JP2017091128A - Movement state estimation system, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of accurately estimating a moving state of a vehicle free from influence by ambient circumstance light.SOLUTION: Disclosed movement state estimation system includes: a road surface image acquisition unit that acquires road surface images of a road surface picked up by a camera installed on the bottom of a vehicle; and a moving state estimation unit that estimates the moving state of the vehicle based on variation of feature points in the road surface images.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a moving state estimation system, method, and program.

  A technique for calculating the amount of movement of a vehicle based on a change in a ground projection image obtained by imaging the front, rear, left and right directions from the vehicle is known (see Patent Document 1).

JP 2008-175717 A

However, in Patent Document 1, there is a problem that a ground projection image obtained by capturing the front, rear, left, and right directions from the vehicle is easily affected by ambient ambient light, and the accuracy of the calculated vehicle movement amount tends to be low.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of estimating a moving state of a vehicle with high accuracy and being hardly affected by ambient light.

  In order to achieve the above object, a moving state estimation system according to the present invention is based on a road surface image acquisition unit that acquires a road surface image obtained by imaging a road surface by a camera provided on the bottom surface of a vehicle, and displacement of feature points in the road surface image. And a movement state estimation unit that estimates a movement state of the vehicle.

  In order to achieve the above object, the moving state estimation method of the present invention includes a road surface image acquisition step of acquiring a road surface image obtained by imaging a road surface with a camera provided on the bottom surface of the vehicle, and displacement of feature points in the road surface image. And a moving state estimating step for estimating the moving state of the vehicle based on the above.

  Furthermore, in order to achieve the above object, the moving state estimation program of the present invention includes a road surface image acquisition function for acquiring a road surface image obtained by capturing a road surface with a camera provided on the bottom surface of the vehicle, and displacement of feature points in the road surface image. And a moving state estimating function for estimating the moving state of the vehicle based on the computer.

  In the present invention configured as described above, the road surface image is an image of the road surface imaged by the camera provided on the bottom surface of the vehicle, so that the influence of ambient ambient light can be suppressed. Accordingly, it is possible to estimate the moving state of the vehicle with high accuracy and not easily affected by ambient ambient light. Since the camera moves together with the vehicle, the moving state of the vehicle relative to the road surface can be estimated based on the displacement of the feature points in the road surface image.

It is a block diagram of a movement state estimation system. 2A is a plan perspective view of the vehicle, FIG. 2B is a side view of the vehicle, and FIGS. 2C to 2F are views showing road surface images. 3A is an imaging timing chart, FIGS. 3B and 3D are side views of the vehicle, FIGS. 3C and 3E are road surface images, and FIG. 3F is a graph of a conversion coefficient. It is a flowchart of a movement state estimation process.

Here, embodiments of the present invention will be described in the following order.
(1) Configuration of moving state estimation system:
(2) Movement state estimation processing:
(3) Other embodiments:

(1) Configuration of moving state estimation system:
FIG. 1 is a block diagram showing a configuration of a navigation device 10 as a movement state estimation system according to an embodiment of the present invention. The navigation device 10 is provided in a vehicle. The navigation apparatus 10 includes a control unit 20 including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Random Access Read Only Memory), and the like, and a recording medium 30, and is stored in the recording medium 30 and the ROM. The control unit 20 executes the programmed program. The control unit 20 executes a navigation program 21 as a movement state estimation program.

  Map information 30 a is recorded on the recording medium 30 of the navigation device 10. The map information 30a includes node data indicating the position of a node set on the road on which the vehicle is traveling, and shape interpolation data indicating the position of a shape interpolation point for specifying the shape of a road section connecting the nodes. , Link data indicating various information about road sections, facility data indicating facility attributes, and the like are included. The control unit 20 draws a map based on the map information 30a.

  The vehicle includes a touch panel display 41, four light emitters 42 a to 42 d, and a camera 43. The touch panel display 41 serves as both an output device that outputs an image to the screen and an input device that receives a user input according to a contact position of a finger or the like on the screen. The touch panel display 41 displays a map under the control of the control unit 20.

  The light emitters 42a to 42d are provided on the lower surface (bottom surface) of the vehicle and output light in the optical axis direction determined by the installation direction. The light emitted from the light emitters 42 a to 42 d may be light in a wavelength band that can be captured by the camera 43. Therefore, if the camera 43 is an infrared camera, the light emitters 42a to 42d emit infrared light, and if the camera 43 is an ultraviolet camera, the light emitters 42a to 42d emit ultraviolet light, and the camera 43 is a visible light camera. The light emitters 42a to 42d may emit visible light. The camera 43 of this embodiment is an infrared camera, and the light emitters 42a to 42d are infrared laser oscillators. However, the light emitters 42a to 42d are not necessarily laser oscillators, and may be light emitters having an optical system that narrows the light flux near the road surface. In the present embodiment, it is assumed that the light emitters 42a to 42d always emit light.

  The camera 43 is a two-dimensional image sensor and is provided on the lower surface (bottom surface) of the vehicle. A road surface image is taken. The camera 43 includes illumination, and the illumination emits illumination light having a wavelength that can be captured by the camera 43. The camera 43 captures a reflection image formed by reflecting the light emitted from the illumination on the road surface. The road surface image is a grayscale image in which each pixel indicates the received light intensity of infrared light. The illumination light of the camera 43 and the laser light of the light emitters 42a to 42d are greatly different in intensity. Among the pixels constituting the road surface image, the pixels that receive the reflected light of the laser light of the light emitters 42a to 42d are clearly defined. Can be determined. When the camera 43 is a color camera, the light wavelength may be different between the illumination of the camera 43 and the light emitters 42a to 42d in the wavelength band of light that the camera 43 can capture.

  FIG. 2A is a perspective plan view of a vehicle (four-wheeled vehicle) for explaining the arrangement of the light emitters 42 a to 42 d and the camera 43. FIG. 2A is a view of the vehicle as viewed from above. As shown in FIG. 2A, a camera 43 is provided in the center of the vehicle body H indicated by a broken line. The camera 43 can capture an imaging range R indicated by a solid line, and captures a reflection image of the imaging range R as a road surface image. The imaging range R is also located at the center of the vehicle body H. Light emitters 42a to 42d are provided in the vicinity of the four corners of the rectangular imaging range R in plan view.

  FIG. 2B is a side view of the vehicle illustrating the light emitters 42 a to 42 d and the camera 43. FIG. 2B is a view of the lower part of the side surface of the vehicle as viewed from a direction parallel to the road surface. An imaging range R indicated by a bold line is formed on a road surface indicated by hatching. Illumination light having a uniform intensity is projected onto the road surface so as to cover the imaging range R. Projection positions Ea to Ed (white triangles) on the road surface of the laser beams (two-dot chain lines) emitted by the light emitters 42 a to 42 d are formed within the imaging range R. In the present embodiment, in a state where the bottom surface of the vehicle is parallel to the road surface, the main optical axis (one-dot chain line) of the camera 43 is orthogonal to the road surface. As shown in FIGS. 2A and 2B, the directions of the laser beams emitted from the light emitters 42a to 42d are different from each other, and the center of the imaging range R (the main light of the camera 43) from each position of the light emitters 42a to 42d. Direction toward the axis).

The navigation program 21 includes a navigation module 21a, a road surface image acquisition module 21b, and a movement state estimation module 21c.
The navigation module 21a causes the control unit 20 to realize a function of displaying a marker indicating the current position of the vehicle on the map. That is, by the function of the navigation module 21a, the control unit 20 displays a map around the current position on the touch panel display 41 based on the map information 30a, and superimposes and displays a marker indicating the current position of the vehicle on the map. To do. In addition, the control part 20 may display the movement planned route searched by the well-known route search method on a map. Furthermore, the control unit 20 may output a voice for guiding the current position and the planned movement route from a speaker (not shown).

  The road surface image acquisition module 21b causes the control unit 20 to realize a function of acquiring a road surface image obtained by capturing the road surface with the camera 43 provided on the bottom surface of the vehicle. Specifically, by the function of the road surface image acquisition module 21b, the control unit 20 continuously acquires road surface images captured by the camera 43 at predetermined time periods (for example, 1 second). The camera 43 and the control unit 20 may be connected by wire or may be connected by a wireless communication line. Furthermore, the ECU (Electronic Control Unit) of the vehicle may acquire a road surface image captured by the camera 43, and the control unit 20 may acquire the road surface image via the ECU.

  The movement state estimation module 21c causes the control unit 20 to realize a function of estimating the movement state of the vehicle based on the displacement of the feature points in the road surface image. The control unit 20 extracts feature points in the road surface image by the function of the movement state estimation module 21c. Except for the projection positions Ea to Ed, the camera 43 captures an imaging range R in which uniform illumination light is projected. Therefore, if the road surface is flat and has a uniform reflectance, the road surface image has a light receiving intensity. A uniform image. However, when there is unevenness or dirt on a part of the road surface, feature points having different received light intensity from other parts appear in the road surface image. For example, the control unit 20 creates a histogram of the light reception intensity of the road surface image, and includes pixels having the highest light reception intensity and pixels corresponding to the projection positions Ea to Ed (hereinafter referred to as projection pixels). Is removed from the road surface image to extract feature points. Note that the projection pixels corresponding to the projection positions Ea to Ed indicate the received light intensity of the reflected light of the laser light emitted from the light emitters 42a to 42d, and therefore are distinguished from the pixels indicating the received light intensity of the reflected light of the illumination light. Can do. In addition, the feature point has a shape corresponding to a shape such as unevenness or dirt on the road surface. The control unit 20 records the characteristics of the feature points (light reception intensity and shape) and the position in the road surface image on the recording medium 30.

  With the function of the movement state estimation module 21c, the control unit 20 estimates the movement state of the vehicle based on the displacement of the feature points between the plurality of road surface images captured at different times. 2C and 2D show road surface images I captured at exposure times t1 and t2 having different start times. A star-shaped feature point Pt1 is extracted from the road surface image I at the exposure time t1, and a feature point Pt2 having a shape similar to the feature point Pt1 is also extracted from the road surface image I at the exposure time t2. The control unit 20 extracts feature points Pt1 and Pt2 whose features (light reception intensity and shape) are similar to or more than the reference as feature points Pt1 and Pt2 corresponding to the same unevenness or dirt on the road surface. The control unit 20 acquires a displacement vector having a feature point Pt1 as a start point and a feature point Pt2 as an end point in the road surface image I. The control unit 20 estimates a direction opposite to the displacement vector in the road surface image I as the moving direction of the vehicle. Further, the control unit 20 converts the length of the displacement vector into a moving distance on the road surface, and estimates the moving speed by dividing the converted moving distance by the difference between the start times of the exposure times t1 and t2. The control unit 20 calculates the movement distance on the road surface by multiplying the movement distance in the road surface image I by the conversion coefficient M. This conversion coefficient M corresponds to the imaging magnification of the camera 43 and is derived from the approach distance G between the camera 43 and the road surface. The approach distance G will be described later. As described above, a movement state (movement speed, movement direction, movement distance) derived from a plurality of road surface images I taken at different times is referred to as a first movement state.

  Furthermore, the function of the movement state estimation module 21c causes the control unit 20 to acquire the displacement of the feature point based on the line image of the feature point in the single road surface image I imaged at a single time, and the feature point The moving state of the vehicle is estimated based on the displacement. FIG. 2E shows a road surface image I captured at an exposure time t3 as a single time. As shown in the figure, a line image Lt3 of feature points appears in the road surface image I. The feature point line image Lt3 is a line having a characteristic light reception intensity, and is a subject blur within a single exposure time t3. That is, when the moving speed is high, the road surface unevenness and dirt move relatively greatly within the single exposure time t3, and the road surface unevenness and dirt image at the exposure start time and the road surface unevenness at the exposure end time. A line connecting images such as unevenness and dirt appears as a feature point line image Lt3. Note that the control unit 20 may determine that the line image Lt3 of the feature point has been extracted when a feature point having an aspect ratio between the long side and the short side that is equal to or greater than a threshold is extracted. The control unit 20 may extract the feature point line image Lt3 by a line detection method such as Hough transform.

  The control unit 20 acquires a vector connecting both ends of the feature point line image Lt3 as a displacement vector. And the control part 20 estimates the direction opposite to the direction of a displacement vector as a moving direction of a vehicle. Note that which is the starting point of both ends of the line image Lt3 of the feature point is unknown only from the single road surface image I, but the displacement vector of the displacement vector is obtained so that a movement direction close to the movement direction estimated immediately before can be obtained. A direction (start point → end point) may be defined. Further, the control unit 20 multiplies the length of the displacement vector by the conversion coefficient M to convert it to a moving distance on the road surface, and estimates the moving speed by dividing the converted moving distance by the length of the exposure time t3. To do. As described above, a moving state (moving speed, moving direction, moving distance) derived from a single road surface image I imaged at a single time is referred to as a second moving state.

  The feature point line image Lt3 also has a characteristic line thickness and received light intensity. Therefore, as shown in FIGS. 2E and 2F, the control unit 20 has feature points whose characteristics (line thickness, light reception intensity) are similar to each other in the road surface image I captured at a plurality of exposure times t3 and t4. The line images Lt3 and Lt4 are extracted as the line images Lt3 and Lt4 of the feature points corresponding to the same unevenness and dirt on the road surface, and the displacement vectors between the line images Lt3 and Lt4 (for example, the start points) of the feature points are extracted. Based on this, it is possible to estimate the first movement state of the vehicle.

  As described above, when the feature point line images Lt3 and Lt4 appear in the single road surface image I, the second movement state at the exposure times t3 and t4 based on the feature point line images Lt3 and Lt4, respectively. Is obtained. Further, when the line images Lt3 and Lt4 of feature points similar to each other appear in the plurality of road surface images I, the first movement state at the exposure times t3 to t4 is obtained based on the displacement between the line images Lt3 and Lt4 of the feature points. It will be.

  With the function of the movement state estimation module 21c, the control unit 20 has a first movement state that is a movement state of the vehicle specified based on the displacement of the feature points between the plurality of road surface images I captured at different times, and a single Both of the second movement state, which is the movement state of the vehicle, obtained from the displacement of the feature point based on the line image of the feature point in the single road surface image I imaged at the time and specified based on the displacement of the feature point. Based on the above, the movement state of the vehicle is estimated. By the function of the movement state estimation module 21c, the control unit 20 estimates a value obtained by averaging the first movement state and the second movement state as a final movement state. As shown in FIG. 3A, the first movement state is estimated for the periods t3 to t4 from the start time of the exposure time t3 to the start time of the exposure time t4, and the second for each of the exposure time t3 and the exposure time t4. The movement state is estimated. For example, the control unit 20 calculates an average value of the movement speed V (t3) as the second movement state at the exposure time t3 and the movement speed V (t4) as the second movement state at the exposure time t4 during the period t3. It is calculated as the movement speed as the second movement state at t4. Then, the control unit 20 determines the average value of the movement speed V (t3 to t4) as the first movement state in the period t3 to t4 and the movement speed as the second movement state in the period t3 to t4. Estimate the moving speed as Note that the control unit 20 may estimate a direction obtained by averaging the movement direction in the first movement state and the movement direction in the second movement state as the final movement direction. Note that the control unit 20 may perform a weighted average of the first movement state and the second movement state according to the reliability of each of the first movement state and the second movement state. For example, as the similarity between the feature point line images Lt3 and Lt4 is smaller, the weight of the second moving state derived from the single feature point line images Lt3 and Lt4 may be increased to perform weighted averaging.

  With the function of the movement state estimation module 21c, the control unit 20 can estimate the current position of the vehicle by accumulating the movement distance in the movement direction. A map is displayed using this current position. The control unit 20 does not have to estimate the current position based only on the road surface image I. For example, the current position may be corrected by a GPS signal, a self-contained navigation trajectory, map matching, or the like, or based on the GPS signal. You may acquire the initial position (initial position which accumulate | stores moving distance) of a position.

  Next, the conversion coefficient M will be described. In the present embodiment, the displacement of the feature point is corrected based on at least one of the inclination of the vehicle body with respect to the road surface and the degree of approach. Specifically, the control unit 20 derives the conversion coefficient M according to the inclination and the approach degree of the vehicle by the function of the movement state estimation module 21c, and converts the distance in the road surface image I into the movement distance on the road surface. By multiplying the conversion factor M, the displacement of the feature point is corrected.

  Further, at least one of the inclination and the approach degree of the vehicle is specified based on the projection positions Ea to Ed of light projected at a plurality of angles with respect to the imaging range R of the road surface image I on the road surface. By the function of the movement state estimation module 21c, the control unit 20 causes the position of the pixel indicating the received light intensity of the reflected light of the laser light emitted from the light emitters 42a to 42d in the road surface image I to be projected positions Ea to Ed on the road surface. Is obtained as the position of the projection pixel corresponding to.

  2B shows a state where the approach distance G from the light emitters 42a to 42d to the road surface is G1, and FIG. 3B shows a state where the approach distance G from the light emitters 42a to 42d to the road surface is G2 smaller than G1. Yes. 2C to 2F show a road surface image I when the approach distance G from the light emitters 42a to 42d to the road surface is G1, and FIG. 3C shows the approach distance G from the light emitters 42a to 42d to the road surface than G1. The road surface image I in the case of small G2 is shown. As the approach distance G is larger, the laser beam intersects the road surface when entering the position closer to the main optical axis of the camera 43. Therefore, as the approach distance G is larger, the positions of the projection pixels corresponding to the projection positions Ea to Ed are the road surface. It approaches the center of the image I. Based on the function of the movement state estimation module 21c, the control unit 20 calculates the approach distance G for each of the light emitters 42a to 42d based on the distance between the position of the projection pixel corresponding to the projection positions Ea to Ed and the center of the road surface image I. presume. Since the approach distance G is estimated for each of the light emitters 42a to 42d, the approach distance G may be different for each of the light emitters 42a to 42d.

  FIG. 3D shows a state in which the approach distance G from the two front light emitters 42a and 42b to the road surface is G2, and the approach distance G from the two rear light emitters 42c and 42d to the road surface is G1. ing. Thus, the inclination with respect to the road surface of the vehicle can be obtained by estimating the approach distance G for each of the light emitters 42a to 42d. Therefore, the control unit 20 can estimate the approach distance G between the position on the road surface corresponding to an arbitrary pixel in the road surface image I and the vehicle body by the interpolation operation by the function of the movement state estimation module 21c. The interpolation calculation may be performed by weighted averaging the approach distance G for each of the light emitters 42a to 42d with the distance from any pixel to the four corners of the road surface image I.

The control unit 20 estimates the approach distance G between the position on the road surface corresponding to the feature points Pt1 and Pt2 and the line images Lt3 and Lt4 of the feature points and the vehicle body, and calculates the conversion coefficient M corresponding to the approach distance G. get. FIG. 3F shows the relationship between the conversion coefficient M and the approach distance G. Since the camera 43 is closer to the road surface as the approach distance G is smaller, the moving distance on the road surface corresponding to one pixel becomes larger.
Therefore, the conversion coefficient M increases as the approach distance G decreases. In addition, the control unit 20 estimates the approach distance G for a pixel in the middle of the feature points Pt1 and Pt2 when converting the distance between the feature points Pt1 and Pt2 into a movement distance on the road surface (first movement state). A conversion coefficient M corresponding to the approach distance G may be used.

  When the approach distance G changes between the exposure times t1 and t2, the control unit 20 may use a conversion coefficient M corresponding to the average value of the approach distances G before and after the change. Further, the control unit 20 converts the lengths of the feature point line images Lt3 and Lt4 into the movement distance on the road surface (second movement state) with respect to the middle point pixels of the feature point line images Lt3 and Lt4. The approach distance G may be estimated and a conversion coefficient M corresponding to the approach distance G may be used. As described above, by obtaining the movement distance using the conversion coefficient M corresponding to the approach distance G, the displacement of the feature point is corrected based on the inclination and the approach degree of the vehicle body with respect to the road surface, and the movement distance is obtained. be able to.

  In the configuration of the present embodiment described above, the road surface image I is an image of the road surface imaged by the camera provided on the bottom surface of the vehicle, so that the influence of ambient ambient light can be suppressed. Accordingly, it is possible to estimate the moving state of the vehicle with high accuracy and not easily affected by ambient ambient light. Since the camera 43 moves together with the vehicle, the moving state of the vehicle relative to the road surface can be estimated based on the displacement of the feature points in the road surface image I.

  The control unit 20 estimates the first movement state of the vehicle based on the displacement of the feature points Pt1 and Pt2 between the plurality of road surface images I taken at different times. Accordingly, the moving speed can be estimated based on a value obtained by dividing the displacement of the feature points between the plurality of road surface images I by the length (time) between the times when the plurality of road surface images I are imaged. Further, the moving direction of the vehicle can be estimated based on the direction of displacement of the feature points between the plurality of road surface images.

  The control unit 20 acquires the displacement of the feature point based on the line images Lt3 and Lt4 of the feature point in the single road surface image I imaged at a single time, and moves the vehicle based on the displacement of the feature point. Estimate the state. The line images Lt3 and Lt4 of the feature points in the single road surface image I are the positions of the feature points at the beginning of the exposure times t3 and t4 of the single road surface image I and the feature points at the end of the exposure times t3 and t4. It can be thought of as a line connecting the positions. Accordingly, the displacement of the feature point can be acquired based on the feature point line images Lt3 and Lt4 in the single road surface image I. The second movement state of the vehicle can be estimated every single exposure time t3, t4, and the second movement state can be obtained quickly. Furthermore, since the moving state of the vehicle is estimated based on both the first moving state and the second moving state, the reliability of the estimated moving state can be improved.

  In addition, since the displacement of the feature point is corrected based on the inclination and proximity of the vehicle body with respect to the road surface, even if the imaging distance of the camera 43 relative to the road surface and the optical axis direction vary, the effect of the variation is offset. By correcting the displacement of the points, the movement state can be estimated with high accuracy. The inclination and the approach degree of the vehicle are specified based on the projection positions Ea to Ed of light projected at a plurality of angles with respect to the imaging range R of the road surface image I on the road surface. Thereby, the projection positions Ea to Ed of the light projected at a plurality of angles can be specified based on the road surface image I, and the inclination and the approach degree of the vehicle can be obtained without providing the cameras 43 individually.

(2) Movement state estimation processing:
Next, the movement state estimation process executed by the navigation program 21 will be described. FIG. 4 is a flowchart of the movement state estimation process. First, the control unit 20 acquires a road surface image I by the function of the road surface image acquisition module 21b (step S100). Next, the control unit 20 extracts a feature point Pt2 from the road surface image I by the function of the movement state estimation module 21c (step S110). 2E and 2F, when the feature point line images Lt3 and Lt4 are extracted, the control unit 20 regards the start points of the feature point line images Lt3 and Lt4 as the feature point positions.

  Next, the control part 20 estimates the approach distance G based on a projection pixel by the function of the movement state estimation module 21c (step S120). The projection pixels are pixels corresponding to the projection positions Ea to Ed on the road surface of the laser light emitted from the light emitters 42a to 42d, and are pixels that indicate the received light intensity corresponding to the intensity of the laser light. The control unit 20 estimates the approach distance G for each of the light emitters 42a to 42d based on the position of the projection pixel in the road surface image I. Here, as the position of the projection pixel in the road surface image I is closer to the center of the road surface image I, a larger distance is estimated as the approach distance G.

  Next, the control unit 20 records frame information (not shown) on the recording medium 30 by the function of the movement state estimation module 21c (step S130). The frame information is information about the current road surface image I (frame), and features (light receiving intensity and shape) and positions of the extracted feature points Pt1 and Pt2 (including the start points of the feature point line images Lt3 and Lt4). And the approach distance G for each of the light emitters 42a to 42d.

  Next, the control unit 20 acquires the frame information of the previous frame by the function of the movement state estimation module 21c (step S140). The previous frame is a road surface image I imaged in the immediately preceding imaging cycle. Here, it is assumed that the previous frame is the road surface image I (FIG. 2C) at the exposure time t1, and the current frame is the road surface image I (FIG. 2D) at the exposure time t2.

  Next, the control unit 20 acquires the displacement vectors of the feature points Pt1 and Pt2 between the frames by the function of the movement state estimation module 21c (step S150). Specifically, the control unit 20 extracts feature points Pt1 and Pt2 corresponding to the same unevenness and dirt on the road surface from two road surface images I taken at different times, and the feature points Pt1 and Pt2 The movement state of the vehicle is estimated based on the displacement between the two. The feature points Pt1 and Pt2 corresponding to the same unevenness and dirt on the road surface are the feature points Pt1 and Pt2 whose features (light reception intensity and shape) are similar to each other from the reference. The control unit 20 acquires a displacement vector having a feature point Pt1 as a start point and a feature point Pt2 as an end point in the road surface image I.

  Next, the control part 20 acquires the conversion factor M corresponding to the approach distance G by the function of the movement state estimation module 21c (step S160). Specifically, the control unit 20 determines the position of the feature points Pt1 and Pt2 (the middle point of the displacement vector connecting the feature points Pt1 and Pt2) based on the approach distance G of the light emitters 42a to 42d at the exposure times t1 and t2. A corresponding approach distance G is acquired, and a conversion coefficient M corresponding to the approach distance G is acquired.

  Next, the control unit 20 estimates the first movement state by the function of the movement state estimation module 21c (step S170). That is, the control unit 20 estimates the direction opposite to the direction of the displacement vector as the moving direction, and two road surface images I are captured of the moving distance obtained by multiplying the length of the displacement vector by the conversion factor M. The moving speed is estimated by dividing by the length of the period between times (t1 to t2).

  Next, the control unit 20 determines whether or not a feature point line image has been extracted by the function of the movement state estimation module 21c (step S180). That is, the control unit 20 determines whether or not a line image of a feature point whose aspect ratio between the short side and the long side is equal to or greater than the threshold has been extracted in step S110. Here, it will be described below that the current frame is the road surface image I at the exposure time t3 and the line image Lt3 of the feature point is extracted.

  When it is determined that the feature point line image Lt3 has been extracted (step S180: Y), the control unit 20 acquires a displacement vector connecting both ends of the feature point line image Lt3 by the function of the movement state estimation module 21c ( Step S200). That is, the control unit 20 acquires a displacement vector that connects the position of the feature point at the exposure start time and the position of the feature point at the exposure end time.

  Next, the control part 20 acquires the conversion factor M corresponding to the approach distance G by the function of the movement state estimation module 21c (step S210). Specifically, the control unit 20 acquires the approach distance G corresponding to the position of the line image Lt3 of the feature point based on the approach distance G of the light emitters 42a to 42d at the exposure time t3, and corresponds to the approach distance G. The conversion coefficient M to be acquired is acquired.

  Next, the control unit 20 estimates the second movement state by the function of the movement state estimation module 21c (step S220). That is, the control unit 20 estimates the direction opposite to the direction of the displacement vector as the moving direction, and calculates the distance obtained by multiplying the length of the displacement vector by the conversion factor M by the length of the single exposure time t3. The moving speed is estimated by dividing.

  Next, the control part 20 outputs the movement state which averaged the 1st movement state and the 2nd movement state by the function of the movement state estimation module 21c (step S230). That is, the control unit 20 outputs the movement state obtained by averaging the first movement state and the second movement state as the finally estimated movement state. As a result, the control unit 20 estimates the current position of the vehicle based on the finally estimated movement state or superimposes the marker indicating the current position on the touch panel display 41 by the function of the navigation module 21a. You can do it.

  On the other hand, when it is not determined that the feature point line image has been extracted (step S180: N), the control unit 20 outputs the first movement state by the function of the movement state estimation module 21c (step S190). That is, since the second moving state cannot be estimated in the current frame, the control unit 20 outputs the first moving state that can be estimated in the current frame as it is. When the moving speed is low, the feature point line image is not extracted and the second moving state cannot be estimated.

(3) Other embodiments:
The above embodiment is an example for carrying out the present invention, and the embodiment can be changed within a range not impairing the effect of the present invention. For example, the control unit 20 may estimate only one of the first movement state and the second movement state.

  For example, the inclination and the approach degree of the vehicle body with respect to the road surface may be acquired based on measurement values of a distance sensor, an acceleration sensor, and a gyro sensor. Furthermore, the inclination and the approach degree of the vehicle may be acquired based on the state of the suspension and the wear state of the tire. Further, the laser light output from the light emitters 42 a to 42 d may be captured by a camera different from the camera 43.

  Further, it is not necessary to consider the inclination of the vehicle body as in the above embodiment, and only the degree of vehicle body approach to the road surface is considered by measuring the approach distance G only at one point (for example, on the main optical axis of the camera 43). May be. That is, the control unit 20 may acquire the conversion coefficient M corresponding to the approach distance G independent of the positions of the feature points Pt1 and Pt2 and the feature point line images Lt3 and Pt4. Furthermore, the control unit 20 may acquire a conversion coefficient M corresponding to a certain approach distance G for each vehicle type without measuring the actual inclination and approach degree of the vehicle body.

  The above embodiment is an example for carrying out the present invention, and various other embodiments can be adopted. The bottom surface of the vehicle is a surface of the vehicle body surface that faces the road surface. The camera may be provided on the bottom surface of the vehicle, but it is desirable that the optical axis direction and the angle of view are set so that the imaging range is as small as possible with the influence of ambient light. For example, the camera may be provided so that the optical axis is located at the center of the vehicle body when viewed from the vertical direction. In addition, the camera may be provided so that the imaging range exists inside a predetermined distance from the outer edge of the vehicle body when viewed from the vertical direction. The camera may be a camera capable of capturing visible light, or may be a camera capable of capturing infrared light or ultraviolet light. For example, by using a camera capable of capturing infrared light, a road surface image with little influence of ambient light can be obtained.

  The road surface image acquisition unit only needs to acquire a road surface image captured at the timing of estimating the movement state, and may periodically estimate the movement state by acquiring the road surface image periodically captured. The road surface image acquisition unit may acquire the road surface image directly from the camera, or may acquire the road surface image via the vehicle. A feature point is a point having an image feature in a road surface image, and is a point where a point having a characteristic shape or color is captured on the road surface. The feature point displacement is the moving distance or moving direction of the feature point. The movement state estimation unit can estimate the movement distance and movement direction of the vehicle based on the movement distance and movement direction of the feature points.

  Further, the movement state estimation unit may estimate the movement state of the vehicle based on the displacement of the feature points between a plurality of road surface images captured at different times. That is, the moving speed can be estimated based on a value obtained by dividing the displacement of the feature points between the plurality of road surface images by the length (time) between the times when the plurality of road surface images are captured. Further, the moving direction of the vehicle can be estimated based on the direction of displacement of the feature points between the plurality of road surface images.

  Further, the movement state estimation unit obtains the displacement of the feature point based on the line image of the feature point in the single road surface image captured at a single time, and the movement state of the vehicle based on the displacement of the feature point. May be estimated. That is, a line image of feature points in a single road image (so-called subject blur) connects the position of the feature point at the beginning of the exposure time of the single road surface image with the position of the feature point at the end of the exposure time. It can be thought of as a line to do. Therefore, the displacement of the feature point can be acquired based on the line image of the feature point in the single road surface image. The movement state of the vehicle can be estimated for each single exposure time, and the movement state can be obtained quickly.

  The movement state estimation unit also includes a first movement state that is the movement state of the vehicle specified based on the displacement of the feature points between a plurality of road surface images captured at different times, and a single image captured at a single time. The displacement of the feature point is acquired based on the line image of the feature point in the one road surface image, and the vehicle is moved based on both the second movement state that is the movement state of the vehicle specified based on the displacement of the feature point. The state may be estimated. Movement of the vehicle based on both the first movement state identified based on the displacement of the feature points between the plurality of road surface images and the second movement state identified based on the line images of the feature points in the single road surface image Since the state is estimated, the reliability of the estimated moving state can be improved.

  Furthermore, the displacement of the feature point may be corrected based on at least one of the inclination of the vehicle body with respect to the road surface and the degree of approach. Thereby, even if the imaging distance of the camera with respect to the road surface and the optical axis direction vary, the movement state can be estimated with high accuracy by correcting the displacement of the feature points so as to cancel the influence of the variation.

  Further, at least one of the inclination and the approach degree of the vehicle may be specified based on the projection positions of light projected at a plurality of angles with respect to the imaging range of the road surface image on the road surface. The projection positions of the light projected at a plurality of angles can be specified based on the road surface image, and at least one of the vehicle inclination and the approach degree can be obtained without providing a camera individually.

  Furthermore, the method of estimating the moving state of the vehicle based on the road surface image obtained by imaging the road surface with the camera provided on the bottom surface of the vehicle can also be applied as a program or a method. In addition, the moving state estimation system, the program, and the method as described above may be realized as a single moving state estimation system or may be realized by using components shared with each part provided in the vehicle. Various embodiments are included. For example, it is possible to provide a navigation system, method, and program that include the above-described moving state estimation system. Further, some changes may be made as appropriate, such as a part of software and a part of hardware. Furthermore, the present invention is also established as a recording medium for a program for controlling the movement state estimation system. Of course, the software recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium to be developed in the future.

DESCRIPTION OF SYMBOLS 10 ... Navigation apparatus, 20 ... Control part, 21 ... Navigation program, 21a ... Navigation module, 21b ... Road surface image acquisition module, 21c ... Movement state estimation module, 30 ... Recording medium, 30a ... Map information, 41 ... Touch panel display, 42a ˜42d ... light emitter, 43 ... camera, ... light receiving intensity, Ea-Ed ... projection position, G ... access distance, H ... vehicle body, I ... road surface image, Lt3 ... line image, Lt3, Lt4 ... line image of feature points, M: conversion coefficient, Pt1, Pt2 ... feature point, R ... imaging range, t3 to t4 ... exposure time, V ... movement speed

Claims (8)

A road surface image acquisition unit that acquires a road surface image obtained by imaging the road surface with a camera provided on the bottom surface of the vehicle;
A movement state estimation unit that estimates a movement state of the vehicle based on a displacement of a feature point in the road surface image;
A moving state estimation system comprising: The movement state estimation unit estimates a movement state of the vehicle based on displacement of the feature points between a plurality of the road surface images captured at different times.
The movement state estimation system according to claim 1. The movement state estimation unit acquires a displacement of the feature point based on a line image of the feature point in a single road surface image captured at a single time, and the vehicle based on the displacement of the feature point Estimate the moving state of
The movement state estimation system according to claim 1. The movement state estimation unit
A first movement state that is a movement state of the vehicle specified based on a displacement of the feature point between the plurality of road surface images captured at different times;
The movement state of the vehicle is acquired based on the displacement of the feature point based on the line image of the feature point in the single road surface image captured at a single time and specified based on the displacement of the feature point. Estimating the moving state of the vehicle based on both second moving states;
The movement state estimation system according to claim 1. The displacement of the feature point is corrected based on at least one of the inclination and the approach degree of the vehicle body with respect to the road surface.
The movement state estimation system according to any one of claims 1 to 4. At least one of the inclination and the approach degree of the vehicle is specified based on a projection position of light projected at a plurality of angles with respect to an imaging range of the road surface image on the road surface.
The movement state estimation system according to claim 5. A road surface image acquisition step of acquiring a road surface image obtained by imaging the road surface with a camera provided on the bottom surface of the vehicle;
A moving state estimating step of estimating a moving state of the vehicle based on displacement of feature points in the road surface image;
A moving state estimation method including: A road surface image acquisition function for acquiring a road surface image obtained by imaging the road surface with a camera provided on the bottom surface of the vehicle;
A movement state estimation function for estimating a movement state of the vehicle based on a displacement of a feature point in the road surface image;
A moving state estimation program that causes a computer to realize the above.

Images