JP2018004435A - Movement amount calculation device and movement amount calculation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a movement amount calculation device capable of calculating a movement amount with low processing load and high accuracy even when three-dimensional objects having different reflection rates are projected in a pickup image.SOLUTION: A movement amount calculation device comprises: an imaging device for imaging an image of a road surface; and an image processing part for calculating a movement amount of a mobile body on the basis of a time-series pickup image picked up by the imaging device. The image processing part calculates a time-series distance image on the basis of the pickup image, and stores it in a memory (S32), and extracts a plurality of first feature points from a first pickup image picked up at a first timing among pickup images, and extracts a plurality of second feature points from a second pickup image picked up at a second timing after the first timing (S36), and tracks each of the first feature points to each of the second feature points (S37), and calculates a deviation image indicating a deviation of a distance from the calculated time-series distance image, and calculates a movement amount of the mobile body on the basis of the feature point whose deviation is a predetermined threshold or less among the plurality of tracked feature points (S39).SELECTED DRAWING: Figure 3

Description

  The present invention relates to a movement amount calculation apparatus and a movement amount calculation method applied to a robot, an automobile, and the like.

  Developed autonomous driving technology that detects information on the surrounding environment and performs driving control according to the situation to improve safety and convenience in mobile units such as land drones and robots that perform dangerous work on behalf of people Has been. In particular, when traveling in an open environment where GPS (Global Positioning System) cannot be used, a technique for estimating the amount of movement with reference to the unevenness of the ground is required.

  As an example of calculating the amount of movement of a moving object, a plurality of map information of the moving environment is stored and measured by the map information and the TOF camera based on the measurement angle of the TOF (Time-of-flight) camera (distance image camera). There is a technique for estimating the self-position based on the obtained distance data (see Patent Document 1). In this document, the accuracy of the movement amount measured by the TOF camera is to be improved with map information.

JP2011-209845A

  However, as in Patent Document 1, even if map information is used, the accuracy is not always improved. For example, if a three-dimensional object having a different reflectance is reflected on the TOF distance image sensor, the accuracy of the distance information acquired by the TOF distance image sensor is lowered due to the influence of the different reflectance, and the moving amount of the moving object may be erroneous.

  The present invention is an invention for solving the above-described problems, and is a movement amount calculation capable of calculating a movement amount with a low processing load and high accuracy even when a three-dimensional object having a different reflectance is reflected in a captured image. An object is to provide an apparatus.

  In order to achieve the above object, a movement amount calculation device of the present invention includes an imaging device that captures an image of a road surface, and an image processing unit that calculates a movement amount of a moving body based on time-series captured images captured by the imaging device. The image processing unit calculates a time-series distance image based on the captured image and stores it in the storage unit (for example, the memory 6) (for example, step S32). A plurality of first feature points are extracted from the first captured image captured at the timing, and a plurality of second feature points are extracted from the second captured image captured at the second timing after the first timing. (Eg, step S36), each of the first feature points is tracked to each of the second feature points (eg, step S37), and a deviation image indicating a deviation of the distance from the calculated time-series distance image is obtained. Calculated (eg, step S 4), among the plurality of feature points to track, based on the feature point of the deviation is less than a predetermined threshold value, and calculates the amount of movement of the moving body (e.g., step S39) it is characterized. Other aspects of the present invention will be described in the embodiments described later.

  According to the present invention, it is possible to provide a movement amount calculation apparatus and a movement amount calculation method for a moving body that are capable of calculating a movement amount with a low processing load and high accuracy even when a three-dimensional object having a different reflectance is reflected in a captured image. can do.

It is a figure which shows the structure of the movement amount calculation apparatus which concerns on this embodiment. It is a figure which shows the detail of the influence by the three-dimensional object from which a reflectance differs, (a) is a figure which shows the structure for verifying the accuracy of distance, (b) is a figure which shows the influence of the distance by a different reflectance. (C) is a figure which shows the influence of the deviation by a different reflectance. It is a flowchart which shows the process of an image process part. It is a figure which shows the detail of step S34, (a) is a case where a mobile body is low speed, (b) is a case where a mobile body is high speed. It is a figure which shows the detail relevant to step S38. It is a figure which shows the movement amount calculation method on a road, (a) is a case where there is no object with a different reflectance, (b) is a case where there is an object with a different reflectance. It is a figure which shows the example of the experimental data based on the image acquired with the imaging device, (a) is a captured image, (b) is an average distance image, (c) is a deviation image discrimination | determination result.

DESCRIPTION OF EMBODIMENTS Embodiments for carrying out the present invention will be described in detail with reference to the drawings as appropriate.
FIG. 1 is a diagram illustrating a configuration of a movement amount calculation apparatus. The moving body 1 captures an image of the surrounding environment, processes the image captured by the imaging apparatus 2, calculates the amount of movement of the moving body 1, and outputs a display or control signal according to the calculation result. It is comprised by the processing apparatus 3 to perform. Note that the movement amount calculation device 10 (optical movement amount estimation device) of the present embodiment includes at least the imaging device 2 and the image processing unit 4.

  The processing device 3 is configured by a computer system or the like, and includes an image processing unit 4 that processes an image captured by the imaging device 2, a control unit (CPU) 5 that performs various controls based on the processed image, a movement amount calculation, and the like. For this purpose, a memory 6 (storage unit) for storing various data used in the control unit, a display unit 7 for outputting a calculation result of the control unit 5, and a bus 8 for connecting these components to each other are provided. . CPU is an abbreviation for Central Processing Unit.

  The imaging device 2 is a distance image sensor installed facing the front of the moving body 1. Distance data is acquired for each pixel (pixel) by the distance image sensor. In the following, for the sake of simplicity, a case where a single standard camera is used will be described. However, any standard camera, wide-angle camera, or stereo camera can be used as long as it has a viewing angle from which feature points can be extracted during driving. A TOF distance image sensor may be used. Each camera finally generates one image, and the imaging apparatus 2 may be configured by combining a plurality of cameras.

  The imaging device 2 acquires an image when a command is input from the control unit 5 or at regular time intervals, and outputs the acquired image and the acquisition time to the image processing unit 4 via the memory 6. The original image and the acquisition time of the acquired image are stored in the memory 6, and an intermediate processed image is created in the image processing unit 4. These intermediate images are also stored in the memory 6 as necessary, and are used for determination and processing by the control unit 5 and the like. The result data used for the processing in the control unit 5 is also stored in the memory 6 as appropriate.

  The bus 8 for transmitting data between the blocks can be configured by an IEBUS (Inter Equipment Bus), a LIN (Local Interconnect Network), a CAN (Controller Area Network), or the like.

  The image processing unit 4 calculates a movement amount based on an image captured by the imaging device 2 while the moving body 1 is traveling. First, feature points on the image transmitted from the imaging device 2 are extracted. Further, feature points on the next transmitted image are also extracted. Then, the feature point extracted first (previous) and the next (current) feature point are tracked (contrast), the amount of movement of the moving object is calculated, and the result is output to the control unit 5.

  The control unit 5 calculates the position of the moving body 1 based on the movement amount calculated by the image processing unit 4, determines the future moving direction and speed, and controls the moving body 1. Then, by displaying a necessary detection result on the display unit 7, information is provided to the operator of the moving body 1.

  FIG. 2 is a diagram showing details of the influence of a three-dimensional object having different reflectivities, (a) is a diagram showing a configuration for verifying the accuracy of the distance, and (b) is a diagram of distances due to different reflectivities. It is a figure which shows an influence, (c) is a figure which shows the influence of the deviation by a different reflectance.

  A plane 20 illustrated in FIG. 2A is a plane installed in parallel to the imaging device 2. The distance M (21) is a constant distance from the imaging device 2 to the center XO of the plane 20. Since the plane 20 is parallel to the imaging device 2, the imaging device 2 can calculate the distance M (21) for each pixel (u, v). The line 22 is a line on the plane 20. Here, for the sake of simplicity, the line 22 is a line passing through the center XO of the image in the same direction as the u pixel on the image. Note that the depth is constant for all pixels on the plane 20 from the imaging device 2.

  A graph 23 illustrated in FIG. 2B represents an example in which the imaging device 2 is a TOF distance image sensor, a plurality of distance images are acquired, and an average distance to the line 22 is calculated. The horizontal axis indicates the pixel position (pixel position) of the straight line 22, and the “0” point position on the horizontal axis corresponds to, for example, the left end when the plane 22 is viewed from the imaging device 2 on the line 22. Using the TOF distance pixel sensor, the distance M is calculated by obtaining the time difference from the phase difference between the irradiated light and the reflected light.

  When the plane 20 has a constant reflectance such as a white board, the relationship between the pixel position and the distance M is data 24. When the distance M to the center XO is M0, the distance M0 at the center point XO is the vertex. And a downwardly convex curve. On the other hand, when the plane 20 is a three-dimensional object having a different reflectance such as a chess board, the relationship between the pixel position and the distance is data 25. In this case, an error occurs in the distance calculated by the imaging device 2, and the value of the data 25 is different from the data 24.

  A graph 26 shown in FIG. 2C represents a deviation in distance obtained by the same distance image as the graph 23. When the plane 20 has a certain reflectance such as a white board, the relationship between the pixel position and the deviation of the distance becomes the data 27. On the other hand, when the plane 20 is a three-dimensional object with different reflectivities such as a chess board, the relationship between the pixel position and the deviation in distance becomes the data 28. In this case, an error occurs in the distance calculated by the imaging device 2, so that the distance deviation becomes large. That is, the distance deviation and accuracy are related, and when the distance deviation is large, the distance accuracy is low. On the other hand, when the distance deviation is small, the accuracy of the distance is high. Therefore, it is possible to distinguish the precision by calculating the deviation using a distance image in time series.

  FIG. 3 is a flowchart showing the processing of the image processing unit. The image processing unit 4 acquires an image (captured image) captured by the imaging device 2 from the memory 6 (step S30). Next, the image processing unit 4 calculates a distance image (step S31). When the imaging device 2 is a stereo camera, the distance is calculated geometrically for each pixel using two images taken simultaneously. When the imaging device 2 is a TOF distance image sensor, the distance is calculated for each pixel based on the TOF principle. According to the TOF principle, the light emitted from the sensor is reflected by the object, and the distance to the object is obtained from the time of flight of light and the speed of light until it reaches the sensor. If the road surface is flat, the pixel position on the image and the actual positional relationship (X, Y, Z) are constant, and the distance may be calculated geometrically. The image processing unit 4 stores the calculated distance image in the memory 6 (step S32).

  The image processing unit 4 determines whether the speed area of the moving body 1 is low speed or high speed (speed area determination, step S33). That is, the image processing unit 4 changes the processing flow according to the speed region. The speed information of the moving body 1 is obtained from sensors mounted on the moving body 1 such as GPS, IMU (Inertia Measurement Unit), and wheel encoder. Moreover, you may estimate the speed of the mobile body 1 by the image odometry technique using the imaging device 2. If the speed is low, the process proceeds to step S34. If the speed is high, the process proceeds to step S35.

  In the case of a low speed, processing is performed in the order of steps S34, S35, S36, and S37 described later. In the case of a high speed, processing is performed in the order of Step S35, Step S36, Step S37, and Step S34. The reason for changing the processing order will be described later.

  The image processing unit 4 calculates a deviation image using the distance image stored in the memory 6 in time series (step S34). Details will be described later with reference to FIG.

  Then, the image processing unit 4 sets a region of interest for extracting feature points on the image acquired in step S30 (step S35). For example, a traveling road surface close to the imaging device 2 with high accuracy and few obstacles is set as a region of interest within a predetermined range. In addition, when the moving body 1 travels on a road, an oncoming vehicle or other obstacle in the adjacent lane affects the self-position estimation, so the traveling lane is set as a region of interest. In addition, when self-position estimation is performed using the ground of the driving environment in an open environment, the environment information captured by the imaging device 2 changes depending on the installation height and angle of the imaging device 2 with respect to the ground, so the ground is reflected. Is set as a region of interest.

  Then, the image processing unit 4 extracts feature points on the image acquired in Step S30 from the region of interest set in Step S35 (Step S36). The feature points are edges, corners, and the like on the image, and techniques such as Canny, Sobel, FAST, Hessian, and Gaussian are used.

  Next, the image processing unit 4 extracts from the image captured at the first timing (previous frame), that is, the image captured before the image captured at the second timing (current frame). The point is tracked on the image captured this time (imaged at the second timing) (step S37). For the tracking, a technique such as Lucas-Kanade method or Shi-Tomasi method is used.

  Then, the image processing unit 4 distinguishes between high accuracy pixels and low accuracy pixels based on the deviation image calculated in step S34, and determines whether or not it is necessary to filter the deviation image (step S38). . Specifically, the deviation of each pixel of the deviation image calculated in step S34 is compared with a predetermined threshold value. When the pixel deviation is larger than the predetermined threshold value (Yes at Step S38), the pixel distance accuracy calculated at Step S31 is low, and therefore, filtering is performed at Step S38a (not used for estimating the moving amount of the moving body 1). On the other hand, if the pixel deviation is equal to or less than a predetermined threshold (predetermined threshold) (No in step S38), the pixel distance accuracy calculated in step S31 is high, and the process proceeds to step S39 without filtering. Note that the determination process in step S38 is performed for each pixel.

Next, the image processing unit 4 calculates the movement amount ΔD _m of the moving body 1 using the feature points not filtered in step S38 (step S39). The point of the feature point that has not been filtered is set to o, and the difference (Δd _mo = d _mo −d ₎ between the relative position d _{(m−1) o of} the feature point extracted in the previous frame and the relative position d _mo of the feature point extracted this time. _{(m-1) o} ) is calculated, and a moving amount ΔD _m of the moving body 1 is calculated using a plurality of Δd _mo . As a calculation method, for example, Rigid Body Transformation, Sliding Window, least square method, median filter, or the like can be used.

  FIG. 4 is a diagram showing details of step S34, where (a) shows a case where the moving body is low speed, and (b) shows a case where the moving body is high speed.

FIG. 4A shows details of the deviation image calculation (step S34) when the moving body 1 travels at a low speed. _{D 1, D 2, ···,} D f (40) , the image ₁ acquired in step S30 to the time series at step S31, the image _2,..., Range image 1 was calculated from the image _f, the distance image 2 ,... Is a distance image f. _{D 1, D 2, ···,} D f of (40) pixels (u, v) (u, v) 1, and _{(u, v) 2, ···} , (u, v) f.

The AVERAGE image 41 is an average distance image obtained from D ₁ , D ₂ ,..., D _f (40). The pixel (u, v) of the AVERAGE image 41 is set to (u, v) _AVE, and is calculated by Expression (1).

(u, v) _AVE = ((u, v) ₁ + (u, v) ₂ + ... + (u, v) _f ) / f (1)

  When the imaging device 2 faces downward and the unevenness of the traveling environment does not change greatly instantaneously, the average is calculated for each pixel in time series using the equation (1), so that the three-dimensional object or the moving body 1 having different reflectivity is obtained. It is possible to reduce the influence of inaccurate pixels due to the vibration of. Further, the AVERAGE image 41 may be configured by processing such as median, mode (mode), moving average (moving average). Note that median, mode, and moving average are used in statistical processing and the like. The median is a method that uses a median value, the mode is a method that uses a mode value, and the moving average smoothes sequence data. It is a technique.

The VAR image 42 is a deviation image obtained from D ₁ , D ₂ ,..., D _f (40). The pixel (u, v) of the VAR image 42 is set to (u, v) _VAR, and is calculated by Expression (2). Further, a numerical value representing the standard deviation or the variation of the distance image obtained in time series in step S21 may be calculated.

(u, v) _VAR = [((u, v) _AVE- (u, v) ₁ ) ²
+ ...
+ ((U, v) _AVE- (u, v) _f ) ² ] / f (2)

The number of distance images (for example, f) is set according to the environment. For example, in an environment where there are many three-dimensional objects with different vibrations and reflectances, the accuracy of the AVERAGE image 41 can be corrected by setting f large. When the moving body 1 travels at a low speed, the distance images D ₁ , D ₂ ,..., D _f (40) do not change greatly instantaneously, so f is set large. When the moving body 1 travels at a high speed, the distance changes greatly, so f is set small.

When the moving body 1 moves, the positional relationship between the pixels (u, v) ₁ , (u, v) ₂ ,..., (U, v) _f and the environment (X, Y, Z) ₁ , (X, Y , Z) ₂ ,..., (X, Y, Z) _f are different, but in the case of low speed, the unevenness does not change greatly instantaneously, and the moving amount of the moving body 1 is assumed to be small. (U, v) _AVE and (u, v) _VAR may be calculated by the following equation (2).

On the other hand, FIG.4 (b) shows the case where the mobile body 1 drive | works at high speed. _{D 1, D 2, ···,} D f (40b) , the image ₁ acquired in step S30 to the time series at step S31, the image _2,..., Range image 1 was calculated from the image _f, the distance image 2 ,... Is a distance image f. _{D 1, D 2, ···,} D f of (40b) pixels (u, v) (u, v) 1, and _{(u, v) 2, ···} , (u, v) f.

However, when the moving body 1 travels at a high speed, the positional relationship between the pixels (u, v) ₁ , (u, v) ₂ ,..., (U, v) _f and the environment (X, Y, Z) ₁ , (X, Y, Z) ₂ ,..., (X, Y, Z) _f are greatly different. Therefore, the feature point tracking of step S37 in FIG. 3 is used to correct the pixel and positional relationship, and the AVERAGE image 41b Is calculated. That is, in FIG. 3, in the case of high speed, the process of step S34 is performed after the process of step S37 (feature point tracking).

The feature points (U, V) ₁ 43a are the feature points extracted in step S36 in the image ₁ . After tracking the feature point 43a at step S37 in the image _1, the feature point 43a in the image ₁ is moved to the feature point (U, V) ₂ 43 b in the image _2. After tracking the feature point 43b in step S37 in the image _2, a feature point 43b in the image ₂ is moved to the feature point (U, V) _f 43c in the image _f.

The pixel (U, V) _AVE of the AVERAGE image 41b is calculated by the equation (3) using the feature points (U, V) ₁ 43a, (U, V) ₂ 43b, (U, V) ₃ 43c.

(U, V) _AVE = ((U, V) ₁ + (U, V) ₂ + ... + (U, V) _f ) / f (3)

Based on the amount of movement of the feature points (U, V) ₁ 43a, (U, V) ₂ 43b, (U, V) ₃ 43c tracked in step S37, other pixels (u, v) are obtained using equation (3). By calculating (u, v) _AVE of ₁ , (u, v) ₂ ,..., (U, v) _f , an AVERAGE image 41b that is an average distance image can be calculated. Further, (U, V) _AVE may be calculated by a process such as median, mode, or moving average.

Accordingly, the feature points (U, V) ₁ 43a, (U, V) ₂ 43b, (U, V) ₃ 43c and (U, V) _AVE tracked in step S37 are substituted into the equation (4) to obtain pixels. (U, V) _VAR and a VAR image 42b which is a deviation image can be calculated.

(U, V) _VAR = [((U, V) _AVE- (U, V) ₁ ) ²
+ ...
+ ((U, V) _AVE− (U, V) _f ) ² ] / f (4)

  When the mobile body 1 travels at a low speed, the VAR image 44 (see FIG. 5) becomes the VAR image 42, and when the mobile body 1 travels at a high speed, the VAR image 44 (see FIG. 5) becomes the VAR image 42b.

FIG. 5 is a diagram showing details relating to step S38. The determination unit 45 is a unit that distinguishes (determines) a high-precision pixel and a low-precision pixel. The value of each pixel (u, v) _VAR of the deviation image calculated in step S34 is compared with a predetermined threshold value Var _th (VAR _th ). (u, v) if the value of _VAR is greater than Var _th, low range accuracy of pixels calculated in step S31. For this reason, when the pixel (u, v) is tracked in step S37, the feature point of the pixel (u, v) is filtered in step S38a (not used for estimating the movement amount of the moving object 1).

On the other hand, (u, v) if the value of _VAR is less than Var _th, high distance precision pixels calculated in step S31. Therefore, when the pixel (u, v) is tracked in step S37, the pixel (u, v) is used for estimating the movement amount of the moving body 1 in step S39. The setting of _Varth is set according to the driving environment.

For example, in the case of an environment such as asphalt where the reflectance is constant, the deviation of the deviation image calculated in step S23 is small. Therefore, in the case of an environment such as off-road where Var _th is set small and the reflectance changes, Var Set _th high. Also, without Var _th fixed value, in accordance with the condition of the environment during traveling may be calculated Var _th for each image. For example, based on such as average or median or mode of all pixels of the calculated deviation image in step S23, may be set Var _th.

  Further, when all the feature points extracted in step S36 are filtered in step S38, it is impossible to estimate the amount of movement of the moving object 1. Therefore, all the feature points extracted in step S36 are used in step S39, and the moving object 1 is used. May be estimated. Further, using the distance images saved in time series in step S32, the movement amount of the moving body 1 is calculated in time series, and the movement that could not be estimated this time based on the movement amount pattern of the moving body 1 calculated in time series The movement amount of the body 1 may be estimated. In addition, when all the feature points extracted in step S36 are filtered in step S38, the movement amount of the moving body 1 may be estimated using only the most accurate pixels among the feature points filtered in step S38. .

  6A and 6B are diagrams illustrating a method for calculating a movement amount on a road. FIG. 6A illustrates a case where there are no objects having different reflectances, and FIG. 6B illustrates a case where there are objects having different reflectances.

An image _A 51a illustrated in FIG. 6A is an image captured by the imaging device 2 in step S30. (u, v) are coordinates in pixel units on the image, and the road 52 is a road surface on which the moving body 1 travels. The feature point 53a is the feature point extracted in step S36 in the image _A 51a.

VAR _A 54a is the deviation image calculated in step S34 with respect to the image _A 51a. Since three-dimensional objects with different reflectivities are not shown in the image _A 51a, the deviation of all pixels is small, and the accuracy of the distance calculated in step S31 is high. The high precision pixel 55 is a pixel having a deviation equal to or smaller than the threshold value Var _th determined in step S38.

On the other hand, an image _B 51b shown in FIG. 6B is an image captured by the imaging device 2 in step S30 after the image _A 51a. The feature point 53b is the feature point extracted in step S36 in the image _A 51a, and the feature point tracked in step S37 in the image _B 51b. The three-dimensional object 56 is a three-dimensional object with different reflectivity reflected in the image _B 51b. The three-dimensional object 56 is, for example, a three-dimensional object having a different reflectance from a road such as a water pool or a plant. The feature point 57 is a feature point extracted in step S25 from the three-dimensional object 56 in the image _B 51b.

VAR _B 54b is the deviation image calculated in step S34 with respect to the image _B 51b. Since the solid object 56 having a different reflectance is reflected in the image _B 51b, the deviation of the pixel in which the solid object 56 is reflected is large in the VAR _B 54b, and the accuracy of the distance calculated in step S31 is low.

The low precision pixel 58 is a pixel having a deviation larger than the threshold value Var _th determined in step S38. Here, the low-accuracy pixel 58 is filtered in step S38, and the movement amount of the moving body 1 is estimated in step S39 using only the feature point 53b corresponding to the high-accuracy pixel 55. In addition, even if there is the high-precision pixel 55 around the low-precision pixel 58, the high-precision pixel 55 around the low-precision pixel 58 may be filtered because it may be affected by the low-precision pixel 58.

  FIG. 7 is a diagram illustrating an example of experimental data based on an image acquired by the imaging device, where (a) is a captured image, (b) is an average distance image, and (c) is a deviation image discrimination result. An image 60 shown in FIG. 7A is an image obtained by the imaging device 2. The imaging device 2 is fixed to a predetermined height of a flat surface 60a that is a horizontal plane, and a chess boat 60c is disposed perpendicularly to the flat surface 60a at a depth away from the imaging device 2 by a predetermined distance. 2A, there is a plane 20 perpendicular to the horizontal plane, and the chess board 60c is disposed on the plane 20. As shown in FIG. The plane 60a is a horizontal plane to which the imaging device 2 is fixed. The plane 60b is a face in front of the imaging device 2. The chess board 60c is a three-dimensional object with different reflectivity fixed to the flat surface 60b. The feature point 60d is the feature point extracted in step S36.

  The average distance image 61 shown in FIG. 7B is an average image (AVERAGE image) calculated in step S34. A dark pixel represents a three-dimensional object close to the imaging device 2, and a bright pixel represents a three-dimensional object far from the imaging device 2.

  The deviation image discrimination result 62 shown in FIG. 7C is a result of discriminating (determining) the accuracy of pixels in step S38 using the deviation image calculated in step S34, and discriminating between high-precision pixels and low-precision pixels. is there. Black pixels are pixels that have been filtered in step S38a, and white pixels are pixels that have not been filtered in step S38.

  The pixel 62a is a pixel with low accuracy distinguished in the present embodiment. Since pixel 62a is on a chess board with different reflectivity, it is filtered in step S38a.

  Further, since the imaging device 2 is a TOF sensor, there is a phenomenon in which the error in the distance calculated by the imaging device 2 increases as the three-dimensional object moves away from the imaging device 2 (pixel 62b). Further, since the imaging device 2 is a TOF sensor, the light emitted from the imaging device 2 is reflected a plurality of times at the corner between the plane 60a and the plane 60b, and the flight time until reaching the sensor changes. The error of the calculated distance becomes large (pixel 62c). In the present embodiment, it is also possible to distinguish pixels with low accuracy such as the pixel 62b and the pixel 62c.

DESCRIPTION OF SYMBOLS 1 Mobile body 2 Imaging device (image acquisition device)
3 Processing Device 4 Image Processing Unit (Processing Unit)
5 Control unit 6 Memory (storage unit)
7 Display unit 8 Bus 10 Movement amount calculation device (optical movement amount estimation device)
40 Distance image 41 AVERAGE image 42 VAR image (deviation image)
53a, 53b, 57 Feature points 54a, 54b Deviation image 55 High-precision pixel 56 Three-dimensional object 58 Low-precision pixel 61 Average distance image 62 Deviation image discrimination result

Claims (8)

An imaging device for capturing an image of a road surface;
An image processing unit that calculates a moving amount of the moving body based on a time-series captured image captured by the imaging device;
The image processing unit
A time-series distance image is calculated based on the captured image and stored in the storage unit,
Among the captured images, a plurality of first feature points are extracted from a first captured image captured at a first timing, and a second captured image is captured at a second timing after the first timing. Extracting a plurality of second feature points from the image;
Tracking each of the first feature points to each of the second feature points;
Calculating a deviation image indicating a deviation of distance from the calculated time-series distance image;
A movement amount calculating apparatus, wherein the movement amount of the moving body is calculated based on a feature point having a deviation equal to or less than a predetermined threshold among the plurality of feature points to be tracked. The movement amount calculation apparatus according to claim 1, wherein the image processing unit calculates a deviation image indicating a deviation for each pixel based on the distance image. The movement amount calculation apparatus according to claim 1, wherein the image processing unit determines whether or not there is distance accuracy based on the deviation image. The movement amount calculation apparatus according to claim 1, wherein the image processing unit sets the predetermined threshold according to a driving environment. The movement amount calculation apparatus according to claim 1, wherein the image processing unit sets the number of distance images when calculating the deviation image according to an environment. The movement amount calculation apparatus according to claim 1, wherein the imaging device is a distance image sensor that is installed facing the front of the moving body. When the imaging device is a stereo camera, the image processing unit geometrically calculates a distance for each pixel using two images taken at the same time, and creates the distance image. The movement amount calculation apparatus according to 1. A moving amount calculation method using a processing unit that calculates a moving amount of a moving body based on a captured image of a time-series road surface,
The processor is
A time-series distance image is calculated based on the captured image and stored in the storage unit,
Among the captured images, a plurality of first feature points are extracted from a first captured image captured at a first timing, and a second captured image is captured at a second timing after the first timing. Extracting a plurality of second feature points from the image;
Tracking each of the first feature points to each of the second feature points;
Calculating a deviation image indicating a deviation of distance from the calculated time-series distance image;
A movement amount calculation method comprising: calculating a movement amount of the moving body based on a feature point having a deviation equal to or less than a predetermined threshold among the plurality of feature points to be tracked.