JP5609667B2 - Motion estimation apparatus and program - Google Patents

Description

  The present invention relates to a motion estimation device and program, and more particularly, to a motion estimation device and program for estimating motion of a moving object.

  Traditionally, RANSAC (Random Sampling Consensus) is used to estimate the motion that follows the motion of the most feature points (optical flow), thereby eliminating the influence of a small number of erroneous optical flows and performing robust motion estimation. A motion measuring device to perform is known (for example, Patent Document 1).

JP 2006-350897 A

  However, in the technique described in Patent Document 1 described above, errors occur in many optical flows in a scene where there are many optical flows on a moving object. Therefore, an error is caused by estimating a motion that follows many optical flows. There is a problem that it occurs.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a motion estimation device and program capable of accurately estimating the motion of a moving object.

  In order to achieve the above object, a motion estimation apparatus according to the present invention includes a search unit that searches for feature points corresponding to a plurality of images from each of a plurality of images obtained by imaging the outside of a moving body, and the search A plurality of sets of the corresponding feature points are repeatedly selected from all the sets of the corresponding feature points searched by the means, and the plurality of sets based on the plurality of sets of the corresponding feature points selected repeatedly A motion representing a relative relationship between the position and orientation of the moving body when each of the images is captured as a candidate for motion estimation of the moving body, and a candidate calculating means for calculating the motion calculated by the candidate calculating means. For each of the motion estimation candidates, reliability calculation means for calculating the reliability indicating how reliable the motion estimation candidate is, and the estimation of the motion calculated by the candidate calculation means Based on each of the complements, a degree of variation of the motion estimation candidate is calculated, and based on the degree of variation of the motion estimation candidate, a usability determination unit that determines whether the motion estimation candidate is usable, and the use When it is determined by the availability determining means that the motion estimation candidate is usable, the motion estimation candidate is calculated based on the reliability of each of the motion estimation candidates calculated by the reliability calculation means. Motion estimation means for selecting any one of them as the motion estimation result of the moving body.

  According to the present invention, there is provided a program for searching a computer for a feature point corresponding to a plurality of images from each of a plurality of images obtained by imaging the outside of the moving body, and the corresponding unit searched by the search unit. When a plurality of sets of corresponding feature points are repeatedly selected from all sets of feature points, and each of the plurality of images is captured based on the plurality of sets of corresponding feature points selected repeatedly. A candidate calculating means for calculating a motion representing a relative relationship between the position and orientation of the mobile body as a motion motion candidate for the mobile body, and each of the motion estimation candidates calculated by the candidate calculating means, Reliability calculation means for calculating the reliability indicating how reliable the motion estimation candidate is, based on each of the motion estimation candidates calculated by the candidate calculation means The degree of variation of the motion estimation candidate is calculated, and based on the degree of variation of the motion estimation candidate, the availability determination unit that determines the availability of the motion estimation candidate, and the availability determination unit determine the availability of the motion. When it is determined that the estimation candidate is usable, one of the motion estimation candidates is selected based on the reliability of each of the motion estimation candidates calculated by the reliability calculation means. , A program for functioning as motion estimation means for selecting as a motion estimation result of the moving body.

  According to the present invention, the search unit searches for feature points corresponding to the plurality of images from each of the plurality of images obtained by imaging the outside of the moving body. The candidate calculation means repeatedly selects a plurality of pairs of corresponding feature points from all the pairs of corresponding feature points searched by the search means, and based on the plurality of sets of corresponding feature points repeatedly selected, The motion representing the relative relationship between the position and orientation of the moving body when each of the plurality of images is captured is calculated as an estimation candidate for the motion of the moving body.

  Then, the reliability calculation means calculates, for each of the motion estimation candidates calculated by the candidate calculation means, a reliability indicating how reliable the motion estimation candidate is. The usability determining means calculates the degree of variation of the motion estimation candidate based on each of the motion estimation candidates calculated by the candidate calculation means, and based on the degree of variation of the motion estimation candidate, Judge whether it can be used.

  Then, when it is determined by the motion estimation means that the motion estimation candidate is usable by the availability determination means, the motion is calculated based on the reliability of each of the motion estimation candidates calculated by the reliability calculation means. Any one of the estimation candidates is selected as the motion estimation result of the moving object. In addition, when it is determined that the motion estimation candidate is unusable by the availability determination unit, the motion estimation unit does not output the motion estimation result of the moving object.

  As described above, when it is determined that the motion estimation candidate cannot be used based on the degree of variation of the motion estimation candidate, the motion estimation result is not output, and the motion estimation candidate variation degree is not calculated. Based on the reliability of each of the motion estimation candidates, if one of the motion estimation candidates is determined to be usable, one of the motion estimation candidates is By selecting as the estimation result, it is possible to accurately estimate the motion of the moving body.

  The candidate calculation means according to the present invention can repeatedly select a plurality of sets of corresponding feature points searched by the search means at random, and calculate respective motion candidate motion candidates. Accordingly, the error is dispersed in each of the motion estimation directions and reflected in the degree of variation of the motion estimation candidates, so that the motion of the moving body can be estimated with higher accuracy.

  The motion estimation apparatus according to the present invention includes a motion measurement unit that measures the motion of the moving body using a gyro sensor, a measurement value of the motion of the mobile body by the motion measurement unit, and an estimation of the motion of the mobile body by the motion estimation unit. An offset correction unit that corrects an offset amount of the measurement value of the movement of the moving object measured using the gyro sensor based on the difference from the result can be further included. Thereby, the movement of the moving body can be measured with high accuracy, and the movement of the moving body can be estimated with higher accuracy.

  As described above, according to the motion estimation apparatus and program of the present invention, when it is determined that a motion estimation candidate is unusable based on the degree of variation of motion estimation candidates, the motion estimation result If the motion estimation candidates are determined to be usable based on the degree of variation of the motion estimation candidates, the motion estimation candidates are determined based on the reliability of each motion estimation candidate. By selecting any one of them as the estimation result of the movement of the moving body, an effect that the movement of the moving body can be estimated with high accuracy can be obtained.

It is a block diagram which shows the motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the motion estimation candidate calculation part of the motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the relationship between the extracted feature point and a motion of a moving body. It is a block diagram which shows the structure of the motion estimation part of the motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is an image figure for demonstrating an epipolar line. It is a figure for demonstrating the variance value of a motion estimation candidate. It is a graph which shows the relationship between the error of an optical flow, and the reliability of a scene. It is a flowchart which shows the content of the motion estimation process routine in the motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the motion estimation candidate calculation processing routine in the motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the exercise | movement candidate selection process routine in the exercise | movement estimation apparatus which concerns on the 1st Embodiment of this invention. (A) The figure which shows the optical flow of the feature point on a distant object, (B) The figure which shows a mode that a motion estimation candidate is calculated using the optical flow of the feature point on a distant object. (A) The figure which shows the optical flow of the feature point on the near object, and (B) The figure which shows a mode that the motion estimation candidate is calculated using the optical flow of the feature point on the near object.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The case where the present invention is applied to a motion estimation device mounted on a vehicle will be described as an example.

  As shown in FIG. 1, the motion estimation apparatus 10 according to the first embodiment includes an image capturing unit 12 including a monocular camera that captures an image ahead of the host vehicle, and yaw motion as the host vehicle. Based on the gyro sensor 14 that measures the angular velocity, the image captured by the image capturing unit 12, and the yaw angular velocity measured by the gyro sensor 14, the motion of the host vehicle is estimated and output to an external device (not shown). And a computer 16. The gyro sensor 14 is an example of a motion measuring unit.

  In the present embodiment, a case where a monocular camera is installed in front of the host vehicle and an image in front of the host vehicle is acquired will be described as an example, but an image outside the host vehicle may be captured. A monocular camera may be installed behind the host vehicle to capture the rear of the host vehicle. The monocular camera may be a normal camera with a field angle of about 40 degrees, a wide-angle camera, or an omnidirectional camera. Further, the type of the wavelength of the image to be captured is not limited. There is no limitation on the installation location, the angle of view, the wavelength, and the number of installations as long as the momentum can be estimated appropriately.

  The computer 16 estimates the three-axis angular velocity of the host vehicle and a component indicating the translation direction as the motion of the host vehicle. The computer 16 includes a CPU, a RAM, and a ROM that stores a program for executing a motion estimation processing routine described later, and is functionally configured as follows. The computer 16 includes an image input unit 18 that acquires a plurality of images captured by the image capturing unit 12, a motion estimation candidate calculation unit 20 that calculates a plurality of motion estimation candidates for the host vehicle based on the plurality of images, and a calculation. Based on the plurality of motion estimation candidates, the motion estimation unit 22 that estimates the motion of the host vehicle, the offset correction unit 24 that corrects the offset amount of the measurement value measured by the gyro sensor 14, and the estimated host vehicle And a motion estimation result output unit 26 that outputs a measured value of the yaw angular velocity subjected to the motion or offset correction as an estimation result of the motion of the host vehicle.

  As shown in FIG. 2, the motion estimation candidate calculation unit 20 extracts a feature point extraction unit 30 that extracts a plurality of feature points that can be easily traced on an image from each of a plurality of images obtained by the image capturing unit 12, and a feature. Corresponding feature that searches corresponding feature points (hereinafter referred to as corresponding points) between the two images from the feature points in each of the two images obtained by the point extraction unit 30, and calculates an optical flow based on the corresponding points. From the point search unit 32 and the optical flow obtained by the corresponding point search unit 32, an optical flow grouping unit 34 that selects eight optical flows in a combination of a plurality of types, and each type of combination of selected optical flows, Position and orientation of the image capturing unit 12 when the image of one of the corresponding points retrieved is captured using the combination of the optical flows as an input Changes in the position and orientation of the image capturing unit 12 (relative relationship between positions and orientations) when the other image for which the corresponding point has been searched are captured as a reference are the movement amounts of the motion of the image capturing unit 12 in the XYZ-axis directions. And a motion calculation unit 36 that calculates the rotation amount with reference to the XYZ axes.

  The feature point extraction unit 30 extracts feature points from two images captured at different times obtained from the image capturing unit 12. A feature point refers to a point that can be distinguished from surrounding points and that makes it easy to obtain a correspondence between different images. The feature points are automatically extracted by using a method (for example, the Tomasi-Kanade method or the Harris operator) that detects pixels in which the gradient value of the change in shading increases two-dimensionally. The number of feature points may be approximately 30 to 500 points for all images, but may be fixed or variable depending on the situation. The method is not limited to the above as long as it is an appropriate extraction method.

  In the method using the Harris operator, feature points are extracted as described below. First, the matrix M is calculated by the following equation (1), where the luminance of the point (u, v) of the image is I (u, v).

However, I _u and I _v represent horizontal and vertical differentiation, respectively, and G _σ represents smoothing by a Gaussian distribution with a standard deviation σ.

Then, using the eigenvalues λ ₁ and λ ₂ of the matrix M calculated by the above equation (1), the corner strength is calculated by the following equation (2).

  However, k is a preset constant, and a value of 0.04 to 0.06 is generally used. In the method using the Harris operator, a point at which the corner intensity is equal to or greater than a threshold value is selected, and the selected point is extracted as a feature point.

  The corresponding point search unit 32 searches the corresponding points between the two images by associating the two points of the feature points extracted from each of the two images by the feature point extracting unit 30. In the association of feature points between images, if the corresponding points are the same, it is assumed that the luminance of the corresponding points in image 1 and image 2 and the surrounding points hardly change, and based on this assumption, each feature Do point association. For example, a set of points having similar brightness distributions in a small area set around a feature point is selected, and the selected set of points is set as a corresponding point. A well-known technique for feature point association (feature point tracking) includes the Lucas-Kanade method (Lucas-Kanade method). The corresponding point search unit 32 calculates an optical flow for each of the searched corresponding points.

  The optical flow grouping unit 34 arbitrarily selects eight optical flows used for motion estimation from all the optical flows calculated for all corresponding points for which feature point association (tracking) has been completed. The motion estimation candidate calculated by the motion calculation unit 36 is determined by the combination of the optical flows selected here. The optical flow grouping unit 34 repeats the selection of eight optical flows N times, and generates a combination of N types of optical flows. The method of selecting the optical flow may be random or any method, but it is preferable to select the optical flow at random so that errors contained in the optical flow are dispersed. Also, eight or more optical flows may be selected.

For each of a plurality of combinations of the eight optical flows selected by the optical flow grouping unit 34, the motion calculation unit 36 captures images when each of the two images is captured from the eight optical flows in the combination. Changes in the position and orientation of the unit 12 (the amount of movement in the XYZ axis direction and the amount of rotation based on the XYZ axis) are calculated. As shown in FIG. 3, the change of the position and orientation is based on the rotation matrix R from the first image to the second image (the rotation amount based on the X axis, the rotation amount based on the Y axis, and the Z axis based on And a translation vector t (a movement amount t _{x in} the X-axis direction, a movement amount t _{y in} the Y-axis direction, and a movement amount t _{z in the} Z-axis direction). Note that the elements of the rotation matrix R and the translation vector t are physical quantities representing conversion of image coordinates between two images.

Here, a calculation method of the rotation matrix R and the translation vector t from the first image to the second image will be described. For the first image coordinates of the corresponding point of the n points and the image coordinates x ₁ of the corresponding point of n points in the second image in the image x _{2 (start} and end points of the optical flow) (n ≧ 8), corresponding points are correct If there is no error, there is a 3 × 3 matrix F that satisfies the following expression (3).

Here, if there are eight or more pairs of corresponding points x ₁ and x ₂ , the basic matrix F can be calculated.

  As described above, when the basic matrix F can be calculated and the calibration matrix K of the image capturing unit 12 is known, the calibration matrix K for correcting image distortion due to the imaging characteristics of the camera, and the basic matrix Using F, the rotation matrix R and the translation vector t can be calculated from the following equations (4) and (5).

  The motion calculated here is set as a motion estimation candidate. By repeating the selection of the optical flow by the optical flow grouping unit 34 and the calculation by the motion calculation unit 36 N times, N motion estimation candidates are obtained.

  Further, as shown in FIG. 4, the motion estimation unit 22 selects, for each of a plurality of motion estimation candidates, a feature point selection unit 40 that selects a feature point whose motion matches the motion estimation candidate, and each motion estimation candidate. A reliability determination unit 42 that determines the reliability of the motion estimation candidate from the number of feature points selected with respect to the motion candidate selection unit 44 that rearranges and selects the motion estimation candidates based on the reliability of the motion estimation candidates; Based on a plurality of motion estimation candidates, the reliability of the optical flow group in the image is determined as the reliability of the scene, and the reliability of the scene determined by the scene reliability determination unit 46 And a use determination unit 48 that determines whether or not the motion estimation candidate selected by the motion candidate selection unit 44 can be output.

  The feature point selection unit 40 selects, for each of a plurality of motion estimation candidates, a feature point whose motion matches the motion estimation candidate from among a plurality of feature points used when calculating the motion estimation candidate. .

For example, as shown in FIG. 5, when the feature point P located at the coordinate x ₁ in the image 1 moves according to the motion represented by the basic matrix F, the coordinate of the feature point P in the image 2 is a line represented by Fx _1. Located on (epipolar line). That is, it can be said that the closer the distance between the corresponding feature point in the image 2 and the epipolar line is, the higher the movement of the feature point is with the motion estimation candidate. Using this property, among the feature points used when calculating the motion estimation candidate, the feature point whose distance from the corresponding epipolar line is equal to or less than the threshold is selected.

  For each of the plurality of motion estimation candidates, the reliability determination unit 42 selects the reliability of the motion estimation candidate based on the number of feature points that are selected by the feature point selection unit 40 and whose motion matches the motion estimation candidate. Determine the degree. The higher the number of feature points, the higher the reliability is determined. As the reliability index, the number of selected feature points may be used as it is, or a result obtained by applying an arbitrary function to the number of feature points may be used.

The motion candidate selection unit 44 rearranges the N motion estimation candidates in descending order of the calculated reliability, and the motion estimation with the highest ranking among the motion estimation candidates M _{1 to} N _N ranked according to the reliability. Candidates are output.

The scene reliability determination unit 46 calculates the reliability of the optical flow group in the image as the reliability of the scene based on the _N motion estimation candidates M _{1 to} M _N obtained by the motion estimation candidate calculation unit 20. First, as illustrated in FIG. 6, the variance value σ of the motion estimation candidates M _{1 to} M _N is calculated as the degree of variation of the motion estimation candidates. Next, the reliability of the scene is calculated based on the variance value σ. For example, the reliability of the scene is calculated according to the following equation (6).
R = log (1 + 1 / σ) (6)
By the above equation (6), the reliability of the scene that is inversely proportional to the optical flow error is obtained (see FIG. 7). FIG. 7 shows the relationship between the error average value [pixel] when an error is randomly given to each optical flow and the reliability of the scene.

In the calculation of the variance value, all of the motion estimation candidates M _{1 to} M _N may be used, or a part selected by comparison with a measurement result using the gyro sensor 14 may be used. Of the three rotational components of motion (pitch, yaw, roll) and the three translational components (tx, ty, tz), the motion component used to calculate the variance value may be one component, multiple components, or multiple A combination of ingredients may be used.

  The use determination unit 48 determines whether the motion estimation result selected by the motion candidate selection unit 44 can be output based on the reliability of the scene determined by the scene reliability determination unit 46. For example, when θ is a threshold value regarding the reliability R of the scene, it is determined that the selected motion estimation result can be output only when R ≧ θ is satisfied. As a result, only a motion estimation result with an error value equal to or less than a certain value is output. On the other hand, when R <θ, the use determination unit 48 determines that the motion estimation result is unusable. For example, in a scene with many erroneous optical flows, since the reliability R of the scene is low, a motion estimation result is not output.

Offset correction unit 24 obtains the measured value of the yaw angular velocity measured by the gyro sensor 14, a value obtained by correcting an offset (steady deviation) of the measurement values of the gyro sensor 14, and outputs as the motion measurement result Y _g . For the output value of the motion estimator 22 and the measured value of the gyro sensor 14, the difference between the average values of the past fixed period from the current time is calculated and set as the offset correction value. The offset correction value is calculated for each time step. However, when all the motion estimation candidates calculated by the motion estimation candidate calculation unit 20 are not adopted, the motion estimation unit 22 calculates an average value except for the input of the time.

When the motion estimation candidate is output from the motion estimation unit 22, the motion estimation result output unit 26 outputs the motion estimation candidate as a motion estimation result, and the motion estimation candidate 22 is not output. In this case, Y _g output from the offset correction unit 24 is output as the motion estimation result. The motion estimation result output unit 26 may output an appropriate character string when a motion estimation candidate is not output from the motion estimation unit 22, or a large value that cannot normally be obtained as a motion estimation value. May be output.

  Next, the operation of the motion estimation apparatus 10 according to the first embodiment will be described. In addition, the case where the motion of the own vehicle is estimated when the own vehicle carrying the motion estimation device 10 is traveling will be described as an example.

  First, in the motion estimation apparatus 10, a motion estimation processing routine shown in FIG. 8 is executed. In step 100, the first image and the second image obtained by imaging the front of the host vehicle at different timings are acquired, and the gyro sensor 14 captures the first image and the second image. Get the measured yaw angular velocity. In step 102, a plurality of motion estimation candidates are calculated based on the first image and the second image acquired in step 100.

  Step 102 is realized by the motion estimation candidate calculation processing routine shown in FIG. First, in step 120, a predetermined number of feature points are extracted from each of the first image and the second image. Then, in step 122, each feature point in the first image extracted in step 120 is tracked in the second image, and each corresponding feature point is searched from the second image. Each optical flow of the corresponding points is calculated.

  In step 124, a variable n indicating the number of motion estimation candidates calculated is set to an initial value of 1. In step 126, a combination of eight optical flows is selected from the plurality of optical flows calculated in step 122. Select at random. In step 128, from the eight optical flows selected in step 126, the relative positional relationship of the image capturing unit 12 between the two image capturing times, that is, between the two times of the host vehicle on which the image capturing unit 12 is mounted. Estimate the motion at.

  In step 130, it is determined whether the variable n is less than a constant N representing the number of motion estimation candidates to be calculated. If N motion estimation candidates are not calculated, in step 132, The variable n is incremented and the process returns to step 126. On the other hand, if the variable n reaches N, the motion estimation candidate calculation processing routine is terminated. As described above, when the motion estimation candidate calculation processing routine is executed, N motion estimation candidates are output.

  In step 104 of the motion estimation processing routine, one motion estimation candidate is selected from the N motion estimation candidates calculated in step 102. Step 104 is realized by the exercise candidate selection processing routine shown in FIG.

  First, in step 140, a motion estimation candidate to be determined is set, and in step 142, an epipolar line in the second image is obtained for each of a plurality of feature points used when calculating the motion estimation candidate to be determined. Ask.

  In step 144, a feature point in which the distance between the corresponding feature point in the second image and the epipolar line is equal to or less than a threshold is selected from the plurality of feature points used in calculating the motion estimation candidate to be determined. To do. In the next step 146, the reliability of the motion estimation candidate to be determined is determined based on the number of feature points selected in step 144.

  In step 148, it is determined whether or not reliability determination has been performed for all motion estimation candidates. If there is a motion estimation candidate for which reliability has not been determined, the process returns to step 140. When the reliability determination is performed for all the motion estimation candidates, the process proceeds to step 150.

  In step 150, based on the reliability determined for each motion estimation candidate, the N motion estimation candidates are rearranged in descending order of reliability, the motion estimation candidate with the highest reliability is selected, and motion candidate selection is performed. The processing routine ends.

  In step 106 of the motion estimation processing routine, the reliability of the scene is determined based on the N motion estimation candidates calculated in step 102. In the next step 108, the reliability of the scene determined in step 106 is compared with a threshold value to determine whether or not the motion estimation candidate selected in step 104 is usable. If the reliability of the scene is equal to or greater than the threshold value, it is determined that the selected motion estimation candidate is usable. In step 110, the motion estimation candidate selected in step 104 is used as the motion estimation result. Output.

  In step 112, the average value of the yaw angular velocity of the motion estimated in the past fixed period and the average value of the yaw angular velocity measured by the gyro sensor 14 in the past fixed period are calculated. In step 114, the above step 112 The difference between the average values of the yaw angular velocities calculated in step 1 is calculated as an offset correction value, and the process returns to step 100.

  On the other hand, if it is determined in step 108 that the reliability of the scene is less than the threshold value and the selected motion estimation candidate is unusable, the offset previously calculated in step 114 is determined in step 116. Using the correction value, the offset amount of the yaw angular velocity acquired in step 100 is corrected. In the next step 118, the yaw angular velocity corrected in step 116 is output as the motion estimation result of the host vehicle, and the process returns to step 100.

  As described above, according to the motion estimation device according to the first embodiment, it is determined that the motion estimation candidate is unusable based on the reliability of the scene based on the variance value of the motion estimation candidate. If the motion estimation candidate is determined to be usable based on the reliability of the scene based on the variance value of the motion estimation candidate without outputting the motion estimation result, each of the motion estimation candidates is output. By selecting any one of the motion estimation candidates as the motion estimation result of the moving body based on the reliability, the motion of the host vehicle can be accurately estimated.

  In addition, when many feature points move incorrectly (including movements different from the movement of the host vehicle) due to changes in lighting conditions or movement of moving objects, motion estimation is calculated from a randomly generated optical flow group. Since each candidate includes an error at random, the variance value of a plurality of motion estimation candidates increases. Therefore, by using the variance value of motion estimation candidates as an index, it is possible to distinguish a scene with many erroneous motion feature points (a scene with a high possibility of causing a large motion estimation error), and a motion estimation error is likely to occur. Whether it is a scene or not can be determined. In addition, by performing this determination, it is possible to extract only a motion with high reliability from the calculated motion estimation candidates.

  In general, a distant object in an image has a lower resolution (the number of pixels representing an object region) than a nearby object. For this reason, the influence of an error per pixel is greater in a distant object than in a nearby object. For example, when the optical flow in the image uniformly has an error of one pixel, the vehicle motion estimation is performed rather than performing the vehicle motion estimation using the optical flow on a nearby object as illustrated in FIG. The estimation accuracy is greatly degraded when the vehicle motion estimation is performed using the optical flow on the distant object as shown in FIG.

  In this embodiment, a plurality of optical flows are selected from the optical flow group, a plurality of types of combinations are created, a motion estimation candidate is calculated from each combination, and the reliability of the entire optical flow group is determined based on the variance value. Evaluating. Therefore, by evaluating the variance value of the motion estimation candidate calculated from the selected optical flow instead of the error of each optical flow pixel, it is possible to reflect the difference in error due to the above-described perspective difference. That is, as shown in FIG. 11B, if the selected optical flow is an optical flow on a nearby object, the error of the optical flow is reduced and the variance of motion candidates is reduced, so that the reliability of the scene is increased. Get higher. On the other hand, as shown in FIG. 12B, if the selected optical flow is an optical flow on a distant object, the error of the optical flow becomes large and the variance of motion candidates becomes large. Lower. Therefore, it is possible to accurately estimate the motion of the host vehicle by reflecting the difference in the influence of the error due to the difference in perspective.

  Further, by correcting the offset amount of the measurement value by the gyro sensor, it is possible to accurately measure the yaw angular velocity of the own vehicle, and thereby it is possible to estimate the motion of the own vehicle more accurately.

  In addition, in motion estimation based on captured images, a plurality of motion estimation candidates are calculated, and when it is determined that the motion estimation candidates are unusable, an appropriate motion can be obtained by using the measurement value of the gyro sensor as the motion estimation result. The estimation result can be acquired stably.

  Next, a second embodiment will be described. In addition, since the motion estimation apparatus according to the second embodiment has the same configuration as that of the first embodiment, the same reference numerals are given and description thereof is omitted.

  The second embodiment is different from the first embodiment in that a motion estimation candidate is selected from motion estimation candidates whose difference from a measurement value using a gyro sensor is within a certain range. .

In the motion estimation apparatus according to the second embodiment, the motion candidate selection unit 44 acquires the measured value of the yaw angular velocity measured by the gyro sensor 14, and the measured values of the yaw angular velocity for the N motion estimation candidates are obtained. Motion estimation candidates for which the difference between them is within a certain range are selected. In addition, the motion candidate selection unit 44 rearranges the selected motion estimation candidates (for example, L motion estimation candidates) in descending order of the calculated reliability, and the motion estimation candidates M ranked according to the reliability. _A motion estimation candidate having the highest rank among _{1 to ML} (L ≦ N) is selected and output.

  In addition, about the other structure and process of the motion estimation apparatus which concern on 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

  In the first to second embodiments described above, the case where the reliability of the motion estimation candidate is determined based on the number of selected feature points has been described as an example. However, the present invention is not limited thereto. Instead, the reliability of the motion estimation candidate may be determined using the average value of the distances between each of the plurality of feature points used in calculating the motion estimation candidate and the epipolar line.

  Moreover, although the case where the availability of the motion estimation candidate is determined after selecting one of the calculated motion estimation candidates has been described as an example, the present invention is not limited to this. If it is determined that the motion estimation candidate is usable, any one of the plurality of calculated motion estimation candidates may be selected.

  In addition, the case where the motion estimation candidate is calculated every time a combination of eight optical flows is selected has been described as an example. However, the present invention is not limited to this, and a plurality of groups composed of combinations of eight optical flows are generated. Then, the motion estimation candidate may be calculated for each group.

  Further, the case where the optical flow is calculated for the searched corresponding points has been described as an example. However, the present invention is not limited to this, and eight corresponding points are calculated from all the searched corresponding points without calculating the optical flow. The motion estimation candidates may be calculated for the eight selected corresponding points.

  Further, the case where the yaw angular velocity is measured by the gyro sensor has been described as an example. However, the present invention is not limited to this, and the three-axis angular velocity may be measured by the gyro sensor, and the yaw angular velocity, pitch angular velocity, In addition, at least one of the roll angular velocity may be measured. Further, at least one of the three translational components may be measured by a sensor.

  The program of the present invention may be provided by being stored in a storage medium.

DESCRIPTION OF SYMBOLS 10 Motion estimation apparatus 12 Image pick-up part 14 Gyro sensor 16 Computer 20 Motion estimation candidate calculation part 22 Motion estimation part 24 Offset correction part 26 Motion estimation result output part 30 Feature point extraction part 32 Corresponding point search part 34 Optical flow grouping part 36 Motion Calculation unit 40 Feature point selection unit 42 Reliability determination unit 44 Motion candidate selection unit 46 Scene reliability determination unit 48 Usage determination unit

Claims (4)

Search means for searching for feature points corresponding to the plurality of images from each of a plurality of images obtained by imaging the outside of the moving body;
From all the sets of corresponding feature points searched by the search means, repeatedly select a plurality of sets of the corresponding feature points, and based on the plurality of sets of the corresponding feature points repeatedly selected, Candidate calculation means for calculating motions representing the relative relationship between the position and orientation of the moving body when each of the plurality of images is captured, as motion motion estimation candidates, respectively.
For each of the motion estimation candidates calculated by the candidate calculation means, reliability calculation means for calculating the reliability indicating how reliable the motion estimation candidate is,
Based on each of the motion estimation candidates calculated by the candidate calculation means, the variation degree of the motion estimation candidates is calculated, and based on the variation degree of the motion estimation candidates, the use of the motion estimation candidates Availability determination means for determining availability; and
When it is determined that the motion estimation candidate is usable by the usability determination unit, the motion estimation is performed based on the reliability of each of the motion estimation candidates calculated by the reliability calculation unit. Motion estimation means for selecting any one of the candidates as a motion estimation result of the moving body;
A motion estimation device including:   The motion estimation apparatus according to claim 1, wherein the candidate calculation unit repeatedly selects a plurality of sets of the corresponding feature points searched by the search unit at random, and calculates each motion estimation candidate of the moving object. . Motion measuring means for measuring the motion of the moving body using a gyro sensor;
Based on the difference between the measured value of the motion of the moving body by the motion measuring means and the estimation result of the motion of the moving body by the motion estimating means, the motion of the moving body measured using the gyro sensor is measured. The motion estimation apparatus according to claim 1, further comprising offset correction means for correcting an offset amount of the measurement value. Computer
Search means for searching corresponding feature points between the plurality of images from each of a plurality of images obtained by imaging the outside of the moving body;
From all the sets of corresponding feature points searched by the search means, repeatedly select a plurality of sets of the corresponding feature points, and based on the plurality of sets of the corresponding feature points repeatedly selected, Candidate calculation means for calculating a motion representing a relative relationship between the position and orientation of the moving body when each of the plurality of images is captured as an estimation candidate for the motion of the moving body,
For each of the motion estimation candidates calculated by the candidate calculation means, reliability calculation means for calculating the reliability indicating how reliable the motion estimation candidate is,
Based on each of the motion estimation candidates calculated by the candidate calculation means, the variation degree of the motion estimation candidates is calculated, and based on the variation degree of the motion estimation candidates, the use of the motion estimation candidates And a determination unit for determining whether or not the motion estimation candidate is usable by the availability determination unit, and each of the motion estimation candidates calculated by the reliability calculation unit is determined. A program for causing any one of the motion estimation candidates to function as motion estimation means for selecting as a motion estimation result of the moving body based on the reliability.

Images