JP4817901B2 - Correlation evaluation system and correlation evaluation method - Google Patents

Description

  The present invention relates to a correlation evaluation system and a correlation evaluation method for evaluating positional reliability of a correlation destination area specified as a correlation destination of a correlation source area.

  2. Description of the Related Art Conventionally, a technique for specifying in detail the position of a correlation destination area as a correlation destination of a correlation source area is known. For example, Patent Document 1 discloses a technique for specifying the position of a correlation destination region in a stereo image at a subpixel level. In this method, the subpixel component is calculated based on the city block distance (hereinafter referred to as CB distance) of the correlation destination region and the CB distance of the correlation destination candidate that is positioned immediately before or after the correlation destination region. By using this sub-pixel component, the amount of deviation (parallax) at the sub-pixel level is calculated by interpolating the amount of deviation of the pixel level (component level of the data area) of the correlation destination area with respect to the correlation source area.

JP 2003-97944 A

  However, in the conventional technique, an error is included in the position of the correlation destination region at the calculated subpixel level due to noise as disturbance that is carried on the image data (data region). This error may reduce the accuracy of various subsequent processes using the position of the correlation destination region as an input.

  An object of the present invention is to improve the accuracy of various processes using the position of a correlation destination area as an input by evaluating the positional reliability of the correlation destination area specified as the correlation destination of the correlation source area. .

  Another object of the present invention is to enable various types of processing with excellent noise resistance by evaluating the positional reliability of the correlation destination region in consideration of the influence of noise.

In order to solve such a problem, the first invention provides a correlation evaluation system that evaluates the positional reliability of a correlation destination area specified as a correlation destination of a correlation source area. The evaluation system includes a matching processing unit, an interpolation processing unit, a reliability evaluation unit, and a data correction unit . The matching processing unit sets a plurality of correlation destination candidates as correlation destination candidates in the correlation source area within a predetermined search range, and indicates the degree of correlation with the correlation source area for each of the plurality of correlation destination candidates. Calculate the value. Then, the matching processing unit identifies one of the plurality of correlation destination candidates as a correlation destination region by comparing the correlation values calculated for each correlation destination candidate. Further, the matching processing unit calculates the position of the correlation destination region as a discrete first coordinate value. The interpolation processing unit converts the position of the correlation destination region to the first coordinate value by interpolation using a correlation value of the correlation destination region and correlation values of two or more correlation destination candidates that are positionally adjacent to the correlation destination region. The second coordinate value having a finer resolution is calculated. The reliability evaluation unit includes a first value indicating an error range of the correlation value according to an amount of change between the correlation value of the correlation destination region and the correlation value of the correlation destination candidate that is positioned adjacent to the correlation destination region. The error width is projected on the coordinate axis indicating the position of the correlation destination area, and specified by the first coordinate value and the second coordinate value based on the second error width generated on the coordinate axis by the projection. The reliability regarding the position of the correlation destination area is evaluated. The data correction unit is provided in front of the matching processing unit, and performs a predetermined correction on the data before correction and outputs the corrected data. In this case, the reliability evaluation unit variably sets the first error width according to the degree of correction of the data after correction with respect to the data before correction.

  In the first invention, the reliability evaluation unit evaluates the reliability lower as the amount of change between the correlation value of the correlation destination region and the correlation value of the correlation destination candidate positioned adjacent to the correlation destination region is smaller. It is desirable to do.

  In the first invention, a representative value calculation unit that calculates a representative value representing the data based on data included in the correlation source region or the search range may be provided. In this case, the reliability evaluation unit variably sets the first error width according to the representative value.

  In the first invention, a sensor and an amplifier may be provided. The sensor outputs a frame region that includes a data group having a predetermined size and includes a region used as a correlation source region or a search range. The amplifier is provided before the matching processing unit, and adjusts the frame region output from the sensor by the gain. In this case, the reliability evaluation unit variably sets the first error width according to the gain.

  Furthermore, in the first invention, a temperature sensor for detecting the ambient temperature may be further provided. In this case, the reliability evaluation unit variably sets the first error width based on the temperature.

The second invention provides a correlation evaluation method for evaluating the positional reliability of the correlation destination area specified as the correlation destination of the correlation source area. In this evaluation method, a predetermined correction is performed on the uncorrected data, and then the corrected data is output, and a plurality of correlation destination candidates that are candidates for the correlation destination in the correlation source region are determined by a predetermined search. By setting the value within the range and calculating the correlation value indicating the degree of correlation with the correlation source region for each of the plurality of correlation destination candidates, by comparing the correlation values calculated for each correlation destination candidate, A step of specifying any one of a plurality of correlation destination candidates as a correlation destination region, a step of calculating a position of the correlation destination region as a discrete first coordinate value, a correlation value of the correlation destination region, a correlation destination region, Calculating a position of the correlation destination region as a second coordinate value having a finer resolution than the first coordinate value by interpolation using correlation values of two or more correlation destination candidates that are adjacent in position; Correlation of correlation destination area And the first error width indicating the error range of the correlation value on the coordinate axis indicating the position of the correlation destination area according to the amount of change between the correlation value and the correlation value of the correlation destination candidate that is positioned adjacent to the correlation destination area. And the reliability of the position of the correlation destination area specified by the first coordinate value and the second coordinate value is evaluated based on the second error width generated on the coordinate axis by the projection. A step of performing. In the step of evaluating the reliability, the first error width is variably set according to the degree of correction of the data after correction with respect to the data before correction.

  According to the present invention, the reliability regarding the position of the correlation destination region is evaluated according to the amount of change in the correlation value of the correlation destination candidate that is positioned adjacent to the correlation destination region based on the correlation value of the correlation destination region. It will be. By using the result of the evaluation related to reliability for various processes using the position of the correlation destination area as an input, the accuracy of these various processes can be improved. For example, by considering the influence of noise and setting the relationship between the change amount of the correlation value and the reliability, various processes with excellent noise resistance can be performed.

(First embodiment)
FIG. 1 is a block configuration diagram of a correlation evaluation system according to the present embodiment. This correlation evaluation system evaluates the positional reliability of the correlation destination area specified as the correlation destination of the correlation source area. Data that defines the first data area is stored in the storage unit 1a, and data that defines the second data area is stored in the storage unit 1b. The correlation source area constitutes a part of the first data area, and the correlation destination area (and the correlation destination candidate) constitutes a part of the second data area. The predetermined input unit outputs a pair of data groups corresponding to the pair of data areas to the storage units 1a and 1b. For example, the pair of data groups includes a pair of frame image data (stereo image data) captured simultaneously. In this case, a stereo camera is used for the input unit.

  The matching processing unit 2 sets a plurality of correlation destination candidates as correlation destination candidates in the correlation source area within a predetermined search range on the second data area. A correlation value S is calculated for each of the plurality of correlation destination candidates. The correlation value S indicates the degree of correlation between the correlation source area and the correlation destination area. The matching processing unit 2 compares the correlation values S for each correlation destination candidate, and thereby specifies any one of a plurality of correlation destination candidates as a correlation destination region. For example, in the ij coordinate system of the first data area and the second data area which are two-dimensional planes, the lower left corner of the data area is the origin, the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. Data is associated with an element of a grid-point data area that takes an integer value on this coordinate. As an example, the correlation source region can be a small region of 4 × 4 elements (4 elements in the horizontal direction and 4 elements in the vertical direction). A correlation destination area having a correlation between the correlation source area and data characteristics is specified in, for example, a horizontally long 128 × 4 search range. In the search range, the correlation value S is calculated while updating the correlation destination candidates by shifting one element at a time in the horizontal direction from the left end to the right end of the search range. The correlation destination candidate is specified by a horizontal shift amount k with respect to the correlation source region, and the correlation value S is expressed as a function S (k) of k. The correlation destination candidate (k = kp) that minimizes the correlation value S (k) is specified as the correlation destination region. The coordinate value kp of the integer level in the horizontal direction of the correlation destination region is calculated and output to the interpolation processing unit 3 together with the correlation value S (kp). Further, the correlation value S (kp−1) of the correlation destination candidate adjacent to the left side (immediately before) of the correlation destination region, and the correlation value S (kp + 1) of the correlation destination candidate adjacent to the right side (immediately after) of the correlation destination region. Is output to the interpolation processing unit 3. Here, on the assumption that a two-dimensional data group is used, a range on the second data area corresponding to an area obtained by moving the correlation source area on the first data area in the horizontal direction is set as a search range. is doing. Instead of this, a three-dimensional or more data group may be used, and a range corresponding to an area obtained by moving the correlation source area in an arbitrary direction may be set as a search range.

As the correlation value S (k), for example, an absolute difference sum can be used. The correlation value S (k) using the sum of absolute differences is calculated as a difference (absolute value) between data of elements corresponding to each other in a pair of data areas to be evaluated, as in Equation 1. This value is expressed as a total value obtained by adding all elements. In the equation, i and j are the lower left coordinates of the correlation source region that specifies the position of the correlation source region, and k is a horizontal shift amount from the correlation source region to the correlation destination candidate. R is data corresponding to each element in the correlation source region, and L is data corresponding to each element in the correlation destination candidate. m and n are the sizes of the correlation source region and the correlation destination candidate. Further, as the correlation value S (k), other known evaluation functions such as a sum of squared differences can be used. The correlation value S (k) using the sum of squared differences is a sum value obtained by calculating the square of the luminance difference between the elements corresponding to each other in the pair of data areas to be evaluated and adding this value for all the elements. Represented as:

  When these correlation evaluation methods are used, the correlation value S (k) decreases as the correlation between a pair of data regions increases, in other words, the similarity between the data characteristics of the two data regions, and the data regions that are completely coincident with each other. Theoretically, the correlation value S (k) becomes zero. However, in actuality, various noises act as disturbances, and therefore, even in a data region that should originally be perfectly matched, the correlation value S (k) is hardly zero.

The interpolation processing unit 3 interpolates using the correlation value S (kp) of the correlation destination region and the correlation values S (kp−1) and S (kp + 1) of the adjacent correlation destination candidates. By this interpolation, the position (displacement amount) of the correlation destination area in the ij coordinate system to which the correlation destination area belongs is calculated as a decimal level coordinate value. Here, the horizontal shift amount of this correlation destination area is specified as the value kr shown in the following Equation 2, that is, the sum of kp (integer part) and ksp (decimal part). The deviation amount kr and the correlation values S (kp−1), S (kp), and S (kp + 1) are output to the reliability evaluation unit 4.

Specifically, the interpolation processing unit 3 calculates the right side of the following equations 3 and 4 to obtain the decimal part ksp. Equation 3 shows the case where the correlation value S (kp−1) is larger than the correlation value S (kp + 1) (in other words, the case where the correlation value S (k) changes largely before the coordinate value kp). . Equation 4 shows a case where the correlation value S (kp−1) is equal to or smaller than the correlation value S (kp + 1) (a case where the correlation value S (k) changes greatly after the coordinate value kp). To be exact, in Equation 4, ksp is negative. In this case, (kp-1) indicates the integer part of the correlation value kr, and (1-ksp) indicates the decimal part of the shift amount kr.

  For example, the interpolation of the subpixel component (decimal component) for the parallax d in the stereo image data is disclosed in Japanese Patent Laid-Open No. 2000-283653, and should be referred to if necessary. Alternatively, a subpixel component interpolation method disclosed in Japanese Patent Laid-Open No. 2003-97944 can be used instead. In this interpolation method, not all of the above three correlation values are used, and the correlation value S (kp) and the smaller correlation value of the adjacent correlation values S (kp-1) and S (kp + 1) are obtained. Used to perform sub-pixel level interpolation.

  First, the reliability evaluation unit 4 calculates the error width E2 based on the variation amount of the adjacent correlation value S (kp-1) or S (kp + 1) with the correlation value S (kp) as a reference, and the error width E1. presume. The error width E1 indicates the error range of the correlation value S (k), and the error width E2 indicates the error range of the deviation amount kr of the correlation destination region. As the error width E1, a fixed value obtained by estimating the variation of noise superimposed on the correlation value S (k) may be set, and as the error width E1, a variable value corresponding to the noise factor that fluctuates the error is used. May be. The case where a variable value is used for the error width E1 will be described in the second embodiment. Next, the reliability evaluation unit 4 evaluates the reliability related to the shift amount kr of the correlation destination region based on the error width E2.

  FIG. 2 is a diagram showing the relationship between the error width E1 of the correlation value S (k) and the error width E2 of the deviation amount kr of the correlation destination candidate (and the decimal part ksp of the deviation amount). The case where the correlation value S (kp−1) is larger than the correlation value S (kp + 1) will be described, but the same applies to the case where the correlation value S (kp−1) is equal to or smaller than the correlation value S (kp + 1). In FIG. 9A, the change ΔS1 between the reference correlation value S (kp) and the adjacent correlation value S (kp−1) is relatively large. The error width E2 is estimated on the coordinates where the horizontal axis is the deviation amount k of the correlation destination candidate and the vertical axis is the correlation value S (k). That is, the error width E2 is estimated by projecting the error width E1 in the vertical axis direction onto the error width E2 in the horizontal axis direction in accordance with the change amount ΔS1 of the correlation value. Point P1 (kp, S (kp)) corresponds to the correlation destination region, and point P0 (kp-1, S (kp-1)) corresponds to the adjacent correlation destination candidate immediately before the correlation destination region. Further, the point P2 (kp + 1, S (kp + 1)) corresponds to the adjacent correlation destination candidate immediately after the correlation destination region. The straight line l1 is a straight line having a slope −m (m is a positive number) passing through the points P0 and P1, and the straight line l2 is a straight line having a slope m passing through the point P2. Here, the change amount ΔS1 of the correlation value corresponds to a change amount of one unit of the deviation amount k of the correlation destination candidate. Accordingly, the change amount ΔS1 is equal to the magnitude m of the slope of the straight line l1. In particular, it is assumed that correlation values S (kp−1), S (kp), and S (kp + 1), which are true values, have an error with an error width E1 around each value. Under this assumption, the error width E2 resulting from the projection of the error width E1 is calculated as the length of the line segment connecting the point A and the point B in FIG. Point A is an intersection of a straight line defining the error width E1 below the straight line l1 and a straight line defining the error width E1 above the straight line l2, and point B is an error width E1 above the straight line l1. This is the intersection of the prescribed straight line and the straight line defining the error width E1 below the straight line l2.

  In FIG. 5B, the correlation value change amount ΔS1 (the absolute value of the slope of the straight line l) is relatively small. Naturally, the error width E1 of the correlation value S (k) is equal to the magnitude of the change amount ΔS1, and the error width E2 of the shift amount kr becomes smaller as the change amount ΔS1 becomes smaller, comparing FIGS. It turns out that becomes large.

It is also confirmed from the error Nksp of the fractional part ksp shown in equations 5 and 6 that the error width E2 fluctuates in accordance with the change amount ΔS1 of the correlation value and the error width E1.

  In the formula, N (0, σ) represents a variable having a predetermined distribution with an average of 0 and a standard deviation of σ. This N (0, σ) is determined based on the noise superimposed on the correlation value S (k), as shown in Equations 7 and 8 below. That is, the standard deviation σ of N (0, σ) corresponds to the error width E1, which is a constant value obtained by estimating the noise variation. For σ, a value confirmed in advance through experiments or simulations is used. Further, the variation of the error Nksp calculated by the mathematical expressions 5 and 6 corresponds to the error width E2. The error Nksp varies depending on the value of the denominator, and the error width E2 also varies accordingly.

  Equation 5 shows the error Nksp of the fractional part ksp when the adjacent correlation value S (kp-1) is larger than the adjacent correlation value S (kp + 1) (in the case of FIG. 2). In this case, the correlation value variation ΔS1 is used as the denominator of the error Nksp. As described above, the change amount ΔS1 indicates the absolute value of the slope of the straight line 1 passing through the points P0 and P1. That is, when the amount of change ΔS1 is large and the slope of the straight line l is steep, the error width E2 (range of values that the error ksp can take) takes a smaller value. Further, when the change amount ΔS1 is small and the slope of the straight line l is gentle, the error width E2 takes a larger value. Equation 6 shows the error Nksp when the adjacent correlation value S (kp−1) is equal to or smaller than the adjacent correlation value S (kp + 1). In this case, the amount of change ΔS2 between the reference correlation value S (kp) and the adjacent correlation value S (kp + 1) is used as the denominator of the error Nksp. This change amount ΔS2 indicates the slope m ′ of the straight line 1 ′ passing through the points P1 and P2. That is, when the change amount ΔS2 is large and the slope m ′ of the straight line l ′ is steep, the error width E2 takes a smaller value. Further, when the change amount ΔS2 is small and the slope of the straight line l ′ is gentle, the error width E2 takes a larger value.

The error Nksp shown in Equation 5 is derived from the decimal part ksp of the deviation amount of Equation 3 as shown in Equations 7-10. That is, in Equation 3, the correlation values S (kp-1), S (kp), and S (kp + 1) include noise components n (kp-1), n (kp), and n (kp + 1), respectively. If so, the decimal part ksp ′ of Equation 7 is obtained. Further, assuming that the noise components n (kp−1), n (kp), and n (kp + 1) have a Gaussian distribution with a standard deviation σ, the decimal part ksp ′ is as shown in Equation 8. Here, N (0, σ) and N ′ (0, σ) are variables having a predetermined distribution with an average value of 0 and a standard deviation of σ.

Assuming Equation 9 in Equation 8, the fractional part ksp ′ of Equation 10 is obtained, and Equation 5 is derived from the second term of the second equation on the right side of Equation 10. Similarly, Expression 6 is derived from Expression 4 through ksp ′ of Expression 11.

In the reliability evaluation unit 4 shown in FIG. 1, the reliability of the correlation amount kr of the correlation destination region is further evaluated based on the error width E2. That is, for example, the reliability value R representing the degree of reliability is expressed by the following equations 12 and 13.

  The reliability value R takes a smaller value as the reliability of the shift amount kr (= kp + ksp) of the correlation destination region is higher. When the error width E2 in the fractional part ksp is relatively large, that is, in the above Equations 5 and 6 indicating the error Nksp, when the numerator is a constant value, the smaller the denominator, the larger the confidence value R becomes. As a result, the reliability is evaluated low. The confidence value R is output to the storage unit 5 in association with the deviation amount kr of the correlation destination region. Various processes are performed based on the data kr and R on the storage unit 5. Here, the denominator of the confidence value R is set to a larger value of the correlation value changes ΔS1 and ΔS2. That is, the amount of change between the larger value of the adjacent correlation values S (kp−1) and S (kp + 1) and the reference correlation value S (kp) is used as the denominator of the confidence value R. Instead, the average value of the change amounts ΔS1 and ΔS2 can be used as the denominator of the reliability value R. Further, a smaller value of the correlation value changes ΔS1 and ΔS2 can be used as the denominator of the reliability value R. That is, the change amount between the smaller value of the adjacent correlation values S (kp−1) and S (kp + 1) and the reference correlation value S (kp) may be used as the denominator of the confidence value R. Is possible. Further, the reliability value R may be a value obtained by dividing the error width E1 of the correlation value (estimated value σ of noise variation) by these values relating to the changes ΔS1 and ΔS2 and a predetermined constant. That is, the reliability value R can be a constant multiple of the error width E2 and can be the same as the error width E2.

  Thus, in this embodiment, the reliability regarding the position of a correlation destination area | region is evaluated according to the variation | change_quantity between a reference | standard correlation value and an adjacent correlation value. By using the evaluation results regarding the reliability and performing various processes with the position of the correlation destination region as an input, the accuracy of those processes can be improved. In particular, by considering the influence of noise and setting the relationship between the amount of change in the correlation value and the reliability, various processes with excellent noise resistance can be performed.

(Second Embodiment)
A more specific apparatus will be described in detail following the correlation evaluation system described above. The following embodiment is an example applied to a stereo monitoring device. FIG. 3 is a block diagram of the stereo monitoring device. Parts having the same functions as those of the above-described correlation evaluation system are denoted by the same reference numerals. The monitoring apparatus is mounted on a vehicle, for example, and processes a pair of frame images (frame regions) to monitor a road ahead of the host vehicle and a monitoring object (such as a preceding vehicle or a pedestrian). In particular, the monitoring device includes a sensor 6, an amplifier 7, a data correction unit 8, a temperature sensor 9, and a representative value calculation unit 10. The reliability evaluation unit 4 sets the error width E1 based on data regarding noise in the amplifier 7, the data correction unit 8, the temperature sensor 9, and the representative value calculation unit 10. That is, here, the noise factors affecting the error width E1 are mainly the gain G in the amplifier 7, the correction degree γ in the data correction unit 8, the ambient temperature T, and the luminance D to be subjected to correlation evaluation. Assume that it is itself. Depending on the state of these noise factors, the reliability evaluation unit 4 variably sets the error width E1.

  The sensor 6 is composed of a pair of cameras incorporating an image sensor such as a CCD or CMOS sensor. The sensor 6 outputs a pair of frame images (a reference image and a comparison image), and each frame image is constituted by a data group having a predetermined size. The amplifier 7 adjusts the frame image output from the sensor 6 with the gain G. This gain G (current set value) is output to the reliability evaluation unit 4. The data correction unit 8 performs predetermined correction on the uncorrected image data and then outputs the corrected image data. For example, data correction is performed by referring to a table in which the relationship between input data and output data is described. This data correction may be set to an appropriate characteristic suitable for processing in the subsequent stage, regardless of whether the characteristic is non-linear or continuous. The degree of correction γ (current set value) of the corrected image data with respect to the uncorrected image data is output to the reliability evaluation unit 4. The reference image data after being processed by each part of the preceding stage of the matching processing unit 2 is stored in the storage unit 1a as digital data of a predetermined luminance gradation. Similarly, the comparison image data is stored in the storage unit 1b.

  The matching processing unit 2 sets a plurality of correlation destination candidates in the correlation source region that is a part of the reference image within the search range on the comparison image. The matching process described in the first embodiment is performed on the correlation between the luminance characteristics of the correlation source area and the search range. By this matching process, the horizontal shift amount kp of the integer level of the correlation destination region is calculated. The shift amount kp corresponds to the parallax dp in the stereo image, and this integer level parallax dp is output to the interpolation processing unit 3 together with the correlation value S (dp) corresponding to the correlation destination region. Further, the correlation value S (dp−1) of the left adjacent correlation destination candidate and the correlation value S (dp + 1) of the right adjacent correlation destination candidate are output to the interpolation processing unit 3.

  The interpolation processing unit 3 performs a coordinate value of a decimal level for the parallax by interpolation using the correlation value S (dp) of the correlation destination region and the correlation values S (dp−1) and S (dp + 1) of the adjacent correlation destination candidates. dsp is calculated. That is, the integer level parallax dp is interpolated by this decimal level coordinate value dsp. The interpolated parallax d (decimal level coordinate value dr), the correlation value S (dp) of the correlation destination region, and the correlation values S (dp−1) and S (dp + 1) of the adjacent correlation destination candidates are reliable. It is output to the sex evaluation unit 4.

  The temperature sensor 9 detects the ambient temperature T of the stereo monitoring device and outputs the current detection value to the reliability evaluation unit 4. For example, the temperature T around the sensor 6 is output to the reliability evaluation unit 4.

  Based on the luminance data D1 to Dn included in the correlation source region, the representative value calculation unit 10 calculates a representative value Drep representing the data. For example, the representative value Drep can be an average value of luminance values of all or a specific part of pixels in the correlation source region, and can be an intermediate value of these luminance values. The representative value calculation unit 10 can calculate a representative value based on luminance data included in the search range instead of the correlation source region.

  The reliability evaluation unit 4 estimates the error width E2 based on the amount of change between the reference correlation value S (dp) and the adjacent correlation value S (dp-1) or S (dp + 1) and the error width E1. Then, the reliability of the parallax d is evaluated based on the error width E2. The error width E1 here is set based on the state of the noise factor riding on the luminance. That is, the reliability evaluation unit 4 detects the gain G, the degree of correction γ, the ambient temperature T, and the representative luminance value Drep, and estimates the noise corresponding to these to set the error width E1 variably. To do. For example, the error width E1 can be set by setting a basic value and a correction value for the error width E1. That is, the correspondence relationship between the combination of the gain G, the correction degree γ, and the temperature T and the basic value of the error width E1 is described in advance in a predetermined basic value table. Further, the correspondence between the representative value Drep and the correction value of the error width E1 is described in advance in a predetermined correction value table. The reliability evaluation unit 4 sets a basic value by referring to the basic value table based on the current value of the gain G, the degree of correction γ, and the ambient temperature T, and based on the representative value Drep of the luminance, The correction value is set with reference to the correction value table. Then, the reliability evaluation unit 4 corrects the basic value with the correction value, thereby setting the error width E1. Each correspondence relationship in the basic value table and the correction value table is appropriately set through experiments or simulations performed in advance. In this experiment or the like, the noise component of the correlation value based on the luminance is estimated while considering the relationship between the noise variation and the state value of the noise factor. Of course, it is also possible to use a predetermined calculation formula instead of all or part of the basic value table and the correction value table and calculate the basic value by this calculation formula. Moreover, it is not always necessary to consider all these four types of noise factors, and it is possible to include other factors. The reliability evaluation unit 4 stores the parallax d and the reliability value R representing the reliability evaluation result in the storage unit 5.

  Using this parallax d and the reliability value R, the object recognition process can be performed more effectively. That is, in the object recognition using the well-known parallax d, the parallax d is the same object, for example, a preceding vehicle, depending on whether or not the parallax d of a plurality of adjacent pixel regions is within a predetermined range. It is determined whether or not a pedestrian or the like is indicated. Based on the determination result, the parallax d is grouped. In the conventional grouping, the parallax d having a value near the determination threshold value that defines the range is determined rigidly. That is, the parallax d determined to belong to the range is grouped, and the parallax d determined to not belong to the range is excluded from the grouping. On the other hand, the determination can be performed more flexibly by considering the reliability of the parallax d as in the present embodiment. In other words, it is possible to perform grouping in consideration of errors in the parallax d. In particular, it is possible to set a more appropriate reliability value R for the parallax d by variably setting the error width of the correlation value S according to the state of the noise factor.

  Also, the parallax d can be weighted using the parallax d and the confidence value R. That is, it is possible to determine stepwise whether the parallax d is valid or invalid based on the reliability value R. As described above, the state of the noise factor can be appropriately reflected in the reliability value R.

  Thus, in this embodiment, the reliability of parallax is evaluated according to the amount of change between the reference correlation value and the adjacent correlation value. By using this reliability evaluation result for various processes using parallax as input, the accuracy of those processes can be improved. In particular, various processes with excellent noise resistance can be performed by setting the amount of change in the correlation value in consideration of the influence of noise.

  In the present embodiment, the application example to a pair of frame images (stereo images) captured at the same time has been described. However, the present invention is not limited to this, and a pair of frame regions having relevance to each other. Widely applicable. As another example of such a frame region, there is a pair of frame images captured at different times, which are targets of so-called optical flow processing. In this case, a monocular camera is used instead of the sensor 6 described above, and frame images are output in time series. Further, such a frame region is not limited to an image (luminance distribution) defined by luminance, but information other than luminance (for example, heat distribution obtained from a far-infrared camera, laser radar or millimeter wave radar) It may be defined by the obtained reflection intensity distribution or the like.

  The present invention can also be applied to an apparatus that performs pattern matching. Here, the correlation source region corresponds to a predetermined template, and the correlation destination region corresponds to a pixel region in a search region that constitutes part or all of the frame image. In particular, an image of a road marking relating to a lane is used as a template, and a correlation with a pixel region on a frame image is evaluated while moving and enlarging the template, thereby specifying a corresponding pixel region. With respect to the position of the specified corresponding pixel region, it is possible to estimate the error width and evaluate the reliability.

  In the present embodiment, the correlation source region is set in the reference image, and the correlation destination region is specified in the comparison image. Instead, in one image, the correlation source region and the correlation source region are specified. The self-matching may be performed by setting both the correlation destination candidates adjacent to the correlation source region. That is, for example, in the reference image, the amount of change in the correlation value between a certain area and the areas adjacent to each other is calculated. Based on the magnitude of this change amount (depth of the valley shape centered around the correlation value 0), the error width of the parallax obtained with the comparison image is predicted, and further, the parallax reliability is set. It is possible.

  Further, in the above-described embodiment, the position of the correlation destination region is specified by the coordinate value of the integer level and the coordinate value of the decimal level. When this is generalized, the former is the discrete first coordinate value. The latter can be expressed as a second coordinate value having a finer resolution than the first coordinate value. The integer level coordinate values are representative of discrete values, but other than these, discrete values in units of subpixels may be used. For example, the coordinate value is calculated as a discrete value for each 0.5 pixel by setting a correlation destination candidate by pixel interpolation while offset by 0.5 pixel within the search range. Further, the interval (offset value) between adjacent discrete values is not necessarily constant.

The block block diagram of the correlation evaluation system of 1st Embodiment A diagram showing the relationship between the error width of the correlation value and the error width of the deviation amount of the correlation destination area Block configuration diagram of the stereo monitoring device of the second embodiment

Explanation of symbols

2 Matching Processing Unit 3 Interpolation Processing Unit 4 Reliability Evaluation Unit 6 Sensor 7 Amplifier 8 Data Correction Unit 9 Temperature Sensor 10 Representative Value Calculation Unit

Claims (6)

In the correlation evaluation system that evaluates the positional reliability of the correlation destination area specified as the correlation destination of the correlation source area,
A plurality of correlation destination candidates as correlation destination candidates in the correlation source region are set within a predetermined search range, and a correlation value indicating a degree of correlation with the correlation source region is set for each of the plurality of correlation destination candidates. By calculating and comparing correlation values calculated for each of the correlation destination candidates, one of the plurality of correlation destination candidates is specified as the correlation destination region, and the position of the correlation destination region is discrete A matching processing unit that calculates the first coordinate value;
The position of the correlation destination region is determined from the first coordinate value by interpolation using the correlation value of the correlation destination region and the correlation values of the two or more correlation destination candidates that are positionally adjacent to the correlation destination region. An interpolation processing unit that calculates a second coordinate value having a fine resolution;
The first error width indicating the error range of the correlation value according to the amount of change between the correlation value of the correlation destination region and the correlation value of the correlation destination candidate that is positioned adjacent to the correlation destination region Is projected on the coordinate axis indicating the position of the correlation destination area, and specified by the first coordinate value and the second coordinate value based on the second error width generated on the coordinate axis by the projection. A reliability evaluation unit that evaluates the reliability related to the position of the correlation destination region ,
A data correction unit that is provided in a preceding stage of the matching processing unit, performs a predetermined correction on the data before correction, and outputs data after correction;
The correlation evaluation system, wherein the reliability evaluation unit variably sets the first error width according to a degree of correction of the data after correction with respect to the data before correction .   The reliability evaluation unit evaluates the reliability lower as the amount of change between the correlation value of the correlation destination region and the correlation value of the correlation destination candidate positioned adjacent to the correlation destination region becomes smaller. The correlation evaluation system according to claim 1, wherein: A representative value calculating unit that calculates a representative value representing the data based on the data included in the correlation source region or the search range;
The correlation evaluation system according to claim 1 or 2, wherein the reliability evaluation unit variably sets the first error width according to the representative value. A sensor configured by a data group of a predetermined size and outputting a frame region including a region used as the correlation source region or the search range;
An amplifier that is provided in a preceding stage of the matching processing unit, and that adjusts a frame region output from the sensor by a gain;
The correlation evaluation system according to claim 1, wherein the reliability evaluation unit variably sets the first error width according to the gain. A temperature sensor for detecting the ambient temperature;
The correlation evaluation system according to claim 1, wherein the reliability evaluation unit variably sets the first error width based on the temperature. In the correlation evaluation method for evaluating the positional reliability of the correlation destination area specified as the correlation destination of the correlation source area,
A step of outputting the corrected data after performing a predetermined correction on the data before correction;
A plurality of correlation destination candidates as correlation destination candidates in the correlation source region are set within a predetermined search range, and a correlation value indicating a degree of correlation with the correlation source region is set for each of the plurality of correlation destination candidates. A calculating step;
Identifying any of the plurality of correlation destination candidates as the correlation destination region by comparing correlation values calculated for each of the correlation destination candidates; and
Calculating the position of the correlation destination region as discrete first coordinate values;
The position of the correlation destination region is determined from the first coordinate value by interpolation using the correlation value of the correlation destination region and the correlation values of the two or more correlation destination candidates that are positionally adjacent to the correlation destination region. Calculating a second coordinate value having a fine resolution,
The first error width indicating the error range of the correlation value according to the amount of change between the correlation value of the correlation destination region and the correlation value of the correlation destination candidate that is positioned adjacent to the correlation destination region Is projected on the coordinate axis indicating the position of the correlation destination area, and specified by the first coordinate value and the second coordinate value based on the second error width generated on the coordinate axis by the projection. and a step of the evaluating the reliability of the position of the correlated destination area being,
In the step of evaluating the reliability , the correlation evaluation method is characterized in that the first error width is variably set according to the degree of correction of the corrected data with respect to the uncorrected data .

Images