JP4694993B2 - Correlation evaluation system and correlation evaluation method - Google Patents

Description

  The present invention relates to a technique for evaluating the correlation of a correlation destination area specified as a correlation destination of a correlation source area, and more particularly to setting of a determination threshold value used in this correlation evaluation.

2. Description of the Related Art Conventionally, a technique for evaluating the correlation of a partial area between a pair of data areas having relevance using a determination threshold value is known. For example, Patent Document 1 discloses a method of evaluating the reliability of distance data by calculating the reliability of distance data calculated from a stereo image and comparing the reliability with a determination threshold value. ing. The determination threshold value is variably set according to the situation. For example, when the camera gain is increased, the determination threshold value is increased in view of the fact that the noise included in the screen increases. The set determination threshold value is uniformly applied to the entire frame image constituting one frame.
JP 2003-269917 A

  However, in the conventional technique, since one determination threshold value is uniformly applied to the entire frame image (data region), when evaluating the correlation for each partial region constituting a part of the frame image, It was difficult to sufficiently eliminate the influence of noise as a disturbance.

  The present invention has been made in view of such circumstances, and an object thereof is to improve the evaluation accuracy in the evaluation of the correlation between the correlation source region and the correlation destination region.

  Another object of the present invention is to enable evaluation of correlation excellent in noise resistance.

In order to solve this problem, the first invention provides a correlation evaluation system that evaluates the correlation of a correlation destination area specified as a correlation destination of a correlation source area. The evaluation system includes an imaging unit, a matching processing unit, a representative value calculation unit, a threshold setting unit, and a determination unit. The imaging unit captures a frame image partially including the correlation source region. The matching processing unit sets a plurality of correlation destination candidates as correlation destination candidates in the correlation source region for each correlation source region, and calculates a correlation value indicating the degree of correlation with the correlation source region for each correlation destination candidate. In addition, the matching processing unit identifies one of a plurality of correlation destination candidates as a correlation destination region by comparing the correlation values calculated for each correlation destination candidate. Based on the data in the sub-region including the correlation source region, the representative value calculation unit calculates, for each sub-region, the average value or intermediate value of the luminance values of all the pixels in the sub-region as a representative value. The threshold setting unit includes a base value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and for each sub-region. The sum of the correction value set based on the representative value is individually set for each correlation source region as a determination threshold value applied to the correlation source region. The determination unit compares the correlation value of the correlation destination region specified by the matching processing unit with the determination threshold value applied to the correlation source region that is the correlation source of the correlation destination region, thereby calculating the correlation value. Evaluate reliability.

In the first invention, it is desirable that the threshold value setting unit sets the determination threshold value of the sub region to be larger as the luminance state in the sub region indicated by the representative value becomes higher.

The second invention provides a correlation evaluation system that evaluates the correlation of the correlation destination area specified as the correlation destination of the correlation source area. The evaluation system includes an imaging unit, a matching processing unit, a representative value calculation unit, a threshold setting unit, and a determination unit. The imaging unit captures a frame image partially including the correlation source region. The matching processing unit sets, for each correlation source region, a search range including a plurality of correlation destination candidates that are correlation destination candidates in the correlation source region, and sets a correlation value indicating the degree of correlation with the correlation source region for each correlation destination candidate. To calculate. In addition, the matching processing unit identifies one of a plurality of correlation destination candidates as a correlation destination region by comparing the correlation values calculated for each correlation destination candidate. Based on the data included in the search range, the representative value calculation unit calculates, for each search range, an average value or an intermediate value of luminance values of all the pixels in the search range as a representative value. The threshold setting unit includes a basic value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and for each search range . The sum of the correction value set based on the representative value is individually set for each search range as a determination threshold applied to the search range. The determination unit evaluates the reliability of the correlation value by comparing the correlation value of the correlation destination region specified by the matching processing unit with a determination threshold value applied to the search range including the correlation destination region. To do.

In the second invention, it is desirable that the threshold setting unit sets the determination threshold of the search range to be larger as the luminance state in the search range indicated by the representative value becomes higher.

  In the first invention or the second invention, a luminance correction unit that performs predetermined luminance correction on the frame image may be further provided before the matching processing unit. In this case, it is preferable that the threshold setting unit variably sets the determination threshold based on the degree of luminance correction.

The third invention provides a correlation evaluation method for evaluating the correlation of the correlation destination area specified as the correlation destination of the correlation source area. In this evaluation method, a step of capturing a frame image partially including a correlation source region and a plurality of correlation destination candidates that are candidates for a correlation destination of the correlation source region are set for each correlation source region. A step of calculating a correlation value indicating a degree of correlation for each correlation destination candidate and identifying one of a plurality of correlation destination candidates as a correlation destination region by comparing the correlation values calculated for each correlation destination candidate And calculating, based on the data in the sub-region including the correlation source region, the average value or intermediate value of the luminance values of all the pixels in the sub-region as a representative value for each sub-region, and the brightness of the frame image A basic value set for each frame image based on at least one of an amplification factor for adjusting the depth and an ambient temperature of the correlation evaluation system, and a correction value set based on a representative value for each sub-region Sum of And setting individually for each correlation reference region as a determination threshold to be applied to the correlation reference region, and the correlation value of the correlation destination area, determined to be applied to the correlation reference region to be a correlation source of the correlated destination region And comparing the correlation value with a threshold value to evaluate the reliability of the correlation value.

4th invention provides the correlation evaluation method which evaluates the correlation of the correlation destination area | region specified as a correlation destination of a correlation origin area | region. In this evaluation method, a step of capturing a frame image partially including a correlation source region and a search range including a plurality of correlation destination candidates that are candidates for a correlation destination in the correlation source region are set for each correlation source region. A correlation value indicating the degree of correlation with the original region is calculated for each correlation destination candidate, and one of a plurality of correlation destination candidates is compared by comparing the correlation values calculated for each correlation destination candidate. A step of calculating as a representative value the average value or intermediate value of the luminance values of all the pixels in the search range based on data included in the search range, and amplification factor for adjusting the brightness, the basic value is set for each frame image based on at least one of the ambient temperature of the correlation evaluation system, the correction value set based on the representative value for each search range Individually for each search range as a determination threshold applied to the search range, a correlation value of the correlation destination region, and a determination threshold applied to the search range including the correlation destination region To evaluate the reliability of the correlation value.

  In the present invention, the determination threshold value used for evaluating the correlation between the correlation source region and the correlation destination region is not the entire data region including the correlation source region or the correlation destination region, but a portion constituting a part thereof Individual sub-area units are set individually. Thus, by setting the determination threshold value finely for each sub-region, it is possible to improve the correlation evaluation accuracy. In particular, if the threshold value for each sub-region is set so as to eliminate this after considering the tendency of noise generated in that area, it is possible to evaluate the correlation with excellent noise resistance. Become.

(First embodiment)
FIG. 1 is a block configuration diagram of a correlation evaluation system according to the present embodiment. This evaluation system 1 evaluates the correlation between the correlation source region and the correlation destination region. Data defining the first data area is stored in the storage unit 2a, and data defining the second data area is stored in the storage unit 2b. The correlation source area constitutes a part of the first data area, and the correlation destination area (and the correlation destination candidate) constitutes a part of the second data area. The predetermined input unit outputs the pair of data groups to the storage units 2a and 2b. As an example of the pair of data groups, a pair of frame image data (stereo image data) captured at the same time can be cited. In this case, a stereo camera is used for the input unit.

  The matching processing unit 3 sets a plurality of correlation destination candidates as correlation destination candidates in the correlation source area within a predetermined search range on the second data area, and calculates a correlation value CB for each correlation destination candidate. . The correlation value CB indicates the degree of correlation between the correlation source region and the correlation destination region. Further, the matching processing unit 3 compares the correlation values CB for each correlation destination candidate, and thereby specifies any one of a plurality of correlation destination candidates as a correlation destination area. As an example, a sum of absolute differences can be used for the correlation value CB. The correlation value CB using this absolute difference sum is a sum obtained by calculating a difference (absolute value) of data between elements corresponding to each other in a pair of data areas to be evaluated, and adding this value for all the elements. Expressed as a value. Also, as the correlation value CB, other known evaluation functions such as a sum of squared differences can be used. The correlation value CB using the sum of squared differences is expressed as a sum value obtained by calculating the square of the difference between the data of the elements corresponding to each other in the pair of data areas to be evaluated, and adding this value for all the elements. Is done. When these correlation evaluation methods are used, the correlation value CB decreases as the correlation between a pair of data regions increases, in other words, the similarity between the data characteristics of the two data regions. Specifically, the correlation value CB becomes 0. However, in practice, various noises act as disturbances, and therefore the correlation value CB rarely becomes 0 even in a data area that should be completely identical. The matching processing unit 3 outputs the correlation value CB to the determination unit 4.

  The determination unit 4 evaluates the reliability of the correlation value CB by comparing the correlation value CB of the correlation destination region with the determination threshold value CBth. In particular, this determination threshold value CBth can be set for the sub-region including the correlation source region. As an example, this sub-region can be a region obtained by dividing the first data region. A determination threshold value CBth is sequentially set for a plurality of sub-regions to be processed. For example, this sub-region may be a region having the same size as the correlation source region, or may be a region including all or a specific part of the correlation source region. Further, the determination threshold value CBth may be set for a search area or the like instead of the sub-area. Furthermore, the determination unit 4 may determine the validity of the data calculated based on the correlation value CB according to the reliability evaluation result of the correlation value CB.

  Determination threshold setting unit 5 sets determination threshold CBth for determining reliability (correlation) of correlation value CB in determination unit 4. The determination threshold value setting unit 5 performs this setting individually for each sub-region based on the representative value Drep of the data D in the sub-region. Specifically, this setting is based on the basic value CBbase set for each first data area by the basic value setting unit 6 (described later) and the correction set for each sub-area by the correction value setting unit 7 (described later). Based on the value CBamd. That is, the determination threshold value setting unit 5 corrects the basic value CBbase with the correction value CBamd, and sets the determination threshold value CBth obtained thereby for each sub-region.

  The base value CBbase and the correction value CBamd can be set in consideration of the following noise factors, for example. That is, here, it is assumed that the factors that determine the magnitude of noise riding on the data D are the data D itself, the amplification factor G, the degree of data correction γ, and the ambient temperature T. The amplification factor G is the ratio of the output signal to the input signal representing data. The amplification factor G is appropriately adjusted in a predetermined amplifier provided in the previous stage of the storage units 2a and 2b. In addition, a predetermined data correction unit is provided in the preceding stage of the storage units 2a and 2b, and the degree of data correction γ is a value corresponding to the rate of change of output data with respect to input data in the correction unit. For example, data correction is performed by referring to a table in which the relationship between input data and output data is described. This data correction may be set to an appropriate characteristic suitable for processing in the subsequent stage, regardless of whether the characteristic is non-linear or continuous. Furthermore, the ambient temperature T is the ambient temperature of the correlation evaluation system 1 detected by a predetermined temperature sensor.

  2-5 is a figure which shows the tendency of the correspondence of each value showing the state of each noise factor mentioned above and the dispersion | variation in noise (standard deviation in noise distribution) as an example. In general, noise is superimposed on data stored in the storage units 2a and 2b. While the characteristics of the individual devices that process the data differ slightly, there is no significant difference in the overall trend for noise variation. For example, when the correlation evaluation system 1 is applied to a stereo monitoring apparatus that processes a stereo image, noise is added to all the images captured by the stereo camera. Although the characteristics of individual cameras are different, the tendency for noise variation is not different.

  FIG. 2 is a diagram illustrating a tendency of the correspondence relationship between data D and noise variation as an example. As shown here, as the value of the data D increases, the variation of noise on the data D tends to increase. For example, in the case of application to a stereo type monitoring device, the data D corresponds to the luminance, but if the amount of light entering each cell of the imaging device of the stereo camera increases, the noise increases accordingly. by. FIG. 3 is a diagram illustrating a tendency of the correspondence relationship between the amplification factor G and noise variation as an example. As the amplification factor G increases, noise variation tends to increase. This is because if the data signal is amplified more greatly, the noise increases accordingly. FIG. 4 is a diagram illustrating a correspondence relationship between luminance values after data correction and noise variation as an example. In the data correction unit, arbitrary correction can be performed as described above. However, when a portion with a small value of the data D is corrected with a larger increase rate (change rate) than a portion with a large value, the data D Amplification of noise in a small part of becomes remarkable. This is because noise increases in accordance with the increase rate. FIG. 5 is a diagram showing a tendency of the correspondence relationship between the ambient temperature T and noise variation as an example. As the ambient temperature T increases, the noise variation tends to increase. This is because, for example, in the case of application to a stereo monitoring device, dark current (noise) generated in the image sensor increases as the temperature rises.

  Referring to FIG. 1 again, the basic value setting unit 6 sets a basic value CBbase for each first data area. Regarding this setting, the current set values of the amplification factor G and the degree of data correction γ are output to the basic value setting unit 6 as needed. Further, the current detected value of the ambient temperature T is output to the basic value setting unit 6 as needed. The base value setting unit 6 refers to a predetermined base value table based on the gain G, the data correction degree γ, and the ambient temperature T, and sets the base value CBbase to be applied to the entire first data area. That is, in this basic value table, a correspondence relationship between the combination of each value of the amplification factor G, the data correction degree γ, and the ambient temperature T, and the basic value CBbase of the determination threshold is described. Based on this, the base value CBbase is set. The correspondence is appropriately set through experiments and simulations conducted in advance. In this experiment or the like, the noise component of the data D is estimated in consideration of the relationship between the noise variation and the noise factor state shown in FIGS. It should be noted that a predetermined calculation formula may be used in place of all or a part of the base value table, and the base value CBbase may be calculated by this calculation formula.

  The correction value setting unit 7 sets the correction value CBamd of the sub area based on the representative value Drep of the data D for each sub area. That is, n data is included in the sub area, and the representative value calculation unit 8 reads the values D1 to Dn of the data from the storage unit 2a and represents the data D1 to Dn in the sub area. A representative value Drep is calculated. The correction value setting unit 7 sets a correction value CBamd for the sub-region by referring to a predetermined correction value table based on the representative value Drep. This correction value table describes the correspondence between the luminance representative value Drep and the determination threshold correction value CBamd. Similar to the above-described basic value table, these correspondences are appropriately set through experiments, simulations, and the like performed in advance while considering the relationship of FIG. Similarly to the above, a predetermined calculation formula may be used instead of all or a part of the correction value table, and the correction value CBamd may be calculated using this calculation formula.

  Thus, here, as noise factors affecting the evaluation of correlation, (1) the magnitude of the value of data D, (2) gain G, (3) degree of data correction γ, (4) ambient temperature T is taken into account. As a result, it is possible to meticulously cope with any noise, and it is possible to evaluate the optimum correlation independent of the noise factor. Of these noise factors (1) to (4), it is preferable to consider the noise factor (1), but other noise factors (2) to (4) may not be considered. . Moreover, even when considering the noise factors (2) to (4), it is not necessary to include all of them, and it is sufficient that at least one of them is included, and other factors can be included. That is, the determination threshold value CBth is variably set for a value representing the state of one noise factor, and the determination threshold value is variably set for each combination of values representing the states of a plurality of noise factors. CBth may be set.

(Second Embodiment)
Following the conceptual description of the correlation evaluation system described above, more specific processing will be described in detail below. This embodiment and the following third to fifth embodiments are application examples of the above-described correlation evaluation system to a stereo monitoring apparatus. In each of the following embodiments, the positional relationship between the region to which the determination threshold value CBth is applied and the correlation source region are different.

  FIG. 6 is a block diagram of the stereo monitoring device 10. Parts having the same functions as those of the above-described correlation evaluation system are denoted by the same reference numerals. The monitoring device 10 is mounted on, for example, a vehicle, and based on sensor information from the stereo camera 11 directed to the front of the host vehicle, a road or a monitoring target (a preceding vehicle, a pedestrian, or the like) ahead of the host vehicle Monitor. The camera 11 includes a main camera 11a and a sub camera 11b. Each camera 11a and 11b includes an image sensor such as a CCD or a CMOS sensor. The camera 11 captures a pair of frame images (stereo images). One reference image, which is a frame image, corresponds to the first data area described above and is stored in the storage unit 2a. Further, the comparison image which is the other frame image corresponds to the second data area and is stored in the storage unit 2b. Each frame image is processed by the gain control amplifiers (GCAs) 12a and 12b, the A / D converter (not shown), the luminance correction unit 13, and the like before being output to the storage units 2a and 2b. ing. By these processes, the analog frame image signal is converted into digital data of a predetermined luminance gradation, and necessary correction is performed.

  The amplification factor G in the amplifier of the correlation evaluation system described above corresponds to the gain G for adjusting the brightness of the frame image in the GCA 11a. The data correction degree γ in the data correction unit corresponds to the luminance correction degree γ applied to the frame image in the luminance correction unit 13. The ambient temperature T is also detected by the temperature sensor 14. Each value of the gain G, the luminance correction degree γ, and the ambient temperature T is output to the basic value setting unit 6 as needed. The base value setting unit 6 variably sets the base value CBbase based on these values. The data D on the data area corresponds to the luminance value, and the representative value calculation unit 8 calculates the representative value Drep based on the luminance values D1 to Dn in the sub area. The correction value setting unit 7 sets the correction value CBamd based on the representative value Drep. Determination threshold setting unit 5 sets determination threshold CBth based on base value CBbase and correction value CBamd. The matching processing unit 3 calculates the correlation value CB and the parallax d of the correlation destination region by a known stereo matching process. The correlation value CB and the parallax d corresponding to the correlation value CB are associated with each other and output to the determination unit 4. The determination unit 4 evaluates the reliability of the correlation value CB based on the determination threshold value CBth, and outputs the parallax d and the evaluation result corresponding to the parallax d to a predetermined object recognition unit. The object recognition unit can recognize the monitoring object or the like based on the parallax d and the corresponding evaluation result.

  FIG. 7 is a flowchart illustrating a determination threshold setting procedure. By the procedure shown here, a pair of frame images corresponding to one frame are sequentially processed. In steps 1 and 2, the basic value CBbase of the determination threshold value in the determination unit 4 is set for the reference image. First, in step 1, the base value setting unit 6 detects the gain G, the brightness correction degree γ, and the ambient temperature T in order to set the base value CBbase. In step 2, the base value setting unit 6 refers to the base value table and sets the base value CBbase based on each value detected in step 1. Here, it is assumed that the base value CBbase is set for each frame of the reference image, but it is also possible to set the same base value CBbase to a plurality of continuous reference images.

  In steps 3 and 4, the correction value CBamd of the determination threshold is set for the sub-region of the reference image. FIG. 8 is an explanatory diagram of a sub area (application area of the determination threshold value CBth). The image plane defined by the frame image data is expressed in the ij coordinate system, with the lower left corner of the image as the origin, the horizontal direction as the i axis, and the vertical direction as the j axis. Here, on the reference image 21a, the sub-region is set as a region having the same size (for example, 4 × 4 pixels) and the same position as the correlation source region 31a. That is, the application area of the determination threshold value CBth matches the correlation source area 31a. In step 3, the representative value calculation unit 8 calculates a representative value Drep of luminance in the correlation source region 31a. For example, the representative value Drep may be an average value of luminance values of all or a specific part of pixels in the sub-region, or may be an intermediate value of these luminance values. In step 4, the correction value setting unit 7 refers to the correction value table and sets the correction value CBamd based on the representative value Drep calculated in step 3.

  In step 5, the determination threshold value setting unit 5 sets the determination threshold value CBth based on the basic value CBbase set in step 2 and the correction value CBamd set in step 4. For example, the determination threshold value CBth can be the sum of the basic value CBbase and the correction value CBamd. Instead, the determination threshold value CBth may be calculated by setting a predetermined calculation formula and substituting the basic value CBbase and the correction value CBamd into the calculation formula. In particular, in the correction value setting unit 7 (and the determination threshold setting unit 5), based on the relationship between noise factors and noise variations (FIGS. 2 to 5), the higher the luminance state in the sub-region, It is preferable to set the correction value CBamd (determination threshold value CBth) large. Thereby, the noise resistance in the evaluation of correlation can be effectively improved.

  In step 6, the matching processing unit 3 performs a matching process on the correlation source region, and calculates a correlation value CB and a parallax d of the correlation destination region. In subsequent step 7, the determination unit 4 evaluates the correlation by comparing the correlation value CB calculated in step 6 with the determination threshold value CBth set for each correlation source region in step 5.

  FIG. 9 is an explanatory diagram for evaluating the correlation in the stereo matching process. In FIG. 8, the matching processing unit 3 sets correlation destination region candidates within a predetermined search range 31b on the comparison image 21b. Here, a search range 31b having a predetermined length from the position (i1−α, j1) on the comparison image 21b with respect to the correlation source region 31a located at the position (i1, j1) on the reference image 21a. Is set. The correlation source region 31a and the search range 31b are on the same horizontal line (epipolar line). This α is a predetermined search margin toward the left. In the search range 31b, the correlation value CB of the luminance is calculated while updating the correlation destination region candidates by shifting one pixel at a time from the left end to the right end in the search range 31b. In FIG. 9, the correlation value CBn of the correlation destination candidate PBn is the correlation value CBn-1 of the immediately adjacent correlation destination candidate PBn-1, and the correlation value CBn + 1 of the immediately adjacent correlation destination candidate PBn + 1. The correlation value CB is the minimum value. This correlation destination candidate PBn is specified as a correlation destination region. Further, the matching processing unit 3 calculates a parallax d that is a deviation amount between the position of the correlation source region in the reference image and the position of the correlation destination region specified in this way in the comparison image. The parallax d and the correlation value CB corresponding to the parallax d are output to the determination unit 4.

  The determination unit 4 evaluates the correlation related to the correlation value CB (reliability of the correlation value CB). For example, as this correlation, it is possible to determine the reliability of the parallax d based on the correlation value CB. . That is, when the correlation value CB is smaller than the determination threshold value CBth, it is assumed that the correlation of the luminance between the correlation source region and the correlation destination region is sufficiently high, and the reliability for the parallax d is set high. . Further, when the correlation value CB is larger than the determination threshold value CBth, the reliability is set to be low, assuming that the luminance correlation is not sufficiently high. The determination threshold value CBth here is variably set according to the magnitude of the noise component of the estimated luminance (correlation value) in steps 1 to 5. For example, when the luminance of the correlation source region is large, noise variation is large (FIG. 2). When the variation in noise is large, it is estimated that noise having a magnitude far from the average is generated. Therefore, in order to prevent erroneous determination due to noise, the determination threshold value CBth is increased. Further, when the ambient temperature T is low, noise variation is small (FIG. 5). In this case, the magnitude of noise is estimated to be relatively small, and the determination threshold value CBth is lowered.

  The correlation is evaluated using the determination threshold value CBth having such a width. Then, it is determined whether the parallax d is valid or invalid based on the reliability information as the evaluation result. That is, the determination unit 4 can filter invalid data and output only effective parallax as distance data to a predetermined distance data memory. Instead, the reliability information may be combined with the parallax d corresponding to the correlation value CB to form distance data and stored in the distance data memory. Further, the reliability information may be stored in the distance data memory in association with the parallax d. Furthermore, although the reliability has two levels of high and low, it is naturally possible to have three or more levels. In this case, two or more determination threshold values are set for each correlation source region (sub-region) in the same manner as described above.

  If the determination in step 8 shown in FIG. 7 is affirmative, that is, if all the specified correlation source regions are to be processed, this processing ends. On the other hand, if the determination in step 8 is negative, that is, if all the correlation source regions are not processed, the correlation source region is updated in step 9 and the process returns to step 3.

  As described above, in the present embodiment, the determination threshold value used for evaluating the correlation between the correlation source region and the correlation destination region is not the entire data region (frame image in the present embodiment) but one of them. These are set individually in units of partial sub-regions (correlation source regions in the present embodiment) that constitute a part. Thus, by setting the determination threshold value finely for each correlation source region, it is possible to improve the evaluation accuracy of the correlation. In particular, if the threshold value for each sub-region is set so as to eliminate this after considering the tendency of noise generated in that area, it is possible to evaluate the correlation with excellent noise resistance. Become. In addition, here, the sub-region that is the determination threshold setting unit is set to the same size as the correlation source region that is the parallax calculation unit (in other words, the correlation value calculation unit). Thereby, since both are matched on a one-to-one basis, the determination threshold value applied to each correlation source region can be set optimally.

(Third embodiment)
FIG. 10 is a diagram for explaining setting of the determination threshold value CBth in the present embodiment. Stereo monitoring apparatuses according to the following embodiments have a configuration similar to the stereo monitoring apparatus 10 of FIG. Similarly, luminance D, gain G, luminance correction degree γ, ambient temperature T, and the like are assumed as main noise factors. In FIG. 10, as in the second embodiment, a search range 32b is set for the correlation source region 32a. In particular, here, a sub-region 33 larger than the correlation source region is set, and the determination threshold value CBth calculated for a certain sub-region 33 is applied to the correlation source region included in the sub-region. For example, the sub area 33 is a vertical strip-shaped area obtained by dividing the reference image 22a. Further, by dividing the sub-region 33, a correlation source region 32a constituting a part thereof is obtained. A plurality of correlation source regions 32a are set in each of the plurality of sub-regions 33. For example, for each correlation source region 32a belonging to the second sub-region 33 from the left, the second sub-region 33 from the left The set determination threshold value CBth is used.

  As described above, in this embodiment, the determination threshold value used for evaluating the correlation between the correlation source region and the correlation destination region is set individually for each sub-region, not for the entire frame image. Compared to setting a determination threshold value for the entire frame image as in the prior art, it is possible to improve the correlation evaluation accuracy. It is also possible to evaluate the correlation with excellent noise resistance.

  Note that the sub-region unit is not limited to that disclosed in FIG. 10, and may be a horizontal strip-like region, for example. Further, it can be set according to the tendency of the captured image. That is, when a stereo monitoring device is mounted on a vehicle, assuming that a road surface is at the bottom of the image, a preceding vehicle is at the center, the sky is at the top, and a three-dimensional object is projected on the side, It is also possible to have a configuration in which a determination threshold value is set for each set area.

(Fourth embodiment)
FIG. 11 is an explanatory diagram for setting the determination threshold value CBth in the present embodiment. Here, the determination threshold value CBth is set for an area constituting a part of the comparison image 23b. That is, as described with reference to FIGS. 8 and 9, in the matching process, a predetermined search range 34b is set for the correlation source region 34a in the reference image 23a. In the present embodiment, a representative value Drep of luminance in the search range 34b (representative value of data in the search range) is calculated, and the determination threshold CBth is individually determined for each search range 34b based on the representative value Drep. Is set.

Also in the present embodiment, as described above, it is possible to improve correlation evaluation accuracy, improve noise resistance, and the like. In particular, the determination threshold value CBth is set larger as the luminance state in the search area 34b becomes higher. Thereby, the noise resistance in the evaluation of correlation can be effectively improved.
(Fifth embodiment)
FIG. 12 is an explanatory diagram for setting the determination threshold value CBth in the present embodiment. In the second to fourth embodiments, the matching processing unit 3 performs a stereo matching process, and the determination unit 4 determines the reliability of the output parallax d. That is, it is assumed that stereo matching processing for distance measurement is performed. Instead, the above-described correlation evaluation system can be applied to an apparatus that performs pattern matching. In this pattern matching, the degree of coincidence between the predetermined template 35 and the pixel region in the predetermined search range 36 constituting part or all of the frame image 24 (evaluation of correlation value reliability) is performed. . The threshold value CBth for this determination is individually set based on the representative value Drep of the luminance in the search range 36 and the like. For example, an image of a road sign relating to a lane can be used as a template. While moving and enlarging this template, the correlation with the pixel area on the frame image is evaluated to identify the corresponding pixel area. Then, it is determined whether or not the brightness characteristics of the template and the brightness characteristics of the corresponding pixel region match, and the degree of matching is evaluated. For this pattern matching, a frame image captured by a monocular camera may be used.

  In the present embodiment, for pattern matching, a determination threshold for correlation evaluation is set for each search range that is a part or all of a frame image. By setting the determination threshold for this search range, it is possible to improve the correlation evaluation accuracy, improve the noise resistance, and the like as described above.

  In each of the above-described embodiments, it is assumed that the pair of data areas are stereo images, but the present invention is not limited to this. Another example of the data area is a pair of frame images that are captured at different times and are to be processed by a so-called optical flow. In this case, a monocular camera is used as the imaging unit instead of the stereo camera described above, and frame images are output in time series. Further, these data areas are not limited to images (luminance distribution) defined by the luminance information, and a pair of data areas having mutual relations can be widely used. That is, the data area may be defined by information other than luminance (for example, a heat distribution obtained from a far-infrared camera, a reflection intensity distribution obtained from a laser radar or a millimeter wave radar, etc.).

The block block diagram of the correlation evaluation system which concerns on 1st Embodiment Diagram showing the trend of correspondence between data values and noise variation The figure which shows the tendency of the correspondence relationship between the amplification factor and the noise variation The figure which shows the correspondence between the value of the data after the data correction and the noise variation Diagram showing the trend of correspondence between ambient temperature and noise variation The block block diagram of the stereo type monitoring apparatus which concerns on 2nd Embodiment. Flow chart showing determination threshold setting procedure Illustration of judgment threshold setting Explanatory diagram of correlation evaluation in stereo matching processing Explanatory drawing of the determination threshold value setting which concerns on 3rd Embodiment Explanatory drawing of the determination threshold value setting which concerns on 4th Embodiment Explanatory drawing of the determination threshold value setting which concerns on 5th Embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Correlation evaluation system 2a, 2b Memory | storage part 3 Matching process part 4 Judgment part 5 Judgment threshold value setting part 6 Base value setting part 7 Correction value setting part 8 Representative value calculation part 10 Stereo monitoring apparatus 11 Stereo camera 12a, 12b Gain Control amplifier 13 Brightness correction section

Claims (7)

In the correlation evaluation system that evaluates the correlation of the correlation destination area specified as the correlation destination of the correlation source area,
An imaging unit that captures a frame image partially including the correlation source region;
A plurality of correlation destination candidates that are correlation destination candidates in the correlation source area are set for each correlation source area, and a correlation value indicating the degree of correlation with the correlation source area is calculated for each correlation destination candidate. A matching processing unit that identifies one of the plurality of correlation destination candidates as the correlation destination region by comparing the correlation values calculated for each of the correlation destination candidates;
Based on data in the sub-region including the correlation source region, a representative value calculation unit that calculates, for each sub-region, an average value or an intermediate value of luminance values of all pixels in the sub-region as a representative value;
A basic value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and the representative value for each sub-region A threshold value setting unit for individually setting each correlation source region as a determination threshold value applied to the correlation source region;
By comparing the correlation value of the correlation destination region specified by the matching processing unit with the determination threshold value applied to the correlation source region that is the correlation source of the correlation destination region, A correlation evaluation system comprising: a determination unit that evaluates reliability.   The threshold value setting unit sets the determination threshold value of the sub-region larger as the luminance state in the sub-region indicated by the representative value becomes higher. Correlation evaluation system. In the correlation evaluation system that evaluates the correlation of the correlation destination area specified as the correlation destination of the correlation source area,
An imaging unit that captures a frame image partially including the correlation source region;
A search range including a plurality of correlation destination candidates that are correlation destination candidates of the correlation source region is set for each correlation source region, and a correlation value indicating a degree of correlation with the correlation source region is set for each correlation destination candidate And a matching processing unit that identifies any one of the plurality of correlation destination candidates as the correlation destination region by comparing the correlation values calculated for each of the correlation destination candidates.
A representative value calculation unit that calculates, for each search range, an average value or an intermediate value of luminance values of all the pixels in the search range as a representative value based on data included in the search range;
A basic value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and the representative value for each search range A threshold value setting unit that individually sets each search range as a determination threshold value applied to the search range, with a correction value set based on
The reliability of the correlation value is evaluated by comparing the correlation value of the correlation destination area specified by the matching processing unit with the determination threshold value applied to the search range including the correlation destination area. A correlation evaluation system.   The threshold value setting unit sets the determination threshold value of the search range larger as the luminance state in the search range indicated by the representative value becomes higher. Correlation evaluation system. A brightness correction unit that is provided in a preceding stage of the matching processing unit and that performs a predetermined brightness correction on the frame image;
The correlation evaluation system according to claim 1, wherein the threshold value setting unit variably sets the determination threshold value based on a degree of the luminance correction. In the correlation evaluation method for evaluating the correlation of the correlation destination area specified as the correlation destination of the correlation source area,
Capturing a frame image partially including the correlation source region;
A plurality of correlation destination candidates that are correlation destination candidates in the correlation source area are set for each correlation source area, and a correlation value indicating the degree of correlation with the correlation source area is calculated for each correlation destination candidate. Identifying any of the plurality of correlation destination candidates as the correlation destination region by comparing the correlation values calculated for each of the correlation destination candidates;
Based on the data in the sub-region including the correlation source region, calculating an average value or an intermediate value of the luminance values of all the pixels in the sub-region as a representative value for each sub-region;
A basic value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and the representative value for each sub-region Individually setting the sum of the correction value set based on the correlation source region for each correlation source region as a determination threshold applied to the correlation source region;
A step of evaluating the reliability of the correlation value by comparing the correlation value of the correlation destination region with the determination threshold value applied to the correlation source region that is the correlation source of the correlation destination region. A correlation evaluation method characterized by comprising: In the correlation evaluation method for evaluating the correlation of the correlation destination area specified as the correlation destination of the correlation source area,
Capturing a frame image partially including the correlation source region;
A search range including a plurality of correlation destination candidates that are correlation destination candidates of the correlation source region is set for each correlation source region, and a correlation value indicating a degree of correlation with the correlation source region is set for each correlation destination candidate Calculating and identifying any one of the plurality of correlation destination candidates as the correlation destination region by comparing the correlation values calculated for each of the correlation destination candidates; and
Based on the data included in the search range, calculating an average value or an intermediate value of the luminance values of all the pixels in the search range as a representative value for each search range;
A basic value set for each frame image based on at least one of an amplification factor for adjusting the brightness of the frame image and an ambient temperature of the correlation evaluation system, and the representative value for each search range Individually setting the sum of the correction value set based on the search range as a determination threshold applied to the search range for each search range;
Comparing the correlation value of the correlation destination region with the determination threshold value applied to the search range including the correlation destination region, and evaluating the reliability of the correlation value. Correlation evaluation method.

Images