JP4389602B2 - Object detection apparatus, object detection method, and program - Google Patents

Description

  The present invention relates to an object detection device, an object detection method, and a program for detecting an object such as a person or an object entering or leaving the space by separating it from the background by using distance information in the space of interest.

  Conventionally, for the purpose of monitoring a space where an object to be detected, such as a person or a car, enters and exits, a technique is known in which a detection object that enters and exits a space of interest is separated and extracted from the background. For example, using a plurality of images captured at different times by a TV camera with a fixed field of view, a difference image is generated in which the difference between the pixel values of each pixel at a corresponding position between the images is a pixel value. There is known a technique for separating a set of pixels that are equal to or more than the threshold defined by the above as a region corresponding to a detection target (moving object) from the background. A gray image or a color image is used as an image picked up by the TV camera.

  The technique described above separates the detection target from the background by using the change in brightness and color between when the detection target exists in the image and when the detection target does not exist. When there is a change in the amount of light or color due to the influence of external light or illumination, there is a problem that even if there is no detection target, it may be mistaken for the presence of the detection target.

Thus, it has been proposed to use distance information to separate and detect a detection target from the background so as not to be affected by light and darkness of the space of interest or color change (see, for example, Patent Document 1). That is, if the detection target is extracted based on the difference between the distance information of the space that does not include the detection target and the distance information of the space that includes the detection target, it is considered that the detection target is hardly affected by external light or illumination. Here, in order to obtain distance information on the assumption that the detection target is detected separately from the background, the resolution can be obtained for each part sufficiently smaller than the area occupied by the detection target in the space of interest. is required.
Japanese Patent Publication No. 8-21149

  By the way, various techniques for obtaining distance information are known, but when obtaining the distance of each part in the space as described above, an error is often included in the measurement result of any part. For example, a stereo image method that uses images captured by two TV cameras, associates each part in the two images captured by both TV cameras, and generates a distance image based on the parallax of both TV cameras. When measuring the distance of each part of the space optically, if the amount of light incident on the TV camera is small, erroneous measurement is likely to occur. Therefore, the distance measurement result at the part where the reflectance of the object existing in the space is low May cause errors.

  As described above, if there is an error in the distance measurement result at any part of the space, the distance difference includes an error other than the detection target in order to separate the detection target from the background. Therefore, in the technique described in Patent Document 1 described above, a detection target is extracted using only a part that does not include an erroneous measurement among the distance of the space that does not include the detection target and the distance information of the space that includes the detection target. Adopt technology. However, if such a technique is employed, information on the erroneously measured part is not used, and thus there is a possibility that a missing part is generated in the detection target. As a result, there is a problem that it cannot be used for the purpose of extracting the feature quantity of the detection target.

  The present invention has been made in view of the above-mentioned reasons, and its purpose is to obtain the reliability of the measurement result when measuring the distance of each part of the space, and to select the detection target candidate obtained from the distance information as the reliability. It is an object to provide an object detection apparatus, an object detection method, and a program capable of extracting a detection target with almost no missing part by performing an evaluation according to the above.

  The invention according to claim 1 is a distance measuring means for measuring the distance of each part of the space of interest, and a difference means for obtaining a difference of distances for each part measured at different times by the distance measuring means for one space of interest. The object extraction means for extracting the existence part of the candidate object to be detected in the space using the difference in distance obtained by the difference means, and the detection target is extracted from the detection target candidate extracted by the object extraction means And an object determination means for extracting the feature quantity of the detection target, and a reliability calculation means for obtaining a reliability for the measurement result of the distance of each part of the space by the distance measurement means. The object extraction means is provided for each part. It has a function to classify the difference in distance for each part by combining the reliability of the measurement results of distances at different times, and the object determination means is detected by the classification result and distance evaluation for each part in the object extraction means Subject And having a function of extracting grouped complement each detection target.

  According to the above configuration, not only the distance difference is used, but also the reliability is obtained for the distance measurement result by the distance measuring means, and the detection target candidates are evaluated based on the reliability. Even when an error is included, it is possible to reduce the erroneous detection of the detection target. As a result, it is possible to extract the detection target with almost no missing part.

  According to a second aspect of the present invention, in the first aspect of the invention, the reliability calculating means sets a neighborhood area for each part whose distance is measured by the distance measuring means, and a measurement result of the distance in the neighboring area of the target part The reliability of the part is calculated as a function of the variance of.

  According to this configuration, since the reliability of the target portion is evaluated using information obtained in the vicinity region of the target portion, it is possible to obtain the reliability of the measurement result using only the distance information obtained from the distance measuring means. Become.

  According to a third aspect of the invention, in the first aspect of the invention, the reliability calculating means sets a neighboring area for each part whose distance is measured by the distance measuring means, and is a first neighboring area of the target part. For each part included in the area, a second area that is a neighboring area is further set, and the reliability of the part as a function of the sum total of the dispersion of the measurement results of the distance in the second area in the first area Is calculated.

  According to this configuration, since the reliability of the target region is evaluated using information obtained from a wider range than the neighboring region set for the target region, the reliability of the measurement result can be determined only by the distance information obtained from the distance measuring means. The degree can be obtained. Moreover, even if the region of interest is the boundary or step portion of the detection target, the influence of the boundary or step can be reduced.

  According to a fourth aspect of the present invention, in the first aspect of the invention, the distance measuring means outputs accompanying information that is a measure of the reliability of the measurement result for the distance of each part of the space, and the reliability calculating means The reliability is obtained by using the accompanying information for the measurement result of each part.

  According to this configuration, since the reliability is obtained using the accompanying information obtained from the distance measuring means, the reliability of the measurement result is evaluated using information other than the distance information obtained by the distance measuring means. The reliability of the measurement result evaluation based on the reliability can be improved.

  According to a fifth aspect of the present invention, in the fourth aspect of the present invention, the distance measuring means associates pixels included in a plurality of images picked up using a plurality of TV cameras with each other to thereby determine the distance between each part of the space. A distance image having a pixel value of each pixel is generated, and as the accompanying information, the density of each pixel of a plurality of images captured by each TV camera is output, and the reliability calculation unit includes: A neighborhood region is set for each pixel of the image, and the reliability of the pixel is calculated as a function of density dispersion in the neighborhood region of the pixel of interest in the distance image.

  According to this configuration, a distance image is generated by a stereo image method as a distance measuring unit, and a density that greatly affects measurement accuracy is selected from any one of a plurality of images used when generating a stereo image. Therefore, the reliability that accurately reflects the measurement result can be obtained.

  According to a sixth aspect of the present invention, in the fourth aspect of the present invention, the distance measuring means associates pixels included in a plurality of images captured using a plurality of TV cameras with each other to thereby determine the distance between each part of the space. A distance image is generated as a pixel value of each pixel, and as ancillary information, a region in which a pixel is associated in a plurality of images captured by each TV camera and a density of each pixel of the plurality of images are output, The reliability calculation means calculates the reliability of the pixels included in the area as the correlation value of the density of the pixels in the associated area.

  According to this configuration, the distance measurement unit adopts a configuration that generates a distance image by the stereo image method, and uses the accuracy of associating a plurality of images used for generating a stereo image as accompanying information. The reliability that accurately reflects the measurement result can be obtained.

  According to a seventh aspect of the present invention, in the first to sixth aspects of the present invention, the object extraction means divides the reliability of the distance measurement result into a plurality of stages, and the reliability of the measurement result at different times for each part. The degree of distance is combined, the difference in distance for each part is classified into three or more types by the combination of the degree of reliability, and whether or not it is a candidate for detection is evaluated for each classification.

  According to this configuration, since the measurement results are classified into three or more types for each part using the reliability of the measurement results, and whether or not each of the classified types is a candidate for a detection target, it is detected by a simple difference image. Even parts that cannot be extracted can be extracted as candidates for detection. That is, it is possible to accurately extract detection target candidates even when the measurement result includes many erroneous measurements.

  According to an eighth aspect of the present invention, in the first to seventh aspects of the present invention, the object determining means is a three-dimensional distance between candidates in the space of interest for the detection target candidates extracted by the object extracting means. The detection target candidates that are equal to or smaller than the first threshold defined by the above are extracted as a group, and the detection targets whose distance from the group is equal to or smaller than the second threshold defined within the projection plane to the distance measuring means are selected as the group. And extracted as a group to be connected, and the connected group is set as one detection target.

  According to this configuration, it is possible to extract a region having a high possibility of being a detection target by extracting a region having a close three-dimensional distance as a group with respect to the detection target candidate, and it is highly likely that the detection target candidate is a detection target. By extracting and linking the parts having a short distance in the projection plane from the detection target candidates to the group, it is possible to accurately group each detection target.

  The invention of claim 9 is an object detection method, wherein the distance of each part of the space of interest is measured by the distance measuring means, and the distance for each part measured at different times by the distance measuring means for one space of interest. The difference between the obtained distances is extracted, the presence part of the candidate object to be detected in the space is extracted, the detection target is extracted from the extracted detection target candidates, and the feature amount of the detection target is extracted. A method for detecting the distance of each part of the space by the distance measuring means, and determining the reliability of the distance of each part by combining the reliability of the measurement results of the distance at different times for each part. Differences are classified, and detection target candidates are grouped and extracted for each detection target based on the classification result and distance evaluation for each part.

  According to this method, not only the difference in distance is used, but also the reliability of the distance measurement result obtained by the distance measuring means is obtained, and the candidate area to be detected is evaluated based on the reliability. Even when an error is included, it is possible to reduce the erroneous detection of the detection target. As a result, it is possible to extract the detection target with almost no missing part.

  A tenth aspect of the present invention is a computer-readable program, the difference means, the object extraction means, the object determination means, and the reliability calculation means in the object detection device according to any one of the first to eighth aspects. Is realized.

  That is, this program makes it possible to configure an object detection device using a general-purpose computer and distance measuring means.

  According to the configuration of the present invention, not only the distance difference is used, but also the reliability of the distance measurement result obtained by the distance measuring means is obtained, and the candidate area to be detected is evaluated based on the reliability. Even when an error is included in the result, there is an advantage that erroneous detection of the detection target can be reduced.

(Embodiment 1)
As shown in FIG. 1, this embodiment includes a distance measuring unit 1 that measures the distance to each part of a space of interest, and uses distances measured at different times by the distance measuring unit 1 for one space of interest. The object is detected.

  In the embodiment described below, as the distance measuring unit 1, the target space is imaged by two TV cameras, the pixels included in the images captured by both TV cameras are associated with each other, and the parallax of both TV cameras is used. Thus, a stereo image method distance measuring means is employed in which the distance is obtained for each pixel. That is, a distance image in which a distance is associated with each pixel is generated from images captured by two TV cameras. The distance measuring means 1 is not limited to the stereo image method, and any technique may be adopted as long as a distance image can be generated.

  The distance measuring means 1 generates a distance image at regular time intervals, and in this embodiment, the detection target is detected from two distance images obtained at different times. Here, to simplify the explanation, it is assumed that the image obtained at the previous time in the time series of the distance image does not include the detection target and includes only the background, and this distance image is referred to as the background distance image. Call. In addition, a distance image of a target for detecting the presence or absence of a detection target is referred to as a target distance image. The background distance image may be generated in advance before the operation of the apparatus.

  The background distance image obtained by the distance measuring means 1 is stored in the background storage means 2, and the target distance image obtained by the distance measuring means 1 is input to the difference means 3. The difference means 3 generates a difference distance image having a pixel value of a difference for each corresponding pixel between the target distance image and the background distance image. When the detection target is included in the target distance image, ideally, in the differential distance image, the pixel value of the region where the detection target exists is a relatively large value, and the pixel value of the other region is almost zero. . Therefore, if the pixel value of the difference distance image is binarized with an appropriate threshold, a region where a detection target exists can be extracted. However, since the actual difference distance image includes a noise component and an error component, in this embodiment, in order to reduce the influence of the noise component and the error component, distance measurement between the target distance image and the background distance image is performed. A reliability calculation unit 4 that calculates an accuracy reliability evaluation value (hereinafter referred to as “reliability”), and a pixel that is a candidate for detection from the difference distance image using the reliability calculated by the reliability calculation unit 4 An object extracting unit 5 for extracting a pixel, and an object for performing a noise component removal, pixel grouping, and the like on a pixel extracted by the object extracting unit 5 and determining whether the pixel is a detection target based on a feature amount of the grouped pixel group Determination means 6 is provided.

  In the reliability calculation means 4 of this embodiment, the reliability of the distance measurement accuracy of each pixel is calculated for both the background distance image and the target distance image. Further, the object extraction unit 5 classifies each pixel of the difference distance image by using a combination of the reliability of each pixel value of the background distance image and the reliability of each pixel value of the target distance image. The object determination unit 6 performs noise component removal and pixel grouping using the pixel classification result obtained by the object extraction unit 5, and uses feature quantities such as position, shape, and size for the grouped pixel group. To determine whether it is a detection target. A part of the distance measuring means 1, the background storage means 2, the difference means 3, the reliability calculation means 4, the object detection means 5, and the object determination means 6 are realized using a computer.

(Operation of reliability calculation means)
Below, operation | movement of the reliability calculation means 4 is demonstrated.

Reliability (1)
In this example, it is estimated that the pixel value difference is small (distance difference is small) in the vicinity region of the pixel of interest in the distance image, and the pixel value of the pixel is significantly different from the other pixels in the vicinity region. The reliability is calculated so that the reliability of the value becomes small. Specifically, a neighboring region having a size of N × M pixels (N and M are natural numbers) is set around the pixel of interest in the distance image, and the variance of pixel values in the neighboring region is obtained. The reliability is obtained using Equation 1 so as to reduce the reliability of the pixel of interest. In Equation 1, Var is variance, R is reliability, D (u, v) is a pixel value of each of the background distance image and the target distance image, and A is a neighborhood area of N × M pixels.

Reliability (2)
The reliability R obtained by the method of reliability (1) can obtain a good result for an object whose distance is substantially constant in the background distance image and the target distance image, but the boundary between the detection target and the background, Since the variance is relatively large at a stepped portion or the like present in the object, the reliability R obtained by Equation 1 is a small value even when the distance measurement accuracy is high. Therefore, in this example, if the accuracy of distance measurement around the boundary or step is high, the reliability is obtained by assuming that the distance is measured with the same degree of accuracy at the boundary or step portion.

  That is, the reliability of the pixel value of the pixel of interest is not calculated from the dispersion of the pixel value in the neighborhood area of the size of N × M pixels centered on the pixel of interest, as shown in Equation 2. It is obtained from the sum of the variances of the neighborhood area set with a size of N × M pixels with each of the pixels included in the neighborhood area of the pixel of interest as the center. In Equation 2, n is a code for distinguishing the pixels in the neighboring region set for the pixel of interest, and 1 ≦ n ≦ N × M, and Var [n] is a neighborhood set around the pixel n. The variance in the region, R is the reliability, D (u, v) is the pixel value of each of the background distance image and the target distance image, and An is a neighborhood region of N × M pixels centered on the pixel n. Region means a region including the entire neighborhood region An. The reliability R calculated by Equation 2 is the sum of the reliability calculated by the method of reliability (1) around the boundary or step even if it is the boundary between the detection target and the background or the step portion in the object. As a result, the effects of boundaries and steps are alleviated.

(Operation of object extraction means)
Below, the technique which classifies each pixel of a difference distance image in the object extraction means 5 is demonstrated.

  Now, it is assumed that the target distance image as shown in FIG. 2A is output from the distance measuring means 1 and the background distance image as shown in FIG. In the example shown in the figure, it is assumed that the part having the higher brightness is closer (the white part is closest). Further, the reliability calculation means 4 calculates the reliability as shown in FIG. 3A for each pixel of the target distance image output from the distance measurement means 1 and stores the background stored in the background storage means 2. Assume that the reliability is calculated for each pixel of the distance image as shown in FIG. In FIG. 3, the brighter the higher the reliability (the white part is the highest).

  The object extraction unit 5 classifies each pixel into three types based on the reliability of the target distance image and the background distance image. In other words, by binarizing the reliability of the target distance image and the background distance image and combining the magnitudes of the reliability, when both are highly reliable, when both are low reliability, Are classified into three types when only high reliability. For example, the reliability regarding the target distance image in FIG. 3A is binarized as shown in FIG. 4A, and the reliability regarding the background distance image in FIG. 3B is binary as shown in FIG. 4B. Suppose that In FIG. 4, the shaded area indicates the low reliability region. When the three types of classification described above are performed by combining the areas shown in FIGS. 4A and 4B, the areas are classified as shown in FIG. In FIG. 4 (c), the white part indicates a highly reliable part, the black part indicates a low reliable part, and the hatched part indicates only one highly reliable part.

  If the three types of regions are classified as described above, a part with high reliability in both the target distance image and the background distance image is extracted as a detection target candidate when Df−Db ≧ thr1 is satisfied. The Here, Df and Db are pixel values (distances) of pixels at the same position in the target distance image and the background distance image, and thr1 is a threshold set slightly larger than the distance measurement accuracy. In short, if a region closer to the background exists, it is determined as a candidate for detection. Further, with respect to a part where only one of the target distance image and the background distance image is highly reliable, all parts are determined as detection target candidates. FIG. 5 shows candidate regions to be detected that are obtained as a result of applying the above-described conditions to the classification result of FIG. The black part in FIG. 5 is a part to be a candidate for detection.

  However, as is clear from FIG. 5, there is a part that is not selected as a detection target candidate in a part of the part that is considered as a detection target (in FIG. 5, the left side that is a detection target candidate in FIG. 5). The bottom of the area is outlined). In order to extract such a part as a candidate for a detection target, if a part with a relatively large change in reliability is extracted from parts where both the target distance image and the background distance image have low reliability, Good. That is, the reason why the reliability has changed between the target distance image and the background distance image is estimated to be due to the occurrence of the detection target, and the difference between the reliability Rf and Rb obtained from the target distance image and the background distance image is estimated. When the absolute value is equal to or greater than the prescribed threshold value thr2, that is, a region satisfying | Rf−Rb | ≧ thr2 is included in the detection target candidates. The result of applying this condition to FIG. 5 is shown in FIG. The object extraction unit 5 can extract all the detection target candidates by the above-described processing.

  In the above example, the reliability of the target distance image and the background distance image is binarized, but it can be divided into three or more stages.

(Operation of object judgment means)
The object extraction unit 5 extracts a part that is a candidate for detection, but there may be a case where a noise component is included or a part that should be grouped is separated. is necessary.

  Therefore, in the object determination unit 6, with respect to the detection target candidates extracted by the object extraction unit 5, a pixel group (hereinafter referred to as “A group”) with high ranging accuracy (high reliability of measurement) and other They are classified into pixel groups (hereinafter referred to as “B group”).

  The conditions for the pixels in the A group are the pixel values Di and Dj (that is, distances) of the pixels i and j for the two pixels i and j in the region that is a candidate for detection in the target distance image, That is, the relationship of Equation 1 in Expression 3 is satisfied between the coordinates (ui, vi) (uj, vj) of the pixel i, j and the prescribed threshold value thrA. Further, the condition of the pixels in the group B is that the coordinates (uk, vk) (ul, vl) of the pixels k and l are set for two pixels k and l in the region that is a candidate for detection in the target distance image. Satisfies the relationship of Equation 2 in Equation 3 with the prescribed threshold value thrB. Further, between the group A and the group B, the expression 3 in the expression 3 is provided between the coordinates (ua, va) (ub, vb) of the pixel a of the group A and the pixel b of the group B and the prescribed threshold value thrAB. When the above relationship is satisfied, it is determined that the detection target is one, and the pixels of the A group and the B group are integrated into one group. The result obtained by the above process is shown in FIG. In FIG. 7, A1 and A2 are groups of pixels of A group, B1 to B4 are groups of pixels of B group, (A1, B2) are integrated into one group, and (A2, B1, B3, B4) is one group. Integrated into. That is, each group becomes an individual detection target.

  By the way, when the pixels of the A group and the B group are integrated into the group as described above, the pixels of the B group that are generated at the positions shown as B5 in FIG. 8 are the group including the A1 and the group including the A2. Any of them may be grouped. In such a case, it is necessary to determine to which group the group B pixels indicated as B5 are to be integrated. For example, if the purpose of detecting the detection target is to monitor so as not to enter the specific area (dangerous area) Ad shown in FIG. 9, B5 may be integrated into the detection target closer to the specific area Ad. It is considered reasonable (detection is easy because the number of pixels in the group increases).

  Here, the distance between the specific area Ad and each group including A1 and A2 is the distance Dd from the distance measuring means 1 to the representative point of the specific area Ad, and the distance from the distance measuring means 1 to the representative point of each group. Evaluate by. The distances D1 and D2 to the representative point of each group are obtained by Equation 4. Equation 4 is an average value of pixel values (distances) included in A1 and A2, and N1 and N2 are the numbers of pixels included in A1 and A2, respectively.

  As described above, when integrating the pixels of the A group and the B group into a group that can be regarded as one detection target, if the noise component is removed in advance for each pixel group, the detection target can be accurately determined. It also reduces the processing load. Here, since the noise component is considered to have a small area occupied by the pixel group, an appropriate threshold is set for the number of pixels (area) of each pixel group of the A group and the B group, and the number of pixels is equal to or less than the threshold value. A certain pixel group may be deleted from the processing target as a noise component.

  However, the larger the distance from the distance measuring means 1, the smaller the area occupied in the image. Therefore, if the area of each pixel group is simply compared with a threshold value, there is a possibility that a distant detection target is deleted. Therefore, if the distance between the representative points of each pixel group is obtained and the threshold value is changed according to the distance, the possibility of misdetecting the detection target as a noise component and deleting it can be reduced. Such a threshold value may be set to be inversely proportional to the distance, for example, and set to a value obtained by multiplying an appropriate coefficient depending on the angle of view of the distance measuring means 1.

  Even if the pixel groups are integrated into one group by the above-described processing, the pixel group to be the B group may have relatively low distance measurement accuracy. For example, when a human body is a detection target, as shown in FIG. 10, there is a possibility that the distance measurement accuracy is low for the hair portion. Therefore, when using for the purpose of calculating | requiring the distance to a detection target, the distance of a hair part cannot be employ | adopted. When the pixel group constituting the group includes a pixel group (B group) with low ranging accuracy, a pixel group with high ranging accuracy in the vicinity of the pixel group in the group from a pixel group with low ranging accuracy. The same number of pixels as the group are extracted, and the average value of the extracted pixels is used in place of the distance of the pixel group with low ranging accuracy. That is, when the pixel value (distance) of the pixel extracted from the pixel group Obj with high ranging accuracy is D (u, v) and the extracted number is Num, the distance Dv is obtained by applying Expression 5. That is, the distance Dv of the pixel group with low ranging accuracy is obtained by interpolation of the distance of the pixel group with high ranging accuracy.

  When the detection target has a shape close to a plane like a human body, the distance Dv to the detection target may be obtained by a weighted average as shown in Equation 6. In Equation 6, D (u, v) is each pixel value (distance) of the detection target in the target distance image, Obj is the pixel to be detected, and R (u, v) is obtained by the reliability calculation means 4. The reliability of each pixel.

  In addition to the distance to the detection target, the object determination means 6 may obtain the position of the detection target and the size as the feature amount. When obtaining the position and size of the detection target, as shown in FIG. 11, a circumscribed rectangle E is set in the area occupied by one group described above. The circumscribed rectangle E can be set by obtaining the minimum value and the maximum value of the coordinate values in the horizontal direction and the vertical direction constituting the group. Since the distance to the detection target can be obtained by Equations 5 and 6, etc., if the distance to the detection target is Do, by using the width w and the height h of the circumscribed rectangle E in the target distance image, The actual height W and width H of the detection target can be obtained by Equation 7. Β in Equation 7 is a constant unique to the distance measuring means 1. Further, the area S that is the feature quantity of the detection target can be approximated by the area of the circumscribed rectangle E as shown in Equation 7, and when the aspect ratio Aspect of the circumscribed rectangle E is obtained as the feature quantity of the detection target, the human body It is also possible to determine whether (Aspect≈0.5) or a small animal such as a dog or cat (Aspect≈1).

(Embodiment 2)
As shown in FIG. 12, the present embodiment includes a distance measuring means 1, a background storage means 2, a difference means 3, a reliability calculation means 4, an object extraction means 5, and an object determination means 6, as in the first embodiment. . However, the background storage means 2 does not store only the background distance image, but stores information used when generating the background distance image in the distance measurement means 1 as background information, and the reliability calculation means 4 stores the background distance. Background information is used in calculating the reliability of each pixel value of the image, and information used when generating the target distance image in the distance measuring means 1 instead of the target distance image is used as target information for calculating the reliability. Specific examples of the background information and the target information will be described later. The background information and the target information become accompanying information that is not the distance information that is the output purpose of the distance measuring means 1.

(Operation of reliability calculation means)
Reliability (3)
This example is an operation example of the reliability calculation unit 4 in the present embodiment, and utilizes the fact that the distance measurement unit 1 employs the stereo image method. The stereo image method generally uses a plurality of images, but here, for the sake of simplicity, two images on the left and right are used. One of the two images obtained by the stereo image method is used as background information and target information. In the stereo image method, it is necessary to associate each pixel included in the left and right images, and the parallax between the two images is obtained by associating the pixels, and the image is obtained by using the distance and the parallax between the two TV cameras. The distance between each pixel is obtained. Therefore, in a region where the change in density is small, it is difficult to associate the pixels on a one-to-one basis. If the association is inaccurate, an error occurs in the distance measurement value.

  Therefore, in the present embodiment, when a pixel associated with one image (for example, the left image) of the two images used in the stereo image method is extracted from the other image (for example, the right image), Equation 8 is obtained. In addition, the dispersion of density in the vicinity of N × M pixels (N and M are natural numbers) centering on the image of interest in the one image is obtained, and the dispersion of density is used with reliability. In this case, the greater the variance, the easier the pixel association, so the reliability of the distance measurement value increases. In Equation 8, Var is variance, R is reliability, I (u, v) is the density of the pixel (u, v), and A is a neighborhood area of N × M pixels.

Reliability (4)
Similar to the reliability (3), this example is based on the knowledge that the quality of the correspondence between the two images in the stereo image method affects the accuracy of the distance measurement (normalization value (normalization) (Correlation value) is obtained by Equation 9 and used as background information and target information for the background distance image and the target distance image. In Equation 9, Il (ul, vl) and Ir (ur, vr) are the density of the left image and the density of the right image, Al and Ar are areas associated with each other in the left and right images, and Avel and Aver are areas, respectively. It is the average value of the concentration in Al and Ar. The evaluation value C obtained by Equation 9 is a value of 0 ≦ C ≦ 1, and becomes larger as the correspondence between the two images is better.

In the reliability (4), a color (at least one of hue, saturation, and brightness) may be used instead of the density.

Reliability (5)
When obtaining the normalized correlation value C in Equation 9, the differential value Amp or the concentration gradient direction Dir shown in Equation 10 may be used instead of using the luminance. In order to obtain the differential value of the pixel of interest, a Sobel filter (Sobel) as shown in FIG. 13 may be applied. FIG. 13A is a Sobel filter for obtaining the horizontal differential value H, and FIG. 13B is a Sobel filter for obtaining the vertical differential value V.

  Other configurations of the present embodiment are the same as those of the first embodiment.

  In the first embodiment and the second embodiment described above, the distance measuring unit 1 is assumed to generate a distance image by the stereo image method using two TV cameras. You may employ | adopt what measures distance using the time difference from receiving an ultrasonic wave) or light transmission to receiving a reflected wave. In this type of distance measuring means 1, as shown in FIG. 14, when the speed of sound or light is Ss, when the time difference from transmission to reception is Tr, the distance D to the object is expressed as D = Ss × Tr / 2. In such distance measuring means 1, it can be said that the greater the received wave intensity Ar of the reflected wave, the higher the reliability of distance measurement. Alternatively, as the sound or light that is intermittently transmitted, the reception intensity A0 at the time of transmission of the sound or light and the reception intensity Ar at the reception of the reflected wave are used, and the ratio between them (= Ar / A0) or the difference between the two (Ar−A0) may be used for the reliability R.

  As the distance measuring means 1 using light, the intensity of the transmitted light is modulated (that is, the intensity of the light is changed with time according to a predetermined rule), and the transmitted light and the reflected light are received. You may employ | adopt the technique which calculates | requires the distance to an object from the phase difference of intensity modulation with a wave. For example, when the intensity of transmitted light is modulated with a sine wave, the received intensity Ar of the received light changes as shown in FIG. 15, and the detection error of the phase difference increases as the received intensity Ar increases. Therefore, the received wave intensity Ar can be used as a measure of the reliability of distance measurement. That is, the received wave intensity Ar can be used as the reliability R.

1 is a block diagram illustrating a first embodiment. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. FIG. 6 is a block diagram illustrating a second embodiment. It is a figure which shows the Sobel filter used for the same as the above. It is operation | movement explanatory drawing of the example of another structure. It is operation | movement explanatory drawing of another structural example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Distance measuring means 2 Background memory means 3 Difference means 4 Reliability calculation means 5 Object extraction means 6 Object determination means

Claims (10)

  A distance measuring means for measuring the distance of each part in the space of interest, a difference means for obtaining a difference in distance for each part measured at different times by the distance measuring means for one space of interest, and a difference means The object extraction means for extracting the existence part of the candidate object to be detected in the space using the difference of the distance, and the detection target is extracted from the detection target candidate extracted by the object extraction means, and the feature amount of the detection target is obtained. An object determining means for extracting, and a reliability calculating means for obtaining a reliability of the distance measurement result of each part of the space by the distance measuring means, wherein the object extracting means is a measurement result of the distance at a different time for each part The function of classifying the difference in distance for each part based on the combination of the reliability of the object, the object determination means for each detection target candidate of the detection target by the classification result and distance evaluation of each part in the object extraction means Object detecting apparatus characterized by having a function of extracting grouped.   The reliability calculation means sets a neighborhood area for each part whose distance is measured by the distance measuring means, and calculates the reliability of the part as a function of the dispersion of the distance measurement result in the vicinity area of the part of interest. The object detection apparatus according to claim 1.   The reliability calculation means sets a neighborhood area for each part whose distance is measured by the distance measuring means, and further includes a second area that is a neighborhood area for each part included in the first area that is the neighborhood area of the site of interest. 2. The object according to claim 1, wherein the reliability of the part is calculated as a function of the total sum of the variances of the distance measurement results in the second area in the first area. Detection device.   The distance measuring means outputs accompanying information that is a measure of the reliability of the measurement result for the distance of each part of the space, and the reliability calculating means uses the accompanying information for the measurement result of each part of the space. The object detection apparatus according to claim 1, wherein:   The distance measuring means generates a distance image in which the distance of each part of the space is the pixel value of each pixel by associating pixels included in the plurality of images captured using a plurality of TV cameras. Then, as the accompanying information, the density of each pixel of the plurality of images captured by each TV camera is output, and the reliability calculation unit sets a neighborhood region for each pixel of the image, and the distance image The object detection device according to claim 4, wherein the reliability of the pixel is calculated as a function of density dispersion in a region near the pixel of interest.   The distance measuring means generates a distance image in which the distance of each part of the space is the pixel value of each pixel by associating pixels included in the plurality of images captured using a plurality of TV cameras. Then, as ancillary information, a region in which a pixel is associated in a plurality of images captured by each TV camera and a density of each pixel in the plurality of images are output, and the reliability calculation unit is configured to output the pixel in the associated region. The object detection apparatus according to claim 4, wherein the reliability of pixels included in the region is calculated as a density correlation value.   The object extracting means classifies the reliability of the distance measurement result into a plurality of stages, combines the reliability levels of the measurement results at different times for each part, and determines the difference in distance for each part as the reliability stage. The object detection apparatus according to claim 1, wherein the object detection device is classified into three or more types based on the combination of the two, and whether or not each of the classifications is a candidate for a detection target is evaluated.   The object determination means extracts detection target candidates whose detection target candidates extracted by the object extraction means have a three-dimensional distance between the candidates within the space of interest as a specified threshold value or less as a group. In addition, a detection target whose distance to the group is equal to or less than a second threshold defined in the projection plane to the distance measuring means is extracted as a group connected to the group, and the connected group is defined as one detection target. The object detection device according to claim 1, wherein the object detection device is an object detection device.   The distance of each part of the space of interest is measured by the distance measuring means, the difference of the distances of each part measured at different times by the distance measuring means for one space of interest is obtained, and the space is calculated using the obtained distance difference. An object detection method for extracting a detection target from an extracted detection target candidate and extracting a feature amount of the detection target from the extracted detection target candidate. Obtain the reliability of the distance measurement results for each part, classify the difference in distance for each part according to the combination of the reliability of the distance measurement results at different times for each part, and determine the classification result and distance for each part. An object detection method, wherein candidates for detection targets are grouped and extracted for each detection target by evaluation.   9. A computer-readable program for realizing a difference unit, an object extraction unit, an object determination unit, and a reliability calculation unit in the object detection apparatus according to claim 1.

Images