JP2006113738A - Device and method for detecting object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detecting device for correctly determining whether a detected object area is a background part or an object. <P>SOLUTION: The device comprises imaging means 101a, 101b for imaging a prescribed monitor area; a three-dimensional information calculating means 102 for calculating three-dimensional information which expresses the position of the monitor area; an object extracting means 103 for extracting the object area where the object exists; a histogram calculation area determining means 104 for determining a histogram calculation area to calculate a histogram; a feature extracting means 105 for extracting a feature image; a reference feature storage means 106 for storing the feature image of reference; an area dividing means 107 for dividing the histogram calculation area into a plurality of areas; a histogram correlation calculating means 108 for calculating a correlation value between data concerning the feature histogram based on the feature image and data concerning a reference feature histogram based on the feature image of reference; and an object determining means 109 for determining whether or not the object exists in the monitor area. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an object detection apparatus for detecting an object by calculating three-dimensional information of a monitoring area using a plurality of cameras.

As a conventional object detection device, a distance image generation unit that generates a distance image of a traveling road surface and an obstacle on the traveling road surface by stereo image processing, and a three-dimensional coordinate position distribution of the traveling road surface and the obstacle on the traveling road surface are generated. When a moving body such as an unmanned dump truck travels on the travel path, the coordinate conversion unit is configured to include a ground detection unit that detects the travel road surface and an obstacle detection unit that detects an obstacle on the travel road surface. In addition, the distance image generation unit generates a distance image of the traveling road surface and the obstacle on the traveling road surface from the image obtained by the stereo camera, and the coordinate conversion unit calculates the three-dimensional coordinate position distribution of each pixel from the generated distance image. Generated, and based on the generated three-dimensional coordinate position distribution, the ground detection unit identifies a pixel group on the traveling road surface to detect the traveling road surface, and a pixel group having a predetermined height or more based on the detected traveling road surface Obstacle detection unit is known to detect that the obstacle (e.g., see Patent Document 1).
Japanese Patent Laid-Open No. 10-143659 (page 4-7, FIG. 1)

  However, in the conventional object detection device, in the environment where there is a uniform texture such as gravel or lawn, if a mishandling occurs in the process of stereo image processing, even if there is no object, the background of gravel or lawn etc. There has been a problem that a pixel group representing a partial image is detected by the obstacle detection unit as an object.

  The present invention has been made to solve the conventional problems, and provides an object detection device capable of accurately determining whether a detected object region is a background part or an object. Objective.

  The object detection apparatus according to the present invention includes a plurality of imaging units that image a predetermined monitoring area, and a three-dimensional representation of the position of the monitoring area based on each image obtained by imaging each of the plurality of imaging units. Three-dimensional information calculation means for calculating information, object extraction means for extracting an object area where an object exists from the monitoring area based on the three-dimensional information calculated by the three-dimensional information calculation means, and the object extraction means Histogram calculation area determination means for determining a histogram calculation area for calculating a histogram based on the extracted object area, and a feature for extracting a feature image from an image obtained by one of the plurality of imaging means Extraction means; reference feature storage means for storing a reference feature image extracted by the feature extraction means; and the histogram calculation area A region dividing unit that divides the histogram calculation region determined by the determining unit into a plurality of regions, data relating to a feature histogram based on the feature image extracted by the feature extracting unit, and a reference stored by the reference feature storing unit Histogram correlation calculation means for calculating a correlation value with data relating to a reference feature histogram based on the feature image of the image, and whether or not an object exists in the monitoring area based on the correlation value calculated by the histogram correlation calculation means It has the structure provided with the object determination means which determines.

  With this configuration, the correlation value between the data relating to the feature histogram based on the image obtained by imaging and the data relating to the reference feature histogram based on the image obtained at the time of reference is calculated, and the monitoring region is calculated based on the correlation value. Therefore, it is possible to accurately determine whether the detected object region is a background portion or an object.

  The object detection apparatus of the present invention calculates a difference between a frame representing an image obtained by one of the plurality of imaging means and a frame a predetermined time before the time when the frame was obtained. Frame difference calculation means for processing, and processing area determination means for determining a histogram calculation area based on the frame difference area obtained from the frame difference and the object area.

  With this configuration, an area where there is a high possibility that an object exists is used as a histogram calculation area, so that when an object exists in the object area extracted from the monitoring area, it is erroneously determined that no object exists. Can be prevented.

  The object detection method of the present invention includes: an imaging step in which a plurality of imaging units image a predetermined monitoring area; and the monitoring area based on each image obtained by imaging each of the plurality of imaging units in the imaging step. A three-dimensional information calculation step for calculating three-dimensional information representing the position of the object, and an object extraction step for extracting an object region where an object exists from the monitoring region based on the three-dimensional information calculated in the three-dimensional information calculation step; A histogram calculation area determination step for determining a histogram calculation area for calculating a histogram based on the object area extracted in the object extraction step, and one imaging means among a plurality of imaging means in the imaging step A feature extraction step for extracting a feature image from the obtained image, and a reference feature image extracted in the feature extraction step Reference feature storing step for storing, region dividing step for dividing the histogram calculation region determined in the histogram calculation region determination step into a plurality of regions, and data relating to the feature histogram based on the feature image extracted in the feature extraction step Based on the correlation value calculated in the histogram correlation calculation step, the histogram correlation calculation step for calculating the correlation value with the data related to the reference feature histogram based on the reference feature image stored in the reference feature storage step And an object determination step for determining whether or not an object exists in the monitoring area.

  By this method, the correlation value between the data relating to the feature histogram based on the image obtained by imaging and the data relating to the reference feature histogram based on the image obtained at the time of reference is calculated, and the monitoring region is calculated based on the correlation value. Therefore, it is possible to accurately determine whether the detected object region is a background portion or an object.

  Further, the object detection method of the present invention includes a frame representing an image obtained by imaging by one of a plurality of imaging means in the imaging step, and a predetermined time before the time when the frame is obtained. A frame difference calculation step for calculating a difference from the frame; and a processing region determination step for determining a histogram calculation region based on the frame difference region obtained from the frame difference and the object region.

  By this method, since the area where there is a high possibility that an object exists is used as the histogram calculation area, it is erroneously determined that the object does not exist when the object exists in the object area extracted from the monitoring area. Can be prevented.

  Furthermore, the object detection method of the present invention has a configuration in which the region dividing step divides the histogram calculation region in a direction perpendicular to a scanning line of the image obtained in the imaging step. Yes.

  With this method, even when a fluctuation in the vertical direction occurs with respect to the scanning line of the image obtained by imaging due to the influence of wind or the like, the influence of this fluctuation on the feature histogram is reduced in order to reduce the influence on the feature histogram. Detection can be prevented.

  Furthermore, the object detection method of the present invention has a configuration in which the region dividing step divides the histogram calculation region in a direction parallel to a scanning line of the image obtained in the imaging step. Yes.

  With this method, even if a shake in the direction parallel to the scanning line of the image obtained by imaging is generated due to the influence of wind or the like, the effect of this shake on the feature histogram is reduced in order to reduce the effect on the feature histogram. Detection can be prevented.

  The present invention provides a histogram correlation calculating means for calculating a correlation value between data relating to a feature histogram based on an image obtained by imaging and data relating to a reference feature histogram based on an image obtained at the time of reference, It is possible to accurately determine whether the detected object region is a background part or an object by providing an object determination unit that determines whether an object exists in the monitoring region based on the correlation value An object detection apparatus is provided.

  Hereinafter, an object detection device according to an embodiment of the present invention will be described with reference to the drawings. Note that the object detection apparatus according to the embodiment of the present invention detects an object by, for example, calculating three-dimensional information of a monitoring area using a plurality of cameras.

(First embodiment)
FIG. 1 shows an object detection apparatus according to a first embodiment of the present invention.

  In FIG. 1, the object detection apparatus according to the first embodiment of the present invention includes imaging means 101a and 101b for imaging a predetermined monitoring area, three-dimensional information calculation means 102 for calculating three-dimensional information, and an object area. , An object extraction unit 103 that extracts a histogram calculation region, a histogram calculation region determination unit 104 that determines a histogram calculation region, a feature extraction unit 105 that extracts a feature image, a reference feature storage unit 106 that stores a reference feature image, and a histogram calculation Area dividing means 107 for dividing an area into a plurality of areas, histogram correlation calculating means 108 for calculating a correlation value between data relating to a feature histogram based on a feature image and data relating to a reference feature histogram based on a reference feature image; , And an object determination unit 109 for determining whether or not an object exists.

  The histogram correlation calculation unit 108 includes a feature histogram generation unit 108a that generates a feature histogram, a reference feature histogram generation unit 108b that generates a reference feature histogram, and a correlation value calculation unit 108c that calculates a correlation value.

  About the object detection apparatus comprised as mentioned above, the operation | movement is demonstrated using FIGS.

  First, the imaging units 101a and 101b image a predetermined monitoring area (step S201).

  Next, the three-dimensional information calculation unit 102 calculates three-dimensional information representing the position of the monitoring area based on the respective images obtained by the imaging units 101a and 101b (step S202). The original information is converted from the imaging means coordinate system to the real space coordinate system.

  Here, a method in which the three-dimensional information calculation unit 102 calculates the three-dimensional information of the monitoring area will be described with reference to FIG.

  In FIG. 3, coordinates (x, y, z) representing the real space coordinate system composed of the x-axis, y-axis and z-axis are coordinates (X, Y) representing the imaging means coordinate system composed of the X-axis and the Y-axis. Is used. However, in order to distinguish between the imaging means 101a and 101b, the coordinates (XL, YL) on the image plane of the imaging means 101a consisting of the XL axis and the YL axis are set on the image plane of the imaging means 101b consisting of the XR axis and the YR axis. The coordinates (XR, YR) are used. Further, the x axis, the XL axis, and the XR axis are parallel to each other, the y axis, the YL axis, and the YR axis are parallel to each other. Further, the z axis and the optical axes of the imaging units 101a and 101b are parallel to each other. The origin O of the real space coordinate system is taken as the midpoint of the projection center of the imaging means 101a and the projection center of the imaging means 101b, and the distance between the projection centers is represented by 2a. Further, the distance between the projection center and the image plane is represented by a focal length f.

  If points p (x, y, z) in the real space are imaged on the image planes of the imaging means 101a and 101b, respectively, are point PL (XL, YL) and point PR (XR, YR). The three-dimensional information calculation means 102 finds a point corresponding to the point PL on the image plane from the image plane of the image pickup means 101a with reference to the image plane of the image pickup means 101a, and based on the principle of triangulation A point p (x, y, z) is calculated.

Hereinafter, a method for calculating the point p (x, y, z) in the real space will be described in detail. Since the imaging means 101a and 101b are set so that their optical axes are parallel to each other and on the same plane, YR = YL. Therefore, the point p (x, y, z) in the real space is
(Equation 1)
x = a (XL + XR) / (XL-XR) (1)
(Equation 2)
y = 2aYL / (XL-XR) (2)
(Equation 3)
z = 2af / (XL-XR) (3)
It is expressed. Here, d = XL-XR is called parallax.

From the equations (1) to (3) described above,
(Equation 4)
XL = (x + a) f / z (4)
(Equation 5)
XR = (x−a) f / z (5)
(Equation 6)
YL = YR = yf / z (6)
So,
(Equation 7)
XL> XR and YL = YR (7)
It becomes.

  Here, the equation (7) indicates that the point PR on the image plane of the imaging unit 101b corresponding to the point PL on the image plane of the imaging unit 101a exists on the same scanning line and in the range of XL> XR. To express. Accordingly, the three-dimensional information calculation unit 102 checks the image similarity for a certain small region along the same scanning line, thereby matching the point on the image plane of the imaging unit 101b corresponding to the point PL on the image plane of the imaging unit 101a. You can find PR.

  Next, a method for evaluating similarity will be described. Here, Miyoshi et al., “Driving support system using three-dimensional image recognition technology”, Automobile Society of Japan Academic Lecture Preprint 924, October 1992, pp. 11-28. This will be described with reference to FIG. 4 based on the method described in 169-172. As shown in FIG. 4, the three-dimensional information calculation unit 102 divides the image 401 into blocks each having a size of n × m pixels based on the image 401 of the imaging unit 101a, and the imaging unit 101b for each divided block. A corresponding area corresponding to the image 402 is found out, and the parallax d is calculated.

Also, the three-dimensional information calculation means 102 uses the similarity evaluation formula (8) for determining the corresponding region, and moves the block 404 one pixel at a time within the search range 405 of XL> XR while comparing the similarity to the block 403. An evaluation value C is calculated, and a position where the similarity evaluation value C is minimum is determined as a corresponding region. Note that Li and Ri represent the luminance at the i-th pixel in the blocks 403 and 404, respectively. The three-dimensional information calculation means 102 calculates the parallax d of all the blocks of the image 401 by performing this similarity evaluation for each block of the image 401, and calculates the three-dimensional information such as the distance z by Expression (3). To do.
(Equation 8)
C = Σ | Li−Ri | (8)
As described above, the three-dimensional information calculation unit 102 calculates the three-dimensional information of the monitoring area. Various methods other than the above method have been proposed as a method for calculating three-dimensional information by stereo image processing, and the present invention is not limited to the above method.

  Next, a method in which the three-dimensional information calculation unit 102 converts from the imaging unit coordinate system to the real space coordinate system will be described with reference to FIGS. 5 and 6.

  As shown in FIG. 5, an imaging means coordinate system (X, Y, Z) composed of an X axis, a Y axis, and a Z axis, and a real space coordinate system (U, V, composed of a U axis, a V axis, and an H axis). , H) are arranged in such a positional relationship that the X axis and the U axis are parallel and the angle formed by the Z axis and the H axis is θ.

Further, as shown in FIG. 6, the imaging unit 601 is installed at a height Hcam from the reference plane 603 existing in the monitoring area, and the object 602 is present at a distance Z1 in the optical axis z direction. When the object 602 exists, the object 602 is imaged at the point (X, Y) on the imaging unit coordinate system, and when the object 602 does not exist, the optical axis to the reference plane 603 imaged at the point (X, Y) described above. If the distance in the direction is Z0, the three-dimensional information calculation unit 102 calculates the height h of the object 602 in the monitoring area by the equation (9).
(Equation 9)
h = Hcam × (1−Z1 / Z0) (9)
In addition, the three-dimensional information calculation unit 102 calculates the distance v in the V-axis direction from the origin of the real space coordinate system (U, V, H) to the object 602 by Expression (10).
(Equation 10)
v = {Z1- (Hcam-h) cos θ} / sin θ (10)
On the other hand, as shown in FIGS. 5 and 6, if the horizontal imaging range at a distance Z from the imaging means 601 is W and the horizontal size of the imaging surface of the imaging means 601 is XW, the three-dimensional information calculation means 102 is , The distance u in the U-axis direction to the object 602 is calculated by the equation (11).
(Equation 11)
u = X / XW × W (11)
Here, when the pitch width of the CCD of the image pickup means is pH, the horizontal image pickup range W is expressed by Expression (12).
(Equation 12)
W = pH × XW × Z / f (12)
As described above, the three-dimensional information calculation unit 102 converts the imaging unit coordinate system (X, Y, Z) into the real space coordinate system (U, V, H).

  The object extraction unit 103 extracts an object region where an object exists from the monitoring region based on the three-dimensional information calculated by the three-dimensional information calculation unit 102 (step S203).

  Here, a method by which the object extraction unit 103 extracts an object region will be described with reference to FIG.

  The object extraction unit 103 divides the reference plane 703 into a predetermined size and divides the reference surface 703 into small regions 702, calculates a representative height value for each small region 702, and creates a height distribution H (U, V). As a method for calculating the height representative value, for example, the highest value is selected from the three-dimensional information 701 belonging to each small region 702. In addition, the object extraction unit 103 stores three-dimensional information at a predetermined reference time, and creates a reference height distribution H0 (U, V) in the same manner as the method for calculating the representative height value. Here, the three-dimensional information at a predetermined reference time stored by the object extracting unit 103 is a real space coordinate system (U, Y) obtained by coordinate transformation of the imaging unit coordinate system (X, Y, Z) by the three-dimensional information calculation unit 102. V, H).

Then, the object extraction unit 103 determines that the small area (U, V) is the object area when Expression (13) with a predetermined threshold value Hth is satisfied.
(Equation 13)
H (U, V) −H0 (U, V) ≧ Hth (13)
The object extraction unit 103 sets each of the adjacent object areas as one object detection area among the detected object areas, specifies the upper left point and the lower right point of the object detection area, and uses the specified result as the object detection area. Store in information. As an example, the object detection area information includes a point (lx (i), ly (i)) at the upper left point of the i-th object detection area and a point (rx (i), ry (i) at the lower right point. ), The object detection area information OBJINFO [i] = (lx (i), ly (i), rx (i), ry (i)) (i = 1,..., CntObj). Here, the number of object detection areas is expressed as CntObj.

  When it is determined by Equation (13) that no object exists, the time may be set as the reference time. This prevents the creation of the reference height distribution H0 (U, V) when an object is present in the monitoring area, thereby reducing the influence of environmental fluctuations.

  In addition, the object extraction unit 103 determines a reference time interval in consideration of a time during which an object in the monitoring area can be detected, and obtains three-dimensional information before the reference time interval from the time when the object area is extracted from the monitoring area. You may have the past information storage means to accumulate | store. In this case, the object extraction unit 103 creates a reference height distribution H0 (V, H) based on the three-dimensional information accumulated by the past information accumulation unit. Thereby, since the reference height distribution H0 (U, V) created at a time close to the time determined by the equation (13) is used, it is possible to reduce the influence due to the fluctuation of the environment.

  The histogram calculation area determination means 104 determines a histogram calculation area for calculating a histogram based on the object area extracted by the object extraction means 103 (step S204).

  Here, a method in which the histogram calculation area determination unit 104 determines the histogram calculation area will be described with reference to FIG.

  As shown in FIG. 9, it is assumed that six object regions 903 are extracted by the object extracting unit 103 in an image 901 obtained by imaging by the imaging unit 101a or 101b.

  The histogram calculation area determination means 104 calculates the upper left point and lower right point of the histogram calculation area 902 based on the object detection area information, and stores the calculated result in the histogram calculation area information. As an example, the histogram calculation area information includes the upper left point of the i-th histogram calculation area 902 as a point (Clx (i), Cly (i)) and the lower right point as a point (Cryx (i), Cry (i )), The histogram calculation area information CalcArea [i] = (Clx (i), Cly (i), Crx (i), Cry (i)). For example, the upper left point of the image 901 is set to the origin (the origin (so that the region size of the object detection region 903 represented by the object detection region information OBJINFO [i]) becomes a histogram calculation region 902 with a region size of a predetermined magnification α. According to the equation (14) with X = 0, Y = 0), the upper left point (Clx (i), Cly (i)) and the lower right point (Cryx (i), Cry (i)) of the histogram calculation region 902 ) It has become to so that.

  On the other hand, the feature extraction unit 105 extracts a feature image from the image obtained by the imaging unit 101a among the imaging units 101a and 101b (step S205).

  Here, a method by which the feature extraction unit 105 extracts feature images will be described.

Since the feature extraction unit 105 performs a ternarization process on the image obtained by the imaging unit 101a among the imaging units 101a and 101b, for example, the feature extraction unit 105 performs a filtering process based on the filter matrix shown in Expression (15). I do.
(Equation 15)
[-1 0 1] (15)
The feature extraction unit 105 sets the pixel value to −1 if the pixel of the image after the filtering process is less than the predetermined ternary zero lower limit threshold, and is equal to or higher than the ternary zero lower limit threshold. A characteristic image is extracted by converting the pixel value to 0 if the pixel value is equal to or less than the maximization zero upper limit threshold and to 1 if it is greater than the ternary zero upper limit threshold. Here, an example of the extracted feature image is shown in FIG.

  The reference feature storage unit 106 stores the reference feature image extracted by the feature extraction unit 105 (step S206).

  Here, a method in which the reference feature storage unit 106 stores the reference feature image will be described.

  The reference feature storage unit 106 stores a reference feature image that is a feature image extracted by the feature extraction unit 105 at the reference time when the reference height distribution H0 (U, V) is created by the object extraction unit 103. The reference time may be the time when it is determined that the object does not exist according to the above-described equation (13), and the reference time interval determined by the object extraction unit 103 from the time when the object region is extracted from the monitoring region. It may be the previous time.

  The area dividing means 107 divides the histogram calculation area determined by the histogram calculation area determining means 104 into a plurality of areas (step S207).

  Here, a method in which the region dividing unit 107 divides the histogram calculation region into a plurality of regions will be described with reference to FIG.

  The area dividing unit 107 is, for example, the width of a small area 702 divided by the object extracting unit 103, as in the histogram calculation unit area 904 in FIG. 9, and the scanning lines of the images obtained by the imaging units 101a and 101b. The histogram calculation area 902 is divided in the vertical direction. Note that the region dividing unit 107 may divide the histogram calculation region 902 at a predetermined interval in a direction parallel to the scanning lines of the images obtained by the imaging units 101a and 101b.

  The feature histogram creating unit 108a creates a feature histogram by counting the frequency of pixels including a predetermined component existing in each region divided by the region dividing unit 107 based on the feature image extracted by the feature extracting unit 105 ( Step S208).

  Here, a method in which the feature histogram creating unit 108a creates a feature histogram will be described.

  When the predetermined component is a pixel value of 1, the feature histogram creation unit 108a counts the frequency of the pixel having a pixel value of 1 in the feature image for each histogram calculation unit region 904, thereby obtaining positive feature histogram information. Get. As an example, the positive feature histogram information includes the positive feature histogram information Hist_p [i] = (hp (i, 0), where hp (i, 0),..., Hp (i, CntHist) is the positive feature histogram information. ,..., Hp (i, CntHist)). Here, the number of histogram calculation unit regions 904 is expressed as CntHist.

  Further, when the predetermined component is set to the pixel value −1, the feature histogram creation unit 108 a counts the frequency of the pixel whose feature image pixel value is −1 for each histogram calculation unit region 904, Get negative feature histogram information. As an example, the negative feature histogram information has negative feature histogram information Hist_m [i] = (hm (i, 0)) where hm (i, 0),..., Hm (i, CntHist) is the negative feature histogram class. ,..., Hm (i, CntHist)).

  Here, FIG. 10 shows an example of a positive feature histogram obtained from the positive feature histogram information and a negative feature histogram obtained from the negative feature histogram information.

  The reference feature histogram creating means 108b counts the frequency of pixels including a predetermined component existing for each area divided by the area dividing means 107 based on the reference feature image stored by the reference feature storage means 106, thereby calculating the reference feature. A histogram is created (step S209).

  Here, a method in which the reference feature histogram creating unit 108b creates a reference feature histogram will be described.

  When the predetermined component is the pixel value 1, the reference feature histogram creating unit 108b counts the frequency of the pixels whose pixel value of the reference feature image is 1 for each histogram calculation unit region 904, thereby obtaining the reference Obtain positive feature histogram information. As an example, the reference positive feature histogram information is the reference positive feature histogram information Hist0_p [i] = (hp0 (i , 0),..., Hp0 (i, CntHist)). Here, the number of histogram calculation unit regions 904 is expressed as CntHist.

  Further, when the predetermined component is the pixel value -1, the reference feature histogram creating unit 108b counts the frequency of the pixel whose pixel value of the reference feature image is -1 for each histogram calculation unit region 904. Thus, reference negative feature histogram information is obtained. As an example, the reference negative feature histogram information includes reference negative feature histogram information Hist0_m [i] = (hm0 (i), where hm0 (i, 0),. , 0),..., Hm0 (i, CntHist)).

  Here, FIG. 11 shows an example of the reference positive feature histogram obtained from the reference positive feature histogram information and the reference negative feature histogram obtained from the reference negative feature histogram information.

  The correlation value calculation unit 108c calculates a correlation value between the data related to the feature histogram created by the feature histogram creation unit 108a and the data related to the reference feature histogram created by the reference feature histogram creation unit 108b (step S210).

  Here, a method of calculating the correlation value by the correlation value calculating unit 108c will be described.

  The correlation value calculating unit 108c calculates a positive correlation value between the positive feature histogram information representing the data related to the feature histogram and the reference positive feature histogram information representing the data related to the reference feature histogram, and negative feature histogram information representing the data related to the feature histogram. And a negative correlation value with reference negative feature histogram information representing data relating to the reference feature histogram.

  Here, the procedure for calculating the positive correlation value will be described. First, the correlation value calculation means 108c calculates the average value AH_p [i] of the positive feature histogram information Hist_p [i] and the reference positive feature histogram information Hist0_p [i for all i = 1,..., CntObj according to the equation (16). ] AH0_p [i] is calculated.

  Next, the correlation value calculation means 108c calculates the positive correlation value Cor_p [i] by the equation (17).

  A procedure for calculating the negative correlation value will be described. The correlation value calculating unit 108c calculates the average value AH_m [i] of the negative feature histogram information Hist_m [i] and the average value AH0_m [i] of the reference negative feature histogram information Hist0_m [i] by Expression (18).

  Next, the correlation value calculation means 108c calculates the negative correlation value Cor_m [i] by the equation (19).

  The object determination unit 109 determines whether an object exists in the monitoring area based on the correlation value calculated by the correlation value calculation unit 108c (step S211).

  Here, a method by which the object determination unit 109 determines whether or not an object exists in the monitoring area will be described.

  The object determination unit 109 determines a determination threshold value for determining whether an object exists in the monitoring area. Next, when both the positive correlation value and the negative correlation value are larger than the determination threshold, the object determination unit 109 determines that no object exists in the object detection area represented by the object detection area information. Further, the object determination unit 109 determines that an object exists in the object detection area represented by the object detection area information when one of the positive correlation value and the negative correlation value is equal to or less than the determination threshold value.

  Here, when the object determination unit 109 determines that no object exists in the object detection area represented by the object detection area information, and when it determines that an object exists in the object detection area represented by the object detection area information. The positive correlation value and the negative correlation value will be described with reference to FIGS.

  When the object determination unit 109 determines that no object exists in the area represented by the object detection area information, the positive correlation value and the negative correlation value increase. For example, when the positive feature histogram information and the negative feature histogram information shown in FIG. 10 and the reference positive feature histogram information and the reference negative feature histogram information shown in FIG. 11 are used, the positive correlation value is 0.9 and the negative correlation value is 0. .9.

  On the other hand, when the object determination unit 109 determines that an object is present in the region represented by the object detection region information, the positive correlation value and the negative correlation value become low. For example, in FIG. 12, an object region 1203 is extracted by the object extraction unit 103 from an image 1201 obtained by imaging by the imaging unit 101a or the imaging unit 101b, and the histogram calculation region 1202 is displayed by the region dividing unit 107 as a histogram. It is assumed that the calculation unit area 1204 is divided. Here, examples of the positive feature histogram information and the negative feature histogram information are shown in FIG. An example of the reference positive feature histogram information and the reference negative feature histogram information is shown in FIG. At this time, when the positive feature histogram information and the negative feature histogram information shown in FIG. 13 and the reference positive feature histogram information and the reference negative feature histogram information shown in FIG. 14 are used, the positive correlation value is 0.4, and the negative correlation value is 0.1.

  According to such an object detection device according to the first embodiment of the present invention, the reference feature based on the data obtained from the image obtained at the time of reference and the data related to the feature histogram based on the image obtained by imaging. Histogram correlation calculation means 108 for calculating a correlation value with data relating to the histogram, and object determination means 109 for determining whether or not an object exists in the monitoring area based on the correlation value are provided, thereby detecting the detected object region. It is possible to accurately determine whether is a background part or an object.

  Further, even when a fluctuation in the vertical direction with respect to the scanning line of the image obtained by imaging due to the influence of wind or the like occurs in the imaging means 101a or 101b, in order to reduce the influence of this fluctuation on the feature histogram, False detection can be prevented.

  Furthermore, in order to reduce the influence of the fluctuation on the feature histogram even when a fluctuation in the direction parallel to the scanning line of the image obtained by imaging due to the wind or the like occurs in the imaging means 101a or 101b, False detection can be prevented.

(Second Embodiment)
Next, an object detection apparatus according to a second embodiment of the present invention is shown in FIG.

  Of the constituent elements constituting the object detection device according to the second embodiment of the present invention, the same constituent elements as those constituting the object detection device according to the first embodiment of the present invention are included. The same code | symbol is attached | subjected and each description is abbreviate | omitted.

  In FIG. 15, the object detection apparatus according to the second embodiment of the present invention includes a frame difference calculation unit 1501 that calculates a frame difference of an image and a processing region determination unit 1502 that determines an object region.

  The operation of the object detection apparatus configured as described above will be described.

  First, the same operations from step S201 to step S203 as in the first embodiment of the present invention are performed.

  Next, the frame difference calculation unit 1501 calculates a difference between a frame representing an image obtained by imaging by the imaging unit 101a and a frame a predetermined time before the frame is obtained.

  Here, a method in which the frame difference calculation unit 1501 calculates a frame difference will be described.

  The frame difference calculation unit 1501 stores the reference image captured by the imaging unit 101a at the reference time. The reference time may be the time when it is determined that the object does not exist according to the above-described equation (13), and the reference time interval determined by the object extraction unit 103 from the time when the object region is extracted from the monitoring region. It may be the previous time.

  Then, the frame difference calculation unit 1501 calculates a frame difference between the image obtained by the imaging unit 101a and the reference image, performs binarization processing with a predetermined difference threshold, and then performs labeling. By executing the above, the upper left point and the lower right point of the difference area are specified, and the specified result is stored in the difference area information. As an example, the difference area information includes the upper left point of the kth difference area as a point (ldx (k), ldy (k)) and the lower right point as a point (rdx (k), rdy (k)). Thus, the difference area information DiffArea [k] = (ldx (k), ldy (k), rdx (k), rdy (k)) (k = 1,..., CntDiff). Here, the number of difference areas is expressed as CntDiff.

  Next, the processing area determination unit 1502 determines an object area based on the frame difference area and the object area obtained from the frame difference.

  Here, a method in which the processing area determination unit 1502 determines the object area will be described.

  The processing area determination unit 1502 searches for overlap between the object detection area information and the difference area information, and sets the difference area information in the histogram calculation area information when there is difference area information overlapping with the object detection area information.

  Next, the same operations from step S205 to step S211 as in the first embodiment of the present invention are performed.

  As described above, according to the object detection device of the second exemplary embodiment of the present invention, since an area where an object is likely to exist is set as a histogram calculation area, an object is not included in the object area extracted from the monitoring area. When it exists, it can prevent erroneously determining that an object does not exist.

  As described above, the object detection device according to the present invention has a correlation value between data relating to a feature histogram based on an image obtained by imaging and data relating to a reference feature histogram based on an image obtained at the time of reference. And a histogram correlation calculation means for calculating the object and an object determination means for determining whether or not an object exists in the monitoring area based on the correlation value, so that the detected object area is a background part or an object. It has the effect of being able to accurately determine whether or not it is present, and is useful as an object detection device that detects an object by calculating three-dimensional information of a monitoring area using a plurality of cameras.

The block diagram of the object detection apparatus in the 1st Embodiment of this invention Flow chart for explaining the operation of the object detection apparatus according to the first embodiment of the present invention. Explanatory drawing of the method of calculating the three-dimensional information in the 1st Embodiment of this invention Explanatory drawing of the evaluation method of the similarity degree of the image in the 1st Embodiment of this invention The figure which shows the positional relationship of the imaging means coordinate system and real space coordinate system in the 1st Embodiment of this invention. Explanatory drawing of the method of coordinate-transforming from the imaging means coordinate system to the real space coordinate system in the 1st Embodiment of this invention Explanatory drawing of the method of extracting the object area | region in the 1st Embodiment of this invention The figure which shows an example of the characteristic image in the 1st Embodiment of this invention Explanatory drawing of the method of creating the histogram calculation area division and feature histogram in the first embodiment of the present invention The figure which shows an example of the characteristic histogram in the 1st Embodiment of this invention The figure which shows an example of the reference | standard feature histogram in the 1st Embodiment of this invention Explanatory drawing of the method of producing the characteristic histogram when it determines with an object existing in the object area | region in the 1st Embodiment of this invention The figure which shows an example of the characteristic histogram when it determines with an object existing in the object area | region in the 1st Embodiment of this invention The figure which shows an example of the reference | standard feature histogram when it determines with an object existing in the object area | region in the 1st Embodiment of this invention The block diagram of the object detection apparatus in the 2nd Embodiment of this invention

Explanation of symbols

101a, 101b Imaging means 102 Three-dimensional information calculation means 103 Object extraction means 104 Histogram calculation area determination means 105 Feature extraction means 106 Reference feature storage means 107 Region division means 108 Histogram correlation calculation means 108a Feature histogram creation means 108b Reference feature histogram creation means 108c Correlation value calculation means 109 Object determination means 1501 Frame difference calculation means 1502 Processing region determination means

Claims (6)

A plurality of imaging means for imaging a predetermined monitoring area;
Three-dimensional information calculation means for calculating three-dimensional information representing the position of the monitoring region based on each image obtained by imaging each of the plurality of imaging means;
Object extraction means for extracting an object region where an object exists from the monitoring region based on the three-dimensional information calculated by the three-dimensional information calculation means;
Histogram calculation area determination means for determining a histogram calculation area for calculating a histogram based on the object area extracted by the object extraction means;
Feature extraction means for extracting a feature image from an image obtained by one of the plurality of imaging means;
Reference feature storage means for storing a reference feature image extracted by the feature extraction means;
Area dividing means for dividing the histogram calculation area determined by the histogram calculation area determining means into a plurality of areas;
Histogram correlation calculation for calculating a correlation value between the data relating to the feature histogram based on the feature image extracted by the feature extraction means and the data relating to the reference feature histogram based on the reference feature image stored by the reference feature storage means Means,
An object detection apparatus comprising: an object determination unit that determines whether or not an object exists in the monitoring region based on the correlation value calculated by the histogram correlation calculation unit. Frame difference calculating means for calculating a difference between a frame representing an image obtained by one of the plurality of imaging means and a frame a predetermined time before the time when the frame was obtained;
The object detection apparatus according to claim 1, further comprising a processing area determination unit that determines a histogram calculation area based on a frame difference area obtained from the frame difference and the object area. An imaging step in which a plurality of imaging means image a predetermined monitoring area;
A three-dimensional information calculation step for calculating three-dimensional information representing the position of the monitoring region based on each image obtained by imaging each of the plurality of imaging means in the imaging step;
An object extraction step of extracting an object region where an object is present from the monitoring region based on the three-dimensional information calculated in the three-dimensional information calculation step;
A histogram calculation region determination step for determining a histogram calculation region for calculating a histogram based on the object region extracted in the object extraction step;
A feature extraction step of extracting a feature image from an image obtained by imaging by one of the plurality of imaging means in the imaging step;
A reference feature storage step of storing the reference feature image extracted in the feature extraction step;
A region dividing step for dividing the histogram calculation region determined in the histogram calculation region determination step into a plurality of regions;
Histogram correlation calculation for calculating a correlation value between the data related to the feature histogram based on the feature image extracted in the feature extraction step and the data related to the reference feature histogram based on the reference feature image stored in the reference feature storage step Steps,
An object detection method comprising: an object determination step of determining whether or not an object exists in the monitoring region based on the correlation value calculated in the histogram correlation calculation step. Frame difference calculation for calculating a difference between a frame representing an image obtained by imaging by one of a plurality of imaging means in the imaging step and a frame a predetermined time before the time when the frame was obtained Steps,
The object detection method according to claim 3, further comprising: a processing region determination step for determining a histogram calculation region based on a frame difference region obtained from the frame difference and the object region. 5. The object detection method according to claim 3, wherein the region dividing step divides the histogram calculation region in a direction perpendicular to a scanning line of the image obtained in the imaging step. 5. The object detection method according to claim 3, wherein the region dividing step divides the histogram calculation region in a direction parallel to a scanning line of the image obtained in the imaging step.