JP4633942B2 - Passing object counting method and counting apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a passing object counting method and a counting apparatus, and more particularly to a passing object counting method and a counting apparatus for counting passing objects from a spatiotemporal image including information on the height of a passing object.
[0002]
[Prior art]
As a method of counting passing objects, a method using a planar spatio-temporal image has been conventionally known. That is, first, height information and velocity information on the counting line are calculated from stereo images of passing objects obtained by the two imaging devices to obtain three-dimensional information. Next, the obtained three-dimensional information is cut into circles at a certain height, and a planar spatio-temporal image is created when passing objects are arranged in the time direction for a certain time. Then, using the created planar spatio-temporal image, passing objects are counted depending on whether or not the object has an area of a certain width or more and a time at a certain height. In addition, by changing the above-mentioned fixed height, it counts whether it is an object having a certain width x time area or more, and repeats this within a fixed height range to count passing objects. It was.
[0003]
[Problems to be solved by the invention]
The conventional passing object counting apparatus as described above has the following problems. In the method of counting by a plane time-space image for a fixed time and determining whether it is an object having a certain width or more width and time area at a fixed height, the counting becomes inaccurate. In addition, by changing the height and counting whether or not the object has an area of a certain width or more and a time area, and repeating this within a certain height range, the counting is accurate but processed. The amount of data that should have been enormous, and the calculation took time.
[0004]
Accordingly, an object of the present invention is to provide a passing object counting device and a counting method capable of reducing the amount of calculation and performing high-speed processing while maintaining high counting accuracy.
[0005]
[Means for Solving the Problems]
To achieve the above object, the passing object counting method according to the first aspect of the present invention counts passing objects 5 passing through the reference line 1 on the plane 2 along the plane 2 as shown in FIG. In the passing object counting method: a spatio-temporal image of the passing object 5 on the reference line 1, and the small areas of the passing object 5 from the plane 2 are arranged in a plurality of regularly arranged small areas on the spatio-temporal image. An image forming step S0 for forming a spatio-temporal image to which the height information of a portion corresponding to is assigned; and a maximum extraction step S1 for extracting a small region having a maximum height from the plurality of small regions; A collecting region determining step S2 for obtaining a collecting region that is a small region surrounding the small region and that is a collection of small regions within a certain height range from the maximum height; and the collecting region obtained in the collecting region determining step S2 The ratio of the area to a predetermined threshold A comparison step S4 for; the area of the set region and a summing step S5, performing counting in one count when the larger becomes than the threshold value. The aggregate area is not limited to 1 and may be plural.
[0006]
If comprised in this way, since it has image formation process S0, local maximum extraction process S1, and gathering area | region determination process S2, the spatio-temporal image to which height information was allocated can be acquired, and from the acquired spatio-temporal image A small region having a maximum height is extracted, and a set region obtained by collecting small regions within a certain height range from the maximum height can be obtained. Further, since the comparison step S4 and the addition step S5 are provided, the area of the obtained collective region is compared with a predetermined threshold value, and 1 count can be performed when the area exceeds the threshold value.
[0007]
Further, as described in claim 2, in the passing object counting method according to claim 1, a step of acquiring velocity information in a traveling direction that intersects the reference line and is parallel to the plane in correspondence with the height information. S24; and a step S7 of weighting the area of the collective region with the speed information may be provided. The direction of speed includes a positive direction and a negative direction for the same speed direction.
[0008]
If comprised in this way, since it has the process of acquiring speed information and the process of weighting the area of a collection area with speed information, the speed of a passing object can be acquired, and the collection area weighted with the acquired speed Can be obtained.
[0009]
Further, as described in claim 3, in the passing object counting method according to claim 1 or 2, attention is paid to one small area of the plurality of small areas, and the small area of interest is based on height information. Is compared with the height of the attention subregion, and when the height of the attention subregion is higher than or equal to any height of the surrounding subregion, the attention subregion is It is good to make it a maximal small region.
[0010]
Further, according to a fourth aspect of the present invention, in the passing object counting method according to the third aspect, the plurality of small regions are arranged in a grid pattern, and the small region that surrounds the four small regions in the vicinity of the small region of interest is the surrounding small region. It is good to be an area. For example, as shown in FIG. 5A, the small region in the vicinity of 4 is a small region of interest P.₂₂4 small areas P on the top, bottom, left and right₁₂, P₃₂, P₂₁, P₂₃Say. If comprised in this way, the small area | region of maximum height can be easily obtained from a spatiotemporal image.
[0011]
Further, as described in claim 5, in the passing object counting method according to claim 4, the attention small region P₂₂Small region P in the vicinity of 8₁₁, P₁₂, P₁₃, P₂₁, P₂₃, P₃₁, P₃₂, P₃₃(For example, see FIG. 5A), a small region within a certain height range from the maximum height is set as a partial region belonging to the maximum small region, and each of the small regions of the contour of the partial region Of the eight subregions in the vicinity, the partial region after repeating a predetermined number of operations for adding a small region within a certain height range from the maximum height to the partial region is determined as the collective region. Step S2 may be provided (see, for example, FIG. 5B).
[0012]
In order to achieve the above object, the passing object counting apparatus 10 according to the sixth aspect of the present invention passes the reference line 1 on the plane 2 along the plane 2 as shown in FIGS. 1 and 3B, for example. In the passing object counting device 10 that counts the passing object 5 that passes through; the spatio-temporal image of the passing object 5 on the reference line 1 and the heights of portions corresponding to a plurality of regularly arranged small regions on the spatio-temporal image Imaging devices 11 and 12 for acquiring information; a maximum extraction means 30 for extracting a small region having a maximum height from the plurality of small regions; a small region surrounding the extracted small region, A set area determining means 31 for obtaining a set area obtained by collecting small areas within a certain range from the maximum height; and a comparison for comparing the area of the set area obtained by the set area determining means 31 with a predetermined threshold value Means 32; surface of the gathering area There an adding means 32 for counting the one count when the larger becomes than the threshold value. Typically, the spatiotemporal image is acquired from the imaging devices 11 and 12 via the processing device 20. That is, the processing device creates a spatiotemporal image from the imaging signals from the imaging devices 11 and 12.
[0013]
For example, as shown in FIG. 12, the surface on which the passing object 5 moves is not limited to a horizontal plane, but may be a surface 2 'inclined like a slope or a staircase. However, for example, as shown in FIG. 12A, height information and speed information are corrected according to the inclination angle as necessary.
[0014]
The reference line 1 that intersects the traveling direction of the passing object 5 is preferably a line perpendicular to the traveling direction of the passing object 5. The depth direction is typically a direction perpendicular to a plane or a vertical direction. For the same traveling direction, the traveling direction has a positive direction and a negative direction.
[0015]
If comprised in this way, since the imaging devices 11 and 12 and the maximum extraction means 30 are provided, a spatiotemporal image will be acquired using the imaging devices 11 and 12, and the maximum height of the passing object 5 on the spatiotemporal image will be obtained. A small region having a certain height maximum can be extracted. In addition, since the collection area determination means 31 and the addition means 32 are provided, a collection area obtained by collecting small areas in a certain height range from the maximum height is obtained, and the area of the collection area is larger than the threshold value One count can be performed.
[0016]
According to a seventh aspect of the present invention, in the passing object counting device according to the sixth aspect of the present invention, the imaging device is an imaging in which a plurality of pixels corresponding to the small region forming the spatiotemporal image are arranged. It is preferable to have an element.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol or a similar code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted.
[0018]
FIG. 1 is a schematic perspective view of a passing object counting device 10 (hereinafter referred to as “counting device 10” as appropriate) according to an embodiment of the present invention. In the figure, the passing object 5 moves in the traveling direction on a horizontal passage 2 which is a plane. An orthogonal coordinate system XYZ is taken so that the XY axis is placed in the passage 2, and the passage 2 extends in parallel with the Y axis. Further, a counting line 1 which is a reference line is taken perpendicular to the extending direction of the passage 2, that is, parallel to the X axis. The traveling direction of the passing object 5 is typically parallel to the extending direction of the passage 2, that is, the Y axis. The passing object 5 is a person in the present embodiment. In the present embodiment, the counting line 1 is perpendicular to the extending direction of the passage 2, but it does not have to be particularly perpendicular. If it is not vertical, the parallax and the speed may be corrected by the angle with the X axis. The counting line 1 is a reference line for counting the passing objects 5 that have passed through this line, and may be a virtual line, and does not need to be physically drawn in the path 2.
[0019]
On the other hand, in order to image the passing object 5 passing through the counting line 1 of the passage 2 above the counting line 1 of the passage 2 in the figure in a vertical direction above the counting line 1 of the passage 2 at a height sufficient to pass the person 5 as a passing object. The first imaging device 11 and the second imaging device 12 are installed. The first imaging device 11 and the second imaging device 12 are installed in parallel at an appropriate interval so that they are parallel to the counting line 1 and the optical axis is parallel to the Z axis. The height sufficient for the person 5 to pass here is, for example, about 3000 mm. Further, the appropriate interval is, for example, about 50 mm.
[0020]
An example of the configuration of the passing object counting device 10 will be described with reference to FIG. The counting device 10 includes a first CCD camera 11 as a first imaging device, a second CCD camera 12 as a second imaging device, and a personal computer 13 as an arithmetic device.
[0021]
The first CCD camera 11 and the second CCD camera 12 are connected to a personal computer 13 so that the personal computer 13 can acquire images captured by the first CCD camera 11 and the second CCD camera 12, respectively. It is configured.
[0022]
The personal computer 13 includes a control unit 14 and controls the counting device 10 as a whole. An imaging device interface 15 is connected to the control unit 14, and the first CCD camera 11 and the second CCD camera 12 are connected to the control unit 14 via the imaging device interface 15 and controlled. Yes.
[0023]
In the control unit 14, the height of the passing object 5, which is the height information of the passing object 5, is acquired in time series from the stereo images acquired by the first CCD camera 11 and the second CCD camera 12. A comparison calculation unit 20 that creates a spatiotemporal image and a counting calculation unit 21 that counts the passing object 5 from the created spatiotemporal image of the passing object 5 are provided. The comparison calculation unit 20 may acquire the speed of the passing object 5 that is the speed information of the passing object 5 in time series in association with the height information when acquiring the height information.
[0024]
A storage unit 18 is connected to the control unit 14 and can store data such as images captured by the first CCD camera 11 and the second CCD camera 12 and calculated information.
[0025]
Further, an input device 16 for inputting information for operating the counting device 10 and an output device 17 for outputting a result processed by the counting device 10 are connected to the control unit 14. The input device 16 is, for example, a keyboard or a mouse, and the output device 17 is, for example, a display or a printer. Although the input device 16 and the output device 17 are illustrated as externally attached to the personal computer 13 in this figure, they may be built in.
[0026]
Further, as shown in FIG. 3, an image processing unit 25 that processes an image captured by the CCD camera, an image captured by the first CCD camera, and an image captured by the second CCD camera are included in the comparison operation unit 20. Corresponding point search unit 26 for obtaining corresponding points from the obtained image, height information calculating unit 27 for calculating the height that is the height information of the passing object 5, and speed information calculation for calculating the speed that is the speed information of the passing object 5. The unit 28 includes a spatiotemporal image creation unit 29 that creates a spatiotemporal image by arranging the calculated height information in the time direction (FIG. 3A). Processing the image includes, for example, acquiring the first image and the second image from the imaging signals from the first CCD camera 11 and the second CCD camera 12. In this embodiment, a spatiotemporal image obtained from an image picked up by a CCD camera is represented by a plurality of pixels, which are a plurality of small regions, arranged in a grid pattern. Here, the pixels on the spatio-temporal image do not need to correspond one-to-one with the pixels on the imaging surface of the CCD camera. For example, the former one pixel may correspond to the latter four pixels.
[0027]
Further, in the count calculation unit 21, a maximum extraction unit 30 that extracts a pixel having a maximum height, which is a small region having a maximum height, from the created spatiotemporal image, and a pixel surrounding the maximum height region is used to determine the maximum height. A region calculation unit 31 for obtaining a collection region that is a set of pixels in a certain height range, and a counting unit 32 that compares the area of the collection region with a threshold and counts the passing object 5 when the area is larger than the threshold. It has.
[0028]
With reference to the flow of FIG. 4, an embodiment of the passing object counting method of the present invention will be described as an operation of the counting device 10 according to the embodiment. First, the comparison operation unit 20 in the control unit 14 of the personal computer 13 performs spatiotemporal image formation as an image formation process (step 0 (displayed as “S0” in the figure, the same applies hereinafter)). As a result, a spatio-temporal image represented by a plurality of pixels having the height information of the passing object 5 is created from the imaging signals from the imaging devices 11 and 12. In the present embodiment, this height information is expressed by luminance. This luminance is represented by a luminance value of 0 to 255 corresponding to a height of about 0 to 185 cm. The spatiotemporal image may be acquired in any way, but a method of acquiring the spatiotemporal image described with reference to FIG. 4 described later may be used.
[0029]
Next, the count calculation unit 21 acquires a spatiotemporal image from the comparison calculation unit 20, and the maximum extraction unit 30 extracts a maximum pixel that is a pixel having a maximum height, that is, a maximum luminance value on the spatiotemporal image. The maximum pixel extraction is performed as an extraction process (step 1). For example, as shown in FIG. 5 (a), the extraction method starts with one pixel P on the space-time image.₂₂Pay attention to. Pixel of interest P₂₂When all of the four neighboring pixels are less than or equal to the luminance value of the target pixel (including the case where they are equal), the target pixel is determined to be the maximum pixel. The 4 neighborhoods of the target pixel are, for example, the upper P of the target pixel.₁₂(Upper in the figure) Lower P₃₂(Bottom) Right P₂₃(Right in the figure) Left P₂₁This refers to the four pixels (left in the figure). Note that a plurality of maximum pixels may exist on one spatiotemporal image, and the plurality of maximum pixels may be adjacent or close to each other, or may be scattered apart from each other.
[0030]
Then, the extracted local maximum pixel is subjected to expansion processing as a collective region determination step by the region calculating unit 31 (step 2) to obtain a collective region. When there are a plurality of maximum pixels on one spatiotemporal image, the same number of collection regions as the maximum pixels are obtained. In particular, when a plurality of maximum pixels are adjacent or close to each other, a plurality of collection regions obtained from these maximum pixels may be set as one collection region.
[0031]
Here, the expansion process will be described with reference to FIG. First, one extracted maximum pixel P₂₂Pixel P in the vicinity of₁₁, P₁₂, P₁₃, P₂₁, P₂₃, P₃₁, P₃₂, P₃₃If each pixel of interest near 8 is within a certain range of the luminance value of the maximum pixel, this pixel is included in the partial region (P₁₁~ P₃₃(Not shown in (b)). The certain range here may be determined according to the characteristics of the passing object to be counted. However, when the target is a person, it is within 10% of the luminance value of the maximum pixel (the luminance value of the maximum pixel is 100%). In this case, it is desirable that the range be 90% or more. Further, if the average value of the luminance values of the eight neighboring pixels is within a certain range of the luminance values of the maximum pixels, all the eight neighboring pixels may be included in the partial area.
[0032]
Next, attention is paid to 8 neighborhoods of each pixel of the contour of this partial area (for example, the pixel P in FIG. 5A).₃₃Pixel P₂₂, P₂₃, P₂₄, P₃₂, P₃₄, P₄₂, P₄₃, P₄₄) If each pixel of interest in the vicinity of 8 is within a certain range of the luminance value of the maximum pixel in this partial area, this pixel is included in the partial area. This process is repeated a predetermined number of times. The predetermined number of times is typically about 50, but may be appropriately determined according to conditions. Increasing the number of times converges to a certain area. After convergence, the area does not change even if it is repeated more times.
[0033]
By this expansion processing, a partial region including pixels within a certain range of the luminance value of the maximum pixel can be obtained as a collective region. In addition, the maximum pixel in this process is a pixel having a maximum luminance value in one collective region (in some cases, a plurality of pixels are present).
[0034]
As shown in FIG. 6, the above expansion processing is processing for obtaining a region (aggregation region) from the maximum height of the plurality of passing objects 5 present on the spatiotemporal image to a certain height. Since the obtained spatio-temporal image is a grayscale image in which the height of the passing object is expressed by luminance, the region of luminance within a certain value can be the maximum cross-sectional area of the passing object up to a certain height. . That is, the expansion process is a process for extracting a cross section of the head portion of a person who is a passing object from the obtained spatiotemporal image. By extracting the cross section of the head portion of the person, for example, even when two persons 5 pass very close to each other, the two persons can be distinguished by obtaining the cross section of the head.
[0035]
Returning to the flowchart of FIG. Next, the collection area obtained by the above-described expansion process is labeled (step 3). Labeling refers to attaching, for example, a number or a code for identifying each region. Then, the area value of the collective region labeled as the comparison step is compared with a preset threshold value (step 4). If the area value is larger than the threshold value (step 4 is Y), one person 5 passes. As a result, counting as an adding step is performed (step 5). Also, if the aggregate area is an aggregate area that has already been counted, it is not counted. At this time, the traveling direction or speed of the area may be obtained in advance, and the traveling direction of the person 5 that has passed may be distinguished according to the traveling direction and reflected in the count. If the area value is smaller than the threshold (step 4 is N), it is determined that the person is not a person 5 and is not counted (step 6).
[0036]
In the above embodiment, the case where the speed information is not acquired has been described. However, this method can be used when the moving speed is almost constant in one way such as an amusement park entrance. In this case, the amount of calculation is small compared with the method of acquiring the speed information, so that high-speed processing is possible.
[0037]
Also, the velocity of the passing object is calculated from the area value of the collective area obtained by the expansion process, and the area value is weighted to the area value of the collective area at a speed corresponding to the labeled collective area described above. It may be calculated (step 7 (indicated by a broken line in the figure)). The reason why the area value is weighted by speed is that, for example, for a person with a high passing speed, the passing speed and the number of passing frames are inversely proportional to each other. A spatio-temporal image generally means an image including a time axis in at least one dimension. Here, an image in which the time axis disappears as a result of weighting the spatio-temporal image with speed is also referred to as a spatio-temporal image for convenience.
[0038]
Furthermore, since there are positive and negative directions of travel, when counting, give the sign of the sign of the direction of travel of the passing object to the speed information that weights the area value of the gathering area. Thus, the direction of travel of the passing object can be distinguished and counted according to the sign of the area value. As a speed information calculation method, for example, a method of acquiring speed information described later with reference to FIG. 9 may be used.
[0039]
In this way, by using the method of obtaining the speed information of the passing object, and calculating the area value by weighting the area value of the collective region at a speed corresponding to the collective region, for example, like a passage in a station premises, Even in the case of face-to-face traffic and the movement speed of each person being different, counting can be performed with high accuracy. It is desirable to use this method for actual counting.
[0040]
The preset threshold is typically the maximum cross-sectional area value of the head of the smallest person among the persons 5 to be counted. There is no significant difference in the cross-sectional area value of the head even when comparing an adult and a child.For example, if the cross-sectional area value of the child's head is set as a threshold, Both adults and children can accurately count without miscounting.
[0041]
In the above counting method, counting is performed if the area value of the collective region is larger than the threshold value. However, if the area value of the collective region is equal to or greater than a certain value, the count value may not be counted. That is, the maximum possible human cross-sectional area of an adult may be set as the second threshold, and the area value may be smaller than the second threshold. In this way, it is possible to prevent counting of passing objects (for example, luggage) having a large cross-sectional area, so that the counting accuracy can be further improved.
[0042]
An example of a method for acquiring a spatiotemporal image will be described with reference to the flow of FIG. Here, an image captured by the first CCD camera 11 is described as a left image, and an image captured by the second CCD camera is described as a right image. First, the control unit 14 of the personal computer 13 acquires the left image and the right image captured by the first CCD camera 11 and the second CCD camera 12 as a stereo image via the imaging device interface 15 (step 10).
[0043]
The control unit 14 acquires, from the acquired stereo image, the left image of the t frame that is the first image as an image captured at a certain time t by the image processing unit 25 in the control unit 14 (step 12). . Further, a right image of a t + Δt frame, which is a second image taken at time t + Δt after Δt has elapsed from time t, is acquired (step 13).
[0044]
Actually, the first CCD camera and the second CCD camera pick up images at each time point simultaneously and in parallel, and the time point t as the first image is shifted by one frame or more as the second image. You may make it employ | adopt the image of t + (DELTA) t. At this time, Δt may be appropriately determined according to the setting conditions of the counting device 10, but it is desirable that Δt be the time from when a certain frame N is imaged until the next frame N + 1 is imaged.
[0045]
Next, edge extraction of the left image of t frame is performed (step 14), and it thins (step 15). Then, the corresponding point search of the right image of the t + Δt frame is performed using the edge points on the counting line 1 of the left image of the t frame (step 16). Edge extraction, thinning, and corresponding point search will be described later with reference to FIG.
[0046]
Next, height information is calculated from the corresponding points obtained by the corresponding point search (step 17). Here, a non-background portion is defined in advance by background removal, and linear interpolation for reflecting the value at the edge point is performed on the portion (step 18). A method for calculating the height information will be described later with reference to FIGS.
[0047]
The obtained height information is arranged in the time direction to create a spatiotemporal image including the height information of the person 5 (step 19).
[0048]
Next, T is the elapsed time since the first frame was acquired.₁Is a preset time T₀Is judged (step 20). T₁Is the predetermined time T₀(Step 20 is Y), the predetermined time T₀Can be acquired (step 21). T₁Is the predetermined time T₀Is not passed (N in step 20), the next frame is acquired and the spatiotemporal images are arranged in the time direction. The determination in step 20 may be made when the number of acquired frames reaches a predetermined number of frames instead of the elapsed time. Predetermined time T₀For example, it is appropriate to set the time to about 20 seconds due to the limitation of the capacity of the storage device.
[0049]
In addition, when the speed of the passing object is required by the above-described counting method (for example, when the area of the gathering area is weighted by speed), it is obtained by the above-described corresponding point search as indicated by a broken line in the figure. The speed information is also calculated from the corresponding points (step 22). Then, linear interpolation is performed in the same manner as the height information (step 23). In this way, speed information is acquired (step 24). A method for calculating the speed information will be described later with reference to FIG.
[0050]
The corresponding point search will be described in more detail with reference to FIG. Here, although a person is expressed by hatching as a passing object in each illustrated frame, this does not indicate a cross section but merely represents a person imaged from above.
[0051]
Here, in order to obtain height information, corresponding points of two images are obtained. First, the correspondence between the left image of t frame and the right image of t + Δt frame is obtained. At this time, the correspondence relationship is not clearly determined for pixels having no feature in the neighboring region. Therefore, the search is performed according to the following flow.
[0052]
First, edge extraction of the left image of t frame is performed to obtain a thinned image, and corresponding point search is performed only at the edge point. Edge extraction refers to extracting a contour, which is a set of edge points of a target passing object, from an image acquired with shading and specifying the coordinates, for example, with coordinates. Thinning refers to, for example, converting an image obtained by edge extraction into an image in which only the outline of a passing object obtained by edge extraction is indicated by a thin line.
[0053]
In order to perform the corresponding point search, first, a point on the specified outline of the left image of the t frame is selected as an edge point, and a window is set in a neighborhood region including the edge point on the grayscale image. Typically, an edge point on the counting line is selected. An edge point is typically an intersection of the fine line and the count line, and usually there are a plurality of edge points, typically 10 to 20 points. For example, the leftmost point among the plurality of edge points is first selected. The window may be set so that the selected edge point is the center. Then, the window is set to the right image of the t + Δt frame with reference to the set window.
[0054]
With respect to these windows, the correlation between the gray values of the left image of t frame and the right image of t + Δt frame is obtained using Equation 1 shown in FIG. Where I_L(I, j) and I_R(I, j) are the gray values in the windows on the left and right images, and μ_{L ,} μ_{R ,} σ_{L ,} σ_RAre the mean and variance of the gray values in the left and right windows, respectively. C takes a value within a range from −1 to +1. The C value is calculated in each window while moving the window of the right image little by little with respect to the selected edge point of the left image, for example, parallel to the count line and moving in the direction orthogonal to the count line. The window of the right image that gives the maximum value among C obtained in this way is regarded as a local region corresponding to the selected window of the left image.
[0055]
That is, the left and right image windows are correlated as described above, and the right image window corresponding to the window including the edge point selected in the left image (C is maximized) is searched. The center of the window of the right image obtained by this search becomes the edge point of the right image corresponding to the edge point of the left image selected previously.
[0056]
The search for the window is performed, for example, in a direction parallel to the count line (X-axis direction) and a vertical direction (Y-axis direction). For example, when the passing object is a person, the direction parallel to the counting line is regarded as a parallax between the left and right images as will be described later. What is necessary is just to search the width | variety of parallax to be performed. Since the vertical direction is regarded as the movement amount of the person as will be described later, the width of the movement amount that the person is expected to move during the time Δt may be searched with reference to the window of the left image that has already been set. If the value of C is increasing in the direction of search, the value of C is further maximized, and the search is continued in the same search direction until the downward trend is reached. You may make it select the window where the value of C becomes the maximum in the already searched range.
[0057]
If the window search is performed in this way, it is not necessary to correlate the windows of the left and right images of the part that clearly does not need to be searched, so that the amount of calculation can be further reduced. As described above, the corresponding points are searched for all the edge points existing in the left image, and the corresponding edge points of the left and right images are obtained.
[0058]
The calculation of depth information and speed information will be described in more detail with reference to FIG. First, a vector connecting corresponding edge points of the left and right images obtained above is considered, and this is separated into a horizontal component (X-axis direction component) and a vertical component (Y-axis direction component). The horizontal component can be viewed as an amount representing the parallax between the left and right images, and the distance to the target passing object can be calculated using this parallax. On the other hand, the vertical component can be regarded as the amount of movement of the attention point in the direction along the path, and based on this, the speed information of the passing object can be obtained. In this way, the three-dimensional height that is the depth information at the edge point and the speed that is the speed information can be collectively obtained in pairs.
[0059]
Since the speeds obtained for one passing object should normally be equal, it is not necessary to obtain the speeds for all the edge points. For example, the speed of the point closest to the center may be adopted. Alternatively, the speed of each edge point is obtained, and an edge point having a protruding value may be re-searched as a search error, or the point may be rejected. Or you may obtain | require all the speed | velocity | rate of each edge point, calculate those average values, and may employ | adopt as the speed | velocity | rate of the object.
[0060]
In addition, for the above-described weighting by speed, the speed for each point may be used, or the speed of the passing object obtained by the average value may be used in common for each point.
[0061]
Here, the left and right images are images that are shifted in time by Δt. However, if Δt is a sufficiently small value, the left and right images do not have a difference enough to support parallax. Conversely, Δt may be determined as such. That is, the length is sufficient to obtain the speed information without any problem for obtaining the parallax. For example, when the target passing object is a pedestrian, it is set to about 1/60 to 1/10 seconds, particularly 1/40 to 1/20 seconds, and more preferably about 1/30 seconds.
[0062]
Here, with reference to FIG. 10, the calculation method of the height z from the plane as the height information of the target passing object will be described. The distance a to the target passing object 5 can be calculated using Expression 2 shown in FIG. Here, w is the distance between the CCD cameras (baseline length), f is the focal length of the CCD camera when the imaging lens is a single lens, and d is the parallax on the imaging surface of the CCD camera. The focal length here is the focal length of the combination lens when a commonly used combination lens is used. Further, if the distance from the imaging lens of the CCD camera to the counting line 1 is h, the height z of the target passing object 5 is calculated by Equation 3 shown in FIG.
[0063]
In the above embodiment, the height information and the speed information are obtained from the left image at the time point t and the right image at the time point t + Δt. However, the height information is acquired from the left image and the right image at the time point t, The speed information may be acquired from the left or right image of t + Δt.
[0064]
A case where the passing object 5 moves on the slope will be described with reference to FIG. For example, when the passing object 5 is moving on an inclined surface 2 ′ such as a staircase or a slope, the plane 2 is a virtual plane 2 perpendicular to the line connecting the reference line 1 and the first and second imaging devices 11 and 12. It is good to do.
[0065]
If the target passing object 5 is a person, it stands in the vertical direction even if it is a slope, so the optical axis of the CCD camera is directed in the vertical direction (FIG. 12A). In this case, the virtual plane 2 is a horizontal plane. In this example, for example, when the angle formed between the inclined surface 2 ′ and the virtual plane 2 is θ, the time of the passing object 5 moving on the inclined surface 2 ′ is obtained in order to obtain the height of the passing object 5 from the virtual plane 2. A value obtained by multiplying the amount of movement of Δt by tan θ may be subtracted from the calculated height of the passing object 5 moving on the slope 2 ′.
[0066]
If the target passing object 5 is, for example, a car, the slope may be used as it is as the plane 2 of the present invention. In this case, the optical axis of the CCD camera is directed in a direction perpendicular to the inclined surface (FIG. 12B).
[0067]
As described above, according to the embodiment of the present invention, compared with the conventional counting method of cutting a spatio-temporal image, for example, 20 sec (predetermined time T₀= 20 sec), the time for processing the spatio-temporal image can be shortened from 160 sec to 21 sec, so that the processing speed can be greatly increased.
[0068]
【The invention's effect】
As described above, according to the present invention, since the image forming step, the maximum extraction step, and the gathering region determination step are provided, the spatiotemporal image to which the height information is allocated can be acquired. It is possible to extract a small region with a maximum height from, and obtain a set region where a small region within a certain height range from the maximum height is gathered, and since it includes a comparison step and an addition step, Compare the area of the gathering area with a predetermined threshold value, and can count one count when it exceeds the threshold value. An object counting device and a counting method can be provided.
[Brief description of the drawings]
FIG. 1 is a schematic perspective block diagram showing an outline of a passing object counting apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration example of a passing object counting apparatus according to an embodiment of the present invention.
FIG. 3 is a block diagram showing a configuration example (a) of a comparison calculation unit and a configuration example (b) of a count calculation unit used in the embodiment of the present invention.
FIG. 4 is a flowchart showing an example of the operation of the passing object counting device according to the embodiment of the present invention or a passing object counting method according to the embodiment of the present invention.
FIG. 5 is an image diagram for explaining a processing example in the case of performing expansion processing in the embodiment of the present invention.
FIG. 6 is a diagram illustrating expansion processing according to the embodiment of the present invention.
FIG. 7 is a flowchart showing an example of a method for acquiring a spatiotemporal image or an example of an operation of a passing object counting device according to an embodiment of the present invention in an embodiment of the present invention.
FIG. 8 is an image diagram for explaining a processing example when searching for corresponding points between the first image and the second image in the embodiment of the present invention.
FIG. 9 is an image diagram for explaining a vector connecting corresponding points of the first image and the second image in the case of FIG. 8;
FIG. 10 is a stylized front view for explaining a method of calculating the height of a passing object from parallax between the first imaging device and the second imaging device in the embodiment of the present invention.
FIG. 11 is a diagram showing a calculation formula used in the description of the embodiment of the present invention.
FIGS. 12A and 12B are diagrams illustrating a case of counting by a virtual plane (a) and a case of using a slope as a plane of the present invention (b) as an example of a method of counting passing objects moving on an inclined surface.
[Explanation of symbols]
1 Count line
2 passage
5 passing objects
10 Passing object counting device
11 First CCD camera
12 Second CCD camera
13 Arithmetic unit
14 Control unit
15 Imaging device interface
16 input devices
17 Output device
18 Storage unit
20 Comparison operation part
21 Counting calculator
25 Image processing section
26 Corresponding point search unit
27 Height information calculator
28 Speed information calculator
29 Space-time image generator
30 maximum extraction part
31 Area calculation part
32 counter

Claims (6)

In a passing object counting method of counting passing objects passing a reference line on a plane along a plane;
A space-time image in the reference line of the passage body, a plurality of small regions which are regularly arranged on said time-space image, from the plane of the portion corresponding to the small region before Symbol passing object An image forming process for forming a spatio-temporal image to which height information is assigned;
A maximum extraction step of extracting a small region having a maximum height from the plurality of small regions;
A collective region determining step for obtaining a collective region that is a small region surrounding the extracted small regions and that is a collection of small regions within a certain height range from the maximum height;
A comparison step of comparing the area of the collection region determined in the collection region determination step with a predetermined first threshold;
An addition step of counting 1 count when the area of the aggregate region is greater than the first threshold;
Passing object counting method. Obtaining velocity information in a traveling direction that intersects the reference line and is parallel to the plane in correspondence with the height information;
Comprising the step of weighting by the speed information on the area of the set region;
The comparison step compares the area weighted in the weighting step with a predetermined second threshold value instead of the area of the collective region,
In the adding step, instead of the area of the collective region, a count of 1 count is performed when the weighted area is larger than the second threshold value.
The passing object counting method according to claim 1. Pay attention to one small area of the plurality of small areas, and based on the height information, compare the height of the small area surrounding the small area of interest with the height of the small area of interest, and When the height of the small region of interest is equal to or higher than the height, the small region of interest is the small region of the maximum,
The passing object counting method according to claim 1 or 2. The plurality of small areas are arranged in a grid pattern, and a small area in the vicinity of 4 of the target small area is defined as the surrounding small area.
The passing object counting method according to claim 3. Of the small regions in the vicinity of 8 of the target small region, a small region within a certain height range from the maximum height is set as a partial region belonging to the maximum small region, and each of the small regions of the outline of the partial region The partial area after repeating the operation of adding a small area within a certain height range from the maximum height to the partial area among the small areas in the vicinity of 8 is determined as the set area. Comprising the steps of:
The passing object counting method according to claim 4. In a passing object counting device for counting passing objects passing through a reference line on a plane along a plane;
An imaging device that acquires a spatiotemporal image of the passing object at the reference line and height information of portions corresponding to a plurality of regularly arranged small regions on the spatiotemporal image;
A maximum extraction means for extracting a small region having a maximum height from the plurality of small regions;
A collection area determination means for obtaining a collection area that is a collection of small areas that surround the extracted small area and that is within a certain height range from the maximum height;
Comparing means for comparing the area of the set area determined by the set area determining means with a predetermined threshold;
Adding means for counting one count when the area of the aggregate region is larger than the threshold;
Passing object counting device.

Images