JPH08185521A - Mobile object counter - Google Patents

Abstract

PURPOSE: To provide a mobile object counter which eliminates miss counting and improves accuracy. CONSTITUTION: Continuous frame image data inputted from a CCD camera 1 through an A/D converter 2 and a digital switch 2 are stored in frame memories 4, 5 and 6 and according to processing flow stored in a program memory 9, a CPU 8 extracts a mobile object area by calculating difference between the frames of image data while using pointer shift. Labelling is performed to this mobile object area by grouping using a pointer, and image data are separated for every independent mobile object area. Then, the number of mobile objects in the mobile object area is judged from the aspect ratio, area ratio and peripheral area ratio of the labelled mobile object area. Thus, the mobile object is extracted for every frame, mobile objects are made correspondent between the frames, the noncorrespondent mobile object is counted as an object moved out of the object area, and the counted result is outputted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving body counting device for counting moving bodies by image recognition.

[0002]

2. Description of the Related Art In recent years, research and development of a moving object counting device for automatically counting moving objects such as people and automobiles by image recognition has been advanced for railways, buses, convenience stores, department stores, streets and the like.

The reason for this is that, for example, in the case of trains and buses, operating schedules are greatly affected by the amount and fluctuations of passengers during peak hours. If the manager can grasp the passenger volume and fluctuations in detail, an operation plan corresponding to the passenger volume can be made and a stable transportation capacity can be secured. Furthermore, it is possible to provide appropriate operation information and services to passengers.
Therefore, if it is possible to automatically carry out passenger volume surveys that currently require manual labor, it will be more effective in terms of management and service, and such automatic counting of moving bodies is not possible by image recognition. It can be mentioned that it is optimal.

In the moving object counting apparatus based on such image recognition, the most effective method for extracting moving objects is the method based on the inter-frame difference of input images, which is resistant to environmental changes and is expected to be processed at high speed and with high accuracy. ing. The method based on the inter-frame difference of the input image is a method of inputting the input image of two frames or three frames into the storage device, calculating the difference between each frame, and determining the moving body region from the difference value.

Further, each extracted moving body area is labeled, the number of moving bodies in each labeled moving body area is determined, the moving bodies are extracted, and the extracted moving bodies are counted.

Therefore, the moving body counting apparatus realized by the above method, the data input means for picking up the image of the target area for counting the moving body and obtaining the image data, the storing means for storing the image data, and the storing means. The moving body area is extracted from the image data read out from the means by a method based on the inter-frame difference, each extracted moving body area is labeled, and the number of moving bodies in each labeled moving body area is determined. It is configured by an arithmetic processing unit that extracts a moving body, counts the extracted moving body, and outputs the count result.

The storage means is composed of a plurality of frame memories and a data memory for storing calculation results such as count results. The calculation processing means is a CPU and the CP.
It is composed of a program memory that stores the U arithmetic program.

Next, the image processing and the arithmetic processing by the arithmetic processing means and the storage means are carried out by extracting the moving body by the inter-frame difference, labeling, determining the number of moving bodies in the moving body area,
Divided into mobile counts. An example of each of these processes will be described below.

[Extraction of Moving Object by Difference between Frames] F (T-
2ΔT), F (T-ΔT), and F (T). Where Δ
T is the sampling interval of the image data, T-2Δ
T, T-ΔT, t represent sampling times.

The difference between F (T-2ΔT) and F (T-ΔT) and the difference between F (T-ΔT) and F (T) are calculated for each pixel, and the difference is determined by a threshold value t ₁ . Quantify.

Each of the binarized difference data is Fb.
If (T-2ΔT, T-ΔT) and Fb (T-ΔT, T), then Fb (T-2ΔT, T-ΔT) = [| F (T-2ΔT)
−F (T−ΔT) |] t ₁ Fb (T−ΔT, T) = [| F (T−ΔT) −F (T)
|] T ₁ . Here, [] t ₁ indicates a binarization operation in which if the value in [] is t ₁ or more, it is “1”, and if it is less than t ₁ , it is “0”.

Next, in order to eliminate the duplication of the mobile unit, Fb
The logical product of (T-2 [Delta] T, T- [Delta] T) and Fb (T- [Delta] T, T) is obtained, and this is set as interframe difference data H (T- [Delta] T) at time T- [Delta] T.

H (T-ΔT) = Fb (T-2ΔT, T-
ΔT) * Fb (T−ΔT, T) Here, * indicates a logical product operation.

By the above processing, the background area is removed and only the moving body area is extracted. H (T-ΔT) is data in which bits are set only in the moving body area.

Next, the data shift operation of the data in the frame memory in the above processing will be described with reference to FIG. In FIG. 33, the extraction of the moving object based on the difference between the frames in step C from the input of the image data in step A is a round unit.

Before entering the first round, the image data 1 is first loaded into the data area 1 of the frame memory, and then the image data 2 is loaded into the data area 2 after the sampling interval ΔT. The first round starts when the image data 3 is taken into the data area 3 after ΔT. The data shift for one round and the accompanying arithmetic processing by the CPU are performed during ΔT. As shown in the figure, image data 1 is data a, image data 2 is data b, and image data 3 is data c.

In the first round, the image data 3 is loaded into the data area 3 in step A (data c), and the binarized difference data (corresponding to Fb above) between the data a and the data b is acquired in step B. The data is written in the data area 1 (data d). Similarly, the binary difference data of the data b and the data c is written in the data area 2 (data e). The data c is stored in the data area 3 as it is. The inter-frame difference data (corresponding to H (T-ΔT) above) obtained by performing the logical product processing of the data d and the data e in step C is written in the data area 1 (referred to as data f), and labeling is performed using this f. Then, the feature amount for association and counting is extracted. The data e and the data c are stored in the data area 2 and the data area 3 as they are.

In the second round, the image data 4 sampled after ΔT of the image data 3 in step A '.
Are taken into the data area 3 (data c '). The data e in the data area 2 is shifted to the data area 1 (data d '), and the data c in the data area 3 is shifted to the data area 2 (data b'). Step B '
Then, the binary difference data of the data b'and the data c'is written in the data area 3. Data c'and data d '
Is saved as is. Step B of the first round below
And the same processing as step C is performed.

After the third round, the same data shift as in the second round is performed, and the sampled image data is always input to the data area 3.

[Labeling] Using the binarized inter-frame difference data H (T-ΔT), the same label is applied to the connected pixels for each pixel with a bit, and each independent effective By giving one label to the area, the extracted effective area is separated into independent areas.

In the labeling process, the binarized image data is scanned in the raster direction from the upper left of the screen, a pixel with a bit is detected, and the same label is given to the base and the pixel connected to the base based on this pixel. . Then, the base is changed one after another by scanning in the raster direction, and the above processing is repeated.

34 to 36 are flowcharts of the labeling process. 35 shows the contents of the subroutine 1 of the step C of FIG. 34, and FIG. 36 shows the contents of the subroutine 2 of the step I of FIG.

In FIG. 34, the number of pixels in one frame is 2
56 x 256. In addition, the difference data between frames H (T
It is assumed that −ΔT) is binarized with the effective area being “1”. In step A, XMAX and YMAX indicate the numbers of pixels in the x direction and the y direction, respectively, and 256 is substituted in each case. HIGH indicates the value of the effective area, and "1" is substituted here. Variable la
“bel” indicates a label number, and “1” is substituted as an initial value.

In step B, IMG (X, Y) is X (takes a value of 0 to XMAX-1) and Y (0 to YM).
The pixel data corresponding to the inter-frame difference data H (T-ΔT) is read using the value of AX-1) as a variable.
It is a three-dimensional array. Each pixel of IMG (X, Y) is sequentially scanned in the raster direction, and if the value of IMG (X, Y) is not HIGH, the process returns to step B via step E to process the next pixel data. The value of IMG (X, Y) is HI
When the GH pixel is found, the process proceeds to step C, and FIG.
Go to subroutine 1 indicated by 5.

In step F of FIG. 35, labeling is performed by substituting label for IMG (X, Y), and this pixel is used as a base.

In step G, the variable CNT indicating the presence or absence of the pixel connected to the base is substituted by "0" and reset, and in step H, the two-dimensional array IMG is set to I (0 to XMAX-).
1)) and J (0-YMAX-1) are used as variables to scan again in the raster direction, and if there is a pixel having the same label as the base, the process proceeds to the subroutine 2 shown in FIG. 36 in step I. .

In step L of FIG. 36, 8 neighboring pixels of pixel IMG (I, J) having the same label as the base, that is, M
And IM are variables with values of -1 to 1
The value of G (I + M, J + N) is checked, and if the value is HIGH, label is assigned to that pixel in step M, and C
Increment NT (CNT = CNT + 1).

After the above processing is performed for the IMG (I + M, J + N) of 8 pixels, the process returns to step I of FIG. 35 at step N.

In FIG. 35, when the processes of steps H and I have been completed for all IMG (I, J), the value of CNT is checked in step K via step J, and CNT ≠
If it is 0, the process returns to step G, CNT is reset, and the processes of steps H and I are repeated. If CNT = 0, it means that there are no connected pixels to be labeled to the base. Therefore, step D in FIG. Return to.

In step D of FIG. 34, label is incremented (label = label + 1), the process returns to step B through step E, a new base is set, and the processes of steps C and D are repeated.

The processes of steps B to D are performed for all IMG (X, Y), and the labeling process is completed.

In the above labeling process, scanning for base setting and scanning for connecting pixel search to the base (L number of scannings when the total number of labels is L), that is, 1 + L scannings are performed. Further, when the moving body region has a complicated shape, scanning for connecting pixel search for one base is performed plural times.

FIG. 37 shows an example of the labeling process when the moving body region has a complicated shape. FIG.
7 (a) is an example of the moving body area. The portion described as H is the extracted moving body area, and the pixel marked with a circle is the base. With respect to this base, Figs. 37 (b) to (e)
Four scans shown in are required.

FIG. 38 is an explanatory diagram of the number of scans for searching a connected pixel for one base. Since the scanning is performed in the raster direction, labeling can be performed by one scanning with respect to the connection of three or more pixels in the solid line direction shown in FIG. 38 (a) (the connection of the circle in FIG. 38 (b)). Is a connection of three or more pixels in the direction of the broken line shown in FIG. 38A (x in FIG. 38B).
Multiple scans are required for the (linking of marks).

After all, the number of scans at the label k is S _k (k
= 1, 2 ... L), the total number of scans is 1+ (S ₁ + S ₂ + ... + S _k + ... + S _L ).

[Determination of Number of Moving Objects in Moving Object Area] The number of moving objects in each labeled moving object area is determined.
When the two moving bodies are integrated (that is, the moving body areas of the two moving bodies are connected), the two moving bodies are labeled as one moving body area, and thus this determination is performed.

The number of individuals in each moving body area is determined by the aspect ratio of each moving body area. This is because when two moving bodies are integrated in the horizontal direction (direction perpendicular to the moving direction), the shape of the moving body region becomes horizontally long as compared with the case of one moving body. The vertical direction (moving direction) may be integrated, but if the target region is limited to a region that is long in the horizontal direction and short in the vertical direction, integration in the vertical direction does not pose a problem.

The aspect ratio is obtained as follows. Consider a circumscribing rectangle surrounded by two circumscribing lines in the moving direction of the moving object region and two circumscribing lines in a direction perpendicular to the moving direction, and let L be the length of the side of the circumscribing rectangle in the moving direction and the vertical direction.
_The aspect ratio is calculated by the following equation, where _{s is} the length of the side in the moving direction and L _i .

Aspect ratio = L _s / L _i and a threshold value t ₂ are provided. If the aspect ratio is t ₂ or more, it is determined that the number of individuals is 1, and if the aspect ratio is less than t ₂ , the number of individuals is 2. Judge as one.

[Counting of Moving Objects] Finally, moving objects in each moving object area are counted.

[0041]

However, the above-mentioned conventional moving body counting apparatus is required to process the input image data in real time on a frame-by-frame basis, and since the amount of data is enormous, it is limited to the arithmetic processing capability of the CPU. Therefore, there is a problem that miscounting occurs because the sampling interval of the image data cannot be shortened sufficiently, and there is also a problem that miscounting occurs due to an erroneous determination of the number of solids in the moving body area It is the current situation that it has not been realized.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a highly accurate moving body counting apparatus capable of performing accurate counting.

[0043]

In order to achieve the above object, a moving object counting apparatus according to a first aspect of the present invention stores data input means for obtaining continuous frame image data and image data input by the input means. While exchanging data with the storage means and the storage means, a moving object area is extracted by obtaining a difference between frames of the image data by pointer shift, and labeling is performed on the moving object area by grouping using a pointer. And separate each moving body region and extract the moving body for each frame by judging the number of moving bodies in the moving body region from the aspect ratio, area ratio and peripheral surface ratio of the labeled moving body region. , The relevant moving body is associated between frames, and the moving body that does not correspond is regarded as having moved out of the target area. Collected by, those comprising an arithmetic processing means for outputting the count result.

According to a second aspect of the present invention, there is provided a moving object counting apparatus which comprises a data input means for obtaining continuous frame image data, a storage means for storing the image data input by the input means, and data exchange with the storage means. However, the image data is spatially differentiated, a characteristic portion of the moving body is extracted by taking an inter-frame difference of the spatially differentiated image data, and labeling is performed on the characteristic portion for each frame. A moving object is extracted, the moving object is associated with each other between frames, the moving object that does not correspond is counted as having moved out of the target area, and the calculation result is output. To do.

[0045]

Therefore, in the moving object counting apparatus according to the first aspect of the present invention, the storage means stores the continuous frame image data inputted from the data input means, and the arithmetic processing means obtains the inter-frame difference of the image data by pointer shift. By extracting the moving body area by this, labeling is performed on the moving body area by grouping using a pointer to separate each moving body area, and the aspect ratio, area ratio and peripheral surface of the labeled moving body area are separated. The moving objects are extracted for each frame by determining the number of moving objects in the moving object region from the ratio, the moving objects are associated between frames, and the unmoving moving objects are counted. Is output, and the interframe difference is calculated by the pointer shift without data shift in the storage means. By speeding up the moving body area extraction processing and labeling processing by performing the labeling by grouping using the pointer and reducing the number of scanning at the time of labeling, there are multiple judgment items when judging the number of moving bodies in the moving body area. Further, by associating the moving bodies between frames, the number of moving bodies can be accurately counted.

According to a second aspect of the present invention, in the mobile object counting device, the storage means stores the continuous frame image data input from the data input means, and the arithmetic processing means:
The image data is spatially differentiated, a characteristic part of the moving body is extracted by taking a difference between frames of the spatially differentiated image data, and the moving body is frame-by-frame by labeling the characteristic portion. Is extracted, and the moving bodies are associated with each other between frames, and the moving bodies that cannot be matched are counted, and the count result is output. By extracting a portion and associating the characteristic portion between frames, the number of moving bodies can be accurately counted.

[0047]

EXAMPLE A first example of the present invention will be described below. FIG. 1 is a block diagram showing the configuration of the first embodiment.

In FIG. 1, reference numeral 1 is a CCD camera for picking up an image of a moving body and outputting analog image data. FIG. 2 is an installation diagram of the CCD camera 1, and the CCD camera 1 is installed right below the ceiling.

In FIG. 1, 2 is an A / D converter for converting the analog image data into digital image data, and 3 is a digital switch for outputting the digital image data by selecting an output destination. The CCD camera 1, the A / D converter 2 and the digital switch 3 constitute data input means 10.

4, 5 and 6 are digital image data input from the digital switch 3 or a CPU 8 which will be described later.
Reference numeral 7 is a frame memory for storing the image-processed data, and 7 is a data memory for storing the data processed by the CPU 8 described later. Frame memory 4,
5, 6 and the data memory 7 constitute a storage means 11.

A CPU 8 counts the number of moving objects by subjecting the digital image data read from the frame memories 4, 5 and 6 to image processing and various arithmetic processing.
And 9 is a program memory that stores a processing program in the CPU 8. The CPU 8 and the program memory 9 constitute the arithmetic processing means 12.

The operation of the first embodiment can be divided into 7 steps as follows.

[STEP1 Input of image data] [STEP2 Extraction of moving body by difference between frames] [STEP3 Labeling] [STEP4 Judgment of moving body area] [STEP5 Judgment of number of moving body in moving body area] [STEP6 Correlation and Count] [STEP 7 Output of Count Result] Here, STEPs 2 to 7 are based on image processing and arithmetic processing by the CPU 8, and are based on interactions between the CPU 8 and the frame memories 4, 5, 6 and the data memory 7.

Therefore, the operation will be described along with each step, and finally the data shift in the frame memories 4, 5, and 6 will be described.

[STEP1 Input of image data] CC
Analog image data input from D camera 1 is A / D
A / D conversion to digital image data by the converter 2,
The digital image data is input to the frame memories 4, 5 and 6 by switching the output destination with the digital switch 3 in units of one frame.

[STEP2 Extraction of Moving Object by Difference between Frames] Image data for three frames that are continuously input are F (T-2ΔT), F (T-ΔT), and F (T). Here, ΔT is a sampling interval of image data, and T−2ΔT, T−ΔT, and T represent sampling times.

The difference between F (T-2ΔT) and F (T-ΔT) and the difference between F (T-ΔT) and F (T) are calculated for each pixel, and the difference is calculated by the threshold value t ₁ . Quantify.

Each of the binarized difference data is Fb.
If (T-2ΔT, T-ΔT) and Fb (T-ΔT, T), then Fb (T-2ΔT, T-ΔT) = [| F (T-2ΔT)
−F (T−ΔT) |] t ₁ Fb (T−ΔT, T) = [| F (T−ΔT) −F (T)
|] T ₁ . Here, [] t ₁ indicates a binarization operation in which if the value in [] is t or more, it is set to “1”, and if it is less than t _1, it is set to “0”.

Next, in order to eliminate the duplication of the mobile unit, Fb
The logical product of (T-2 [Delta] T, T- [Delta] T) and Fb (T- [Delta] T, T) is obtained, and this is set as interframe difference data H (T- [Delta] T) at time T- [Delta] T.

H (T-ΔT) = Fb (T-2ΔT, T-
ΔT) * Fb (T−ΔT, T) Here, * indicates a logical product operation.

By the above processing, the background area is removed,
An area with a bit of H (T-ΔT) is extracted as a moving body area, but in reality, a noise area is also included.
The area extracted here will be referred to as an effective area.

Further, for the purpose of removing noise and reducing memory capacity, inter-frame difference data H (T-ΔT)
Is subjected to the following blocking. H (T−ΔT) is n ×
A small region of n pixels, that is, a block is divided into M × N blocks, and a threshold value t ₂ is set. Let k be the number of pixels in which bits are set out of the n × n pixels in the block (the number of pixels whose data is “1”), and if k> t ₂ , set the data of that block as “1” as an effective area, k ≦ t
If it is ₂ , the data in the block is set to "0" as a noise area. Blocked H (T-ΔT) is B
Assuming that (T−ΔT), B (T−ΔT) = << H (T−ΔT) >> t ₂ = << Fb (T−2ΔT, T−ΔT) * Fb (T−ΔT,
T) >> t ₂ Here, <<>> t ₂ indicates a blocking calculation process by the threshold value t ₂ of the binarized image data in <<>>. Further, the total number of blocks of B (T-ΔT) M × N, that is, the total number of pixels is 1 / n ² of the total number of pixels of H (T-ΔT).

FIG. 3 is an explanatory diagram of the moving body extraction based on the above-mentioned difference between frames. In FIG. 3, data a, b, and c are the above-mentioned input image data F (T-2ΔT), F, respectively.
Corresponding to (T-ΔT) and F (T), and the data d and e are the above-described binarized difference data Fb (T-2ΔT, T-Δ.
T) and Fb (T-ΔT, T). The data f corresponds to the above B (T-ΔT).

[STEP3 Labeling] Using the above-described blocked inter-frame difference data B (T-ΔT), the same label is given to the connected pixels for each pixel having a bit, and each independent effective By giving one label to the area, the extracted effective area is separated into independent areas.

The labeling process is divided into three processes of the first scanning, the grouping and the second scanning. In the first scan, scan the binarized data in the raster direction from the upper left of the screen, detect the pixels with a bit set and label (base), and if the pixel with a bit set is detected in the subsequent scan, , It is checked whether or not it is connected to the base, and if it is connected, the same label as the base is given, and if not connected, a different label is given to make a new base. Since it is not possible to apply the same label to all the connected pixels only by the above processing, it is associated with whether different labels are connected or independent at the same time.

Grouping is a process of grouping a plurality of labels that should be the same label into one group based on the above-mentioned association between the two labels.

In the second scan, the label value is changed according to the above grouping.

4 to 6 are flowcharts of the labeling process. FIG. 4 shows the processing of the first scanning, and FIG. 5 shows the processing of the second scanning. Further, FIG. 6 shows the contents of the subroutine of step I of FIG. 5 and shows the grouping processing.

In FIG. 4, the number of pixels in one frame is set to 25.
The size is 6 × 256. In addition, the inter-frame difference data B (T-
ΔT) is binarized with the effective area being “1”. In step A, XMAX and YMAX indicate the numbers of pixels in the x direction and the y direction, respectively, and 256 is substituted in each case. HIGH indicates the value of the effective area, and "1" is substituted here. Variable lab
el indicates a label number, and "0" is substituted as an initial value. LK (L1, L2) is a two-dimensional array having L1 and L2 indicating label numbers as variables,
"0" is substituted as the initial value. When L1 and L2 are connected, LK (L1, L2) becomes "1".

In step B, IMG (X, Y) is X (takes a value of 0 to XMAX-1) and Y (0 to YM).
The pixel data corresponding to the inter-frame difference data B (T-ΔT) is read in 2).
It is a three-dimensional array. This IMG (X, Y) is sequentially scanned in the raster direction, and if the value of IMG (X, Y) is not HIGH, the process returns to step B via step G to move to the processing of the next pixel data. The value of (X, Y) is HI
If it is GH, go to step C.

In step C, IMG (X, Y) shown in FIG.
4-direction data of IMG (X-1, Y), IMG
(X-1, Y-1), IMG (X, Y-1), IMG
If all the values of (X + 1, Y-1) are "0", the label is incremented in step D (label = l
abel + 1) and IMG (X, Y) are substituted for label, and then the process returns to step B via step G to process the next pixel data.

If any value of the four-direction data is not "0" in step C, the four-direction data is checked clockwise in step E, and the first data other than "0", that is, the label, is IMG (X). , Y), and if there is a label having a value different from the label first found in the four-direction data in step F, both labels are L1 and L.
After setting LK (L1, L2) = 1 as 2, then step G
Then, the process returns to step B via to move to the processing of the next pixel data.

FIG. 8 shows the state of image data in the process of the labeling process of the first embodiment. FIG.
8A is image data showing an effective area before labeling, FIG. 8B is image data after the first scanning, and FIG.
(C) shows the image data after the second scanning. In FIG. 8A, the part in which "1" is described is the effective area. FIG. 8A shows the result of the first scanning as shown in FIG.
Labels 1 to 4 as shown in FIG. FIG.
The circle mark in (b) indicates the base.

The above processing is performed for all the pixels of B (T-ΔT), the label N is assigned to the variable N indicating the maximum number of labels in step H of FIG. The subroutine shown in 6 is executed to perform grouping.

In step M of FIG. 6, k is a variable indicating a label value before grouping which takes a value of 1 to N,
q (k ') is a one-dimensional array in which the label values after the grouping of the label k are entered (the variable k'has a value of 0 to N), p
(K) is a pointer array into which the address of q (k ') is inserted. Q (0) = 0, p (k) = & q as initial settings
(0). Here, & is a symbol indicating an address value, and & q (0) indicates an address value of q (0). With this initial setting, all labels before grouping are address values &
The setting is that data of q (0), that is, "0" is converted.

At step N, L1 = 1 to N-1, L2 = 2
~ N, LK for L1 and L2 where L1 <L2
(L1, L2) is checked, and if LK (L1, L2) = 1 is not satisfied, the process proceeds to step V, and if LK (L1, L2) = 1, proceeds to step O.

In step O, * p (L1) is a symbol indicating the data value contained in the address value indicated by the pointer p (L1). For example, p (3) = & q (3), q.
If (3) = 1, then * p (3) = 1, which indicates that label 3 is converted to label 1.

If * p (L1) = * p (L2) = 0, in step P, q (L1) = L1 p (L1) = & q (L1) p (L2) = & q (L1) And proceed to * p (L1) = * p (L2)
If not = 0, go to step Q.

In step Q, * p (L1) = 0 and * p
If (L2) ≠ 0, in step R, p (L1) = p (L2) is set, and the process proceeds to step V. * p (L1) = 0 and * p
If (L2) ≠ 0, the process proceeds to step S.

In step S, * p (L1) ≠ 0 and * p
If (L2) = 0, then in step T, p (L2) = p (L1) is set and the operation proceeds to step V. * p (L1) ≠ 0 and * p
If (L2) = 0 is not satisfied, in step U, MAX [* p (L1), * p (L2)] = MIN [* p (L1), * p (L2)] and the process proceeds to step V. Where MAX [] = MIN
[] Is a process of substituting the value of the smaller variable in [] for the larger variable.

Through step V, the next LK (L1, L
When the process of 2) is performed and the processes of all LKs (L1, L2) are completed, the process returns to step I of FIG.

With that, the grouping process is completed.

Next, an example of the above grouping process is shown in FIG.
Also, description will be made with reference to FIGS. Assume that the following association data has been obtained in the first scan.

Data1: LK (2,7) = 1 data2: LK (3,4) = 1 data3: LK (3,6) = 1 data4: LK (6,7) = 1 In this case, label 2 to label 4 Is associated with 2 → 7 →
6 → 3 → 4 (data1 → data4 → data3 → d
First understood by ata2). It is called an association between two data that wraps around such an intricately intertwined association.

FIG. 9 shows a pointer array p based on the above data.
(K) and the array q (k ') are shown in the initial setting state, and FIGS. 10 and 11 show the process of grouping processing when data1 to data4 are referred to in order.

In FIG. 10, * p is set by the initial setting.
(2) = * p (7) = q (0) = 0, so data1
Q (2) = 2, p (2) = & q (2), p
(7) = & q (2). Next, * p (3) = * p
Since (4) = q (0) = 0, q in the processing of data2
(3) = 3, p (3) = & q (3), p (4) = & q
(3).

In FIG. 11, * p (3) = q (3) =
3 (≠ 0), * p (6) = q (0) = 0, so dat
In the processing of a3, p (6) = & q (3). Finally * p
(6) = q (3) = 3 (≠ 0), * p (7) = q (2)
= 2 (≠ 0), q (3) = 2 in the processing of data4
And

After all, * p (2) = * p (3) = * p (4)
= * P (6) = * p (7) = 2, and two groups that were grouped separately at the processing stage of data3, that is, p (2) and p containing & q (2)
The group of (7) and p (3) containing & q (3),
The group of p (4) and p (6) is the processing of data4,
It is unified by setting q (3) = q (2).

Finally, the second scan is performed based on the result of the grouping process. In FIG. 5, in step J, the image data labeled by the first scan is scanned in the raster direction, and * p (IMG (X, Y)) is checked for the pixel for which IMG (X, Y) ≠ 0. , * P (I
If MG (X, Y) = 0, return to step J via step L to move to the processing of the next pixel, * p (IMG
If (X, Y)) ≠ 0, in step K, the label value after grouping is substituted as IMG (X, Y) = * p (IMG (X, Y)), and the process proceeds to step L.

Then, the above processing is performed for all the pixels, and the second scanning is completed. By such second scanning, the image data labeled in the first scanning shown in FIG. 8B is converted as shown in FIG. 8C.

[STEP4 Judgment of Mobile Object Area] The judgment of the mobile object area is performed using the above-mentioned labeled inter-frame difference data B (T-ΔT). Considering the distance between the CCD camera 1 and the moving body and the size of the moving body,
A threshold t ₄ indicating the area area, that is, the number of pixels, which serves as a criterion, is provided. If the area of the labeled effective area, that is, the number of pixels is t ₄ or more, it is determined as a moving body area, and if it is less than t ₄ , Remove as a noise area.

FIG. 12 is an explanatory diagram of the determination of the moving body area. In FIG. 12, the label 1, the label 2, the label 3 and the threshold value t ₄ are shown by the area. Since the effective area area of each of the label 1 and the label 2 is t ₄ or more, the area is determined to be a moving body area, but the effective area area of the label 3 is smaller than t _{4 and} is removed as a noise area. Therefore, it is determined that there are two moving body areas.

Finally, assuming that the number of moving body regions is n and the label value is L _k (k = 1, 2 ... n), L _k is 1 to
The label value of each moving body region is converted to take the value of n. For example, when there are four moving body areas and their label values are 1, 3, 4, and 6, the label values of the respective moving body areas are 1, 2, 3, and 4.

[STEP5 Judgment of Number of Moving Objects in Moving Object Area] From frame difference data B (T-ΔT) from which the noise area is removed, aspect ratios, area ratios, and peripheral surfaces which are characteristic amounts of individual moving object areas. By calculating the ratio and comparing the calculated value and the threshold value provided for each feature amount,
The number of moving bodies in the moving body area is determined.

Each feature amount is obtained by the following formula. A region circumscribed by two circumscribed lines of the moving body region drawn in the set moving direction of the moving body and two circumscribed lines of the moving body region drawn in the direction perpendicular to the set moving direction of the moving body And Ls is the length of the side of the circumscribed rectangle in the set movement direction, Li
Where A is the length of the side of the circumscribed rectangle in the direction perpendicular to the set movement direction: Aspect ratio As = Ls / Li Area ratio Mn = (area of moving body area) / (area of circumscribed rectangle) (≦ 1) Peripheral surface ratio Su = (Perimeter of moving body area) ² / (area of moving body area) The reason for determining the number of solids by the area ratio is that the aspect ratio is the above when the moving body moves diagonally with respect to the set moving direction. In this case, even if the number of individuals is two, the value becomes close to 1, and an erroneous determination occurs. However, when the two moving bodies move diagonally with respect to the set movement direction, the area of the circumscribing rectangle becomes larger than when moving in the set movement direction, and as a result, the area ratio becomes smaller. is there.

The reason for determining the number of solids based on the peripheral surface ratio is that the above-mentioned aspect ratio and area ratio result in an erroneous determination in the moving body region of the number of solids 2 with shadows and noise portions. This is because, in the integrated moving body region, a concave portion is formed in the region at the contact portion, and the peripheral length with respect to the area of the moving body region, that is, the peripheral surface ratio becomes larger than in the case where the number of solids is one.

FIG. 13 is a flowchart of the processing for determining the number of moving bodies in the moving body area. In FIG. 13, the aspect ratio As is calculated in step A, As is compared with the aspect ratio threshold value in step B, and if As is less than or equal to the threshold value, in step H the number of moving bodies in this moving body region is calculated. If it is determined that the number is two and As is larger than the threshold value, the process proceeds to step D.

The area ratio Mn is calculated in step C, the area ratio threshold is compared with Mn in step D, and if Mn is less than the threshold value, the number of moving bodies in this moving body region is 2 in step H. If As is larger than the threshold value, the process proceeds to step E.

In step E, the peripheral surface ratio Su is calculated, and in step F, Su is compared with the peripheral surface ratio threshold value. If Su is larger than the threshold value, in step H the number of moving bodies in this moving body area is calculated. If it is determined that the number is two and Su is equal to or less than the threshold value, it is determined in step G that the number of moving bodies in this moving body region is one.

14 to 17 each show an example of the moving body area. The moving body area of FIG. 14 (moving body area 1
15) has one moving body, and the moving body area (shown as moving body area 2) in FIG. 15 has two moving bodies.
The number of moving objects is 2 in the moving object area (referred to as area 3) in FIG.
The actual moving direction is diagonal to the set moving direction, the moving body region of FIG. 17 (hereinafter referred to as moving body region 4)
Indicates that the number of moving objects is two and each includes a shadow portion and a noise portion. Further, FIG. 18 shows an example of the set threshold value of each feature amount and the calculation result of each feature amount of the mobile body regions 1 to 4.

A process for determining the number of moving bodies in the moving body regions 1 to 4 will be described with reference to FIGS. 13 to 18. The aspect ratio is calculated for all mobile regions. Mobile unit area 2
Aspect ratio of (actual solid number 2) is 0.50
Because of the following, the number of solids is determined to be 2, and the area ratio and the peripheral surface ratio are not calculated.

Next, the area ratio is calculated for the moving body regions 1, 3, and 4. Since the area ratio of the moving body region 3 (actual number of solids 3) is less than the area ratio threshold value 0.60, the number of solids is 2. Therefore, the peripheral surface ratio is not calculated.

Further, the peripheral surface ratio is calculated for the moving body regions 1 and 4, and the peripheral surface ratio of the moving body region 1 (actual solid number 1) is 1 or less because the peripheral surface ratio threshold value is 18.0 or less. Since the peripheral surface ratio of the moving body region 4 (actual solid number 1) is larger than the peripheral surface ratio threshold value 18.0, it is determined to be the solid number 4.

From the above, it is accurately determined that the moving body area 1 is the number of solids 1 and the moving body areas 2 to 4 is the number of solids 2.

[STEP 6 Correlation and Count] Correspondence is established between each mobile unit area, that is, each label, between two consecutive frames. Considering the number of moving objects in the label, the labels between two frames whose center position is the shortest distance are associated as the same moving object, and the moving objects with the labels that cannot be associated at that time are associated with each other. The number of people is incremented on the assumption that it has moved to the outside of the target area.

FIG. 19 is a flowchart of the association and counting process. 20 shows step H of FIG.
21 is the contents of the subroutine 1 in FIG.
This is the contents of the subroutine 2 in step I.

In FIG. 19, it is assumed that the moving body region extraction data of frame 1 and the moving body region extraction data of frame 2 are associated with each other, and that frame 1 is the previous frame of frame 2 (for example, frame 1 is the time). T-2ΔT
, Frame 2 was sampled at time T-ΔT). Also L _k
(K = 1, 2 ... m) is the label in frame 1, L ′
Let _h (h = 1, 2 ... n) be a label in frame 2.

In step A, the label L _k (k = 1, 2 ...
m) center coordinates (i _k , j _k ) of the moving body region, label L
′ _H , (h = 1, 2, ..., N) center coordinates (i
_'H, determined using the center or centroid of the circumscribing rectangle _j'h),
The label _{L k (k = 1,2 ... m} ) mobile number n _k of the moving object in the region of the label _{L'h (h = 1,2 ... n} ) mobile number _n'k mobile region of Then, Mx (L _k ) = i _k My (L _k ) = j _k Mc (L _k ) = n _k (k = 1, 2 ... m) Mx ′ (L ′ _h ) = i ′ _h My ′ _{(L'h) = j'h Mc'} (L'h) = n' and _{h (h = 1,2 ... n)} .

In step B, k = 1, and in steps C and D, the search target label for the label L _{1 is} selected from L ′ _h . Labels downstream of the label L _{1 with} respect to the set movement direction are to be searched.

When there is no label to be searched in step E, the label L ₁ is judged to have disappeared in frame 2 (moved to the outside of the target area) in step S, and the number of moving bodies is incremented. The label L _{2 is} incremented and the processing of the label L ₂ is started.

If there is a label to be searched in step E, the number of moving objects Mc (L ₁ ) of label L ₁ is found in step G.
If = 1, the process proceeds to subroutine 1 shown in FIG. 20 in step H, and in step F, the number Mc of moving objects with label L ₁
If (L ₁ ) = 2, the process proceeds to the subroutine 2 shown in FIG. 21 in step I.

If the number of search target labels is two or more in step J of FIG. 20, the distance between the label L ₁ and each search target label is calculated as the central coordinate data Mx (L ₁ ) and M in step K.
y (L ₁ ) and Mx ′ (L ′ _h ), My ′ (L ′ _h ).
Then, in step L, the search target label having the shortest distance to the label L ₁ is selected as the selection label, and the process proceeds to step M. In step J, if the number of the search target labels is 1, this label is selected as the selection label. Go to M.
In step M, the selected label and the label L ₁ are associated with each other as the same moving body, and the process returns to step H in FIG.

If the number of search target labels is two or more in step O of FIG. 21, the distance between the label L ₁ and each search target label is calculated as the center coordinate data Mx (L ₁ ) and M in step P.
y (L ₁ ) and Mx ′ (L ′ _h ), My ′ (L ′ _h ).
And go to step Q. Label L _{1 in} step Q
The number Mc of moving objects of the search target label having the shortest distance from
If ′ (L ′ _h ) = 1, it is determined in step R that the frame 2 is divided, and in step S, the search target label having the shortest distance to the label L ₁ and the search target label having the next shortest distance are selected. If the number of moving objects Mc '(L' _h ) of the search target label having the shortest distance to the label L _{1 in} step Q is 2 as a selected label, the distance to the label L _{1 in} step T Is selected as the selected label and the process proceeds to step U.

If the number of labels to be searched is one in step O, the number M of moving objects in the label to be searched is found in step V.
When c ′ (L ′ _h ) is searched and Mc ′ (L ′ _h ) = 2, the process proceeds to step U with the search target label as the selection label in step W, and Mc ′ (L ′ _h ) = in step V. If it is 1, it is determined in step X that the two moving bodies have split and one has disappeared, the moving logarithmic count is incremented, and in step W, the search target label is set as the selected label and the process proceeds to step U. Then, in step U, the selected label and the label L ₁ are associated with each other as the same moving body, and in step I of FIG.
Return to

In FIG. 19, when the processing of step G or step I is completed, k is incremented in step B and the processing of label L ₂ is started.

In this way, the labels L _{1 to} L _m are associated with each other.

FIG. 22 shows an example of correspondence and counting. In FIG. 22, a, b, and c indicate labels in the moving body extraction data 1, a ′, b ′, and c ′ indicate labels in the moving body extraction data 2, and a ″,
b ″ and c ″ represent labels in the moving body extraction data 3. In addition, it is assumed that the mobile body extraction data 1, the mobile body extraction data 2, and the mobile body extraction data 3 are sampled in this order (for example, sampled at times T-2ΔT, T-ΔT, and T, respectively).

Mobile object extraction data 1 and mobile object extraction data 2
Correspondence and counting are performed as follows. It is assumed that the number of moving bodies of each label is determined as follows.

Number of solids of label a = 2 Number of solids of label a ′ = 1 Number of solids of label b = 1 Number of solids of label b ′ = 1 Number of solids of label c ′ = 1 Search target of label a is label a Labels a'and b'downstream of the position. The label a has two solids, and the labels a ′ and b ′ each have one solid,
Label a is determined to have split, and labels a'and b '
Is associated with. The search target of the label b is the label a ′,
Although they are b'and c ', the label c'which is the shortest distance is associated.

The association and counting between the moving body extraction data 2 and the moving body extraction data 3 are performed as follows. It is assumed that the number of moving bodies of each label is determined as follows.

Number of solids of label a ′ = 1 Number of solids of label a ″ = 1 Number of solids of label b ′ = 1 Number of solids of label b ″ = 1 Number of solids of label c ′ = 1 Number of solids of label c ″ = 1 Since the label a'is the only search target for the label a ', the label a'is associated. The label b'is judged to have disappeared because there is no search target label, and the number of people is incremented by 1. The label c' Search target is label a ″,
Although it is c ″, the label c ″ located at the shortest distance is associated. Incidentally, as a result, since the label as the source of the association of the label b ″ does not exist in the moving body extraction data 2, it newly appears in the moving body extraction data 3.

Summarizing the above, label a = label a ′ = label a ″ label b = label b ′ ≠ label b ″ label c ′ = label c ″, the mobile body count number = 1 (label b ′).

[STEP7 Output of Count Result] Finally, the count result of the moving body is output. STEP2-6
If the reset is not applied, the cumulative number of moving objects counted up to the previous round is added to the number of moving objects counted in the current round, and the result is output. If a reset is applied, it is counted in the current round. Outputs the number of mobile units that have been created.

The operation of each step has been described above. Finally, the data shift in the frame memories 4, 5 and 6 will be described.

The data shift of the frame memories 4, 5, 6 is performed by the point value shift method in order to speed up the processing. This method shifts the pointer value indicating the start address of the frame memory of the input image data and the processed data for each round,
Since this is equivalent to shifting the input image data and the processed data, it is not necessary to shift the entire data, and this is a method that enables continuous calculation processing.

FIG. 23 is an explanatory diagram of the pointer value shift method. The data areas 1, 2, 3 are frame memories 4, 5,
6 respectively. Before entering the first round, the image data 1 is first captured in the data area 1 of the frame memory, and subsequently, the image data 2 after the sampling interval ΔT.
Are taken into area 2. Further, after ΔT, the first round starts when the image data 3 is loaded into the data area 3. The data shift for one round and the accompanying arithmetic processing by the CPU are performed during ΔT. As shown in the figure, image data 1 is data a, image data 2 is data b, and image data 3 is data c.

In the first round, in step A, the image data 3 is loaded into the data area 3 (data c), and in step B, the binarized difference data of the data a and the data b (above Fb) is acquired in the data area 1. To (data d). Similarly, the binary difference data of the data b and the data c is written in the data area 2 (data e).
The data c is stored in the data area 3 as it is. In step C, the inter-frame difference data B (T-ΔT), which is the logical product of the data d and the data e, is written in the data area 1 (data f), and the feature amount is extracted from this f. The data e and the data c are stored in the data area 2 and the data area 3 as they are.

In the second round, the image data 4 sampled after ΔT of the image data 3 in step A ′
Are taken into the data area 1. (Data c '). The data e is stored in the data area 2 as it is (data d ′
The data c is stored in the data area 3 as it is. (Data b '). Data b'at step B '
And binarized difference data of data c'in the data area 3
Write in. The data c ′ and the data d ′ are saved as they are. In step C ′, the inter-frame difference data obtained by performing the logical product processing of the data d ′ and the data e ′ is written in the data area 2 (data f ′). The data c ′ and the data d ′ are saved as they are.

After the third round, the same processing as in the second round is performed. However, the data area that performs the same operation as the previous round is shifted by the pointer. For example, the image data sampled after the image data 4 is taken into the data area 2.

Next, FIG. 24 is a flow chart of each processing by the CPU 8 of the first embodiment, in which only the part relating to the pointer value shift is shown in detail. Also in FIG.
Shows the subroutine in step E of FIG. Then, in step G, association and counting are performed, and in step H, the count result is output.

Next, FIG. 24 is a flow chart of each processing by the CPU 8 of the first embodiment, in which only the part relating to the pointer value shift is shown in detail. Also in FIG.
Shows the subroutine in step E of FIG.

In FIG. 24, ADD1, ADD2, A
DD3 is the start address value of the data area 1, the data area 2, and the data area 3, respectively, and is AREA1, AREA.
2 and AREA3 are ADD1, ADD2 and ADD, respectively
Any of the three is a variable that is exclusively input, and is called a data area pointer.

In step A, as initial values, AREA1 is ADD1, AREA2 is ADD2, and AREA3 is A.
Substitute DD3 respectively.

In step B, the image data is loaded into the data area indicated by AREA1, in step C the image data is loaded into the data area indicated by AREA2, and in step D, A
The image data is taken into the data area indicated by REA3, and step E is proceeded to.

In step J of FIG. 25 which is a subroutine of step E, the data area indicated by AREA1 and AREA2
The difference data between the data areas indicated by is calculated and binarized,
The result is written in the data area indicated by AREA1, the difference data between the data area indicated by AREA2 and the data area indicated by AREA3 is calculated and binarized in step K, the result is written in the data area indicated by AREA2, and AREA1 is written in step L. And the data area indicated by AREA2 are ANDed and then blocked.
The result is written in the data area indicated by AREA1.

In step F of FIG. 24, the characteristic amount of the data area indicated by AREA1 is extracted and labeled, the correspondence and counting are performed in step G, and the counting result is output in step H.

At step I, TEMP is used as a pointer to temporarily stock the start address, and TEMP is stored in ARE.
The pointer value of A1 is substituted, the pointer value of AREA2 is substituted for AREA1, the pointer value of AREA3 is substituted for AREA2, and the pointer value of TEMP, that is, the pointer value of AREA1 is substituted for AREA3, whereby the pointer value is shifted. This made the second round of A
The target area of REA1 is data area 2, the target area of AREA2 is data area 3, and the target area of AREA3 is data area 1. FIG. 26 shows how the data area pointer value changes.

Up to this point, the processing of the first round has been completed,
After the second round, the processes of steps D to I are repeated for each round.

According to the above-described first embodiment, the moving object area is extracted from the image data without performing the data shift in the frame memory by the pointer shift, and the labeling is performed by performing the grouping by using the pointer. By making the number of scans twice, the processing speed is increased to narrow the sampling interval of the image data, and in the determination of the number of moving bodies in the moving body region, the aspect ratio as well as the area ratio and the peripheral surface ratio are used as the determination items. By associating the moving bodies between frames, the number of moving bodies can be accurately counted.

Next, the second embodiment will be described. In the second embodiment, a characteristic area in the moving body is extracted, and the number of moving bodies is counted by focusing on this area. At this time, ideally, a region that does not change in shape and has no difference in shape or size between individual moving bodies is defined as a characteristic region. In the second embodiment, the moving body is human and the characteristic area is the head.

The configuration of the second embodiment is similar to that of the first embodiment shown in FIG. 1, and therefore its explanation is omitted.

The operation of the second embodiment can be divided into the following 6 steps.

[STEP1 Input of Image Data] [STEP2 Spatial Differentiation by Sobel Operator] [STEP3 Extraction of Moving Object by Interframe Difference] [STEP4 Extraction and Labeling of Head Region] [STEP5 Correlation and Count] [STEP6 Count Result Output] Therefore, the operation is explained according to each step, and STEP1
The operation of STEP 6 and STEP 6 is the same as that of the first embodiment, and therefore its explanation is omitted.

[STEP2 Spatial Differentiation by Sobel Operator] Image data for three frames that are continuously input is F (T-2ΔT), F (T-ΔT), F (T).
And Here, ΔT is a sampling interval of image data, and T−2ΔT, T−ΔT, and T represent sampling times.

F (T-2ΔT), F (T-ΔT), F
Each pixel of (T) is differentiated by the sobel operator in the scanning line direction, and the result is Dx (T-2ΔT), Dx (T-Δ
T) and Dx (T). In addition, F (T-2ΔT), F
Each pixel of (T-ΔT) and F (T) is differentiated by the sobel operator in the direction perpendicular to the scanning line, and the result is Dy (T
-2ΔT), Dy (T-ΔT), and Dy (T).

Image data F (T-2ΔT), F (T-Δ
The edge size of each pixel of T) and F (T) is set to D (T-2
ΔT), D (T−ΔT), and D (T) are calculated by the following formula.

D (T-2ΔT) = | Dx (T-2ΔT) | + | Dy (T-2ΔT) | D (T-ΔT) = | Dx (T-ΔT) | + | Dy (T-ΔT) | D (T) = | Dx (T) | + | Dy (T) | Alternatively, D (T-2ΔT) = {Dx (T-2ΔT)} ² + {Dy (T-2ΔT)} ² D (T -ΔT) = {Dx (T- ΔT)} 2 + {Dy (T-ΔT)} 2 D (T) = {Dx (T)} 2 + {Dy (T)} 2 further, the image data F (T- 2ΔT), F (T-ΔT),
The direction of the edge of each pixel of F (T) is θ (T-2ΔT),
θ (T−ΔT) and θ (T) are calculated by the following formula.

Θ (T−2ΔT) = tan ⁻¹ {Dy (T−2ΔT) / Dx (T−2ΔT)} θ (T−ΔT) = tan ⁻¹ {Dy (T−ΔT) / Dx (T− ΔT)} θ (t) = tan ⁻¹ {Dy (T) / Dx (T)} [STEP3 Extraction of Moving Object by Inter-frame Difference]
In order to remove noise components of D (T-2ΔT), D (T-ΔT), and D (T), D (T-2ΔT) and D (T-Δ
T), D (T−ΔT) and the difference between D (T) and Db (T−2ΔT, T−ΔT) and Db (T), which are binarized by the threshold t ₁ . -ΔT, T)
Is calculated by the following formula.

Db (T-2ΔT, T-ΔT) = [| D
(T-2ΔT) -D (T-ΔT) |] t ₁ Db (T-ΔT, T) = [| D (T-ΔT) -D (T)
|] T ₁ Here, [] t ₁ indicates a binarization operation in which if the value in [] is t ₁ or more, it is “1”, and if it is less than t, it is “0”.

Further, θ (T-2ΔT) and θ (T-ΔT),
The difference between θ (T−ΔT) and θ (T) is calculated, and the difference is binarized by a threshold value t ₂ to obtain θb.
(T-2ΔT, T-ΔT) and θb (T-ΔT, T) are calculated by the following formula.

Θb (T-2ΔT, T-ΔT) = [| θ
(T- ₂ [Delta] T)-[theta] (T- [Delta] T) |] t2 [theta] _b (T- [Delta] T, T) = [| [theta] (T- [Delta] T)-[theta] (T).
|] T ₂ FIG. 27 shows Db (T-2ΔT, T-ΔT) or Db (T
-ΔT, T) showing the difference data corresponding to
For example, letting A and a be the edges of the person and the head at time T-2ΔT, B and b are the edges of the person and the head at time T-ΔT.

As shown by the broken line portion in FIG. 27, Db (T-
In 2ΔT, T-ΔT) and Db (T-ΔT, T), the edge may be interrupted at the overlapping portion of the moving body, which is θb (T-2ΔT, T-ΔT) and θb (T-ΔT).
Since it is corrected by T, T, Db (T-2ΔT, T
Logical sum D of -ΔT) and θb (T-2ΔT, T-ΔT)
b '(T-2ΔT, T-ΔT) and Db (T-ΔT,
T) and θb (T−ΔT, T) OR'd Db ′ (T−Δ
T, T) is calculated by the following equation.

Db '(T-2ΔT, T-ΔT) = Db
(T-2ΔT, T-ΔT) + θb (T-2ΔT, T-Δ
T) Db ′ (T−ΔT, T) = Db (T−ΔT, T) + θb
(T−ΔT, T) Here, + indicates a logical sum.

Next, in order to eliminate the duplication of the moving body edge, Db '(T-2ΔT, T-ΔT) and D'b (T-Δ
The logical product of (T, T) is obtained, and this is set as interframe difference data H (T-ΔT) at time T-ΔT.

H (T-ΔT) = Db '(T-2ΔT, T
−ΔT) * Db ′ (T−ΔT, T)} where * represents a logical product.

FIG. 28 is an explanatory diagram of duplicated erasure of a mobile body edge by logical product of mobile body edges. In FIG. 28, the difference data 1 and the difference data 2 are Db ′, respectively.
(T-2ΔT, T-ΔT) and Db '(T-ΔT,
Corresponds to T). In the difference data 1 or the difference data 2, the edge of the same person who is moving appears twice.
In the difference data 1, A and a are the person and the head edge thereof at time T−2ΔT, and B and b are the person and the head edge thereof at time T−ΔT.
In the difference data 2, B and b are the person and the head edge thereof at the time T-ΔT, and C and c are the person and the head edge thereof at the time T. By taking the logical product of the difference data 1 and the difference data 2, only the edges B and b at time T-ΔT remain, and the duplication is removed.

Further, for the purpose of removing noise and reducing memory capacity, inter-frame difference data H (T-ΔT)
Is subjected to the following blocking. H (T−ΔT) is n ×
It is divided into small regions of n pixels, that is, blocks, and the threshold
Provide ₃ . The number of pixels in which bits are set out of n × n pixels in a block is k. If k> t ₃ , the data of the block is set as “1” as an effective area, and if k <t ₃ is set as a noise area of the block. The data is set to "0". Blocked H (T-ΔT) is converted into B (T-ΔT)
When T), B (T-ΔT ) = "H (T-ΔT)" t 3 = "Db '(T-2ΔT, T-ΔT) * Db' (T-Δ
T, T) >> t ₃ . Here, <<>> t ₃ indicates a blocking operation process by the threshold value t ₃ of the binarized image data in <<>>.

[STEP4 Extraction and Labeling of Head Region] The head region is extracted from H (T-ΔT) or B (T-ΔT) by template matching. There are several matching templates with different radii to accommodate variations in height and head size, and H (T-
[Delta] T) or B (T- [Delta] T), the center of the area surrounded by the edge is obtained, the distance between the center and each pixel forming the edge is calculated, and the difference between the calculated distance and the distance in the matching template is calculated as the pixel. If the sum of the differences is within the set threshold value, it is extracted as the head region.

Then, labeling is applied to the pixel corresponding to the center of the obtained head region. Therefore, the number of moving objects is equal to the number of labels.

[STEP 5 Correspondence and Count] Each head region data, that is, each label is associated between two consecutive frames, and a label that cannot be associated is out of the measurement target region (field of view of the CCD camera at that time). Outside) and increment the people count assuming it has moved.

There are two associating methods, and one of them is appropriately selected and used according to the sampling interval of the image data and the moving speed of the moving body. When the sampling interval of the image data is short or the moving speed of the moving person is slow, the labels between the two frames having the shortest distance to the center of gravity of the head are associated as the same person (first association method). If the sampling interval of the image data is long or the moving speed of the moving person is high, the labels between the two frames having the most similar feature amount such as the shape are associated as the same person (second association method).

FIG. 29 is a flowchart of the associating and counting process using the first associating method. Figure 2
9, the head region extraction data of frame 1 and the head region extraction data of frame 2 are associated with each other, and it is assumed that frame 1 is the previous frame of frame 2 (for example, frame 1 is sampled at time T-2ΔT). , Frame 2 was sampled at time T-ΔT). _Lk (k = 1, 2 ... m) is a label in frame 1 and L' _h (h = 1, 2 ... n) is a label in frame 2.

In step A, the label L _k (k = 1, 2 ...
center coordinates (i _k head region of m), j _k) and the label _{L 'h (h = 1,2 ...} n) the center coordinates (i head area of' _h, j _'h) the center of the circumscribed rectangle Ya The center of gravity is used to obtain Mx (L _k ) = i _k My (L _k ) = j _k (k = 1,2 ... m) Mx ′ (L ′ _h ) = i ′ _h My ′ (L ′ _h ) = J ′ _h (h = 1, 2 ... n).

In step B, k = 1, and in steps C and D, the label to be searched for the label L _{1 is} selected from L ′ _h . Labels downstream of the label L _{1 in} the flow of person movement are to be searched.

If there is a search target label in step E, the number of search target labels is detected in step F. If there is one search target label, this label is selected in step G and selected in step H. Label and Label L ₁
Corresponds to the same person, increments k in step B, and starts processing of label L ₂ .

When there are two or more search target labels in step F, the distance between the label L ₁ and each search target label is calculated as the center coordinate data Mx (L ₁ ), My (L ₁ ) and Mx '(L in step I. ′ _H ), My ′ (L ′ _h ), and in step J, the search target label having the shortest distance from the label L ₁ is selected as the selected label, and the process proceeds to step H.

If there is no search target label in step E, it is determined in step K that the label L ₁ has disappeared (moved to the outside of the target area), the number of people is incremented, and the process proceeds to step B.

Next, FIG. 30 is a flowchart of the associating and counting process using the second associating method.
In the following description of FIG. 30, settings that overlap with FIG. 29,
Descriptions of symbols and formulas are omitted.

In FIG. 30, in step A, the label L _k
(K = 1,2 ... m) the center coordinates data Mx (L _k) of the head region of _{= i k My (L k)} = j k and the label L _'h head region of (h = 1,2 ... n) center coordinate data _{Mx '(L' h) =} i in addition to the _{'h My' (L 'h} ) = j' h, circularity of each label, the aspect ratio, the characteristic amounts such as area ratio determined. An area bounded by two outer tangent lines of the head area drawn in the set moving direction of the moving body and two outer tangent lines of the head area drawn perpendicular to the set moving direction of the moving body is a circumscribed rectangle, Let S be the area of the head region, L be the length of the outline of the head region, Ls be the length of the side of the circumscribed rectangle in the set movement direction, and Li be the length of the side of the circumscribed rectangle in the direction perpendicular to the set movement direction. , Circularity = 4πS / L aspect ratio = Ls / Li area ratio = S / (LsLi)

Here, the circularity and the aspect ratio of these feature quantities are adopted to label L _k (k = 1, 2 ...
m), the circularity is p _k , and the aspect ratio is q _k . Mp (L _k ) = p _k Mq (L _k ) = q _k (k = 1, 2 ... m) and the label L ′ _h (h = 1 , 2 ... n) has a circularity of p ′ _h and an aspect ratio of q ′ _h , Mp ′
Let (L ′ _h ) = p ′ _h Mq ′ (L ′ _h ) = q ′ _h (h = 1, 2 ... n).

In step B, k = 1, and in steps C and D, the label to be searched for the label L _{1 is} selected from L ′ _h . Labels downstream of the label L _{1 in} the flow of person movement are to be searched.

If there is a search target label in step E, the number of search target labels is detected in step F, and if there is one search target label, then in step G this label is selected and the process proceeds to step L.

When there are two or more search target labels in step F, the difference between the feature amount of the label L _{1 and} the feature amount of the search target label is calculated for each feature amount in step I, and step J
Then, the process advances to step L with the label having the smallest sum of the differences in the feature quantities as the selected label. The sum of the difference between the feature _{amount, | Mp '(L' h} ) -Mp (L 1) | + | Mq '(L'
_h ) -Mq (L ₁ ) | Here, h = number of the search target label.

In step L, the difference between the feature amounts of the label L ₁ and the selected label and the set threshold value are compared for each feature amount. If all the feature amount differences are smaller than the threshold value, the selection is performed in step H. The label and the label L ₁ are associated as the same person, k is incremented in step B, and the label L ₂
Processing is started, and if the difference between any of the feature quantities is equal to or greater than the threshold value, the process proceeds to step K. Assuming that the threshold of the difference in circularity is t ₄ and the threshold of the difference in aspect ratio is t ₅ , the conditions that can be associated are | Mp ′ (L ′ _h ) −Mp (L ₁ ). | <T ₄ and | Mq ′ (L ′ _h ) −Mq (L ₁ ) | <t ₅ . Here, h = number of the selected label.

When there is no label to be searched in step D,
Alternatively, if the difference between any of the feature amounts is equal to or larger than the threshold value in step L, it is determined in step K that the label L ₁ has disappeared, the number of people is incremented, and the process proceeds to step B.

FIG. 31 shows an example of association and counting by the first association method used when the frame interval is short or the moving speed is fast. FIG.
1, a, b, and c represent labels in the moving body extraction data 1, a ′, b ′, and c ′ represent labels in the moving body extraction data 2, and a ″, b ″, and c ″ represent moving body extraction data. 3 shows a label in 3. Also, it is assumed that the moving body extraction data 1, the moving body extraction data 2, and the moving body extraction data 3 are sampled in this order (for example, time T-2ΔT,
It is assumed that they are sampled at T-ΔT and T, respectively.

Mobile object extraction data 1 and mobile object extraction data 2
Correspondence and counting are performed as follows. The search target of the label a is the labels a ′ and b ′ which are located downstream of the position of the label a in the moving body extraction data 2,
The label a ′ at the shortest distance is associated. Label b
Since only the label b ′ is searched for, the label b and the label b ′ are associated with each other. The search target of the label c is the labels a ′, b ′, and c ′, but the label c ′ with the shortest distance.
Are associated with.

The association and counting between the moving body extraction data 2 and the moving body extraction data 3 are performed as follows. The search target of the label a ′ is the labels a ″ and c ″, but the label a ″ with the shortest distance is associated. The search target of the label c ″ is also the labels a ″ and c ″, but with the shortest distance. The label c ″ is associated. The label b ′ is determined to have disappeared because there is no search target label, and the number of people is incremented by 1. As a result, the label b ″ is obtained.
Is a new appearance.

Summarizing the above, label a = label a ′ = label a ″ label b = label b ′ ≠ label b ″ label c = label c ′ = label c ″, the number of moving persons = 1 person (label b ′) .

FIG. 32 is an explanatory diagram of association and counting by the second association method used when the frame interval is long or the moving speed is slow.

In FIG. 32, association and counting between the moving body extraction data 1 and the moving body extraction data 2 are performed as follows. Label to be searched for label a is label a ′
Therefore, if | Mp ′ (a ′) − Mp (a) | <t ₁ and | Mq ′ (a ′) − Mq (a) | <t ₂ , the label a and the label a ′ are associated with each other. Otherwise, it is determined that the label a has disappeared, and the number of people count is incremented by one. The label b is determined to have disappeared because there is no search target label, and the number of people count is incremented by one. Labels to be searched for the label c are labels a ′ and c ′. The sum of the differences between the feature amounts of the label c and the label a ′ and the sum of the differences between the feature amounts of the label c and the label c ′ are calculated, and the search target label having the smaller sum of the differences is associated with the label c. The sum of the differences between the feature amounts of the label c and the label a ′ is | Mp ′ (a ′) − Mp (c) | + | Mq ′ (a ′) −
The sum of the differences between the feature amounts of the label c and the label c ′ calculated by Mq (c) | is | Mp ′ (c ′) − Mp (c) | + | Mq ′ (c ′) −
It is calculated by Mq (c) |. The label b is determined to have disappeared because there is no search target label, and the number of people count is incremented by one.

In the second associating method, it is also possible to compare and associate the previous label with the feature amount without narrowing down the search target label by paying attention to the moving direction. This method is applied when the direction of movement varies from person to person.

As described above, according to the second embodiment, it is not necessary to determine the number of moving objects in the moving object area by spatially differentiating the image data and extracting the characteristic area of the person, that is, the head area. Therefore, the number of moving bodies can be accurately counted.

[0184]

As is apparent from the above first embodiment,
According to the moving object counting apparatus of claim 1, the moving object region is extracted from the image data without performing the data shift in the frame memory by the pointer shift, and the grouping is performed by using the pointer so that the number of scanning times of the labeling is 2. By speeding up the process, the sampling interval of the image data is narrowed, and when determining the number of moving bodies in the moving body area, the area ratio and the peripheral surface ratio are used as the judgment items in addition to the aspect ratio, and the movement between frames is performed. By associating the bodies, the number of moving bodies can be accurately counted.

As is clear from the second embodiment described above, according to the moving body counting apparatus of the second aspect, the image data is spatially differentiated to extract the head region which is a characteristic region of the person. Since there is no need to determine the number of moving bodies in the moving body area, the number of moving bodies can be accurately counted.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a moving body counting apparatus in a first embodiment and a second embodiment of the present invention.

FIG. 2 is a setting diagram of a CCD camera according to the first and second embodiments of the present invention.

FIG. 3 is an explanatory diagram of a moving body extraction process by a 3-frame method in the first embodiment of the present invention.

FIG. 4 is a flowchart of a first scanning unit of labeling processing according to the first embodiment of the present invention.

FIG. 5 is a flowchart of a second scanning unit of labeling processing according to the first embodiment of the present invention.

FIG. 6 is a flowchart of a grouping processing unit of labeling processing according to the first embodiment of the present invention.

FIG. 7 is an explanatory diagram of 4-direction data.

FIG. 8 is an example of a labeling process according to the first embodiment of the present invention.

FIG. 9 is a diagram showing an initialized state of pointer arrays p (k) and arrays q (k ′) in the grouping processing according to the first embodiment of this invention.

FIG. 10 is a diagram showing a state of pointer arrays p (k) and arrays q (k ′) after data1 processing and data2 processing in an example of the grouping processing of the first exemplary embodiment of the present invention.

FIG. 11 is a diagram showing a state of pointer arrays p (k) and arrays q (k ′) after data3 processing and data4 processing in an example of the grouping processing of the first exemplary embodiment of the present invention.

FIG. 12 is an explanatory diagram of moving body region determination in the first embodiment of the present invention.

FIG. 13 is a flowchart of a process for determining the number of moving bodies in a moving body area according to the first embodiment of the present invention.

FIG. 14 is an example of a mobile unit area in the first embodiment of the present invention (mobile unit area 1).

FIG. 15 is an example of a mobile unit area in the first embodiment of the present invention (mobile unit area 2).

FIG. 16 is an example of a mobile unit area in the first embodiment of the present invention (mobile unit area 3).

FIG. 17 is an example of a mobile unit area in the first embodiment of the present invention (mobile unit area 4).

FIG. 18 is an example of a feature amount calculation result and a set threshold value of the moving body regions 1 to 4 in the moving body number determination processing in the moving body region according to the first embodiment of the present invention.

FIG. 19 is a flowchart (main routine) of association and counting processing in the first embodiment of the present invention.

FIG. 20 is a flowchart (subroutine 1) of association and counting processing in the first embodiment of the present invention.

FIG. 21 is a flowchart (subroutine 2) of association and counting processing in the first embodiment of the present invention.

FIG. 22 is a diagram showing an example of association and count processing in the first embodiment of the present invention.

FIG. 23 is an explanatory diagram of a pointer value shift method in the first embodiment of the present invention.

FIG. 24 is a flowchart (main routine) of CPU processing in the first embodiment of the present invention.

FIG. 25 is a flowchart (subroutine) of CPU processing in the first embodiment of the present invention.

FIG. 26 is a diagram showing how the data area pointer changes in the pointer value shift method according to the first embodiment of the present invention.

FIG. 27 is the difference data Db in the second embodiment of the present invention.
And a part of the edge is broken.

FIG. 28 is an explanatory diagram of duplicated erasure by logical product in the second embodiment of the present invention.

FIG. 29 is a flowchart of association and counting by the first association method according to the second embodiment of the present invention.

FIG. 30 is a flowchart of an associating and counting method according to a second associating method in the second embodiment of the present invention.

FIG. 31 is an example of the association and count processing by the first association method in the second exemplary embodiment of the present invention.

FIG. 32 is an example of an associating and counting process by the second associating method in the second embodiment of the present invention.

FIG. 33 is an explanatory diagram of data shift in the conventional example.

FIG. 34 is a flowchart of a labeling process in a conventional example (main routine).

FIG. 35 is a flowchart of a labeling process in a conventional example (subroutine 1).

FIG. 36 is a flowchart of a labeling process in a conventional example (subroutine 2).

FIG. 37 is an example of labeling in the conventional example.

FIG. 38 is an explanatory diagram of the number of labeling scannings in the conventional example.

[Explanation of symbols]

 1 CCD Camera 2 A / D Converter 3 Digital Switch 4, 5, 6 Frame Memory 7 Data Memory 8 CPU 9 Program Memory-10 Data Input Means 11 Storage Means 12 Arithmetic Processing Means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display part H04N 7/18 K

Claims (2)

[Claims] 1. Data input means for obtaining continuous frame image data, storage means for storing image data input by the input means, and the image data by pointer shift while exchanging data with the storage means. The moving object area is extracted by taking the difference between frames, the moving object area is labeled by grouping using a pointer to separate each moving object area, and the aspect ratio of the labeled moving object area is determined. , The moving body is extracted for each frame by determining the number of moving bodies in the moving body region from the area ratio and the peripheral surface ratio, and the moving bodies are associated between the frames, and the moving bodies that cannot be matched are identified. A moving object power source, comprising: an arithmetic processing unit that counts as having moved outside the target area and outputs the count result. Cement equipment. 2. Data input means for obtaining continuous frame image data, storage means for storing the image data input by the input means, and spatial differentiation of the image data while exchanging data with the storage means. Then, the characteristic portion of the moving body is extracted by taking the difference between the frames of the spatially differentiated image data, and the moving body is extracted for each frame by labeling the characteristic portion, The moving object counting is performed by associating the moving objects with each other, counting the unmoving moving objects as having moved out of the target area, and outputting the count result. apparatus.