JP4674920B2 - Object number detection device and object number detection method - Google Patents

Description

  The present invention relates to a target number detection device and a target number detection method that can detect the number of a plurality of types of objects such as people and cars that move in an arbitrary direction by capturing a moving image.

  Currently, recognition of moving objects is an important issue in surveillance camera systems, intelligent road traffic systems, and robot vision. Also, by constantly monitoring and recording the flow of people and the degree of congestion, accidents that occur due to the concentration of people in one place can be prevented, availability / congestion information provided, and staff assignment plans in the store There is also a need to monitor the flow of people and the degree of congestion in order to use them for such strategies.

In the case of a system that automatically monitors the flow of people and the degree of congestion, it is necessary to be able to recognize the entire situation such as the flow and number of moving objects at high speed and robustly. However, it is very difficult to recognize a moving object automatically by a computer. Examples of factors that make recognition difficult include the following.
(1) There are a plurality of people and various types of moving objects such as bicycles in the camera image.
(2) Even in the same moving object, there are movements in various directions and speeds.
(3) The scale (size) of the object in the screen varies due to the distance between the camera and the object, the adult and child, and the height difference.

  There are many researches on detecting and recognizing moving objects, but most of them perform segmentation and tracking of moving objects, which incurs a calculation cost proportional to the number and type of objects, and makes many objects faster. And it was difficult to recognize with high accuracy. Also, the detection accuracy was low due to the difference in scale.

On the other hand, the following patent document 1 filed by the present inventors discloses a technique for extracting a higher-order local autocorrelation feature from a still image and estimating the number of objects using a multivariate analysis technique. Has been.
In addition, the present inventors have studied abnormal motion recognition that recognizes the difference in motion of an object from the entire image, and the following patent document 2 filed by the present inventors includes a three-dimensional higher-order local autocorrelation feature. (Hereinafter, also referred to as CHLAC data) has been disclosed.

Japanese Patent No. 2834153 JP 2006-079272 A

  When it is desired to know the overall situation such as the number of moving objects and their flow, position information of each object is not necessary. It is important to understand the overall situation: one person walking to the right, two people walking to the left, and one bike running to the left. It is fully possible to grasp the flow of people and the degree of congestion only by information about the situation and its changes.

  In the abnormal motion recognition technology described above, CHLAC data extracted from the entire moving image plane is used as the motion feature, and this CHLAC data is a position-invariant value that does not depend on the target location or time. Further, when there are a plurality of objects in the screen, there is an additive property that the entire feature value is the sum of the individual feature values. That is, for example, when there are two “people walking to the right”, the feature value is twice the feature value of one “person walking to the right”. Therefore, it is conceivable to apply CHLAC data to the detection of the number of moving objects and their moving directions.

When the above-mentioned CHLAC data is applied to the detection of the number of moving objects and their flow, the feature value varies depending on the scale (size) of the target on the video screen and the type of movement (speed and direction). Therefore, there has been a problem that the number detection accuracy is lowered.
An object of the present invention is to solve the problems of the conventional examples as described above, and to acquire a plurality of types of objects such as a person or a car that takes in a moving image and moves in a predetermined direction using a stereoscopic higher-order local autocorrelation feature The object number detection device and the object number detection method that can detect the number of objects with high accuracy by type are provided.

  The target number detection device according to the present invention includes a difference binary data generation unit that generates inter-frame difference data from video data including a plurality of image frame data obtained by photographing a detection target, and binarizes the difference data. A feature data extracting means for extracting feature data from the three-dimensional data composed of the binarized inter-frame difference data by a cubic higher-order local autocorrelation, and a plurality of factor vectors for one detection target generated in advance by learning are arranged. A coefficient acquisition means for obtaining a coefficient of each factor vector from the factor matrix and the feature data, an addition means for adding a plurality of the coefficients for one detection target, and rounding off the decimal point of the output value of the addition means The main feature is that it is provided with an integerizing means for generating the number by converting it into an integer.

  In addition, the above-described target number detection apparatus is further characterized in that it further includes a learning unit that generates a factor matrix based on feature data obtained from the learning data. In the above-described target number detection device, the learning unit generates difference data between frames from moving image data including a plurality of image frame data obtained by photographing a detection target that is learning data, and binarizes the difference. Using featured data generating means, feature data extracting means for extracting feature data from three-dimensional data consisting of a plurality of inter-frame difference binarized data by means of high-order local autocorrelation, and a known number of objects in learning data In addition, there is a feature in that a factor matrix generation unit that obtains a factor matrix by factor analysis from the feature data corresponding to a plurality of learning data is provided.

  Further, in the above-described target number detection device, a plurality of factor vectors corresponding to one detection target included in the factor matrix are a plurality of different at least one of a scale, a moving speed, and a moving direction of the target on the screen. It is also characterized in that it is generated from learning data.

  Another target number detection apparatus according to the present invention includes a difference binary data generation unit that generates inter-frame difference data from moving image data including a plurality of image frame data obtained by photographing a detection target, and binarizes the difference data. Feature data extracting means for extracting feature data from three-dimensional data comprising a plurality of inter-frame difference binarized data by means of three-dimensional local autocorrelation, and at least one of a scale, a moving speed, and a moving direction of a target on the screen A learning means for generating a coefficient matrix for calculating the number of detection targets based on feature data obtained from a plurality of different learning data, a coefficient matrix previously generated by the learning means, and recognition data. A number generation means for obtaining a number from the obtained feature data; and an integerization means for rounding off an output value of the number generation means to an integer. The main feature that.

  The target number detection method of the present invention includes a step of generating a factor matrix based on three-dimensional higher-order local autocorrelation based on learning data, and inter-frame difference data from moving image data composed of a plurality of image frame data obtained by photographing the detection target. A step of generating and binarizing, a step of extracting feature data by three-dimensional high-order local autocorrelation from three-dimensional data composed of a plurality of latest inter-frame difference binarized data, A step of obtaining a coefficient of each factor vector from the factor matrix in which a plurality of factor vectors are arranged for the detection target and the feature data, a step of adding the plurality of the coefficients for one detection target, and a decimal point of the output value of the adding means The main feature is that it includes a step of generating a number by rounding to an integer.

The present invention has the following effects.
(1) For one detection target, prepare a factor matrix in which multiple factor vectors corresponding to targets with different scales and moving speeds are prepared in advance by learning using factor analysis, and add the coefficients of each factor vector during recognition Then, the number is generated by rounding off to an integer, so that the variation in the sum of the coefficients is small and matches the number of objects to be recognized with high accuracy. Therefore, robust recognition can be made with respect to differences in scale, speed and direction of the object and dynamic changes thereof, and the number detection accuracy is improved.
(2) Since a plurality of objects are recognized at the same time without cutting out the objects, the amount of calculation for feature extraction and number identification determination is small. Further, the calculation amount is constant regardless of the number of objects. Therefore, real-time processing is possible.
(3) A coefficient matrix can be generated in advance by learning based on multiple regression analysis using images of objects with different scales, moving speeds, and directions, and the number can be calculated directly and at high speed. Robust detection of speed, direction and scale is possible.

It is a block diagram which shows the structure of the object number detection apparatus by this invention. It is explanatory drawing which shows the outline | summary of the object number detection process by this invention. It is explanatory drawing which shows the autocorrelation process coordinate in a three-dimensional pixel space. It is explanatory drawing which shows the example of an autocorrelation mask pattern. It is explanatory drawing which shows the content of the real-time process of the moving image by this invention. It is explanatory drawing which shows the example of the factor matrix produced | generated in learning mode. It is a flowchart which shows the content of the object number detection process (learning mode) by this invention. It is a flowchart which shows the content of the object number detection process (recognition mode) by this invention. It is a flowchart which shows the content of the pixel CHLAC data extraction process of S13.

  In the following embodiments, an example in which the object is a person walking to the left or right will be described. However, the object may be any moving body or moving body that can be photographed as a moving image, shape, size, color, brightness, etc. It can be applied to an object in which one of the above changes.

FIG. 1 is a block diagram showing the configuration of a target number detection apparatus according to the present invention. The video camera 10 outputs moving image frame data of a target person or car in real time. The video camera 10 may be a monochrome camera or a color camera. The computer 11 may be a known personal computer (PC) provided with a video capture circuit for capturing a moving image, for example. The present invention is realized by creating, installing, and starting a processing program to be described later in any known computer 11 such as a personal computer.
The monitor device 12 is a well-known output device of the computer 11 and is used, for example, to display the detected number of objects to the operator. The keyboard 13 and the mouse 14 are well-known input devices used for input by the operator. In the embodiment, for example, moving image data input from the video camera 10 may be processed in real time, or may be temporarily stored after being stored in a moving image file and processed. Further, the video camera 10 may be connected to the computer 11 via an arbitrary communication network.

FIG. 2 is an explanatory diagram showing an outline of the target number detection processing according to the present invention. For example, a 360 × 240 pixel gray scale (monochrome multi-value) moving image is captured by the video camera 10 and is sequentially captured by the computer 11.
The absolute value of the difference from the captured frame data (a) with the luminance value of the same pixel in the immediately preceding frame is obtained, and a difference 2 is set to 1 if this value is equal to or greater than a predetermined threshold, for example, and 0 otherwise. The valued frame data (c) is obtained. Next, CHLAC data is calculated for each pixel from the last three difference binary frame data (d) by a method described later, and this pixel-corresponding CHLAC data is added by one frame to obtain frame-corresponding CHLAC data (f ) The above processing is common to the learning mode and the recognition mode.

  In the learning mode, by executing the process (h) of adding frame-corresponding CHLAC data (g) in a predetermined region (for example, 30 frames in time width), a plurality of learning data-corresponding CHLAC features corresponding to a plurality of learning data Ask for data. Then, a factor matrix is obtained by factor analysis (i) using the number information of each factor of the known object in the learning data. The factor matrix (j) corresponds to one object, for example, “a person who walks to the right” and “a person who has a large scale that walks quickly to the right”, “a person who has a small scale that normally walks to the right”, etc. The factor vector data of are arranged.

On the other hand, in the recognition mode, (M) CHLAC feature data is obtained by executing the process (l) for adding the frame-corresponding CHLAC data (k) in the most recent predetermined area (for example, 30 frames of time width). Then, using the factor matrix (j) obtained in advance in the learning mode (N), the number of objects is estimated by a method described later.
In the number estimation process (N), the coefficient of each factor vector is obtained, a plurality of coefficients corresponding to one object are added, and then the number is calculated by rounding off the fractional part. This process makes it possible to recognize the difference in the scale and speed of the target and the dynamic changes thereof.

  Details of the processing will be described below. FIG. 7 is a flowchart showing the contents of the target number detection process (learning mode) according to the present invention. In S10, unprocessed learning data is selected. The learning data is for two types of subjects, for example, “person walking to the right” and “person walking to the left”, each with a different speed of movement (normal, fast, running) and scale (large (near), medium , Small (far)) moving image data. Any number of two types of objects may be mixed. Note that the number of objects, the moving speed, and the scale in the learning data are known. At this time, the learning data correspondence CHLAC data is cleared.

  In S11, frame data is input (read into a memory). The image data at this time is, for example, 256 gray scale data. In S12, "motion" information is detected for the moving image data, and difference data is generated for the purpose of removing stationary objects such as the background.

  As a method of taking the difference, an inter-frame difference method that extracts a change in luminance of a pixel at the same position between adjacent frames is adopted, but an edge difference that extracts a luminance change portion in a frame or both are adopted. May be. If each pixel has RGB color data, the distance between the two RGB color vectors may be calculated as difference data between the two pixels.

  Further, binarization is performed by automatic threshold selection to remove color information and noise unrelated to “movement”. As a binarization method, for example, a fixed threshold value, a discriminative least square automatic threshold value method disclosed in Non-Patent Document 1 below, a threshold value 0, and a noise processing method (all differences except for a difference of 0 in a grayscale image have motion = 1 and a method of removing noise by a known noise removing method).

In the discriminative least squares automatic threshold method, noise is detected in a scene where there is no target at all. Therefore, when the threshold value of the luminance difference value to be binarized is smaller than a predetermined lower limit value, the lower limit value is set as the threshold value. By the above preprocessing, the input moving image data becomes a column of frame data (c) having logical values of “moved (1)” and “not moved (0)” as pixel values.
Noriyuki Otsu, “Automatic threshold selection method based on discriminant and least-squares criteria” IEICE Transactions D, J63-D-4, P349-356, 1980.

In S13, pixel CHLAC data which is 251 dimensional feature data is extracted for each pixel of one frame, and is added by one frame to obtain frame-corresponding CHLAC data.
Here, the three-dimensional higher order local autocorrelation (CHLAC) feature will be described. The Nth-order autocorrelation function can be expressed as the following Equation 1.

Here, f is a pixel value (difference value), and N displacements a _i (i = 1,..., N) viewed from the reference point (target pixel) r and the reference point are two in the difference binarized frame. It is a three-dimensional vector having dimensional coordinates and time as components.

  A high-order autocorrelation function can be considered innumerable depending on the direction of displacement and the order of the order, but the high-order local autocorrelation function is limited to a local region. In the three-dimensional high-order local autocorrelation feature, the displacement direction is limited to a 3 × 3 × 3 pixel local area centered on the reference point r, that is, in the vicinity of 26 of the reference point r. The integral value of Equation 1 corresponding to a set of displacement directions is one element of the feature amount. Therefore, the feature amount elements are generated in the number of combinations of the displacement directions (= mask patterns).

  The number of elements of the feature quantity, that is, the dimension of the feature vector corresponds to the type of mask pattern. In the case of a binary image, since the pixel value 1 is multiplied by 1 regardless of how many times it is, a term of square or higher is deleted as an overlapping term of a square with a different multiplier only. In addition, patterns that overlap in the integration operation (parallel movement: scan) of Equation 1 are deleted while leaving one representative pattern. Since the expression on the right side of Equation 1 always includes a reference point (f (r): the center of the local area), the representative pattern includes the center point, and the entire pattern fits within the local area of 3 × 3 × 3 pixels. select.

  As a result, the number of types of mask patterns including the center point is 352 with a total number of selected pixels of one: one, two: 26, three: 26 × 25/2 = 325. However, if overlapping patterns are removed by the integration operation (parallel movement: scan) of Equation 1, there are 251 types of mask patterns. That is, the three-dimensional higher-order local autocorrelation feature vector for one three-dimensional data is 251 dimensions.

  When the pixel value is a multi-value gray image, for example, when the pixel value is a, the correlation value is a (0th order) ≠ a × a (primary) ≠ a × a × a (secondary). Thus, even if the selected pixels are the same, those having different multipliers cannot be redundantly deleted. Therefore, in the case of multi-value, the number is 2 when the number of selected pixels is 1 and 26 when the number of selected pixels is 2, and the number of mask patterns is 279 in total.

  FIG. 3 is an explanatory diagram showing autocorrelation processing coordinates in a three-dimensional pixel space. In FIG. 3, the xy planes of three difference frames of t−1 frame, t frame, and t + 1 frame are shown side by side. In the present invention, a correlation is obtained for pixels inside a cube of 3 × 3 × 3 (= 27) pixels centered on the reference pixel of interest. The mask pattern is information indicating a combination of pixels to be correlated, and data of pixels selected by the mask pattern is used for calculation of correlation values, but pixels not selected by the mask pattern are ignored. In the mask pattern, the target pixel (center pixel: reference point) is always selected.

  FIG. 4 is an explanatory diagram showing an example of an autocorrelation mask pattern. FIG. 4A shows the simplest 0th-order mask pattern of only the target pixel. (2) is a primary mask pattern example in which two hatched pixels are selected, (3) and (4) are tertiary mask pattern examples in which three hatched pixels are selected, There are many other patterns. As described above, the number of types of mask patterns is 251 except for overlapping patterns. That is, the three-dimensional high-order local autocorrelation feature vector for 3 × 3 × 3 pixel three-dimensional data has 251 dimensions, and the element value is 0 or 1.

  Returning to FIG. 7, in S14, the frame-corresponding CHLAC data is added to the learning data-corresponding CHLAC data for each element. In S15, it is determined whether or not the processing of all frames of the learning data has been completed. If the determination result is negative, the process proceeds to S13, but if the determination is affirmative, the process proceeds to S16. In S16, CHLAC data corresponding to learning data is stored. In S17, it is determined whether or not the processing of all the learning data is completed. If the determination result is negative, the process proceeds to S10, but if the determination is affirmative, the process proceeds to S18.

  In S18, factor analysis is performed based on the number data of known factors to obtain a factor matrix. Here, factor analysis will be described. First, the factor in the embodiment is the type of the object identified by the shape, scale, moving speed, and the like. In the embodiment, for example, “a person with a large scale walking to the right at a normal speed” in one object “a person walking to the right” is one factor. It becomes.

For example, a three-dimensional higher-order local autocorrelation feature vector extracted from learning data in which only one factor exists on the screen is equivalent to the factor vector. That is, the factor vector is a characteristic vector unique to each factor.
Here, if a moving image as stereoscopic data is composed of a combination of m factor vectors f _j (0 ≦ j ≦ m−1), the cubic higher-order local autocorrelation feature z obtained from the stereoscopic data is additive. And by position invariance, the linear combination of f _j is expressed as:

Here, F is defined as a factor matrix, a coefficient a _j when expressed by linear combination is defined as a factor addition amount, and a obtained by arranging the coefficients a and b is defined as a factor addition vector. E represents an error. The factor addition amount represents the number of objects corresponding to the factor. For example, if f ₀ is a factor that a person walks to the right, a ₀ = 2 indicates that there are two people walking to the right in the moving image. Therefore, if a factor addition vector can be obtained, it can be understood how many objects are present in the screen. For this purpose, a factor matrix is acquired in advance by learning, and a factor addition vector is obtained using the factor matrix during recognition.

In the learning mode, a factor matrix F = [f ₀ ; f ₁ ;..; F _m−1 ] ^T is obtained. As the teacher signal, a factor addition vector a which is the number corresponding to each factor is given. The specific learning process is described below. The number of moving image data used as learning data is N, the cubic higher-order local autocorrelation feature corresponding to the i-th (1 ≦ i ≦ N) learning data is z _i , and the factor addition vector is a _i = [a _i0 ; a _i1 ;... a _{i (m−1)} ]. At this time, the factor matrix F can be obtained explicitly by minimizing the error e in Equation 3 below.

  The mean square error of Equation 3 is as follows.

R _aa and R _az autocorrelation matrix of each a _i, a cross-correlation matrix of a _i and z _i. At this time, F that minimizes the error e solves Equation 5 below, and the solution is obtained explicitly in the range of linear algebra as in Equation 6.

The advantages of this learning method are the following three points.
(1) There is no need to cut out and teach individual objects.
(2) The system automatically and adaptively acquires factors necessary for recognition simply by teaching the number of objects in the screen.
(3) Since the solution is obtained explicitly in the range of linear algebra, it is not necessary to consider the convergence of the solution or the convergence to the local solution, and the amount of calculation is small.

FIG. 6 is an explanatory diagram illustrating an example of a factor matrix generated in the learning mode. In this example, a factor matrix including two types of “person walking to the right” and “person walking to the left” is shown as a target. The “person who walks to the right” has nine factor vectors f _{0 to} f ₁₆ (even subscripts) with different moving speeds (running, fast walking, normal walking) and scales (large, medium, small), Nine factor vectors f _{1 to} f ₁₇ (subscripts are odd numbers) belong to the “person walking to the left”. The image shown in FIG. 6 is an example of a binary difference image of learning data corresponding to each factor vector.

  FIG. 8 is a flowchart showing the contents of the target number detection process (recognition mode) according to the present invention. In S20, the process waits until a frame is input. In S21, frame data is input. In S22, difference data is generated and binarized as described above. In S23, pixel CHLAC data is extracted for each pixel of one frame, and one frame is added to obtain frame-corresponding CHLAC feature data. The processes of S21 to S23 are the same as S11 to S13 in the learning mode described above. In S24, the frame-corresponding CHLAC data is stored. In S25, CHLAC feature data is obtained by adding the frame-corresponding CHLAC data of the latest predetermined time width.

  FIG. 5 is an explanatory diagram showing the contents of real-time processing of a moving image according to the present invention. The CHLAC feature data obtained in S24 is a frame sequence. Therefore, a time window having a certain width is set in the time direction, and a frame set in the window is set as one three-dimensional data. Each time a new frame is input, the time window is moved, and the old frame is deleted to obtain finite three-dimensional data. The length of this time window is desirably set equal to or longer than one cycle of the operation to be recognized.

  Actually, only one frame is stored in order to obtain a difference in image frame data, and frame-corresponding CHLAC data corresponding to the frame is stored for a time window. That is, when a new frame is input at time t, the frame-corresponding CHLAC data corresponding to the immediately preceding time window (t−1, t−n−1) has already been calculated. However, in order to calculate the frame CHLAC data, the last three difference frames are necessary, but since the (t-1) frame is the end, the frame CHLAC data is calculated up to the one corresponding to the (t-2) frame. Yes.

  Therefore, using the newly input t frame, frame-corresponding CHLAC data corresponding to the (t−1) frame is generated and added to the CHLAC feature data. In addition, the CHLAC data corresponding to the oldest (t-n-1) frame is subtracted from the CHLAC feature data. Through such processing, the CHLAC feature data corresponding to the time window is updated.

  Returning to FIG. 8, in S26, a factor addition amount (coefficient) a of each factor vector is obtained based on a known factor matrix obtained by learning. When there is a three-dimensional higher-order local autocorrelation feature z obtained from a moving image to be recognized, z should be expressed as a linear combination of factor vectors f obtained by learning, as shown in Equation 3. Accordingly, at this time, a factor addition vector a which is a vector of coefficients that minimizes the error e is obtained.

  A specific process for obtaining the factor addition amount a that minimizes the error e in Equation 3 will be described below. The least square error is expressed by Equation 7 below.

  The coefficient a that minimizes this is obtained explicitly by solving Equation 8 below and Equation 9.

The factor addition amount a obtained in this way is not an integer, but a real number including less than a decimal point. In S27, the sum of the coefficients of a plurality of factors belonging to the same object is obtained. That is, for example, the sum of the coefficients of nine factors (f ₀ , f ₂ , f ₄ ... F ₁₆ ) belonging to the “person moving to the right” shown in FIG.
In S28, the fraction of the total of the coefficients is rounded off to an integer and output as the number for each object. In S29, it is determined whether or not the process is ended. If the determination result is negative, the process proceeds to S20, but if the determination is affirmative, the process ends.

  In the conventional number recognition using CHLAC data, the factor addition amount, which is the coefficient of each factor, is simply rounded off and used as the number recognition result. However, it did not recognize the number when there were factors with different scales and speeds. Therefore, as a result of various experiments conducted by the present inventor, for one object, the factors are divided according to the difference in scale and walking speed in the screen, and the factor addition amount of the factors belonging to the same object is added. It was found that using the method of rounding off makes it possible to recognize robustly the difference in speed and speed.

  FIG. 9 is a flowchart showing the contents of the pixel CHLAC data extraction process in S13. In S30, the correlation value group data corresponding to 251 correlation patterns is cleared. In S31, one unprocessed pixel (reference point) is selected (a pixel of interest that is a reference point in the frame is sequentially scanned). In S32, one unprocessed correlation mask pattern is selected.

In S33, the correlation value is calculated by multiplying the difference value (0 or 1) of the position corresponding to the pattern by using the above-described Expression 1. This process corresponds to the calculation of f (r) f (r + a1)... F (r + aN) in the above-described equation 1.
In S34, it is determined whether or not the correlation value is 1. If the determination result is affirmative, the process proceeds to S35, but if not, the process proceeds to S36. In S35, 1 is added to the correlation value data corresponding to the mask pattern. In S36, it is determined whether or not processing has been completed for all mask patterns. If the determination result is affirmative, the process proceeds to S37, but if not, the process proceeds to S32.

  In S37, it is determined whether or not the processing has been completed for all pixels. If the determination result is affirmative, the process proceeds to S38, but if not, the process proceeds to S31. In S38, the correlation value group data with one frame added is output as frame-corresponding CHLAC data.

  In the factor analysis of the first embodiment, in order to obtain a measurement result to be obtained, a factor vector specific to the type, movement, scale, etc. of each moving object is obtained during learning, and the target is in the form of the sum of coefficients of each factor vector during recognition. The number of was determined. At this time, factors corresponding to the difference in scale and speed were prepared, and those coefficients were added and then rounded off, so that robust recognition of changes in the target scale and speed could be achieved. Since this method obtains a feature vector corresponding to each factor, it is useful for methods using them, for example, traffic volume measurement and abnormality detection.

However, as a result of experiments, if you want to know only the number, it is possible to measure the number rapidly and robustly by using multiple regression analysis, which is a more direct method than factor analysis. found.
In order to perform robust recognition with respect to scale and speed using multiple regression analysis, learning is performed using learning data including targets of various scales and speeds as in the case of factor analysis. However, the teacher signal for the learning data is given differently from the factor analysis.

  In factor analysis, teacher signals including differences in scale and speed are used and the coefficients to be detected are added at the time of recognition. In multiple regression analysis, the signals added in advance at the stage of the teacher signal are used. That is, a teacher signal that ignores differences in scale and speed is used.

  For example, if there is data that includes large, medium, and small scales as "person walking to the right", the factor analysis divides them and gives a teacher signal so that "one person with large scale walking to the right" is one person. Regression analysis simply gives the number of people walking to the right, ignoring differences in scale and speed. At the time of recognition, it is not necessary to perform addition, and it is possible to measure the number of people robustly due to differences in scale and speed. Hereinafter, specific contents will be described.

The multiple regression analysis used in Example 2, when the desired measurement result when a feature quantity z _i obtained was a _i, the smallest least squares error of the output y _i = B ^T z _i and a _i This is a method for determining such a coefficient matrix B. In this case, the optimum coefficient matrix is uniquely obtained, and by using the obtained optimum coefficient matrix B, the system can calculate the measurement value (number) for the new input feature vector at high speed. A detailed calculation method will be described below.

<< Learning Phase >>
The number of three-dimensional data used as learning data, that is, the number of learning data is N, the three-dimensional higher-order local autocorrelation feature corresponding to the i-th (1 ≦ i ≦ N) three-dimensional data is z _i , and the teacher signal is a _i = [a _{i 0} , a _i1 ,..., a _{i (m−1)} ] ^T. The teacher signal ignores the difference in scale and speed, even if the learning data is data that has "people walking to the right" and "people walking to the left" with various scales and speeds. number, and number) ^T of the person walking to the left. The mean square error between the teacher signal a _i and the output y _i = B ^T z _i is as follows.

R _zz and R _za are each a cross-correlation matrix of the autocorrelation matrix and z _i and a _i of z _i. At this time, B, which minimizes the mean square error, is obtained explicitly in the range of linear algebra as shown in Expression 12 by solving Expression 11 below.

<< Recognition phase >>
In recognition, the number of objects can be directly obtained by multiplying the obtained feature vector by the coefficient matrix B obtained in the learning phase as follows.

  When multiple regression analysis is used, each factor vector is not obtained directly, so additional detections are required for anomaly detection using the distance from the subspace formed by each factor vector and traffic measurement. Information is not available. Therefore, it is necessary to use the methods of the first embodiment and the second embodiment depending on the object and the situation. Further, by combining the two methods, it is possible to improve both the processing speed and the recognition accuracy.

  As mentioned above, although the Example was described, this invention is applicable to the traffic measurement system which measures the number of the vehicles and the person who passed the inside of a screen, for example. In the system of the embodiment, the number of objects in the screen is output in real time. For example, the passing number per hour cannot be obtained directly from the system of the embodiment. Therefore, the number of passing objects per unit time is obtained by time integrating the number information output from the system of the present invention, and dividing this integrated value by the average time passing through the screen determined from the average moving speed of the object. Can be calculated. The average time for the object to pass through the screen can also be estimated from the variation in the number information output from the system of the present invention.

  Also, the present invention can be modified as follows. In the embodiment, an example in which a plurality of factor vectors having different scales, moving speeds, and the like for one target are all generated from learning data by factor analysis has been disclosed. For example, a factor vector corresponding to a large scale and a small scale are supported. The factor vector may be obtained from other factor vectors by interpolation or extrapolation, such as by generating a factor vector corresponding to the scale from the factor vector.

  In the embodiment, an example in which various learning data is used for the scale and speed of the target image is disclosed. However, the number of the movements of the target can be as robust as the scale and speed. For example, as an application example of robust counting using factor analysis, it is possible to photograph a person walking in various directions from above and measure the total number of persons moving in any direction.

  As factors for the direction in which a person walks, for example, eight directions of up and down, left and right, diagonally upper right (lower) and diagonally upper left (lower) are adopted. And it learns factors in 8 directions. At the time of recognition, each factor addition amount is obtained using the learned factor matrix, and the factor addition amount is added as in the case of scale and speed, and rounded off to obtain the number of pedestrians. In addition, as a direction to prepare, it can increase / decrease according to a use. When using multiple regression analysis, the directionality is ignored and only the number of pedestrians is used as a teacher signal.

  By the above method, the number of objects can be measured robustly even when an object moving in various directions is handled. Practical applications include (scramble) counting the number of pedestrians and vehicles using a camera that has been photographed from the top, etc., counting the number of moving organisms and particles, especially the number of fine organisms and particles using a microscope. . It is possible to compare the number of objects that are stationary and those that are moving, and to analyze movement trends.

DESCRIPTION OF SYMBOLS 10 ... Video camera 11 ... Computer 12 ... Monitor apparatus 13 ... Keyboard 14 ... Mouse

Claims (2)

Differential binary data generation means for generating inter-frame difference data from moving image data composed of a plurality of image frame data obtained by photographing a detection target, and binarizing the difference data;
Feature data extraction means for extracting feature data by three-dimensional higher-order local autocorrelation from three-dimensional data composed of a plurality of latest inter-frame difference binarization data;
Generating a coefficient matrix for calculating the number of detection targets by multiple regression analysis based on the feature data obtained from a plurality of learning data having different scales and movement speeds on the screen; in regression analysis, the scale of the object, as a teacher signal of the moving speed is different training data, the scale of the object, ignoring the difference in moving speed, and the learning means using a numerical obtained by adding the number of the target,
Using the coefficient matrix obtained in advance, a number generating means for obtaining the number from the feature data obtained from the data to be recognized,
An object number detecting apparatus comprising: an integerizing means for rounding off an output value of the number generating means to an integer. A step in which a computer generates inter-frame difference data from moving image data composed of a plurality of image frame data obtained by photographing a detection target, and binarizes the data;
A computer extracting feature data by means of three-dimensional higher-order local autocorrelation from three-dimensional data consisting of a plurality of the binarized inter-frame difference data,
The computer generates a coefficient matrix for calculating the number of detection targets by multiple regression analysis based on the feature data obtained from a plurality of learning data having different scales and movement speeds on the screen. step wherein the multiple regression analysis, the scale of the object, as a teacher signal of the moving speed is different training data, the scale of the object, ignoring the difference in moving speed, using a numerical value obtained by adding the number of target ,
A computer using the coefficient matrix to determine a number value from feature data obtained from the data to be recognized;
The computer includes a step of rounding off the decimal value of the number value to an integer, and the target number detection method.

Images