JPH1125278A - Behavior analysis estimating device for traveling object and its waviness association processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the reliability and estimation accuracy of a cross- correlation coefficient in the case of a rigid and non-rigid object accompanying gray level change by analyzing similarity based on a cross-correlation method, changing a mean gray level of two images about an average of gray levels in a small block to find maximum similarity and estimating a motion vector. SOLUTION: An image inputting part 11 uses a camera, etc., and inputs a time series image. An image accumulating part 12 accumulates it as a time series image. A comparing part 13 splits each image into small blocks among not less than two continuous two-dimensional images, extracts image feature quantity and compares image feature quantity at different times. A compensation cross-correlating part 14 finds maximum similarity by analyzing similarity based on a cross-correlation method, changing a mean gray level of two images about an average of a gray level in a small block, estimates a motion vector, performs waviness association processing when waviness exists in the gray level and performs analysis estimation of the behavior of a traveling object.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the field of detecting the movement of a moving object with a camera / radar and analyzing and predicting the movement, and more particularly to predicting the change in precipitation from weather radar echo images and the behavior of fluid in fluid engineering. Analysis prediction, such as analysis of

[0002]

2. Description of the Related Art Conventionally, when estimating a movement vector of an object having a rigid system in an image, a model having almost no change in illumination on the object is often used. The detection method is
For example, as shown in the document “Computer Vision” by Donna H. Ballard, Computer Vision Association of Japan, a method based on an optical flow or a cross-correlation method is mainly used.

On the other hand, it can be said that there is no appropriate movement vector for a non-rigid object. This is due to the fact that attributes such as contours and gray levels of an object change at the same time even between consecutive images, so that a clear association cannot be made. That is, for a complex object, a cross-correlation method that follows statistical similarity is often applied. For example, when a weather radar echo image is used, the precipitation pattern changes non-rigidly, but the cross-correlation method is applied.

In any case, when estimating the movement vector of an object between images based on the analysis result of the cross-correlation method, it is assumed that there is almost no change in the gray value distribution of the object between images. Have been. In the case of a rigid system, it is assumed that there is almost no change in lighting, and in the case of a non-rigid system, a model is assumed that there is no generation or disappearance due to a convection phenomenon especially in the case of a weather radar image. In addition, it is assumed that the gray value of the object is noise-free.

[0005]

However, in the case of a rigid or non-rigid object, the distribution of gray values actually changes, and the prediction based on the analysis of the behavior assuming that there is no change in the gray value distribution, Naturally, there is a problem that an error is generated between the actual change and the actual change.

An object of the present invention is to improve the reliability of a cross-correlation coefficient and to improve the prediction accuracy when a rigid or non-rigid object is accompanied by a change in gray value, and a behavior analysis prediction device therefor. An object of the present invention is to provide a swell correspondence processing method.

[0007]

SUMMARY OF THE INVENTION The present invention provides a behavior analysis / prediction device for a moving object, which is a behavior analysis / prediction device using an image of a moving object, comprising: an image input means for inputting an image; Each image is divided into small blocks between an image accumulating unit that accumulates as a time-series image and two or more continuous two-dimensional images accumulated in the image accumulating unit, and an image feature amount is extracted by image processing. Means for comparing image feature values at different times by using a cross-correlation method based on a cross-correlation method while changing the average gray value of two images with respect to the average value of gray values in a small block obtained by the comparing means. By analyzing the characteristics and finding the maximum similarity point and estimating the movement vector, when the swell exists in the grayscale value, it is possible to estimate the movement vector corrected by associating the swell. Seki and means, and output means for outputting the result of the analysis.

The undulation correspondence processing method of the behavior analysis / prediction device of the moving object according to the present invention is a undulation correspondence processing method of the behavior analysis / prediction device using the image of the moving object, wherein the image of the movability object is time-series. A procedure for inputting by the image input means, a procedure for storing the input time-series images in the image storage means, and a comparing means for extracting two or more continuous two-dimensional images from the plurality of images stored in the image storage means. A procedure of dividing each extracted image into small blocks, extracting image feature amounts by image processing, and comparing image feature amounts at different times, and correcting cross-correlation means, in the small block obtained by the comparison means. The similarity is analyzed based on the cross-correlation method while the average gray value of the two images is changed with respect to the average value of the gray value of, and the maximum similarity point is found to estimate the movement vector. If kneading is present comprises a step of estimating a motion vector by performing a correction by undulation association process, a procedure for outputting the output means the result of the analysis by correcting the cross-correlation means.

In the procedure for obtaining the image feature amount, the difference between the images is obtained and extracted as the spatial frequency component of the temporal variation of the gray value. In the procedure for estimating the movement vector, the periodic component and the variation of the gray value are calculated. It is preferable to perform the cross-correlation analysis while corresponding to the time direction and correcting the average value of the grayscale values between the correlated images accordingly.

In the procedure for estimating a motion vector in the correction cross-correlation means, a procedure for linearly predicting a motion vector by a linear extrapolation method using a motion vector obtained by analyzing similarity between two images as an initial vector. And a procedure for predicting fluid fluctuation by applying the advection term of the fluid equation.

In the procedure for comparing image feature values, it is preferable to perform a smoothing process and an isolated point removing process on each image as preprocessing.

It is sufficiently conceivable that various gray levels are added to the gray level distribution of a fluid-based object observed by a radar or a camera, thereby disturbing the true gray level distribution. In particular, the rainfall phenomenon depends on the strength of the convection change, and the grayscale value (true value) itself greatly fluctuates as an image feature amount. Therefore, when estimating the movement vector using the statistical similarity analysis by the cross-correlation method, it is appropriate to use the result of the cross-correlation method while correcting the grayscale value. When the swell of the gray value due to convection is found, the spatial frequency component is analyzed, and the gray value is corrected between the correlated images while associating the periodic component and the gray value fluctuation in the time direction.

[0013]

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram of a moving object behavior analysis prediction device provided with a swell correspondence processing method and a swell correspondence processing method according to an embodiment of the present invention, and FIG.
Is a block diagram of a moving object behavior analysis prediction device,
(B) is a flowchart of a swell correspondence processing method according to the embodiment of the present invention. In the figure, reference numeral 11 denotes an image input unit,
12 is an image storage unit, 13 is a comparison unit, 14 is a corrected cross-correlation unit, 15 is an output unit, and S101 to S111 are processing procedures.

A behavior analysis and prediction apparatus using an image of a moving object according to an embodiment of the present invention includes an image input unit 11 for inputting a time-series image using a camera or radar, and the input image as a time-series image. Each image is divided into small blocks between an image storage unit 12 to be stored and two or more continuous two-dimensional images stored in the image storage unit 12, and image features are extracted and different by image processing to be different. Based on the cross-correlation method, the comparing unit 13 that compares the image feature amounts at the time and the average value of the gray values in the small blocks obtained by the comparing unit 13 are changed while changing the average gray value of the two images. Correction cross-correlation unit 14 that analyzes the characteristics and finds the maximum similarity point to estimate the movement vector, performs swell association processing when swell exists in the grayscale value, and analyzes and predicts the behavior of the moving object. , And an output means for outputting the result of the analysis by correcting the cross-correlation unit 14.

In the undulation associating processing method according to the embodiment of the present invention, when processing is started (S101), images of a moving object are input in time series by the image input unit 11 using a camera or radar (S102). The time-series image input by the storage unit 12 is stored (S103).

The comparison unit 13 extracts two or more continuous two-dimensional images from the plurality of images stored in the image storage unit 12 (S104), divides each extracted image into small blocks (S105), The image features are extracted by the processing (S106), and the image features at different times are compared (S107).

The correction cross-correlation unit 14 corrects the average value of the gray values in the small blocks obtained by the comparison unit 13 while changing the average gray value of the two images, and performs cross-correlation. The similarity is analyzed based on the method (S
108), the maximum similarity is found, and the movement vector is estimated (S109). By analyzing the similarity while changing the average gray value of the two images, if there is a undulation in the gray value, a undulation association process is performed and the result is corrected.

The result analyzed by the output unit 15 is output (S110), and the process is terminated (S111).

The supplementary description of the extraction of two or more two-dimensional images in S104 will be given. Basically different time 2
Image processing such as detection of a moving amount can be performed using only one image. However, the actual precipitation patterns are often complex and non-stationary,
It is desirable to include averaging. As one of the methods, for example, six consecutive frames in the past are extracted, the first three frames and the second three frames of the extracted six frames are averaged once, respectively, and the moving amount is calculated using the average value of the first half and the average value of the second half. Image processing such as detection of

When obtaining the image feature value in S106, the difference between the images to be compared is calculated and extracted as the spatial frequency component of the temporal variation of the gray value, and the periodic component and the variation of the gray value are extracted in the time direction. In step S108, the cross-correlation analysis is performed while correcting the average value of the grayscale values between the images to be correlated.

In the comparison of the image feature amounts at different times in S107, a smoothing process and an isolated point removing process are performed on each image as preprocessing.

In the processing in the correction cross-correlation unit 15, a procedure of linearly predicting a moving vector by a linear extrapolation method using a moving vector obtained by analyzing the similarity between the two images in S109 as an initial vector. Predicting fluid fluctuations by applying the advection term of the fluid equation.

FIGS. 2A and 2B show an example of a change in a time-series pattern having a change in gray value. FIG. 2A shows a first example, and FIG. 2B shows a second example. Visually, in both the first example and the second example, a clear pattern can be seen in the pattern, but when the cross-correlation coefficient of each set is calculated, the pattern becomes considerably small. This is because, because the correlation function uses the gray value as a variable, if the average value of the gray values is large, the correlation coefficient will be low regardless of the similarity of the contour shapes.

FIG. 3 is a graph before and after the correction of the average gray value distribution and the cross-correlation coefficient change of two images having different successive times. Even if there is a similarity in the outline shape, the similarity in the gray value may be low. In terms of temporal transition, the average gray level change differs even for continuous images, and the cross-correlation coefficient becomes 0.5 or less.

When comparing the images at the two times, the cross-correlation coefficient is normalized and the cross-correlation coefficient is calculated so that the respective average grayscale values are close to each other. Thus, a result close to that obtained when the correspondence is visually obtained is obtained. That is, the estimation accuracy of the movement vector is improved, and at the same time,
The prediction accuracy is improved.

FIG. 4 is a graph showing an evaluation result when a change in precipitation pattern is predicted by applying the movement vector estimation method of the present invention to a weather radar echo image. It can be understood that the prediction accuracy when the present invention is not applied and when the present invention is applied is improved when the present invention is applied.

In this case, any of the methods uses the formula (1)
Predicts changes in precipitation patterns using a method based on GPS.

In the equation (1), I (x, y, t) is a gray value of an image, and is a position (x, y) and a time (t). λ is the diffusion coefficient, ε is the extinction constant,

[0029]

[Outside 1] Is the advection vector, and the movement vector estimated by the present invention was inserted.

In the comparison experiment of evaluation, the prediction accuracy is compared by applying a corrected cross-correlation method to a motion vector estimated by an uncorrected cross-correlation method by providing a motion vector in which all pixels are filled in by linear interpolation. did. Equation (1)
Are the diffusion term, the advection term, the springing term, and the suction term from the time term and the first term on the right side. Equation (1)

[0031]

(Equation 1) In this case, the pixels of the image are discretized, and the time term is set to the forward difference, and the time integration is continued for the required number of prediction steps.

[0032]

(Equation 2) That is, as a result of the time integration, I (x, y, t) is updated,
This is the exact result of the predicted precipitation pattern.
The source and sink terms are binarized continuous images,
A positive area and a negative area when the difference image is obtained are given to each. For the diffusion coefficient, a linear table of the area change and the diffusion coefficient is statistically created in advance.

As a result of the numerical simulation, when the rainfall pattern is predicted using the advection vector (movement vector) estimated by the corrected cross-correlation coefficient and the equation (1), the shading that increases as the prediction time elapses. It can be seen that the prediction error of the value is suppressed lower than the increase when the cross-correlation coefficient without correction is used.

[0034]

As described above, according to the present invention,
When associating moving objects with continuous gray value changes, normalization is performed so that the average gray value between images at different times is the same, and then the cross-correlation coefficient is calculated. And the accuracy of estimating the movement vector is improved. Therefore, even when the prediction is performed based on the movement estimated in this way, there is an effect that the prediction accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a moving object behavior analysis prediction device including a swell correspondence processing method and a swell correspondence processing method according to an embodiment of the present invention. (A) is a block diagram of a mobile object behavior analysis prediction device. (B) is a flowchart of a swell correspondence processing method according to the embodiment of the present invention.

FIG. 2 is an example of a change in a time-series pattern having a change in gray value. (A) is a first example. (B) is a second example.

FIG. 3 is a graph before and after correction of average gray value distribution and cross-correlation coefficient change of two images having different continuous times.

FIG. 4 is a graph of an evaluation result when a change in a precipitation pattern is predicted by applying the movement vector estimation method of the present invention to a weather radar echo image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Image input part 12 Image storage part 13 Comparison part 14 Correction cross-correlation part 15 Output part S101-S111 Processing procedure

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI G06T 1/00 G06F 15/62 385 // G01S 13/95 G01S 13/95

Claims (5)

[Claims] 1. A behavior analysis prediction device using an image of a moving object, comprising: an image input unit for inputting an image; an image storage unit for storing the input image as a time-series image; A comparison unit that divides each image into small blocks between two or more consecutive two-dimensional images stored, extracts image feature amounts by image processing, and compares image feature amounts at different times; With respect to the average value of the gray values in the small block obtained by the comparison means, the similarity is analyzed based on the cross-correlation method while changing the average gray value of the two images, the maximum similar point is found, and the movement vector is calculated. Correction cross-correlation means that enables estimation of a motion vector that has been corrected by associating undulations when undulations exist in the grayscale values, and output that outputs the analyzed result Behavior Analysis prediction apparatus using the image of the moving body, characterized in that it comprises a stage, a. 2. A swell-correlation processing method for a behavior analysis / prediction device using an image of a moving object, comprising: a step of inputting the image of the moving object in time series by an image input means; A procedure for storing in the image storing means, and a comparing means for extracting two or more continuous two-dimensional images from the plurality of images stored in the image storing means, dividing each of the extracted images into small blocks, Extracting image feature values by processing and comparing the image feature values at different times; and correcting cross-correlation means to calculate the average value of the grayscale values in the small block obtained by the comparison means. By analyzing the similarity based on the cross-correlation method while changing the average gray value, finding the maximum similarity point and estimating the movement vector, if there is undulation in the gray value, correlate with the undulation Correlating processing of a behavior analysis prediction device, comprising: a procedure of estimating a movement vector by performing a correction based on a logical process; and a procedure of outputting a result analyzed by the corrected cross-correlation unit by an output unit. Method. 3. In the step of obtaining the image feature quantity, a difference between the images is obtained and extracted as a spatial frequency component of a temporal variation of a grayscale value. 3. The undulation associating processing method of the behavior analysis prediction device according to claim 2, wherein the variation of the gray value is made to correspond to the time direction, and the cross-correlation analysis is performed while correcting the average value of the gray values between the correlated images. 4. In the procedure for estimating the movement vector in the correction cross-correlation means, a movement vector obtained by analyzing similarity between two images is used as an initial vector, and the movement vector is linearized by a linear extrapolation method. 3. The undulation associating processing method of the behavior analysis prediction device according to claim 2, further comprising: a procedure of predicting the fluid variation by predicting the fluid advection term of the fluid equation. 5. The undulation associating processing method of the behavior analysis prediction device according to claim 2, wherein in the step of comparing the image feature amounts, a smoothing process and an isolated point removing process are performed on each image as preprocessing.