JPH05233816A - Moving image feature detecting method - Google Patents

Abstract

PURPOSE:To automatize the detection of the motion of a moving image by loading a linear group filter to an obtained Fourier spectrum and detecting the feature of the motion of data by calculating a peak value from that result. CONSTITUTION:When a white circle 42 is moved on a screen 41 shown by (x) and (y) coordinates while setting velocity Vx for the (x) axis direction component to '1' and setting velocity Vy for the (y) axis direction component to '1', a three-dimensional FET image F (Fx, Fy, Ft) is obtained by executing three-dimensional FET (high-speed Fourier transformation) to the moving image data. In respect to the obtained power spectrum, the Vy is changed from -Ny to Ny, average power spectrum is obtained at every Vy by the linear group filter and the peak of the average power spectrum is calculated. Concerning data on the lane of Fy=0 of the power spectrum, the velocity component Vx in the (x) axis direction of the object is calculated similarly. Further, when there is one peak, it is judged that there is one object, and one velocity component Vx/Vy is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing of moving images, and more particularly, to a video database system for providing various services using moving images, and more particularly to a technique effective in the fields of broadcasting and entertainment. Is.

[0002]

2. Description of the Related Art FIG. 2A shows moving image data as a frame image. For example, in the present television, 30 frames 21 per second are reproduced seamlessly so that the image appears to be moving. A major problem in handling this moving image data is what to do with the unit of handling. It is considered that the problem is that the amount of data is more than two digits larger than that of conventional text data and the like, and the continuity as a moving image is maintained. Therefore, in order to handle moving image data in a database or the like, it is necessary to perform feature detection as shown in FIG. 2B and then disassemble into units as shown in FIG. 2C to create an index. ..

For the feature detection of FIG. 2B, it is possible to use attributes such as color, size, and motion. Among these, a method for automatically detecting a motion that can be divided into a simple uniform motion, a periodic motion, and a general motion of a combination thereof from moving image data has not yet been developed. Conventionally, the most commonly used motion detection is an analysis method using optical flow.
This method is to detect the change of each element between frames to represent the motion. An example of analysis using this method is shown in FIG.
Shown in.

[0004]

When the method using the above optical flow is used for the feature detection of FIG. 2B, there are the following problems. In other words, the optical flow method has so far been studied only to the point where a human can judge the diagram shown in FIG. 7 which is made into a flow. It is necessary to further perform image processing and the like from the obtained flow, but the method has not been developed. Therefore, it is difficult for the optical flow method to automatically perform motion detection. Also,
When the optical flow method is applied to an image in which multiple objects are moving, the flow becomes complicated.

On the other hand, an object of the present invention is to automatically detect the motion of a moving image, which has been conventionally performed by human judgment.

The above and other objects and novel characteristics of the present invention will be apparent from the description of this specification and the accompanying drawings.

[0007]

In order to achieve the above-mentioned object, the present invention relates to data representing moving images in which each screen is continuously formed, and inside and between each screen. It is characterized in that a Fourier spectrum is obtained from the information, a linear group filter is applied to the obtained Fourier spectrum, and a peak value is obtained from the result, whereby the characteristic of the movement of the data is detected. The term "straight line group filter" in the present specification refers to a filter having a certain width and extracting only data existing on a straight line group arranged at equal intervals.

[0008]

According to the above-mentioned means, it becomes possible to detect all the motion characteristics automatically and without human judgment.

The structure of the present invention will be described below together with one embodiment.

[0010]

EXAMPLE FIG. 2 shows an overall view of an indexing method focusing on motion detection in moving image data according to this example. FIG. 2A shows moving image data as a frame image. For example, in the present television, 30 frames 21 per second are reproduced seamlessly so that the image appears to be moving. In order to handle moving image data in a database or the like, the motion feature detection 22 of (b) is performed by the method of the present invention described below, and then decomposed into units as shown in (c) to create indexes 23 to 26. I'll do it.

FIG. 3 specifically shows an example of feature detection focusing on movement. The moving image data is decomposed into several scenes 31 to 33 with the feature of motion. The characteristics of each movement are shown in (a). Here, the detection of motion refers to the vectors indicated by the arrows 34 to 36 shown in (a). For example, in the first example 37 of (a), the black round object is moving to the lower right, in the second example 38 the black square object is moving to the left, and in the third example 39, the black elongated object. Things are characterized as moving directly above.

The moving image feature detecting method of this embodiment will be described below with reference to the flowchart of FIG. The sample data of FIG. 4 is used to describe this example. In FIG. 4, it is assumed that the white circle 42 is moving at a speed Vx of the x-axis direction component of 1 and a speed Vy of the y-axis direction component of 1 on the screen 41 indicated by the x and y coordinates. On an actual frame, an image in which the circle 42 is moved by the time represented by the time interval between the frames is represented.

(Step 101) The moving image data is defined as the luminance spatiotemporal distribution f (x, y, t) of each pixel. Here, x is the horizontal direction, y is the vertical direction, t is the pixel coordinate in the time direction, and the range is x = 0 to Nx−1, y = 0 to Ny−.
Let 1, t = 0 to Nt-1.

(Step 102) The three-dimensional FFT image F (Fx, Fy, Ft) defined by the following [Equation 1] is applied to the moving image data to obtain a three-dimensional FFT image F (Fx, Fy, Ft).

[0015]

[Equation 1]

Here, Fx, Fy, Ft are frequency axes corresponding to x, y, t, and the range is Fx = 0 to Nx-1, Fy =
0 to Ny-1 and Ft = 0 to Nt-1. Next, 3D FF
The power spectrum of the T image is obtained as in [Equation 2].

[0017]

## EQU2 ## | F (Fx, Fy, Ft) | ² FIG. 5 shows a contour diagram of this power spectrum. The figure shows the surface of Ft = 0 as a result represented by three-dimensional data.

(Step 103) Vy is changed from -Ny to Ny with respect to the obtained power spectrum, and each Vy is changed.
The average power spectrum is obtained for each time by the following linear group filter of [Equation 3]. Fx = 0 and Fy and Ft are

[0019]

[Equation 3]

For the region satisfying the above condition, the average power spectrum obtained by normalizing the total sum of power spectra by the number of pixels is obtained. Here, Δw corresponds to ½ of the width of the linear region. The result obtained by this algorithm is shown in FIG. In this figure, the vertical axis represents the average power spectrum and the horizontal axis represents y.
The velocity Vy of the axial component is shown. The velocity Vy of the y-axis direction component is changed by 0.1. Also in the figure line wi
dth indicates the width of the linear region of the linear group filter.

(Step 104) The peak of the average power spectrum is obtained. As a method of obtaining the peak, for example, an existing general method such as obtaining a curve obtained by differentiating once from the obtained curve and obtaining it from the point where 0 is crossed is used. The velocity Vy of the y-axis direction component with respect to this peak is the velocity component of the object in the y-axis direction.

(Step 105) (Step 106) Similar to Step 103 and Step 104, the power spectrum F
For the data on the surface of y = 0, the velocity component Vx of the object in the x-axis direction is obtained.

(Step 107) It is judged whether the number of peaks obtained in steps 104 and 106 is one or plural. When there is one peak, the process proceeds to step 108, and when there are a plurality of peaks, the process proceeds to step 109.

(Step 108) In Step 107,
When the number of peaks is one, it is determined that the number of objects is one, and one velocity component Vx, Vy is obtained.

(Step 109) In Step 107,
When there are a plurality of peaks, it is determined that there are a plurality of objects, and further, regarding the data on the surface of Ft = 0 of the power spectrum,
By the same method as steps 103 to 106, the moving direction Vy of the object
By obtaining / Vx, the velocity component of each object can be correctly associated.

What is characteristic of FIG. 5 in the above step 102 is that the data existing areas are concentrated from the upper left to the lower right. This indicates that the motion in which the ratio of the velocity components in the x-axis direction and the y-axis direction is 1 is characteristic. According to the present invention, the speed can be automatically obtained based on the above algorithm, and it is possible to automatically detect the motion of a moving image, which has been impossible in the past.

Although the invention made by the present inventor has been specifically described based on the embodiments, the invention is not limited to the above-described embodiments and various modifications can be made without departing from the scope of the invention. Needless to say.

[0028]

As described above, according to the present invention,
It has become possible to automatically detect the motion of moving images, which was previously impossible, and it is expected to be extremely useful in fields such as indexing when inputting data to database systems that use moving images, which is expected to dramatically increase demand in the future. It is effective for.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating a moving image feature detection method of the present invention.

FIG. 2 is an overall view showing an indexing method focusing on motion detection in moving image data according to this example.

FIG. 3 is a diagram specifically showing an example of feature detection focusing on motion.

FIG. 4 is a diagram showing sample data used in an example of the present invention.

FIG. 5 shows a three-dimensional FFT power spectrum obtained by Fourier transform.

FIG. 6 is a graph showing an average power spectrum.

FIG. 7 is a diagram showing an example of flow analysis obtained by a conventional optical flow method.

[Explanation of symbols]

21 ... Frame, 22 ... Motion feature detection, 23-26 ...
Indexes 31 to 33 ... Video data, 34 to 36 ...
Arrow.

Claims (1)

[Claims] 1. A method for detecting a characteristic of a motion of data for data representing a moving image in which each screen is continuously formed, and a Fourier spectrum is obtained from information inside and between the screens. The method for detecting a moving image feature is characterized in that the obtained Fourier spectrum is subjected to a linear group filter, and the peak value is obtained from the result to obtain the characteristic of the data.