JPH10177655A - Moving image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To grasp series of movement of object in an image as a whole while almost fixing arithmetic quantity in spite of size of movement by providing the information of movement in inputted moving image data based on the information of stripe-shaped patterns. SOLUTION: A space/time axis transforming part 10 inputs moving image data S101, stores these data, transforms the coordinate system of moving image data expressed on a three-dimensional space and outputs a pattern S102 on a two-dimensional space to a projection pattern preparing part 102. The projection pattern preparing part 102 inputs the twodimensional pattern 102, projects it for each of plural directions and outputs a projected pattern S103 to a stripe direction detecting part 103. The stripe direction detecting part 103 inputs the projected pattern S103, calculates its variance for each direction and outputs the direction of maximum covariant value as stripe direction data S104 to a moving amount output part 104. The moving amount output part 104 inputs plural pieces of stripe direction data S104, calculates the amount of movement corresponding to the moving image data S101, and outputs moving image movement data S105.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing apparatus for obtaining movement information of an object in a moving image.

[0002]

2. Description of the Related Art Literature 1 "Television Society," Image Information Compression ", Ohmsha, p. 92, 1991 ", motion detection of a moving image is a technique necessary for configuring a moving image processing apparatus. For example, in a moving image encoding device, the amount of code can be reduced by predicting the motion of an image and encoding only the prediction error. Also, in a system that analyzes and models the movement of an object and uses it, motion detection of a moving image is indispensable.

As a method for detecting such a motion of a moving image, there is a method disclosed in the above-mentioned reference 1.

FIG. 2 is a diagram for explaining a method of detecting a motion of a moving image according to the related art. In FIG. 2, reference numeral 201 denotes a block in the current frame, reference numeral 202 denotes a block in the previous frame, reference numeral 203 denotes a pixel Xi in the block 201,
Reference numeral 204 represents a pixel Xi-fj in the block 202, respectively.

In the method described in the above document, an image is divided into blocks, and pattern matching is performed in block units. The pattern matching method is, for example, the sum of the absolute value S of the difference between the pixel Xi in the block 201 in the current frame and the pixel Xi-fj in the block 202 in the previous frame, which is shifted by j pixels, is expressed by (1 ) Is obtained according to the following formula, and a shift position j that minimizes the evaluation amount S is searched for to detect a motion amount. The sum Σ in the equation (1) is for all the pixels Xi belonging to the block 201.

S = Σ | Xi−Xi-fj | (1) In this block matching method, the current frame is divided into blocks, and a similar position to the previous frame is searched for each block, so that the inter-frame Detect movement.

[0007]

However, the following problems (1) to (3) have occurred in the motion detection by the above-described block matching method.

(1) If the movement between frames is large, it is necessary to set a wide search range for pattern matching.
As a result, the amount of calculation increases.

(2) Since only the amount of motion for each block between frames can be obtained, a series of motions of an object in an image cannot be grasped globally. For this reason, it is difficult to apply the present invention to a system that analyzes and models the movement of an object.

(3) When noise is included in image data, the evaluation value S is not always minimized with respect to a correct motion amount, so that a motion detection error is likely to occur.

[0011]

In order to solve such a problem, a moving image processing apparatus according to the present invention receives moving image data represented in a three-dimensional space of a horizontal axis, a vertical axis and a time axis, and receives the input data. Through coordinate axis conversion processing for moving image data, coordinate axis conversion means for forming coordinate axis conversion data that can include a striped pattern corresponding to movement in input moving image data, and information on the striped pattern in the formed coordinate axis conversion data. Motion information forming means for obtaining information on motion in the input moving image data based on the motion information.

In the moving picture processing apparatus of the present invention, the coordinate axis conversion means receives the input through the coordinate axis conversion processing for the input moving picture data represented in the three-dimensional space of the horizontal axis, the vertical axis and the time axis. Forming coordinate axis conversion data that may include a striped pattern corresponding to the movement in the moving image data, and the motion information forming unit performs processing on the input moving image data based on the striped pattern information in the formed coordinate axis conversion data. Get movement information.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION

(A) First Embodiment Hereinafter, a first embodiment of a moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing the configuration of a moving image processing apparatus according to the first embodiment. In FIG. 1, the moving image processing apparatus according to the first embodiment includes a spatiotemporal axis conversion unit 101, a projection pattern creation unit 102, a stripe direction detection unit 103, and a motion amount output unit 104.

The spatio-temporal axis conversion unit 101 converts the moving image data S
101 is input and stored, and the coordinate system (x, y, t) of the moving image data represented in the three-dimensional space (x is the horizontal axis of each frame constituting the moving image, y is the vertical axis of each frame, t
Is a time axis), and a pattern on a two-dimensional space using (t, x) for each y value as a coordinate axis and a two-dimensional space using (t, y) for each x value as a coordinate axis Pattern S1
02 is output to the projection pattern creation unit 102.

The projection pattern creation unit 102 receives the two-dimensional pattern S102, performs projection in a plurality of directions, and outputs the projection pattern S103 to the stripe direction detection unit 103.

The fringe direction detector 103 detects the projection pattern S1
03, the variance in each direction is calculated, and the direction in which the variance value is maximized is obtained.
Is output to the motion amount output unit 104.

The motion amount output unit 104 receives a plurality of stripe direction data S104, calculates a motion amount corresponding to the moving image data S101, and outputs moving image motion data S105.

(A-2) Operation of First Embodiment Next, the operation of the moving image processing apparatus of the first embodiment, which includes the above-described units 101 to 104, will be described.

The moving image data S101 is converted to a spatiotemporal axis conversion unit 1.
When input to 01, the spatio-temporal axis conversion unit 101 stores moving image data for a predetermined time. At this point, the spatio-temporal axis conversion unit 101 has stored the pixel data defined on the (x, y, t) three-dimensional space. next,
The spatiotemporal axis conversion unit 101 sequentially creates a (t, x) two-dimensional pattern for each y value and a (t, y) two-dimensional pattern for each x value, and projects these two-dimensional patterns S102. Output to the pattern creation unit 102.

FIG. 3 is an explanatory diagram showing characteristics of the conversion pattern in the spatiotemporal axis conversion unit 101.

FIG. 3A shows a frame pattern at an initial time t = 0 when a pattern in a frame moves horizontally as time elapses as indicated by an arrow.
In this case, the (t, x) two-dimensional pattern corresponding to a certain y value yc is a striped pattern as shown in FIG. When the pattern on the moving image moves at a constant speed in the horizontal direction, the striped pattern becomes a linear pattern in the same direction, and its inclination dx / dt matches the horizontal moving speed. Further, in this case, (t,
y) The two-dimensional pattern is not always a striped pattern as shown in FIG.

FIG. 3D shows a frame pattern at an initial time t = 0 in a case where the pattern in the frame moves vertically with the passage of time like a mark. In this case, (t, x) corresponding to a certain y value yc
The two-dimensional pattern is not always a striped pattern as shown in FIG. On the other hand, in this case, a certain x
The (t, y) two-dimensional pattern corresponding to the value xc of FIG.
As shown in (f), a stripe pattern is formed. When the pattern on the moving image moves at a constant speed in the vertical direction, the striped pattern becomes a linear pattern in the same direction, and its slope dy / dt matches the vertical moving speed.

FIG. 3 (g) shows a frame pattern at an initial time t = 0 when the pattern in the frame moves obliquely with the passage of time as indicated by an arrow. In this case, (t,
x) As shown in FIG. 3H, the two-dimensional pattern does not form a strictly striped pattern, but exhibits a pattern close to the striped pattern. Similarly, the (t, y) two-dimensional pattern corresponding to a certain x value xc also exhibits a pattern close to a striped pattern as shown in FIG. FIG.
The directions of the stripe-like patterns seen in (h) and (i) correspond to the moving speeds dx / dt and dy / dt of the moving image, respectively.

As described above, when the moving image moves linearly, the two-dimensional pattern (a set of two-dimensional patterns) S102 output from the spatiotemporal axis conversion unit 101 becomes a striped pattern. It turns out that it is easy.

When the obtained two-dimensional pattern S102 is input to the projection pattern creation unit 102, the projection pattern creation unit 102 uses the two-dimensional pattern S102 for each running direction θ (for example, −9).
Two-dimensional pattern S1 (from 0 ° to 90 ° in 1 ° increments)
The projection pattern S103 of No. 02 is calculated and output to the stripe direction detection unit 103.

FIG. 4 is a diagram for explaining a method of calculating the projection pattern S103 in the projection pattern creation unit 102.

FIG. 4 (a) shows a 1 having a striped pattern.
The projection data (PR (r) in the figure) is shown when the two two-dimensional patterns 401 are projected in a direction (shown by an arrow) inclined clockwise by θ from the vertical direction. . Projection data PR for each projection position r
As a calculation method of (r), for example, the following (2)
Apply the formula. The sum 総 in equation (2) is
This is for all the elements of the pixel set A (θ, r).

PR (r) = ΣL (x, y, θ, r) P (x, y) / S (θ, r) (2) Here, FIGS. As illustrated in the geometrical relationship implied by the parameters, A (θ,
r) is a projection line at the projection angle θ and the projection position r (θ,
r) is a set of pixels existing on L (x, y, θ,
r) indicates that the projection line (θ, r) is a pixel set A (θ, r)
Is the length of passage when passing through a certain pixel (x, y) belonging to
P (x, y) is the pixel value (gradation) of the pixel (x, y), and S (θ, r) is the length of the projection line (θ, r) passing through the two-dimensional pattern 401. .

Note that (x, y) in the immediately preceding paragraph is
It is used in a different sense from (x, y) in FIG. For example, in the case of the projection processing for the (t, x) two-dimensional pattern in FIG. 3, (t, x) in FIG. 3 corresponds to (x, y) in the immediately preceding paragraph.

The above parameters A (θ, r), L
(X, y, θ, r) and S (θ, f) are calculated each time the projection pattern S103 is calculated based on the projection direction, the projection position, and the mutual geometric positional relationship of the two-dimensional pattern. The calculation may be performed, or the calculation may be performed in advance and stored as a table, and the table may be read from the table when calculating the projection pattern S103.

When the projection pattern S103 is output from the projection pattern creation unit 102, the variance value V (θ) of the projection pattern is output by the stripe direction detection unit 103 for each projection direction θ.
Is calculated. Subsequently, the stripe direction detection unit 103
The projection direction θ when the variance value V (θ) is the maximum is calculated, and the projection direction θ is output to the motion amount output unit 104 as the stripe direction data S104.

FIG. 5 is a diagram for explaining the characteristics of the projection pattern with respect to the striped pattern. As an example, when a two-dimensional pattern ((t, x) two-dimensional pattern or ((t, y) two-dimensional pattern) 501 having a vertical stripe pattern is projected in the vertical direction, the upper part of FIG. As shown,
When a projection pattern 502 having a large variance is obtained and the projection is performed in another direction, for example, in the horizontal direction, FIG.
As shown on the right side of FIG.
Is obtained.

That is, when the projection is performed on the stripe pattern, the variance value V of the projection pattern corresponding to the stripe direction θ is obtained.
Since (θ) increases, the fringe direction detection unit 103 calculates the direction θ in which the variance value V (θ) is maximum, so that the fringe direction data S104 is correctly output.

When the maximum variance is smaller than a predetermined threshold value, the stripe direction detecting section 103 regards that there is no motion and outputs data indicating this as the stripe direction data S10.
Output as 4.

For example, (t, y) shown in FIG.
Since no clear fringe pattern exists for the two-dimensional pattern, a sufficiently large variance value cannot be obtained in the processing in the fringe direction detecting unit 103, and the identification data indicating that there is no vertical movement is the fringe direction data. It will be output as S104.

The fringe direction detecting unit 103 outputs fringe direction data S1
When 04 is output, the motion amount output unit 104 inputs and stores the stripe direction data S104. Further, the motion amount output unit 104 generates a stripe direction corresponding to all the (t, x) two-dimensional patterns and the (t, y) two-dimensional patterns constituting the moving image data S101 input to the image processing apparatus. After inputting the data S104, (x, y, t)
The motion of the moving image in the three-dimensional space is expressed as, for example, a motion vector and output as moving image motion data S105.

As described above, (t, x) 2 shown in FIG.
The stripe direction θ of the two-dimensional pattern is the moving speed dx / dt in the x direction from the initial time t = 0 used for the calculation, and the stripe direction θ of the (t, y) two-dimensional pattern shown in FIG.
Is the moving speed dy / dt in the y direction from the initial time t = 0 used for the calculation. Therefore, the moving speed dx / dt at a certain value of x and the moving speed dy at a certain value of y
/ Dt, coordinates (x, y) at the initial time t = 0
At the coordinates (x, y).
, The motion vectors of all the pixels at the initial time t = 0 can be obtained. By sequentially shifting the initial time, moving image motion data S10, which is a time change of a motion vector at each pixel position, is obtained.
5 can be obtained.

FIG. 6 shows an example of presentation using the moving image motion data S105. As shown in the figure, the frame images are arranged with time and the movement of an arbitrary point on the (x, y) plane is tracked. Is displayed. In this embodiment, by using the moving image motion data S105, the motion of the image in the (x, y, t) three-dimensional space can be grasped globally. Various movements can be presented.

(A-3) Effect of First Embodiment As described above, in the first embodiment, the moving image data represented on the (x, y, t) three-dimensional space is represented by (t, x). After slicing on the plane and the (t, y) plane, projection is performed for each of a plurality of directions, and the motion amount of the moving image is detected based on the direction in which the variance of the projection pattern is maximized. 2 on the sliced (t, x) plane and (t, y) plane
When the motion of a moving image can be approximated by a parallel movement that does not include deformation or rotation, the dimensional pattern generally tends to be a striped pattern. , That is, the direction corresponding to the motion of the moving image, so that the motion of the moving image can be correctly detected. In addition, since the motion can be detected collectively on the <t, x) plane and the (t, y) plane, the motion of the moving image can be grasped globally. Therefore, it is possible to solve the problem of the method of detecting a motion by performing pattern matching on a block basis and detect motion of a moving image with high accuracy and globally.

(B) Second Embodiment Next, a second embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of the Second Embodiment FIG. 7 is a block diagram showing the configuration of a moving image processing apparatus according to the second embodiment. 7, the moving image processing apparatus according to the second embodiment includes a spatiotemporal axis conversion unit 701, a finite Radon conversion unit 702, a stripe direction detection unit 703, a candidate angle storage unit 705, and a motion amount output unit 704. I have.

The spatio-temporal axis conversion unit 701 executes the same conversion processing as the spatio-temporal axis conversion unit 101 of the first embodiment on the input moving image data S701, and obtains each y
And a pattern S702 in the (t, y) two-dimensional space for each value of x and a pattern S702 in the (t, y) two-dimensional space for each value of x.

The finite Radon transform unit 702 receives the two-dimensional pattern S702, performs finite Radon transform, and outputs the projection pattern S703 to the stripe direction detecting unit 703.

The fringe direction detector 703 detects the projection pattern S7
03, the variance of each projection pattern is calculated, the projection pattern number S706 having the variance value equal to or larger than a predetermined value is output to the candidate angle storage unit 705, and the candidate angle data S707 is stored in the candidate angle storage unit 705. And calculates the stripe direction data S704 and outputs the calculated data to the motion amount output unit 704.

The candidate angle storage unit 705 receives the projection pattern number S706 and outputs the corresponding candidate angle data S707 to the stripe direction detection unit 703.

As in the case of the motion amount output unit 104 of the first embodiment, the motion amount output unit 704 receives the stripe direction data S704, calculates the motion amount corresponding to the moving image data S701, and It outputs data S705.

(B-2) Operation of the Second Embodiment Next, the operation of the moving image processing apparatus according to the second embodiment, which includes the above-described units 701 to 705, will be described.

The moving image data S701 is converted to a spatiotemporal axis
01, which is performed by the spatiotemporal axis conversion unit 701 on the moving image data S701 is the same as in the first embodiment, and a description thereof will be omitted.

When the two-dimensional pattern S702 output from the spatio-temporal axis conversion unit 701 is input to the finite Radon conversion unit 702, the two-dimensional pattern
Finite Radon transform is performed on the dimensional pattern S702, and the resulting projection pattern S703 is output to the stripe direction detection unit 703.

Here, the finite Radon transform is one of the methods of converting a two-dimensional pattern into a projection pattern as described in, for example, Reference 2.

Reference 2 “F. Matus, J. Flusser,“ Image
Representation via a Finite Radon Transform '', IE
EE Transactions on Pattern Analysis and Machine In
telligence, vol.15, no.10, pp.996-1006, 1993]
In the pattern projection method described in the embodiment, the parameters L (x, y, θ, r) and A are determined based on the projection direction, the projection position, and the mutual geometrical positional relationship of the two-dimensional pattern.
Although it was necessary to obtain (θ, r) and S (θ, r) by real number calculation including multiplication and division, unlike the finite Radon transform, most of the projection patterns were formed only by simple integer addition operations. It has the characteristic that the original two-dimensional pattern can be completely reconstructed by the inverse transformation.

FIG. 8 is a diagram for explaining the rules of projection of the finite Radon transform. In the finite Radon transform, p × p pixels (p
(Prime number) pattern areas, there are p + 1 types of projection patterns. The number for identifying this type is hereinafter referred to as a projection pattern number. In FIG. 8, as an example, FIGS. 8 (a), (b), (c), (d), (e), and (f) show projection pattern numbers i = 0, 1, 2,.
The projection method in the case of p-2, p-1, and p is shown.

For the projection pattern number i = 0 (see FIG. 8A), the average of the pixel values arranged in the vertical direction with respect to the horizontal position r on the lower end line of the two-dimensional pattern is calculated. This is defined as projection data PR (r).

For a projection pattern number i = 1 (see FIG. 8B), a pixel shifted right by +1 and upward by +1 with respect to the horizontal position r on the lower end line of the two-dimensional pattern. The average of the pixel values of the group is calculated, and this is set as projection data PR (r). However, when the pixel exceeds the right end of the two-dimensional pattern, the excess is calculated using the pixel value shifted rightward from the left end of the pattern.

For a projection pattern number i = 2 (see FIG. 8C), a pixel group shifted by +2 to the right and +1 to the upper direction with respect to the horizontal position r on the lower end line of the two-dimensional pattern Is calculated, and this is used as projection data PR (r). However, as in the case of the projection pattern number i = 1, when the pixel exceeds the right end of the two-dimensional pattern, the excess is calculated using the pixel value shifted rightward from the left end of the pattern.

Similarly, the projection pattern number i = 3,
With respect to 4,..., P−1, the average of the pixel values of the pixel group shifted by + i in the right direction and +1 in the upward direction with respect to the horizontal position r on the lower end line of the two-dimensional pattern is calculated. Is the projection data PR (r) (note that, for the projection pattern numbers i = p−2 and p−1, FIGS.
(E)).

For the projection pattern number i = p (see FIG. 8F), the average of the pixel values arranged in the horizontal direction is calculated with respect to the vertical position r on the left end line of the two-dimensional pattern. This is defined as projection data PR (r).

FIG. 8 (g) shows that each projection pattern number i =
The projection data PR (r) for 0, 1,..., P are arranged in order from the bottom in the upward direction, and this is conversion data by the finite Radon transform (projection pattern S703).

The finite Radon transform unit 702 calculates the projection pattern S
After outputting 703, the stripe direction detection unit 703 inputs the projection pattern S703 and calculates a variance value V (i) for each projection pattern number i. Subsequently, the stripe direction detection unit 703 calculates a projection pattern number S706 when the variance value V (i) is equal to or larger than a predetermined threshold, and outputs the calculated projection pattern number S706 to the candidate angle storage unit 705.

The candidate angle storage unit 705 stores candidate angle data including the stripe direction angle candidate and reliability in advance corresponding to each projection pattern number, and receives the projection pattern number S706. Then, the corresponding candidate angle data S7
07 is output to the stripe direction detection unit 703.

FIG. 9 is an explanatory diagram of the candidate angle data stored in the candidate angle storage unit 705 in advance.

In FIG. 9, reference numeral 901 denotes an area of a two-dimensional pattern for performing a finite Radon transform, and a plurality of hatched squares represent a group of pixels contributing to projection data PR (r) for a certain projection pattern number i. is there. The stripe direction angle candidate is a direction angle of a stripe when a stripe pattern along the pixel group is assumed, and in order to determine the angle, a plurality of straight lines connecting the pixel group are adjacent to each other. A pixel having a small distance between pixels is selected, and the direction of the straight line is calculated. In the second embodiment, three straight lines are selected in ascending order of the distance between pixels, and the angles are calculated. That is, regarding the straight line connecting the pixels P, Q, and R shown in FIG. 9 as the stripe direction angle candidates, the angle θ1 with respect to the straight line PQ, the angle θ2 with respect to the straight line PR,
The angle θ3 with respect to the straight line QR is applied. The reliability is the reciprocal of the distance between adjacent pixels, that is, the reciprocal (L1) of the length information of the line segment PQ (hereinafter, the square of the distance is applied to the angle candidate θ1). , Angle candidate θ2
And the inverse (L3) of the length information of the line segment QR is applied to the angle candidate θ3.

Here, the coordinates of the pixel P are represented by (0, 0), and the coordinates of the pixel Q are represented by (i, 1). FIG.
Are represented by (ji, j), the coordinates of the pixel R are represented by (ji + ip, j + 1). Note that j is an integer part obtained by dividing p-1 by i.

Therefore, the above-mentioned candidate angle data (angle candidate and reliability) is calculated according to the projection pattern number i.
This value is calculated as in equation (3), and this value is stored in the candidate angle storage unit 705 in advance.

Θ 1 = atan (1 / i) (i = 1, 2,..., P−1) = 90 ° (i = 0) = 0 ° (i = p) θ 2 = atan ｛(j + 1) / (ji + i) −p)｝ θ3 = atan ｛j / (ji-p)｝ L1 = 1 / (i ² +1) L2 = 1 / ｛(ji + ip) ² + (j + 1) ² ｝ L3 = 1 / ｛(ji- p) ² + j ² … (3) Upon inputting the candidate angle data S707, the stripe direction detection unit 703 receives the projection pattern number S706 (i = i1, i2,
, In), the variance value V (i) and the stripe direction angle candidates θ1 (i), θ2 (i), θ3 (i), and the reliability L1 (i), L2 (i), L3 ( Based on i), stripe direction data S704 (θ) is determined and output. When the stripe direction data is determined, any suitable method may be used as long as a stripe direction angle candidate having a large variance value and a high reliability is selected. In the second embodiment, first, all input stripe direction angle candidates are grouped by regarding those having similar angles as those belonging to the same group, and further, the stripe direction angle candidates included in the same group are grouped. The sum of the variance value and the reliability is calculated, and the representative value of the stripe direction angle candidate of the group having the largest sum of product values is output as the stripe direction data S704.

In the first embodiment, when projection is performed on a two-dimensional pattern of a stripe pattern, the projection direction in which the variance of the projection pattern is large corresponds to the stripe direction 1: 1. In the case of the second embodiment, as shown in FIG. 9, even if the variance value increases for one projection pattern number, the direction of the stripe is not uniquely determined. Therefore, in the second embodiment, as described above, the stripe direction angle candidates and their reliability are prepared in advance in accordance with the projection pattern numbers, and the projection pattern number group having a large variance is comprehensively determined. The detection accuracy of the fringe direction data is improved.

If any of the above sums of products is smaller than a predetermined threshold value, the stripe direction detection unit 703 regards that there is no motion, and outputs data indicating this as the stripe direction data S7.
04 is output. For example, since a clear stripe pattern does not exist for the (t, y) pattern shown in FIG. 3C, a sufficiently large product-sum value cannot be obtained in the processing in the stripe direction detection unit 703. The identification data indicating that there is no movement in the vertical direction is output as the stripe direction data S704.

The fringe direction detecting unit 703 outputs fringe direction data S7
When 04 is output, the motion amount output unit 704 inputs and stores the stripe direction data S704. Further, the motion amount output unit 704 generates the stripe direction data corresponding to all the (t, x) two-dimensional patterns and the (t, y) two-dimensional patterns constituting the moving image data S701 input to the image processing apparatus. After inputting S704, (x, y, t)
The motion of the moving image in the three-dimensional space is expressed as, for example, a motion vector and output as moving image motion data S705.

(B-3) Effect of Second Embodiment As described above, in the second embodiment, (x, y,
t) Moving image data represented on a three-dimensional space is represented by (t,
After slicing on the x) plane and the (t, y) plane, finite Radon transform is performed, and the motion amount of the moving image is detected based on the projection pattern number whose output variance is equal to or greater than a predetermined value. I do. The sliced two-dimensional pattern on the (t, x) plane and the (t, y) plane generally tends to be a striped pattern when the motion of a moving image can be approximated by a parallel movement that does not include deformation or rotation. Therefore, the projection direction corresponding to the projection pattern number at which the variance value increases becomes the direction of the pattern of the striped pattern, that is, the direction corresponding to the motion of the moving image, so that the motion of the moving image can be correctly detected. In addition, since the motion can be detected collectively on the (t, x) plane and the (t, y) plane, the motion of the moving image can be grasped globally. Therefore, it is possible to solve the problem of the conventional method in which motion is detected by performing pattern matching in block units, and to detect motion of a moving image with high accuracy and globally.

Further, in the second embodiment, in addition to the same effects as those of the first embodiment, most of the projection patterns can be formed only by the addition operation, so that the operation amount can be reduced. In addition, for moving image data previously expressed in the data format of the finite Radon transform, there is also an effect that the motion of the moving image can be detected without reconstructing the moving image by the finite Radon inverse transform.

(C) Third Embodiment Next, a third embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(C-1) Configuration of the Third Embodiment FIG. 10 is a block diagram showing the configuration of a moving image processing apparatus according to the third embodiment.

In FIG. 10, a moving image processing apparatus according to the third embodiment includes a spatio-temporal axis conversion unit 1001, a finite Radon conversion unit 1002, a low-frequency cutoff filter unit 1006, a stripe direction detection unit 1003, and a candidate angle storage unit 1005. And a motion amount output unit 1004. Here, the spatio-temporal axis conversion unit 1001, the finite Radon conversion unit 1002, the stripe direction detection unit 1
003, candidate angle storage unit 1005 and motion amount output unit 10
04 is the same as that of the second embodiment described above,
The description of the function is omitted.

The low-frequency cutoff filter unit 1006 receives the projection pattern S1003 from the finite Radon transform unit 1002, and cuts off the low-frequency component of the projection pattern S1003 to obtain a projection filter pattern S1013 obtained by the stripe direction detection unit 1003. Is output to

The fringe direction detecting unit 1003 has the same basic functions as those of the second embodiment, but the input data is different from that of the second embodiment as described above. Therefore, the stripe direction detection unit 1003 receives the projection filter pattern S1013, calculates the variance of each projection filter pattern, and outputs a projection pattern number S1006 having a variance value equal to or greater than a predetermined value to the candidate angle storage unit 1005. And the candidate angle data S from the candidate angle storage unit 1005
1007 is input and the fringe direction data S1004 is calculated and output.

(C-2) Operation of Third Embodiment Next, the operation of the moving image processing apparatus according to the third embodiment, which includes the units 1001 to 1006 as described above, will be described.

First, the input moving image data S1001
On the other hand, the processing sequentially performed by the spatio-temporal axis conversion unit 1001 and the finite Radon conversion unit 1002 is the same as that of the second embodiment, and a description thereof will be omitted.

When the projection pattern S1003 is output from the finite Radon conversion unit 1002,
03 creates a pattern obtained by removing the low-frequency component of the projection pattern S1003, and generates a projection filter pattern S101.
3 is output to the stripe direction detection unit 1003.

FIG. 11 is an explanatory diagram of the operation of the low-frequency cutoff filter section 1003. In FIG. 11, reference numeral 1101 denotes one of the projection patterns S1003 (PR (r)), and reference numeral 1102 denotes a pattern (L (L)) obtained by subjecting the projection pattern 1101 to low-pass filtering.
PR (r)), and reference numeral 1103 denotes a low-frequency cutoff pattern (HPR (r)) obtained by subtracting the low-pass pattern 1102 from the projection pattern 1101.

When the projection pattern 1101 is input, the low-frequency cutoff filter unit 1003 first calculates a pattern 1102 from which only a low-frequency component is extracted by a conventionally known one-dimensional convolution filter. As the convolution filter, for example, in Expression (4), a
= 1/3, b = 1/3, and c = 1/3 may be used. Further, a low-pass cutoff filter unit 1003
By calculating the expression (5), the projection pattern 11
01, a projection filter pattern 1 in which low-frequency components are cut off
103 is obtained and output to the stripe direction detection unit 1003.

LPR (r) = a · PR (r−1) + b · PR (r) + c · PR (r + 1) (4) HPR (r) = PR (r) −LPR (r) (5) Further, as another calculation method in the low-frequency cutoff filter unit 1003, the projection filter pattern 1103 may be directly obtained by Expression (6) without using LPR (r). Here, the filter coefficients d, e,
As f, any conventionally known one can be used as long as it has a property of blocking low-frequency components. For example, d = − ／,
e = 1 and f = − ／ can be applied.

HPR (r) = d · PR (r−1) + e · PR (r) + f · PR (r + 1) (6) Even when any of the above methods is used, the low-frequency cutoff filter unit 1
By 006, a pattern in which low-frequency components are blocked from the projection pattern S1003 is obtained as a projection filter pattern S1013.

When the low-frequency cutoff filter unit 1006 outputs the projection filter pattern S1013, the fringe direction detection unit 1003 inputs the projection filter pattern S1013 and outputs the variance value V (i) for each projection pattern number i. Is calculated. Subsequently, the fringe direction detection unit 1003 calculates the variance value V
The projection pattern number S1006 when (i) is equal to or greater than a predetermined threshold is calculated and output to the candidate angle storage unit 1005. The candidate angle storage unit 1005 is a block that performs the same operation as the candidate angle storage unit 1005 of the second embodiment.
Corresponding to the projection pattern number, candidate angle data including a stripe direction angle candidate and reliability are stored in advance, and when the projection pattern number S1006 is input, the corresponding candidate angle data S1007 is detected in the stripe direction. Output to the unit 1003. The stripe direction detection unit 1003 calculates candidate angle data S
When 1007 is input, similarly to the second embodiment,
The stripe direction data S1004 is determined and the motion amount output unit 100
4 is output.

As described above, in the second embodiment, the fringe direction detecting unit 603 determines the fringe direction based on the variance of the projection pattern by the finite Radon transform. The fringe direction detection unit 1003 determines the fringe direction based not on the projection pattern itself by the finite Radon transform but on the variance of the pattern in which the low-frequency component is cut off.

According to Reference 2 described above, it has been mathematically proved that a frequency filter on a two-dimensional pattern can be replaced with a frequency filter for a projection pattern after the finite Radon transform of the two-dimensional pattern. Therefore, this third
Determining the fringe direction based on the variance value of the projection pattern in which the low-frequency component is cut off as in the embodiment of Example 2 means that only the two-dimensional stripe pattern in which the low-frequency component is cut off, that is, only the outline of the stripe is extracted This is equivalent to determining the stripe direction based on the variance of the projection pattern of the pattern.

Since the outline pattern of the stripes represents the direction of the stripes more remarkably than the original stripe pattern, the second
Compared with the embodiment, it is possible to enhance the detection accuracy of the fringe. Moreover, in order to obtain the same effect as in the third embodiment, the original fringe pattern is subjected to a finite Radon transform after passing through a low-pass cutoff filter, and the fringe direction is determined based on the variance of the projected pattern. Is more effective than the method for determining This is because when the original fringe pattern is passed through the low-pass filter, 2
Although it is necessary to perform dimensional filtering, in the third embodiment, the same effect can be obtained only with a one-dimensional filter for the projection pattern, so that the processing amount can be reduced.

The fringe direction data S
When 1004 is output, the motion amount output unit 1004 inputs and stores the stripe direction data S1004, and
The motion amount output unit 1004 outputs all (t, s) constituting the moving image data S1001 input to the moving image processing apparatus.
x) When the input of the stripe direction data S1004 corresponding to the two-dimensional pattern and the (t, y) two-dimensional pattern is completed,
The motion of the moving image in the (x, y, t) three-dimensional space is expressed, for example, as a motion vector, and the moving image motion data S100
Output as 5.

(C-3) Effects of the Third Embodiment According to the third embodiment, although not described in detail, the same effects as in the above-described second embodiment can be obtained.

In addition to this, according to the third embodiment, a low-frequency cutoff pattern is obtained from a projection pattern using a simple one-dimensional filter, and the direction of the striped pattern is detected based on the low-frequency cutoff pattern. Therefore, an effect equivalent to detecting the direction of the stripe only from the contour portion of the stripe pattern can be obtained. That is, the motion of the moving image can be detected with higher accuracy.

(D) Fourth Embodiment Next, a fourth embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(D-1) Configuration of Fourth Embodiment FIG. 12 is a block diagram showing the configuration of a moving image processing apparatus according to the fourth embodiment. 12, a moving image processing apparatus according to the fourth embodiment includes a spatio-temporal axis conversion unit 1201,
A projection pattern creation unit 1202, a stripe direction detection unit 1203,
Motion amount output unit 1204 and low-frequency cutoff filter unit 1205
It is composed of Here, the spatiotemporal axis conversion unit 120
1. Projection pattern creation unit 1202, stripe direction detection unit 120
3 and the motion amount output unit 1204 are the same as the corresponding elements in the first embodiment, and a detailed description thereof will be omitted below.

The newly provided low-frequency cutoff filter unit 1205 in addition to the configuration of the first embodiment receives the two-dimensional pattern S1202 from the spatiotemporal axis conversion unit 1201 and
The two-dimensional filter pattern S1212 obtained by blocking the low-frequency component of the two-dimensional pattern S1202 is output to the projection pattern creation unit 1202. Therefore, the projection pattern creation unit 1202 of the fourth embodiment has the same function as that of the first embodiment, but has a different input.

Here, the low-pass cutoff filter section 1205 is added to the configuration of the second embodiment in comparison with the third embodiment.
As in the case of providing the low-frequency cutoff filter unit 1006 in the embodiment, the stripe direction is detected after the contour is extracted, thereby improving the detection accuracy of the stripe direction.

(D-2) Operation of the Fourth Embodiment Next, the operation of the moving image processing apparatus according to the fourth embodiment, which includes the above-described units 1201 to 1205, will be described.

When the moving image data S1201 is input,
After the spatio-temporal axis conversion unit 1201 stores moving image data for a predetermined time, the pixel data defined on the (x, y, t) three-dimensional space is converted into (t, x) for each y value. 2) A two-dimensional pattern and a (t, y) two-dimensional pattern for each value of x are sequentially created, and this two-dimensional pattern S1202 is output to the low-frequency cutoff filter unit 1205.

When such a two-dimensional pattern S1202 is input to the low-frequency cutoff filter unit 1205, the low-frequency cutoff filter unit 1205 creates a pattern from the two-dimensional pattern S1202 by removing its low-frequency components. The projection pattern creation unit 12 as the two-dimensional filter pattern S1212
02 is output.

As the low-frequency cutoff filter section 1205, a conventionally known method may be applied, and for example, a two-dimensional composition filter for performing the filtering shown in Expression (7) can be applied.

HP (x, y) = a · P (x−1, y−1) + b · P (x, y−1) + c · P (x + 1, y−1) + d · P (x−1, y) + eP (x, y) + fP (x + 1, y) + gP (x-1, y + 1) + hP (x, y + 1) + iP (x + 1, y + 1) (7) , P (x, y) are pixels (x, y) on the two-dimensional pattern S1202 input to the low-pass cutoff filter unit 1205.
y), the pixel value HP (x, y) is the pixel value of the pixel (x, y) on the two-dimensional filter pattern S1212, a, b,
c, d, e, f, g, h, and i are filter coefficients (for example, -1, -1, -1, and-, respectively) that determine the characteristics of the filter.
1, 8, -1, -1, -1, -1). Accordingly, when the two-dimensional pattern S1202 is a striped pattern, the pattern obtained by extracting only the contour of this strip is 2
It is obtained as a dimensional filter pattern S1212.

In the case of the fourth embodiment, in the following processing, the variance of the projection pattern with respect to the two-dimensional filter pattern S1212 is obtained. Since the variance value for the pattern is obtained, the difference between the variance value in the fringe direction and the variance value in the other directions becomes larger. Therefore, the detection accuracy of the fringe direction data S1204 is improved.

The operation after the projection pattern creation unit 1202 is the same as that of the first embodiment, and the description is omitted.

(D-3) Effects of the Fourth Embodiment According to the fourth embodiment, the same effects as those of the above-described first embodiment can be obtained, although not described in detail.

In addition to this, according to the fourth embodiment, since low-frequency components are removed from the two-dimensional pattern before the stripe pattern is projected, the variance in the stripe direction and the other The difference between the variance value in the direction and the variance value in the direction is increased, so that the effect of improving the accuracy of motion detection is obtained.

(E) Fifth Embodiment Next, a fifth embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

Although the above-described first and fourth embodiments are effective, as shown in FIG. 3 (g), the input moving image data moves diagonally, and the moving speed Is small, even if the spatio-temporal axis is converted, the striped pattern in the data after the conversion is not so clear, and thus the detection accuracy is reduced depending on the moving direction as compared with other moving directions.

From the fifth embodiment to the twelfth embodiment which will be described later, in consideration of such a point, a case where a clear stripe pattern is exhibited regardless of the moving direction of the input moving image data. And detects the amount of motion based on the clear striped pattern.

(E-1) Configuration of Fifth Embodiment FIG. 13 is a block diagram showing the configuration of a moving image processing apparatus according to the fifth embodiment. In FIG. 13, the moving image processing apparatus according to the fifth embodiment includes a geometric conversion unit 1301, a projection variance calculation unit 1302, and a motion detection unit 1303.

The geometric conversion unit 1301 receives moving image data S1301 (shown as A (X, Y, T)) from outside the apparatus, and receives a control parameter S104 from the motion detection unit 1303.
(Illustrated by θ) and input the geometric transformation data S1302 (C
(Illustrated by (P, T, Q)) to form the projection variance calculation unit 13
02 is output. Geometric transformation data S1302
Includes a striped pattern depending on the value of the control parameter S104.

The projection variance calculation unit 1302 includes the geometric transformation unit 1
The geometric transformation data S1302 is input from 301, the control parameter S1305 (illustrated by φ) is input from the motion detection unit 1303, and the projection distribution data S1303 of the projection pattern obtained by projecting the geometric transformation data S1302 in the direction defined by the control parameter S1305 ( (Illustrated by PV) and output to the motion detection unit 1303.

The motion detecting section 1303
1 and a control parameter S1305 to the projection variance calculation unit 1302, respectively.
Projection variance data S1303 from the projection variance calculation unit 1302
And the control parameters S1304 and S1305 at which the maximum projection dispersion data S1303 is generated.
The motion amount S1306 (illustrated by (V, α)) is formed based on the value of, and is output to the outside of the apparatus.

Here, the moving image data A (X, Y, T) is expressed on a three-dimensional space composed of two coordinate axes X and Y indicating a two-dimensional space and a coordinate axis T indicating time. Digital image data having a size of p × p × p. This moving image data A (X, Y, T) is, for example,
It is created by performing analog / digital conversion of video data captured by an imaging device such as a video camera, and then dividing the video data into an area having a size of p × p × p (hereinafter, referred to as a cube).

The motion amount (V, α) is data representing the motion corresponding to the cube, and the motion magnitude V
(The unit is pixels / frame) and the direction of movement α (the unit is radians [rad]).

The geometric transformation data C (P, T, Q) is represented by p × p on a three-dimensional space composed of three coordinate axes P, T, and Q.
Xp, of which the coordinate axis T is the same as the coordinate axis T on which the moving image data A (X, Y, T) is defined.

The control parameter S1304 (θ) and the control parameter S1305 (φ) are parameters associated with candidate values of the motion amount in the process of determining the motion amount (V, α). From another point of view, the control parameter S1304 (θ) is the input moving image data A (X, Y,
T) defines the amount of rotation with respect to the (X, Y) plane, and the control parameter S1305 (φ) defines the projection direction in the projection dispersion calculation unit 102.

(E-2) Operation of the Fifth Embodiment Next, the operation of the moving picture processing apparatus of the fifth embodiment composed of the units 1301 to 1303 as described above will be described.

This device starts operation in response to an operation start command from outside. At this time, first, the motion detection unit 1303
Sets two control parameters θ and φ ([rad]) to initial values, and sets the control parameter θ to the geometric transformation unit 1
In 301, the control parameter φ is assigned to the projection dispersion calculation unit 1302.
Respectively. For example, as an initial value, θ =
φ = −π / 2 [rad].

Next, the geometric transformation unit 1301 receives the control parameter θ from the motion detection unit 1303 and, based on this, the moving image data A (X, Y, T) inputted from outside.
To generate geometric transformation data C (P, T, Q),
Output to the projection variance calculation unit 1302.

FIG. 14 is a diagram for explaining the operation of converting data A (X, Y, T) into data C (P, T, Q) by the geometric conversion unit 1301.

The geometric conversion unit 1301 stores the moving image data A
When (X, Y, T) is input, first, the frame data B obtained by rotating the frame data at each time T by an angle-?
(P, Q, T) is created. Here, P and Q represent coordinate axes constituting the frame data after rotation. Next, a pattern on a two-dimensional plane composed of a coordinate axis P and a coordinate axis T is cut out for each Q of the moving image data B (P, Q, T), and this is geometrically transformed data C (P, T, T). Q).

FIG. 15 shows a case where the moving image data A (X, Y, T) and the moving image data B (P,
It is a figure explaining the method of rotation to (Q, T).

When the given control parameter θ is | θ | ≦
If it is π / 4 (FIG. 15A), a rotation process is performed according to the equation (8). Here, rnd means an operation for converting to an integer, and ｛x｝ mod p is an operation for converting a number x using p as a radix. ｛X｝ mod p = x-np (n is 0 ≦ x-
np <p).

B (P, Q, T) = A (P, ｛rnd (Ptan θ) + Q｝ mod p, T) (8) In this case, as shown in FIG. ) Line segments in the angle θ direction on the plane (A, B, etc. in the figure) are (P,
Q) On a plane, it is converted into a line segment in the angle 0 (horizontal) direction. Also, two line segments at different positions on the (X, Y) plane may be converted as the same horizontal line segment on the (P, Q) plane, as indicated by C1 and C2 in the figure. .
In any case, the rotation operation converts the straight line in the angle θ direction into a horizontal straight line, and all the pixels on the plane before rotation correspond one-to-one with the pixels on the plane after rotation. This ensures pixel preservation before and after rotation,
This means that the loss of information due to rotation is prevented.

On the other hand, given control parameter θ is |
If θ |> π / 4 (FIG. 15B), rotation is performed according to equation (9).

B (P, Q, T) = A ({rnd (Ptan (π / 2−θ)) + Q} mod p, P, T) (9) (2) In this case, FIG. As shown in the figure, the line segment (D, E, etc. in the figure) in the angle θ direction on the (X, Y) plane is (P,
Q) On a plane, it is converted into a line segment in the angle 0 (horizontal) direction. Also, two line segments at different positions on the (X, Y) plane may be converted as the same horizontal line segment on the (P, Q) plane, such as F1 and F2 in the figure. .
In any case, the rotation operation converts the straight line in the angle θ direction into a horizontal straight line, and all the pixels on the plane before rotation correspond one-to-one with the pixels on the plane after rotation. This ensures pixel preservation before and after rotation,
This means that the loss of information due to rotation is prevented.

Therefore, the moving image data A (X, Y, T)
Is a predetermined direction θ on the (X, Y) plane with time.
[rad] (| θ | <π / 2), if moving at the speed of the magnitude V [pixel / frame], as shown in FIG.
In (P, Q, T), the motion has a magnitude Vp [pixel / frame] in the horizontal direction (positive direction of the P axis). However, the value of Vp follows equation (10).

Vp = Vcos θ (when | θ | ≦ π / 4) = Vsin θ (when | θ |> π / 4) (10) Further, at this time, the two-dimensional pattern C ( P,
T, Q) forms a striped pattern according to the magnitude of motion Vp on the moving image data B (P, Q, T). Assuming that the direction of the striped pattern is φ, the relationship of equation (11) holds.

Tan φ = 1 / Vp (11) On the other hand, as time elapses, the moving image data A (X, Y, T) has a predetermined direction θ + π [rad] (| θ
| <Π / 2) at a speed of V [pixel / frame], the motion on the moving image B (P, Q, T) is large in the horizontal direction (negative direction of the P axis). Vp [pixel / frame]. However, Vp is equal to equation (10). As a result, the direction φ of the striped pattern appearing on the two-dimensional pattern C (P, T, Q) for each Q is as shown in Expression (12).

Tan φ = −1 / Vp (12) From the equations (11) and (12), the moving image data A (X, Y, T)
Is in the range of -π / 2 to + π / 2, the direction φ of the striped pattern on the two-dimensional pattern C (P, T, Q)
Is positive, otherwise φ is negative. In addition, it can be seen that between the moving image data A (X, Y, T) having the same magnitude of motion and the opposite direction, the magnitudes are equal only by changing the sign of φ.

Also, moving image data A (X, Y, T)
When moving in a direction other than θ or θ + π over time on the (X, Y) plane, a two-dimensional pattern B for each T
Since the movement on (P, Q, T) is not in the horizontal direction (direction parallel to the P axis), the two-dimensional pattern C (P,
In (T, Q), a striped pattern is not necessarily formed.

Next, when geometric transformation data C (P, T, Q) is input to the projection variance calculation unit 1302, the projection variance calculation unit 1302 stores this data. Further, the projection variance calculation unit 1302 receives the control parameter φ from the motion detection unit 1303 and inputs the geometric transformation data C (P,
T, Q), and outputs the projection dispersion data PV to the motion detection unit 1303.

FIG. 16 is a diagram for explaining the operation of the projection variance calculator 1302 according to the fifth embodiment.

The projection variance calculating unit 1302 projects the stored geometric transformation data C (P, T, Q) in the direction of the angle φ on the two-dimensional pattern of each Q, and
Calculate R1. That is, the two-dimensional pattern C for each Q
On (P, T, Q), a coordinate axis s is defined in a direction orthogonal to the angle φ direction, and for each point on the coordinate axis s, the sum of pixel values on a straight line in the angle φ direction passing therethrough is calculated. Is projection data PR1 for s. Further, the projection dispersion calculation unit 13
No. 02 calculates the total number of pixels that have contributed to the calculation for each s when obtaining the projection data PR1. Further, the projection variance calculation unit 1302 obtains projection data PR2 obtained by normalizing the projection data PR1 with the number of pixels that have contributed to the projection. That is, the projection data PR1 corresponding to each s is represented by PR1 (s),
Assuming that the projection data PR2 is PR2 (s) and the number of pixels contributing to projection is L (s), the projection variance calculation unit 1302 calculates (1
3) Calculate the projection data PR2 based on the equation.

PR2 (s) = PR1 (s) / L (s) (13) Next, the projection variance calculation unit 1302 converts the projection variance data PV into (14) based on the projection data PR2 obtained for each Q. ) And outputs it to the motion detection unit 1303. Where E
(X) represents the average value of the data x over all Qs.

PV = E (PR2 ² ) −E (PR2) (14) As shown in FIG. 16, a two-dimensional pattern C (P,
When (T, Q) forms a striped pattern, the change in the projection data PR2 is maximum and the projection dispersion data PV is also maximum for an angle φ equal to the direction of the stripe.
On the other hand, for an angle φ that does not match the direction of the stripes, the change in projection data PR2 is not clear, and as a result, projection dispersion data PV does not have a large value.

Next, upon input of the projection variance data PV from the projection variance calculation unit 1302, the motion detection unit 1303 stores this data in the maximum projection variance data PVm in which it is stored.
(In the initial state, the value 0 is stored.) If the projection dispersion data PV is larger, this value is stored in the maximum projection dispersion data P.
Vm, and the two control parameters θ and φ at this time are stored as θm and φm, respectively.

Then, the motion detecting section 1303 changes and resets the control parameters θ and φ within the range of -π / 2 to π / 2, and sets the geometric transformation section 1301 and the projection variance calculation section 13.
02 is output. In the fifth embodiment, the change is performed in units of π / 100. Each time the control parameters are reset, the geometric conversion unit 13
01, projection variance calculation unit 1302 and motion detection unit 1303
Is performed.

As a result, in the motion detecting section 1303, of the combinations of all the control parameters θ and φ, the parameter θ when the striped pattern on the geometric transformation data C (P, T, Q) becomes the clearest The projection dispersion data PV becomes the maximum value PVm with respect to the stripe direction φ at that time, and θ and φ at that time are stored as θm and φm.

As described above, the moving image data A (X,
When (Y, T) moves in the direction of θ or θ + π, a clear stripe pattern is formed in the geometric transformation data C (P, T, Q) obtained from the geometric transformation unit 1301 using the angle θ as a control parameter. , And the stripe direction is the projection variance calculation unit 13
02 is the control parameter φ when the projection dispersion data PV obtained from the motion detection unit 13 has the maximum value.
The parameters θm and φm stored in 03 are values corresponding to the motion of the moving image data A (X, Y, T).

Therefore, the motion detection unit 1303 calculates the motion amount S1306 ((V, α)) of the moving image data A (X, Y, T) based on the equations (15) and (16). Output to outside. The expressions (15) and (16) are obtained from the expressions (10) to (12).

V = 1 / | tan φm · cos θm | (when | θm | ≦ π / 4) = 1 / | tan φm · sin θm | (when | θm |> π / 4) (15) α = θm (in the case of | φm | ≦ π / 2) = θm + π (in the case of | φm |> π / 2) (16) As a result of the above operation, the moving image processing apparatus uses the moving image data A (X , Y, T).

As described above, in the fifth embodiment, the geometric conversion unit 1301 converts the moving image data A (X, Y, T) into the geometric conversion data C (P, T,
Q), the projection variance calculation unit 1302 calculates the projection variance data PV of the geometric transformation data C (P, T, Q) based on the control parameter φ, and the motion detection unit 1303 determines that the projection variance data PV is the maximum. Control parameters θ and φ are determined. The processing in the geometric transformation unit 1301 is performed by converting the moving image data A (X, Y, T) on the (X, Y) plane into an angle −θ.
To create a three-dimensional pattern B (P, Q, T), and then exchange the coordinate axes to convert the geometric transformation data C (P, T, T).
When moving image data A (X, Y, T) moves in the direction of angle θ or θ + π on the (X, Y) plane in order to obtain Q), the geometry on the (P, T) plane for each Q Conversion data C
A clear stripe pattern is formed on (P, T, Q), and the direction φ of the stripe corresponds to the magnitude of the movement. Accordingly, the projection variance calculation unit 1302 calculates the projection variance value of the geometric transformation data C (P, T, Q) in the direction φ, and the motion detection unit 1303 calculates the control parameter θ when the projection variance value becomes the maximum. And φ by detecting
The motion amount of the moving image A (X, Y, T) can be determined from the values of θ and φ.

(E-3) Effect of Fifth Embodiment As described above, in the fifth embodiment, the geometric transformation data C (P, P) on the (P, T) plane for each Q including the time axis T , T,
Q) Since the amount of motion is detected using the features of the above pattern, it is possible to accurately detect the motion even if the inter-frame motion is large, as compared with the conventional method of detecting the motion between frames by block matching. A series of motions can be grasped globally, and motion detection errors due to noise in an image can be reduced.

Further, according to the fifth embodiment, no matter what the moving direction of the input moving image data, the case where a clear stripe pattern is formed by changing the control parameter θ is created. Since the motion amount is detected based on the striped pattern, the motion amount can be detected well regardless of the moving direction of the input moving image data.

(F) Sixth Embodiment Next, a sixth embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(F-1) Configuration of the Sixth Embodiment FIG. 17 is a block diagram showing the configuration of a moving image processing apparatus according to the sixth embodiment. In FIG. 17, the moving image processing apparatus according to the sixth embodiment includes a geometric conversion unit 1701, a horizontal projection dispersion calculation unit 1702, and a motion detection unit 1703.

The geometric conversion unit 1701 receives moving image data S1701 (shown by A (X, Y, T)) from outside the apparatus, and receives control parameters S1704 from the motion detection unit 1703.
(Indicated by θ and φ) and input geometric transformation data S1702
(Shown by D (R, S, Q)) and output to the horizontal projection dispersion calculation unit 1702. Here, depending on the value of the control parameter S1704 (θ, φ), the geometric transformation data S1702 (D (R, S, Q)) is shifted in the horizontal direction (R
Direction).

The horizontal projection variance calculation unit 1702 receives the geometric transformation data S1702 from the geometric transformation unit 1701, and outputs the projection dispersion data S1703 of the horizontal projection pattern.
(Shown by PV) and output to the motion detection unit 1703.

The motion detecting section 1703 includes a
1 and the projection variance data S1703 from the horizontal projection variance calculation unit 1702, and the motion amount S1706 based on the control parameter S1704 when the maximum projection variance data S1703 is generated.
(Shown by (V, α)) and output to the outside of the apparatus.

Here, the geometric transformation data D (R, S, Q)
Is data having a size of p × p × p on a three-dimensional space composed of three coordinate axes R, S, and Q. The meaning of the moving image data A (X, Y, T), the amount of motion (V, α), and the control parameters θ and φ are the same as in the fifth embodiment.

(F-2) Operation of the Sixth Embodiment Next, the operation of the moving image processing apparatus according to the sixth embodiment, which includes the above-described units 1701 to 1703, will be described.

This moving image processing apparatus starts operation in response to an operation start command from the outside. At this time, first, the motion detection unit 1703 determines two control parameters θ and φ (however,
| Θ |, | φ | ≦ π / 2) to initial values (eg, θ = φ =
−π / 2 [rad]) and outputs the result to the geometric transformation unit 1701.

Next, the geometric conversion unit 1701 receives the control parameters θ and φ from the motion detection unit 1703, and based on the control parameters θ and φ, outputs the moving image data A (X,
Y, T) are converted to generate geometric conversion data D (R, S, Q), and output to the horizontal projection dispersion calculation unit 1702.

FIG. 18 is a diagram for explaining a conversion operation for obtaining data D (R, S, Q) from data A (X, Y, T) by the geometric conversion unit 1701.

The geometric conversion unit 1701 converts the moving image data A
When (X, Y, T) is input, first, the frame data B obtained by rotating the frame data at each time T by an angle-?
(P, Q, T) is created. Next, moving image data B
For each Q of (P, Q, T), a pattern on a two-dimensional plane composed of a coordinate axis P and a coordinate axis T is cut out and created as coordinate exchange data C (P, T, Q). Further, the geometric transformation unit 1701 calculates the plane data C for each Q
Plane data D obtained by rotating (P, T, Q) by an angle -φ
(R, S, Q) is created and output to the horizontal projection dispersion calculation unit 1702 as geometric transformation data.

From data A (X, Y, T) to data B
Since the conversion into (P, Q, T) is performed by the same method as in the first embodiment, a detailed description of the conversion method is omitted here. Note that, as described in the first embodiment, by this conversion, data B having the following characteristics is obtained.
(P, Q, T) is created.

The straight line in the direction of angle θ on (X, Y) is converted to a straight line in the horizontal direction. The movement of the speed V in the direction of the angle θ on (X, Y) is converted into the movement of the speed Vp (defined in the equation (10)) in the horizontal direction.

The conversion from data B (P, Q, T) to data C (P, T, Q) is performed by the same method as in the first embodiment. C (P, T, Q) having various characteristics are created.

The value of Vp for the movement of velocity V in the direction of angle θ at A (X, Y, T) (= the movement of velocity Vp in the horizontal direction for B (P, Q, T)) Is formed, and the direction φ of the stripe is determined by the expression (11) and the expression (12).
Follow the formula. The movement of A (X, Y, T) is represented by θ on the (X, Y) plane.
Or, when moving in the θ + π direction, the data C for each Q
A clear stripe pattern is formed in (P, T, Q), but otherwise, no stripe pattern is formed. A (X, Y, T) having the same magnitude of motion and opposite directions have the same sign but different magnitudes in the stripe direction φ. Note that the moving direction of A (X, Y, T) is -π / 2.
Φ is positive in the range of + π / 2, and φ is negative in other cases.

The conversion from data C (P, T, Q) to data D (R, S, Q) is performed on data A (X, Y, T).
To data B (P, Q, T). That is, the geometric transformation unit 1701 receives the control parameter φ from the motion detection unit 1703 and
Each plane pattern data C (P, T, Q) is rotated by -φ to obtain geometric transformation data D (R, S, Q). This rotation is performed by converting data A (X, Y, T) to data B (P, Q,
This is based on the same method as the conversion to T), and a detailed description is omitted.

By this processing, for data C (P, T, Q) having the same stripe direction as the control parameter φ, a stripe pattern along the horizontal direction (direction parallel to the R axis) is obtained.
It is formed on the geometric transformation data D (R, S, Q) for each Q. On the other hand, for data C (P, T, Q) having a stripe direction different from control parameter φ, no horizontal stripe pattern is formed on geometrically transformed data D (R, S, Q).

Next, when the geometric transformation data D (R, S, Q) is input to the horizontal projection variance calculation unit 1702, the horizontal projection variance calculation unit 1702 stores this data. Further, the horizontal projection variance calculation unit 1702 calculates the projection variance data PV on the geometric transformation data D (R, S, Q) for each Q, and outputs the calculated data to the motion detection unit 1703.

FIG. 19 is a diagram for explaining the operation of the horizontal projection variance calculation unit 1702 according to the sixth embodiment.

The horizontal projection variance calculation unit 1702 projects the stored geometric transformation data D (R, S, Q) in the horizontal direction (direction parallel to the R axis) for each Q, and Calculate PR1. That is, (R, S) for each Q
In the plane, the sum of the pixel values D (R, S, Q) is obtained in the horizontal direction, and this is defined as projection data PR1. Next, based on the projection data PR1 obtained for each Q, the horizontal projection variance calculation unit 1702 calculates the projection variance data PV by Expression (17), and outputs the calculated variance to the motion detection unit 1703. Where E
(X) represents the average value of the data x over all Qs.

PV = E (PR1 ² ) −E (PR1) (17) As shown in FIG. 19, the two-dimensional pattern D (R,
When (S, Q) forms a striped pattern extending in the horizontal direction, the change in the projection data PR1 is maximum, and the projection dispersion data PV is also maximum. On the other hand, when the two-dimensional pattern D (R, S, Q) for each Q does not form a stripe pattern along the horizontal direction, the change in the projection data PR1 is not clear, and as a result, the projection dispersion The data PV does not become a large value.

In the sixth embodiment, the data C
As a conversion method from (P, T, Q) to data D (R, S, Q), data A (X, Y, T) to data B (P,
Q, T), the different line segments on the data C (P, T, Q) are combined, and the data D (R, S, Q) are combined as one line segment. May appear above. The discontinuous line on the (R, S) plane in FIG. 19 indicates this. However, since only one discontinuous line occurs in the (R, S) plane, the projection data PR1
There is little effect on

Next, when the motion estimation unit 1703 receives the projection variance data PV from the horizontal projection variance calculation unit 1702, the motion estimation unit 1703 stores this data in the maximum projection variance data P
Vm (in the initial state, the value 0 is stored). If the projection dispersion data PV is larger, this value is stored as the maximum projection dispersion data PVm, and the two control parameters θ and φ at this time are set to θm, respectively. And φm.

Next, the motion detecting unit 1703 changes and resets the control parameters θ and φ within the range of -π / 2 to π / 2, and outputs the result to the geometric transformation unit 1701. In the sixth embodiment, the change is performed, for example, in units of π / 100. Each time the above control parameters are reset,
As described above, the operations of the geometric transformation unit 1701, the horizontal projection dispersion calculation unit 1702, and the motion detection unit 1703 are performed.

As a result, in the motion detecting section 1703, the parameter θ when the striped pattern on the geometric transformation data D (R, S, Q) becomes the clearest among all the combinations of the control parameters θ and φ. And φ, the projection dispersion data PV becomes the maximum value PVm, and θ
And φ will be stored as θm and φm.

As described above, the moving image data A (X,
When (Y, T) moves in the direction of θ or θ + π, when the angle θ and an appropriate angle φ according to the movement are given to the geometric transformation unit 1701 as control parameters, the geometric transformation data D (R,
Since a clear striped pattern is formed in S, Q), the parameters θm and φm stored by the motion detection unit 1703 are values corresponding to the motion of the moving image data A (X, Y, T). .

Therefore, the motion detecting section 1703 calculates the motion amount S1706 ((V) of the moving image data A (X, Y, T) based on the equations (15) and (16) as in the first embodiment. ,
α)) is calculated and output to the outside.

As a result of the above operation, the moving picture processing device can know the moving amount (V, α) for the moving picture data A (X, Y, T).

As described above, in the sixth embodiment, the geometric conversion unit 1701 converts the moving image data A (X, Y, T) into the geometric conversion data D (R, S, S) based on the control parameters θ and φ. Q), and the horizontal projection dispersion calculation unit 170
2 is the geometric transformation data D based on the control parameter φ.
The projection variance data PV of (R, S, Q) is calculated, and the motion detection unit 1703 determines the control parameters θ and φ when the projection variance data PV is maximized. Geometric transformation unit 170
The processing in (1) converts the moving image data A (X, Y, T) into (X,
Y) Three-dimensional pattern B rotated by an angle -θ on a plane
After creating (P, Q, T), 3
A dimensional pattern C (P, T, Q) is created, and this is rotated by -φ on the (R, S) plane to convert the geometric transformation data D
In order to obtain (R, S, Q), moving image data A (X, Y,
T) moves in the direction of the angle θ or θ + π on the (X, Y) plane, the angle φ corresponding to the magnitude of the movement is used as a control parameter, so that each Q can be moved on the (R, S) plane. A clear striped pattern along the horizontal direction is formed in the geometric transformation data D (R, S, Q). Therefore, the horizontal projection variance calculation unit 1702 calculates the geometric transformation data D (R, S, Q)
Of the horizontal direction is calculated, and the motion
03 detects the control parameters θ and φ when the projection dispersion value is the maximum, so that the amount of motion of the moving image A (X, Y, T) can be determined from the values of θ and φ.

(F-3) Effects of the Sixth Embodiment According to the sixth embodiment, the (P, T) plane for each Q including the time axis T is geometrically converted to the (R, S) plane. To
As a result, the amount of motion is detected using the feature of the pattern on the geometric transformation data D (R, S, Q) on the (R, S) plane in which the time change of the moving image is expressed. Compared to the method of detecting the movement between frames, even if the movement between frames is large, the movement can be accurately detected, and a series of movements can be grasped globally.
Motion detection errors due to noise in the image can be reduced.

Further, according to the sixth embodiment, no matter what the moving direction of the input moving image data, the case where a clear stripe pattern is formed by changing the control parameter θ is created. Since the motion amount is detected based on the striped pattern, the motion amount can be detected well regardless of the moving direction of the input moving image data.

Further, according to the sixth embodiment, the moving image data is converted into the horizontal geometric conversion data by the geometric conversion unit 1701, and the horizontal projection variance calculation unit 1702
Need only calculate the variance of the projection data PR1 in the horizontal direction only, so that the number of pixels contributing to projection is always constant,
Therefore, unlike the fifth embodiment, it is not necessary to calculate the number of pixels contributing to projection and normalize the projection data based on the number of pixels, so that the amount of calculation can be reduced.

If the amount of calculation is to be further reduced, the data D (R, S, Q) given from the geometric transformation unit 1701 to the horizontal projection variance calculation unit 1702, for example, as shown in FIG. Thus, Q1 = 0, Q2 = p
/ 2, Q3 = p-1 three data D (R, S, Q
1), D (R, S, Q2) and D (R, S, Q3) may be given.

(G) Seventh Embodiment Next, a moving image processing apparatus according to a seventh embodiment of the present invention will be described in detail with reference to the drawings.

(G-1) Configuration of Seventh Embodiment FIG. 21 is a block diagram showing the configuration of a moving image processing apparatus according to the seventh embodiment. In FIG. 21, the moving image processing apparatus according to the seventh embodiment includes a geometric transformation unit 2101, a horizontal projection dispersion calculation unit 2102, a motion detection unit 2103, and a search range setting unit 2104.

The geometric conversion unit 2101 and the horizontal projection variance calculation unit 2102 function in the same manner as the corresponding elements of the sixth embodiment.

The motion detecting section 2103 includes a search range setting section 2
104, search range data S2105 (shown by G) is input, horizontal variance calculation unit 2102 receives projection variance data S2103, outputs control parameter S2104 to geometric transformation unit 2101, and outputs motion amount S2106 outside the apparatus.
(Shown by (V, α)). The motion detection unit 2103 determines the range of the control parameter S2104 to be given to the geometric transformation unit 2101 based on the search range data S2105 (shown by G) from the search range setting unit 2104.

The search range setting unit 2104 newly provided in the seventh embodiment is provided with a moving image data S21 from the outside.
01 is input and the search range data S2105 is output to the motion detection unit 2103.

Here, the search range data G is V1, V
2, consisting of four data, α1 and α2,
The minimum value of the magnitude V of the motion amount, the maximum value of V, the minimum value of the direction α of the motion amount, and the maximum value of α.

(G-2) Operation of the Seventh Embodiment Next, the operation of the moving image processing apparatus of the seventh embodiment composed of the above-described units 2101 to 2104 will be described.

The moving image processing apparatus starts operation in response to an operation start command from outside. At this time, first, the search range setting unit 2104 externally receives the moving image data A (X, Y,
T) is input, and a motion vector between frames is calculated for a plurality of representative regions therein. In the seventh embodiment, moving image data A (X,
Block areas are set at seven places in total, including each surface of the cube constituting (Y, T) and the center of the cube.

FIG. 22 is a diagram showing an example of this representative area. As shown in FIG. 22, seven blocks are set as the representative region, including a block B0 located at the center of the cube and blocks B1 to B6 located near six squares.

The search range setting unit 2104 calculates a motion vector for each of these seven block areas. As a method for calculating the motion vector, pattern matching in block units described in the related art is applied. That is, the search range setting unit 2104 detects the amount of deviation from an adjacent frame by pattern matching for each block, and stores this as a motion vector. In order to detect the amount of the shift, image data outside the range of the cube is also required.
Operates in advance to input image data in a wider area in the X and Y directions than the cube to be processed.

The calculation of the motion vector in the search range setting unit 2104 indicates that the motion vector MVi = (vxi,
When v yi) is obtained, next, the search range setting unit 2104
The magnitude Vi [pixel / f of the motion vector for each block Bi
rame] and direction αi [rad] (−π to + π), and search range data G = (V1, V
2, α1, α2) and outputs the result to the motion detection unit 2103. That is, the calculation shown in Expression (18) is performed. In addition,
The parameter for searching for the minimum and maximum values in equation (18) is for i.

V1 = min ｛√ (vxi ² + vyi ² )｝ V2 = max ｛√ (vxi ² + vyi ² )｝ α1 = min ｛tan ^-1 (vyi / vxi)｝ α2 = min ｛tan ^-1 (vyi / vxi)｝ (18) Next, when the motion detection unit 2103 inputs the search range data G, first, (V, α) = (V1,
α1) is set. Further, the motion detection unit 2103 determines (1
The control parameters θ (| θ | ≦ π / 2) and φ corresponding to the motion amount candidates are calculated based on the expression 9), and the geometric conversion unit 2
Output to 101. Note that the relationship between the amount of motion (V, α) and the control parameters θ and φ is the same as that in the fifth embodiment, and the expression (19) is described in the description of the fifth embodiment and (10) It is derived from the relationship of Expressions (12) to (12).

Θ = α (when | α | ≦ π / 2) = α ± π (when | α |> π / 2) φ = tan ⁻¹ (1 / Vcos α) (| θ | ≦ π / 4) = tan ⁻¹ (1 / Vsin α) (when | θ |> π / 4) (19) Next, the geometric transformation unit 2101 inputs the control parameter θ from the motion detection unit 2103. Based on this, the moving image data A (X, Y, T) input from the outside is converted to create geometric conversion data D (R, S, Q), which is output to the horizontal projection dispersion calculation unit 2102. Note that the geometric transformation unit 2101
The details of the operation are the same as those of the sixth embodiment, and the description is omitted.

As a result, the geometric transformation data D (R, S,
Q) is the moving image data A (X,
For the control parameters θ and φ corresponding to the motion amounts (V, α) of (Y, T), a striped pattern is formed along the horizontal direction (the direction parallel to the R axis). On the other hand, moving image data A
For control parameters θ and φ that do not correspond to the amount of motion of (X, Y, T), geometric transformation data D (R, S,
Q) No stripe pattern along the horizontal direction is formed on the top.

Next, when the geometric transformation data D (R, S, Q) is input to the horizontal projection variance calculation unit 2102, the horizontal projection variance calculation unit 2102 stores this data. further,
The horizontal projection variance calculation unit 2102 calculates the projection variance data PV on the geometric transformation data D (R, S, Q) for each Q, and outputs it to the motion detection unit 2103. Horizontal projection dispersion calculator 21
02 is calculated by the same method as in the sixth embodiment, and as a result, the two-dimensional pattern D (R, S, Q) for each Q is a stripe pattern along the horizontal direction. Is formed, the projection variance data PV becomes the maximum, whereas in other cases, the projection variance data PV does not become a large value.

Next, when the motion detection unit 2103 receives the projection variance data PV from the horizontal projection variance calculation unit 2102, the motion detection unit 2103 stores this data in the maximum projection variance data P
Compared with Vm (the value 0 is stored in the initial state), if the projection variance data PV is larger, this value is stored as the maximum projection variance data PVm and the motion amount candidate (V, α) at this time is stored. I do.

Next, the motion detecting section 2103 changes and resets the motion amount candidate (V, α) within the range of V1 ≦ V ≦ V2 and α1 ≦ α ≦ α2, based on the equation (19). After obtaining the control parameters θ and φ, they are output to the geometric transformation unit 2101. In the seventh embodiment, the above change is performed in units of 0.5 [pixel / frame] for V and in units of π / 100 for α. Every time the control parameters are reset, the operations of the geometric transformation unit 2101, the horizontal projection dispersion calculation unit 2102, and the motion detection unit 2103 are performed in the same manner as described above.

As a result, in the motion detection section 2103, the search range set by the search range
The motion amount candidate (V, α) when the stripe pattern on the geometric transformation data D (R, S, Q) is the clearest is stored.

As described above, the geometric transformation data D (R,
The striped pattern on (S, Q) becomes the clearest depending on the control parameter θ and the control parameter θ obtained from the relationship of equation (19) according to the movement (V, α) of the moving image A (X, Y, T). Since the geometric transformation unit 2101 is operated using φ, the motion amount candidate when the above-mentioned stripe pattern becomes the clearest is the moving image A
It is equal to the actual motion amount of (X, Y, T).

Therefore, the motion detection section 2103 outputs the motion amount (V, α) stored therein to the outside.

As a result of the above operation, the moving picture processing apparatus can know the moving amount (V, α) for the moving picture data A (X, Y, T).

As described above, in the seventh embodiment, the search range setting unit 2104 sets the moving image data A (X,
(Y, T), a motion vector is obtained from the representative area, a search range of the motion amount is determined based on the motion vector, and the motion detection unit 2103 sets the motion amount candidate and the corresponding control parameters θ and φ based on the search range. Then, the geometric transformation unit 2101 determines whether the moving image data A (X,
Y, T) into geometric transformation data D (R, S, Q),
The horizontal projection variance calculation unit 2102 calculates the projection variance data PV of the geometric transformation data D (R, S, Q) based on the control parameter φ, and the motion detection unit 2103 calculates the projection variance data P
The amount of motion when V is maximized is determined. Geometric transformation unit 2
In the process at 101, the three-dimensional pattern B (P, Q, T) is created by rotating the moving image data A (X, Y, T) by an angle -θ on the (X, Y) plane, Creates a three-dimensional pattern C (P, T, Q) by exchanging, and rotates it by −φ on the (R, S) plane to obtain geometric transformation data D (R, S, Q). Moving image data A (X,
When (Y, T) moves in the direction of the angle θ or θ + π on the (X, Y) plane, the angle φ corresponding to the magnitude of the movement is used as a control parameter, so that the (R, S) plane of each Q A clear striped pattern along the horizontal direction is formed in the upper geometric conversion data D (R, S, Q). Therefore, the horizontal projection variance calculation unit 2102 outputs the geometric transformation data D (R, S,
By calculating a horizontal projection variance value of Q), the motion detection unit 2103 detects a motion amount corresponding to the control parameters θ and φ when the projection variance value is the maximum, thereby obtaining a moving image A ( (X, Y, T).

(G-3) Effects of the Seventh Embodiment According to the seventh embodiment, the same effects as those of the above-described sixth embodiment can be obtained.

In addition, according to the seventh embodiment,
Since the search range of the amount of motion is limited in advance by the search range setting unit 2104, the amount of calculation can be reduced as compared with the sixth embodiment. Moreover, in the seventh embodiment, the search range setting unit 2104 detects a motion vector, determines a search range that is likely to be a motion amount, and the motion detection unit 2103 detects a motion amount within the search range. In addition, a motion detection error due to the decrease in the amount of calculation hardly occurs, and the amount of calculation is small and highly accurate motion detection is possible.

(H) Eighth Embodiment Next, an eighth embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(H-1) Configuration of Eighth Embodiment FIG. 23 is a block diagram showing the configuration of a moving image processing apparatus according to the eighth embodiment. In FIG. 23, the moving image processing apparatus according to the eighth embodiment includes a geometric conversion unit 2301, a projection variance calculation unit 2302, a motion detection unit 2303, and a filter unit 2310. Here, the geometric converter 2301, the projection variance calculator 2302, and the motion detector 23
03 function in the same way as the corresponding components of the fifth embodiment described above, and detailed description thereof will be omitted.

The filter unit 2310 newly provided in the eighth embodiment converts the geometric transformation data S 2302 (C (P, T, Q)) inputted from the geometric transformation unit 2301 from the motion detection unit 2303. Control parameter S230
5 (φ) by performing low-frequency cutoff processing on the geometrically transformed edge data S2310 (C ′ (P,
T, Q)) to the projection variance calculation unit 2302.

Therefore, the projection variance calculation unit 2 of this embodiment
Reference numeral 302 denotes projection dispersion data S2 for the geometric transformation edge data S2310 input from the filter unit 2310.
303 (PV) is obtained and output to the motion detection unit 2303.

(H-2) Operation of Eighth Embodiment Next, each of the units 2301 to 2303, 2310 as described above
The operation of the moving image processing apparatus according to the eighth embodiment will be described.

This moving image processing apparatus starts operation in response to an operation start command from outside. At this time, first, the motion detection unit 2303 sets the two control parameters θ and φ to initial values, and sets the control parameter θ among them to the geometrical conversion unit 23.
01, and outputs the control parameter φ to the filter unit 2310 and the projection variance calculation unit 2302, respectively.

Thereafter, the current control parameters θ and φ
Transformation unit 2301, projection dispersion calculation unit 230
2. Each time a series of processes of the motion detection unit 2303 and the filter unit 2310 ends, the motion detection unit 2303 changes the control parameters θ and φ within a predetermined range.

The operations of the geometric converter 2301, the projection variance calculator 2302, and the motion detector 2303 for certain control parameters θ and φ are the same as those in the fifth embodiment, and will be described in detail.

Moving image data A (X,
Geometric transformation data C (P, T, Q) obtained by transforming (Y, T) by the geometric transformation unit 2301 as shown in FIG. 24 is input to the filter unit 2310.

The geometric conversion data C
When (P, T, Q) is input, the filter unit 2310
Detects the edge pattern parallel to the angle φ direction in the two-dimensional pattern C (P, T, Q) for each Q, and projects the detected geometrically transformed edge data C ′ (P, T, Q) by projection dispersion. Output to calculation section 2302.

This geometrically transformed edge data C ′ (P, T,
As a calculation method of Q), a conventionally known edge detection method can be applied. In this embodiment, in order to particularly detect an edge pattern parallel only to the angle φ direction, for example, the geometric conversion edge data C ′ is obtained by the following procedures (1) and (2).
(P, T, Q) is calculated.

Procedure (1) Slope (gp, gt) of pixel value for each pixel of pattern C (P, T, Q) = (√C / √P,
√T / √P) is calculated.

Procedure (2) Two conditions, p1 <√ (gp ²
+ Gt ² ) and p2 <tan ⁻¹ (gt / gp) <p3
C ′ (P, T, Q) = 1 (edge region) for pixels satisfying the following condition, and C ′ (P, T, Q) for other pixels.
Q) = 0 (non-edge area). Here, p1 is a parameter representing the minimum value of the amount of change between pixels determined to be edges. For example, when a pixel value is described in 8 bits, it is a constant of about 16 to 64. p2 and p3 are parameters for determining that the edge is parallel to the angle φ direction, and p2 = φ−Δφ + π / 2 and p3 = φ + Δφ + π /
2, and Δφ is a constant of about π / 50.

By applying such a calculation method, only edges having a certain steep change among edges parallel to the direction close to the angle φ direction are detected, and based on this, the geometrically transformed edge data C ′ (P , T, Q) are output to the projection variance calculation unit 2302.

In FIG. 24 described above, the filter unit 2310 extracts an edge pattern parallel to the angle φ in the geometric transformation data C (P, T, Q), and extracts the geometric transformation edge data C ′.
The case where (P, T, Q) is created is shown.

The projection variance calculator 2302 has a difference that the geometrically transformed edge data C ′ (P, T, Q) is input as input data, but the processing content is the same as that of the fifth embodiment. It is the same as the calculation unit. That is, the control parameter φ is input from the motion detection section 2303, and each Q is input.
The projection variance data PV is calculated on the geometric transformation data C ′ (P, T, Q) for each and output to the motion detection unit 2303.

FIG. 25 is a diagram for explaining the operation of the projection variance calculation unit 2302 according to the eighth embodiment. FIG.
(A) illustrates the operation of the configuration of the eighth embodiment, while (b) of FIG.
10 illustrates an operation when there is no 10 (see the fifth embodiment), that is, illustrates a comparative example for explaining an effect of the filter unit 2310.

As shown in FIG. 25A, when the two-dimensional pattern C ′ (P, T, Q) for each Q forms a striped pattern, the angle φ equal to the direction of the striped pattern is obtained. In contrast, the change of the projection data PR2 becomes the maximum, and the projection dispersion data PV also becomes the maximum. On the other hand, for an angle φ that does not match the direction of the stripes, the change in projection data PR2 is not clear, and as a result, projection dispersion data PV does not have a large value.

On the other hand, if the geometric transformation data C (P, T, Q) is directly input to the projection variance calculation unit 2302 without using the filter unit 2310 (see the fifth embodiment), Even if the two-dimensional pattern C (P, T, Q) forms a striped pattern, as shown in FIG. 25B, the projection data PR1 and PR2 have a small variation in change, and the projection dispersion data PV May not be the maximum. This occurs when the dynamic range of the pixel values constituting the input image is narrow, because the pixel values between the two regions bordering the edge are close.

For such an input image, the eighth
As in the embodiment, the filter unit 2310 detects an edge of the geometric transformation data C (P, T, Q) in advance, and projects the geometric transformation edge data C ′ (P, T, Q) obtained as a result. If the pattern is obtained, as shown in FIG.
The projection data PR2 will have a clear change.

Therefore, the filter unit 2310 in the eighth embodiment has an effect of giving a clear change to the projection data for the striped pattern.

Since the same operation as in the fifth embodiment is performed except for the difference in operation due to the provision of the filter unit 2310 as described above, further description of the operation will be omitted.

As described above, in the eighth embodiment, the geometric conversion unit 2301 converts the moving image data A (X, Y, T) into the geometric conversion data C based on the control parameter θ.
(P, T, Q), and the filter unit 2310 converts the geometrically transformed data C (P, T, Q) based on the control parameter φ.
Geometric edge data C ′ (P, T,
Q), and the projection variance calculation unit 2302 generates the geometrically transformed edge data C ′ (P, T,
The projection dispersion data PV of Q) is calculated, and the motion detection unit 230 is calculated.
3 determines the control parameters θ and φ when the projection dispersion data PV is maximized. In the processing performed by the geometric transformation unit 2301, the moving image data A (X, Y, T) is rotated by an angle -θ on the (X, Y) plane, and the three-dimensional pattern B (P, Q,
T), the geometric transformation data C (P, T, Q) is obtained by exchanging the coordinate axes. After that, the filter unit 2310 further performs the geometric transformation representing the edge of the geometric transformation data C (P, T, Q). Since the edge data C ′ (P, T, Q) is calculated, if the moving image data A (X, Y, T) moves in the angle θ or θ + π direction on the (X, Y) plane, , The geometrically transformed edge data C ′ (P, T) on the (P, T) plane
T, Q), a clear stripe pattern is formed.
The direction φ of the stripe corresponds to the magnitude of the movement.

Therefore, the projection variance calculation unit 2302 calculates the projection variance value of the geometrically transformed edge data C ′ (P, T, Q) in the direction φ, and the motion detection unit 2303 maximizes the projection variance value. By detecting the control parameters θ and φ in the case, the moving image A (X, Y, T) is obtained from the values of θ and φ.
Can be determined.

(H-3) Effects of the Eighth Embodiment The eighth embodiment can provide the same effects as those of the moving picture processing device of the fifth embodiment described above.

In addition to this, according to the eighth embodiment,
A filter unit 2310 is provided, and the geometric transformation edge data C ′ is provided.
After performing the projection variance calculation process on (P, T, Q), the motion amount is detected. Therefore, regardless of the dynamic range of the pixel values constituting the input image, the motion amount is stably determined. Can be detected.

(I) Ninth Embodiment Next, a ninth embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(I-1) Configuration of the Ninth Embodiment FIG. 26 is a block diagram showing the configuration of a moving image processing apparatus according to the ninth embodiment. In FIG. 26, the moving image processing apparatus according to the ninth embodiment includes a geometric conversion unit 2601, a horizontal projection variance calculation unit 2602, a motion detection unit 2603, and a filter unit 2610.

The above-described eighth embodiment is similar to the above-described fifth embodiment.
Although a filter unit is added to the technical idea of the sixth embodiment, the ninth embodiment is obtained by adding a filter unit to the technical idea of the sixth embodiment described above.
Therefore, the geometric transformation unit 2601 and the horizontal projection dispersion calculation unit 2
602 and the motion detection unit 2603 each function similarly to the corresponding component of the sixth embodiment.
The detailed description is omitted.

The filter unit 2610 newly provided in the ninth embodiment performs the low frequency cutoff processing on the geometric conversion data S2602 input from the geometric conversion unit 2601, and obtains the geometric conversion edge data S2610 ( D '
(R, S, Q)) to the horizontal projection dispersion calculation unit 2602. The meaning of providing the filter unit 2610 is the same as that of the eighth embodiment.

As described above, the horizontal projection variance calculation unit 2602 does not receive the geometric transformation data S2602 output from the geometric transformation unit 2601, but the filter unit 261.
Since the geometric transformation edge data S2610 output from 0 is input, the horizontal projection variance calculation unit 2602 processes the geometric transformation edge data S2610 to obtain projection variance data S2603 (PV) and outputs it to the motion detection unit 2603. I do.

(I-2) Operation of the Ninth Embodiment Next, the components 2601 to 2603 and 2610 as described above
The operation of the moving image processing apparatus according to the ninth embodiment will be described.

As described above, the geometric transformation unit 260
1, horizontal projection dispersion calculation unit 2602 and motion detection unit 260
3 operate and operate in the same manner as the corresponding components of the sixth embodiment, and therefore, the description of the operation thereof is omitted, and the following mainly describes the operation of the filter unit 2610.

The geometric conversion unit 2601 sequentially converts the input moving image data A (X, Y, T) as shown in FIG. 27 and as described in the operation of the sixth embodiment. The obtained geometric transformation data D (R, S, Q) is applied to the filter unit 2
610.

The geometric conversion data D is supplied to the filter unit 2610.
When (R, S, Q) is input, the filter unit 2610
Detects the edge pattern parallel to the horizontal direction in the two-dimensional pattern D (R, S, Q) for each Q, and converts the obtained geometrically transformed edge data D ′ (R, S, Q) to the horizontal direction. Output to the projection variance calculation unit 2602.

This geometrically transformed edge data D '(R, S,
As the calculation method of Q), similarly to the eighth embodiment, a conventionally known edge detection method can be applied. In the ninth embodiment, the method described in the eighth embodiment is modified, and the geometric transformation edge data D ′ (R, S, Q) is calculated by the following procedures (1) and (2).

Procedure (1) The gradient (gr, gs) of the pixel value for each pixel of the pattern D (R, S, Q) = (√D / √R,
√D / √S) is calculated.

Procedure (2) Two conditions, p1 <√ (gr ²
+ Gs ² ) and p2 <tan ⁻¹ (gs / gr) <p3
D ′ (R, S, Q) = 1 (edge region) for pixels satisfying the following condition, and D ′ (R, S, Q) for other pixels.
Q) = 0 (non-edge area). Here, p1 is a parameter representing the minimum value of the amount of change between pixels determined to be edges. For example, when a pixel value is described in 8 bits, it is a constant of about 16 to 64. p2 and p3 are parameters for judging that they are parallel in the horizontal direction (direction parallel to the R axis), and p2 = −Δφ + π / 2 and p3 = Δφ +
π / 2, and Δφ is a constant of about π / 50.

By applying this calculation method, only edges having a certain degree of sharp change among edges close to the horizontal direction are detected, and geometric transformation edge data D '(R, S, Q) is output based on the detected edges. I do.

As a result, as shown in FIG.
The filter unit 2610 converts the geometric transformation data D (R, S, Q)
The middle horizontal edge pattern is extracted, and geometric transformation edge data D '(R, S, Q) is created.

The horizontal projection variance calculation unit 2602 has a difference that the geometrically converted edge data D ′ (R, S, Q) is input as input data.
This is the same as the horizontal projection variance calculation unit of the embodiment. That is, the horizontal projection variance calculation unit 2602 uses the projection variance data P ′ on the geometric transformation edge data D ′ (R, S, Q) for each Q.
V is calculated and output to the motion detection unit 2603.

FIG. 28 is a diagram for explaining the operation of the horizontal projection dispersion calculation unit 2602 according to the ninth embodiment. FIG. 28A illustrates the operation according to the configuration of the ninth embodiment, while FIG. 28B illustrates the operation when the filter unit 2610 is not used, and conversely. For example, a comparative example for explaining the effect of the filter unit 2610 is shown.

As shown in FIG. 28A, when the two-dimensional pattern D '(R, S, Q) for each Q forms a striped pattern along the horizontal direction, the projection data PR1 Is maximum, and the projection dispersion data PV is also maximum.
On the other hand, when D ′ (R, S, Q) does not form a stripe pattern along the horizontal direction, the change in the projection data PR1 is not clear, and as a result, the projection dispersion data PV has a large value. Not be.

On the other hand, if the filter section 2610 is not used,
When the geometric transformation data D (R, S, Q) is directly input to the projection variance calculation unit 2602, even if the two-dimensional pattern D (R, S, Q) for each Q forms a striped pattern. As shown in FIG. 28B, the change of the projection data PR1 is not so large, and there is a possibility that the projection dispersion data PV does not become the maximum. This occurs when the dynamic range of the pixel values constituting the input image is narrow, because the pixel values between the two regions bordering the edge are close.

For such an input image, as in the ninth embodiment, the filter unit 2610 detects the edge of the geometric conversion data D (R, S, Q) in advance, and obtains the resulting geometric image. If a projection pattern is obtained for the converted edge data D '(R, S, Q), the projection data PR1 has a clear change as shown in FIG.

Therefore, the filter section 2610 according to the ninth embodiment has the effect of giving a clear change to the projection data for the striped pattern.

Except for the difference in operation due to the provision of the filter unit 2610 as described above, the same operation as in the sixth embodiment is performed, so that further description of the operation will be omitted.

As described above, in the ninth embodiment, the geometric conversion unit 2601 converts the moving image data A (X, Y, T) into the geometric conversion data D (R, S, Q), the filter unit 2610 creates geometric transformation edge data D ′ (R, S, Q) indicating an edge in the geometric transformation data D (R, S, Q), and the horizontal projection variance calculation unit 2602 Geometric transformation edge data D '(R,
(S, Q) of the projection variance data PV
603 determines the control parameters θ and φ when the projection dispersion data PV is maximized.

The processing in the geometric conversion unit 2601 is performed by rotating the moving image data A (X, Y, T) by an angle -θ on the (X, Y) plane to obtain a three-dimensional pattern B (P, Q, T). Is created, the three-dimensional pattern C (P, T,
Q) is created and further rotated by −φ on the (R, S) plane to obtain geometric transformation data D (R, S, Q).
Thereafter, the filter unit 2610 outputs the geometric transformation data D
Since the geometric transformation edge data D '(R, S, Q) representing the edge of (R, S, Q) is calculated, the moving image data A
(X, Y, T) is the angle θ or θ + π on the (X, Y) plane.
When moving in the direction, the angle φ corresponding to the magnitude of the movement is used as a control parameter, so that (R,
S) Geometric transformation edge data D '(R, S, Q) on a plane
Has a clear stripe pattern along the horizontal direction.

Accordingly, the horizontal projection variance calculation unit 2602 calculates the horizontal variance of the geometrically transformed edge data D ′ (R, S, Q), and the motion detection unit 2603 determines that the maximum variance is maximum. By detecting the control parameters θ and φ in the following case, the moving image A (X,
Y, T).

(I-3) Effects of the Ninth Embodiment The ninth embodiment can also provide the same effects as those of the above-described sixth embodiment.

In addition to this, according to the ninth embodiment,
The (P, T) plane for each Q including the time axis T is geometrically converted to the (R, S) plane, and thereby the time change of the moving image is expressed on the (R, S) plane. Since the amount of motion is detected using the feature of the pattern on the converted edge data D '(R, S, Q), the amount of motion is detected stably regardless of the dynamic range of the pixel values constituting the input image. be able to.

(J) Tenth Embodiment Next, a moving image processing apparatus according to a tenth embodiment of the present invention will be described in detail with reference to the drawings.

(J-1) Configuration of the Tenth Embodiment FIG. 29 is a block diagram showing the configuration of a moving image processing apparatus according to the tenth embodiment. In FIG. 29, the first
0 of the moving image processing apparatus according to the first embodiment.
1. It comprises a horizontal projection variance calculation unit 2902 with a filter and a motion detection unit 2903.

In the tenth embodiment, similarly to the ninth embodiment, a filtering function is added to the technical idea of the sixth embodiment. But the first
0 is the same as the filter unit 2610 of the ninth embodiment.
By providing a horizontal projection variance calculation unit 2902 with a filter obtained by fusing the horizontal projection variance calculation unit 2602 with the horizontal projection variance calculation unit 2602, a filtering function is added to the technical idea of the sixth embodiment described above. Therefore, the geometric conversion unit 2901 and the motion detection unit 2903 function in the same manner as the corresponding components of the sixth embodiment, and a detailed description thereof will be omitted.

Horizontal variance calculator with filter 2902
Is obtained from the geometric transformation data S2902 by the geometric transformation unit 2901.
(D (R, S, Q)), and the projection variance data S2 obtained after performing projection processing and low-frequency cutoff processing on it.
903 (PV) is output to the motion detection unit 2903.

(J-1) Operation of the Tenth Embodiment Next, the operation of the moving image processing apparatus according to the tenth embodiment, which includes the units 2901 to 2903 as described above, will be described.

As described above, the geometric transformation unit 290
1 and the motion detection unit 2903 function and operate in the same manner as the corresponding components of the sixth embodiment, so that the description of the operation is omitted, and the horizontal projection variance calculation unit with filter 2902 will be described below. The operation will be mainly described.

The geometric conversion unit 2601 sequentially converts the input moving image data A (X, Y, T) as shown in FIG. 18 and described in the operation of the sixth embodiment. The obtained geometric transformation data D (R, S, Q) is provided to the horizontal projection dispersion calculation unit with filter 2902.

When the geometric transformation data D (R, S, Q) is input to the horizontal projection variance calculation unit with filter 2902, the horizontal projection variance calculation unit with filter 2902 firstly converts the geometric transformation data D ( R, S, Q) to calculate the projection data PR1 in the horizontal direction.

Further, the horizontal projection calculation unit with filter 290
2 performs edge detection processing (filtering) on the projection data PR1 and outputs the result to the projection data PR2.
And As the edge detection method, a conventionally known edge detection method can be applied. For example, a convolution filter based on local data on the projection data PR1 is used. That is, the projection data PR2 is (20)
It can be calculated by an equation. Here, w (i) is a filter coefficient, for example, w (0) = 1.0, w
(1) = w (−1) = 0.5, w (eise) = 0. The sum Σ in the equation (20) is for i.

PR2 (s) = | Σw (i) PR1 (s + i) | (20) By applying the convolution filter expressed by the equation (20), if the change in the projection data PR1 is small, the projection data PR2 becomes The value becomes close to 0. Conversely, if the change in the projection data PR1 is large, the projection data PR2 has a large value. Therefore, the projection data PR2 has a value corresponding to the change amount of the projection data PR1.

Next, the horizontal projection dispersion calculation unit with filter 29
02 calculates the variance of the projection data PR2 over all the Q values, and outputs this to the motion detection unit 2903 as projection variance data PV.

FIG. 30 is a diagram for explaining the operation of the horizontal projection variance calculating unit with filter 2902 according to the tenth embodiment.

The inputted geometric transformation data D (R, S,
If Q) forms a horizontal stripe pattern (= if the geometric transformation unit 2901 is operated using the control parameters θ and φ corresponding to the motion of the moving image), if the dynamic range of the image is narrow, The projection data PR1 obtained by the horizontal projection dispersion calculation unit with filter 2902 may not show a clear change. When the projection data PR2 is obtained by the calculation of the expression (20), a pattern having a clear change according to the change point of the projection data PR1 is obtained. Therefore, the projection dispersion data PV has a large value.

On the other hand, the input geometric transformation data D (R,
S, Q) do not form a horizontal stripe pattern (= control parameters θ and φ that do not correspond to the motion of a moving image)
When the geometric transformation unit 2901 is operated by using), the projection data PR1 has a pattern with less change than in the above case, so that the projection data PR2 also has no clear change, and therefore, the projection variance. The data PV has a small value.

Although it is possible to know the difference between the presence and absence of a striped pattern by the variance value for the projection data PR1, as in the tenth embodiment, the difference between the projection data PR2 and the edge-detected projection data PR2 is determined. By using the variance value, the difference between the variance value in the case of exhibiting a striped pattern and the variance value in the case of not exhibiting a striped pattern is widened, so that it is easy to determine both cases based on the variance value. become.

The processing performed by the motion detecting section 2903 thereafter is the same as in the sixth embodiment, as described above.

As described above, in the tenth embodiment, the geometric transformation unit 2901 transforms the moving image data A (X, Y, T) into the geometric transformation data D (R, S, S) based on the control parameters θ and φ. Q), and the horizontal projection variance calculation unit with filter 2902 calculates the projection variance data PV based on the change amount of the projection data of the geometric transformation data D (R, S, Q),
The motion detection unit 2903 determines the control parameters θ and φ when the projection dispersion data PV is maximized.

The processing in the geometric transformation unit 2901 is such that the moving image data A (X, Y, T) is rotated on the (X, Y) plane by an angle -θ to obtain a three-dimensional pattern B (P, Q, T). Is created, and only three Q values are changed by exchanging the coordinate axes.
A dimensional pattern C (P, T, Q) is created, and this is rotated by -φ on the (R, S) plane to convert the geometric transformation data D
In order to obtain (R, S, Q), moving image data A (X, Y,
T) moves in the direction of the angle θ or θ + π on the (X, Y) plane, the angle φ corresponding to the magnitude of the movement is used as a control parameter, so that each Q can be moved on the (R, S) plane. A striped pattern along the horizontal direction is formed on the geometric transformation data D (R, S, Q). Further, by the processing in the horizontal projection dispersion calculation unit with filter 2902, the projection dispersion value becomes a large value regardless of the dynamic range of the striped pattern.

Therefore, the motion detector A 2903 detects the control parameters θ and φ when the projection variance value is maximized, and the motion image A (X,
Y, T).

(J-3) Effects of the Tenth Embodiment The tenth embodiment can provide the same effects as the ninth embodiment described above.

In addition, since the filtering process is performed on the projection data PR1 which is one-dimensional data, the ninth
There is also an effect that the amount of calculation can be reduced as compared with the embodiment.

(K) Eleventh Embodiment Next, an eleventh embodiment of the moving image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(K-1) Configuration of Eleventh Embodiment FIG. 31 is a block diagram showing a configuration of a moving image processing apparatus according to the eleventh embodiment. In FIG. 31, the first
The moving image processing apparatus according to the first embodiment includes a geometric conversion unit 310
1, horizontal projection dispersion calculation unit 3102, motion detection unit 310
3, a filter unit 3110 and a pattern superimposing unit 3111.

As is clear from the comparison from FIGS. 31 and 26, the eleventh embodiment is characterized in that a pattern superimposing section 3111 is further provided in the configuration of the ninth embodiment. is there. Therefore, the geometric transformation unit 31
01, horizontal projection dispersion calculation unit 3102, motion detection unit 310
3 and the filter unit 3110 function similarly to the corresponding components of the ninth embodiment. However, the horizontal projection variance calculation unit 3102 differs from the ninth embodiment in the input data due to the provision of the pattern superimposition unit 3111, and the processing is slightly different from that of the ninth embodiment. ing.

The pattern superimposing section 3111 includes the filter section 3
110, the geometric transformation edge data S3110 (D ′
(R, S, Q)) and (R, S) for each Q value
Superimposed data S3111 (E) obtained by superimposing two-dimensional data
(R, S)) to the horizontal projection dispersion calculation unit 3102. The superimposition data E (R, S) is data defined on a plane defined by the axis R and the axis S among the three axes defining the geometrically transformed edge data D ′ (R, S, Q). .

The horizontal projection variance calculator 310 of this embodiment
2 is superimposed data S311 from the pattern superimposing unit 3111
1 and inputs the projection dispersion data S3 to the motion detection unit 3103.
103 (PV) is output.

(K-2) Operation of Eleventh Embodiment Next, each of the units 3101 to 3103 and 311 as described above
The operation of the moving image processing apparatus according to the eleventh embodiment, which includes 0, 3111, will be described.

As described above, the geometric transformation unit 310
1. Since the motion detecting unit 3103 and the filter unit 3110 each function and operate similarly to the corresponding components of the ninth embodiment, the description of the operations is omitted.
Hereinafter, the operation of the pattern superimposing unit 3111 will be mainly described, and the point that the horizontal projection dispersion calculating unit 3102 operates differently from the ninth embodiment due to the provision of the pattern superimposing unit 3111 will be described.

When the geometrically transformed edge data D ′ (R, S, Q) is input to the pattern superimposing section 3111, the pattern superimposing section 3111 outputs the two-dimensional pattern D ′ (R, S,
Q) is superimposed to calculate superimposition data E (R, S). In the eleventh embodiment, as this pattern superimposition method, E (R, S) = {D '(R, S, Q) is calculated (the sum Σ is about Q). Pixels (R, R) on the converted edge data D ′ (R, S, Q)
If the value of S) is reflected on the value of the corresponding pixel (R, S) on the superimposition data E (R, S), another suitable method may be used.

FIG. 32 is a diagram illustrating the operation of pattern superimposing section 3111. As described in the ninth embodiment, by using the control parameters θ and φ corresponding to the motion of the moving image A (X, Y, T), the geometrically transformed edge data D ′ (R, S, Q) is An edge pattern extending in the horizontal direction as shown in FIG. 32 is formed. These patterns differ according to the value of each Q, but have a common feature of forming an edge pattern extending in the horizontal direction. Therefore, by superimposing these patterns, more edge patterns in the horizontal direction are formed on the superimposition data E (R, S). On the other hand, when the control parameters θ and φ that do not correspond to the motion of the moving image A (X, Y, T) are used, the pattern on the superimposition data E (R, S) is the edge pattern aligned in the horizontal direction. Not formed.

When superimposition data E (R, S) is input to horizontal projection variance calculation section 3102, horizontal projection variance calculation section 31
02 calculates projection data PR1 in the horizontal direction (direction parallel to the R axis) with respect to the superimposition data E (R, S),
Further, a variance value of the projection data PR1 is obtained, and this is output to the motion detection unit 3101 as projection variance data PV.

Therefore, since the projection data PR1 has a large change with respect to the superimposed data E (R, S) having an edge pattern extending in the horizontal direction as shown in FIG. 32, the projection dispersion data PV has a large value. . On the other hand, the superimposition data E
When an edge pattern along the horizontal direction is not formed at (R, S) (= using the control parameters θ and φ that do not correspond to the motion of the moving image, the geometric conversion unit 3101, the filter unit 3110, and the pattern superimposition unit 3111 ), The projection data PR1 does not have a clear change,
As a result, the projection dispersion data PV has a small value.

The motion detecting section 3103 calculates
When V is input, it operates similarly to the ninth embodiment.

As described above, in the eleventh embodiment, the geometric conversion unit 3101 converts the moving image data A (X, Y, T) into the geometric conversion data D (R, S, Q), the filter unit 3110 creates geometrically transformed edge data D ′ (R, S, Q) indicating an edge in the geometrically transformed data D (R, S, Q), and the pattern superimposing unit 3111 creates the geometrically transformed edge data D ′ (R, S, Q). Edge data D ′ (R, S,
Q) The pattern for each Q is superimposed to form a superimposed pattern E
(R, S) is calculated, the horizontal projection variance calculation unit 3102 calculates the projection variance data PV of the superimposed pattern E (R, S), and the motion detection unit 3103 controls the control parameter when the projection variance data PV is maximized. Determine θ and φ.

The processing in the geometric transformation unit 3101 is such that the moving image data A (X, Y, T) is rotated by an angle -θ on the (X, Y) plane to obtain a three-dimensional pattern B (P, Q, T). Is created, and the three-dimensional pattern C (P, T,
Q) is created and further rotated by −φ on the (R, S) plane to obtain geometric transformation data D (R, S, Q).
After that, the filter unit 3110 outputs the geometric transformation data D
Since the geometric transformation edge data D '(R, S, Q) representing the edge of (R, S, Q) is calculated, the moving image data A
(X, Y, T) is the angle θ or θ + π on the (X, Y) plane.
When moving in the direction, the angle φ corresponding to the magnitude of the movement is used as a control parameter, so that (R,
S) Geometric transformation edge data D '(R, S, Q) on a plane
Has a clear stripe pattern along the horizontal direction.

Therefore, the pattern superimposing section 3111 superimposes the geometrically transformed edge data D '(R, S, Q) to create superimposed data E (R, S), and the horizontal projection dispersion calculating section 3102
Calculates the projection variance in the horizontal direction of the superimposition data E (R, S), and the motion detection unit 3103 detects the control parameters θ and φ when the projection variance is maximized. The motion amount of the moving image A (X, Y, T) can be determined from the value of φ.

(K-3) Effects of the Eleventh Embodiment According to the eleventh embodiment, the same effects as those of the ninth embodiment can be obtained.

In addition to the above, according to the eleventh embodiment, the calculation of the projection dispersion data PR1 is based on the superposition data E1.
Since it suffices to perform only on (R, S), the amount of calculation can be reduced as compared with the ninth embodiment in which processing is performed on geometrically transformed edge data D ′ (R, S, Q). Can be.

(L) Twelfth Embodiment Next, a twelfth embodiment of the moving picture processing apparatus according to the present invention will be described in detail with reference to the drawings.

(L-1) Configuration of Twelfth Embodiment FIG. 33 is a block diagram showing the configuration of a moving image processing apparatus according to the twelfth embodiment. In FIG. 33, the first
The moving image processing apparatus according to the second embodiment includes a geometric conversion unit 330
1, a Hough transform unit 3302, a motion detecting unit 3303, a filter unit 3310, and a pattern superimposing unit 3311.

The geometric transformation unit 3301 receives moving image data S3301 (A (X, Y, T)) from the outside of the apparatus, and receives a control parameter S3304 (θ) from the motion detection unit 3303.
Is input and the geometric transformation data S3302 (C (P, T,
Q)) and outputs it to the filter unit 3310. This geometric transformation unit 3301 is the same as the geometric transformation unit in the eighth embodiment.

The filter unit 3310 includes a geometric conversion unit 330
1, the geometric transformation data S3302 is input, and the geometric transformation edge data S3310 (C ′ (P, T, Q)) is output to the pattern superimposing unit 3311. The filter unit 3310 according to the twelfth embodiment includes a motion detection unit 330
3, the control parameter φ is not given, and the filter unit 3310 forms the geometric transformation edge data S3310 by two-dimensional filtering without considering the filtering direction.

[0295] The pattern superimposing section 3311 includes the filter section 3
The superimposition data S3311 (F (P, T)) is obtained by inputting the geometric transformation edge data S3310 from 310, and is output to the Hough transform unit 3302.

The Hough transform section 3302 calculates the pattern superposition 33
11 to obtain the stripe characteristic data S3303 ((φ, HV)).
3 is output. The function of the Hough transform unit 3302 will be clarified in the operation description.

[0297] The motion detecting section 3303 includes the geometric transforming section 330.
1 and outputs the control parameter S3304 to the Hough transform unit 3
A stripe characteristic data S3303 is input from 302, a motion amount S3306 ((V, α)) is obtained based on the control parameter S3304 and the stripe characteristic data S3303, and output to the outside of the apparatus.

Here, the superimposition data F (P, T) is defined on the plane formed by the axes P and T among the three axes defining the geometrically transformed edge data C ′ (P, T, Q). Data. Further, the fringe characteristics (φ, HV) are determined by the parameters φ
Represents an amount corresponding to the control parameter φ in the eighth embodiment, and the parameter HV represents a Hough transform evaluation value.

(L-2) Operation of the Twelfth Embodiment Next, each of the parts 3301 to 30330 and 331 as described above
The operation of the moving image processing apparatus according to the eleventh embodiment, which is composed of 0 and 3311, will be described. Note that only the geometric transformation unit 3301 has the same function as the geometric transformation unit in the eighth embodiment.

The moving image processing apparatus starts operation in response to an operation start command from outside. At this time, first, the motion detecting unit 3303 determines the control parameter θ (where | θ | = π /
The initial value is set in 2), and this is output to the geometric transformation unit 101.

[0301] The geometric conversion unit 3301 is
3 operates in the same manner as the geometric conversion unit in the eighth embodiment, and converts the moving image data A (X, Y, T) input from the outside into the geometric conversion data C.
It is converted to (P, T, Q) and output to the filter unit 3310.

Next, when the geometric transformation data C (P, T, Q) is input to the filter unit 3310, the filter unit 331
0 detects an edge pattern in the two-dimensional pattern C (P, T, Q) for each Q, and outputs the detected geometric transformation edge data C ′ (P, T, Q) to the pattern superimposing unit 3311. .

The geometric transformation edge data C ′ (P, T,
As a calculation method of Q), a conventionally known edge detection method can be applied. Here, it is assumed that the geometric transformation edge data C ′ (P, T, Q) is calculated using a two-dimensional convolution filter having a size of 3 × 3 pixels. That is,
According to equation (21), the geometrically transformed edge data C ′ (P, T,
Q) was calculated. The sum 総 in the equation (21) is for i and j. W (i, j) is a filter coefficient, for example, w (0,0) = 4, w
(-1,0) = w (1,0) = w (0, -1) = w
(0, 1) = − 1, w (else) = 0. By the processing of the equation (21), a pattern obtained by extracting an edge portion from the geometrically transformed data C (P, T, Q) is obtained as geometrically transformed edge data C ′ (P, T, Q).

C ′ (P, T, Q) = ΣΣ w (i, j) C (P + i, T + j, Q) (21) Next, the pattern superposition unit 3311 outputs the geometrically converted edge data C ′ (P, T, Q), the two-dimensional pattern C ′ (P, T, Q) for each Q is superimposed and superimposed data F
Calculate (P, T). The operation of the pattern superposition unit 3311 in the twelfth embodiment is the same as the operation of the pattern superposition unit in the eleventh embodiment. That is,
As a method of superposition, F (R, S) = {C ′ (P, T,
Q) is calculated.

At this point, the moving image A (X, Y,
When the geometric transformation unit 3301 is operated using the control parameter θ corresponding to the movement of T), the superimposition data F (P, T)
Is formed with a striped pattern. On the other hand, the moving image A (X,
When the geometric transformation unit 3301 is operated using the control parameter θ that does not correspond to the movement of (Y, T), no stripe pattern is formed in the superimposition data F (P, T).

Next, the superimposition data F is
When (P, T) is input, the superimposed pattern F (P, T)
Calculate fringe characteristic data (φ, HV) including the direction φ of the middle fringe and the Hough transform evaluation value HV indicating the likelihood,
Output to the motion detection unit 3303.

FIG. 34 is a view for explaining the operation of Hough transform section 3302 for performing such processing.

The Hough transform unit 330 of the twelfth embodiment
2 first converts the superimposition data F (P, T) into Hough transform data H (a, b). That is, the superimposition data F
In (P, T), the value exceeds a predetermined threshold, in other words, for (P, T) constituting an edge,
Voting is performed on the data H (a, b) for the point group (a, b) satisfying the relational expression of T = Pa + b (ie, H
(The value of (a, b) is incremented.) Here, when the superimposition data F (P, T) forms a striped pattern, the relationship of equation (22) holds between the direction φ of the striped pattern and the parameter a.

A = tan φ (22) Accordingly, in the Hough transform data H (a, b) for such superimposed data F (P, T), the value concentrates on the parameter a satisfying the expression (22). It will be easier.

Therefore, Hough transform section 3302 outputs data H
(A, b) is projected in a direction parallel to the axis b, and a parameter a that gives a maximum value HV = maxΣH (a, b) in the projection data is detected (a search for the maximum value max uses a as a parameter, The sum Σ is for b). Further, the Hough transform unit 3302 obtains a parameter φ from the parameter a at this time based on the equation (22), and obtains the stripe characteristic data (φ, H
V).

Therefore, when the superimposition data F (P, T) forms a stripe pattern, the parameter φ indicates the direction of the stripe, and the Hough transform evaluation value HV has a large value. On the other hand, when the superimposition data F (P, T) does not form a striped pattern, the value does not concentrate on the specific parameter a on the Hough transform data H (a, b), and as a result, the Hough transform evaluation value HV Is a small value.

In the Hough transform, even when a straight line in an image is distorted or includes noise, the inclination of the straight line can be detected with high accuracy by a statistical process of “voting”.
Therefore, according to Hough transform section 3302, superimposed data F
Even when (P, T) is not a strict striped pattern (such as when the motion of the moving image is slightly fluctuating or when noise is included in the moving image), the direction of the stripe is accurately detected. be able to.

In the eighth embodiment, the fringe direction is detected based on the projection data for the geometrically transformed edge data C ′ (P, T, Q).
When (P, T, Q) deviates from the strict striped pattern to some extent, a clear change may not appear in the projection data. The configuration of the Hough transform unit 3302 in the twelfth embodiment is expected to be able to detect the direction of the stripe more accurately than in the eighth embodiment.

Next, the motion detecting section 3303 receives the stripe characteristic data (φ, HV) from the Hough transform section 3302,
The maximum Hough transform evaluation value HVm in which this data is stored internally.
(In the initial state, the value 0 is stored.) If the input Hough transform evaluation value HV is larger, this value is stored again as the maximum projection variance data HVm.
Control parameter θ and parameter φ
And φm.

Next, the motion detecting section 3303 changes and resets the control parameter θ within the range of -π / 2 to π / 2, and outputs the result to the geometric transformation section 3301. In this embodiment, the change is made in units of π / 100. Every time the control parameters are reset, the operations of the geometric conversion unit 3301, the filter unit 3310, the pattern superimposition unit 3311, the Hough conversion unit 3302, and the motion detection unit 3303 are performed in the same manner as described above.

As a result, in the motion detection section 3303, the superimposition data F
The Hough transform evaluation value HV becomes the maximum value HVm with respect to the parameter θ when the stripe pattern on (P, T) becomes clearest and the stripe direction φ at that time, and θ and φ at that time are θm and φm. Will be stored as

As described above, the moving image data A (X,
Y, T) moves in the direction of θ or θ + π, the geometric transformation edge data C ′ (P, P) obtained from the geometric transformation unit 3301 and the filter unit 3310 using the angle θ as a control parameter.
T, Q) has a clear striped pattern,
Superimposition data F obtained from pattern superimposition section 3311
Since a clear stripe pattern is also formed on (P, T), and the stripe direction is the parameter φ when the Hough transform evaluation value HV obtained by the Hough transform unit 3302 is the maximum value, motion detection is performed. The parameters θm and φm stored by the unit 3303 are values corresponding to the motion of the moving image data A (X, Y, T).

Therefore, in the same manner as in the above-described embodiment, the motion detecting section 3303 calculates the motion amount S330 of the moving image data A (X, Y, T) based on the parameters θm and φm.
6 ((V, α)) and outputs it to the outside.

As described above, in the twelfth embodiment, the geometric conversion unit 3301 converts the moving image data A (X, Y, T) into the geometric conversion data C based on the control parameter θ.
(P, T, Q), and the filter unit 3310 converts the geometric transformation data C (P, T, Q) based on the control parameter φ.
Geometric edge data C ′ (P, T,
Q), and the pattern superimposing unit 3311 superimposes a plane pattern for each Q of the geometric transformation edge data C ′ (P, T, Q) to create superimposed data F (P, T), and the Hough transform unit 3302 calculates the Hough transform evaluation value HV indicating the fringe direction φ and its certainty based on the superimposition data F (P, T), and the motion detection unit 3303 calculates the control parameter when the Hough transform evaluation value HV becomes the maximum. Determine θ and φ.

In the processing performed by the geometric transformation unit 3301, the moving image data A (X, Y, T) is converted to an angle −θ on the (X, Y) plane.
To create a three-dimensional pattern B (P, Q, T), and then exchange the coordinate axes to convert the geometric transformation data C (P,
T, Q), and then the filter unit 3310
Calculates geometric transformation edge data C ′ (P, T, Q) representing an edge of the geometric transformation data C (P, T, Q).
When the moving image data A (X, Y, T) moves in the angle θ or θ + π direction on the (X, Y) plane, (P,
T) Geometric transformation edge data C ′ (P, T, Q) on a plane
Has a clear stripe pattern, and the direction φ of the stripe corresponds to the magnitude of the movement.

Therefore, the Hough transform section 3302 performs a Hough transform on the superimposed data F (P, T) to calculate the direction φ of the stripe and the Hough transform evaluation value HV, and the motion detecting section 3303 determines that the Hough transform evaluation value HV is maximum. By detecting the control parameters θ and φ in the case of, the moving image A (X,
Y, T).

(L-3) Effects of the Twelfth Embodiment The twelfth embodiment can also provide substantially the same effects as the twelfth embodiment.

In addition, according to the twelfth embodiment, the pattern superimposing unit 3311 superimposes the plane data for each Q of the geometric transformation edge data C ′ (P, T, Q) to superimpose data F ( P, T), and the Hough transform unit 3302 detects the stripe direction on the superimposed data F (P, T) by the Hough transform. Therefore, the geometrically transformed edge data C ′ (P, T) as in the eighth embodiment is obtained. The motion can be detected with higher accuracy than when the fringe direction is detected based on the projection data on (T, Q).

(M) Other Embodiments The present invention is not limited to the above embodiments, but allows various modified embodiments under the same technical concept. Some examples are as follows.

(M-1) In the first to eleventh embodiments, the case has been described where the moving image processing apparatus is configured to detect the fringe direction based on the variance of the projection data. Alternatively, an evaluation value indicating the amount of change in the projection data may be set in advance, and the stripe direction may be detected based on the evaluation value. For example, a differential (or difference) pattern when the projection data is regarded as a one-dimensional signal sequence may be obtained, and the sum of the absolute values thereof may be used as the evaluation value. In this case, the larger the fluctuation of the projection data, the larger the evaluation value becomes, so that the same effect as in the first to eleventh embodiments can be obtained.

(M-2) In the fifth embodiment, the geometric conversion unit 1301 converts moving image data A (X, Y, T) into a three-dimensional pattern B (P, Q, T), and further performs geometric conversion. The data is converted into data C (P, T, Q), and this is converted into a projection dispersion
Although the case of outputting to the 302 has been described, as another configuration, the geometric transformation unit 1301 is configured to output the parameters (X, Y, T) corresponding to the geometric transformation data C (P, T, Q) required by the projection variance calculation unit 1302. ) Is calculated and output, and the projection variance calculator 1302 calculates the parameters (X, Y,
T) and the corresponding moving image data A (X, Y, T)
May be directly input from the outside to calculate the projection pattern. The expression of claim 7 includes this case. Further, similar modifications can be made to the embodiment based on the fifth embodiment.

(M-3) Similarly, in the sixth embodiment, the geometric transformation unit 1701 converts the moving image data A (X, Y, T) into 3
It is converted to a three-dimensional pattern B (P, Q, T), further converted to a three-dimensional pattern C (P, T, Q), further converted to a three-dimensional pattern D (R, S, Q), and Although the case of outputting to the projection variance calculation unit 1702 has been described, as another configuration, the geometric transformation unit 1701 performs the geometric transformation data D (R, S,
The parameter (X, Y, T) corresponding to Q) is calculated and output, and the horizontal projection variance calculation unit 1702 receives the parameter (X, Y, T) and receives the corresponding moving image data A (X , Y, T) may be directly input from the outside to calculate a projection pattern or the like. The expression of claim 16 is:
This case is included. Further, an embodiment based on the sixth embodiment may be similarly modified.

(M-4) In the seventh embodiment, the search range setting unit 2104 obtains a motion vector for a representative region in the moving image data A (X, Y, T), and based on the motion vector, Has been described. However, as another configuration, the motion vector data A (X, Y, T) and the motion vector data are also input, and the search range setting unit 2104 searches the search range data based on the motion vector. May be determined. In this case, it is not necessary to perform the motion detection processing in the moving image processing apparatus. As an example of such data input, encoded image data (for example, MPEG data) can be used.

(M-5) The configuration of the sixth, eighth to twelfth embodiments may be provided with a search range setting unit similar to the search range setting unit 2104 of the seventh embodiment.

(M-6) The filtering method (edge detection method) in the filter unit in each embodiment is not limited to the above-described method, and a conventionally known edge detection method may be applied.

For example, in the eighth embodiment, a pixel having a large amount of change in the pixel value in a direction orthogonal to the angle φ direction in the plane pattern is regarded as an edge, and the other pixels are determined as non-edges. The converted edge data C ′ (P, T, Q) may be created. At this time, the values of the geometric transformation edge data C ′ (P, T, Q) are 0 and 1
, Or multi-valued data according to the amount of change in the pixel value. further,
As another simpler configuration of the filter unit 2610, a processing configuration that does not depend on the angle φ, for example, by using a two-dimensional convolution filter having a size of 3 × 3 pixels to convert the geometrically transformed edge data C ′ (P, T, Q). You may calculate.

In the ninth and eleventh embodiments, for example, in a D (R, S, Q) plane pattern for each Q, pixels having a large amount of change in pixel value in a direction orthogonal to the horizontal direction are defined as edges. And the other pixels are determined to be non-edges, whereby the geometrically transformed edge data D ′ (R, S,
Q) may be created. At this time, the value of the geometric transformation edge data D ′ (R, S, Q) may be binary data consisting of 0 and 1, or may be multi-valued data corresponding to the amount of change in the pixel value. Is also good. Further, as another simpler configuration of the filter unit, the geometric transformation edge data D ′ (R, S, Q) may be calculated using, for example, a two-dimensional convolution filter having a size of 3 × 3 pixels.

(M-7) In each of the fifth and subsequent embodiments, the calculation of the geometrically transformed edge data and the calculation of the projection variance data are performed for all regions in the target cube. The same effects as those of the embodiments can be obtained even when the operation is performed only in a part of the region in order to reduce the amount.

For example, in the ninth embodiment, the geometric transformation unit 2601 is configured to determine the geometric transformation data D (R, S, Q) only for three appropriate values in the range that Q can take. The filter unit 2610 and the horizontal projection variance calculation unit 2602 may also be configured to perform processing only on the above three values of Q, and as a result, the obtained motion amount is determined from the already described sixth embodiment. Almost the same value as the obtained motion amount is obtained.

(M-8) In each of the fifth and subsequent embodiments, the control parameters θ and / or φ are replaced by time division to search for values θm and φm according to the amount of motion. Alternatively, the values θm and φm according to the amount of motion may be searched for by parallel processing for different values of the control parameters θ and / or φ.

(M-9) In each of the above embodiments, the input moving image data is a time series of frames, but the time axis direction is added to blocks obtained by dividing a frame vertically and horizontally.
It may be dimensional data.

(M-10) The fifth through twelfth embodiments may employ the motion amount output method as in the first through fourth embodiments. The above-described first to fourth embodiments may employ a motion amount output method as in the embodiment.

(M-11) The finite Radon transform as described in the second embodiment may be applied to the creation of a projection pattern in another embodiment, if applicable.

[0339]

As described above, according to the present invention, moving image data represented on a three-dimensional space of a horizontal axis, a vertical axis, and a time axis is input, and the coordinate axis conversion processing is performed on the moving image data. Coordinate axis conversion means for forming coordinate axis conversion data that may include a striped pattern corresponding to movement in the input moving image data, and input moving image data based on information on the striped pattern in the formed coordinate axis conversion data. With the motion information forming means for obtaining the motion information in, the amount of calculation is substantially constant regardless of the magnitude of the motion, and a series of motions of the object in the image can be grasped globally. Furthermore, even if noise is included in the image data, it is possible to realize a moving image processing device in which a motion detection error hardly occurs.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a first embodiment.

FIG. 2 is an explanatory diagram of a conventional motion detection method.

FIG. 3 is an explanatory diagram of an operation of a spatio-temporal axis conversion unit according to the first embodiment.

FIG. 4 is a diagram illustrating the operation of a projection pattern creation unit according to the first embodiment.

FIG. 5 is a diagram illustrating the operation of the fringe direction detection unit according to the first embodiment.

FIG. 6 is an explanatory diagram of an output method of a motion amount output unit according to the first embodiment.

FIG. 7 is a block diagram illustrating a configuration of a second embodiment.

FIG. 8 is an explanatory diagram of an operation of a finite Radon conversion unit according to the second embodiment.

FIG. 9 is an explanatory diagram of an operation of a fringe direction detection unit according to the second embodiment.

FIG. 10 is a block diagram illustrating a configuration of a third embodiment.

FIG. 11 is a diagram illustrating the operation of a low-frequency cutoff filter unit according to the third embodiment.

FIG. 12 is a block diagram illustrating a configuration of a fourth embodiment.

FIG. 13 is a block diagram illustrating a configuration of a fifth embodiment.

FIG. 14 is a diagram (part 1) illustrating the operation of the geometric transformation unit according to the fifth embodiment.

FIG. 15 is a diagram (part 2) illustrating the operation of the geometric transformation unit according to the fifth embodiment.

FIG. 16 is an explanatory diagram of the operation of the projection variance calculation unit of the fifth embodiment.

FIG. 17 is a block diagram illustrating a configuration of a sixth embodiment.

FIG. 18 is an explanatory diagram of an operation of a geometric transformation unit according to the sixth embodiment.

FIG. 19 is an explanatory diagram of the operation of the horizontal projection variance calculation unit of the sixth embodiment.

FIG. 20 is an explanatory diagram of an operation of a geometric conversion unit according to a modified embodiment of the sixth embodiment.

FIG. 21 is a block diagram illustrating a configuration of a seventh embodiment.

FIG. 22 is a diagram illustrating the operation of a search range setting unit according to the seventh embodiment.

FIG. 23 is a block diagram showing the configuration of the eighth embodiment.

FIG. 24 is an explanatory diagram illustrating operations of a geometric conversion unit and a filter unit according to the eighth embodiment.

FIG. 25 is an explanatory diagram of an effect obtained by providing a filter unit according to the eighth embodiment.

FIG. 26 is a block diagram showing a configuration of a ninth embodiment.

FIG. 27 is an explanatory diagram of operations of a geometric conversion unit and a filter unit according to the ninth embodiment.

FIG. 28 is an explanatory diagram of an effect obtained by providing a filter unit according to the ninth embodiment.

FIG. 29 is a block diagram showing the configuration of the tenth embodiment.

FIG. 30 is an explanatory diagram of the operation of the horizontal projection variance calculation unit with filter according to the tenth embodiment.

FIG. 31 is a block diagram illustrating a configuration of an eleventh embodiment.

FIG. 32 is a diagram illustrating the operation of the pattern superimposing unit according to the eleventh embodiment.

FIG. 33 is a block diagram showing a configuration of a twelfth embodiment.

FIG. 34 is a diagram illustrating the operation of the Hough transform unit according to the twelfth embodiment.

[Explanation of symbols]

101, 701, 1001, 1201 ... space-time axis conversion unit, 102, 1202 ... projection pattern creation unit, 103, 703, 1003, 1203 ... stripe direction detection unit, 104, 704, 1004, 1204 ... motion amount output unit, 702 , 1002: finite Radon transform unit, 705, 1005: candidate angle storage unit, 1006, 1205: low-pass cutoff filter unit, 1301, 1701, 2101, 2301, 2601,
2901, 3101, 3301 ... geometric conversion unit, 1302, 2302 ... projection dispersion calculation unit, 1303, 1703, 2103, 2303, 2603,
2901, 3103, 3303: motion detectors, 1702, 2102, 2602, 3102: horizontal projection variance calculator, 2104: search range setting unit, 2310, 2610, 3110, 3310: filter unit, 2902: horizontal projection variance calculation with filter Section, 3111, 3311 ... pattern superimposition section.

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 18, 1996

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0212

[Correction method] Change

[Correction contents]

Procedure (1) Slope (gp, gt) of pixel value for each pixel of pattern C (P, T, Q) = (∂C / ∂P,
∂T / ∂P) is calculated.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0237

[Correction method] Change

[Correction contents]

Procedure (1) Slope (gr, gs) of pixel value for each pixel of pattern D (R, S, Q) = (∂D / ∂R,
∂D / ∂S) is calculated.

Claims (22)

[Claims] 1. Moving image data represented in a three-dimensional space of a horizontal axis, a vertical axis, and a time axis is input, and a coordinate axis conversion process is performed on the moving image data to correspond to the motion in the input moving image data. Coordinate axis conversion means for forming coordinate axis conversion data that may include a striped pattern; and motion information forming means for obtaining information on movement in input moving image data based on information on the striped pattern in the formed coordinate axis conversion data. A moving image processing apparatus comprising: 2. The coordinate axis converting means according to claim 1, wherein said moving image data includes a time axis and a horizontal axis for each value in the vertical axis direction.
First coordinate axis conversion data expressed on a three-dimensional space, and second coordinate axis conversion data expressed on a two-dimensional space of a time axis and a vertical axis for each value in the horizontal axis direction of the input moving image data The motion information forming means is configured to input the moving image data based on the information of the striped pattern in the first coordinate axis conversion data and the information of the striped pattern in the second coordinate axis conversion data. The moving image processing apparatus according to claim 1, wherein information on a motion in the motion image is obtained. 3. The motion information forming means, for each of the first and second coordinate axis conversion data,
A projection pattern creation unit that creates projection patterns for various projection directions; and detects a stripe direction that is a projection direction of a projection pattern having the largest variation among a plurality of projection patterns obtained for the first coordinate axis conversion data. And a fringe direction detecting unit for detecting a fringe direction which is a projection direction of a projection pattern having the largest variation among a plurality of projection patterns obtained with respect to the second coordinate axis conversion data; 3. The image processing apparatus according to claim 2, further comprising: a motion amount output unit that obtains information on a motion in the input moving image data based on information on the stripe direction and the stripe direction according to the second coordinate axis conversion data. The moving image processing apparatus according to the above. 4. A low-frequency cutoff process for input data between the coordinate axis conversion means and the projection pattern creation unit or between the projection pattern creation unit and the stripe direction detection unit. The moving image processing apparatus according to claim 3, further comprising an area cutoff filter unit. 5. The moving image processing apparatus according to claim 3, wherein a finite Radon transform unit that performs a finite Radon transform on the input data is applied as the projection pattern creating unit. 6. The fringe direction detecting unit is configured to perform a projection pattern based on a finite Radon transform that can be directly given from the finite Radon transform unit, or the low frequency cut-off filter unit output from the finite Radon transform unit. For each projection pattern by the finite Radon transform that can be given via the storage unit that stores the stripe direction angle candidate and the reliability corresponding thereto, wherein the stripe direction detection unit is configured to input, 6. The moving image according to claim 5, wherein, among the plurality of projection patterns obtained by the finite Radon transform, the stripe direction is detected based on the stripe direction angle candidate and the reliability of the projection pattern whose fluctuation is large. Processing equipment. 7. The coordinate axis conversion means according to claim 2, wherein said moving image data includes a horizontal axis and a vertical axis for each value in a time axis direction.
After rotating the data in the three-dimensional space by the angle variable Θ, the moving image data represented in the three-dimensional space of the horizontal axis, the vertical axis, and the time axis after the rotation is converted into the vertical axis direction (or the horizontal axis direction). Is converted into data expressed on a two-dimensional space of a time axis and a horizontal axis (or a vertical axis) for each value of the above. 2. The motion information in input moving image data is obtained based on an angle variable が in which a striped pattern appears and an angle Φ of the striped pattern in the coordinate axis conversion data.
3. The moving image processing device according to claim 1. 8. The motion information forming means obtains a projection pattern of an angle variable Φ for each coordinate axis conversion data for each angle variable Θ from the coordinate axis conversion means, and obtains information of a projection pattern having the largest variation. And a motion detector that obtains information on the motion in the input moving image data based on the angle variables Θ and Φ in the projection pattern having the largest fluctuation. Item 8. The moving image processing device according to Item 7. 9. A vertical axis direction in the moving image data expressed in a three-dimensional space of the input horizontal axis, vertical axis, and time axis after rotation between the coordinate axis conversion unit and the motion information forming unit. Or a low-frequency cut-off filter unit for performing low-frequency cut-off processing on data in a two-dimensional space of a time axis and a horizontal axis (or a vertical axis) for each value in the horizontal axis direction. Item 9. The moving image processing device according to item 7 or 8. 10. Two-dimensional spatial data of a time axis and a horizontal axis (or a vertical axis) for each value in a vertical axis direction (or a horizontal axis direction) after filtering output from the low-frequency cutoff filter unit, The moving image processing apparatus according to claim 9, further comprising a pattern superimposing unit that superimposes and synthesizes in a vertical axis direction (or a horizontal axis direction) and outputs the resultant to the motion information forming unit. 11. The projection pattern variation calculation section performs a finite Radon transform on input data, and obtains information on a projection pattern having the largest variation among projection patterns by the finite Radon transform. The moving image processing device according to any one of claims 8 to 10. 12. The projection pattern variation calculation unit includes a storage unit for storing, for each projection pattern that may be generated by the finite Radon transform, a stripe direction angle candidate and a reliability corresponding thereto, The information of the projection pattern having the largest variation is obtained based on the stripe direction angle candidate and the reliability of the projection pattern having a large variation among the plurality of projection patterns by the finite Radon transform. 12. The moving image processing device according to item 11. 13. The projection pattern fluctuation calculation unit obtains a projection pattern of an angle variable Φ with respect to coordinate axis conversion data from the coordinate axis conversion means, performs low-frequency cutoff processing on each projection pattern, and performs low-frequency cutoff processing. 9. The moving image processing apparatus according to claim 8, wherein information of a projection pattern having the largest variation among subsequent projection patterns is obtained. 14. A vertical axis direction (moving image data) expressed in a three-dimensional space of a horizontal axis, a vertical axis, and a time axis after rotation between the coordinate axis converting means and the motion information forming means. Or horizontal axis direction) for data in a two-dimensional space of a time axis and a horizontal axis (or a vertical axis) for each value,
A low-frequency cutoff filter for performing low-frequency cutoff processing, and the motion information forming means performs Hough transform on each coordinate axis conversion data for each angle variable の from the low-frequency cutoff filter, A Hough transform unit that obtains the angle Φ and the reliability of the striped pattern in the coordinate axis conversion data, and the angle Φ and the angle variable Θ with the highest reliability
The moving image processing apparatus according to claim 7, comprising a motion detecting unit that obtains information on a motion in the input moving image data. 15. Two-dimensional spatial data of a time axis and a horizontal axis (or a vertical axis) for each value in a vertical axis direction (or a horizontal axis direction) after filtering output from the low-frequency cutoff filter unit, 15. The moving image processing apparatus according to claim 14, further comprising a pattern superimposing unit that superimposes and synthesizes in a vertical axis direction (or a horizontal axis direction) and outputs the resultant to the motion information forming unit. 16. The coordinate axis conversion means rotates data in a two-dimensional space of a horizontal axis and a vertical axis for each value in the time axis direction of the input moving image data by an angle variable Θ,
The moving image data expressed in the three-dimensional space of the horizontal axis, the vertical axis, and the time axis after the rotation is converted into the time axis and the horizontal axis (or the vertical axis) for each value in the vertical axis direction (or the horizontal axis direction). Is converted into data represented on a two-dimensional space of the following two-dimensional space, and further, the two-dimensional space of the time axis and the horizontal axis (or the vertical axis) for each value in the vertical axis direction (or the horizontal axis direction) after the conversion The above data is rotated by an angle variable Φ. 2. The moving image processing apparatus according to claim 1, wherein information on movement in the moving image data is obtained. 17. The motion information forming means obtains a horizontal (or vertical) projection pattern with respect to each coordinate axis conversion data of each combination of the angle variables Θ and Φ from the coordinate axis conversion means, A one-way projection pattern variation calculation unit that obtains information on a projection pattern having a large variation, and a motion detection unit that obtains information on the motion in input moving image data based on the angle variables Θ and Φ in the projection pattern with the largest variation 17. The moving image processing device according to claim 16, wherein: 18. A time axis and a horizontal axis (or a vertical axis) for each value of a vertical axis direction (or a horizontal axis direction) after rotation inputted between the coordinate axis converting means and the motion information forming means.
18. The moving image processing apparatus according to claim 16, further comprising a low-frequency cutoff filter unit that performs low-frequency cutoff processing on the data in the two-dimensional space. 19. Two-dimensional spatial data of a time axis and a horizontal axis (or a vertical axis) for each value in a vertical axis direction (or a horizontal axis direction) after filtering output from the low-frequency cutoff filter unit, 19. The moving image processing apparatus according to claim 18, further comprising a pattern superimposing unit that superimposes and synthesizes in a vertical axis direction (or a horizontal axis direction) and outputs the resultant to the motion information forming unit. 20. The one-directional projection pattern variation calculation unit obtains a horizontal (or vertical) projection pattern for each coordinate axis conversion data of a combination of each angle variable Θ and Φ from the coordinate axis conversion means. 18. The method according to claim 17, wherein a low-frequency cutoff process is performed on each projection pattern to obtain information on a projection pattern having the largest variation among the projection patterns after the low-frequency cutoff process. Moving image processing device. 21. Estimating a motion vector between frames for a representative region in a moving image in the input moving image data, or inputting the motion vector from the outside, and searching for a motion amount based on the motion vector 21. The moving image processing apparatus according to claim 7, further comprising a search range setting unit that sets a range in which the angles Θ and Φ can be taken. 22. In the rotation processing of the angle Θ or Φ by the coordinate axis converting means, all the pixels of the two-dimensional data before rotation correspond to all the pixels of the two-dimensional data after rotation on a one-to-one basis. The rotation processing is performed.
A moving image processing apparatus according to any one of 5, 17 to 21.