JPH07123409A - Moving image analysing and synthesizing device - Google Patents

Abstract

PURPOSE:To provide the moving image analysing and synthesizing device for efficiently encoding moving image data with picture quality not visually inferior. CONSTITUTION:The moving image data changing with time are handled as the three-dimensional spatial image data of a two-dimensional plane x-y and a time base direction. An information change analysis part 12 analyzes the three-dimensional change of the original moving image data. A feature point detection part 13 extracts a feature point by calculating the maximal and minimal values of the analyzed result. A curved surface is provided by linking the feature points. A feature point encoding part 14 encodes the feature points. An information encoding part 15 analyzes and compresses the information on the curved surface composed of the feature points. A total encoding part 16 encodes the data of the feature point encoding part 14 and the information encoding part 15. A synthesizing part 20 restores the original moving images by performing the opposite processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to, for example, image recording,
The present invention relates to an apparatus and method for performing image processing and encoding processing used in fields such as communication, and more particularly to a moving image analysis / synthesis apparatus capable of efficiently encoding moving image data.

[0002]

2. Description of the Related Art Although many efficient image compression methods have been considered, there is still a demand for further improvement in image quality and compression rate. Regarding a two-dimensional still image, an attempt has been made to faithfully reproduce the edge portion of the two-dimensional still image in consideration of the fact that human visual characteristics are sensitive to the edge where the brightness changes abruptly. However, it has not yet been known that such an attempt was made for moving images. Therefore, it is desired to develop an efficient image compression technique for moving images.

A moving image can be regarded as a three-dimensional space-time image by arranging spatial two-dimensional signals in the time direction. As for human visual characteristics, motion detection is also performed in the temporal direction in parallel with the normal edge detection in the spatial direction for this spatiotemporal image. Sensitive reactions, especially to moving edges, have been observed in nerve cells that control visual information processing. Therefore, it is suggested that moving edges are as important as spatial edge detection was for vision. In conventional image compression using motion detection, motion detection is performed by some method, the prediction is performed in the time direction using the method, and the error is encoded.

[0004]

However, since this method is not a technique of paying attention to a moving edge and transmitting the information there faithfully as described above, a sufficiently high efficiency encoding is achieved for a moving image. It has not been.

Therefore, it is an object of the present invention to provide an image compression technique capable of achieving a high image quality and a high compression ratio for a moving image, and an apparatus using the image compression technique. That is, it is an object of the present invention to provide a moving image analysis / synthesis apparatus capable of visually expanding and interpolating an image in a spatial direction and a temporal direction, and achieving a high compression rate with a visually comparable image quality. And

[0006]

In order to solve the above-mentioned problems and to achieve the above-mentioned object, the present invention provides a moving image analysis / synthesis apparatus which compresses and encodes a moving image for compression coding. And a moving picture synthesizing device for decoding the encoded signal. The moving image analysis apparatus analyzes original moving image data that changes with time as three-dimensional space image data distributed in a two-dimensional plane and a three-dimensional space in the time axis direction, and means for extracting feature points. A means for expressing the extracted feature points as a curved surface that spreads in three dimensions, a means for calculating an extreme value of curvature from a cutting curve of the curved surface, a curve connecting the extreme values of curvature in the time axis direction, and image data serving as a reference. And a means for compressing the data of the curved surface. The moving image synthesizing device receives the coded signal from the moving image analyzing device, performs a process which is practically the reverse of the process in the moving image analyzing device, and performs similar interpolation on the compressed data to obtain the original moving image. Restore the data.

Specifically, the means for extracting the feature points in the moving image analysis apparatus extracts the DC component by convolving the original image data with a smoothing function, and multiresolution analysis is performed on the image data excluding the DC component. And feature points are extracted. Further, specifically, the means for calculating the extreme value in the moving image analysis apparatus convolves the smoothing function with the image data excluding the DC component, and determines the portion where the value becomes 0 as the extreme value. More specifically, the means for expressing the curved surface in the moving image analysis apparatus expresses the curved surface by connecting the extreme value points that minimize the distance between the extreme values of curvature.

[0008]

In the moving image analyzer, continuous moving image data is analyzed as three-dimensional spatial image data in the time axis t direction in addition to the xy plane, and feature points are extracted. Further, in the moving image analysis apparatus, these extracted feature points are represented as a curved surface that spreads in a three-dimensional space, and the extreme values of curvature, that is, the maximum value and the minimum value, are obtained from the cutting curve of the curved surface. Furthermore, in the moving image analysis apparatus, the extreme value of the curvature is represented as a curve connected in the time axis direction, and this curve and a reference, for example,
A curved surface is expressed by the cutting curve of the first frame, and the curved surface is compressed and encoded. The moving picture synthesizing device receives the above-mentioned coded signal, reverses the above-mentioned coded processing, and restores the original moving picture data.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention regards a moving image as image information (luminance signal, color difference signal, etc.) distributed in a three-dimensional space formed by an image plane and a time axis, analyzes the change, and detects the change. The original moving image is approximated using the information at the feature points.
In this embodiment, the approximation interpolates image information of points other than the edge curved surface using the detection of the edge curved surface in the three-dimensional space and the analysis result at that point. The location of the edge point is also information, so this must also be encoded,
Here, a geometric structure called a curved surface is expressed by a parametric function expression, and the function is analyzed to compress image data.

A moving image analyzing / synthesizing apparatus according to an embodiment of the present invention will be described below in detail with reference to the drawings. For ease of explanation, consider that only black and white moving images are sent to the frame memory and the moving images for the F frames are collectively encoded. FIG. 1 is a block diagram of a moving image analysis / synthesis apparatus 1 of this embodiment. The moving image analysis / synthesis device (system) 1 includes an analysis unit (moving image analysis device) 10 and a synthesis unit (moving image synthesis device) 20. Analysis unit 10 and synthesis unit 2
A signal transmission system, for example, a magnetic recording medium such as a magnetic tape or a magnetic disk, or a signal transmission system in a communication system exists between 0 and 0, but is omitted in the drawing. The analysis unit 10 includes an image memory 11, an information change analysis unit 12, a feature point detection unit 13, a feature point coding unit 14, an information coding unit 15, and a comprehensive coding unit 16. The synthesizing unit 20 has an overall reproducing unit 21, an information reproducing unit 22, and an original image reproducing unit 23.

First, the outline of the processing contents of the above-mentioned components will be described. The image memory 11 stores the input monochrome moving image in F
The frames are recorded, and the information change analysis unit 12 performs a three-dimensional analysis of the brightness change of the moving image. The feature point detection unit 13
From the analysis result of the information change analysis unit 12, the place where the characteristic point exists is expressed as a point in the three-dimensional space. The feature point encoding unit 14 expresses the place where the feature point, which is the analysis result of the feature point detection unit 13, as a curved surface. The information encoding unit 15 executes the analysis for reproducing the original moving image at the location of the above-mentioned feature point, and the analysis result is obtained by the overall encoding unit 1.
Send to 6. The comprehensive encoding unit 16 includes the feature point encoding unit 1
4 and the information sent from the information coding unit 15 are integrated and appropriately coded.

The general reproducing section 21 receives the information encoded by the general encoding section 16 via a signal transmission system (not shown), and from the information, the feature point encoding section 14 and the information encoding. The output information of the unit 15 is reproduced. Information reproducing unit 22
Selects a feature point in the three-dimensional space based on the reproduction result regarding the position of the feature point, and arranges the reproduction result regarding the analysis result of the information encoding unit 15 at that position. The information reproducing unit 22 also interpolates image components at locations outside the feature points,
In the visually important three-dimensional space of the original moving image, the portion where the brightness changes drastically is faithfully approximated, and the other portion is approximated gently.

Further, details of the processing contents of the above-mentioned respective parts will be described. Image memory 11 The image memory 11 sequentially and continuously inputs moving images of F frames from an image signal output device (not shown) such as a camera or a video and stores them. This input image data is represented as original image data (or original image data or initial image data) I ₀ (x, y, t). The parameter x is (x-
y) The position in the horizontal (x-axis) direction on the two-dimensional plane, the parameter y indicates the position in the vertical (y-axis) direction, and the parameter t indicates the passage of time. The symbol I indicates the brightness (intensity) of a monochrome image. The symbol I _o indicates the brightness in the initial state (or of the original image). The original image data I ₀ (x, y, t) stored in the image memory 11 has an image plane (image plane) defined as an xy plane and a time-series change in the time (t) axis. Can be handled as information distributed in the three-dimensional space replaced with. The three-dimensional original image data I ₀ (x, y, t) stored in the image memory 11 is read from the image memory 11 and analyzed for information change in order to be analyzed by the information change analysis unit 12. It is output to the unit 12.

Information Change Analysis Unit 12 The information change analysis unit 12 analyzes the three-dimensional change of the original image data I ₀ (x, y, t) read from the image memory 11. Specifically, as the analysis of the three-dimensional change, the information change analysis unit 12 performs a directional filtering process in each direction at multiple resolutions to which different analysis scales are applied. The multi-resolution, means to split the signals for each band by using a plurality of filters having different band characteristics in frequency space, the information change analysis unit 12, a plurality of scales sigma _j, Safukkusu (index) j = Perform steps j1, j2, ..., JJ. The information change analysis unit 12 first analyzes the DC component independently. If the window function or the smoothing function is G (x, y, t: σ _j ) represented as a Gaussian function, the DC component has a smoothing function G (x, y, t: which corresponds to the coarsest resolution j = jJ. corresponding to the filter output of σ _i ). This means that the original image data I ₀ is represented by the following equation 1.
This means that the DC component is extracted by convolving the smoothing function G (x, y, t: σ _i ) with (x, y, t).

[0015]

[Equation 1]

One symbol * represents one convolution operation. Since there are three * in the formula 1, the parameter x,
It is shown that three times of y, t, that is, a three-dimensional convolution operation is performed. Original image data I ₀ (x, y,
The image data I (x, y, t) that does not include the DC component obtained by subtracting the DC component represented by Expression 1 from t) is represented by the following equation.

[0017]

[Equation 2]

Hereinafter, the image data I (x, y, t) that does not include the DC component expressed by the equation 2 will be simply referred to as image data. The information change analysis unit 12 uses the image data I (x, y, t) expressed by the equation 2.
Analyze the changes in. The analysis processing of the information change analysis unit 12 will be described below. First, the information change analysis unit 12 applies an analysis filter function represented by the following equations 3 to 5 obtained by partially differentiating the smoothing function G (x, y, t: σ _i ) to the image data I (x, y, t). Fold it up.

[0019]

[Equation 3]

[0020]

[Equation 4]

[0021]

[Equation 5]

The calculation formulas are shown in Formulas 6-8.

[0023]

[Equation 6]

[0024]

[Equation 7]

[0025]

[Equation 8]

By performing the above calculation, the information change analysis unit 12 has performed the multi-resolution analysis.

Characteristic Point Detecting Section 13 The characteristic point detecting section 13 obtains the extreme values of the analysis result of the information change analyzing section 12, that is, the locations (positions or points) of the maximum and minimum values. To describe the specific processing, the feature point detection unit 13 differentiates the results of Expressions 6 to 8 again in the same direction to obtain the zero-cross point. The zero-cross point indicates an extreme value, and the feature point detection unit 13 stores the position where the extreme value occurs. The calculation in the feature point detection unit 13 is
This is almost the same as the calculation in the information change analysis unit 12, and the smoothing function G (x, y, t: σ _i ) is x, y, t.
The zero-cross point of the filter output that is second-order differentiated (partial differential) in each direction of is to be obtained. The second partial differential of the smoothing function G (x, y, t: σ _i ) is expressed by the following equation 9
~ 11.

[0028]

[Equation 9]

[0029]

[Equation 10]

[0030]

[Equation 11]

The feature point detection unit 13 applies the image data I (x, y, t) to the analysis filter function defined as the second partial differential.
Fold up. The calculation formulas are shown in 12 to 14 below.

[0032]

[Equation 12]

[0033]

[Equation 13]

[0034]

[Equation 14]

The feature point detection unit 13 calculates each of the zero-cross points by performing the calculation of Expressions 12-14. here,
In Expression 12, the convolution operation result W _xx I (x, y, t:
The point (position) where σ _j ) becomes 0 is Px _i (x _xi , y _xi , t _xi : σ
_j )). However, index i = 1, 2, ..., N
For _x ^j, it is assumed that there is a N _x ^j-number of the zero-crossing point. Similarly, in Expression 13, the convolution operation result W _yy
The point (position) at which I (x, y, t: σ _j ) becomes 0 is Py _h (x _yh , y
_yh , t _yh : σ _j ). However, index h = 1,
, ..., N _y ^j, there are N _y ^j zero-cross points. Similarly, in Expression 14, the point (position) at which the convolution operation result W _tt I (x, y, t: σ _j ) becomes 0 is Pt _k (x _tk , y _tk , t _tk : σ _j ). Represent However, it is assumed that there are N _t ^j zero-cross points for the indexes k = 1, 2, ..., N _t ^j . In order to simplify the expression, the point (position) Px _i (x _xi , y _xi , t _xi : σ _j ) is changed to the point Px _i.
(j), point Py _h (x _yh , y _yh , t _yh : σ _j ) to point Py _h (j), point Pt
Let _k (x _tk , y _tk , t _tk : σ _j ) be the point Pt _k (j).

As described above, the feature point detection unit 13 calculates the equations 12 to 14 to obtain the zero-cross point, that is, the maximum value or the minimum value and the position thereof.

Feature Point Coding Section 14 The feature point coding section 14 calculates the points calculated by the feature point detecting section 13.
Px _i (j), point Py _h (j), and point Pt _k (j) are regarded as points on the curved surface, and the points Px _i (j) and P
Encoding by representing y _h (j) and the point Pt _k (j).
FIG. 6 is a flowchart showing the processing of the feature point encoding unit 14. The details will be described later. The feature point encoding unit 14 first sets the point Px _i (j), the point Py _h (j), and the point Pt _k (j), if there are overlapping points, and sets them as one feature point. Create a set P (j). That is, the set of feature points P (j) has a convolution operation result W _xx I (x, y, t: σ _j ) of 0 or W _yy I
(x, y, t: σ _j ) is 0, or W _tt I (x, y, t: σ _j ) is 0. The element is represented as Pp (j). However, it is assumed that p = 1,2, ..., Np is composed of Np feature points in total. Here, in order to regard the feature points distributed in three dimensions as the feature points distributed in two dimensions, the feature point encoding unit 14 defines the three-dimensional space formed by x-y-t on the xy plane. Slice in parallel with and drop (project) on a two-dimensional plane. Then, the feature point coding unit 14 connects the feature points on the two-dimensional plane to extract a curve. Here, connection means focusing on a certain feature point Pp (j), and centering on that pixel (3 ×
3) A square image composed of pixels is processed, and if a feature point that is also an element of P (j) of the feature point set is included therein, that point is included in the same curve. Furthermore, the curves on the respective two-dimensional planes obtained by the above processing are connected in the time axis direction (t direction) and expressed as a curved surface. First, paying attention to a certain curve, the number of points in the vicinity of nine pixels (3 × 3) of each element point (feature point) of this curve and the curve of the next frame is calculated. Then, curved lines are formed by connecting the most matched curves.

Next, the feature point coding unit 14 expresses the grouped curved surface by a curve obtained by connecting a cutting curve and extreme values (maximum value, minimum value) of curvature. FIG. 2 shows an example of a curved surface. First, the feature point encoding unit 14 slices a curved surface in parallel with the xy plane. As a result, the curved surface becomes a cutting curve on each plane. Then, the feature point encoding unit 14 holds information on the position of the curve of the first frame, which is the reference. Next, the feature point encoding unit 14 sets the curvature κ for each cutting curve.
(s) is calculated, and the positions of the maximum and minimum values (extreme values) of curvature are calculated. However, the variable s is a parameter indicating the position from the starting point. A method for obtaining the curvature κ (s) from the curve will be described later. Then, the feature point encoding unit 14 connects the positions of the obtained extreme values of the curvature in the t (time axis) direction.
Here, the feature point encoding unit 14 is
An extreme point P _{ni of} curvature included in the curve L _i on a frame
And the extreme point of curvature included in the curve L _{i + 1} on the next frame
Calculate the distance D defined by the following formula for P _{i + 1m} .

[0039]

[Equation 15]

Then, the feature point coding unit 14 connects the extreme points having the minimum distance D to each other. As described above, the feature point encoding unit 14 expresses a curved surface by the information of the position of the first frame serving as a reference and the information in which the positions of the extreme values on each frame are connected in the t direction. Feature point coding unit 1
4 holds the newly appearing curve in the memory (not shown). In Figure 3, the first reference
The appearance of the curved surface represented by the frame and the connecting line in the t direction is shown.

Information Encoding Unit 15 The information encoding unit 15 analyzes information on the curved surface formed by the above-mentioned feature points and compresses the analysis result. The reason for performing compression on these will be described. That is, these feature points are points that give the extreme value of the information analyzed by the information change analysis unit 12, and the extreme value changes only gently on the connected feature points. More specifically, the above-mentioned point P _gj (j) is represented by a curved surface, and the feature point encoding unit 14 encodes its position (x, y, t), but the information encoding unit 15 , Analysis result (convolution operation result) W _x I (x, y, t: σ _i ), W _y I (x, y, t: σ _i ),
W _t I (x, y, t: σ _i ) is encoded by a similar method. In the encoding of the analysis result of the curved surface in the information encoding unit 15, the analysis result at the position of the feature point expressed in the feature point encoding unit 14 is used as the information of the curved surface for compression. That is, the information encoding unit 15 on the curved surface
The analysis result in (1) is represented only by the analysis result of the position of the connecting line that connects the cutting curve and the curvature of the first frame represented by the feature point encoding unit 14 in the time direction. As described above, the information encoding unit 15 compresses the analysis result on the feature points.

Total Encoding Unit 16 The overall encoding unit 16 encodes the data from the feature point encoding unit 14 and the information encoding unit 15. To briefly describe this encoding, the total encoding unit 16 further compresses the redundancy of both, or performs bit allocation for quantization. If necessary, the total coding unit 16 performs coding processing for error correction. Further, the comprehensive encoding unit 16 records the result and sends it to the synthesizing unit 20.

The processing of the synthesizing unit 20 will be described below. General reproduction section 21 The general reproduction section 21 inputs the encoded data of the general encoding section 16 and performs the reverse operation of the general encoding section 16. That is,
The general reproduction section 21 decodes the information of the feature point coding section 14 and the information coding section 15, that is, the data relating to the location of the feature point and the data necessary for reconstructing the information above it.

Information reproducing unit 22 The information reproducing unit 22 determines the position of the characteristic point from the data equivalent to the output of the characteristic point encoding unit 14 obtained from the general reproducing unit 21,
Also, the positions of all the feature points in the three-dimensional space defined by (x-y-t) and the information at the positions are restored from the information equivalent to the output of the information encoding of the information encoding unit 15. That is, the information reproducing unit 22 includes the feature point encoding unit 14
The reverse processing of the compression performed by the information encoding unit 15 is performed.
In other words, the information reproducing unit 22 restores the characteristic points grouped as a curved surface and the result of the information change analysis unit 12 above the characteristic points.

FIG. 4 is a diagram showing a curved surface restored by the information reproducing section 22. The information reproducing unit 22 first reads the reference first frame information. Next, the information reproducing unit 22 reads the information in which the extreme values of the curvature of each frame are connected in the t direction. Furthermore, the information reproducing unit 22 determines the position of the i-th extreme value P ₁ (s _i ) of the first frame and (i + 1) of the first frame.
The position of the extremum P ₁ (s _{i + 1} ) and the corresponding position of the jth extremum P ₂ (s _j ) of the next frame and the position of the (j + 1) th extremum Select P2 (s _{j + 1} ). Information reproducing unit 22
For the four points, the i-th and (i + 1) on the first frame
Using the position information of the curve surrounded by the extreme value of the th, the position of the curve surrounded by the extreme values of the i′th and (i + 1) ′ th points on the next frame is interpolated. The information reproducing unit 22 performs the above operation for all frames and restores all position information. The interpolation method will be described later. Similarly, the information reproducing unit 22 restores the result of the information change analyzing unit 12 by sequentially interpolating the analysis result on the position information of the first frame in the t direction.

Original image reproducing unit 23 The original image reproducing unit 23 performs interpolation approximation on the basis of the information which is the extreme value of the analysis result of the information change analysis unit 12 obtained by the above operation, and (x-y- The result of the information change analysis unit 12 in the entire three-dimensional space of t) is restored, and the inverse transformation of equations 6 to 8 is performed on the result, and the original image data I
Restore ₀ (x, y, t). In this example, the original image reproduction unit 23
Uses the Convex Projection method (convex projection) to repeat interpolation and inverse transformation several times to converge. The original image reproduction unit 23 first uses (x, y, t
) Independent direction. That is, the original image reproducing unit 23 regards YT one-dimensional data as a whole for all y and all time t in the x direction, and the analysis result Wx I (x, y, t: σ _i ) To use Convex Pro
By the jection method, the projection that makes that point an extremum is taken. The original image reproducing unit 23 also operates in the y direction and the t direction.
Processing is performed in the same manner as in the x direction. The original image reproducing unit 23 performs the inverse transformation on the obtained approximate data by the method defined by the equation 22, and projects it again on the original analysis space by the method defined by the equations 6 to 8. The original image reproducing unit 23 repeats this operation several times to obtain the analysis result WxI (x, y, t: σ _i ) at the extreme value,
Wy I (x, y, t: σ _i ) and Wt I (x, y, t: σ _i ) are converged. The original image reproducing unit 23 finally performs the inverse conversion and performs DC conversion.
The image data I (x, y, t) that does not include the component is restored. The inverse transformation may be performed by convolving the analysis result with the filter function represented by the following equation.

[0047]

[Equation 16]

[0048]

[Equation 17]

[0049]

[Equation 18]

The symbol C (u, v, w) in the above equation is defined by the following equation.

[0051]

[Formula 19]

The left sides of the equations 16 to 18 are smoothing functions (Gaussian functions) Gx (u, v, w: σ _i ) and Gy (u, respectively).
v, w: σ _i ), Gt (u, v, w: σ _i ), the Fourier transform of Gt (u, v, w: σ _i ).
* On the right shoulder represents a complex function, and − on the head of the right side represents a complex conjugate. The image data I (x, y, t) that does not include the DC component is obtained by the following equation using the inverse transform filter defined by the above equation.

[0053]

[Equation 20]

[0054]

[Equation 21]

[0055]

[Equation 22]

Finally, the original image reproducing section 23 adds the DC component S _iJ (x, y, t) to the image data I (x, y, t) not containing the DC component, as shown in the following equation. Original moving image data I
Restore ₀ (x, y, t).

[0057]

[Equation 23]

In this way, the synthesizing section 20 was able to reproduce the original image data I ₀ (x, y, t). The calculation algorithm (method) of the curvature from the curve used by the feature point encoding unit 14 and the interpolation algorithm (method) of the curved surface used by the information reproducing unit 22 will be described in detail below.

Curvature Calculation Method First, a curvature calculation method in the feature point encoding unit 14 will be described. The feature point encoding unit 14 calculates the curvature from the input curve and expresses the curve by the curvature. Now, the two-dimensional position of the curve data string P (s) is represented as the parameter s by the following formula.

[0060]

[Equation 24]

Function of parameter l (el) (x (l), y (l))
If there is a derivative of, the curvature κ (l) can be calculated as

[0062]

[Equation 25]

As

[0064]

[Equation 26]

It is assumed that Expression 26 is derived from the following expression.

[0066]

[Equation 27]

The following equation is obtained by subjecting the equation 27 to the first-order differentiation and the second-order differentiation with respect to the parameter x.

[0068]

[Equation 28]

By substituting the expression 28 into the expression 27, the following expression is obtained.

[0070]

[Equation 29]

The following equations are obtained from the equations 29 and 27.

[0072]

[Equation 30]

Here, the parameter s for x (s), y (s)
The first-order differential and the second-order differential due to are expressed by the following equations.

[0074]

[Equation 31]

Therefore, the following equation is obtained.

[0076]

[Equation 32]

From equations 30 and 32, the curvature κ (l) is expressed by the following equation.

[0078]

[Expression 33]

From the above, the feature point encoding unit 14 uses the equation 33
Can be used to find the curvature κ (s) of the curve data string P (s). If the curvature κ (s) is obtained, the curved surface is known. However, since the data string P (s) is quantized position data, the obtained curvature becomes oscillatory. In order to improve this, the feature point encoding unit 14 uses the data string P (s)
Fold the smoothing function into. A one-dimensional Gaussian function g (s, σ) is used as the smoothing function. Here, the parameter σ represents a scale for each resolution.
The one-dimensional Gaussian function is represented by the following formula.

[0080]

[Equation 34]

Here, the data string P (s, σ) smoothed by using the Gaussian function is represented by the following equation.

[0082]

[Equation 35]

When one convolution is represented by one symbol *, the following equation is obtained.

[0084]

[Equation 36]

In order to obtain the curvature κ (s), the respective first and second derivatives of X (s, σ) and Y (s, σ) in the equation 36 are required. Assuming that the first derivative of the one-dimensional Gaussian function g (s, σ) is g1 (s, σ) shown in the following formula 37 and the second derivative is g1 (s, σ) shown in the formula 38, X
The first and second derivatives of (s, σ) and Y (s, σ), respectively, are defined by equations 39-42.

[0086]

[Equation 37]

[0087]

[Equation 38]

[0088]

[Formula 39]

[0089]

[Formula 40]

[0090]

[Formula 41]

[0091]

[Equation 42]

By substituting the expression 33 into the expressions 39 to 42, the curvature κ (l) at each resolution can be obtained. This curvature κ (l)
The curved surface is obtained from. That is, the feature point encoding unit 14 performs the above-described calculation to obtain the curvature κ (l).

Curved Surface Interpolation Method The curved surface interpolation process performed by the information reproducing section 22 will be described below with reference to FIGS. Now, as shown in Fig. 5, on the curve L _i from the origin Feature points P _in , P _{in + 1} and curve Li
_{i + 1} characteristic point on the P _{i + 1n,} P _i; the vector to _{1n + 1,}
Let P _in , P _{in + 1} , P _{i + 1n + 1} , P _{i + 1 n + 1j} . here,
From the data between the characteristic points Pin and Pin + 1, the characteristic points Pi + 1n and P
Consider interpolating data between i + 1n + 1. When the distance between the feature points Pin and Pin + 1 is L1 and the distance between the pin Pin and Pin + 1 is L2, the following equation is obtained.

[0094]

[Equation 43]

[0095]

[Equation 44]

Here, the unit vector on the curve L1 is e
If the values are 1 and e2, the following equation is obtained.

[0097]

[Equation 45]

[0098]

[Equation 46]

Therefore, the height h1 and the length l1 at the characteristic point Pa on the curve Li are respectively expressed by the following equations.

[0100]

[Equation 47]

[0101]

[Equation 48]

Similarly, the unit vector on the curve Li + 1 is e
When 1 ′ and e2 ′ are given, they are represented by the following formula.

[0103]

[Equation 49]

[0104]

[Equation 50]

Therefore, the height h2 and the length l2 at the characteristic point Pa 'on the curve Li + 1 are respectively expressed by the following equations.

[0106]

[Equation 51]

[0107]

[Equation 52]

Here, if the height h2 and the length l2 of the point Pa 'on the curve Li + 1 are determined as follows, they are expressed by the following equations.

[0109]

[Equation 53]

[0110]

[Equation 54]

From the equations 51 to 54, the characteristic point Pa ′ on the curve Li + 1
= (Pax ', Pay') can be calculated based on the following formula.

[0112]

[Equation 55]

[0113]

[Equation 56]

As described above, in the embodiment of the present invention, a black and white image is exemplified as the original image data I ₀ (x, y, t), but in the implementation of the present invention, a black and white moving image is used. Not limited to this, color moving image data can be encoded while improving the image quality and the compression rate. As the color image data, for example, in the case of R, G, B image data, R, G
The above-described analysis processing may be performed for each of G and B.

[0115]

According to the present invention, it is possible to achieve a high compression ratio of moving image data with a visually comparable image quality. That is,
According to the present invention, it is possible to achieve a high compression ratio with a visually comparable image quality in a spatial direction and an image expansion or compensation in the time direction and a visually comparable image quality.

[Brief description of drawings]

FIG. 1 is a block diagram of a moving image analysis / synthesis apparatus according to an exemplary embodiment of the present invention.

FIG. 2 is a graph showing processing in the moving image analysis / synthesis apparatus of the present invention.

FIG. 3 is a graph showing a process in the moving image analysis / synthesis apparatus of the present invention, in which the first frame and the time axis (t).
It is a graph which shows the appearance of the curved surface represented by the direction connection line.

FIG. 4 is a graph showing processing in the moving image analysis / synthesis apparatus of the present invention, which is an information reproducing unit 22 shown in FIG.
It is a figure which shows the curved surface restored in FIG.

FIG. 5 is a graph showing processing in the moving image analysis / synthesis apparatus of the present invention, and is a diagram showing curved surface interpolation.

FIG. 6 is an encoding processing flowchart of a feature point encoding unit 14 shown in FIG.

7 is a processing flowchart of an information reproducing unit 22 showing the curved surface interpolation processing shown in FIG.

[Explanation of symbols]

 1 ... Moving image analysis / synthesis device 10 ... Analysis unit 11 ... Image memory 12 ... Information change analysis unit 13 ... Feature point detection unit 14 ... Feature point coding unit 15 ... Information coding unit 16 ... Comprehensive encoding unit 20-Combining unit 21-Comprehensive reproducing unit 22-Information reproducing unit 23-Original image reproducing unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location 9061-5L G06F 15/70 365

Claims (7)

[Claims] 1. A means for analyzing original time-varying moving image data as three-dimensional space image data distributed in a two-dimensional plane and a three-dimensional space in the time axis direction, and extracting feature points, and the extraction. Means for expressing the defined feature points as a three-dimensional curved surface, means for calculating an extreme value of curvature from a cutting curve of the curved surface, a curve connecting the extreme values of curvature in the time axis direction, and the reference image data. A moving image analysis / synthesis apparatus comprising: means for expressing a curved surface by a cutting curve; and means for compressing data on the curved surface. 2. A synthesizing means for receiving the compressed data, performing processing substantially reverse to the processing in the moving image analysis apparatus, and performing similarity interpolation to restore the original moving image data. 1. The moving image analysis / synthesis apparatus according to 1. 3. The feature point extracting means extracts a DC component by convolving a smoothing function with the original image data, and multi-resolution analysis is performed on the image data excluding the DC component to extract the feature point. The moving image analysis / synthesis apparatus according to claim 1. 4. The means for calculating the extremum convolves a smoothing function on the image data excluding the DC component,
4. The moving image analysis / synthesis apparatus according to claim 3, wherein a portion where the value becomes 0 is determined as an extreme value. 5. The moving image analyzing / synthesizing apparatus according to claim 4, wherein the means for representing the curved surface represents the curved surface by connecting extreme value points that minimize the distance between the extreme values of curvature. 6. The synthesizing means receives the compressed data, decompresses the compressed data, restores the compressed data to an extreme value of curvature, and restores the curvature from the restored data. Means for calculating the extreme value of, a means for restoring a curved surface that spreads three-dimensionally from the extreme value, a means for restoring feature points from the restored curved surface, and original moving image data from the restored restored feature points. 3. The moving image analysis / synthesis apparatus according to claim 2, further comprising means for restoring. 7. The means for extracting the feature points has an image memory and an information change analysis section, and the means for expressing as the three-dimensional curved surface has a feature point encoding section, the extreme value of the curvature. Means for calculating the information encoding unit, means for expressing a curved surface by the connected curve and the cutting curve of the reference image data has an information encoding unit, and means for compressing the curved surface data The moving image analysis / synthesis apparatus according to claim 5, further comprising: a comprehensive encoding unit.