JPH10257436A - Automatic hierarchical structuring method for moving image and browsing method using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To speedily and exactly extract a scene by performing the hybrid encoding of applied moving image, dividing this image into hierarchical structure and extracting the scene by integrating shots corresponding to the similarity of divided shots. SOLUTION: An MPEG1 moving image is not completely decoded but only irreducibly minimum information is decoded and high-speed processing is enabled. Then, processing reduction is attained by utilizing the characteristics of MPEG1 encoding algorithm such as inter-frame prediction or the possession of simplified image due to the decoding of DC component. Then, only an I picture is decoded and further, the reduced image of the source frame is provided by simplified decoding. Besides, it is necessary to decode information not to be directly used for decoding B and P pictures but since the I picture is encoded while being closed in the frame, it is not necessary to decode this picture. Besides, since the DC component of intramicroblock in the I picture is decoded without using the IDCT of a large calculation amount, a DC image can be provided at high speed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic hierarchical structure structuring method for an encoded moving image for obtaining a video browser based on the moving image and its analysis data. And a browsing method using the same.

[0002]

2. Description of the Related Art At present, moving picture information is not out of the range of simple reproduction of video. That is, a moving image is captured in frame units, and if a code is added at the time of shooting, it can be extracted by the code, and a special relationship such as detecting or reproducing a desired frame using the reproduction time as a factor. Only in the obvious case can the frame be extracted.

[0003]

However, it is extremely difficult to extract a desired frame from given moving image information without adding any code, and if there is a time constraint, it becomes impossible to extract it. For example, one frame is generally 3
Since it is 1/0 second, 1800 frames per minute, 1
108,000 frames per hour.

Therefore, there is a problem that an arbitrary frame cannot be extracted in a short time from the conventional general moving image information.

[0005]

According to the present invention, an encoded moving image is divided into shots, shots are integrated using the similarity of each shot, and scenes are extracted to form an automatic hierarchical structure. By using this data to create a moving image browsing tool, the above-mentioned conventional problems were solved.

That is, the present invention is characterized in that a coded moving image is divided into shots, and the shots are integrated to extract a scene by using the similarity of each divided shot. The automatic hierarchical structuring method described above is characterized in that the moving picture is encoded by MPEG. Also, when a shot is detected from an encoded moving image, high-speed processing is performed using the characteristics of MPEG, and a representative frame is extracted when calculating the degree of similarity between shots. It is characterized by. Next, it is characterized in that the similarity between shots is obtained by fuzzy inference, and the scene extraction processing is characterized by being obtained by the defined degree of connection between shots. Still another invention divides an encoded moving image into shots, and then uses the similarity of each of the divided shots, integrates the shots, extracts a scene, and automatically arranges the moving image into a hierarchical structure. Using this hierarchically structured data, you can grasp the contents of the entire video,
A moving image browsing method characterized by facilitating detection of a desired scene or shot.

[0007] The above encoding is based on the compression algorithm of MPEG1. Here, the official name of MPEG1 is "C
odding of moving pictures
and associated audio for
digital storage media at
up to about 1.5 Mbit / s ".

The hybrid coding is performed by DCT, quantization, motion compensation inter-frame prediction, and entropy coding. However, since the individual methods are known methods, detailed description thereof will be omitted.

Next, an MPEG1 encoding / decoding system will be described with reference to FIG. The video input enters the video encoder through pre-processing, then enters the storage medium via the system multiplexing, then demultiplexes the system, enters the video decoder, and pre-processes to the video output.

In the MPEG1 video encoder, the input image shown in FIG. 2 is processed into data. In the MPEG1 video decoder, an input buffer is processed into a display buffer as shown in FIG.

As mentioned above, MPEG1 is a CD-RO
It is intended to be used for storage media such as M. For storage media, trick modes such as fast forward, rewind, midway playback, and reverse playback are required. In order to realize such a trick mode, MPEG1 employs a group of pictures.
s, hereinafter referred to as GOP).

[0012] In MPEG1, encoded image data is created on the basis of previous and subsequent screen data, and thus does not become complete information for only one screen. For this reason, random access is possible in units of a GOP in which several pieces of screen data are grouped. In other words, at least one picture in a GOP always includes screen data (I picture) closed by only one picture without using information of the previous and next screens, and other data in the GOP is based on this data. Screen data can be reproduced. In addition, one G
OPs usually group about 15 pictures (FIG. 4).

In MPEG1, both forward prediction from a past reproduced image and backward prediction from a future reproduced image are performed. This is called bidirectional prediction.

In order to realize bidirectional prediction, MPEG1 defines three types of images: I picture, P picture, and B picture.

In addition to these, a D picture (DC coded image) is defined. This is coded only with information in a frame and is composed only of DC components of DCT coefficients, and does not coexist in the same sequence with the other three picture types.

In MPEG1, the introduction of B-pictures for bidirectional prediction greatly improves the prediction efficiency and contributes to the improvement of image quality at high compression.

As shown in FIG. 6, the image data has a hierarchical structure of a sequence, GOP, picture, slice, macroblock (MB), and block.

[0018] The sequence layer means that a bit stream representing a sequence of video starts with a sequence header, is followed by one or several GOPs, and ends with a 1
End with sequence end codes. A sequence header can be placed immediately before any GOP, but in a sequence header in a sequence of video, all data elements other than the quantization matrix need to be the same as the first sequence header.

As a result, random access in the middle of the sequence becomes possible.

The GOP layer includes one GOP.

Next, the picture layer includes one of I picture, P picture, B picture and D picture.

The slice layer is a group of a series of arbitrary macroblocks starting from the upper left of the image and continuing to the lower right in the raster scan order. Although there is no gap between slices, the position of the slice may be different for each screen. Since a synchronization signal is assigned to the head of slice data, there is an advantage that synchronization can be restored in the next slice even if there is a data reading error during decoding. In addition, since decoding of slice data can be performed independently for each slice, parallel processing can be performed in slice units to speed up decoding.

Next, the macroblock layer is such that the macroblock is composed of a luminance component of 16 pixels × 16 lines and two color difference components of 8 pixels × 8 lines corresponding to spatial positions in the image. One macro block includes four luminance blocks and two chrominance blocks. The block order and arrangement in the macro block are as shown in FIG. Motion compensation and inter-frame prediction are performed for each macro block.

Further, the block layer is a DCT processing unit composed of a luminance component or a color difference component composed of 8 pixels × 8 lines.

[0025]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention provides an automatic moving image hierarchical structure in which an encoded moving image is divided into shots, shots are integrated using similarity of each shot, and scenes are extracted. It is a method of conversion.

Further, there is provided a moving image browsing method for grasping the entire contents of a moving image by using the hierarchically structured data and facilitating retrieval of a desired scene, shot or frame.

According to the present invention, it is extremely easy to grasp the contents of a moving image and to search for a desired scene.

[0028]

An embodiment of the present invention will be described with reference to the drawings.

First, a moving image is divided into shots that can be extracted based on physical feature amounts and are relatively easy to detect. Then, a scene is extracted by integrating shots according to the similarity between the divided shots. At this time, since it is difficult to handle the moving image as a shot, some representative frames are selected from the shot (FIG. 7).

Generally, since a moving image has a large data amount, the processing amount is enormous. Furthermore, in the case of an encoded moving image, decoding is required, so that the processing amount further increases.

According to the present invention, high-speed processing is enabled by decoding only the minimum necessary information without completely decoding the MPEG1 moving image. The processing amount is reduced by utilizing the characteristics of the MPEG1 encoding algorithm such as acquisition of a simplified image. Therefore, a method for simplified decoding of an I picture in an MPEG moving image and a method for comparing frames will be described, and then details of each process will be described along the flow.

A moving image is composed of a number of still images (frames). Therefore, image information of each frame is indispensable for analysis of a moving image. But MP
The decoding of the EG1 moving image requires a relatively large amount of processing, and it is difficult to realize high-speed processing.

Therefore, only I pictures, not all frames, are decoded.
A reduced image of the original frame is obtained by simple decoding. This simple decoding utilizes the fact that decoding the DC component of the DCT coefficient yields the average color of the original block. That is, only the DC component of the DCT coefficients of each block is decoded, and an image representative of each block is created with the obtained average color (FIG. 8).

The image obtained in this manner is called a DC image. Since the size of each block is 8 × 8, the DC image is 縦 the size of the original image both vertically and horizontally.

For decoding a B picture and a P picture,
It is necessary to decode not only itself but also information that is not directly used, such as motion vector information and the picture of the reference destination,
Since the I-picture is encoded in a closed frame, there is no need to decode such information. In addition, D of an Intra macroblock in an I picture
The C component can be decoded by the following equation (1) without performing a computationally intensive IDCT. Therefore,
DC images can be obtained very quickly.

[0036]

(Equation 1)

Here, Y _k , Cb _{k ′} and Cr _{k ′} are the luminance and color difference components of the average color of each block ( _k and _{k ′} are block numbers), DY _k , DCb _{k ′} and DCr are the DC of each block. Component.

Actually, an MPEG1 moving image (G
Table 1 shows the processing time when all the frames are decoded and when only the DC picture of the I picture is decoded for the OP of the type shown in FIG. It can be seen that decoding can be performed in about 1/20 the processing time as compared with the case where all frames are decoded.

[0039]

[Table 1]

FIG. 9 shows an example of a DC image and its original image.

The similarity used for comparison between frames is roughly classified into comparison of pixel values and comparison of color histograms. Color histogram comparisons are convenient to use as similarities because they are less affected by camera and subject movement, but on the other hand, completely different images have the same color histogram because they do not contain any spatial information. The case becomes a problem. Some attempts have been made to provide spatial information for comparison by color histograms, but all require some complicated processing.

[0042] In the present invention, the D _Histarea of formula (2) as a distance based on the color histogram, using a D _Pixsum of formula (3) as the distance by the pixel value, by calculating the similarity by combining the two values A similarity that takes into account spatial information is obtained without impairing the simplicity of the processing.

[0043]

(Equation 2)

[0044]

(Equation 3)

Here, a simplified fuzzy inference is used as a technique for calculating the similarity from two values. By using fuzzy inference, it is possible to describe the relationship between the distance based on the color histogram and the distance based on the pixel value, and the similarity between the images without strictly formulating the relationship. Further, inference by simplified fuzzy inference is simple and can be executed at high speed.

The fuzzy rule used at this time is as shown in the following equation (4).

[0047]

(Equation 4)

Where i is the rule number, I is the number of rules,
c _i is a real numerical value representing the consequent part, and takes a value of [0, 1]. A _a and B _b are membership functions of the feature values, respectively, and “sm” as shown in FIG.
Three membership functions of "all", "medium", and "large" are set.

The degree of conformity to this rule is obtained by equation (5), and then the final inference result, that is, the similarity s between images, is obtained by equation (6). Note that s takes a value of [0, 1].

[0050]

(Equation 5)

[0051]

(Equation 6)

For the detection of a shot, a cut point between shots is detected.

The detection of a cut point is nothing but the operation of detecting a point having a low correlation between frames.
Instead of directly comparing the feature amounts of the frames to check the correlation, the correlation is checked based on the encoding state of MPEG1, and the cut point is detected. That is, in MPEG1, the fact that compression is performed by prediction from the correlation between frames is used, and conversely, the correlation between frames is checked by checking how prediction is performed.

By using the MPEG1 encoding information without newly examining the feature amount of the frame, the amount of calculation can be reduced, and since it is not necessary to decode all information, high-speed processing can be performed. It is possible.

As a processing procedure, first, a cut point is detected from a state of reference in a B picture, and further, a reference in a P picture and a change in an I picture are checked for confirmation. The following describes an example of a GOP where N = 15 and M = 3 as shown in FIG.

In the B picture, reference is made from both the front and rear I or P pictures. That is, in a B picture, generally, (IMB), (FMB), (BMB),
(BiMB) there are four types of macroblocks MB,
Either IMB without reference or forward FM
B, backward BMB, and bidirectional InMB. The state of the reference at this time is as shown in FIG.

In a shot, that is, when the correlation between frames is high, the number of references to the past and the future is almost the same. The bias is greatly biased, and the configuration of the macroblock is biased. The situation at this time is shown in FIG.
(B), (c) and (d). However, FIG. 12 is an extreme case, and actually, the reference beyond the cut point is not completely eliminated.

As can be seen from this, it is possible to determine the state of reference to the preceding and succeeding frames of the B picture from the configuration of the macroblock type of the B picture. This is defined as the dependency relat of the B picture as in Expression (7).

[0059]

(Equation 7)

Here, N _F , N _B and N _Bi are the numbers of FMB, BMB and BiMB included in the B picture, respectively.

[0061] relat is, the difference between the N _F and N _B is large,
Also, the smaller the N _Bi , the larger its absolute value, indicating that the bias of the reference is large.

Now, in a GOP as shown in FIG. 11, attention is focused on two P pictures (or I pictures) and two B pictures sandwiched between them (for example, f ₇ , f ₈ , f ₈₎ .
₉ , f ₁₀ ), which is expressed as P ₁ B ₂ B ₃ P ₄ , all cut points are always P ₁ | B ₂ B ₃ P ₄ , P ₁ B
₂ | B ₃ P ₄ and P ₁ B ₂ B ₃ | P ₄ (| denotes a cut point). These are shown in FIG.
(B), (c) and (d). In this case, in any case, as can be seen from FIG. 12, a reference bias occurs in both B ₁ and B ₂ , and the absolute value of the dependency increases.

Therefore, if the following equation (8) is satisfied, P ₁ | B
_{2 B 3 P 4, P 1} B 2 | B 3 P 4, P 1 B 2 B 3 | P 4
It is determined that a cut point exists in one of the following forms.

[0064]

(Equation 8)

Next, P ₁ | B ₂ B ₃ P ₄ , P ₁ B ₂ | B
₃ P ₄ , P ₁ B ₂ B ₃ │P ₄ is determined to determine an accurate cut point. As can be seen from Expression 7, relat takes a positive value when there are many references from the past, and a negative value when there are many references from the future. By utilizing this, the cut point can be determined as in equation (9).

[0066]

(Equation 9)

As described above, the cut point can be detected from the reference information of the B picture.

In the case of the detection only by referring to the B picture, erroneous detection may occur when there is a noise or an instantaneous screen change such as an object crossing in front of the camera. This is considered because the distance between the B picture and the picture of the reference destination is short. Therefore, the result is confirmed using not only the B picture but also the reference information of the P picture which refers to a farther picture.

If a cut point exists between a P picture and a reference I or P picture, reference is hardly made, so that the number of IMBs should increase.
Therefore, when the following equation (10) is satisfied, the cut point obtained from the B picture in between is regarded as being erroneously detected, and is removed.

[0070]

(Equation 10)

[0071] However, N _I can IM that is included in the P-picture
The number of B and N is the number of all MBs in the P picture.

Confirmation is made using a comparison between I-pictures further away from the P-picture.

For a GOP of the type shown in FIG.
_The two B pictures (f ₁ , f ₁₎ before the _three I pictures
Regarding the cut point according to ₂ ), the result using the P picture cannot be confirmed. I picture f ₃ is do no references. In order to detect erroneous detection that occurs in this portion, it is necessary to confirm the result using the I picture.

The I-picture (f ₃ ), the I-picture in the _immediately preceding GOP, and the color histogram distance D _histarea (Equation (2)) for each DC image are examined, and the following equation (1) is obtained.
If 1) is satisfied, the cut point obtained from the B picture in between is regarded as an erroneous detection and is removed.

[0075]

[Equation 11]

Next, selection of a representative frame of a shot will be described. Here, the representative frame is a short unit compared to the entire moving image, but is a set of 150 frames (in the case of 30 _fps ) of a shot of 5 seconds, for example. Processing such as detection is difficult. Therefore, in general, when handling a shot, a frame representing the shot is selected from the shots, and processing such as comparison and display is performed using the representative frame.

In the present invention, when a shot is integrated and a scene is extracted, the similarity between shots is used. It is also used to simply show the contents of a shot when presenting the analyzed moving image structure to the user. Therefore, it is necessary to select a frame that best represents the content of the shot.

In the conventional research dealing with shots, there are many cases where the leading frame or the center frame of the shot is mechanically used as the representative frame.
However, it is hard to say that the frame selected in such a manner well represents the content of the shot. Therefore, in the present invention, the frame closest to the average of the frames included in the shot is selected as the representative frame.

Further, not only when the shot has almost no motion, but also (1) the motion of the camera (pan
Zoom)). (2) A camera or an object in an image that keeps moving. (3) The movement is very intense. And so on. In such a shot, it is difficult to represent the entire shot in one frame, and there is a risk of dropping useful information. Therefore, such a shot is represented by a plurality of representative frames.

Furthermore, by selecting a plurality of representative frames, it is possible to reduce the influence of a cut point detection error. In other words, due to the detection error of the cut point,
When a plurality of shots are originally combined into one, if only one representative frame is selected, only information of one shot is originally used. On the other hand, if a plurality of shots are selected based on the content, the information of each shot can be used for scene extraction without discarding.

In order to select a minimum necessary representative frame, first, clustering of frames in a shot is performed. A frame closest to the average is selected from each cluster obtained as a result, and this is set as a representative frame of the shot.

However, if all the frames in the shot are set as the representative frame candidates, the processing amount may increase when the shot becomes long. Therefore, only I pictures are used as candidates. As a result, not only the processing amount for selection but also the processing amount for decoding from the original moving image can be reduced. In general, in MPEG1, the quantization characteristics of I, P, and B pictures are often changed in order to increase the coding efficiency. Therefore, it is also convenient that the I picture often has the best quality.

Further, instead of completely decoding the I picture, the DC image described above is used to reduce the processing amount at the time of decoding.

The specific processing procedure is as follows (FIG. 13). In the case of a shot that does not include any I picture, that is, in the case of a very short shot, the P picture that appears first in the shot is used as the representative frame, and when there is no I picture, the B picture is used as the representative frame. In the case of a very short shot, it can be said that there is almost no change during the shot, so such mechanical processing is sufficient.

(1) An I picture included in a shot is simply decoded to extract a DC image.

(2) During shots, D of a portion with little movement
Let the C image be the initial cluster. Whether the motion is small is determined by checking the number of IMBs included in the B and P pictures (Equation (12)).

[0087]

(Equation 12)

(3) Clustering is performed based on the initial cluster, and the I picture in the shot is classified into several clusters. Clustering is performed using the group average method, and the distance between elements is D _histarea (Equation (2))
Is used. Clustering is performed until the smallest one of the distances between clusters exceeds a threshold.

(4) After the clustering, one representative frame is selected from each cluster. First, an average DC image is created by averaging DC images of I pictures in a cluster (Equation (13)).

[0090]

(Equation 13)

(5) The I picture I _k having the DC image DC _k with the smallest distance D _pixsum (Equation (3)) from the average DC image is set as the representative frame.

As described above, a representative frame of a shot is selected.

In the actual processing, the selection of the representative frame is performed in parallel with the detection of the cut point in order to efficiently obtain the DC image of the I picture and the macroblock information of the P and B pictures.

For example, in a conversation scene or the like, since a speaker is often photographed alternately, similar shots are repeated. As described above, one scene often includes several similar shots. Focusing on this property, shots are integrated to extract a scene.

The similarity between shots uses the similarity s (formula (6)) between the representative frames of each shot.
However, since one shot may have a plurality of representative frames, the similarities of all the combinations of the representative frames are checked, and the maximum value is set as the similarity of the shots (FIG. 14).

The simplest method of extracting a scene from the similarity between shots is, as shown in FIG. 15, when there are similar shots (similarity is extremely high), all of the shots are regarded as the same scene. It is.

However, in this case, (1) the setting of the threshold value for similarity or dissimilarity greatly affects the result. (2) There is no set of shots having a very high similarity, Even when there are many sets of shots with similarity of, there is a problem that the same scene cannot be regarded and the flexibility is lacking.

Then, shot shot _n and shot s
A degree of connection connect _{n, n + 1} representing the degree to which hot _{n + 1} is continuous (that is, belonging to the same scene) is defined as in Expression (14), and a scene is extracted using this degree of connection.

[0099]

[Equation 14]

Here, N represents the range of shots to be compared. s _ij is the similarity between shot _i and shot _j .

As described above, the coupling degree connect
_{n and n + 1} are shots shot _n and shot shot
It is obtained not only from _{n + 1} but also from the similarity s _ij between all shots in the vicinity. For example, in FIG.
the connect _{3, 4} not only the similarity _{s 34} shots shot ₃ and shot shot _4, shot sho
similarity _{s 25} of t ₂ and shot shot ₅ is also obtained using. This is because even if shots shot ₃ and shot ₄ are not completely similar, if shots shot ₂ and shot ₅ are similar, shots shot ₃ and shot ₄ are considered to belong to the same scene. is there.

However, shots that are far apart in time are unlikely to belong to the same scene. Rather, shots that happen to have a high degree of similarity even though they belong to different scenes may exist. The range of shots to be compared is limited to N in order to prevent undetection due to various causes as much as possible.

The coupling degree con obtained by the equation (13)
The change of nect _{n, n + 1} is, for example, as shown in FIG.

From such a change in the degree of coupling, a scene change is determined and a scene is extracted. Here, when the difference between the peak and the valley of the change is larger than the threshold value threshold _SCENE , a cut point having a coupling degree corresponding to the valley is set as a scene change point.

[0105]

According to the present invention, a given moving image is hybrid-encoded, divided into a hierarchical structure, and shots are extracted by integrating shots based on the similarity between the divided shots. There is an effect that scene extraction is performed quickly and accurately. However, since it is relatively easy to extract a shot or a frame from a scene, there is an effect that a scene, a shot or a cut can be accurately extracted from a moving image in a short time.

Since the above processing can be automated by incorporating appropriate software into the hardware currently in use, a desired scene, shot or cut can be automatically provided by an appropriate input instruction.

According to the experimental results, the cut detection results using the moving images in Table 2 are shown in Table 3.

[0108]

[Table 2]

[0109]

[Table 3]

Table 4 shows the processing time of cut point detection.

[0111]

[Table 4]

Table 5 shows the processing time of cut point detection by a simpler algorithm.

[0113]

[Table 5]

Next, scene extraction was performed using shots obtained as a result of cut point detection. Figure 1 shows the change in coupling
8, 19 and 20 are shown.

Table 6 shows the result of scene extraction from the degrees of connection shown in FIGS. The threshold value for detecting a scene change point was threshold _SCENE = 0.3.

From Table 6, 75% or more of the scene change points can be detected for all the moving images, which is sufficiently practical in consideration of the high speed and the fact that this method does not use semantic analysis or knowledge. It can be said that. In addition, 1 of the number of detections
About か ら to ３ is one point from the actual scene change point.
A shift is detected before or after the shot. This is due to an algorithmic defect that, for a shot adjacent to a scene change point, if a shot having a high degree of similarity to the shot does not exist in a scene to which the shot originally belongs, the degree of connection is reduced.

[0117]

[Table 6]

The processing time of the entire structural analysis processing is as shown in Table 7, and it can be seen that a sufficient high speed is maintained.

[0119]

[Table 7]

[Brief description of the drawings]

FIG. 1 is an MPEG1 encoding / decoding system according to the present invention.

FIG. 2 is a block diagram of the MPEG1 video encoder.

FIG. 3 is a diagram of the MPEG1 video decoder.

FIG. 4 is a view showing an example of a GOP.

FIG. 5 is a diagram showing an arrangement of screens on an original image and a stream.

FIG. 6 is a diagram showing the hierarchical structure of MPEG1.

FIG. 7 is a flowchart of a structure analysis process.

FIG. 8 is a diagram showing the generation of a DC image.

FIG. 9 is a view showing an example of a DC image.

FIG. 10 is a shape diagram of a membership function used for fuzzy inference for obtaining similarity.

FIG. 11 is a view showing an example of a GOP.

FIG. 12 (a) is also a normal reference diagram. (B) The figure which shows the case where there is also a cut point between the past P picture. (C) The figure which shows the case where there is a cut point between B pictures. (D) A diagram in a case where there is a cut point between the same and a future P picture.

FIG. 13 is a flowchart showing a representative frame selection algorithm, in which (a) a DC image is extracted. (B) The figure which determines an initial cluster. (C) Diagram of clustering. (D) A diagram for creating an average image from DC images in a cluster. (E) A diagram showing the image closest to the average image as a representative frame.

FIG. 14 is a view showing the similarity between shots.

FIG. 15 is a diagram of the same simple scene extraction.

FIG. 16 is a diagram showing the coupling degree when N = 3.

FIG. 17 is a view showing an example of a change in the degree of connection connect _{n, n + 1} .

FIG. 18 is an exemplary view showing a change in the coupling degree of the moving image A;

FIG. 19 is an exemplary view showing a change in the coupling degree of the moving image B;

FIG. 20 is a diagram showing a change in the coupling degree of the moving image C;

Claims (7)

[Claims] 1. An automatic hierarchy of moving images, wherein an encoded moving image is divided into shots, and shots are integrated and shots are extracted using similarities of the divided shots. Structured method. 2. The method according to claim 1, wherein encoding of the moving image is performed by MPEG. 3. The moving image automatic hierarchical structuring method according to claim 1, wherein when a shot is detected from an encoded moving image, high-speed processing is performed using characteristics of MPEG. 4. The method according to claim 1, wherein a representative frame is extracted when calculating the degree of similarity between shots. 5. The method according to claim 1, wherein the similarity between shots is obtained by fuzzy inference. 6. The moving image automatic hierarchical structuring method according to claim 1, wherein the scene extraction processing is performed based on a degree of connection between the defined shots. 7. The encoded moving image is divided into shots, and the shots are integrated by using the similarity of each of the divided shots to extract a scene to automatically form a hierarchical structure of the moving image. A moving image browsing method characterized by using a hierarchically structured data to easily grasp the contents of the entire moving image and to detect a desired scene or shot.