JP4648183B2 - Continuous media data shortening reproduction method, composite media data shortening reproduction method and apparatus, program, and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable highly compressed abbreviated reproduction, while maintaining comprehensiveness so that a user can observe and search contents in a short period of time. <P>SOLUTION: The invention includes the process steps of: dividing input continuous data into frame sections; calculating the change amount of a feature parameter of each frame section; extracting a frame in which the change amount of the feature parameter is larger than a predetermined value; clustering the extracted frame; calculating the change amount of a cluster unit which is clustered; restructuring the data by linking only clusters in each of which the change amount of the cluster unit is larger than a predetermined value; and outputting the linked restructured data. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a continuous media data shortening playback method, a composite media data shortening playback method and apparatus, a program, and a computer-readable recording medium for shortening playback of continuous media data. The present invention relates to a continuous media data shortening / reproducing method, a composite media data shortening / reproducing method and apparatus, a program, and a computer-readable recording medium for enabling overview and search of contents.

  The method of playing back continuous media by shortening the time is roughly divided into "high-speed playback" that shortens the playback time by playing at high speed without losing the entire content of the continuous media as much as possible. “Partial playback” that shortens the playback time by means of “Summary playback” that analyzes the content of continuous media semantically, extracts semantically important sections, and shortens the playback time. is there.

  Video is a composite media composed of images and sounds, and images are classified into moving images and still images. Originally, sound includes sound, music, sound and the like, but here, sound is used as including the meaning of sound. In particular, in order to explain the method for continuous media, the video will be described as consisting of video and audio.

  Audio is a typical example of continuous media. The frequency of the reproduced sound is high when fast-forwarding with a normal tape recorder or the like, and it is difficult to grasp the content at high-speed reproduction beyond the limit of about twice. In order to improve this, sound is played back at the same frequency as the original sound by dividing into frames and thinning them out at regular intervals or regularly, or by detecting the pitch period and thinning out regularly in units of pick section waveforms. However, a technique that enables high-speed reproduction has been proposed (see, for example, Patent Document 1 and Patent Document 2).

  Also, a method for detecting a voiced section and a silent section using information unique to speech, deleting the silent section, and reproducing only the voiced section (for example, see Patent Document 1), or a detected voiced section. Is further classified into a vowel section, a consonant section, a transition section between a vowel section and a consonant section, and a noise section, and a device that changes the degree of compression and reduces speech deterioration (see, for example, Patent Document 3). Proposed.

  However, even if these methods are used, the reproduction speed is limited to about 2 to 3 times, and it becomes difficult to grasp the contents at a speed higher than this. In addition, use of audio-specific information is not applicable to other continuous media, so it is necessary to divide the processing for each medium, and it applies to high-speed playback of composite media (for example, video that is a composite of audio and video) In some cases, the processing is complicated.

  Another example of continuous media is moving images, which can be played at high speeds by increasing the playback frequency like normal video recorders, fast-forwarding, etc. A possible approach has been proposed. However, as with voice, it is difficult to grasp the content at high speed playback above a certain speed.

  Also, a method has been proposed in which a certain physical quantity (brightness change amount, cut point, etc.) is extracted from a moving image, and a portion that satisfies a certain condition is extracted and shortened (for example, see Patent Document 4 and Patent Document 5). However, since information specific to moving images is applied, it cannot be applied to other media such as audio.

  In addition, as the various shortened playback (high-speed playback, partial playback, summary playback) techniques using a combination of moving images and sounds, the above-described combination or an extended method thereof has been proposed (for example, Patent Document 5, Patent Document) 6, see Patent Document 7). However, these also have the same drawbacks as the above-mentioned methods, or focus on the moving image and only the partial sound attached to the moving image is reproduced, so that it is difficult to grasp the entire flow.

In view of these drawbacks, a continuous media data high-speed playback method, a composite media data high-speed playback method, etc. as a continuous media high-speed playback technology that improves playback speed with a versatile and easy-to-integrate framework that does not specialize in video and audio Has been proposed (see, for example, Patent Document 8). However, even if this method is used, high-speed playback of about 3 to 5 times speed is the limit in the case of media including sound.
JP-A-6-202691 JP 2000-259200 A Japanese Patent Laid-Open No. 9-152889 JP-A-4-237284 JP-A-6-233227 JP-A-8-116514 JP 2003-169298 A JP 2005-204003 A

  The above-described conventional continuous media data high-speed playback method, etc., when the playback speed reaches a certain level or higher, corresponds to the minimum constituent unit of continuous media (for example, in the case of speech, it corresponds to phonology, syllables, words, phrases, etc. (Corresponding to the movement of the moving object etc.) is degenerated to the extent that it does not retain its original form, so that there is a problem that the intelligibility is remarkably lowered and the contents cannot be understood. For example, in the case of speech, if it exceeds 3 to 5 times speed, phoneme loss becomes excessive, and it becomes impossible to draw an outline of meaning as words or sentences.

  Also, in the high-speed playback method of composite media data, etc., since the amount of change integrated based on the amount of change of frames synchronized between each media is calculated, when the number of media increases, compared to the compression rate as composite media, There is a problem in that the local compression rate of each medium itself is increased, and the minimum structural unit, which is the above-described problem, quickly reaches a state where the original structure is degenerated. For example, in the case of composite media of audio and moving images, even if the overall speed is set to 5 × speed, the local compression rate of the audio exceeds the limit equivalent to 5 × speed, and the contents of the audio cannot be heard.

  The present invention has been made in view of the above-described actual situation of the prior art, and its purpose is to shorten the compression more highly while maintaining the intelligibility so that the user can overview and search the contents in a short time. Continuous media data shortening / reproducing method, composite media data shortening / reproducing method and apparatus and program, and computer-readable recording that can be reproduced and can be applied to other continuous media and composite media without specializing in audio and video The purpose is to provide a medium.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1) is a continuous media data shortening reproduction method for shortening and reproducing continuous media data,
A media input / feature parameter change amount calculating step (step 1) in which the media input / feature parameter change amount calculation means divides the input continuous media data into frame intervals and calculates the change amount of the feature parameter in each frame interval; ,
Frame frame extraction means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in the intelligibility of continuous media data, the amount of change in characteristic parameter is extracted is limited to large kina frame An extraction step (step 2);
A clustering step (step 3) in which the clustering means clusters the extracted frames by grouping the extracted frames having a small time interval with the extracted frame as the same cluster ;
A cluster change amount calculating unit (step 4) in which a cluster change amount calculating unit calculates a change amount of clustered clusters;
The media reconstruction means uses a product obtained by multiplying the total compression rate by which the inverse of the total compression rate is equal to or higher than the playback speed by multiplying the total number of frames of continuous media data by the total compression rate, and the total number of frames included in the cluster. A medium reconstructing step (step 5) of selecting a cluster having a large change amount in cluster units, concatenating frames included in the selected cluster while maintaining an order relation, and reconfiguring the clusters. ,
The output means performs an output step (step 6) for outputting the connected reconstruction data.

The present invention (Claim 2 ) is a composite media data shortening playback method for shortening and playing back composite media data composed of a plurality of continuous media data,
A media input / feature parameter change amount calculating unit divides each input continuous media data into frame sections, and calculates a change amount of a feature parameter in each frame section; a media input / feature parameter change amount calculation step;
A variation parameter extraction means for calculating a variation amount of the integrated feature parameter from the variation amount of the feature parameter of each continuous media data;
Frame extracting means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in the intelligibility of continuous media data, the amount of change in integrated characteristic parameter is limited to large kina frame extraction A frame extraction step,
A clustering step in which the clustering means clusters the extracted frames by grouping the extracted frames with a small time interval as the same cluster ; and
A cluster change amount calculating means for calculating a change amount of clustered cluster units, and a cluster change amount calculating step;
The media reconstruction means uses a product obtained by multiplying the total compression rate by which the inverse of the total compression rate is equal to or higher than the playback speed by multiplying the total number of frames of continuous media data by the total compression rate, and the total number of frames included in the cluster. A medium reconfiguration step of selecting a cluster having a large amount of change in cluster units, concatenating and reconfiguring frames included in the selected cluster in an orderly relationship ,
An output means performs an output step of outputting the reconstruction data.

The present invention (Claim 3 ) is a composite media data shortening reproduction method for shortening and reproducing composite media data composed of a plurality of continuous media data,
A media input / feature parameter change amount calculating unit divides each input continuous media data into frame sections, and calculates a change amount of a feature parameter in each frame section; a media input / feature parameter change amount calculation step;
Frame frame extraction means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in the intelligibility of continuous media data, the amount of change in characteristic parameter is extracted is limited to large kina frame An extraction step;
Clustering means, by grouping frames issued extracted, what time interval between the extraction frame is smaller as the same cluster, clustering step of clustering,
A time compression step in which the time compression means compresses each continuous media data in a clustered cluster unit;
A cluster change amount calculating means for calculating a change amount of the time-compressed cluster unit, and a cluster change amount calculating step;
The media reconstruction means uses a product obtained by multiplying the total compression rate by which the inverse of the total compression rate is equal to or higher than the playback speed by multiplying the total number of frames of continuous media data by the total compression rate, and the total number of frames included in the cluster. A medium reconfiguration step of selecting a cluster having a large amount of change in cluster units, concatenating and reconfiguring frames included in the selected cluster in an orderly relationship ,
An output means performs an output step of outputting the reconstruction data.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 4 ) is a continuous media data shortening / reproducing apparatus for shortening and reproducing continuous media data,
Media input / feature parameter change amount calculation means 110 that divides input continuous media data into frame sections and calculates a change amount of a feature parameter in each frame section;
The frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in the intelligibility of continuous media data, the frame extraction unit 120 the amount of change feature parameters are extracted is limited to large Kina frame,
Clustering means 130 for clustering the extracted frames by grouping those having a small time interval with the extracted frame as the same cluster ;
A cluster change amount calculating means 140 for calculating a change amount of clustered cluster units;
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. Media reconstructing means 150 that selects a cluster having a large amount of change in cluster units, concatenates frames included in the selected cluster while maintaining an order relation, and reconfigures the medium;
And output means 160 for outputting the reconstructed data connected.

The present invention (Claim 5 ) is a composite media data shortening / reproducing apparatus for shortening and reproducing composite media data composed of a plurality of continuous media data,
Media input / feature parameter change amount calculating means for dividing each input continuous media data into frame sections and calculating a change amount of a feature parameter in each frame section;
Change amount parameter extraction means for calculating the change amount of the integrated feature parameter from the change amount of the feature parameter of each continuous media data;
The frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in the intelligibility of continuous media data, a frame extracting means the amount of change integrated feature parameters are extracted is limited to large kina frame ,
Clustering means for clustering the extracted frames by grouping those having a small time interval with the extracted frame as the same cluster ;
Cluster change amount calculating means for calculating the change amount of clustered clusters,
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. In addition, media reconfiguration means for selecting a cluster having a large change amount in cluster units, concatenating frames included in the selected cluster while maintaining an order relation, and reconfiguring ;
Output means for outputting reconstructed data.

The present invention (Claim 6 ) is a composite media data shortening / reproducing apparatus for shortening and reproducing composite media data composed of a plurality of continuous media data,
Media input / feature parameter change amount calculating means for dividing each input continuous media data into frame sections and calculating a change amount of a feature parameter in each frame section;
Serial frame number of a frame divided by the playback speed that does not cause extreme lowering of intelligibility the total number of frames in the continuous media data, a frame extracting means the amount of change in characteristic parameter is extracted is limited to large Kina frame,
Clustering means for clustering the extracted frames of each continuous media data by grouping the frames having a small time interval with the extracted frames as the same cluster ;
A time compression means for time compressing each continuous media data in clustered cluster units;
A cluster change amount calculating means for calculating a change amount of the time-compressed cluster unit;
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. In addition, media reconfiguration means for selecting a cluster having a large change amount in cluster units, concatenating frames included in the selected cluster while maintaining an order relation, and reconfiguring ;
Output means for outputting reconstructed data.

The present invention (Claim 7 ) is a program that causes a computer to function as the apparatus according to Claims 4 to 6 .

The present invention (Claim 8 ) is a computer-readable recording medium storing a program that causes a computer to function as the apparatus according to Claims 4 to 6 .

  As described above, according to the present invention, the first step of extracting a frame having a large amount of change in the characteristic parameter and the second step of reconstructing data by connecting only clusters having a large amount of change in cluster units. The compression process has a two-stage configuration including the compression process of No. 1. The first stage performs compression corresponding to high-speed playback with a quality that holds the minimum structural unit of continuous media, and the second stage performs continuous compression corresponding to partial playback. Thinning is performed in units larger than the minimum constituent unit of the media and compression is performed, and there is an advantage that both high-speed playback and partial playback are effective. In other words, when the compression ratio is low, shortened playback is mainly performed by high-speed playback, and when the compression ratio is high, it is possible to seamlessly shift to shortened playback with the effect of partial playback and the compression ratio is high. Even if it becomes, there exists an advantage which can be shortened, without causing the remarkable fall of intelligibility.

  In particular, when applied to audio as continuous media, shortened playback with intelligibility is possible even with a shortening of 1/5 or less, which corresponds to 5 times or more speed.

  In addition, the quasi-synchronized composite media data shortened playback method of the present invention eases restrictions on frame-to-cluster synchronization, thereby reducing the increase in the compression rate of each media itself even if the number of media increases. Thus, there is an advantage that a higher compression rate can be realized.

  In addition, the index of feature parameter change used in the present invention and the clustering of extracted frames are general-purpose, can be applied to any continuous media, can be handled uniformly on different media, and can be shortened arbitrarily. There is an advantage that the playback time and an arbitrary total compression rate can be specified.

  Because of these advantages, it can be said that the method is more effective than the conventional method in applications such as overview and search of media content that emphasizes short time rather than completeness and semantic consistency.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 is a configuration diagram of the continuous media data shortening / reproducing apparatus according to the first embodiment of the present invention.

  As shown in the figure, the continuous media data shortening playback apparatus includes a continuous media input unit 101, a change amount parameter extraction unit 102, a playback speed instruction unit 103, a clustering unit 104, a cluster change amount calculation unit 105, and a total compression rate instruction unit. 106, a reproduction cluster selection unit 107, a continuous media reconstruction unit 108, and a continuous media output unit 109.

  The portion configured by the continuous media input unit 101 and the change amount parameter extraction unit 102 corresponds to a unit that divides continuous media data into frame sections and calculates a change amount of a feature parameter in each frame section.

  The playback speed instruction unit 103 and the clustering unit 104 are composed of a media input / feature parameter change amount calculating means for extracting a frame whose feature parameter change amount is larger than a predetermined value and a clustering means for clustering the extracted frames. It corresponds to.

  The cluster change amount calculation unit 105 corresponds to a cluster change amount calculation unit that calculates a change amount of clustered clusters.

  The portion composed of the total compression rate instruction unit 106, the reproduction cluster selection unit 107, and the continuous media reconstruction unit 108 is a media reconstruction unit that reconstructs data by concatenating only clusters in which the amount of change in cluster units is larger than a predetermined value. It corresponds to a configuration means.

  The continuous media output unit 109 corresponds to an output unit that outputs connected reconstructed data.

  Hereinafter, each component will be described.

  FIG. 4 is a flowchart of the operation in the first embodiment of the present invention.

  The continuous media input unit 101 reads the input continuous media data into a buffer (not shown) and sends it to the change amount parameter extraction unit 102 (step 101). For example, when the input data is analog data, the input of the continuous media input unit 101 may be read while being converted into digital data, or may be read directly as file format continuous media data. Alternatively, continuous media data stored in the memory may be read.

  The data amount to be read by the continuous media input unit 101 may be read all at once, may be read periodically in a fixed unit amount, or may be read in an arbitrary amount dynamically.

  If necessary, the digital data of the input continuous media is adjusted to the variation parameter extraction unit 102 before being read into a buffer (not shown) or when being output from the buffer to the variation parameter extraction unit 102. Processing for format conversion may be added.

  The data amount output from the read buffer (not shown) to the change amount parameter extracting unit 102 may be the entire continuous media data delivered in a lump or may be divided into fixed units and delivered periodically. Alternatively, an arbitrary amount may be sequentially passed dynamically. The amount of data to be output may be the same as or different from the amount of data to be read (in the case of being different, buffer link is performed by the continuous media input unit 101).

  When the continuous media data to be input is in a format required by the variation parameter extraction unit 102 and all of the data is to be collectively transferred to the variation parameter extraction unit 102, the function of the continuous media input unit 101 is changed. A configuration including the parameter extraction unit 102 is also possible.

  The change amount parameter extraction unit 102 divides the continuous media data from the continuous media input unit 101 into small sections (frame sections) having a fixed period, and calculates a change amount of a feature parameter representing each frame section (step 102). ). For example, after calculating a feature parameter representative of a frame section from continuous media data, the amount of change may be calculated from the time series of the feature parameter, or a feature parameter representative of a frame section directly from continuous media data The amount of change may be calculated.

  The calculation of the feature parameter representing the frame section and the amount of change of the feature parameter may be calculated from only the data in the frame section, or may be calculated including data outside the frame section.

  The feature parameter may be a scalar or a vector of two or more dimensions. The change amount of the feature parameter is a binary distance between the feature parameter of the frame to be calculated and the feature parameter of the previous frame or the subsequent frame (which can be defined by a distance function that satisfies the mathematical distance axiom. For example, Manhattan distance, Euclidean distance Power distance, Chebyshev distance, Mahanobis distance, etc. may be used) or may be a half of the binary distance between the feature parameter of the previous frame and the feature parameter of the subsequent frame of the frame to be calculated. A value calculated using a distance or the like from the feature parameter n value including a plurality of frames before and after the frame to be performed may be used.

  The playback speed instruction unit 103 sets a local playback speed in order to prevent the minimum structural unit of continuous media from being degenerated to the extent that it does not retain its original shape. Is designated to the clustering unit 104 (step 103). At this time, the total compression rate is acquired from the total compression rate instruction unit 106, and when the reciprocal of the total compression rate is lower than the set reproduction speed, the reciprocal of the total compression rate is instructed to the clustering unit 104.

Based on the playback speed from the playback speed instruction unit 103, the clustering unit 104 calculates the total number of frames of data input to the number of extracted frame change parameter extraction unit / playback speed (1).
Are extracted by the number of extracted frames from the one with the larger change amount of the feature parameter calculated by the change amount parameter extraction unit 102, and the extracted frames are clustered (step 104). Clustering can be determined by a method such as setting a threshold and grouping extracted frames whose time interval between adjacent extracted frames is smaller than this threshold as the same cluster. What is necessary is just to be able to group the extracted frames. For example, in addition to setting the threshold value as a constant, the threshold value can also be dynamically determined from the distribution of time intervals between adjacent extracted frames.

  The cluster variation calculation unit 105 calculates the cluster unit variation clustered by the clustering unit 104 (step 105). The amount of change for each cluster can be calculated from the amount of change in the feature parameter of the frame in the cluster (for example, the maximum value, average value, median value, minimum value, etc. of the feature parameter change amount of the frame in the cluster can be calculated. However, the present invention is not limited to this, and the amount of change in cluster units may be calculated using another feature parameter representing the cluster section.

The total compression rate instruction unit 106 instructs the reproduction cluster selection unit 107 on the total compression rate, which is the compression rate for the entire continuous media for shortened playback (step 106). Alternatively, the total compression rate may be calculated and indicated from the shortened playback time. At this time, the total compression rate is
Shortened playback time / playback time of the entire continuous media data (2)
Can be calculated from

Based on the total compression rate obtained by the total compression rate instruction unit 106, the reproduction cluster selection unit 107 selects the number of frames included in the selected cluster from the clusters having a large change amount in cluster units of the cluster change amount calculation unit 105. The sum of
Total number of frames of continuous media data * Total compression rate (3)
Selection is made until the number of frames calculated in (1) is reached (step 107). Up to clusters whose total number of frames exceeds the above formula (3) may be selected clusters, clusters immediately before exceeding may be selected clusters, and the total number of frames of clusters and continuous media data until just before exceeding. * Total compression rate
-Total number of frames included in the cluster just before
(4)
The frame data of the number of frames calculated in (1) may be additionally supplemented so that the total number of frames matches equation (3). The frame data to be additionally replenished may be blank data (for example, silence, white noise, pink noise, etc. for audio, single color frame image for moving images, etc.), or the same frame data as that immediately before replenishment may be used repeatedly. Alternatively, the original frame data following the frame data immediately before replenishment may be selected by raising the number of frames of the equation (4). Further, the present invention is not limited to these, and any method that can determine the selected frame based on the total compression rate may be used.

  The continuous media reconstruction unit 108 concatenates and reconstructs the frame data of continuous media data corresponding to the frame of the cluster selected by the reproduction cluster selection unit 107 while maintaining the order relationship (step 108). For example, when connecting, smoothing processing may be added to the data before and after the connection for reducing the discontinuity of the data before and after the connection. Corresponding continuous media data may be received from the continuous media input unit via the change amount parameter extraction unit 102, clustering unit 104, cluster change amount calculation unit 105, and reproduction cluster selection unit 107, or the continuous media input unit 101. You may receive directly from.

  The continuous media output unit 109 outputs the continuous media data reconstructed by the continuous media reconstruction unit 108 (step 109). For example, the output may be output to an external output device as needed, or may be output as a file to a recording medium for the purpose of later playback, or output to a storage medium such as a memory, and another device The application may be used sequentially.

[Second Embodiment]
In this embodiment, a synchronous composite media data shortening / playback apparatus will be described.

  FIG. 5 is a block diagram of a synchronous composite media abbreviated playback apparatus according to the second embodiment of the present invention.

As shown in the figure, the synchronous composite media abbreviated playback apparatus includes _n continuous media input units 201 from a continuous media input unit 201 ₁ to a continuous media input unit 201 _n , and a parameter extraction unit 202 ₁ to a parameter extraction unit 202. _n change parameter extraction unit 202 of n, parameter synthesis unit 210, playback speed instruction unit 203, clustering unit 204, cluster change amount calculation unit 205, total compression rate instruction unit 206, playback cluster selection unit 207, continuous media reproduction configuration unit ₂₀₈₁ from continuous media reconstruction unit 208 _n n successive media reconstruction unit 208, and a n consecutive media output unit 209 of the continuous media output unit 209 _n from the continuous media output unit 209 _1.

  The n continuous media may be n types of continuous media, the same type of n channels of continuous media, or j types of continuous media of k channels (j * k = n). Without being limited thereto, the total may be n continuous media.

A portion composed of _n continuous media input units 201 from the continuous media input unit 201 ₁ to the continuous media input unit 201 _n and n change parameter extraction units 202 from the parameter extraction unit 202 ₁ to the parameter extraction unit 202 _n This corresponds to media input / feature parameter change amount calculation means for dividing each continuous medium into frame sections and calculating the change amount of the feature parameter of each continuous media data in each frame section.

  The parameter synthesizing unit 210 corresponds to a change parameter extraction unit that calculates the change amount of the feature parameter of the continuous media integrated from the change amount of the feature parameter of the plurality of continuous media data.

  The portion constituted by the reproduction speed instruction unit 203 and the clustering unit 204 corresponds to a frame extraction unit that extracts a frame with a large change amount of the integrated feature parameter and a clustering unit that clusters the extracted frames.

  The cluster change amount calculation unit 205 corresponds to cluster change amount calculation means for calculating the change amount of clustered clusters.

Section, consisting of a total compression rate instructing unit 206 and the reproducing cluster selection unit 207 from the continuous media reconstruction unit 208 ₁ of the continuous media reconstruction unit 208 _n n consecutive media reconstruction unit 208, the amount of change in cluster units This corresponds to a media reconstruction unit that connects only large clusters and generates reconstruction data for each continuous media data.

The _n continuous media output units 209 from the continuous media output unit 209 ₁ to the continuous media output unit 209 _n correspond to output means for outputting each connected reconstructed data.

  The operation in the above configuration will be described.

  FIG. 6 is a flowchart of the operation in the second embodiment of the present invention.

The continuous media input unit 201 ₁ to the continuous media input unit 201 _n read the continuous media data n from the corresponding input continuous media data 1 into respective buffers (not shown), and the corresponding parameter extraction units. send synchronization from 202 ₁ to the parameter extraction unit 202 _n (step 201).

  Each continuous media input unit 201 has the same amount of data sent to each parameter extraction unit 202 between the continuous media input units 201 in addition to the processing of the continuous media input 101 of the continuous media data shortening / playback apparatus in the first embodiment. Match the time (not necessarily the same size, the amount that will be the same time when played back), and the same timing (there is no need to match the operating time exactly, but the data processing position and order are the same) Send in sync with.

  The amount and timing of reading each continuous media data between each continuous media input unit 201 may be synchronized, but may not be synchronized.

Parameter extracting unit 202 ₁ parameter extraction unit 202 _n from divides each successive media data received from the continuous media input unit 201 corresponding to the respective predetermined period of small sections (frame sections) representative of the respective frame section The change amount of the characteristic parameter is calculated (step 202).

  Each parameter extraction unit 202 performs the same processing as the change amount parameter extraction unit 102 in the first embodiment, except that each parameter extraction unit 202 has a common frame section length (frame period).

Parameter combination unit 210 obtains the amount of change in each of the feature parameters from the parameter extraction unit 202 _n n number of parameter extraction unit 202 from the parameter extraction unit 202 _1. The amount of change in each parameter is time-series data of the same number of frames, and is converted into time-series data of one change amount by weighted average to obtain an integrated feature parameter change amount (the weighted average is a feature of each feature). This includes normalization when the unit of parameter change is different and weighting each continuous medium, and also includes the case of arithmetic average, geometric average, etc. The normalization method also includes, for example, the respective normalization methods. By dividing the amount of change of each feature parameter by the maximum value of the amount of change of the feature parameter, normalization may be performed from 0 to 1, or by dividing by the average value of the amount of change of each feature parameter. Alternatively, normalization may be performed by dividing by the variance of the change amount of each feature parameter) (step 203).

  The playback speed instruction unit 203 performs the same processing as the playback speed instruction unit 103 of the continuous media data shortening playback device in the first embodiment (step 204).

  The clustering unit 204 uses the change amount of the integrated feature parameter calculated by the parameter synthesis unit 210 instead of the clustering unit 104 and the change amount parameter extraction unit 102 of the continuous media data shortening / playback device according to the first embodiment. Except for this, the same processing is performed (step 205).

  The cluster change amount calculation unit 205 performs the same processing as the cluster change amount calculation unit 105 of the continuous media data shortening / playback apparatus in the first embodiment (step 206).

  The total compression rate instruction unit 206 performs the same processing as the total compression rate instruction unit 106 of the continuous media data shortening / reproducing apparatus in the first embodiment (step 207).

  The playback cluster selection unit 207 performs the same processing as the playback cluster selection unit 107 of the continuous media data shortening playback device in the first embodiment (step 208).

Continuous Media reconstruction unit 208 _n from continuous media reconstruction unit 208 _1, each frame section data of continuous media data corresponding to the frame number of playback clusters selected by the reproduction cluster selection unit 207, maintaining the order relationship Then, they are connected and reconfigured (step 209). For example, when connecting, smoothing processing may be added to the data before and after the connection in order to reduce the discontinuity of the data before and after the connection. The smoothing process may be applied to all the continuous media reconstruction units 208 or a part of the continuous media reconstruction units 208. Each corresponding continuous media data is received from each continuous media input unit 201 via each parameter extraction unit 202, parameter synthesis unit 210, clustering unit 204, cluster change amount calculation unit 205, and reproduction cluster selection unit 207. Alternatively, it may be received directly from each continuous media input unit 201.

The continuous media output units 209 ₁ to 209 _n output the continuous media data reconstructed by the corresponding continuous media reconstruction units 208 respectively (step 210).

  The continuous media output unit 209 may or may not output in synchronization. For example, when outputting to an external output device as needed, it may be output in synchronization, or output as a file to a recording medium or output to a storage medium such as a memory for later playback. If the devices and applications can be used sequentially, there is no need to synchronize.

[Third Embodiment]
In this embodiment, a semi-synchronous composite media data shortening reproduction apparatus will be described.

  FIG. 7 is a block diagram of a quasi-synchronous composite media data shortening / playback apparatus according to the third embodiment of the present invention.

As shown in the figure, the quasi-synchronous composite media data compaction reproducing apparatus includes n-number of continuous media input unit 301 of the continuous media input unit 301 _n from the continuous media input unit 301 _1, the parameters from the parameter extraction unit 302 ₁ Extraction unit 302 _n parameter extraction unit 302, playback speed instruction unit 303, clustering unit 304, intra-cluster time compression unit 311, total compression rate instruction unit 306, playback cluster selection unit 307, continuous media reconstruction unit 308 ₁ to _n continuous media reconstruction units 308 _n to n continuous media reconstruction units 308, and continuous media output units 309 ₁ to _n continuous media output units 309 _n to n continuous media output units 309.

  The n continuous media may be n types of continuous media, the same type of n channels of continuous media, or j types of continuous media of k channels (j * k = n). Without being limited thereto, the total may be n continuous media.

The parts composed of the n continuous media input units 301 from the continuous media input unit 301 ₁ to the continuous media input unit 301 _{n and} the n parameter extraction units 302 from the parameter extraction unit 302 ₁ to the parameter extraction unit 302 _n are respectively This is equivalent to a media input / feature parameter change amount calculation means for dividing the continuous media data of each frame into frame sections and calculating the amount of change of the feature parameter of each continuous medium in each frame section.

  The portion composed of the playback speed instruction unit 303 and the clustering unit 304 integrates the frame extraction means for extracting a frame with a large amount of change in the feature parameter of each continuous media, and the extracted frame of each continuous media data, This corresponds to clustering means for clustering.

  The intra-cluster time compressing unit 311 corresponds to a time compressing unit that temporally compresses each continuous media data in clustered cluster units.

  The cluster change amount calculation unit 305 corresponds to a cluster change amount calculation unit that calculates a change amount of a cluster unit that is time-compressed.

Section, consisting of n consecutive media reconstruction unit 308 of the total compression rate instruction unit 306 and the reproducing cluster selection unit 307 and the continuous media reconstruction unit 308 _n from continuous media reconstruction unit 308 _1, the compressed time Cluster Corresponding to reconstruction means for generating reconstructed data for each continuous media data by concatenating only time-compressed clusters having a large unit variation.

The _n continuous media output units 309 from the continuous media output unit 309 ₁ to the continuous media output unit 309 _n correspond to output means for outputting each continuous reconstructed data.

  The operation in the above configuration will be described.

  FIG. 8 is a flowchart of the operation in the third embodiment of the present invention.

The continuous media input unit 301 ₁ to the continuous media input unit 301 _n perform the same processing as the continuous media input unit 201 ₁ to the continuous media input unit 201 _n of the synchronous composite media abbreviated playback apparatus of the second embodiment described above. Perform (step 301).

The parameter extraction unit 302 ₁ to the parameter extraction unit 302 _n perform the same processing as the parameter extraction unit 202 ₁ to the parameter extraction unit 202 _n of the synchronous composite media abbreviated playback device of the second embodiment described above (step 302). ).

  The playback speed instruction unit 303 sets a local playback speed in order to prevent the minimum structural unit of each continuous medium from degenerating so as not to retain its original shape. The clustering unit 304 is instructed from the playback speed 1 to the playback speed n, which serve as a reference for extracting a frame with a large amount of change in each feature parameter (step 303). At this time, the total compression rate is acquired from the total compression rate instruction unit 303, and when each playback speed is larger than the reciprocal of the total compression rate, the playback speed is replaced with the reciprocal of the total compression rate and the clustering unit 304 is instructed. .

The clustering unit 304, based on the playback speed n from the playback speed 1 from the reproduction speed instruction unit 303, for each media data, calculates the number of extracted frames of the formula (1) described above, the parameters extracted from the parameter extraction unit 302 _n The change amount of each feature parameter is acquired from the n parameter extraction units 302 of the unit 302 _{n, and} the number of extracted frames is extracted for each media data from the larger feature parameter change amount. Clustering is performed based on the time intervals of sections not extracted in all media data (step 304).
If there is no section that is not extracted in all media data, or if there are few sections, the overlap frequency of the extracted sections may be calculated and clustered based on the time interval of the sections with a low overlap frequency of the extracted sections, Clustering may be performed on the basis of the magnitude of the change amount obtained by integrating the change amounts of the feature parameters of the frame. Further, the present invention is not limited to these, and any method that can perform clustering based on each reproduction speed and the amount of change of each characteristic parameter may be used. The clustering method is the same as that of the continuous media data shortening / playback apparatus in the first embodiment.

  The intra-cluster time compression unit 311 performs time compression on the clusters clustered by the clustering unit 304 by relaxing the frame synchronism between media within the cluster, and generates a time-compressed cluster (step 305). In the time compression method, the number of extracted frames is calculated for each media data in the cluster, and the longest extracted frame number is determined from the calculated number. With respect to the media data smaller than the longest extracted frame number, the frame extraction is performed again by the longest extracted frame number from the frame having a large change amount of the characteristic parameter in the cluster section. As a result, frames of the maximum number of extracted frames can be extracted from all media data in the cluster, and when the frames extracted by the respective media data are concatenated, the length of the maximum number of extracted frames equal to or less than the number of frames as a cluster is reduced. A compressed cluster is generated. This time-compressed cluster loses frame synchronization between media within the cluster, but the cluster is compressed in synchronization.

  The cluster change amount calculation unit 305 includes a cluster change amount calculation unit 205 of the synchronous composite media data shortening reproduction apparatus in the second embodiment described above, and the intra-cluster time compression instead of the cluster clustered by the clustering unit 204. The same processing is performed except that the time-compressed cluster in the unit 311 is used (step 306).

  The total compression rate instruction unit 306 performs the same process as the total compression rate instruction unit 206 of the synchronous composite media data shortening / playback apparatus in the second embodiment described above (step 307).

  The playback cluster selection unit 307 performs the same processing as the playback cluster selection unit 207 of the synchronous composite media data shortening playback device in the second embodiment described above (step 308).

The continuous media reconstruction unit 308 ₁ to the continuous media reconstruction unit 308 _n are connected to the continuous media reconstruction unit 208 ₁ to the continuous media reconstruction unit 208 of the synchronous composite media data shortening reproduction apparatus in the second embodiment described above. Processing similar to _n is performed (step 309).

The continuous media output unit 309 ₁ to the continuous media output unit 309 _n are the same as the continuous media output unit 209 ₁ to the continuous media output unit 209 _n of the synchronous composite media data shortening / playback device in the second embodiment described above. Processing is performed (step 310).

  The first to third embodiments described above are not limited to the configurations shown in FIGS. 3, 5, and 7, and various applications are possible.

  Further, the continuous media data shortening / reproducing apparatus and the composite media shortening / reproducing apparatus of the present invention can be realized only by hardware using a logic circuit or the like, and are executed by using a computer and software executed thereby. It is also possible to do.

  The software can be distributed via a content-readable recording medium or a communication line.

[First embodiment]
An embodiment in which the continuous media data shortening / reproducing apparatus is applied to sound will be described with reference to FIGS. 3 and 9 to 13.

  Voice data (for example, 10 kHz sampling, 16-bit linear PCM, which is spoken with “Aki Aki” (phoneme symbol “aki aki”)) input at the continuous media input unit 101 will be described. FIG. 9 shows an example of a speech waveform of frequency and other speech codes (which may be used). Each phoneme symbol indicates a section of the phoneme that is pronounced.

  In the variation parameter extraction unit 102, when the present embodiment is described with the frame section of the audio data (for example, the frame section length is 10 ms), 100 points of audio data becomes the audio data of the frame section length. The voice power is calculated as a representative characteristic parameter. For the calculation of audio power, for example, using 256 points (25.6ms audio data) including audio data outside the frame interval centered on the frame interval, after applying a Blackman window with a window length of 256 points, the audio power is calculated. Power can be calculated and used as a representative value for the frame interval (the window length may be other than 256 points, the window shape may be other than the Blackman window, and the windowing calculation is not necessarily performed). Also good).

  Here, an example of audio power corresponding to the audio waveform of FIG. 9 is shown in FIG. In the case of voice power, it is a scalar time series. In addition to voice power, Δ (delta) power, FFT coefficient, LPC coefficient, cepstrum coefficient, Δ (delta) cepstrum coefficient, mel frequency cepstrum coefficient (MFCC), Δ ( Delta) Mel frequency cepstrum coefficient (ΔMFCC), various speech analysis parameters similar to these, and combinations thereof can be used, and not only scalars but also vectors.

  As the amount of change of the feature parameter, for example, an absolute value of a difference between frames of audio power is calculated. It may be the absolute value of the difference between the audio power of the frame and the previous frame, the absolute value of the difference of the audio power of the frame and the subsequent frame, or half the difference of the audio power of the previous frame and the subsequent frame. It may be an absolute value or may be calculated by combining audio powers of a plurality of frames before and after. Moreover, you may calculate the absolute value of (delta) power directly from audio | voice data.

  Here, FIG. 11 shows an example of the amount of change in audio power corresponding to the audio power in FIG. The voice power is a scalar, and an example of the difference calculation has been shown. However, when generalized including the case of a vector, it can be replaced with a binary distance calculation. As an example in which a vector is replaced by distance calculation, a mel frequency cepstrum distance (MFCD) can be calculated from an MFCC or Δ (delta) MFCC of a multidimensional vector.

  The total compression rate instruction unit 106 designates the total compression rate or the shortened playback time that the user wants to shorten. Here, description will be made using an example in which the total compression rate is designated as 1/6 (corresponding to 6 times speed in high-speed playback).

  In the playback speed instruction unit 103, a local component in order to prevent a minimum constituent unit (here, phonemes, words, etc.) of a continuous medium (here, phonemes, words, etc.) from degenerates (which corresponds to being inaudible here) to an original form. FIG. 12 shows an example in which the playback speed is set to 3 × speed with few missing phonemes. FIG. 13 shows an example in which the speed is set to 6 × speed equivalent to the total compression rate for comparison. The graph of (c) of FIG. 13 is an example in which a threshold value of the amount of change in audio power is set based on a 6 × speed reproduction speed, and shows a frame in which a shaded portion whose value is above the threshold value is extracted. . The graph of FIG. 13D is an example of concatenating the extracted frames. In the original speech “Akiaki (phoneme symbol“ aki aki ”)”, the phoneme / i / is missing and the phoneme / k / Is an example where the section is too short to be heard. As a result, the extracted voice is “Oh”, and the meaning as a word is lost. On the other hand, FIG. 12 is an example in which the 3 × speed is set appropriately with few phoneme omissions, and the threshold value of the power change amount is set based on the 3 × speed playback speed and extracted from the graph of FIG. At this stage, all phonemes of the original speech “Aki Aki (phoneme symbol“ aki aki ”) remain.

  The clustering unit 104 performs processing and clustering for extracting frames based on the local reproduction speed from the one with the large amount of change in audio power. The graph of FIG. 12C is an example in which, as clustering, a threshold of the maximum section length for cluster division is set, and if the sections of adjacent extracted frames are closer than the threshold of the maximum section length, they are integrated as the same cluster. It is. Here, two clusters are formed.

  The cluster change amount calculation unit 105 calculates the change amount in units of clusters for the formed cluster. For example, by taking the average of the feature parameter change amounts of the extracted frames included in the formed cluster, the change amount in cluster units can be calculated, but various methods are possible.

  The reproduction cluster selection unit 107 selects a cluster having a large change amount in cluster units. Here, although the total compression rate is 1/6, since frame extraction is performed based on the triple speed of the local reproduction speed, it is further reduced to 1/2 by selecting the cluster. (D) of FIG. 12 is an example in which a cluster having a large amount of change in cluster units is selected from the clusters formed in FIG. As a result, the voice “Aki” is generated. Only one of the two words will be played back, but shortened playback at a level that can be understood as a word if it is interpreted that the cluster with the larger amount of change in cluster units represents the characteristics of the audio data. It can be said that it is appropriate. This is because, as shown in FIG. 13D, even if a frame is extracted from two words, it is meaningless if the contents cannot be seen.

  The continuous media reconstruction unit 108 extracts audio data corresponding to the frame selected by the reproduction cluster selection unit 107, and concatenates and reconstructs the data while maintaining the order relationship.

  When audio data is connected, a smoothing process may be applied to the connected part. If the audio fragment data are connected as they are, they are perceived as noise during reproduction due to discontinuity, but the discontinuity can be eliminated and noise can be reduced by smoothing processing.

  As an example of the smoothing process, there is a method of taking a moving average of audio data in a certain section before and after connection. For example, a moving average of data of a total of 5 points before and after may be taken, or a number other than 5 points may be taken. A smoothing method other than moving average may be used.

  For example, the continuous media output unit 109 may output to an audio output device such as a speaker or a headphone and reproduce it at any time, or may output a file for the purpose of reproducing later.

[Second Embodiment]
In this embodiment, a semi-synchronized composite media data shortening / reproducing apparatus will be described with reference to FIG. 14 based on the terms in FIGS.

  FIG. 14 is an example of the operation of the semi-synchronous composite media data shortening reproduction method according to the second embodiment of the present invention, in which audio is applied to the continuous media 1 and moving images are applied to the continuous media 2.

FIG. 14A shows the frame period of the original data obtained by clustering the frame sections clustered by the clustering unit 204 via the continuous media input unit 201 ₁ , the continuous media input unit 202 ₂ , the parameter extraction unit 202 ₁ , and the parameter extraction unit 202 _2. This is an example of mapping together.

  FIG. 14B shows an example in which extracted frames are connected to FIG. 14A while maintaining frame synchronization.

  FIG. 14C shows an example in which FIG. 14B is further compressed by the intra-cluster time compressing unit 311 after being further compressed in the cluster. As can be seen from a comparison between FIGS. 14B and 14C, although the frame of the audio and the moving image is not synchronized, the movement of the movement of the phonology of the audio and the moving object of the moving image is maintained in a state where the synchronization is performed in cluster units. It can be shortened while maintaining intelligibility.

  Thereafter, the cluster change amount calculation unit 205 calculates the change amount in cluster units, and the reproduction cluster selection unit 207 selects a cluster having a large change amount in cluster units according to the total compression rate.

The continuous media reconstruction unit 208 ₁ and the continuous media reconstruction unit 208 ₂ extract media data corresponding to the frame sections in the selected cluster, and connect and reconfigure them while maintaining the order relationship.

The continuous media output unit 209 ₁ and the continuous media output unit 209 ₂ may output to an audio output device such as a speaker or a headphone and a display device such as a display at any time, or may be a file for playback later. It may be output. When outputting a file, the sound and the moving image may be output to separate files, or may be converted into a composite format and output to a single file.

  The present invention is not limited to the above-described embodiments and examples, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a technique for shortening and reproducing various continuous media.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the continuous media data shortening reproduction | regeneration apparatus in the 1st Embodiment of this invention. It is a flowchart of the operation | movement in the 1st Embodiment of this invention. It is a block diagram of the synchronous type composite media data shortening reproduction | regeneration apparatus in the 2nd Embodiment of this invention. It is a flowchart of the operation | movement in the 2nd Embodiment of this invention. It is a block diagram of the shortening reproducing | regenerating apparatus of the semi-synchronous type composite media data in the 3rd Embodiment of this invention. It is a flowchart of the operation | movement in the 3rd Embodiment of this invention. It is an example of the audio | voice waveform of 1st Example of this invention. It is an example of the audio | voice power of 1st Example of this invention. It is an example of the variation | change_quantity of the audio | voice power of 1st Example of this invention. It is an example of the reconfigure | reconstructed audio | speech data in the case of clustering of 1st Example of this invention. It is an example of the reconfigure | reconstructed audio | voice data in the case of no clustering of 1st Example of this invention. It is an example of operation | movement of the semi-synchronous composite media data shortening reproduction method of 2nd Example of this invention.

Explanation of symbols

101, 201, 301 Continuous media input unit 102, 202, 302 Change parameter extraction unit 103, 203 Playback speed designation unit 104, 204, 304 Clustering unit 105, 205, 305 Cluster change calculation unit 106, 206, 306 Total compression Rate indication unit 107, 207, 307 Playback cluster selection unit 108, 208, 308 Continuous media reconstruction unit 109, 209, 309 Continuous media output unit 110 Media input / feature parameter change amount calculation unit 120 Frame extraction unit 130 Clustering unit 140 Cluster Change amount calculation means 150 Media reconstruction means 160 Output means 210 Parameter composition section 311 Intracluster time compression section

Claims (8)

A continuous media data shortened playback method for shortening and playing back continuous media data,
Media input / feature parameter change amount calculation means divides the input continuous media data into frame sections, and calculates a change amount of the feature parameter in each frame section;
Frame extracting means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in intelligibility of the continuous media data, the amount of change in the characteristic parameters are limited to a large kina frame extraction A frame extraction step,
A clustering step in which the clustering means clusters the extracted frames by grouping the extracted frames with a small time interval as the same cluster ; and
A cluster change amount calculating means for calculating a change amount of clustered cluster units, and a cluster change amount calculating step;
The media reconstruction unit is configured to multiply the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the reproduction speed by multiplying the total number of frames of the continuous media data by the total compression rate, and the frames included in the cluster. A medium reconstructing step of selecting a cluster having a large change amount of the cluster unit so that the total number of the same is substantially the same, concatenating and reconfiguring frames included in the selected cluster in an ordered relationship ; ,
An output step in which the output means outputs the concatenated reconstruction data; and
A method for shortening and reproducing continuous media data. A composite media data shortening playback method for shortening and playing back composite media data composed of a plurality of continuous media data,
A media input / feature parameter change amount calculating unit divides each input continuous media data into frame sections, and calculates a change amount of a feature parameter in each frame section; a media input / feature parameter change amount calculation step;
A change parameter extraction means for calculating a change amount of the integrated feature parameter from the change amount of the feature parameter of each continuous media data,
Frame extracting means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in intelligibility of the continuous media data, the amount of deformation feature parameters the integration is limited to large kina frame A frame extraction step to extract
A clustering step in which the clustering means clusters the extracted frames by grouping the extracted frames with a small time interval as the same cluster ; and
A cluster change amount calculating means for calculating a change amount of clustered cluster units, and a cluster change amount calculating step;
The media reconstruction unit is configured to multiply the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the reproduction speed by multiplying the total number of frames of the continuous media data by the total compression rate, and the frames included in the cluster. A medium reconstructing step of selecting a cluster having a large change amount of the cluster unit so that the total number of the same is substantially the same, concatenating and reconfiguring frames included in the selected cluster in an ordered relationship ; ,
An output step for outputting the reconstructed data by an output means;
Synchronous type composite media data shortening reproduction method characterized by performing. A composite media data shortening playback method for shortening and playing back composite media data composed of a plurality of continuous media data,
A media input / feature parameter change amount calculating unit divides each input continuous media data into frame sections, and calculates a change amount of a feature parameter in each frame section; a media input / feature parameter change amount calculation step;
Frame extracting means, the frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in intelligibility of the continuous media data, the amount of change in the characteristic parameters are limited to a large kina frame extraction A frame extraction step,
Clustering means, by grouping the frames issued extracted, what time interval between the extraction frame is smaller as the same cluster, clustering step of clustering,
A time compression step in which the time compression means compresses each continuous media data in a clustered cluster unit;
A cluster change amount calculating means for calculating a change amount of the time-compressed cluster unit, and a cluster change amount calculating step;
The media reconstruction unit is configured to multiply the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the reproduction speed by multiplying the total number of frames of the continuous media data by the total compression rate, and the frames included in the cluster. A medium reconstructing step of selecting a cluster having a large change amount of the cluster unit so that the total number of the same is substantially the same, concatenating and reconfiguring frames included in the selected cluster in an ordered relationship ; ,
An output step for outputting the reconstructed data by an output means;
A quasi-synchronous composite media data shortening reproduction method characterized in that A continuous media data shortening / playback device that shortens and plays back continuous media data,
Media input / feature parameter change amount calculating means for dividing the input continuous media data into frame sections and calculating a change amount of the feature parameter in each frame section;
The frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in intelligibility of the continuous media data, the frame extraction unit amount of change in the feature parameters are extracted is limited to large kina frame ,
Clustering means for clustering the extracted frames by grouping the extracted frames with a small time interval as the same cluster ;
Cluster change amount calculating means for calculating the change amount of clustered clusters,
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of the continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. Medium reconfiguring means for selecting a cluster having a large change amount in cluster units, concatenating and reconfiguring frames included in the selected cluster in an ordered relationship ; and
An output means for outputting the concatenated reconstruction data;
A continuous media data shortening / reproducing apparatus comprising: A composite media data shortening / reproducing apparatus for shortening and reproducing composite media data composed of a plurality of continuous media data,
Media input / feature parameter change amount calculating means for dividing each input continuous media data into frame sections and calculating a change amount of a feature parameter in each frame section;
A change amount parameter extracting means for calculating a change amount of the integrated feature parameter from the change amount of the feature parameter of each continuous media data;
Frame extracting a frame number of a frame divided by the reproduction speed the total number of frames that do not cause an extreme reduction in intelligibility of the continuous media data, the amount of deformation feature parameters the integration extracted is limited to large kina frame Means,
Clustering means for clustering the extracted frames by grouping the extracted frames with a small time interval as the same cluster ;
Cluster change amount calculating means for calculating the change amount of clustered clusters,
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of the continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. as such, selecting the cluster unit of the amount of change is large clusters, and the media reconstructing means frame connected to maintain the order relationship, to reconstruct included in the selected cluster becomes,
Output means for outputting the reconstructed data;
A synchronous composite media data shortening / reproducing apparatus characterized by comprising: A composite media data shortening / reproducing apparatus for shortening and reproducing composite media data composed of a plurality of continuous media data,
Media input / feature parameter change amount calculating means for dividing each input continuous media data into frame sections and calculating a change amount of a feature parameter in each frame section;
Serial frame number of a frame divided by the playback speed that does not cause extreme lowering of intelligibility the total number of frames in the continuous media data, the frame extraction unit amount of change in the feature parameters are extracted is limited to large kina frame ,
Clustering means for clustering the extracted frames of each continuous media data by grouping the frames having a small time interval with the extracted frames as the same cluster ;
A time compression means for time compressing each continuous media data in clustered cluster units;
A cluster change amount calculating means for calculating a change amount of the time-compressed cluster unit;
For the total compression rate at which the reciprocal of the total compression rate is equal to or higher than the playback speed, the multiplication value obtained by multiplying the total number of frames of the continuous media data by the total compression rate is approximately the same as the total number of frames included in the cluster. as such, selecting the cluster unit of the amount of change is large clusters, and the media reconstructing means frame connected to maintain the order relationship, to reconstruct included in the selected cluster becomes,
Output means for outputting the reconstructed data;
A quasi-synchronous composite media data shortening / reproducing apparatus characterized by comprising: Computer
A program which functions as the apparatus according to claim 4 . Computer
A computer-readable recording medium storing a program that functions as the apparatus according to claim 4 .

Images