JP2005204003A - Continuous media data fast reproduction method, composite media data fast reproduction method, multichannel continuous media data fast reproduction method, video data fast reproduction method, continuous media data fast reproducing device, composite media data fast reproducing device, multichannel continuous media data fast reproducing device, video data fast reproducing device, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a continuous media fast reproduction technique that realizes faster reproduction while holding comprehension as much as possible so that a user can grasp contents in a short period of time and is not limited to audio and a moving picture, but applicable to other continuous media and composite media. <P>SOLUTION: A continuous media data fast reproducing device comprises a means of dividing continuous media data into frame sections and calculating variation quantities of feature parameters in each of the frame sections, a means of reconstituting data by connecting only frames which are large in variation quantity of feature parameters, and a means of reproducing the connected reconstituted data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for reproducing continuous media data at high speed, and more particularly to a technique for enabling a user to grasp the contents of continuous media data in a short time.

  Video is a composite media composed of images and sounds, and images are classified into moving images and still images. Originally, sound includes sound, music, sound and the like, but here, sound is used as including the meaning of sound. In particular, in order to explain the method for continuous media, the video will be described as consisting of video and audio.

  Audio is a typical example of continuous media. The frequency of the reproduced sound is high when fast-forwarding with a normal tape recorder or the like, and it is difficult to grasp the content at high-speed reproduction beyond the limit of about twice. In order to improve this, sound is played back at the same frequency as the original sound by dividing it into frame units and thinning them out at regular intervals and regularly, or by detecting the pitch period and regularly thinning out pitch unit waveform units. A method that enables high-speed reproduction has been proposed (see, for example, Patent Documents 1 and 2).

  In addition, using sound-specific information, a method of detecting a voiced section and a silent section, deleting the silent section, and reproducing only the voiced section (for example, refer to Patent Document 1), or detecting a detected voiced section. Furthermore, an apparatus (for example, see Patent Document 3) that classifies into a vowel section, a consonant section, a transition section between a vowel section and a consonant section, and a noise section to change the degree of compression and reduce speech deterioration has been proposed. Yes.

  However, even if these methods are used, the reproduction speed is limited to about 2 to 3 times, and it becomes difficult to grasp the contents at a speed higher than this. In addition, using audio-specific information is not applicable to other continuous media, so it is necessary to divide the processing for each media, and it applies to high-speed playback of composite media (for example, video that is a composite of audio and video) In some cases, the processing is complicated.

  Another example of continuous media is moving images, which can be played at high speeds by increasing the playback frequency like normal video recorders, fast-forwarding, etc. A possible approach has been proposed. However, as with voice, it is difficult to grasp the content at high speed playback above a certain speed.

  In addition, a method has been proposed in which a certain physical quantity (brightness change amount, cut point, etc.) is extracted from a moving image, and a part that satisfies a certain condition is extracted and shortened (see, for example, Patent Documents 4 and 5). However, since information specific to moving images is applied, it cannot be applied to other media such as audio.

In addition, as the various high-speed playback and shortened playback technologies based on combinations of moving images and sounds, the above-described combinations or extended methods thereof have been proposed (see, for example, Patent Documents 5, 6, and 7). However, these also have the same drawbacks as the above-mentioned method, and focus on the moving image, and the sound is only partially reproduced, so that it is difficult to grasp the entire flow.
JP-A-6-202691 JP 2000-259200 A Japanese Patent Laid-Open No. 9-152889 JP-A-4-237284 JP-A-6-233227 JP-A-8-116514 JP 2003-169298 A

  The above prior art has a limit because it is a method of reproducing at high speed uniformly in whole or in a partial section, and in order to shorten it further, it results in eliminating versatility by adding processing specific to each medium. .

  The present invention has been made in view of the above-described prior art, and the purpose of the present invention is to enable higher-speed playback while maintaining as much intelligibility as possible so that the user can grasp the contents in a short time. In addition, it is intended to provide a continuous media high-speed playback technology that can be applied to other continuous media and composite media without specializing in audio and video.

  Accordingly, in order to solve the above problem, the invention according to claim 1 is a continuous media data high-speed reproduction method for reproducing continuous media data by shortening the continuous media data, and dividing the continuous media data into frame segments. A process for calculating a feature parameter change amount, a process for reconstructing data by concatenating only frames having a large feature parameter change quantity, and a process for reproducing the connected reconstructed data. And

  According to a second aspect of the present invention, in the first aspect, the continuous media data is audio data or moving image data.

  According to a third aspect of the present invention, there is provided a composite media data high-speed playback method for shortening and playing back composite media data comprising a plurality of continuous media data, wherein each continuous media data is divided into frame sections. The process of calculating the change amount of the feature parameter of the section, the process of calculating the change amount of the integrated feature parameter from the change amount of the feature parameter of each continuous media data, and only the frame with the large change amount of the integrated feature parameter It has a process of generating reconstructed data for each piece of continuous media data and a process of reproducing the reconstructed data.

  According to a fourth aspect of the present invention, there is provided a multi-channel continuous media data high-speed reproduction method for reproducing multi-channel continuous media data by shortening the multi-channel continuous media data. The process of calculating the amount of change of the feature parameter of the section, the process of calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of the continuous media data of each channel, and the frame with the large amount of change of the integrated feature parameter And a process of generating reconstruction data for each channel and a process of reproducing the reconstruction data.

  According to a fifth aspect of the present invention, there is provided a video data high-speed reproduction method for reproducing video data composed of video data and audio data by shortening the video data, wherein the video data is divided into video data and audio data. Dividing the moving image data into frame sections and calculating the amount of change in the feature parameters in each frame section; dividing the audio data into frame sections and calculating the amount of change in the feature parameters in each frame section; The process of calculating the change amount of the feature parameter of the video data from the change amount of the feature parameter of the data and the change amount of the feature parameter of the audio data, and only the frame of the moving image data having the large change amount of the feature parameter of the video data is connected. The process of generating reconstructed video data and connecting only frames of audio data with large changes in video data feature parameters Generating reconstructed audio data, synthesizing the reconstructed video data and reconstructed audio data to generate reconstructed video data, and reproducing the generated reconstructed video data. It is characterized by.

  According to a sixth aspect of the present invention, there is provided a continuous media data high-speed playback device that shortens and plays back continuous media data. The continuous media data is divided into frame sections, and the amount of change in the characteristic parameter of each frame section is determined. It is characterized by comprising means for calculating, means for reconstructing data by concatenating only frames with a large amount of change in characteristic parameters, and means for reconstructing the concatenated reconstructed data.

  According to a seventh aspect of the present invention, in the sixth aspect, the continuous media data is audio data or moving image data.

  The invention according to claim 8 is a composite media data high-speed playback device that shortens and plays back composite media data composed of a plurality of continuous media data, and divides each continuous media data into frame sections. A means for calculating the change amount of the feature parameter of the section, a means for calculating the change amount of the integrated feature parameter from the change amount of the feature parameter of each continuous media data, and only a frame having a large change amount of the integrated feature parameter. It is characterized by having means for connecting and generating reconstructed data for each continuous media data and means for reproducing the reconstructed data.

  According to a ninth aspect of the present invention, there is provided a multi-channel continuous media data high-speed playback device for reproducing multi-channel continuous media data by shortening the multi-channel continuous media data. Means for calculating the amount of change of the feature parameter of the section, means for calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of the continuous media data of each channel, and a frame with a large amount of change of the integrated feature parameter And a means for generating reconstructed data for each channel and a means for reproducing the reconstructed data.

  According to a tenth aspect of the present invention, there is provided a video data high-speed playback device that shortens and plays back video data composed of video data and audio data, and means for dividing the video data into video data and audio data; Means for dividing the moving image data into frame sections and calculating the amount of change in the feature parameter in each frame section; means for dividing the audio data into frame sections and calculating the amount of change in the feature parameter in each frame section; A means for calculating the amount of change of the feature parameter of the video data from the amount of change of the feature parameter of the data and the amount of change of the feature parameter of the audio data is connected to only the frame of the video data having a large amount of change of the feature parameter of the video data. Only the frame of the audio data having a large amount of change in the feature parameter of the video data is connected. Generating reconstructed audio data, synthesizing the reconstructed video data and reconstructed audio data, generating reconstructed video data, and reproducing the generated reconstructed video data It is characterized by that.

  The invention described in claim 11 is a program, characterized in that the method or apparatus according to any one of claims 1 to 10 described above is described as a computer program and can be executed. And

  The invention described in claim 12 is a recording medium in which a program which records the method or apparatus described in any one of claims 1 to 10 so as to be executable by a computer is recorded. And

  It should be noted that a frame with a large amount of change in the characteristic parameter referred to in the claims can be determined by a method such as setting a threshold and adopting a larger one than this threshold, but is not limited thereto. What can extract only the part with high intelligibility of data is sufficient.

  The multi-channel continuous media data referred to in the claims means multiplexing of the same type of media such as bilingual audio, multi-point audio, and multi-point video.

  In the present invention, the compression rate is higher than that of the prior art by extracting only highly intelligible parts and compressing them non-linearly, instead of compressing them uniformly and linearly for the whole or partial sections and performing non-linear compression. High-speed playback with reduced time becomes possible.

  The method of calculating the highly intelligible part corresponds to the fact that the data is converted into time-series parameters that express the characteristics of the data content, and the large part of the parameters expresses the characteristics well. Can be seen as a high part. By concatenating and reproducing partial data (frames) corresponding to a part with a large change amount of the parameter, it is possible to perform high-speed reproduction with high intelligibility while retaining the characteristics of the data contents as much as possible.

  According to the present invention, it is possible to perform high-speed playback that is non-linear with respect to the time axis, and there are advantages in that the intelligibility is high and the compression rate is high because many features of the contents of the media are left as compared with linear high-speed playback.

  In addition, the feature parameter change amount used in the present invention is general-purpose and can be applied to any continuous media parameter, and high-speed playback giving priority to various features by selecting parameters is possible. There are advantages such as being able to handle different media in a unified manner and enabling high-speed playback that integrates multi-channel media.

  In particular, when applied to audio as continuous media, high-distance reproduction with higher intelligibility and higher compression rate than conventional linear high-speed reproduction is possible. The upper limit of the compression rate that maintains intelligibility varies depending on the content of the audio, but in general, it was about 2 to 3 times in the conventional linear high speed playback, but in the present invention, it is about 5 times high speed playback. Is possible.

  In addition, when applied to video that is a composite media, synchronized high-speed playback is possible while retaining both video and audio features.

  Embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing an example of a continuous media high-speed playback device of the present invention. As shown in FIG. 1, the continuous media high-speed playback apparatus includes a continuous media input unit 1, a change parameter extraction unit 2, a change parameter analysis / playback frame selection unit 3, a playback speed instruction unit 4, and a continuous media reconstruction unit 5. And a continuous media output unit 6. Note that the high-speed audio playback device is one in which audio media is applied as continuous media in the above-described device.

  The portion constituted by the continuous media input unit 1 and the change amount parameter extraction unit 2 corresponds to means for dividing the continuous media data into frame sections and calculating the change amount of the characteristic parameter in each frame section.

  The part constituted by the change amount parameter analysis / playback frame selection unit 3, the playback speed instruction unit 4, and the continuous media reconstruction unit 5 corresponds to means for connecting only frames having a large change amount.

  The continuous media output unit 6 corresponds to means for reproducing the connected reconstructed data.

  The continuous media input unit 1 reads the input continuous media data into a buffer and sends it to the change amount parameter extraction unit 2. For example, when the input data is analog data, the input of the continuous media input unit 1 may be read while being converted into digital data, or may be read directly as file format continuous media data. Alternatively, continuous media data stored in the memory may be read.

  As for the amount of data to be read, the entire data may be read at once, a fixed amount of data may be read periodically, or an arbitrary amount may be dynamically read.

  In addition, if necessary, before the data is read into the buffer or when it is output from the buffer to the change amount parameter extraction unit 2, processing for converting the format of the input digital data of the continuous media according to the change amount parameter extraction unit 2 is added. May be.

  The amount of data to be output from the read buffer to the change amount parameter extraction unit 2 may be transferred as a whole of the continuous media data in a lump, or may be periodically divided and transferred in a fixed unit amount. Any amount may be sequentially passed dynamically. The amount of data to be output may be the same as or different from the amount of data to be read (if different, buffering is performed by the continuous media input unit 1).

  If the input continuous media data is in the format required by the variation parameter extraction unit 2 and all the data is transferred to the variation parameter extraction unit 2 at once, the function of the continuous media input unit 1 is changed. A configuration including the quantity parameter extraction unit 2 is also possible.

  The change amount parameter extraction unit 2 divides the continuous media data from the continuous media input unit 1 into small sections (frame sections) having a fixed period, and calculates a change amount of a characteristic parameter representing each frame section. For example, after calculating a feature parameter representative of a frame section from continuous media data, the amount of change may be calculated from the time series of the feature parameter, or a feature parameter representative of a frame section may be directly calculated from continuous media data. The amount of change may be calculated.

  The calculation of the feature parameter representing the frame section and the amount of change of the feature parameter may be calculated from only the data in the frame section, or may be calculated including data outside the frame section.

  The feature parameter may be a scalar or a vector of two or more dimensions. The change amount of the feature parameter is a binary distance between the feature parameter of the frame to be calculated and the feature parameter of the previous frame or the subsequent frame (which can be defined by a distance function that satisfies the mathematical distance axiom. For example, Manhattan distance, Euclidean distance , Power distance, Chebyshev distance, Mahalanobis distance, etc. may be used.) Or may be a half of the binary distance between the feature parameter of the previous frame and the feature parameter of the subsequent frame of the frame to be calculated. A value calculated using a distance or the like from the characteristic parameter n value including a plurality of frames before and after the frame to be calculated may be used.

  The reproduction speed instruction unit 4 instructs the change amount parameter analysis / reproduction frame selection unit 3 on the reproduction speed for high-speed reproduction. And

To calculate the number of playback frames to be selected by the change amount parameter analysis / playback frame selection unit 3.

  Further, for example, when continuous media data is processed at once, a playback time may be indicated instead of the playback speed. At this time,

From this, the number of playback frames can be calculated.

  The variation parameter analysis / reproduction frame selection unit 3 selects the number of reproduction frames instructed by the reproduction speed instruction unit 4 from the larger one of the characteristic parameter variation calculated by the variation parameter extraction unit 2.

  The continuous media reconstruction unit 5 concatenates and reconstructs the frame section data of the continuous media data corresponding to the number of playback frames selected by the variation parameter analysis / playback frame selection unit 3 while maintaining the order relation.

  For example, when connecting, smoothing processing may be added to the data before and after the connection in order to reduce the discontinuity of the data before and after the connection. Corresponding continuous media data may be received from the continuous media input unit 1 via the variation parameter extraction unit 2, the variation parameter analysis / playback frame selection unit 3, or may be received directly from the continuous media input unit 1. Good.

  The continuous media output unit 6 outputs the continuous media data reconstructed by the continuous media reconstruction unit 5. For example, the output may be output to an external output device as needed, or may be output as a file on a recording medium for the purpose of later reproduction, or output to a storage medium such as a memory and another device, Applications may be used sequentially.

(Second embodiment)
FIG. 2 is a block diagram showing an example of a composite media high-speed playback apparatus according to the present invention. As shown in FIG. 2, the composite media high-speed playback apparatus extracts n continuous media input units from continuous media 1 input unit 21-1 to continuous media n input unit 21-n and parameter n extraction from parameter 1 extraction unit 22-1. N-variation parameter extraction unit, parameter synthesis unit 23, variation parameter analysis / reproduction frame selection unit 24, reproduction speed instruction unit 25, and continuous medium 1 reconstruction unit 26-1 The configuration unit 26-n includes n continuous media reconstruction units and the continuous media 1 output unit 27-1 to n continuous media output units 27-n.

  Note that the multi-channel continuous media high-speed playback device is a case where the multi-channel continuous media configured with n channels is applied to the above-described composite media high-speed playback device instead of the composite media configured with n types of media. .

  N continuous media input units from continuous media 1 input unit 21-1 to continuous media n input unit 21-n and n change parameter extraction units from parameter 1 extraction unit 22-1 to parameter n extraction unit 22-n The part constituted by and corresponds to means for dividing each continuous media data of the composite media high-speed playback device into frame sections, and calculating a change amount of a characteristic parameter of each continuous media data in each frame section.

  The parameter synthesizing unit 23 corresponds to means for calculating the amount of change of the feature parameter of the continuous media data integrated from the amount of change of the feature parameter of the plurality of continuous media data of the composite media high-speed playback device.

  It is composed of a change parameter analysis / playback frame selection unit 24, a playback speed instruction unit 25, and a continuous media 1 reconstruction unit 26-1 to n continuous media reconstruction units 26-n. The portion corresponds to means for connecting only frames with a large amount of change in the characteristic parameters integrated in the composite media high-speed playback apparatus.

  The n continuous media output units from the continuous media 1 output unit 27-1 to the continuous media n output unit 27-n correspond to means for reproducing the connected reconstructed data of the composite media high-speed playback device.

  The continuous media 1 input unit 21-1 to the continuous media n input unit 21-n respectively read the corresponding continuous media data 1 from the continuous media data 1 read into the respective buffers, and corresponding parameter 1 extraction units. The data is sent from 22-1 to the parameter n extraction unit 22-n in synchronization.

  In addition to the same processing as the continuous media input unit 1 of the continuous media high-speed playback device, each continuous media input unit sets the same amount of data to be sent to each parameter extraction unit between the continuous media input units. It is not limited to this, and is sent in synchronism with the same timing (the amount of processing time and the order of data is the same, although it is not strictly necessary to match the operation time) in accordance with the amount of time that is the same when reproduced.

  The data amount and timing for reading each continuous media data between the continuous media input units may be synchronized, but may not be synchronized.

  The parameter 1 extraction unit 22-1 to the parameter n extraction unit 22-n divide each piece of continuous media data received from the corresponding continuous media input unit into small sections (frame sections) having a fixed period, and The amount of change of the characteristic parameter that represents the section is calculated.

  Each parameter extraction unit performs the same process as the change amount parameter extraction unit 2 of the continuous media high-speed playback device, except that the frame length (frame period) of each parameter extraction unit is common.

  The parameter synthesizing unit 23 acquires the amount of change of each feature parameter from the n parameter extracting units of the parameter 1 extracting unit 22-1 to the parameter n extracting unit 22-n. The amount of change of each feature parameter is time-series data of the same number of frames, and is converted into time-series data of one change amount by weighted average to obtain an integrated feature parameter change amount (weighted average This includes normalization when the unit of change amount of the characteristic parameter is different and weighting each continuous media, and also includes the case of arithmetic average, geometric average, etc. By dividing the value of the change amount of each feature parameter by the maximum value of the change amount of the feature parameter, normalization from 0 to 1 may be performed, or dividing by the average value of the change amount of each feature parameter. Or may be normalized by dividing by the variance of the amount of change of each feature parameter.

  The playback speed instruction unit 25 performs the same processing as the playback speed instruction unit 4 of the continuous media high speed playback device.

  The variation parameter analysis / reproduction frame selection unit 24 selects the number of reproduction frames instructed from the reproduction speed instruction unit 25 from the larger variation amount of the integrated feature parameter calculated by the parameter synthesis unit 23.

  The continuous media 1 reconstructing unit 26-1 to the continuous media n reconfiguring unit 26-n are frames of the respective continuous media data corresponding to the frames of the number of playback frames selected by the change parameter analysis / playback frame selection unit 24. The section data is connected and reconfigured while maintaining the order relation.

  For example, when connecting, smoothing processing may be added to the data before and after the connection in order to reduce the discontinuity of the data before and after the connection. The smoothing process may be applied to all continuous media reconstruction units or a part of continuous media reconstruction units.

  Each corresponding continuous media data may be received from each continuous media input unit via each parameter extraction unit, parameter synthesis unit 23, change parameter analysis / reconstruction frame selection unit 24, You may receive directly from a continuous media input part.

  The continuous media 1 output unit 27-1 to the continuous media n output unit 27-n respectively output the continuous media data reconstructed by the corresponding continuous media reconstruction unit.

  Each continuous media output unit may or may not output in synchronization. For example, when outputting to an external output device at any time, it may be output in synchronization, or for later playback, it may be output as a file on a recording medium or output to a storage medium such as a memory. When devices and applications can be used sequentially, there is no need to synchronize.

(Third embodiment)
FIG. 3 is a block diagram showing an example of an embodiment of a high-speed video playback apparatus according to the present invention. As shown in FIG. 3, the high-speed video playback apparatus includes a video input unit 31, a video / audio separation unit 32, a video parameter extraction unit 33, an audio parameter extraction unit 34, a parameter synthesis unit 35, a change parameter analysis / playback frame selection. A unit 36, a playback speed instruction unit 37, a moving image reconstruction unit 38, an audio reconstruction unit 39, a moving image / sound synthesis unit 40, and a video output unit 41.

  The part constituted by the video input unit 31 and the moving image / audio separating unit 32 corresponds to means for separating the video data of the video high-speed playback device into moving image data and audio data.

  The moving image parameter extraction unit 33 corresponds to a unit that divides moving image data of the video high-speed playback device into frame sections and calculates the amount of change in the characteristic parameter in each frame section.

  The audio parameter extraction unit 34 corresponds to means for dividing the audio data of the video high-speed playback device into frame sections and calculating the amount of change in the characteristic parameter in each frame section.

  The parameter synthesizer 35 corresponds to means for calculating the amount of change of the feature parameter as the video data from the amount of change of the feature parameter of the moving image data and the amount of change of the feature parameter of the audio data of the video high-speed playback device.

  The portion constituted by the change parameter analysis / playback frame selection unit 36, the playback speed instruction unit 37, and the moving image reconstruction unit 38 is only a frame of moving image data having a large amount of change in the characteristic parameter of the video data of the video high speed playback device. It corresponds to a means for connecting the two.

  The portion constituted by the change parameter analysis / reproduction frame selection unit 36, the reproduction speed instruction unit 37, and the audio reconstruction unit 39 is only a frame of audio data having a large amount of change in the characteristic parameter of the video data of the high-speed video reproduction apparatus. It corresponds to a means for connecting the two.

  The moving image / audio synthesizing unit 40 corresponds to a unit that synthesizes the reconstructed moving image data and the reconstructed audio data that are connected by the high-speed video playback device, and generates reconstructed video data.

  The video output unit 41 corresponds to means for playing back the reconstructed video data generated by the video high-speed playback device.

  The video input unit 31 is a case where the continuous media data of the continuous media input unit 1 of the continuous media high-speed playback device is video data, and outputs the data to the video / audio separation unit 32 instead of the variation parameter extraction unit 2 To do.

  The video / audio separation unit 32 separates the video data from the video input unit 31 into video data and audio data, and outputs the video data to the video parameter extraction unit 33 and the audio data to the audio parameter extraction unit 34, respectively. .

  The moving image parameter extraction unit 33 is for the case where the continuous media data of the variation parameter extraction unit 2 of the continuous media high-speed playback device is moving image data.

  The audio parameter extraction unit 34 is used when the continuous media data of the variation parameter extraction unit 2 of the continuous media high-speed playback device is audio data.

  The parameter synthesizing unit 35 performs the same processing as the parameter synthesizing unit 23 of the composite media high-speed playback device, and the amount of change of the moving image feature parameter calculated by the moving image parameter extracting unit 33 and the sound feature parameter calculated by the audio parameter extracting unit 34. The change amount of the integrated video feature parameter is calculated from the change amount.

  The playback speed instruction unit 37 performs the same processing as the playback speed instruction unit 4 of the continuous media high speed playback device.

  The change parameter analysis / playback frame selection unit 36 performs the same processing as the change parameter analysis / playback frame selection unit 24 of the composite media high-speed playback device.

  The moving image reconstruction unit 38 is one in which continuous media data is moving image data as one of the continuous media reconstruction units of the composite media high-speed playback device.

  The audio reconstruction unit 39 is one of continuous media reconstruction units of the composite media high-speed playback device, and is used when the continuous media data is audio data.

  The moving image / sound synthesis unit 40 synthesizes the reconstructed moving image data from the moving image reconstruction unit 38 and the reconstructed audio data from the sound reconstruction unit 39 into video data in synchronization.

  The video output unit 41 performs the same processing as the continuous media output unit 6 of the continuous media high-speed playback device, and outputs the reconstructed video data from the moving image / sound synthesis unit 40.

  The first to third embodiments of the present invention are not limited to those shown in FIG. 1, FIG. 2, and FIG. 3, and combinations thereof, for example, composite media are processed in multiple channels, or Changes such as multi-channel processing of video can be made.

  Also, the continuous media high-speed playback device, audio high-speed playback device, composite media high-speed playback device, multi-channel continuous media high-speed playback device, and video high-speed playback device of the present invention may be realized only by hardware using a logic circuit or the like. It is possible to realize this using a computer and software executed thereby. The software can be distributed via a computer-readable recording medium or a communication line.

  An embodiment of a high-speed audio playback device will be described with reference to FIGS. 1 and 4 to 7.

  Voice data (for example, 10 kHz sampling, 16-bit linear PCM, which is spoken with “Aki (phoneme symbol“ aki ”)” input at the continuous media input unit will be described in this embodiment. An example of a speech waveform is shown in FIG. Each phoneme symbol indicates a section of the phoneme that is pronounced.

  In the variation parameter extraction unit 2, when the present embodiment is described assuming that the frame section of the audio data (for example, the frame section length is 10 ms), the sound data of 100 points becomes the voice data of the frame section length. The voice power may be calculated as a representative voice parameter. For the calculation of the audio power, for example, 256 points (25.6 ms of audio data) including the audio data outside the frame interval centered on the frame interval are used, and after applying the Blackman window with a window length of 256 points, the audio power is calculated. Power can be calculated and used as a representative value for the frame interval (the window length may be other than 256 points, the window shape may be other than the Blackman window, and the windowing calculation is not necessarily performed). It is good.)

  Here, an example of the voice power corresponding to the voice waveform of FIG. 4 is shown in FIG. In the case of voice power, it is a scalar time series. In addition to voice power, Δ (delta) power, FFT coefficient, LPC coefficient, cepstrum, Δ cepstrum, various voice analysis parameters similar to these, and combinations thereof may be used. It can be a vector as well as a scalar.

  For the change amount of the audio parameter, for example, the absolute value of the difference between frames of the audio power is calculated. It may be the absolute value of the difference in audio power between the current frame and the previous frame, the absolute value of the audio power difference between the current frame and the subsequent frame, or two minutes of the audio power difference between the previous frame and the subsequent frame. The absolute value of 1 may be used, or may be calculated by combining the audio power of a plurality of frames before and after. Alternatively, the absolute value of Δ power may be calculated directly from the audio data.

  Here, FIG. 6 shows an example of the amount of change in audio power corresponding to the audio power in FIG. The voice power is a scalar, and an example of the difference calculation has been shown. However, when generalized including the case of a vector, it can be replaced with a binary distance calculation.

  The change parameter analysis / playback frame selection unit 3 selects the number of frames corresponding to the playback time (total time / playback speed) from the one with the large amount of change in the audio power (the playback speed is selected from the playback speed instruction unit 4). (You can specify the playback time directly instead of the playback speed.)

  This corresponds to setting a threshold value in the graph of the amount of change in audio power in FIG. 7, and shows a frame in which a shaded portion whose value is above the threshold value is selected.

  The continuous media reconstruction unit 5 extracts audio data corresponding to the frame selected by the change amount parameter analysis / playback frame selection unit 3, and connects and reconstructs the data while maintaining the order relationship.

  When the audio data is connected, a smoothing process may be applied to the connected part. If the audio fragment data are connected as they are, they are perceived as noise during reproduction due to discontinuity, but the discontinuity can be eliminated and noise can be reduced by smoothing processing.

  As an example of this smoothing process, there is a method of taking a moving average of audio data in a fixed section before and after connecting. For example, a moving average of data of a total of 5 points before and after may be taken, or a number other than 5 points may be taken. A smoothing method other than moving average may be used.

  The shaded portion of the speech waveform graph (a) corresponding to the shaded portion of the speech power change graph (c) in FIG. 7 is an example of speech data of a selected frame (d ) Is an example of reconstructed audio data. According to this, not only the silent section is deleted, but also the stationary part with a small change in phoneme is deleted and the part contributing to the intelligibility of the speech remains, so the utterance content “Aki (phoneme symbol“ aki ”)” is also included. It is an example of being compressed in a short time. Thus, there is no guarantee that all phonemes remain, but this is an example in which high-speed playback of an arbitrary playback time (playback speed) that maintains intelligibility compared to the prior art can be realized without changing the pitch of the playback sound.

  For example, the continuous media output unit 6 may output to and reproduce from an audio output device such as a speaker or headphones as needed, or may output a file for the purpose of reproduction later.

  An embodiment of a continuous media high-speed playback device for moving images will be described with reference to FIG.

  Moving image data (for example, the present embodiment is described at 30 frames / second (fps), but may be other than 30 fps) is input to the continuous media input unit 1.

  The change amount parameter extraction unit 2 extracts a frame image (still image) from the moving image data, and for example, sets one frame image as a frame section (1/30 second) in which the change amount parameter is calculated. Moreover, it is good also as a frame area in the some frame image unit.

  As the change amount parameter (change amount of the feature parameter of the moving image data), for example, the amount of change in luminance between the frame image of the frame interval to be calculated and the frame image of the previous frame interval is calculated.

  Further, the luminance change amount in pixel units of the frame image of the frame section to be calculated and the frame image of the subsequent frame section may be calculated, or the pixel unit of the frame image of the previous frame section and the frame image of the subsequent frame section May be calculated by combining the frame images of the preceding and succeeding plural frame sections.

  When a frame section is formed in units of a plurality of frame images, a change amount parameter of the frame section may be calculated from a luminance change amount in pixel units between frame images in the frame section.

  Also, other feature values may be used instead of the luminance change amount in pixel units, and the change amount of the frame image is extracted directly from the encoded moving image data without extracting the pixel information from the frame image. The amount of change may be calculated directly from the difference data in the case of an image code that holds difference data between frames such as MPEG code.

  The change parameter analysis / playback frame selection unit 3 selects the number of frame sections corresponding to the playback time (total time / playback speed) from the one with the large value of the change amount parameter (the playback speed is the playback speed instruction unit 4). (You can specify the playback time directly instead of the playback speed.)

  The continuous media reconstruction unit 5 extracts a frame image corresponding to the frame section selected by the variation parameter analysis / playback frame selection unit 3, and connects them while maintaining the order relation to reconstruct the moving image data.

  For example, the continuous media output unit 6 may output to a display device such as a display and reproduce it at any time, or may output a file for the purpose of reproducing later.

  An embodiment of the high-speed video playback apparatus will be described with reference to FIG.

  Video data (for example, the present embodiment will be described with a moving image of 30 fps and audio of 44.1 kHz sampling, but conditions other than 30 fps and 44.1 kHz may be used) are input to the video input unit 31.

  The moving image / audio separation unit 32 separates moving image data and audio data from the video data.

  The moving image parameter extraction unit 33 and the audio parameter extraction unit 34 calculate the amount of change of each feature parameter by setting the frame section length in common, for example, 1/30 seconds (may be other than 1/30 seconds). The moving image parameter extraction unit 33 calculates a change amount parameter using one frame image as a frame section. The audio parameter extraction unit 34 divides the frame interval length of the audio data into 1470 points of audio data corresponding to 1/30 seconds, and calculates the amount of change in the audio parameters.

  The parameter synthesizing unit 35 receives the moving image variation parameter and the audio variation parameter, takes a weighted average of the respective variation parameters, and sets them as an integrated video variation parameter.

  For example, the video change parameter is normalized by the average value of the video change parameter, and the audio change parameter is normalized by the average value of the audio change parameter. A mean square of the change amount parameter may be calculated and used as an integrated image change amount parameter. As a result, the reproduction is performed with emphasis on the point where either the moving image or the sound is changed, and the synchronized high speed reproduction is possible corresponding to the change of both the moving image and the sound.

  Alternatively, each change parameter may be normalized, multiplied by a weighting parameter corresponding to each continuous medium, averaged, and used as an integrated video change parameter. Accordingly, it is possible to change the balance of each of the moving image-oriented high-speed reproduction and the audio-oriented high-speed reproduction, thereby realizing a range of adjustment and selection according to the type of video content and the purpose of use. Also, by allowing 0 as the weighting parameter, it is possible to perform high-speed video playback using only moving image parameters and high-speed video playback using only audio parameters.

  The change parameter analysis / playback frame selection unit 36 selects the number of frame sections corresponding to the playback time (total time / playback speed) from the video having the largest value of the change amount parameter (the playback speed is a playback speed instruction). Obtained from the section 37. The playback time may be directly specified instead of the playback speed).

  The moving image reconstructing unit 38 extracts moving image data corresponding to the frame section selected by the change amount parameter analyzing / reproducing frame selecting unit 36, and connects and reconstructs the images while maintaining the order relation.

  The audio reconstruction unit 39 extracts audio data corresponding to the frame section selected by the change amount parameter analysis / reproduction frame selection unit 36, and concatenates and reconfigures while maintaining the order relationship.

  The moving image / sound synthesis unit 40 synthesizes video data from the reconstructed moving image data and audio data. Since the video reconstruction unit 38 and the audio reconstruction unit 39 select data in the same frame section, the synchronization between the original video data and the video and audio of the reconstructed video data is maintained.

  For example, the video output unit 41 may output and reproduce an audio output device such as a speaker or headphones and a display device such as a display as needed, or may output a file for the purpose of reproduction later.

  An example of a composite media high-speed playback device (or a combination of a video high-speed playback device and a multi-channel continuous media high-speed playback device) for multi-point video will be described with reference to FIGS.

  Each continuous media input unit in FIG. 2 includes the video input unit 31 and the moving image / sound separation unit 32 in FIG. 3, and each continuous media output unit includes the moving image / sound synthesis unit 40 and the video output unit 41 in FIG. By configuring, the N-point video can be interpreted as a composite media high-speed playback device that combines 2N continuous media. As a result, it is possible to perform high-speed playback of synchronized multi-point video that emphasizes changes at any point and changes in either moving images or audio.

The block block diagram which shows an example of a continuous media high-speed reproduction apparatus. The block block diagram which shows an example of a composite media high-speed reproduction apparatus. The block block diagram which shows an example of embodiment of a video high-speed reproduction apparatus. The figure which shows an example of the audio | voice waveform of audio | voice data. The figure which shows an example of the audio | voice power corresponding to an audio | voice waveform. The figure which shows an example of the variation | change_quantity of audio | voice power. The figure which shows an example of the audio | voice data which were reconfigure | reconstructed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Continuous media input part 2 ... Change amount parameter extraction part 3 ... Change amount parameter analysis / reproduction | regeneration frame selection part 4 ... Play speed instruction | indication part 5 ... Continuous media reconstruction part 6 ... Continuous media output part 21 ... Continuous media input part 22 ... parameter extraction unit 23 ... parameter synthesis unit 24 ... change parameter analysis / playback frame selection unit 25 ... playback speed instruction unit 26 ... continuous media reconstruction unit 27 ... continuous media output unit 31 ... video input unit 32 ... video / audio separation Unit 33 ... Video parameter extraction unit 34 ... Audio parameter extraction unit 35 ... Parameter synthesis unit 36 ... Change amount parameter analysis / reproduction frame selection unit 37 ... Playback speed instruction unit 38 ... Video reconstruction unit 39 ... Audio reconstruction unit 40 ... Video・ Speech synthesis unit 41 ... Video output unit

Claims (12)

A continuous media data high-speed playback method for shortening and playing back continuous media data,
A process of dividing continuous media data into frame sections and calculating a change amount of a feature parameter in each frame section;
The process of reconstructing data by concatenating only frames with large feature parameter variations,
And a step of reproducing the concatenated reconstructed data. A method of rapidly reproducing continuous media data.   2. The continuous media data high-speed playback method according to claim 1, wherein the continuous media data is audio data or moving image data. A composite media data high-speed playback method for shortening and playing back composite media data composed of a plurality of continuous media data,
A process of dividing each continuous media data into frame sections and calculating a change amount of a characteristic parameter of each frame section;
The process of calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of each continuous media data,
A process of generating reconstructed data for each continuous media data by concatenating only the frames with a large amount of change of the integrated feature parameters,
A method of reproducing the reconstructed data; and a method of reproducing the composite media data at a high speed. A multi-channel continuous media data high-speed playback method that shortens and plays back multi-channel continuous media data,
A process of dividing continuous media data of each channel into frame sections and calculating a change amount of a feature parameter of each frame section;
The process of calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of the continuous media data of each channel,
A process of generating reconstructed data for each channel by concatenating only the frames with a large amount of change of the integrated feature parameters,
And a step of reproducing the reconstructed data, and a multi-channel continuous media data high-speed reproduction method. A video data high-speed playback method for shortening and playing back video data consisting of video data and audio data,
Dividing video data into video data and audio data;
The process of dividing the video data into frame sections and calculating the amount of change in the feature parameter in each frame section;
Dividing audio data into frame sections and calculating the amount of change in feature parameters in each frame section;
Calculating the amount of change of the feature parameter of the video data from the amount of change of the feature parameter of the video data and the amount of change of the feature parameter of the audio data;
A process of generating reconstructed video data by concatenating only video data frames having a large amount of change in the feature parameter of the video data,
A process of generating reconstructed audio data by concatenating only frames of audio data having a large amount of change in the feature parameter of the video data;
A process of synthesizing the reconstructed video data and the reconstructed audio data to generate reconstructed video data,
And a step of playing back the generated reconstructed video data. A continuous media data high-speed playback device that shortens and plays back continuous media data,
Means for dividing continuous media data into frame sections, and calculating a change amount of a characteristic parameter of each frame section;
Means for reconstructing data by concatenating only frames with a large amount of change in feature parameters;
A continuous media data high-speed playback device comprising: means for playing back the reconstructed data linked together.   7. The continuous media data high-speed playback device according to claim 6, wherein the continuous media data is audio data or moving image data. A composite media data high-speed playback device that shortens and plays back composite media data composed of a plurality of continuous media data,
Means for dividing each continuous media data into frame sections and calculating a change amount of a characteristic parameter of each frame section;
Means for calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of each continuous media data;
Means for generating reconstructed data for each continuous media data by concatenating only frames with a large amount of change in the integrated feature parameters;
Means for reproducing the reconstructed data, and a high speed playback apparatus for composite media data. A multi-channel continuous media data high-speed playback device that shortens and plays back multi-channel continuous media data,
Means for dividing continuous media data of each channel into frame sections, and calculating a change amount of a characteristic parameter of each frame section;
Means for calculating the amount of change of the integrated feature parameter from the amount of change of the feature parameter of the continuous media data of each channel;
Means for generating reconstructed data for each channel by concatenating only frames with a large amount of change in the integrated feature parameters;
Means for reproducing the reconstructed data, and a multi-channel continuous media data high-speed reproducing device. A video data high-speed playback device that shortens and plays back video data consisting of video data and audio data,
Means for dividing video data into video data and audio data;
Means for dividing the moving image data into frame sections and calculating the amount of change in the feature parameter of each frame section;
Means for dividing the audio data into frame sections and calculating the amount of change in the feature parameter of each frame section;
Means for calculating the change amount of the feature parameter of the video data from the change amount of the feature parameter of the video data and the change amount of the feature parameter of the audio data;
Means for generating reconstructed video data by concatenating only frames of video data having a large amount of change in the characteristic parameter of the video data;
Means for generating reconstructed audio data by concatenating only frames of audio data having a large amount of change in the characteristic parameter of the video data;
Means for synthesizing the reconstructed video data and the reconstructed audio data to generate reconstructed video data;
And a means for reproducing the generated reconstructed video data. A program according to any one of claims 1 to 10, wherein the method or apparatus according to any one of claims 1 to 10 is written in a computer program so as to be executable. A recording medium having recorded thereon a program in which the method or apparatus according to any one of claims 1 to 10 is executable by a computer.

Images