JP3124239B2 - Video information detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a video information detecting apparatus, and more particularly, to detecting a change frame of a video scene used in a system for processing video information composed of, for example, moving images and audio. The present invention relates to a video information detection device.

[0002]

2. Description of the Related Art There is provided a moving image detection system in which a user detects a video of a predetermined scene from a medium storing video information composed of a moving image and audio.

As a conventional method of detecting this kind of video change point, that is, a scene change frame, for example, "Study on Moving Image Database Handling" (IEICE Technical Report, IE89-33, pp. 49-56, 198)
There is one described in 9).

Here, a histogram of luminance is extracted for each frame of a moving image to be processed, the sum of histogram differences from the immediately preceding frame is calculated, and it is determined that there is a change when the difference is large. The frame is changed.

FIG. 13 is a block diagram showing a configuration of a moving image detecting device for detecting a scene change frame, and FIG. 14 is a block diagram showing a configuration of a whole image change calculating unit.

In FIG. 13, reference numeral 11 denotes a moving image data storage unit, 12 denotes an entire image change calculating unit for calculating an entire image change, and 13 denotes a scene change detecting unit.

The moving image data storage unit 11 is constituted by, for example, a hard disk, and has a moving picture data (MPEG).
An encoded signal is stored by an encoding method such as Expert Group).

The image data 14 read from the moving image data storage unit 11 is input to an entire image change calculation unit 12, and the entire image change amount 15 calculated by the entire image change calculation unit 12 is calculated by a scene change detection unit. 13 is input.

In FIG. 14, the entire image change calculator 12 includes a luminance histogram calculator 17 for calculating a luminance histogram of moving image data and a luminance histogram difference calculator 18.

The luminance histogram 19 calculated by the luminance histogram calculator 17 is input to a luminance histogram difference calculator 18.

In the above configuration, the whole image change calculating section 12 stores a certain one of the moving image data from the moving image data storage section 11 which stores the image data of each frame in the moving image sequence.
Image data for a frame is input, a luminance histogram 19 of the frame is calculated by a luminance histogram calculator 17, and a luminance histogram difference calculator 18 calculates in advance the image data of the frame input immediately before the frame. The sum of the difference values with the calculated luminance histogram is calculated. Brightness histogram difference calculator 18
The sum of the difference values with the luminance histogram calculated by
It is output to the scene change detection unit 13 as the entire image change amount 15.

Here, the luminance value Y of the image data of each frame stored in the moving image data storage unit 11 is represented by Y (x, y, t), where 0 ≦ x <xw (x is a horizontal pixel position, xw Is the number of pixels in the horizontal direction. 0 ≦ y <yw (y is the position of the vertical pixel, yw is the number of pixels in the vertical direction), t is the frame number, and the luminance value Y is, for example, 0 ≦ Y (x, y, t) <256. And take the value of.

The luminance histogram is represented by yhist (i, t), where 0 ≦ i <256 (i is the luminance value), where t is a frame number, and yhist (i, t) is the luminance of the image data indicated by the frame number t. The number of pixels whose value is i.

The sum of the difference values with the immediately preceding frame luminance histogram represented as the entire image change amount 15 is dyhist
Assuming that (t), dyhist (t) is represented by Expression (1) shown in Expression 1.

[0015]

(Equation 1)

The scene change detecting section 13 includes the dyhis
The value of t (t) is compared with a predetermined threshold value. If the value of t (t) is larger than the predetermined threshold value, it is determined that the data variation between frames is large, that is, that the video content has changed. Output.

The advantage of using the above-mentioned luminance histogram is that only the entire fluctuation can be stably detected without being affected by the minute fluctuation of the image.

[0018]

However, such a conventional moving image detection system has a problem that both the accuracy of detecting a scene change frame and the amount of calculation are insufficient for use in a video processing system. was there. In other words, if an attempt is made to detect a scene change only from image information with sufficient accuracy, the amount of calculation becomes complicated and enormous, and a change point of image information alone cannot detect a meaningful change point. For example, in some cases, even if the amount of change obtained from image information is relatively large, the change is not so large in terms of video, or vice versa.

An object of the present invention is to provide a video information detecting apparatus which can reduce the probability of erroneously detecting a scene change frame and improve the scene change detection performance with a small amount of audio processing operation. I do.

[0020]

According to the present invention, there is provided a video information detecting apparatus for storing video data for a video scene composed of video data and audio data synchronized in time series with each other. Storage means, audio data storage means for storing audio data, image change frame calculation means for calculating a change point with respect to the time series of moving image data as an image change frame, and a change point with respect to the time series of audio data as audio data. A video information detecting device comprising: an audio change frame calculating means for calculating a change frame, and a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame. The calculating means includes the audio
Average signal level value for a given number of samples of
Is configured to calculate a frame in which the sum of the band outputs of the band-pass filter is continuously smaller than a predetermined threshold for a predetermined section.

A video information detecting apparatus according to the present invention comprises: a video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series; , An image change frame calculating means for calculating a change point with respect to a time series of moving image data as an image change frame, and an audio for calculating a change point with respect to the time series of audio data as an audio change frame. A video information detecting device comprising: a change frame calculating means; and a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame. Time series of the frequency spectrum of The amount of change is adapted to calculate a larger frame than a predetermined threshold value.

A video information detecting device according to the present invention comprises: a video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other; , An image change frame calculating means for calculating a change point with respect to a time series of moving image data as an image change frame, and an audio for calculating a change point with respect to the time series of audio data as an audio change frame. A video information detecting device comprising: a change frame calculating means; and a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame. Depending on the nature of the audio Frame in which the signal level of the data is continuously smaller than a predetermined threshold for a predetermined section, and a frame in which the amount of change in the time series of the frequency spectrum of the audio data is larger than the predetermined threshold is adaptively selected and calculated. Constitute.

A video information detecting apparatus according to the present invention comprises: a video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other; , An image change frame calculating means for calculating a change point with respect to a time series of moving image data as an image change frame, and an audio for calculating a change point with respect to the time series of audio data as an audio change frame. A video scene change frame calculating unit configured to calculate a video scene change frame based on the image change frame and the audio change frame, wherein the image change frame calculation unit includes an image change frame; as a frame, one frame of video data And the image data of one frame of the moving image data is locally changed with respect to the image data of the frame adjacent to the frame. It is configured to calculate a frame that changes dynamically.

A video information detecting apparatus according to the present invention comprises: a video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other; , An image change frame calculating means for calculating a change point with respect to a time series of moving image data as an image change frame, and an audio for calculating a change point with respect to the time series of audio data as an audio change frame. A video scene change frame calculating means for calculating a video scene change frame based on an image change frame and an audio change frame, wherein the video scene change frame calculation means comprises: Calculated as a change frame and Is configured to calculate all frames calculated as frames that change entirely and frames that are appropriately selected from frames calculated as audio change frames and frames calculated as images where the image changes locally. I do.

A video information detecting device according to the present invention comprises: a video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series; , An image change frame calculating means for calculating a change point with respect to a time series of moving image data as an image change frame, and an audio for calculating a change point with respect to the time series of audio data as an audio change frame. A video scene change frame calculating means for calculating a video scene change frame based on an image change frame and an audio change frame, wherein the video scene change frame calculation means comprises: Audio change frames using data After calculating the calculated audio change frame as the video scene change frame, the image change frame calculation unit calculates a section in the video scene where it is necessary to further calculate the video scene change frame. An image change frame is calculated, and a frame calculated as an image change frame in a section is calculated as a video scene change frame.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A video information detecting apparatus according to the present invention is a video information system using a high-efficiency coding method, which is a device for storing and reproducing video, for example, MP
The present invention can be applied to a case where a signal in which a video / audio signal is highly efficient, such as EG2, is stored in a disk.

FIG. 1 is a block diagram showing the overall configuration of the video information detecting device according to the first embodiment of the present invention. In the description of the video information detecting apparatus according to the present embodiment, the same components and the same signal portions as those of the moving image detecting system shown in FIGS. 13 and 14 are denoted by the same reference numerals.

In FIG. 1, the video information detecting device 20 comprises:
A moving image data storage unit 11 (moving image data storage means) for storing image data of each frame in the moving image sequence,
Image entire change calculation unit 12 (image change frame calculation means) for calculating the entire image change, audio data storage unit 21 (audio data storage means) for storing audio data, and silence detection unit for detecting silence information based on the audio data 22 (audio change frame calculating means), and a scene change detecting section 23 (video scene change frame calculating means) for detecting a scene change based on the entire image change amount and the silence information.

This embodiment shows a case where a signal in which a video / audio signal is converted into a high efficiency signal such as MPEG2 is stored on a disk. The moving image data storage unit 11 is an optical disk, a magneto-optical disk, a hard disk, or the like. (HD), or a digital VTR.

The image data 14 read from the moving image data storage unit 11 is input to an entire image change calculation unit 12, and the entire image change amount 15 calculated by the entire image change calculation unit 12 is calculated by a scene change detection unit. 23.

The audio data 24 read from the audio data storage unit 21 is input to the silence detecting unit 22 and the silence information 2 detected by the silence detecting unit 22 is stored.
5 is input to the scene change detection unit 23.

The image entire change calculation unit 12 performs the processing shown in FIG.
4 includes a luminance histogram calculator 17 for calculating a luminance histogram of the moving image data and a luminance histogram difference calculator 18.

FIG. 2 is a block diagram showing the structure of the silence detecting section 22.

In FIG. 2, a silence detecting section 22 includes an audio frame forming section 111 for converting a plurality of continuous samples of audio data into a frame, and a power calculating section 1.
12 and a silence determination unit 113.

The audio data 24 read from the audio data storage unit 21 is input to the audio frame forming unit 111,
Are output to the power calculation unit 112. Power calculation unit 1
The power data 115 calculated by 12 is input to the silence determination unit 113, and the silence information 25 determined by the silence determination unit 113 is input to the scene change detection unit 23.

Next, the operation of the video information detecting device 20 configured as described above will be described.

The operation of the entire image change calculator 12 is the same as that of the conventional example, and the dyhi expression (1) shown in the equation (1) is used.
st (t) is output to the scene change detection unit 23 as the entire image change amount 15.

On the other hand, the audio data 24 is inputted to the silence detecting section 22, and the silence detecting section 22 outputs the silence information 25 by interpreting the time when the input level decreases as a break of the contents of the video sequence.

The operation of the silence detector 22 will be described with reference to FIG.

The audio data 24 stored in the audio data storage section 21 has a resolution of 1 sample.
The 6-bit sample frequency is, for example, 45 kHz.
Basically, it suffices to satisfy the audio band (about 20 kHz).

The audio frame forming section 111 forms a plurality of continuous samples of the audio data 24 input in time series into a frame. The framed audio data 24 is a unit of audio processing.

The audio frame period can be arbitrarily set between several ms and several tens of ms. Here, for simplicity of explanation, the same 33 ms as a moving image, that is, one frame is composed of 1500 samples. explain. The frame period of the audio data is not limited to the numerical value of the present embodiment.

Assuming that the audio data 24 is os (tt) tt is a sample number, and that the audio frame data 114 is of (t) t is a frame number, of (t) is os (t * 1500) and os (t * 1500 + 1). ,…,
os (t * 1500 + 1499).

The power calculator 112 sets the input level of each audio frame data 114 as pow (t),
It is calculated according to equation (2) shown in equation 2.

[0045]

(Equation 2)

The audio data 24, ie, os
(Tt) is input without including a DC component. Also, pow (t) is output to the silence determination unit 113 as power data 115.

The silence determination unit 113 compares the power data 115 of each audio frame with a predetermined threshold value,
When frames smaller than the predetermined threshold value are continuous for a predetermined number of frames (on the order of several hundred ms), these frame numbers are output as silence information 25. The frame number indicated by the silence information 25 has a high possibility that the audio signal is interrupted, that is, the frame number indicates a break even in the video content.

The scene change detecting section 23 detects a scene change based on the entire image change amount 15, that is, dyhist (t) and the silence information 25, and when dyhist (t) is smaller than a predetermined threshold value and the frame number When “t” is included in the silent information 25, it is output as a scene change frame 26.

As described above, the video information detecting device 20 according to the first embodiment includes a moving image data storage unit 1 for storing image data of each frame in a moving image sequence.
1. an entire image change calculator 12 for calculating an entire image change;
Audio data storage unit 2 for storing audio data
1, includes a silence detector 22, scene change detecting section 23 for detecting a scene change based on the entire image variation amount and the silence information detected silence information based on audio data, the silence detector 22, a predetermined audio data Average number of samples signal
Level value or each band output of band pass filter
Is calculated as a frame in which both the image data and the audio data level change frames are determined to be the change frames of the video scene. The probability of detecting a scene change frame is reduced, and the scene change detection performance can be improved with a small amount of computation for audio processing.

[0050] Particularly, in this embodiment, only volume rather than a predetermined value or more frames, so that the scene change frame meaningful, the silence detector 22, O
Average signal level value for a predetermined number of audio data samples,
Alternatively, the sum of each band output of the band-pass filter is
Since a frame that is smaller than the predetermined threshold is calculated continuously for a predetermined fixed section, it is possible to prevent a scene change frame from being erroneously detected. In other words, in a mode in which the volume is simply a frame of a predetermined value or more,
For example, in the scene switching frame, only a loud voice or a frame with a large BGM volume is detected, but in the present embodiment, silence information corresponding to a break in the sound sequence generated is detected by scene change detection. Therefore, accurate scene change detection can be performed.

The method of calculating the power data 115 described above is not limited to the method described in the present embodiment, but may be any method that represents the audio signal level in the corresponding frame. For example, band data of a plurality of frequency banks may be applied to the audio data of each frame, and the sum of the band pass filter outputs of the frequency banks may be realized as power data.

FIG. 3 is a block diagram showing the overall configuration of a video information detecting device according to the second embodiment of the present invention. In the description of the video information detecting device according to the present embodiment, the same components and the same signal portions as those of the moving image detecting device shown in FIGS. 1 and 2 are denoted by the same reference numerals, and the description of the overlapping portions will be omitted.

In FIG. 3, the video information detecting device 30 comprises:
A moving image data storage unit 11 that stores image data of each frame in the moving image sequence, an entire image change calculation unit 12 that calculates an entire image change, an audio data storage unit 21 that stores audio data, and a spectrum based on the audio data. Spectrum change detection unit 31 for detecting change information
(Audio change frame calculating means) and a scene change detecting section 32 (video scene change frame calculating means) for detecting a scene change based on the entire image change amount and the spectrum change information.

The image data 14 read from the moving image data storage unit 11 is input to the entire image change calculation unit 12, and the entire image change amount 15 calculated by the entire image change calculation unit 12 is calculated by the scene change detection unit. 32.

The audio data 24 read from the audio data storage unit 21 is input to a spectrum change detection unit 31, and the spectrum change information 33 detected by the spectrum change detection unit 31 is transmitted to the scene change detection unit 32. Is entered.

FIG. 4 is a block diagram showing the configuration of the spectrum change detecting section 31.

In FIG. 4, the spectrum change detecting section 31
Is composed of an audio frame composing unit 111, a frequency spectrum calculating unit 211, and a spectrum change judging unit 212, which collectively frame a plurality of continuous samples of audio data.

The audio data 24 read from the audio data storage unit 21 is input to the audio frame forming unit 111,
The audio frame data 114 output from is input to the frequency spectrum calculation unit 211. The frequency spectrum data 213 calculated by the frequency spectrum calculation unit 211 is input to the spectrum change determination unit 212, and the spectrum change information 33 determined by the spectrum change determination unit 212 is input to the scene change detection unit 32. .

Next, the operation of the video information detecting device 30 configured as described above will be described.

The operation of the entire image change calculator 12 is the same as that described above, and the dyhis shown in the equation (1) shown in the above equation (1).
t (t) is output to the scene change detection unit 32 as the entire image change amount 15.

On the other hand, the audio data 24 is input to the spectrum change detecting section 31, and the spectrum change detecting section 31 interprets the time at which the input frequency spectrum changes as a break of the contents of the video sequence and converts the spectrum change information 33. Output.

Referring to FIG. 4, spectrum change detecting section 31
Will be described.

The operation of the audio frame composing section 111 is the same as the operation of the first embodiment.

The frequency spectrum calculator 211 receives the audio frame data 114 and performs a band pass filter calculation for a plurality of frequency banks.

The center frequencies of the frequency banks are logarithmically spaced, for example, 10, 20, 50, 100, 200, 500, 100
There are 11 bands of 0, 2000, 5000, 10000 and 20000 Hz.

The center frequency is set at logarithmic intervals, and is determined so as to cover the entire audio band. The number of bands is 1
It should be about 0 to 30.

In the above embodiment, 11 bands are taken as an example, but the number of bands is not limited to the numerical value of this embodiment.

The frequency spectrum at each audio frame number t is represented by bpf (w, t). w is the band number and the order in which the center frequency is small,
In this embodiment, 0 ≦ w <11.

The spectrum change determination unit 212 compares the frequency spectrum data 213 of each audio frame with that of the vicinity of the immediately preceding frame according to the following equation (3).

Dbpf (w, t, n) = | bpf (w, t) −bpf (w, t−n) | (3) Here, for example, 1 ≦ n ≦ 5 and dbpf (w, t) , N), the number of times satisfying dbpf (w, t, 1)> THL1 (THL1 is a predetermined threshold) is larger than a predetermined number of bands, and dbpf (w, t−1, nn)> THL2 (nn) = 1
(4, THL2 is a predetermined threshold value) is less than the predetermined threshold value, that is, when it is determined that the frequency spectrum variation has started from the frame, the frame is output as the spectrum change information 33.

The scene change detecting section 32 determines that dyhist (t) is smaller than a predetermined threshold value and that the frame number t is smaller than the predetermined threshold value, based on the input entire image change amount 15, that is, dyhist (t) and the spectrum change information 33. When it is included in the change information 33, it is output as the scene change frame 26.

As described above, the video information detecting device 30 according to the second embodiment includes a moving image data storage unit 1 for storing image data of each frame in a moving image sequence.
1. an entire image change calculator 12 for calculating an entire image change;
Audio data storage unit 2 for storing audio data
1. A spectrum change detection unit 31 that detects spectrum change information based on audio data, and a scene change detection unit 32 that detects a scene change based on the entire image change amount and the spectrum change information, A frame in which the change amount of the frequency spectrum of the audio data with respect to the time series is larger than a predetermined threshold is calculated, and the change frames of both the image data and the audio data frequency spectrum which is information that accurately indicates the state of the audio data are calculated. Since the change frame is determined, the probability of erroneously detecting the scene change frame is reduced, and the scene change detection performance can be improved.

In the present embodiment, a method of detecting a change frame in audio data by using frequency spectrum analysis has been described. However, the present invention is not limited to the frequency spectrum. It is also possible to use an analysis method that can extract the properties of audio data.

FIG. 5 is a block diagram showing the overall configuration of the video information detecting device according to the third embodiment of the present invention. In the description of the video information detecting device according to the present embodiment, the same components and the same signal portions as those of the moving image detecting device shown in FIGS.

In FIG. 5, the video information detecting device 40 comprises:
A moving image data storage unit 11 that stores image data of each frame in the moving image sequence, an entire image change calculation unit 12 that calculates an entire image change, an audio data storage unit 21 that stores audio data, and audio data of each frame. A voice determination unit 41 for determining whether the signal is a voice signal or a signal other than voice from the state of the frequency spectrum, a silence detection unit 22 for detecting silence information based on audio data, and a spectrum change detecting spectrum change information based on the audio data. The detection unit 31 includes a scene change detection unit 42 that detects a scene change based on the entire image change amount, silence information, and spectrum change information.

The image data 14 read from the moving image data storage unit 11 is input to the entire image change calculation unit 12, and the entire image change amount 15 calculated by the entire image change calculation unit 12 is used as a scene change detection unit. 42.

The audio data 24 read from the audio data storage unit 21 is stored in the audio determination unit 41,
The signals are input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. On the other hand, the voice determination result 43 determined by the voice determination unit 41 is input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. The silence information 25 detected by the silence detection unit 22 and the spectrum change information 33 detected by the spectrum change detection unit 31
Is input to the scene change detection unit 42.

FIG. 6 is a block diagram showing the structure of the voice judging section 41. The spectrum change detecting section 31 shown in FIG.
The same components as those described above are denoted by the same reference numerals.

In FIG. 6, the audio determination unit 41 is composed of an audio frame construction unit 111 for converting a plurality of continuous samples of audio data into a frame, a frequency spectrum calculation unit 211, and an audio band level comparison unit 301.

The audio data 24 read from the audio data storage unit 21 is input to the audio frame forming unit 111,
The audio frame data 114 output from is input to the frequency spectrum calculation unit 211. The frequency spectrum data 213 calculated by the frequency spectrum calculation unit 211 is input to the audio band level comparison unit 301, and the audio judgment result 43 determined by the audio band level comparison unit 301 is output to the silence detection unit 22 and the spectrum change Each is input to the detection unit 31.

Next, the operation of the video information detecting device 40 configured as described above will be described.

The operation of the entire image change calculation unit 12 is the same as described above, and the dyhis
t (t) is output to the scene change detection unit 42 as the entire image change amount 15.

The audio determination section 41 determines whether the audio data of each frame is an audio signal or a non-audio, for example, background music (BGM) signal from the state of the frequency spectrum.

In FIG. 6, the processing contents of the audio frame forming unit 111 and the frequency spectrum calculating unit 211 are as follows.
This is the same as the second embodiment.

The audio band level comparing section 301 compares the value of the frequency spectrum data 215 in the audio signal band (from about 200 Hz to about 6000 Ηz) with the value of the frequency spectrum data 215 outside the audio signal band. When the value is larger than the predetermined threshold value, “1” is output. When the value is smaller than the predetermined threshold value, “0” is output as the voice determination result 43.

When the voice judgment result 43 is 1, the silence detecting section 22 performs the same operation as in the first embodiment and detects the scene change as the silence information 25 using the frame number of the frame whose level is smaller than the predetermined threshold. When the voice judgment result 43 input is 0, the operation is stopped and the frame number is not output as the silence information 25.

When the voice judgment result 43 input is 0, the spectrum change detection unit 31 performs the same operation as in the second embodiment and sets the frame number of the frame whose spectrum change is larger than the predetermined threshold value as the spectrum change information 33. When the input is output to the scene change detection unit 42 and the sound determination result 43 input is 1, the operation is stopped, and the frame number is not output as the spectrum change information 33.

That is, when the level fluctuation such as voice is large, the silence information 25 is used, and when the probability that the level fluctuation is not a change frame like BGM is not high, the spectrum change information 33 is adaptively applied. By using
Improve audio change frame detection accuracy.

The scene change detection unit 42 calculates the dyhist (t), the silence information 15 and the spectrum change information 33 from the input image entire change amount 15,
(T) is smaller than a predetermined threshold value and the frame number t
Is included in the silence information 25 or the spectrum change information 33, it is output as the scene change frame 26.

As described above, the video information detecting device 40 according to the third embodiment is
A moving image data storage unit 11 that stores image data of each frame in the moving image sequence, an entire image change calculation unit 12 that calculates an entire image change, an audio data storage unit 21 that stores audio data, and audio data of each frame. A voice determination unit 41 for determining whether the signal is a voice signal or a signal other than voice from the state of the frequency spectrum, a silence detection unit 22 for detecting silence information based on audio data, and a spectrum change detecting spectrum change information based on the audio data. A detection unit 31 includes a scene change detection unit 42 that detects a scene change based on the entire image change amount, silence information, and spectrum change information, and includes level information and audio information that accurately indicate the state of image data and audio data. Using the data frequency spectrum adaptively, change frames and Since the way, it is possible to improve the scene change detection performance.

FIG. 7 is a block diagram showing the overall configuration of a video information detecting device according to the fourth embodiment of the present invention. In the description of the video information detecting device according to the present embodiment, the same components and the same signal portions as those of the moving image detecting device shown in FIGS. 1 to 6 are denoted by the same reference numerals, and the description of the overlapping portions will be omitted.

In FIG. 7, the video information detecting device 50 comprises:
A moving image data storage unit 11 that stores image data of each frame in the moving image sequence, an entire image change calculation unit 12 that calculates an entire image change, an audio data storage unit 21 that stores audio data, and an audio data storage unit 21 that stores audio data of each frame. A level fluctuation determining unit 51 that calculates the magnitude of the level fluctuation before and after the frame and determines the level fluctuation in a form reflecting the result, a silence detecting unit 22 that detects silence information based on the audio data, Spectrum change detection unit 3 for detecting spectrum change information
1. It is composed of a scene change detecting section 52 for detecting a scene change based on the entire image change amount, silence information and spectrum change information.

The image data 14 read from the moving image data storage unit 11 is input to the entire image change calculation unit 12, and the entire image change amount 15 calculated by the entire image change calculation unit 12 is calculated by the scene change detection unit. 52 is input.

The audio data 24 read from the audio data storage unit 21 is input to the level fluctuation determination unit 51, the silence detection unit 22, and the spectrum change detection unit 31, respectively. On the other hand, the level change determination result 53 determined by the level change determination unit 51 is
2 and the spectrum change detection unit 31. Then, the silence information 25 detected by the silence detection unit 22 and the spectrum change information 33 detected by the spectrum change detection unit 31 are input to the scene change detection unit 52.

FIG. 8 is a block diagram showing the structure of the level fluctuation judging section 51. The same components as those of the silence detecting section 22 of FIG. 2 are denoted by the same reference numerals.

In FIG. 8, the level fluctuation determining section 51
It is composed of an audio frame forming unit 111, a power calculating unit 112, and a power change amount calculating unit 401, which collectively frame a plurality of continuous samples of audio data.

The audio data 24 read from the audio data storage unit 21 is input to the audio frame forming unit 111,
Are output to the power calculation unit 112. Power calculation unit 1
The power data 115 calculated by 12 is input to the power change amount calculation unit 401, and the power change amount calculation unit 40
The level fluctuation determination result 53 calculated in step 1 is input to the silence detection unit 22 and the spectrum change detection unit 31, respectively.

Next, the operation of the video information detecting device 50 configured as described above will be described.

The operation of the entire image change calculation unit 12 is the same as that described above, and the dyhis
t (t) is output to the scene change detection unit 42 as the entire image change amount 15.

The level change judging section 51 calculates the level change level of the audio data of each frame before and after the frame, and outputs a level change judgment result 53 in a form reflecting the result.

In FIG. 8, the processing contents of the audio frame forming unit 111 and the power calculating unit 112 are the same as those in the first embodiment.

The power variation calculator 401 outputs “1” as the level variation determination result 53 when the following equation (4) shown in Expression 3 is satisfied, and outputs “0” when the following equation (4) is not satisfied.

[0103]

(Equation 3)

When the level change determination result 53 input is 1, the silence detecting section 22 performs the same operation as in the first embodiment, and sets the frame number of the frame whose level is smaller than the predetermined threshold as silence information 25 as the scene change information 25. When the level change determination result 53 input is 0, the operation is stopped and the frame number is not output as the silence information 25.

When the level change determination result 53 input is 0, the spectrum change detection section 31 performs the same operation as in the second embodiment to change the frame number of the frame whose spectrum change is larger than the predetermined threshold value into the spectrum change information 33. Is output to the scene change detection unit 52. When the level change determination result 53 input is 1, the operation is stopped, and the frame number is not output as the spectrum change information 33.

That is, if the level fluctuation before and after is large, the silence information 25 is used. If the level fluctuation is small, the spectrum change information 3 is used.
3 is used to improve the accuracy of audio change frame detection.

The scene change detecting section 52 obtains dyhis from the input entire image change amount 15, ie, dyhist (t), silence information 25 and spectrum change information 33.
When t (t) is smaller than a predetermined threshold value and the frame number t is included in the silence information 25 or the spectrum change information 33, it is output as a scene change frame 26.

As described above, the video information detection device 50 according to the fourth embodiment stores the video data storage unit 1 for storing the image data of each frame in the video sequence.
1. an entire image change calculator 12 for calculating an entire image change;
Audio data storage unit 2 for storing audio data
1. A level fluctuation determining unit 51 that calculates the level fluctuation level of the audio data of each frame before and after the frame and determines the level fluctuation in a form reflecting the result, and silence detecting silence information based on the audio data. Detector 2
2. It has a spectrum change detection unit 31 for detecting spectrum change information based on audio data, a scene change detection unit 52 for detecting a scene change based on the entire image change amount, silence information, and the spectrum change information. Since the level information and the audio data frequency spectrum, which accurately indicate the state of the audio data, are adaptively used to determine the change frame of the video scene,
This can be realized by simple processing, and the scene change detection performance can be improved and the cost can be reduced.

FIG. 9 is a block diagram showing the overall configuration of a video information detecting device according to the fifth embodiment of the present invention. In the description of the video information detecting device according to the present embodiment, the same components and the same signal portions as those of the moving image detecting device shown in FIGS. 1 to 6 are denoted by the same reference numerals, and the description of the overlapping portions will be omitted.

In FIG. 9, the video information detecting device 60 comprises:
A moving image data storage unit 11 that stores image data of each frame in the moving image sequence, an entire image change calculation unit 12 that calculates an entire image change, and comparing the image data of each frame with the image data of an adjacent frame, An image local change calculation unit 61 for calculating a local change amount of an image; an audio data storage unit 21 for storing audio data; and determining whether audio data of each frame is an audio signal or a signal other than audio from a state of a frequency spectrum. A voice determination unit 41, a silence detection unit 22 that detects silence information based on audio data, a spectrum change detection unit 31 that detects spectrum change information based on audio data, an entire image change amount, an image local change amount, silence information, It comprises a scene change detecting section 62 for detecting a scene change based on the spectrum change information.

The image data 14 read from the moving image data storage unit 11 is input to the entire image change calculation unit 12 and the local image change calculation unit 61, respectively. 15, the image local change amount 63 calculated by the image local change calculation unit 61
Is input to the scene change detection unit 62.

The audio data 24 read from the audio data storage unit 21 is stored in the audio determination unit 41,
The signals are input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. On the other hand, the voice determination result 43 determined by the voice determination unit 41 is input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. The silence information 25 detected by the silence detection unit 22 and the spectrum change information 33 detected by the spectrum change detection unit 31
Is input to the scene change detection unit 62.

FIG. 10 is a block diagram showing the structure of the image local change calculation section 61.

In FIG. 10, the image local change calculator 61
Is composed of an inter-frame difference calculator 511 and a local change amount calculator 512.

The image data 14 is input to the inter-frame difference calculator 511, and the inter-frame difference data 513 calculated by the inter-frame difference calculator 511 is input to the local change amount calculator 512.

Next, the operation of the video information detecting device 60 configured as described above will be described.

The operation of the entire image change calculation unit 12 is the same as that described above, and the dyhis expression shown in the above equation (1) is used.
t (t) is output to the scene change detection unit 62 as the entire image change amount 15.

Further, the voice judging section 41, the silence detecting section 22,
The operation of the spectrum change detection unit 31 is the same as the operation of the third embodiment.

The image local change calculation section 61 calculates the local change amount of the image by comparing the image data of each frame with the image data of the adjacent frame.

More specifically, the inter-frame difference calculator 511
Calculates the following equation (5), and calculates the inter-frame difference data 513.
Is output to the local change amount calculation unit 512.

Dy (x, y, t) = Y (x, y, t) −Y (x, y, t−1) (5) In the local change amount calculation unit 512, | dy (x, y, t) t) | <THL SMALL (predetermined threshold) (6) The number of pixels satisfying the following expression is larger than the predetermined threshold, and | dy (x, y, t) |> THL BIG (predetermined threshold) (7) When the sum of the absolute values of the difference data in the pixels satisfying the above expression (7) is larger than a predetermined threshold value (in other words,
The change in the screen is local, but the change is large, that is, when it is centered in meaning but small,
When any of the above three conditions is not satisfied, the difference data absolute value sum value is output as 0 as the image local change amount 63.

The scene change detecting section 62 receives the image local change amount 63, the entire image change amount 15, dyhist (t), the silence information 25 and the spectrum change information 33,
(A) dyhist (t) is smaller than a predetermined threshold, and the frame number t is included in the silence information 25 or the spectrum change information 33.

(B) The image local change amount 63 is larger than a predetermined threshold value, and the frame number t is included in the silence information 25 or the spectrum change information 33.

When the above condition (a) or (b) is satisfied, the scene change frame 26 is output.

Here, (a) shows a scene change due to a total change of an image, and (b) shows a condition by a combination of a local change of an image and a change of audio.

As described above, the video information detecting device 60 according to the fifth embodiment includes a moving image data storage unit 1 for storing image data of each frame in a moving image sequence.
1. an entire image change calculator 12 for calculating an entire image change;
By comparing the image data of each frame with the image data of an adjacent frame, an image local change calculation unit 61 for calculating a local change amount of the image, an audio data storage unit 21 for storing audio data, and an audio data for each frame are stored. A voice determination unit 41 for determining whether a signal is a voice signal or a signal other than voice from the state of a frequency spectrum, a silence detection unit 22 for detecting silence information based on audio data, and a spectrum change detecting spectrum change information based on audio data. A detection unit 31, an entire image change amount, an image local change amount,
Comprising a scene change detection unit 62 for detecting a scene change based on silence information and spectral change information, the image change frame
As an example, a frame in which the image data of one frame of the moving image data completely changes with respect to the image data of the frame adjacent to the frame, and an image data of one frame of the moving image data which is adjacent to the frame Frame that changes locally with respect to the image data of the
Since it is configured to calculate, it is possible to detect not only obvious changes from the appearance of the image but also large changes that are not necessarily obvious from the appearance but are ambiguous by using the audio data. And high scene detection performance.

FIG. 11 is a block diagram showing the overall configuration of a video information detecting device according to the sixth embodiment of the present invention. In the description of the video information detection apparatus according to the present embodiment, the same components and the same signal portions as those of the moving image detection apparatus shown in FIGS.

In FIG. 11, a video information detecting device 70
Is a moving image data storage unit 11 for storing image data of each frame in a moving image sequence, an entire image change calculation unit 12 for calculating an entire image change, and comparing image data of each frame with image data of an adjacent frame. , An image local change calculation unit 61 that calculates a local change amount of an image, and an audio data storage unit 2 that stores audio data.
1. An audio determination unit 41 that determines whether the audio data of each frame is an audio signal or a signal other than audio from the state of the frequency spectrum, a silence detection unit 22 that detects silence information based on the audio data, A spectrum change detection unit 31 for detecting the spectrum change information, a whole change scene detection unit 71 for detecting a scene change based on the entire image change amount, silence information and the spectrum change information, and an image local change amount and the whole change frame information. It comprises a scene change detecting section 72 for detecting a scene change.

The image data 14 read from the moving image data storage unit 11 is input to the entire image change calculator 12 and the local image change calculator 61, respectively, and the entire image change calculated by the entire image change calculator 12 is calculated. The quantity 15 is input to the overall change scene detection unit 71. The image local change amount 63 calculated by the image local change calculator 61 is input to the scene change detector 72.

The audio data 24 read from the audio data storage unit 21 is stored in the audio determination unit 41,
The signals are input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. On the other hand, the voice determination result 43 determined by the voice determination unit 41 is input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. The silence information 25 detected by the silence detection unit 22 and the spectrum change information 33 detected by the spectrum change detection unit 31
Is input to the entire scene change detection unit 71. further,
The entire change frame detected by the entire change scene detection unit 71 is input to the scene change detection unit 72 together with the image local change amount 63 calculated by the image local change calculation unit 61.

Next, the operation of the video information detecting device 70 configured as described above will be described.

The operation of the entire image change calculation unit 12 is the same as that described above, and the dyhis
t (t) is output to the entire scene change detecting unit 71 as the entire image change amount 15.

The operations of the voice determination unit 41, the silence detection unit 22, and the spectrum change detection unit 31 are the same as those of the third embodiment, and the operation of the image local change calculation unit 61 is the same as that of the fifth embodiment. The operation is the same.

The whole-change scene detecting section 71 uses the dyhist based on the inputted whole-image change amount 15, dyhist (t), silence information 25 and spectrum change information 33.
(T) is smaller than a predetermined threshold value and the frame number t
Is included in the silence information 25 or the spectrum change information 33, the frame number is output to the scene change detection unit 72 as the entire change frame 73.

The scene change detecting unit 72 receives the image local change amount 63 and the entire change frame 73, and (a) the entire change frame 73 includes a frame number.

(B) The image local change amount 63 is larger than a predetermined threshold value, and the frame number t is included in the silence information 25 or the spectrum change information 33.

When the above (a) or (b) is satisfied, the scene change frame 26 is output.

Here, (a) shows a scene change due to a total change of an image, and (b) shows a condition by a combination of a local change of an image and a change of audio. In this case, the threshold value described in (b) is changed according to the number of frames included in the entire change frame 73.

When the number of frames included in the whole change frame 73 is large, the threshold value is set to a relatively large value so that a frame satisfying the condition (b) is hardly detected. Conversely, when the number of frames included in the entire change frame 73 is small, the threshold value is set to a small value so that a frame satisfying the condition (b) is easily detected.

As a result, the number of frames finally detected as a scene change can be kept within a certain range.

As described above, the video information detecting device 70 according to the sixth embodiment includes a moving image data storage unit 1 for storing image data of each frame in a moving image sequence.
1. an entire image change calculator 12 for calculating an entire image change;
By comparing the image data of each frame with the image data of an adjacent frame, an image local change calculation unit 61 for calculating a local change amount of the image, an audio data storage unit 21 for storing audio data, and an audio data for each frame are stored. A voice determination unit 41 for determining whether the signal is a voice signal or a signal other than voice from the state of the frequency spectrum, a silence detection unit 22 for detecting silence information based on audio data, and a spectrum change detecting spectrum change information based on the audio data. A detecting unit 31, a whole scene change detecting unit 71 for detecting a scene change based on the whole image change amount, silence information and spectrum change information, and a scene change detecting for detecting a scene change based on the image local change amount and the whole change frame information Part 72
And all of the frames calculated as audio change frames and images calculated as frames where the image completely changes, and the frames calculated as audio change frames and images calculated as the locally change frame , A frame appropriately selected from the above is calculated, even under a situation where a request to make the number of scene change frames constant from the host system occurs (for example,
When it is desired to quickly view the digest of the video scene at the same time regardless of the variation of the video fluctuation), a scene change frame detection method that satisfies the same is realized, so that high-performance scene detection performance can be realized.

FIG. 12 is a block diagram showing the overall configuration of the video information detecting device according to the seventh embodiment of the present invention. In the description of the video information detecting device according to the present embodiment, the same components and the same signal portions as those of the moving image detecting device shown in FIGS. 1 to 5 are denoted by the same reference numerals, and the description of the overlapping portions will be omitted.

In FIG. 12, a video information detecting device 80
A moving image data storage unit 11 for storing image data of each frame in a moving image sequence, a screen change frame calculation unit 81 for calculating a screen change frame, an audio data storage unit 21 for storing audio data, and an audio data for each frame. A voice determination unit 41 that determines whether the data is a voice signal or a signal other than voice from the state of the frequency spectrum; a silence detection unit 2 that detects silence information based on the audio data
2. A spectrum change detection unit 31 for detecting spectrum change information based on audio data, an audio change frame detection unit 82 for detecting an audio change frame based on silence information and spectrum change information, and based on an image change frame and an audio change frame. And a scene change frame detecting section 83 for detecting a scene change frame.

The image data 14 read from the moving image data storage section 11 is input to a screen change frame calculation section 81, and the image change frame 84 calculated by the screen change frame calculation section 81 is converted into a scene change frame detection section. 83
Is input to

The audio data 24 read from the audio data storage unit 21 is stored in the audio determination unit 41,
The signals are input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. On the other hand, the voice determination result 43 determined by the voice determination unit 41 is input to the silence detection unit 22 and the spectrum change detection unit 31, respectively. The silence information 25 detected by the silence detection unit 22 and the spectrum change information 33 detected by the spectrum change detection unit 31
Is input to the audio change frame detection unit 82.
Further, the audio change frame 85 detected by the audio change frame detection unit 82 is input to the screen change frame calculation unit 81 and the scene change frame detection unit 83.

Next, the operation of the video information detecting device 80 configured as described above will be described.

The operations of the voice judging section 41, the silence detecting section 22, and the spectrum change detecting section 31 are the same as those of the third embodiment.

The audio change frame detecting section 82 receives the silence information 25 and the spectrum change information 33, and receives the frame number and the spectrum change information 3 contained in the silence information 25.
3 is changed to the audio change frame 8
5 is output to the image change frame detection unit 81 and the scene change frame detection unit 83.

The image change frame detecting section 81 performs the following operation.

(1) The minimum interval of a frame finally detected as a scene change frame is determined in advance.

(2) Audio change frame detecting section 82
When the frame interval of the audio change frame 85 input from the CPU is smaller than the predetermined frame interval, the processing for detecting a scene change from image data is not performed.

(3) Audio change frame detector 82
When the frame interval of the audio change frame 85 input from the third unit is larger than the predetermined frame interval, the third
A scene change is detected from the image data by the same method as in the embodiment, and the detected scene change frame number is set as the image change frame 84 and the scene change frame detecting unit 83
Output to

By the above operation, the scene change detection processing using a huge amount of moving image data (about 1000 times the audio data) can be suppressed to the minimum necessary.

The scene change frame detecting section 83 receives the image change frame 84 and the audio change frame 85 and inputs the frame number included in the audio change frame 85 and the frame number included in the image change frame 84 into the scene change frame 26. Output as

As described above, the video information detecting device 80 according to the seventh embodiment stores the video data storage unit 1 for storing the image data of each frame in the video sequence.
1. Screen change frame calculation section 81 for calculating a screen change frame, audio data storage section 21 for storing audio data, audio for determining whether the audio data of each frame is an audio signal or a signal other than audio from the state of the frequency spectrum. A determination unit 41, a silence detection unit 22 for detecting silence information based on audio data, and a spectrum change detection unit 3 for detecting spectrum change information based on audio data
1. an audio change frame detection unit 82 that detects an audio change frame based on silence information and spectrum change information, and a scene change frame detection unit 83 that detects a scene change frame based on an image change frame and an audio change frame Calculating an audio change frame using the data, calculating the calculated audio change frame as a video scene change frame, and then calculating a video scene change frame in the video scene. Since the image change frame is calculated and the frame calculated as the image change frame in the section is calculated as the video scene change frame, the scene change detection by the audio data that requires a small amount of calculation is performed in the first stage, A huge amount of moving image data Use scene change detection process can be suppressed to the minimum required, it is possible to perform scene change detection in the calculation amount is small, low-cost hardware as a whole.

In each of the above embodiments, for example, the MPE
The present invention can be applied to a video information detecting device using the G system, but can be applied to all devices that perform video / audio coding processing.

Further, the present invention is not limited to the name of the video information detecting device, but can be appropriately changed as in an encoding device or the like within the technical idea of the present invention. It goes without saying that the embodiment may be incorporated.

[0158]

In the image information detecting apparatus according to the present invention, the audio change frame calculation means, the audio data
Average signal level value for a given number of samples, or band
Since the sum of the output of each band of the pass filter is configured to continuously calculate a frame in which the sum is smaller than a predetermined threshold for a predetermined section, the probability of erroneously detecting a scene change frame is reduced and audio processing is reduced. The scene change detection performance can be improved with the amount of calculation.

In the video information detecting apparatus according to the present invention, the audio change frame calculating means is configured to calculate a frame in which the change amount of the frequency spectrum of the audio data with respect to the time series is larger than a predetermined threshold. The probability of detecting a change frame is reduced, and the scene change detection performance can be improved.

[0160] In the video information detecting apparatus according to the present invention, the audio change frame calculating means determines whether the signal level of the audio data is continuously lower than the predetermined threshold for a predetermined section depending on the nature of the audio data. Is configured to adaptively select and calculate a frame in which the amount of change in the frequency spectrum of the time series with respect to the time series is larger than a predetermined threshold value, so that the level information and the audio information that accurately indicate the state of the image data and audio data By determining the change frame of the video scene by using the data frequency spectrum adaptively, the scene change detection performance can be improved.

[0161] In the video information detecting apparatus according to the present invention, the image change frame calculating means determines that the image data of one frame of the moving image data is the same as that of the frame adjacent to the frame as the image change frame. Since the image data is calculated so as to calculate a frame that dynamically changes and a frame in which the image data of one frame of the moving image data locally changes with respect to the image data of the frame adjacent to the frame. It is possible to detect not only obvious changes from but also large changes that are not always obvious from the appearance but are ambiguous by using the audio data, thereby realizing high-performance scene detection performance.

In the video information detecting apparatus according to the present invention, the video scene change frame calculating means calculates all the frames calculated as audio change frames and the frames whose images are totally changed, and the audio change frames. And a frame which is appropriately selected from frames calculated as a frame where an image locally changes, so that a situation in which a higher system requests a fixed number of scene change frames occurs. However, since the scene change frame detection method that satisfies the above condition is adopted, high-performance scene detection performance can be realized.

In the video information detecting apparatus according to the present invention, the video scene change frame calculating means calculates an audio change frame using the audio data, and calculates the calculated audio change frame as a video scene change frame. , For a section in the video scene where it is necessary to further calculate a video scene change frame,
Since the image change frame is calculated by the image change frame calculation means and the frame calculated as the image change frame in the section is calculated as the video scene change frame, the scene change detection by audio data that requires a small amount of calculation can be performed. By performing the processing at the first stage, the scene change detection process using a huge amount of moving image data can be suppressed to the minimum necessary, and the scene change can be detected with low-cost hardware with a small amount of calculation as a whole.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a video information detection device according to a first embodiment to which the present invention has been applied.

FIG. 2 is a block diagram showing a configuration of a silence detecting section of the video information detecting device.

FIG. 3 is a block diagram illustrating a configuration of a video information detection device according to a second embodiment to which the present invention has been applied.

FIG. 4 is a block diagram illustrating a configuration of a spectrum change detection unit of the video information detection device.

FIG. 5 is a block diagram illustrating a configuration of a video information detection device according to a third embodiment to which the present invention has been applied.

FIG. 6 is a block diagram illustrating a configuration of an audio determination unit of the video information detection device.

FIG. 7 is a block diagram illustrating a configuration of a video information detection device according to a fourth embodiment to which the present invention has been applied.

FIG. 8 is a block diagram showing a configuration of a level fluctuation determining unit of the video information detecting device.

FIG. 9 is a block diagram illustrating a configuration of a video information detection device according to a fifth embodiment to which the present invention has been applied.

FIG. 10 is a block diagram showing a configuration of an image local change calculation unit of the video information detection device.

FIG. 11 is a block diagram illustrating a configuration of a video information detection device according to a sixth embodiment to which the present invention has been applied.

FIG. 12 is a block diagram illustrating a configuration of a video information detection device according to a seventh embodiment to which the present invention has been applied.

FIG. 13 is a block diagram showing a configuration of a conventional video information detecting device.

FIG. 14 is a block diagram showing a configuration of a whole image change calculating unit of a conventional video information detecting device.

[Explanation of symbols]

11 moving image data storage unit (moving image data storage means),
12 Image whole change calculation unit (image change frame calculation unit), 20, 30, 40, 50, 60, 70, 80 Video information detection device, 21 audio data storage unit (audio data storage unit), 22 silence detection unit (audio Change frame calculating means), 23, 32, 42, 52,
62, 72 scene change detection section (video scene change frame calculation means), 31 spectrum change detection section (audio change frame calculation means), 41 sound determination section, 51 level change determination section, 61 image local change calculation section, 71 overall change Scene detector, 81 screen change frame calculator,
82 audio change frame detector, 83 scene change frame detector

Claims (6)

(57) [Claims] 1. A video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other, and audio data storage for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the audio change frame calculation means comprises:
Average signal level value for a given number of samples of
Is a video information detecting apparatus for calculating a frame in which a total sum of band outputs of a band-pass filter is smaller than a predetermined threshold continuously for a predetermined section. 2. A video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other, and audio data storage for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculation means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the audio change frame calculation means comprises: Time system of the frequency spectrum of A video information detecting apparatus for calculating a frame in which a change amount for a column is larger than a predetermined threshold. 3. A moving image data storage unit for storing moving image data for a video scene composed of moving image data and audio data synchronized in time series with each other, and audio data storage for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculation means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the audio change frame calculation means comprises: Due to the nature of A frame in which the signal level of the audio data is continuously lower than a predetermined threshold for a predetermined section and a frame in which the amount of change in the frequency spectrum of the audio data with respect to time series is larger than a predetermined threshold are adaptively selected and calculated. A video information detecting device, characterized in that: 4. A video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other, and audio data storage for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculation means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the image change frame calculation means comprises: Frey
As an example, a frame in which the image data of one frame of the moving image data completely changes with respect to the image data of a frame adjacent to the frame, and an image data of one frame of the moving image data, A video information detecting apparatus for calculating a locally changing frame with respect to image data of a frame adjacent to the frame. 5. A moving image data storing means for storing moving image data for a video scene composed of moving image data and audio data synchronized in time series with each other, and audio data storing for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the video scene change frame calculation means comprises: Calculated as a change frame All the frames calculated as frames in which the image changes entirely and the frames calculated as the audio change frames, and appropriately selected from the frames calculated as the frames in which the image changes locally, A video information detection device, which calculates the video information. 6. A video data storage means for storing video data for a video scene composed of video data and audio data synchronized in time series with each other, and audio data storage for storing the audio data. Means, an image change frame calculation means for calculating a change point in the time series of the moving image data as an image change frame, and an audio change frame calculation means for calculating a change point in the time series of the audio data as an audio change frame And a video scene change frame calculating means for calculating a video scene change frame based on the image change frame and the audio change frame, wherein the video scene change frame calculation means comprises: Using the audio data After calculating a change frame and calculating the calculated audio change frame as the video scene change frame, the image of a section in the video scene where the video scene change frame needs to be further calculated is obtained. A video information detection device, wherein a video frame change unit calculates a video change frame and calculates a frame calculated as the video change frame in the section as the video scene change frame.