JPH0965287A - Method and device for detecting characteristic scene for dynamic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain information assisting the user for video image selection by discriminating whether or not a scene is an important scene in a video image, specifying its range simply at a high speed, discriminating and classifying to which field a video image belongs (news, sport relay or the like). SOLUTION: This device is provided with a means entering an object dynamic image to a processer in time series in the unit of frames, a means buffering plural frames entered in the past in the processing unit, and a means discriminating whether or not a feature variable of the buffered frame has a characteristic in which the feature variable approaches that of the newest frame monotonously in the order from the older frames. Furthermore, when it is discriminated true by the discrimination means, a means is provided to extract a video period till a succeeding special video effect is detected or a prescribed after the frame as an important scene. Moreover, in addition to the special video effect, a means detecting a change and a state of various video images (change of cut, display of caption, layout or the like) and a means discriminating kinds of video images based on the appearing order and combination is provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for quickly looking at a moving picture such as a video or movie in a short time, and more particularly to cutting from a moving picture stored on a video tape or a video disc. Dissolve (continuous cuts) between (uninterrupted moving image sections taken by one camera)
When there are A and B, a characteristic scene detection method and apparatus for identifying a scene representative of a moving image by detecting a special video effect in which A fades out and B fades in at the transition of the cut) Regarding

[0002]

2. Description of the Related Art In recent years, in addition to ordinary television broadcasting, satellite broadcasting, cable television, etc. are becoming widespread, and the number of broadcasting channels is increasing. In the future, if a broadband communication infrastructure called an information highway is developed, it will be easier to deliver broadcasts, and more broadcasters than the current number will enter, accelerating the increase in the number of channels. It is a very time-consuming and time-consuming task to distinguish and select useful information and unnecessary information for each individual viewer from the information broadcast in such a large amount. for that reason,
Research on technology for efficiently creating summary information (digest) for quickly grasping video contents is under way.
The most basic and indispensable process in creating a digest is to select important scenes from the video. If a computer can automatically determine the importance of a scene in a video, the digest creation becomes very easy. For example, in Japanese Unexamined Patent Publication No. 4-294694, in a baseball relay, the correspondence between the movement result of a moving object in a video and a certain event (movement of a runner to a home position and a score, etc.) ), A method of selecting scenes with high importance is shown.

[0003]

However, in the motion analysis of a moving object, the accuracy and processing speed are not sufficient in the current state of the art of image recognition, and the motion pattern obtained thereby corresponds to a specific event. There is a problem that does not always correspond. Even if an event is correctly detected, it is extremely difficult to automatically determine which range before and after that event should be cut out as an important scene. Furthermore, although the digest itself is much shorter than watching the entire video, there is still a need to watch for a certain amount of time.
There is a need for a technology that enables a more brief overview.

An object of the present invention is to provide a method for easily and quickly determining whether an important scene in a video and identifying its range. In addition, it is to determine what kind of field (news, sports relay, etc.) the video belongs to and classify it, and provide it as information that helps the user to select the video.

[0005]

[Means for Solving the Problems] In many cases, broadcast video is provided with various video effects that emphasize important scenes on the broadcast station side. This property is particularly noticeable in the case of sports broadcasting, for example, it has the feature of showing a replay when a score is scored. In many cases, replay video images captured by cameras with different viewpoints cannot be used to determine whether they are replay video or not, but when switching to such replay video, dissolve or wipe Such special video effects are used to make it clear to the viewer that a temporary break from normal broadcasting is possible.
Furthermore, similar video effects are used when returning to normal broadcasting. Therefore, it is possible to select important scenes by detecting such special video effects.

Therefore, the target moving image is input to the processing device in time series in frame units, and in the processing device, the color or brightness of each pixel in the frame is spread over a plurality of continuous frame groups, It is checked whether the color or luminance value of the first frame of the frame group tends to monotonously approach the color or luminance value of the last frame, and the entire screen is determined from the number of pixels satisfying the condition. Is calculated, and when the evaluation value is out of a predetermined allowable range, a scene change due to a special video effect such as dissolve occurs in a section that spans a plurality of consecutive frames. It is determined that there is, and it is determined that the section or its vicinity is a characteristic point in the moving image.

Further, a target moving image is input to the processing device in time series on a frame-by-frame basis, or a target sound is input to the processing device in time series, and the processing device outputs a plurality of images including cut changes and color tones. Means for detecting changes in the image feature amount of each type and means for detecting changes in the voice feature amount including a speaker change as necessary, and the detecting unit detects that a change has occurred or a plurality of changes The type of the program is discriminated based on the feature amount, which is generated at the same time or in a specific order.

[0008]

[Operation] The scene to be replayed by broadcasting is the part that the expert has determined to be important, and if such a replay scene can be detected, the digest creation becomes extremely easy. According to the present invention, it is possible to detect a scene change due to a special video effect including a dissolve, and therefore it is possible to accurately extract an important scene that is preceded and followed by such a special effect. At the same time, the range of the scene can be obtained.

Furthermore, the means for discriminating the type of the program determines the type of the video based on the characteristic amount which is obtained by simultaneously or in a specific order that a plurality of types of changes in the image characteristic amount including cut changes and color tones occur. Because it is automatically determined,
If the type of video is not of interest to the viewer, it can be rejected without watching the digest video, and efficient video selection can be performed. Also, in determining the type of this video,
Since the judgment is made based on a simple change in the image or voice and its combination, the processing can be performed at high speed.

[0010]

EXAMPLE An example of the present invention will be described in detail below.

FIG. 1 is an example of a schematic block diagram of a system configuration for implementing the present invention. A display device 1 such as a CRT displays an output screen of the computer 4. Instructions to the computer 4 can be performed using the input device 5 such as a keyboard or a pointing device. The moving image reproducing device 10 is a tuner device for receiving broadcast programs such as terrestrial broadcasting, satellite broadcasting, cable television, or a device for reproducing moving images recorded on an optical disk, a video tape or the like. The video signal output from the moving image reproducing apparatus is A / S of 3
It is converted into digital image data by the D converter and sent to the computer. Inside the computer, the digital image data enters the memory 9 through the interface 8 and the CPU stores the program according to the program stored in the memory 9.
Processed by 7. When a number (frame number) is assigned to each frame of the moving image handled by 10 in order from the beginning of the moving image, the frame number is sent to the moving image reproducing apparatus through the control line 2 to generate a moving image of the scene. You can recall the image and play it back. Further, various kinds of information can be stored in the external information storage device 6 as required for processing. The memory 9 stores various data created by the processing described below and is referred to when necessary.

A method of detecting a dissolve, which is one of cut changes due to a special image effect, in selecting an important scene will be described in detail below.

FIG. 2 is an example of a flowchart of a moving image dissolve detection program executed on the system shown in FIG. The program is stored in the memory 9 and C
The PU 7 first sets various variables necessary for executing the program to initial values as initialization processing (20
0). Next, 0 is substituted for each element of the m two-dimensional array B (x, y) that stores the brightness value of each pixel of the past frame image (202). When the frame image size is w × h, x
Is from 0 to w-1 and y is from 0 to h-1. Processing 20
In 4, the frame image output by the moving image reproducing apparatus 10 is captured (204). In the process 206, the variable eval containing the evaluation value is set to 0, and the initial value 0 is assigned to the loop counter. Then, the following processing from 208 to 228 is performed for all pixels in the frame image.

In the processing from 208 to 228, the characteristic peculiar to the dissolve is detected. Where dissolve is
As shown in FIG. 3, the cut change has a section in which the frame images A and C of the front and rear cuts are mixed as shown by B before and after the cut change. As for the mixing ratio of A and C in B, the ratio reverses over time from the state of A of 100% and C of 0% at the start of dissolve, and finally A
The dissolution is completed when 0 is 0% and C is 100%. In the case of a grayscale image, the brightness value of A is Ba and the brightness value of B is B
When the brightness values of b and C are Bc, and the mixing ratio of C is α (0 ≦ α ≦ 1), it can be approximated by the formula of Bb = Ba × (1−α) + Bc × α. By transforming this equation, Bb = (Bc-
In the case of a dissolve in which Ba) × α + Ba, and the mixing ratio α increases monotonically from 0, the value of Bb also monotonically increases or decreases from Ba to Bc. Therefore, the brightness value of the pixel is always stored in the buffer for the past m frames, and the m
Dissolve can be detected by checking whether the luminance value is monotonically increasing or decreasing in the frame length section. If the value of m is set from 8 to 15,
Experimentally good results are obtained.

First, in step 208, the m-th array Bm of the two-dimensional array B storing the brightness values of past frames
Substitute the luminance value of the pixel represented by the coordinates (x, y). Then, 1 is assigned to the loop counter i and 0 is assigned to the variable num. Next, the brightness value B1 (x, y) stored in the first array
And the values of the m-th array Bm (x, y) are compared (212), and then the brightness value Bi (x, y) stored in the i-th array is transferred to the next array Bi + 1 (x, y). It is compared whether it is larger than the value of y) (214, 216). When B1 (x, y) is larger than Bm (x, y), the value of num is incremented by 1 when Bi (x, y) is larger than Bi + 1 (x, y). Conversely, if B1 (x, y) is smaller than Bm (x, y), then Bi (x, y) is smaller than Bi + 1 (x, y).
The value of is incremented by 1 (218). In the subsequent process 220, Bi
By substituting the value of Bi + 1 (x, y) into (x, y), the m array B is shifted one by one in order, and the brightness for m frames is always counted from the latest frame. Make sure the value is stored as a buffer. In process 222, the loop counter i is incremented by 1, and process 214 is performed when B1 (x, y) is greater than Bm (x, y) at the time of process 212 until i becomes greater than m, and process 216 otherwise. Then, the process is repeated (224). When num is larger than the threshold value th1 (226), the value of eval is incremented by 1 (228), assuming that the pixel at the coordinates (x, y) is monotonically increasing or decreasing. Natural moving images usually have irregular fluctuations due to noise, etc. Also, since the dissolve speed becomes uneven when a human performs a dissolve operation, it is not constant. A margin is provided by providing. In order to perform the above process for all the pixels in the frame image, the process returns to 208 and is repeated (230 to 236). This puts the number of pixels that satisfy the dissolve feature into the eval. Finally, eval is the threshold th2
Is checked (238), and if it is exceeded, it is determined that there is a dissolve, and the dissolve detection processing (240) is executed. Finally, the process returns to the process 204, and the process from 204 is repeated until the end of the video.

In the above method, the eval is higher even when there is a camera movement such as zooming or panning. This is because when the camera moves, the brightness of each pixel in the frame image changes accordingly, and in such a change, there are not a few pixels whose brightness monotonously increases or monotonically decreases. Therefore, there are cases where it is difficult to distinguish between dissolve and camera movement. Therefore, the following describes a dissolve detection method that allows the dissolve to be more clearly understood.

In general, the dissolve time is often one second (30 frames in the case of NTSC video) or more. Therefore, in the section where the dissolve is applied, m
= 8, 22 frames, and when m = 15, the state of high eval continues for 15 frames or more. on the other hand,
In the case of camera movement, the value is not as high as when it is dissolved, and the high state does not always continue.
Therefore, the sum of eval values for the past n frames
When sum is taken, a remarkable difference appears between the value of sum when dissolving and the sum when moving the camera. FIG.
This is a dissolve detection method with the above concept added.

First, as initialization processing, various variables necessary for executing the program are set to initial values (40
0). Next, 0 is assigned to each element of the m two-dimensional array B (x, y) that stores the luminance value of each pixel of the past frame image, and n values that store eval values for the past n frames are stored. All variables E1 to En of 0 are set to 0 (402). When the frame image size is w × h, x is 0 to w-1 and y is 0 to h
Takes a value up to -1. In process 404, the moving image reproducing apparatus 1
The frame image output by 0 is captured (40
4). Thereafter, the processing from 206 to 236 shown in FIG. 2 is executed to obtain an eval (406). Then, the value of eval is assigned to En. The sum from E1 to En is calculated as sum, and the value of Ej + 1 is successively substituted into Ej so that the latest eval value is always stored in E1 to En (408 to 412). . Finally, it is determined whether sum is larger than the threshold value th3 (414), and if it is larger, the dissolve detection processing 240 is performed. If not, the processing is returned to the processing 404 and repeated without doing anything.

In the dissolve detection processing 240, a scene sandwiched between dissolves is selected as an important scene. When the dissolve detection method of FIGS. 2 and 4 is executed, a graph showing the time transition of the evaluation value as shown in FIG. 5 can be obtained. The evaluation value has a characteristic that it does not show a large value for a moment in the dissolve section, but shows a triangular change that increases rapidly and decreases rapidly. And 2 which forms the base of the triangle
The vertices roughly correspond to the starting and ending points of the dissolve. When creating a digest, it is unsightly if the part where a special video effect such as a dissolve is applied remains at the beginning or end, so cut out the section 507 from the point where the dissolve ends to the point before the next dissolve begins. To do. Therefore, in addition to the first threshold value 500 used for determining whether or not a dissolve occurs in the above-described dissolve detection method, a second threshold value 502 lower than that is used. Then, when the dissolve as the starting point of the important scene is detected, the point 506 at which the evaluation value falls below the second threshold for the first time after the point 504 where the evaluation value exceeds the first threshold is set as the starting point of the important scene. . At this time, the start point may be delayed with a margin. When a dissolve is detected as the end point of the important scene, a point 508 at which the evaluation value falls below the second threshold for the first time from the point 510 at which the evaluation value exceeds the first threshold to the past.
Is the end point of the important scene. At this time, like the start point, the end point may be set earlier with a margin.
The time between dissolves can be used to determine whether the detected dissolve indicates the starting point or the ending point of the important scene. If normal broadcasting continues, there is no dissolve, so the time interval between dissolves becomes long, and in important situations, the interval is relatively short. By playing back the important scenes thus obtained in order, a digest can be created.

In the above embodiment, a monotonic change in luminance was investigated, but a similar change in color can be used. Color is three-dimensional information, unlike luminance, which is one-dimensional information. Therefore, it is not possible to simply examine the monotonic change based on the increase or decrease of the value. Here, the monotonic change from color A to color B means that when two colors are mapped in a three-dimensional color space, the distance from color A is gradually increased while the distance from color B is gradually reduced. It can be understood as a tendency. Therefore, instead of the two-dimensional array B that stores the luminance values of the past frames in FIG. 2, a two-dimensional array B ′ that stores colors is used, and if the color difference between each color in B ′ and B′1 increases. At the same time, if it is determined that they are arranged in a form in which the color difference from B'm is reduced, then the same method as in the case of luminance can be used.

The fact that a scene using a special video effect such as dissolve as described above can be regarded as an important scene is actually limited to some programs such as sports broadcasts. In addition, since special video effects frequently appear during commercials inserted between sports programs, it is often over-detected under the condition of a section simply sandwiched between dissolves. Of course, there is no problem in practical use as long as the detected video is a little shorter than the original video. However, if important scenes can be extracted with higher accuracy, the time required to understand the outline can be further saved. Therefore, we will provide a means to distinguish what type of video the digest is for and use it to select important scenes.

FIGS. 6 and 7 are views showing events occurring in news programs and sports programs along the time axis, respectively. Here, let us consider the fact that the characteristics of images and sounds change greatly as an event. In the figure,
The seven items, 1) composition, 2) color tone, 3) speaker, 4) subtitle, 4) dissolve, 6) replay, and 7) slow playback, are given as examples. The appearance and combination of such events have characteristics depending on the type of program, and the programs can be classified based on the characteristics. For example, in a news program, cuts that appear on the entire surface of the caster appear multiple times at intervals, so an image of the same composition, more specifically, a skin color, which is the color of the face, occupies a large area near the center. There is a feature that the image appears multiple times. In addition, the speakers at that time are often the same person, and subtitles frequently appear in the entire program. On the other hand, in the case of sports broadcasting, broadcasting is often performed by switching between multiple cameras installed at fixed positions, and images with the same or very similar composition frequently appear. Especially in the case of baseball and soccer, the main color tone is green, which is the color of the lawn. Also,
It is characterized by frequent use of replay and slow playback. Furthermore, in the case of CM, there is little break in sound, B
GM is often used, the color tone is vivid, and there are many cuts.
It is characterized by a short time length. In this way, the type of the video can be estimated to some extent from the combination pattern of a plurality of events in the video.
And, the events listed here are relatively easy to obtain and require only high reliability, while the techniques of image recognition and voice recognition are required. That is, it is not necessary to recognize the semantic content of images such as stories.

FIG. 8 is an example of a block diagram of a system for discriminating the type of video. The input video is digitized by the image capturing unit 800 and the audio capturing unit 802 for each of the image signal and the audio signal. The digitized data is sent to the event detection unit 804,
Event detection processing is performed by dedicated detection units 806 to 820 provided for each type. The detected event is counted by the event counter unit 822 for each type of event. Further, the co-occurrence counter section 824 increments the counter corresponding to the combination of the events by 1 only when a plurality of events appear at the same time or in a prescribed order. The appearance frequency distribution of various events obtained by these counters is compared and collated by the comparing unit 828 to determine which kind of program the event appearance frequency distribution is close to.

Next, each block in FIG. 8 will be described in detail.

The cut point detecting unit 806 of the event detecting unit 804 detects a cut transition. The method is shown in, for example, IPSJ Journal Vol.33, No.4, "Automatic indexing method and object search method in color video image" by the inventors, and JP-A-4-111181. You can use other methods. The event-specific counter unit 822 counts the number of cut points.

The same composition detecting unit 806 detects whether or not a picture of the same composition or a similar composition appears by tracing back to the past within a predetermined time. An image comparison method typified by template matching can be used for this. Specifically, for each pixel at the same coordinate position of the two frame images to be compared, the luminance difference or the color difference is obtained, and the total sum for the entire screen is calculated. To do. If the degree of difference is smaller than a predetermined threshold value, it can be determined that they are the same or have high similarity. Here, it takes a long processing time to detect whether or not all the frame images in the video have the same composition, and it is wasteful to consider the feature of a moving image having a high image similarity between consecutive frame images. . Therefore, only the image of the cut point is to be examined in conjunction with the cut point detection. The event-specific counter section counts the number of frames having the same composition.

The color tone detecting section 810 detects whether or not a picture of the same color tone or a similar color tone appears by going back to the past within a predetermined time. This includes, for example,
A chromaticity distribution for the entire frame screen is available.
This is a feature quantity that indicates how much each color is used, regardless of composition. Specifically, for each of the two frame images to be compared, the color of the pixel expressing the image is divided into about 64 colors, and the number of each of these colors existing in the frame image is counted. Then, the sum of the absolute values of the differences of the obtained frequency distributions is used as the color difference. If the degree of difference is smaller than a predetermined threshold value, it can be determined that they are the same or have high similarity. Regarding the color tone, it is efficient to target only the image of the cut point for the same reason as the composition. The event-specific counter unit counts the number of frames having the same color tone. Further, the color tone detecting unit may check which color is used most by using the frequency distribution obtained on the way. Specifically, prepare counters for each color such as red, blue, and green in the event-specific counter section, and increase the red counter if there are many red colors, and increase the green counter if there are many green colors. To

The caption detection unit 812 detects whether a caption appears in the video. As the method, for example, the method shown in Japanese Patent Application No. 5-330507 by the inventors can be used. In the event counter section 822, the number of appearances of subtitles is counted.

The dissolve detecting section 814 detects a special effect such as dissolve in an image. The method is as described in the first half of the present invention. The event counter unit 822 counts the number of appearances of dissolves.

The replay detection unit 816 detects whether or not the completely same video appears by going back to the past within a predetermined time. This can be performed by comparing frame images by template matching or the like as in the same composition detection unit 808. However, if template matching is performed for each frame between the moving images to be compared, it takes too much processing time, so each frame is converted into a code of several characters and the matching of the code string is used to match the moving images. And The amount of information is extremely small in a code unit corresponding to one frame, but since a moving image is composed of many frames, one moving image also contains a large number of codes and a sequence of codes in the moving image. Has a sufficient amount of information to identify a piece of moving image. A method of collating moving images based on this concept is disclosed in Japanese Patent Laid-Open No. 7-114567 by the inventors.

The slow playback detecting section 818 detects the slow playback video. Slow playback is realized by continuously displaying frame images at intervals longer than in standard playback (1/2 slow is 2 times, 1/4 slow is 4 times), so in the case of slow playback video, The images digitized by the image capturing unit 800 are characterized in that a plurality of exactly the same images continue (two in 1/2 slow, four in 1/4 slow).
Sheet). Therefore, in order to determine whether or not the slow reproduction is performed, two consecutive frames are examined and the image difference degree is examined by the template matching. Then, the transition of the dissimilarity for a certain period of time is examined, and if the dissimilarity repeats a large value and a small value in a specific cycle, it is determined to be slow reproduction. For example, in the case of 1/2 throw, since the same image continues two by two, the difference degree repeats a small value and a large value alternately. In the case of 1/4, a small value is repeated three times, and a large value is repeated once. However, in the case of a moving image, two consecutive frame images are similar to each other even if the slow reproduction is not performed. Therefore, it is necessary to make the threshold value lower to determine the magnitude of the difference. In the event counter unit 822, the number of appearances of slow reproduction is counted.

The same-speaker detecting section 820 detects whether or not the same speaker has spoken retroactively within a predetermined time. For example, it is possible to find the autocorrelation of the voice and check whether or not the frequencies having the largest values match. The event-specific counter unit 822 counts the number of utterances of the same speaker.

The co-occurrence counter section 824 counts when some of the above events appear simultaneously or in a specific order. Counters are prepared for each combination of detected events. For example, in the case of the same composition and the same speaker speaking, the counter corresponding to the simultaneous occurrence of the composition event and the speaker event is incremented by one. Similarly, when there is a dissolve and slow reproduction is detected immediately after that, the counter corresponding to the consecutive occurrence of the dissolve event and the slow reproduction event is incremented by one.

The comparison unit 828 refers to the clock 826,
It is compared what kind of program the tendency of the appearance frequency of the event in the video in a certain time from the time t1 to the time t2 is close. Prior to comparison, first check typical events for each type, such as news programs and sports programs, and assign a value so that the more important events characterize the program, the higher the ranking, and the more each event is ranked. Create a rank list for each event.
For comparison, the normalized appearance frequency value of each event is multiplied by the value described in this rank list and weighted, and the sum of the values of each event thus obtained exceeds the threshold value. If so, it is determined that the program is of a type corresponding to the rank list.

The event thus obtained is shown in FIG.
Alternatively, as shown in FIG. 7, in a table format with one side as the time axis,
A list can be displayed on the display 1 of FIG.
Even if the computer cannot automatically judge by this list display, the user can guess the type of program by using such information as one clue and using the information, experience and knowledge obtained from others together. There is a possibility. Further, when a program of a type not taught to the computer is newly input, an important event or a combination of events may be selected and registered from this list display. This can be done by a direct and visual operation such as clicking the change point of each event on the list or the display portion of the section using the pointing device 5 such as the mouse shown in FIG. 1 for the user. It will be very convenient.

The present invention can be realized not only by using PC / WS but also as one function of TV, VTR and the like.

[0037]

According to the present invention, an important scene and its range can be obtained at the same time, and a digest video can be automatically created. Generally, the replayed scene is often an important scene, but in the present invention, the replayed scene during broadcasting can be accurately detected by detecting the section of the special video effect including the dissolve.

Furthermore, the means for discriminating the type of the program based on the characteristic amount which is obtained by simultaneously or in a specific order that a plurality of types of changes in the image characteristic amount including the cut change and the color tone occur Since it is automatically determined, if the video is of a type that is not of interest to the viewer, it can be rejected without looking at the digest video, and there is an effect that efficient video selection can be performed. Further, in the determination of the type of the video, the determination can be performed based on the simple change of the image and the sound and the combination thereof, so that the processing can be performed at high speed.

[Brief description of drawings]

FIG. 1 is a system block diagram for realizing an embodiment of the present invention.

FIG. 2 is a flow chart of a program for detecting a dissolve.

FIG. 3 is a diagram showing a concept of dissolve.

FIG. 4 is a flowchart of another program for detecting a dissolve.

FIG. 5 is a graph showing a time transition of an evaluation value when a program for detecting a dissolve is executed.

FIG. 6 is a typical event chart of a news program.

FIG. 7 is a typical event chart of sports broadcasting.

FIG. 8 is a block diagram of a system for classifying images.

[Explanation of symbols]

1 ... Display, 2 ... Control signal line, 3 ... A / D converter, 4 ... Computer, 5 ... Input device, 6 ... External information storage device, 7 ... CPU, 8 ... Connection interface, 9 ... Memory, 10 ... Movie Image reproduction device, 11 ... Keyboard.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Katsumi Taniguchi 1-280, Higashi Koigokubo, Kokubunji, Tokyo Metropolitan Research Center, Hitachi Ltd.

Claims (9)

[Claims] 1. A method for detecting a characteristic scene of a moving image, which detects a transition of a scene of a special video effect including a dissolve from a moving image composed of a plurality of continuous images, in which the target moving image is time-series in units of frames. Inputting to the processing device, in the processing device, the color or brightness of each pixel in a frame is spread over a plurality of consecutive frame groups, and from the color or brightness value of the first frame of the frame group to the final value. It is checked whether or not the color or brightness of the frame tends to become monotonous, and the evaluation value representing the change in the entire screen is calculated from the number of pixels satisfying the condition, and the evaluation value is predetermined. When it is outside the permissible range, it is determined that there is a scene change due to a special video effect including dissolve in the section that spans the plurality of consecutive frame groups, and the section is also broken. Wherein the scene detection process of a moving image, characterized by determining the neighborhood as a characteristic feature of the moving image. 2. A method for detecting a characteristic scene of a moving image of a moving image according to claim 1, wherein a buffer for storing the brightness of a plurality of frames for each pixel is provided as means for determining whether or not the tendency tends to be monotonous. If the number of luminances in which the difference from the immediately preceding luminance shows a positive value in the buffer occupies the majority of the total number of luminances stored in the buffer, it is monotonically increased. A method for detecting a characteristic scene of a moving image, characterized in that when the number of luminances having a negative difference from the luminance of occupies a majority of the total number of luminances stored in the buffer, it is determined to be monotonically decreasing. 3. The method for detecting a characteristic scene of a moving image of a moving image according to claim 1, wherein the evaluation values representing the tendency of the time transition of the entire screen are summed retroactively for a certain period of time, and the sum is dissolved. A method for detecting a characteristic scene in a moving image, which is characterized by determining that there is a scene change due to a special video effect including the. 4. A method for detecting a characteristic scene of a moving picture of a moving picture according to claim 1, wherein a scene change is caused by a special video effect including the detected dissolve and a scene change is caused by a special video effect including a subsequent dissolve. A method for detecting a characteristic scene of a moving image, wherein the section is extracted as an important scene when the section is within a certain time. 5. The method for detecting a characteristic scene of a moving picture according to claim 4, wherein the section of the important scene is the remaining section excluding the duration of the special video effect. A method for detecting characteristic scenes in moving images. 6. A moving image classification method for determining what kind of program a moving image consisting of a plurality of continuous images is, in a processing device, a target moving image is time-sequentially processed in frame units. A unit for inputting or inputting a target voice in a time series to a processing device, the processing device detecting a change in a plurality of types of image feature amounts including a cut change and a color tone, and a speaker if necessary. A means for detecting a change in the voice feature quantity including a change is provided, and the change detection means for the image feature quantity and the voice feature quantity causes a change, or a plurality of changes occur simultaneously or in a specific order. A method for classifying moving images, characterized in that the type of program is discriminated based on the feature amount. 7. The moving image classification method according to claim 6, wherein the detected change points of the plurality of types of image feature amounts or audio feature amounts and the change sections thereof are in a table format with the time axis as one axis. A method for classifying moving images, which is characterized by displaying a list in. 8. A characteristic scene detection device for a moving image, which detects a transition of a scene of a special video effect including a dissolve from a moving image composed of a plurality of continuous images, in which the target moving image is time-sequentially frame by frame. Input means for inputting, means for buffering a plurality of frames input in the past, and feature values of the buffered frames that monotonically approach the feature value of the newest frame in order from the oldest one Means for deciding whether or not there is, and if the decision means decides that there is a special video effect, the video from that frame for a certain period of time or until the next special video effect is detected A characteristic scene detection device for a moving image, characterized by comprising means for extracting a section as an important scene. 9. A moving image characteristic scene detection apparatus according to claim 9, wherein in addition to the special video effect, a means for detecting a change or state of a video including a cut transition, subtitle display, composition, and the like. A device for detecting a characteristic scene of a moving image, which is provided with a means for discriminating the type of video from the order or combination of the appearance of.