JP2008085540A - Program, detection method and detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a cut change including a blend cut change. <P>SOLUTION: A feature extraction part 811 extracts the features of a first image, a second image and a third image, and a similarity degree calculation part 813 calculates the similarity degree of the features of the two images among the first image, the second image and the third image. A reduced image generation part 814 generates the reduced images of the first image, the second image and the third image, a synthesis image generation part 816 generates the synthesis images of the reduced images of the first image and the third image positioned before and after the second image timewise, and a similarity degree calculation part 817 calculates the similarity degree of the reduced image of the second image and the synthesis image. A judgement part 818 judges whether or not the cut change is present between the first image and the third image on the basis of the similarity degree of the features of the two images among the first image, the second image and the third image and the similarity degree of the reduced image of the second image and the synthesis image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a program, a detection method, and a detection apparatus, and more particularly, to a program, a detection method, and a detection apparatus that can detect cut changes with high accuracy.

  The process of detecting a cut change from an image is useful when performing image analysis, encoding, retrieval, or the like.

  Here, “cut” means a group of spatially continuous image segments (image clips), and “cut change” means a cut switching point, that is, temporally or spatially discontinuous image clips connected to each other. It means the change point of the image that is the bound.

  The cut change may be referred to as a shot change, a scene change, an image change point, or the like, but is referred to as a cut change in this specification.

  By the way, as a conventional cut change detection method, a statistic difference method, a pixel difference method, an encoded data method, an edge method, and the like are known, but it is recognized that the most accurate method is the histogram difference method. It is.

  In the histogram difference method, a histogram difference value is used.

  Here, in the histogram, each pixel of an image of a certain frame is divided into an appropriate number of gradations (generally 16 to 64 gradations are commonly used) corresponding to the luminance or color pixel value. By voting for each element (bin), the luminance or color frequency distribution of the image is obtained.

  The histogram difference method includes a simple histogram difference method and a divided histogram difference method.

  In the simple histogram difference method, for example, a difference between histograms of images of two frames to be processed is calculated as an evaluation amount (see, for example, Non-Patent Document 1).

  In the simple histogram difference method, generally, as the difference between histograms, the absolute sum of the difference values between the same bins of the histograms regarding the entire histogram (hereinafter referred to as the histogram difference absolute sum as appropriate) is calculated as the evaluation amount.

  In the divided histogram difference method, for example, the entire image of each of the two-frame images to be processed is divided into a predetermined number of blocks (for example, 16 blocks in Non-Patent Document 1), and the histograms of each block are shared. The difference between the histograms is calculated, and only a predetermined number of blocks are used for calculating the evaluation amount from the one with the smaller difference between the histograms (see, for example, Non-Patent Documents 1 and 2).

  By the way, in general, a cut change often occurs between adjacent images as described above. Such a cut change is hereinafter referred to as a normal cut change as appropriate. On the other hand, there is a cut change in which two cuts are connected in an image in which the preceding and following images are mixed. This is called blend cut change.

  In the normal cut change detection method, as described above, the similarity between two images to be processed is calculated as an evaluation amount, and when the similarity is lower than a predetermined threshold, that is, the dissimilarity is If it is higher than the predetermined threshold, it is determined that the change in the image between the two images is a cut change.

  However, with such a method, it is difficult to accurately detect the blend cut change. That is, in FIG. 1 showing an example of normal cut change, frames F61 to F63 are a series of temporally adjacent images, and two images sequentially detect cut changes from frame F61. Be targeted. That is, first, the frame F61 and the frame F62 are processed, and then the frame F62 and the frame F63 are processed.

  In FIG. 1, if the similarity between the frame F61 for shooting a person and the frame F62 for shooting a house is low and the similarity is equal to or less than a predetermined threshold, the frame F61 and the frame It is determined that the image change during F62 is a normal cut change. Further, if the similarity between the frame F62 whose home is the subject of shooting and the frame F63 whose home is the subject of shooting is high, and the similarity is not less than the threshold value, it is determined that it is not a cut change. The

  On the other hand, in FIG. 2 showing an example of blend cut change, frames F71 to F73 are a series of temporally adjacent images, and two images in order from the frame F71 are cut change detected. To be processed. That is, first, the frame F71 and the frame F72 are processed, and then the frame F72 and the frame F73 are processed.

  The frame F72 in which the person and the house are the shooting target is an image in which the frame F71 before the person is the shooting target and the frame F73 after the house is the shooting target are mixed. Since the two cuts are connected, the frame F72 is a blend cut change.

  Such a blend cut change is, for example, when a cut change occurs between fields in the same image (this cut change is hereinafter referred to as a field cut change as appropriate), or when adjacent images are mixed by an image filter. This occurs when adjacent images are mixed at the time of encoding or decoding, or when adjacent images are mixed at the time of editing.

  In FIG. 2, the frame F72 includes portions similar to the frame F71 or F73, and the frame F72 has a high similarity to both the frame F71 and the frame F73. It is difficult to detect a blend cut change even if a method of detecting a cut change based on the degree of similarity is employed.

  Patent Document 1 proposes a method for detecting a field cut change which is one of blend cut changes. In the method of Patent Document 1, a cut change is detected based on the similarity between two frames of one frame instead of the similarity between two adjacent images.

  However, when the method of Patent Document 1 is used, there is a possibility that the cut change cannot be detected when the image of each frame is an image in which the shooting target moves quickly.

  For example, in FIG. 3, frames F <b> 81 to F <b> 83 are a series of temporally adjacent images and represent images in which a person gradually moves away from the house in the left direction in the screen. The images in the frames F81 to F83 are images in which a person is moving at high speed in the left direction in the screen, and no cut change occurs.

  In FIG. 3, the similarity between the frame F81 of the image including the whole person and a part of the left side of the house, the similarity of the frame F82 of the image including the entire range of the person and the left side of the house, and the frame F82 and the person. Although the similarity between the right half of the image and the frame F83 of the image including the entire house is high, the movement of the person is fast, so the frame F81 of the image including the entire person and a part of the left side of the house The degree of similarity between the right half of the frame and the frame F83 of the image including the entire house is low.

  When the method of Patent Document 1 described above is used, for example, when the similarity between every other frame F81 and frame F83 among the frames F81 to F83 to be cut change detected is obtained, Since the similarity between F81 and frame F83 is low, there is a high possibility that an image change between frames F81 to F83 is erroneously determined as a blend cut change.

  Patent Document 2 proposes a method for detecting a field cut change using encoded data. However, since this method uses a characteristic amount peculiar to an encoded image of a specific encoding method, it can be applied only to an image encoded using the encoding method.

  In Patent Document 3, an absolute sum of the difference values between pixels of two images to be processed (hereinafter referred to as an absolute difference sum as appropriate) is obtained, and further, a difference value between absolute difference sums is obtained. Thus, a method for distinguishing and detecting a normal cut change and a field cut change has been proposed. However, even when this method is used, it is difficult to accurately detect blend cut changes (including field cut changes).

IPSJ Journal, Nagasaka Goro, Tanaka Joe, April 1992, Vol.33, No.4, P.543-550 Comparison of Video Shot Boundary Detection Techniques, John S. Boreczky, Lawrence A. Rowe.Storage and Retrieval for Image and Video Databases (SPIE) (1996) p170-179 JP 2000-295624 A JP 2002-64823 A JP 2000-324499 A

  As described above, it has been difficult for the conventional method to accurately detect the blend cut change. In particular, there is a possibility that an erroneous detection of a blend cut change may occur when the image of each frame is an image in which the shooting target moves quickly.

  The present invention has been made in view of such a situation, and makes it possible to accurately detect a cut change including a blend cut change.

  A program or a detection method according to one aspect of the present invention is a program or a detection method for causing a computer to execute a detection process for detecting a cut change of an image, wherein the characteristics of the first image, the second image, and the third image are characterized. Extracting, calculating the similarity of the characteristics of two images of the first image, the second image, and the third image, and calculating the first image, the second image, and the third image; A reduced image of the first image, a combined image of the reduced image of the first image and the third image positioned before or after the second image in time, and the second image The similarity between the reduced image of the image and the composite image is calculated, the similarity between the features of two images of the first image, the second image, and the third image, and the second image Based on the similarity between the reduced image and the composite image, During the serial first image and the third image comprises determining whether there is cut change.

  According to another aspect of the present invention, there is provided a detection device that detects a cut change of an image, a feature extraction unit that extracts features of a first image, a second image, and a third image; First similarity calculating means for calculating the similarity of the features of two images of the image, the second image, and the third image; the first image; the second image; A reduced image generating means for generating a reduced image of the third image, and a combined image of the reduced image of the first image and the third image located before or after the second image in time Composite image generating means, second similarity calculating means for calculating the similarity between the reduced image of the second image and the composite image, the first image, the second image, and the third image Similarity between features of two of the images, and reduction of the second image Based on the similarity of the image and the synthetic image, between the first image and the third image, and a determining means for determining there is cut change.

  In one aspect of the present invention, features of the first image, the second image, and the third image are extracted, and two of the first image, the second image, and the third image are extracted. The feature similarity is calculated, and reduced images of the first image, the second image, and the third image are generated. In addition, a combined image of the reduced image of the first image and the third image positioned before or after the second image in time is generated. The similarity between the reduced image of the second image and the synthesized image is calculated, the similarity between the features of the two images of the first image, the second image, and the third image, and the reduction of the second image. Based on the similarity between the image and the composite image, it is determined whether a cut change exists between the first image and the third image.

  According to one aspect of the present invention, a cut change including a blend cut change can be detected.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

A program or a detection method according to one aspect of the present invention includes:
In a program or a detection method for causing a computer to execute a detection process for detecting an image cut change,
Extracting features of the first image, the second image, and the third image (for example, step S601 in FIG. 6);
Calculating the similarity of the features of two images of the first image, the second image, and the third image (for example, step S602 in FIG. 6);
Generating reduced images of the first image, the second image, and the third image (for example, step S603 in FIG. 6);
Generating a composite image of the reduced image of the first image and the third image positioned before or after the second image in time (for example, step S604 in FIG. 6);
Calculating the similarity between the reduced image of the second image and the composite image (for example, step S605 in FIG. 6);
Based on the similarity between the characteristics of two images of the first image, the second image, and the third image, and the similarity between the reduced image of the second image and the synthesized image, It is determined whether a cut change exists between the first image and the third image (for example, step S606 in FIG. 6).
Comprising steps.

A detection device according to one aspect of the present invention includes:
In a detection device (for example, the personal computer 1 in FIG. 4) that detects an image cut change,
Feature extraction means (for example, feature extraction unit 811 in FIG. 5) for extracting features of the first image, the second image, and the third image;
First similarity calculation means for calculating the similarity of the features of two images of the first image, the second image, and the third image (for example, the similarity calculation unit 813 in FIG. 5). When,
Reduced image generating means for generating reduced images of the first image, the second image, and the third image (for example, the reduced image generating unit 814 in FIG. 5);
Composite image generation means for generating a composite image of the reduced image of the first image and the third image positioned before or after the second image in time (for example, the composite image generation unit in FIG. 5) 816),
Second similarity calculation means (for example, a similarity calculation unit 817 in FIG. 5) that calculates the similarity between the reduced image of the second image and the composite image;
Based on the similarity between the characteristics of two images of the first image, the second image, and the third image, and the similarity between the reduced image of the second image and the synthesized image, And a determination unit (for example, a determination unit 818 in FIG. 5) for determining whether or not a cut change exists between the first image and the third image.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 4 is a block diagram showing a configuration example of an embodiment of a detection apparatus to which the present invention is applied.

  In FIG. 4, a personal computer 1 as a detection device includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a bus 24, an input / output interface 25, an input unit 26, and an output. Unit 27, storage unit 28, communication unit 29, drive 30, and removable medium 31.

  The CPU 24, ROM 22, RAM 23, and input / output interface 25 are connected to the bus 24, and the bus 24, input unit 26, output unit 27, storage unit 28, communication unit 29, and drive 30 are connected to the input / output interface 25. It is connected.

  The CPU 21 executes various processes according to programs stored in the ROM 22 or the storage unit 28. Further, the CPU 21 executes various processes in response to commands input from the input unit 26, and outputs processing results to the output unit 27.

  The ROM 22 stores a program executed by the CPU 21 and the like.

  The RAM 23 appropriately stores programs executed by the CPU 21 and data. The RAM 23 includes a buffer that temporarily stores an image input from the outside, for example.

  The input unit 26 includes a keyboard, a mouse, a microphone, and the like.

  The output unit 27 includes a display, a speaker, and the like.

  The storage unit 28 includes, for example, a hard disk and stores programs executed by the CPU 21 and various data.

  The communication unit 29 communicates with an external device via a network such as the Internet or a local area network.

  A program may be acquired via the communication unit 29 and stored in the storage unit 28.

  When a removable medium 31 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is loaded, the drive 30 drives them to acquire programs and data recorded therein. The acquired program and data are transferred to and stored in the storage unit 28 as necessary.

  As shown in FIG. 4, a program recording medium that stores a program that is installed in a computer and can be executed by the computer is a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, or a removable medium 31 that is a package medium composed of a semiconductor memory, ROM 22, and a hard disk that constitutes the storage unit 28. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 29 that is an interface such as a router or a modem as necessary. Done.

  The CPU 21 of the personal computer 1 functions as a cut change detection device that detects a cut change by executing a program stored in the ROM 22 or the storage unit 28.

  FIG. 5 is a block diagram showing a configuration of an embodiment of a cut change detection device 51 configured by software of the personal computer 1.

  In FIG. 5, the cut change detection device 51 inputs an external image by reading it from, for example, a buffer of the RAM 23 and performs a cut change detection process.

  In FIG. 5, the cut change detection device 51 includes a feature extraction unit 811, a feature amount buffer 812, a similarity calculation unit 813, a reduced image generation unit 814, a reduced image buffer 815, a composite image generation unit 816, a similarity calculation unit 817, And a determination unit 818.

  An image input to the cut change detection device 51 is supplied to the feature extraction unit 811.

  Now, the latest three images input to the cut change detection device 51, that is, the previous image, the previous image, or the current image in time order, are respectively referred to as a first attention image, a second attention image, Alternatively, the third attention image is used.

  The feature extraction unit 811 extracts the feature amounts of the first to third attention images supplied thereto and supplies them to the feature amount buffer 812.

  As such feature amount, for example, a luminance or color histogram described later with reference to FIGS. 8 and 9 is adopted, but in addition, a general cut change detection method can also be adopted, for example, As shown in Non-Patent Document 1 and Non-Patent Document 2, an average value or variance value of luminance, a DCT coefficient or motion vector obtained by encoding, an edge obtained by edge detection, or the like can be used.

  The feature amount buffer 812 stores the feature amounts of the first to third attention images supplied from the feature extraction unit 811. The feature amount buffer 812 also stores at least the latest three images, that is, the first to third attention images.

  The similarity calculation unit 813 reads the first to third attention images from the feature amount buffer 812, and calculates the similarity between the feature amount of the first attention image and the feature amount of the third attention image (hereinafter, similar as appropriate). The first similarity 1, which is 1 degree), the second similarity 1, which is the similarity 1 between the feature amount of the first attention image and the feature amount of the second attention image, and the second attention image A third similarity 1 that is a similarity 1 between the feature quantity and the feature quantity of the third target image is calculated, and the first to third similarity degrees 1 are supplied to the determination unit 818, respectively.

  As the similarity 1, for example, the histogram similarity described later with reference to FIG. 10 is adopted, but in addition to this, the one used in a general cut change detection method can be adopted. As shown in Document 1 or Non-Patent Document 2, the average value or the difference value between the luminance values of two adjacent images, the difference absolute sum or the difference square sum of two adjacent images ( Square distance), absolute difference of histogram difference, difference value of DCT coefficient, magnitude of prediction error that is error between image predicted using motion vector and actual image, difference in number of edges or distribution of edges, etc. Can be used.

  The reduced image generation unit 814 is supplied with the image input to the cut change detection device 51, that is, the first to third attention images. The reduced image generation unit 814 generates reduced images obtained by reducing the first to third attention images supplied thereto, and supplies the reduced images to the reduced image buffer 815. As the reduced image, for example, an image reduced to a size of about 16 pixels × 16 pixels from 8 pixels × 8 pixels is used.

  The reduced image buffer 815 stores the reduced images of the first to third attention images supplied from the reduced image generation unit 814. The reduced image buffer 815 also stores at least the latest three images, that is, the first to third attention images.

  The composite image generation unit 816 reads, from the reduced image buffer 815, a reduced image of the first target image and a reduced image of the third target image that are temporally before or after the second target image, respectively. A synthesized image obtained by synthesizing the reduced image of the first image of interest and the reduced image of the third image of interest is generated and supplied to the similarity calculation unit 817.

For example, the pixel value G (x, y) of the point (x, y) of the composite image is expressed by F ₁ (x, y) as the pixel value of the reduced image of the first image and the pixel of the reduced image of the third image. When the value is F ₃ (x, y), it is expressed by Expression (1) using predetermined constants a and b.

  Here, x or y represents the X coordinate or Y coordinate of each image, respectively.

  The similarity calculation unit 817 reads the reduced image of the second attention image from the reduced image buffer 815, and uses the reduced image of the first attention image and the reduced image of the third attention image supplied from the composite image generation unit 816. A similarity 2 that is a similarity between the synthesized image and the reduced image of the second image is calculated and supplied to the determination unit 818.

  The similarity 2 is, for example, a value obtained by inverting the sign of the sum of squares of the difference value between the same pixels of the synthesized image and the reduced image, or the maximum (maximum correlation) of the correlation function between the synthesized image and the reduced image. Value) or the like.

  The determination unit 818 uses the first to third similarity 1 supplied from the similarity calculation unit 813 and the similarity 2 supplied from the similarity calculation unit 817 to determine the first attention image and the second attention. It is determined whether the boundary between the images or the boundary between the second attention image and the third attention image is a cut change, that is, whether there is a cut change between the first attention image and the third attention image. To do.

  Next, the cut change detection process by the cut change detection device 51 will be described with reference to the flowchart of FIG.

  An image input to the cut change detection device 51 is supplied to the feature extraction unit 811.

  Now, the three most recent images input to the cut change detection device 51, that is, the previous image, the previous image, or the current image in time order, are represented by a frame F511, a frame F512, or The image is a frame F513. The similarity between the image of the frame F511 that is a subject of photographing and the image of the frame F513 that is subject of photographing of the house is low, and the image of the frame F512 is the image of the previous frame F511 and the image of the subsequent frame F513. Therefore, the similarity between the image of the frame F511 and the image of the frame F512 and the similarity between the image of the frame F512 and the image of the frame F513 are high. That is, the frame F512 includes a blend cut change that is a boundary that connects the cut up to the frame F511 and the cut after the frame F513.

  In step S601, the feature extraction unit 811 extracts the feature amounts of the frames F511 to F513 supplied thereto.

  Specifically, for example, when image data of the original image shown in FIG. 8 is input, a histogram as shown in FIG. 9 is extracted and generated as a feature amount.

  In FIG. 9, the horizontal axis represents luminance and the vertical axis represents frequency. Since the original image P101 has a background gray area and a central white and black area, the histogram H101 has a relatively small luminance (black), an intermediate luminance (gray), and a relatively large luminance ( The frequency is concentrated in each of (white).

  The histogram as the feature amount is stored in the feature amount buffer 812.

  In step S602, the similarity calculation unit 813 reads the feature amounts of the frames F511 to F513 from the feature amount buffer 812, and the first similarity that is the similarity between the feature amount of the frame F511 and the feature amount of the frame F513. 1, the second similarity 1 that is the similarity between the feature amount of the frame F511 and the feature amount of the frame F512, and the third similarity 1 that is the similarity between the feature amount of the frame F512 and the feature amount of the frame F513. The first to third similarity 1 is calculated and supplied to the determination unit 818.

  Specifically, for example, when generating a histogram as a feature amount, an overlapping rate between histograms may be adopted as the similarity between the histograms.

  For example, as shown in FIG. 10, the overlapping rate of the histogram H111 and the histogram H112 is the area of the portion common to the histogram H111 and the histogram H112 (the area of the hatched portion of the histogram).

  In addition, for the histogram similarity, a similarity between vectors (hereinafter, appropriately referred to as histogram vectors) having the frequency of each element (bin) of the histogram as a component may be used. As the similarity between histogram vectors, for example, the so-called Manhattan distance (substantially equal to the overlap ratio between histograms) that is an absolute distance between histogram vectors, the Euclidean distance that is the square distance between histogram vectors, and the like. May be used.

  The reduced image generation unit 814 is supplied with the image input to the cut change detection device 51, that is, the images of the frames F511 to F513.

  In step S <b> 603, the reduced image generation unit 814 generates reduced images obtained by reducing the images of the frames F <b> 511 to F <b> 513 supplied thereto, and stores the reduced images in the reduced image buffer 815. That is, an image having a size displayed on a display unit such as an LCD (Liquid Crystal Display) is reduced to, for example, a size of 8 pixels × 8 pixels. This reduction is performed, for example, by dividing the original image into 8 × 8 blocks and obtaining an average value of each block.

  In step S604, the composite image generation unit 816 reads, from the reduced image buffer 815, a reduced image of the frame F511 temporally preceding the frame F512 and a reduced image of the frame F513 temporally subsequent to the frame F512, respectively. In accordance with 1), a combined image obtained by combining the reduced image of the frame F511 and the reduced image of the frame F513 is generated and supplied to the similarity calculation unit 817.

  In step S605, the similarity calculation unit 817 reads the reduced image of the frame F512 from the reduced image buffer 815, and combines the reduced image of the frame F511 and the reduced image of the frame F513 supplied from the combined image generation unit 816 with each other. The similarity 2 that is the similarity of the reduced image of the frame F512 is calculated. Specifically, a value obtained by reversing the sign of the square distance between the composite image and the reduced image, or the maximum value of the correlation function of both is calculated. The similarity 2 as the calculation result is supplied to the determination unit 818.

  In step S <b> 606, the determination unit 818 determines the first target image based on the first to third similarity 1 supplied from the similarity calculation unit 813 and the similarity 2 supplied from the similarity calculation unit 817. It is determined whether the boundary between the second attention image or the boundary between the second attention image and the third attention image is a cut change. Thereafter, the cut change detection process is terminated.

  Next, the determination process performed by the determination unit 818 in step S606 in FIG. 6 will be described with reference to the flowchart in FIG.

  The determination unit 818 is supplied with the first to third similarity 1 from the similarity calculation unit 813 and the similarity 2 from the similarity calculation unit 817, respectively.

  In step S631, the determination unit 818 determines whether the similarity 1 (first similarity 1) between the image of the frame F511 and the image of the frame F513 is greater than a predetermined threshold value. When the first similarity 1 is larger than the threshold value, in step S635, the determination unit 818 determines that the image change in the frames F511 to F513 is not a cut change.

  For example, as shown in FIG. 12A, when the images of the frames F511 to F513 are hardly changed, or as shown in FIG. 12B, the images of the frames F511 and F513 are hardly changed. If the image of the frame F512 is different in brightness compared to the frames F511 and F513 (for example, when a flash is applied or the frame is dropped), it is determined that the image is not a cut change.

  On the other hand, when it is determined in step S631 that the first similarity 1 is not greater than the threshold value, in step 632, the determination unit 818 determines the similarity 1 (second similarity 1 between the images of the frames F511 and F512). ) Is larger than a predetermined threshold value set in advance.

  If it is determined in step S632 that the second similarity 1 is greater than the threshold value, in step S633, the determination unit 818 determines that the similarity 1 (third similarity 1) between the images of the frames F512 and F513 is in advance. It is judged whether it is larger than a predetermined threshold value that has been set.

  If it is determined in step S633 that the third similarity 1 is greater than the threshold, in step S634, the determination unit 818 determines whether the similarity 2 is greater than a predetermined threshold.

  If it is determined in step S634 that the similarity 2 is not greater than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is smaller than the threshold, and the similarity 1 between the images of the frames F511 and F512 is less than the threshold. If the similarity between the images of the frames F512 and F513 is larger than the threshold and the similarity 2 is smaller than the threshold, the determination unit 818 cuts the image changes in the frames F511 to F513 in step S635. Judge that it is not a change.

  For example, as shown in FIG. 12C, the images of frames F511 to F513 are images in which a person moves away from the house with a fast movement, and the images of frames F511 and F513 are not similar. If the images of the frames F511 and F512 are similar, the images of the frames F512 and F513 are similar, and the image obtained by combining the reduced images of the frames F511 and F513 and the reduced image of the frame F512 are not similar, the cut change It is determined that it is not.

  If it is determined in step S634 that the similarity 2 is greater than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is less than the threshold, and the similarity 1 between the images of the frames F511 and F512 is greater than the threshold. When the similarity 1 between the images of the frames F512 and F513 is greater than the threshold and the similarity 2 is greater than the threshold, the determination unit 818 determines whether the image changes in the frames F511 to F513 are blend cut in step S636. Judge that it is a change.

  For example, as shown in FIG. 12G, the image of the frame F511 that is a person image and the image of the frame F513 that is a house image are not similar, but the image of the frame F512 is a combination of a person and a house. Since these are images, the images of the frames F511 and F512 are similar, the images of the frames F512 and F513 are similar, and the image obtained by combining the reduced images of the frames F511 and F513 is similar to the reduced image of the frame F512. It is determined that it is a blend cut change.

  On the other hand, if it is determined in step S632 that the second similarity 1 is smaller than the threshold value, in step S637, the determination unit 818 determines that the similarity 1 (third similarity 1) between the frame F512 and the frame F513 is in advance. It is judged whether it is larger than a predetermined threshold value that has been set. If it is determined in step S637 that the third similarity 1 is greater than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is smaller than the threshold, and the similarity 1 between the images of the frames F511 and F512 is If it is smaller than the threshold value and the similarity 1 between the images of the frames F512 and F513 is larger than the threshold value, in step S639, the determination unit 818 determines that the image change in the frames F511 to F513 is a normal cut change.

  For example, as shown in FIG. 12D, the image of the frame F511 that is a person image and the image of the frame F513 that is a house image are not similar, and the image of the frame F512 is a house image. If the images of F511 and F512 are not similar and the images of F512 and F513 are similar, it is determined that the image is a normal cut change.

  If it is determined in step S633 that the third similarity 1 is smaller than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is smaller than the threshold, and the similarity 1 between the images of the frames F511 and F512 is If it is larger than the threshold and the similarity 1 between the images of the frames F512 and F513 is smaller than the threshold, in step S639, the determination unit 818 determines that the image change in the frames F511 to F513 is a normal cut change.

  For example, as shown in FIG. 12E, the image of the frame F511 that is a person image and the image of the frame F513 that is a house image are not similar, and the image of the frame F512 is a person image. If the images of F511 and F512 are similar and the images of the frames F512 and F513 are not similar, it is determined that the image is a normal cut change.

  If it is determined in step S637 that the third similarity 1 is smaller than the threshold, in step S638, the determination unit 818 determines whether the similarity 2 is greater than a predetermined threshold. In step S638, when it is determined that the similarity 2 is larger than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is smaller than the threshold, and the similarity 1 between the images of the frames F511 and F512 is smaller than the threshold. When the similarity 1 between the frame F512 and the frame F513 is smaller than the threshold and the similarity 2 is larger than the threshold, in step S636, the determination unit 818 determines the image change in the frames F511 to F513 by blend cut change. Judge that there is.

  For example, as shown in FIG. 12G, the image of the frame F511 that is a person image and the image of the frame F513 that is a house image are not similar, and the image of the frame F512 is a combination of a person and a house. Even if it is determined that the images of the frames F511 and F512 and the images of the frames F512 and F513 are not similar to each other, an image obtained by combining the reduced images of the frames F511 and F513 and the frame F512 If the reduced images are similar, it is determined to be a blend cut change.

  On the other hand, when it is determined in step S638 that the similarity 2 is smaller than the threshold, that is, the similarity 1 between the images of the frames F511 and F513 is smaller than the threshold, and the similarity 1 between the frames F511 and F512 is smaller than the threshold. When the similarity 1 between the frames F512 and F513 is smaller than the threshold and the similarity 2 is smaller than the threshold, the determination unit 818 determines that the image change in the frames F511 to F513 is a normal cut change. .

  For example, as shown in FIG. 12F, the image of the frame F511 that is a human image and the image of the frame F513 that is a mountain image are not similar, and the image of the frame F512 is a house image. When the images of F511 and F512 are not similar, and the images of frames F512 and F513 are also not similar, the image obtained by combining the reduced images of frames F511 and F513 and the reduced image of frame F512 are similar. If not, it is determined to be a normal cut change.

  As described above, the determination unit 818 in FIG. 5 is based on the first to third similarity 1 supplied from the similarity calculation unit 813 and the similarity 2 supplied from the similarity calculation unit 817. It is determined whether the image change between the frames F511 to F513 is a normal cut change or a blend cut change.

  In addition, the determination unit 818 determines that each of the first to third similarity 1 and the similarity 2 is an appropriate threshold set for each of the first to third similarity 1 and the similarity 2. The threshold is determined to determine whether the change is large or small, and based on the combination of the threshold determination results for each of the first to third similarities 1 and 2, it is determined whether the change is a cut change. When the value is equal to the threshold value, it is determined as either large or small.

  FIG. 13 summarizes the relationship between the determination process of FIG. 11 and the image pattern of the three frames of FIG.

  In FIG. 13, when the similarity column is “large”, it indicates that the similarity is greater than a predetermined threshold, and when the similarity column is “small”, the similarity is less than the predetermined threshold. Represents that.

  In the second line from the top in FIG. 13, when the similarity 1 between the frames F511 and F513 is “large”, the cut is performed regardless of whether the other similarities are “large” or “small”. It is determined that it is not a change. An image pattern corresponding to such a case is shown in FIG. 12A or FIG. 12B, for example.

  In the third line from the top of FIG. 13, the similarity 1 between the frames F511 and F513 is “low”, the similarity 1 between the frames F511 and F512 is “high”, and the similarity 1 between the frames F512 and F513 is “high”. When the similarity 2 is “high”, it is determined that the blend cut change has occurred. An image pattern corresponding to such a case is shown in FIG. 12G, for example.

  In the fourth line from the top of FIG. 13, the similarity 1 between the frames F511 and F513 is “small”, the similarity 1 between the frames F511 and F512 is “large”, and the similarity 1 between the frames F512 and F513 is “high”. When the similarity 2 is “small”, it is determined that the cut change is not made. An image pattern corresponding to such a case is shown in FIG. 12C, for example.

  In the fifth line from the top of FIG. 13, the similarity 1 between the frames F511 and F513 is “low”, the similarity 1 between the frames F511 and F512 is “high”, and the similarity 1 between the frames F512 and F513 is “low”. ", It is determined that the cut is a normal cut change regardless of whether the degree of similarity 2 is" large "or" small ". An image pattern corresponding to such a case is shown in FIG. 12E, for example.

  In the sixth line from the top in FIG. 13, the similarity 1 between the frames F511 and F513 is “low”, the similarity 1 between the frames F511 and F512 is “low”, and the similarity 1 between the frames F512 and F513 is “high”. ", It is determined that the cut is a normal cut change regardless of whether the degree of similarity 2 is" large "or" small ". An image pattern corresponding to such a case is shown in FIG. 12D, for example.

  In the seventh line from the top in FIG. 13, the similarity 1 between the frames F511 and F513 is “small”, the similarity 1 between the frames F511 and F512 is “small”, and the similarity 1 between the frames F512 and F513 is “small”. When the similarity 2 is “high”, it is determined that the blend cut change has occurred. An image pattern corresponding to such a case is shown in FIG. 12G, for example.

  In the eighth line from the top in FIG. 13, the similarity 1 between the frames F511 and F513 is “small”, the similarity 1 between the frames F511 and F512 is “small”, and the similarity 1 between the frames F512 and F513 is “small”. When the similarity 2 is “small”, it is determined that the normal cut change is made. An image pattern corresponding to such a case is shown in FIG. 12F, for example.

  For example, a blend cut change that is an image change at the connection of an image section using an image effect accompanied by a continuous image change such as fade, dissolve, and wipe can also be detected as a cut change. It is possible to accurately detect the cut change including.

  Note that the determination unit 818 in FIG. 5 is based on the first to third similarities 1 and 2 using a statistical discrimination method such as a Bayes identification method, a neural network method, and a support vector machine method. The discriminator for discriminating whether it is a cut change may be provided.

  In each of the above-described processes, it has been described that an image is handled in units of frames. However, an image can be handled in units of fields.

  Further, in this specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

  The present invention can be configured by hardware in addition to software.

  The present invention can be applied to all detection devices that process images, such as broadcast equipment, image editing equipment, camcorders, personal computers for image processing, DVD recorders, and hard disk recorders.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure which shows the example of a normal cut change. It is a figure which shows the example of a blend cut change. It is a figure which shows the example of a quick motion of imaging | photography object. It is a block diagram which shows the structure of one Embodiment of the personal computer to which this invention is applied. It is a block diagram which shows the structure of one Embodiment of the cut change detection apparatus comprised with a personal computer. It is a flowchart explaining a cut change detection process. It is a figure explaining the production | generation of a synthesized image. It is a figure which shows the example of an original image. It is a figure which shows the example of a histogram. It is a figure explaining the overlapping rate of histograms. It is a flowchart explaining the determination process of step S606 of FIG. It is a figure which shows the relationship between an image pattern and a cut change. It is a figure which shows the relationship between the similarity 1 and the similarity 2, and the determination result of cut change.

Explanation of symbols

  1 personal computer, 21 CPU, 22 ROM, 23 RAM, 24 bus, 25 I / O interface, 26 input unit, 27 output unit, 28 storage unit, 29 communication unit, 30 drive, 31 removable media, 51 cut change detection device, 811 feature extraction unit, 812 feature amount buffer, 813 similarity calculation unit, 814 reduced image generation unit, 815 reduced image buffer, 816 composite image generation unit, 817 similarity calculation unit, 818 determination unit

Claims (3)

In a program for causing a computer to execute a detection process for detecting a cut change of an image,
Extracting features of the first image, the second image, and the third image;
Calculating the similarity of the features of two images of the first image, the second image, and the third image;
Generating reduced images of the first image, the second image, and the third image;
Generating a composite image of the reduced image of the first image and the third image located before or after in time with respect to the second image;
Calculating a similarity between the reduced image of the second image and the composite image;
Based on the similarity between the characteristics of two images of the first image, the second image, and the third image, and the similarity between the reduced image of the second image and the synthesized image, A program comprising a step of determining whether a cut change exists between a first image and the third image. In the detection method for detecting the cut change of the image,
Extracting features of the first image, the second image, and the third image;
Calculating the similarity of the features of two images of the first image, the second image, and the third image;
Generating reduced images of the first image, the second image, and the third image;
Generating a composite image of the reduced image of the first image and the third image located before or after in time with respect to the second image;
Calculating a similarity between the reduced image of the second image and the composite image;
Based on the similarity between the characteristics of two images of the first image, the second image, and the third image, and the similarity between the reduced image of the second image and the synthesized image, A detection method comprising a step of determining whether a cut change exists between the first image and the third image. In the detection device that detects the cut change of the image,
Feature extraction means for extracting features of the first image, the second image, and the third image;
First similarity calculation means for calculating the similarity of features of two images of the first image, the second image, and the third image;
Reduced image generating means for generating reduced images of the first image, the second image, and the third image;
A composite image generating means for generating a composite image of the reduced image of the first image and the third image located before or after the second image in time;
Second similarity calculation means for calculating the similarity between the reduced image of the second image and the composite image;
Based on the similarity between the characteristics of two images of the first image, the second image, and the third image, and the similarity between the reduced image of the second image and the synthesized image, A detection device comprising: determination means for determining whether a cut change exists between the first image and the third image.

Images