JP2008085539A - Program, detection method, detector, image processing method and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the erroneous detection of a cut change caused accompanying the minute fluctuation of luminance or a color. <P>SOLUTION: A fine histogram generation part 401 generates the respective fine histograms of the luminance or the color of a first image and a second image being the processing object of cut change detection and a feature distribution calculation part 402 calculates feature distributions by executing filter processing to the fine histograms. Then, a similarity degree calculation part 404 calculates the similarity degree of the respective feature distributions of the first image and the second image. A judgement part 405 judges whether or not the boundary of the first image and the second image is the cut change on the basis of the similarity degree of the feature distributions. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a program, a detection method, a detection device, an image processing method, and an image processing device, and in particular, a program, a detection method, a detection device, and an image processing method that enable highly accurate cut change detection. And an image processing apparatus.

  The process of detecting a cut change from an image is useful when performing image analysis, encoding, retrieval, or the like.

  Here, “cut” means a group of spatially continuous image segments (image clips), and “cut change” means a cut switching point, that is, temporally or spatially discontinuous image clips connected to each other. It means the change point of the image that is the bound.

  The cut change may be referred to as a shot change, a scene change, an image change point, or the like, but is referred to as a cut change in this specification.

  By the way, as a conventional cut change detection method, a statistic difference method, a pixel difference method, an encoded data method, an edge method, and the like are known, but it is recognized that the most accurate method is the histogram difference method. It is.

  In the histogram difference method, a histogram difference value is used.

  Here, in the histogram, each pixel of an image of a certain frame is divided into an appropriate number of gradations (generally 16 to 64 gradations are commonly used) corresponding to the luminance or color pixel value. By voting for each element (bin), the luminance or color frequency distribution of the image is obtained.

  The histogram difference method includes a simple histogram difference method and a divided histogram difference method.

  In the simple histogram difference method, for example, a difference between histograms of images of two frames to be processed is calculated as an evaluation amount (see, for example, Non-Patent Document 1).

  In the simple histogram difference method, generally, as the difference between histograms, the absolute sum of the difference values between the same bins of the histograms regarding the entire histogram (hereinafter referred to as the histogram difference absolute sum as appropriate) is calculated as the evaluation amount.

  In the divided histogram difference method, for example, the entire image of each of the two-frame images to be processed is divided into a predetermined number of blocks (for example, 16 blocks in Non-Patent Document 1), and the histograms of each block are shared. The difference between the histograms is calculated, and only a predetermined number of blocks are used for calculating the evaluation amount from the one with the smaller difference between the histograms (see, for example, Non-Patent Documents 1 and 2).

  When the histogram difference method (simple histogram difference method and divided histogram difference method) is used, there is a possibility that a minute change in luminance may be erroneously determined as a cut change.

  Next, with reference to FIG. 1 to FIG. 3, the relationship between the minute variation in luminance and the histogram when the histogram difference method is used will be described.

  FIG. 1 shows changes in the histogram when a cut change is correctly detected.

  In FIG. 1, frames F31 to F33 are a series of images that are temporally adjacent to each other, and an image of two frames in order from the frame F31 is a processing target for detecting a cut change. That is, first, the frame F31 and the frame F32 are processed, and then the frame F32 and the frame F33 are processed.

  Histograms H31 to H33 are luminance histograms relating to the entire image of the frames F31 to F33, respectively.

  In FIG. 1, the change in the image between the frame F31 and the frame F32 is mainly due to the movement of the person as the object to be photographed, so the distributions of the histogram H31 and the histogram H32 are almost the same.

  In this case, as the difference between the histogram H31 and the histogram H32, for example, if the histogram difference absolute sum of the histogram H31 and the histogram H32 is calculated, the histogram difference absolute sum becomes sufficiently small. Therefore, since the difference between the histogram H31 and the histogram H32 is sufficiently small, it is possible to correctly determine that the image change between the frame F31 and the frame F32 is not a cut change.

  On the other hand, since the subject to be photographed is completely different between the frame F32 and the frame F33 (the former is a person and the latter is a mountain), the histogram is between the histogram H32 and the histogram H33. It has changed greatly.

  In this case, as the difference between the histogram H32 and the histogram H33, for example, when calculating the absolute difference of the histogram difference between the histogram H32 and the histogram H33, the absolute difference of the histogram difference is sufficiently large. Since the difference is sufficiently large, a change in the image between the frame F32 and the frame F33 can be correctly determined as a cut change.

  FIG. 2 shows a change in the histogram when a slight change in luminance occurs.

  In FIG. 2, frames F41 to F45 are a series of temporally adjacent images, and two images in order from the frame F41 are set as processing targets for detecting cut changes. That is, first, the frame F41 and the frame F42 are processed, and then the frame F42 and the frame F43 are processed. Hereinafter, similarly, every two frames of images are processed.

  Histograms H41 to H45 are luminance histograms relating to the entire images of the frames F41 to F45, respectively.

  The frames F41 to F45 are, for example, monochromatic images, and the brightness of each image is the same throughout the image.

  In FIG. 2, the number of bins in the histogram is 64, and the bin width is 4 gradations. In addition, the bin division boundary is, for example, between 76 gradations and 77 gradations, between 80 gradations and 81 gradations, and the like. In the frames F41 to F45, minute luminance fluctuations 73, 75, 77, 79, and 81 occur, and the luminance of the entire image increases little by little. Further, since the brightness of the entire image of each of the frames F41 to F45 is the same, the frequencies of the histograms H41 to H45 are concentrated in one bin. The portion of the histogram where the frequency is concentrated in a certain bin is hereinafter referred to as a histogram peak as appropriate.

  The change in the image of the frames F41 to F45, that is, the minute change in luminance is a change in the image within the cut, not the cut change.

  In the frame F43, the brightness of the entire image is slightly increased from 75 to 77 as compared with the frame F42, and the brightness of the entire image straddles between one of 76 and 77 of the bin dividing boundary. Compared with the histogram H42, H43 is greatly changed so that the histogram peak moves to the bin on the right. For example, when the absolute difference of the histogram difference between the histogram H42 and the histogram H43 is calculated, the absolute difference of the histogram difference is the largest difference between the histograms, so that the switching point between the frame F42 and the frame F43 is mistaken for a cut change. It will be detected.

  Similarly, in the frame F45, the luminance of the entire image is slightly increased from 79 to 81 as compared with the frame F44, and the luminance of the entire image straddles between one of the bin dividing boundaries 80 and 81. Therefore, the histogram H45 changes so that the histogram peak moves to the right adjacent bin as compared with the histogram H44. For example, if the absolute difference of the histogram difference between the histogram H44 and the histogram H45 is calculated, the absolute difference of the histogram difference is the largest difference between the histograms. Therefore, the switching point between the frame F44 and the frame F45 is mistaken for a cut change. It will be detected.

  Such false detection of cut change is typically observed particularly when processing a single-color image, as described above. However, even when processing a general image that is a multi-color image, the luminance is detected. Alternatively, this is observed when the number of pixels straddling the boundary of the bins of the luminance or color histogram greatly changes with a slight color variation. As described above, the blurring of the difference between the histograms of the two images accompanying the slight change in the luminance or color between the two frames of the processing target causes a decrease in cut change detection accuracy. It is one.

  In Patent Document 1, by performing a process of matching the average value of the luminance of the entire images of the two images to be processed, the entire histogram of each of the two images is shifted in the direction of the luminance axis ( Hereinafter, a method for calculating the difference between the histograms after the histogram shift has been proposed. If this method is adopted in a relatively simple example as shown in FIG. 2, for example, the histograms after the histogram shift have almost the same shape and position, so that the blur between the histograms is absorbed, and erroneous detection is suppressed. Is done.

  However, in the case shown in FIG. 3, a false detection occurs. That is, FIG. 3 shows changes in the histogram when a slight luminance variation occurs in each of the left half and the right half of the image.

  In FIG. 3, a frame F51 and a frame F52 are two images to be processed that are temporally adjacent to each other, and a histogram H51 and a histogram H52 are histograms of luminance relating to the entire images of the frames F51 and F52, respectively.

  The left half region and the right half region of the frame F51 and the frame F52 are respectively monochrome images.

  In FIG. 3, the number of bins in the histogram is 64, and the bin width is 4 gradations. In addition, the bin division boundary is, for example, between 76 gradations and 77 gradations, between 80 gradations and 81 gradations, and the like.

  In FIG. 3, the luminances of the left half and the right half of the frame F51 are 75 and 81, respectively, and the luminances of the left half and the right half of the frame F52 are 77 and 83, respectively. Accordingly, the histogram H51 has a histogram peak in each of the bins including 75 and 81, and the frame F52 has a histogram peak in each of the bins including 77 and 83.

  A change in the image between the frames F51 and F52, that is, a minute change in luminance is a change in the image within the cut, and therefore the boundary between the frames F51 and F52 is not a cut change.

  The left half of the frame F52 is slightly increased from 75 to 77 in luminance compared to the left half of the frame F51, and the luminance of the left half of the image is between 76 and 77, which is one of the bin dividing boundaries. It straddles between. On the other hand, in the right half of the frame F52, the luminance of the entire image is slightly increased from 81 to 83 compared with the right half of the frame F51, but does not straddle the bin dividing boundary. Therefore, between the histogram H51 and the histogram H52, only the left histogram peak moves to the bin on the right side across the division boundaries 76 and 77. As a result, for example, when calculating the absolute difference of the histogram difference between the histogram H51 and the histogram H52, if the absolute difference of the histogram difference exceeds a predetermined threshold value, the change in the image between the frame F51 and the frame F52 is erroneously detected as a cut change. It will be.

  In this way, when the shape of the histogram changes with a slight change in luminance, it is difficult to absorb the blur due to the difference between the histograms only by the histogram shift. That is, the histogram shift does not function effectively if the change in the image is slightly more complicated than that shown in FIG.

  Therefore, Patent Document 2 proposes a method of voting not only to a bin for voting but also to a bin adjacent to the bin at a predetermined rate when generating a histogram.

  When such a method is used, if the amount of variation in the brightness of the entire image is sufficiently small relative to the width of the bin of the histogram, an effect of reducing the blurring of the difference between the histograms can be expected. If the amount of variation is not negligible relative to the bin width of the histogram, blurring of differences between histograms cannot be absorbed only by voting on adjacent bins.

IPSJ Journal, Nagasaka Goro, Tanaka Joe, April 1992, Vol.33, No.4, P.543-550 Comparison of Video Shot Boundary Detection Techniques, John S. Boreczky, Lawrence A. Rowe.Storage and Retrieval for Image and Video Databases (SPIE) (1996) p170-179 JP 2006-121274 A JP 2004-282318 A

  As described above, with the histogram difference method, it has been difficult to reduce false detection of cut changes caused by minute variations in luminance or color.

  The present invention has been made in view of such a situation, and is intended to reduce false detection of cut changes caused by minute variations in luminance or color.

  A program or a detection method according to a first aspect of the present invention is a program or a detection method for causing a computer to execute a detection process for detecting a cut change of an image, and for each luminance or color of the first image and the second image. Generating a fine histogram, filtering the fine histogram to calculate a feature distribution, calculating a similarity of the feature distribution of each of the first image and the second image, and calculating the feature distribution And determining whether a boundary between the first image and the second image is a cut change based on the similarity.

  The program according to the first aspect of the present invention further includes a step of performing a thinning process on the feature distribution, and the similarity calculation step includes the thinning of each of the first image and the second image. The degree of similarity of the feature distribution after processing can be calculated.

  In the similarity calculation step, the correspondence relationship of the samples is determined so that the shape of the feature distribution of each of the first image and the second image most closely matches, and the sample corresponding by the correspondence relationship A step of calculating the degree of similarity of the feature distribution of each of the first image and the second image by comparing each other can be provided.

  The inspection apparatus according to the first aspect of the present invention is a detection apparatus for detecting cut change of an image, a fine histogram generation means for generating a fine histogram of luminance or color of each of the first image and the second image, Feature distribution calculating means for calculating a feature distribution by performing filtering on the fine histogram; similarity calculating means for calculating the similarity of the feature distribution of each of the first image and the second image; And determining means for determining whether a boundary between the first image and the second image is a cut change based on the similarity of the feature distribution.

  A program or an image processing method according to a second aspect of the present invention is a program or an image processing method for causing a computer to execute image processing for calculating similarity between images. A fine color histogram is generated, a feature distribution is calculated by filtering the fine histogram of the first image and the second image, and each of the first image and the second image is calculated. Calculating the similarity of the feature distribution.

  The image processing apparatus according to the second aspect of the present invention is a fine histogram generating means for generating fine histograms of luminance or color of each of the first image and the second image in the image processing apparatus for calculating the similarity of the images. Each of the first image and the second image, a feature distribution calculating unit that performs a filtering process on the fine histograms of the first image and the second image to calculate a feature distribution; Similarity calculating means for calculating the similarity of the feature distribution;

  According to a third aspect of the present invention, there is provided a program or an image processing method for detecting whether an input image boundary is a cut change in a program or an image processing method for causing a computer to execute image processing for grouping image cuts. Generating a fine histogram of luminance or color of each image constituting each of the detected first cut and second cut, and for each of the fine histograms of the first cut and the second cut Applying filter processing to calculate the feature distribution, integrating the feature distribution of each image constituting the first cut, and integrating the feature distribution of each image constituting the second cut, Calculating the similarity between the feature distribution accumulated in the first cut and the feature distribution accumulated in the second cut; Based on the similarity, comprising the step of grouping the first cut and the second cut.

  In the detection step of detecting the cut change, a fine histogram of brightness or color of each of the first image and the second image is generated, and the fine histogram of the first image and the second image is generated. Each of the first image and the second image is calculated to calculate a feature distribution by filtering each of the first image and the second image. Based on the similarity of the feature distribution, the first distribution is calculated. A step of determining whether a boundary between the first image and the second image is a cut change may be provided, and the first cut and the second cut may be cuts separated by the cut change.

  An image processing apparatus according to a third aspect of the present invention is an image processing apparatus for grouping cuts of images. The detection means detects whether the boundary of the input image is a cut change, and the detected first cut. And fine histogram generating means for generating a fine histogram of luminance or color of each image constituting each of the second cut, and filtering processing for the fine histogram of the first cut and the second cut, respectively. A feature distribution calculating means for calculating the feature distribution and integrating the feature distribution of each image constituting the first cut, and integrating the feature distribution of each image constituting the second cut, respectively. The feature distribution integrating means, the feature distribution obtained by integrating the first cut, and the feature distribution obtained by integrating the second cut It comprises a similarity calculation means for calculating, on the basis of the similarity, and a grouping means for grouping the first cut and the second cut.

  In the first aspect of the present invention, fine histograms of luminance or color of the first image and the second image are generated, and the feature distribution is calculated by performing filtering on the fine histogram. Also, the similarity between the feature distributions of the first image and the second image is calculated. Based on the similarity between the feature distributions, it is determined whether the boundary between the first image and the second image is a cut change. Determined.

  In the second aspect of the present invention, fine histograms of luminance or color of the first image and the second image are generated, and filtering processing is performed on the fine histograms of the first image and the second image. To calculate the feature distribution. Then, the similarity between the feature distributions of the first image and the second image is calculated.

  In the third aspect of the present invention, it is detected whether the boundary of the input image is a cut change, and the brightness or color of each image constituting each of the detected first cut and second cut is detected. A fine histogram is generated, and a feature distribution is calculated by filtering each of the fine histograms of the first cut and the second cut. In addition, the feature distribution of each image constituting the first cut is integrated, the feature distribution of each image constituting the second cut is integrated, and the feature distribution of the first cut is integrated, A similarity with the integrated feature distribution of the second cut is calculated. Then, based on the similarity, the first cut and the second cut are grouped.

  According to the first aspect of the present invention, erroneous detection of cut change can be reduced. In particular, it is possible to reduce false detection of cut changes caused by minute variations in luminance or color.

  According to the second aspect of the present invention, the similarity between images can be obtained. In particular, it is possible to obtain the similarity between images even if a slight change in luminance or color occurs.

  According to the third aspect of the present invention, cuts can be grouped easily and reliably. In particular, even if a slight change in luminance or color occurs, cuts can be grouped easily and reliably.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

A program or a detection method according to the first aspect of the present invention includes:
In a program or a detection method for causing a computer to execute a detection process for detecting an image cut change,
A fine histogram of brightness or color of each of the first image and the second image is generated (for example, step S401 in FIG. 6),
A feature distribution is calculated by performing filtering on the fine histogram (for example, step S402 in FIG. 6),
Calculating the similarity of the feature distribution of each of the first image and the second image (for example, step S404 in FIG. 6);
Based on the similarity of the feature distribution, it is determined whether the boundary between the first image and the second image is a cut change (for example, step S405 in FIG. 6).
Comprising steps.

The program according to the first aspect of the present invention includes:
A step of performing a thinning process on the feature distribution (for example, step S403 in FIG. 6) is further provided,
In the similarity calculation step, the similarity of the feature distribution after the thinning process of each of the first image and the second image can be calculated.

The similarity calculation step includes
A sample correspondence relationship is determined so that the shape of the feature distribution of each of the first image and the second image best matches (for example, step S421 in FIG. 11),
The similarity of the feature distribution of each of the first image and the second image is calculated by comparing the samples corresponding to each other in the correspondence relationship (for example, step S422 in FIG. 11).
Steps can be provided.

The inspection apparatus according to the first aspect of the present invention includes:
In a detection device (for example, the personal computer 1 in FIG. 4) for detecting an image cut change,
Fine histogram generation means (fine histogram generator 401 in FIG. 5) for generating fine histograms of luminance or color of each of the first image and the second image;
A feature distribution calculating unit (for example, the feature distribution calculating unit 402 in FIG. 5) that performs filtering on the fine histogram to calculate a feature distribution;
Similarity calculation means (for example, the similarity calculation unit 404 in FIG. 5) for calculating the similarity of the feature distribution of each of the first image and the second image;
And a determination unit (for example, a determination unit 405 in FIG. 5) that determines whether a boundary between the first image and the second image is a cut change based on the similarity of the feature distribution.

A program or an image processing method according to the second aspect of the present invention includes:
In a program or an image processing method for causing a computer to execute image processing for calculating similarity between images,
A fine histogram of the luminance or color of each of the first image and the second image is generated (for example, step S431 in FIG. 15),
A feature distribution is calculated by filtering the fine histograms of the first image and the second image (for example, step S432 in FIG. 15),
The similarity of the feature distribution of each of the first image and the second image is calculated (for example, step S434 in FIG. 15).
Comprising steps.

The image processing apparatus according to the second aspect of the present invention provides:
In an image processing apparatus that calculates the similarity of images (for example, the similarity calculation apparatus 461 in FIG. 14),
Fine histogram generation means (for example, fine histogram generator 471 and fine histogram generator 474 in FIG. 14) that generates fine histograms of luminance or color of each of the first image and the second image;
Feature distribution calculating means for calculating a feature distribution by filtering the fine histograms of the first image and the second image (for example, the feature distribution calculating unit 472 and the feature distribution calculating unit 475 in FIG. 14). When,
A similarity calculation unit (for example, a similarity calculation unit 477 in FIG. 14) that calculates the similarity of the feature distribution of each of the first image and the second image.

A program or an image processing method according to the third aspect of the present invention includes:
In a program or an image processing method for causing a computer to execute image processing for grouping image cuts,
It is detected whether the boundary of the input image is a cut change (for example, steps S471 to S477 in FIG. 18),
A fine histogram of luminance or color of each image constituting each of the detected first cut and second cut is generated (for example, step S478 in FIG. 19),
A feature distribution is calculated by filtering each of the fine histograms of the first cut and the second cut (for example, step S479 in FIG. 19),
The feature distributions of the images constituting the first cut are respectively integrated, and the feature distributions of the images constituting the second cut are respectively integrated (for example, step S481 in FIG. 19).
Calculating a similarity between the feature distribution obtained by integrating the first cut and the feature distribution obtained by integrating the second cut (for example, step S483 in FIG. 19);
Based on the similarity, the first cut and the second cut are grouped (for example, step S484 in FIG. 19).
Comprising steps.

In the detection step of detecting the cut change,
A fine histogram of each luminance or color of the first image and the second image is generated (for example, step S471 in FIG. 18),
Filtering the fine histograms of the first image and the second image, respectively, to calculate a feature distribution (for example, step S472 in FIG. 18),
Calculating the similarity of the feature distribution of each of the first image and the second image (for example, step S474 in FIG. 18);
Based on the similarity of the feature distribution, it is determined whether the boundary between the first image and the second image is a cut change (for example, steps S475 to S477 in FIG. 18).
Provided steps,
The first cut and the second cut may be cuts separated by the cut change.

The image processing apparatus according to the third aspect of the present invention is:
In an image processing apparatus (for example, the image grouping apparatus 501 in FIG. 17) that groups image cuts,
Detection means (for example, the cut change detection unit 514 in FIG. 17) for detecting whether the boundary of the input image is a cut change;
Fine histogram generation means (for example, a fine histogram generation unit 511 in FIG. 17) for generating a fine histogram of luminance or color of each image constituting each of the detected first cut and second cut;
A feature distribution calculating unit (for example, a feature distribution calculating unit 512 in FIG. 17) that performs a filtering process on the fine histograms of the first cut and the second cut, respectively, to calculate a feature distribution;
Feature distribution integration means (for example, the feature distribution of FIG. 17) that integrates the feature distribution of each image that constitutes the first cut and that accumulates the feature distribution of each image that constitutes the second cut. Integration unit 515),
Similarity calculation means (for example, the similarity calculation unit 517 in FIG. 17) that calculates the similarity between the feature distribution accumulated in the first cut and the feature distribution accumulated in the second cut. ,
Grouping means (for example, a grouping unit 518 in FIG. 17) for grouping the first cut and the second cut based on the similarity.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 4 is a block diagram showing a configuration example of an embodiment of a detection apparatus to which the present invention is applied.

  In FIG. 4, a personal computer 1 as a detection device includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a bus 24, an input / output interface 25, an input unit 26, and an output. Unit 27, storage unit 28, communication unit 29, drive 30, and removable medium 31.

  The CPU 24, ROM 22, RAM 23, and input / output interface 25 are connected to the bus 24, and the bus 24, input unit 26, output unit 27, storage unit 28, communication unit 29, and drive 30 are connected to the input / output interface 25. It is connected.

  The CPU 21 executes various processes according to programs stored in the ROM 22 or the storage unit 28. Further, the CPU 21 executes various processes in response to commands input from the input unit 26, and outputs processing results to the output unit 27.

  The ROM 22 stores a program executed by the CPU 21 and the like.

  The RAM 23 appropriately stores programs executed by the CPU 21 and data. The RAM 23 includes a buffer that temporarily stores an image input from the outside, for example.

  The input unit 26 includes a keyboard, a mouse, a microphone, and the like.

  The output unit 27 includes a display, a speaker, and the like.

  The storage unit 28 includes, for example, a hard disk and stores programs executed by the CPU 21 and various data.

  The communication unit 29 communicates with an external device via a network such as the Internet or a local area network.

  A program may be acquired via the communication unit 29 and stored in the storage unit 28.

  When a removable medium 31 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is loaded, the drive 30 drives them to acquire programs and data recorded therein. The acquired program and data are transferred to and stored in the storage unit 28 as necessary.

  As shown in FIG. 4, a program recording medium that stores a program that is installed in a computer and can be executed by the computer is a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, or a removable medium 31 that is a package medium composed of a semiconductor memory, ROM 22, and a hard disk that constitutes the storage unit 28. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 29 that is an interface such as a router or a modem as necessary. Done.

  The CPU 21 of the personal computer 1 functions as a cut change detection device that detects a cut change by executing a program stored in the ROM 22 or the storage unit 28.

  FIG. 5 is a block diagram showing a configuration of an embodiment of a cut change detection device 51 configured by software of the personal computer 1.

  In FIG. 5, the cut change detection device 51 inputs an external image by reading it from, for example, a buffer of the RAM 23 and performs a cut change detection process.

  In FIG. 5, the cut change detection device 51 includes a fine histogram generation unit 401, a feature distribution calculation unit 402, a thinning processing unit 403, a similarity calculation unit 404, and a determination unit 405.

  The fine histogram generation unit 401 is supplied with the image input to the cut change detection device 51. The fine histogram generation unit 401 generates a fine histogram which is a histogram having the smallest possible division width, for example, one gradation division width in the case of 256 gradations, as a histogram of an image supplied thereto, and a feature distribution It supplies to the calculation part 402.

  The feature distribution calculation unit 402 performs filter processing on the fine histogram supplied from the fine histogram generation unit 401 and supplies the resulting feature distribution to the thinning processing unit 403. For the filter processing by the feature distribution calculation unit 402, it is desirable to use an appropriately designed low-pass filter, but a filter having a non-linear response may be used.

  The decimation processing unit 403 performs decimation processing on the feature distribution supplied from the feature distribution calculation unit 402 at intervals corresponding to sampling frequencies in a range in which the feature distribution is faithfully reproduced (permitted by the sampling theorem). And the feature distribution after the thinning process obtained as a result is supplied to the similarity calculation unit 404.

  Note that the thinning processing by the thinning processing unit 403 is mainly processing for reducing the amount of data to save the calculation amount and the storage capacity, and may be omitted if not necessary.

  The similarity calculation unit 404 compares the feature distribution of the first image after the thinning process supplied from the thinning processing unit 403 with the feature distribution of the second image after the thinning process, and performs the first after the thinning process. The similarity between the feature distribution of the image and the feature distribution of the second image after the thinning process is calculated.

  Similarity between feature distributions is obtained by obtaining the distance between the feature distribution of the first image and, for example, the feature distribution of the second image in the frame before the first image, as in the normal histogram difference method. The similarity between feature distributions may be increased as the distance decreases.

  The similarity calculation unit 404 supplies the similarity to the determination unit 405.

  The determination unit 405 performs threshold determination to determine whether the similarity supplied from the similarity calculation unit 404 is larger or smaller than an appropriate threshold set for the similarity, and is a cut change. Determine whether.

  Next, the cut change detection process performed by the cut change detection device 51 of FIG. 5 will be described with reference to the flowchart of FIG.

  An image input to the cut change detection device 51 is supplied to the fine histogram generation unit 401 in FIG. In step S <b> 401, the fine histogram generation unit 401 generates a fine histogram of the image supplied thereto and supplies it to the feature distribution calculation unit 402.

  Now, as shown in FIG. 7A, it is assumed that frames F211 and F212 are images to be processed. These images are images having the same luminance distribution as the frames F51 and F52 of FIG. That is, the left half of the image of the frame F211 has a luminance of 75 and the right half has a luminance of 81, the left half of the image of the frame F212 has a luminance of 77, and the right half has a luminance of 83. The fine histogram H211 and the fine histogram H212 in FIG. 7B are fine histograms generated from the images of the frames F211 and F212, respectively. The fine histogram H211 has histogram peaks at luminances 75 and 81, and the fine histogram H212 has histogram peaks at luminances 77 and 83, respectively.

  In the fine histogram H211 of FIG. 7, if the input image is an N-gradation image, it is desirable that the bin division number be N stages. In that case, the discontinuity in the calculation of the histogram generation described later remains at the quantization level that the image originally has, and no new error is introduced. From the viewpoint of saving computation amount and storage capacity, the number of bin divisions can be less than N. However, the coarser the division, the higher the histogram generation as in the general histogram difference method. An error caused by the discontinuity of the operation is introduced.

  In step S <b> 402, the feature distribution calculation unit 402 performs filtering processing on the fine histogram supplied from the fine histogram generation unit 401, and supplies the resulting feature distribution to the thinning processing unit 403.

  The feature distribution C211 in FIG. 7C is obtained by performing filtering on the fine histogram H211, and the feature distribution C212 is obtained by performing filtering on the fine histogram H212.

  For the filtering process, it is desirable to use a low-pass filter as shown in FIG. By using a low-pass filter, the difference in histograms due to minute fluctuations in luminance can be eliminated without introducing new errors due to discontinuity in the calculation of histograms, as in the case of voting on adjacent bins. Can absorb blur.

  In addition, since the high-frequency component of the feature distribution obtained by the filter processing using the low-pass filter is reduced compared to the histogram, thinning processing is performed within a range that complies with the sampling theorem, so that the subsequent calculation amount is kept low. The error due to the filtering process or the thinning-out process can be prevented from being introduced, or can be suppressed to a very small error.

  Therefore, by adopting the above-described method, it is possible to calculate the similarity between target images using a feature distribution from which an error due to the discontinuity of the histogram generation operation inherent in the histogram difference method is eliminated. Therefore, it is possible to detect cut change with higher accuracy.

  In the filter processing by the feature distribution calculation unit 402, in addition to the above-described low-pass filter, a nonlinear filter that is a filter having a nonlinear response can be used.

  For example, a nonlinear filter that compresses or expands the feature value axis logarithmically by taking the logarithm of the feature distribution can be used.

  FIG. 9 shows a feature distribution when the above-described processing using the nonlinear filter is performed on an image in which the feature distribution is concentrated at a specific luminance.

  FIG. 9A shows an image to be filtered, FIG. 9B shows a feature distribution when the image is processed by a linear filter, and FIG. 9C shows a feature distribution when the image is processed by a nonlinear filter. Indicates.

  The original image in FIG. 9A is an image in which the foreground of a simple color object is positioned in front of a complex background.

  In the feature distribution of FIG. 9B, the feature distribution is concentrated on the brightness of the simple color object. In this case, as described above, the luminance variation of the simple color object becomes excessively dominant in the detection result of the cut change.

  On the other hand, in the feature distribution of FIG. 9C, since the luminance that is too large is suppressed by non-linear filtering, it is possible to suppress the feature distribution from being concentrated on a specific luminance.

  In step S <b> 403, the thinning processing unit 403 performs thinning processing on the feature distribution supplied from the feature distribution calculation unit 402, for example, every four gradations, and resembles the feature distribution after the thinning processing obtained as a result. This is supplied to the degree calculation unit 404. By performing the thinning process on the feature distribution C211 and the feature distribution C212 in FIG. 7C, the feature distribution D211 and the feature distribution D212 in FIG. 7C are obtained, respectively.

  In step S404, the similarity calculation unit 404 obtains the feature distribution of the first image after the thinning process and the feature distribution of the second image (image of the previous frame) after the thinning process supplied from the thinning processing unit 403. The comparison is performed, and similarity calculation processing is performed to calculate the similarity between the feature distribution of the first image after the thinning process and the feature distribution of the second image after the thinning process.

  In order to execute the similarity calculation process, the similarity calculation unit 404 of FIG. 5 is configured as shown in FIG. That is, the similarity calculation unit 404 includes a corresponding sample determination unit 431 and a similarity determination unit 432.

  The corresponding sample determination unit 431 compares the feature distribution after the thinning process of the first image supplied from the thinning processing unit 403 with the feature distribution after the thinning process of the second image, and compares the first image with the second image. Correspondences between samples are determined so that the shapes of the respective feature distributions of the images of the images match best, and correspondence information representing the correspondences is supplied to the similarity determination unit 432.

  The similarity determination unit 432 obtains, for example, the distance between the feature distribution of the first image and the feature distribution of the second image based on the correspondence relationship information supplied from the correspondence sample determination unit 431, and the smaller the distance is, the smaller the distance is. The feature distribution similarity that is a higher similarity is supplied to the determination unit 405.

  With reference to the flowchart of FIG. 11, the similarity calculation processing by the similarity calculator 404 of FIG. 10 will be described.

  In step S421, the corresponding sample determination unit 431 compares the feature distribution after the thinning process of the first image and the feature distribution after the thinning process of the second image, which are supplied from the thinning processing unit 403, The correspondence relationship between samples is determined as shown in FIG. 12 so that the shapes of the feature distributions of the image and the second image most closely match, and correspondence information representing the correspondence relationship is determined as a similarity. Supplied to the unit 432. Here, the sample represents a feature distribution element (bin).

  The upper diagram of FIG. 12 shows the feature distribution of the frame F231 as the first image, and the lower diagram of FIG. 12 shows the feature distribution of the frame F232 as the second image.

  In FIG. 12, by matching the samples of the feature distribution of the frame F231 and the feature distribution of the frame F232 as indicated by the arrows, the shapes of the two feature distributions most closely match each other.

  Examples of the method for obtaining the correspondence between feature distributions include a method for obtaining the maximum correlation between the feature distributions, a method for obtaining the square error between the feature distributions, or a method for obtaining the correspondence between the feature distributions. There is a method for obtaining the centers of gravity to coincide.

  Alternatively, since the shape of the feature distribution is generally distorted due to a change in luminance, the correspondence relationship may be determined using a non-linear method such as a DP (Dynamic Programming) matching technique.

  In step S422, the similarity determination unit 432 obtains, for example, the distance between the feature distribution of the first image and the feature distribution of the second image based on the correspondence relationship information supplied from the corresponding sample determination unit 431, A feature distribution similarity that is a similarity that increases as the distance decreases is supplied to the determination unit 405.

  Returning to FIG. 6, next, in step S <b> 405, the determination unit 405 sets the similarity (feature distribution similarity) supplied from the similarity calculation unit 404 to the appropriate threshold set for the similarity. A comparison is made to determine whether it is a cut change.

  In addition, the determination unit 405 uses other so-called statistical discrimination methods such as a Bayes identification method that obtains the probability of an event according to Bayes' theorem, a neural network method, and a support vector machine method that applies statistical learning theory. It may be determined whether it is a cut change.

  As described above, in the histogram difference method, there is a possibility that a cut change is erroneously detected when a minute luminance fluctuation occurs. The reason is that a bin separator is sandwiched when obtaining a histogram. It is in performing discontinuous calculation. When the method of Patent Document 2 is used, the discontinuity of this operation can be reduced, so that erroneous detection of cut change can be suppressed to some extent, but the effect of the bin break itself has not been solved, It will be limited. On the other hand, in the present embodiment, since the samples are associated so that the shape of the feature distribution matches, it is possible to reduce the influence of the bin separation, and therefore, it is possible to reduce false detection of cut change. .

  By the way, a request to “arrange still images such as photographs in the order of similarity”, that is, a similarity between one image and another image among a plurality of images is obtained, and the plurality of images are determined. When there is a request to arrange them in order of high similarity, for example, the similarity between two images shown in FIG. 13 as one image and another image is quantitatively evaluated. There is a need to.

  FIG. 14 is a block diagram showing a configuration of an embodiment of a similarity calculation apparatus to which the present invention for calculating the similarity of two images for such a case is applied.

  In FIG. 14, the similarity calculation device 461 includes a fine histogram generation unit 471, a feature distribution calculation unit 472, a thinning processing unit 473, a fine histogram generation unit 474, a feature distribution calculation unit 475, a thinning processing unit 476, and a similarity calculation unit. 477.

  In FIG. 14, the fine histogram generation unit 471, the feature distribution calculation unit 472, and the thinning processing unit 473 are configured similarly to the fine histogram generation unit 401, the feature distribution calculation unit 402, and the thinning processing unit 403 in FIG. . Further, the fine histogram generation unit 474, the feature distribution calculation unit 475, and the thinning processing unit 476 are configured similarly to the fine histogram generation unit 401, the feature distribution calculation unit 402, and the thinning processing unit 403 in FIG. Furthermore, the similarity calculation unit 477 is configured similarly to the similarity calculation unit 404 in FIG.

  The similarity calculation unit 477 acquires the feature distribution of the first image after the thinning process from the thinning processing unit 473, and acquires the feature distribution of the second image after the thinning process from the thinning processing unit 476. To do. Then, the similarity calculation unit 477 calculates the similarity between the feature distribution of the first image after the thinning process and the feature distribution of the second image after the thinning process, and the similarity between the first image and the second image Is output as the degree of image similarity.

  Next, still image similarity calculation processing by the similarity calculation device 461 in FIG. 14 will be described with reference to the flowchart in FIG.

  The fine histogram generation unit 471 and the fine histogram generation unit 474 are supplied with images input from the outside, that is, a first image and a second image to be processed for calculating similarity.

  In step S431, the fine histogram generator 471 and the fine histogram generator 474 generate fine histograms of the first image and the second image supplied thereto, respectively, and the feature distribution calculator 472 and the feature distribution calculator 475. To supply.

  In step S432, the feature distribution calculation unit 472 and the feature distribution calculation unit 475 respectively filter the fine histogram of the first image and the fine histogram of the second image supplied from the fine histogram generation unit 471 and the fine histogram generation unit 474, respectively. Processing is performed, and the characteristic distribution obtained as a result is supplied to the thinning processing unit 473 and the thinning processing unit 476.

  In step S433, the thinning processing unit 473 and the thinning processing unit 476 perform thinning on the feature distribution of the first image and the feature distribution of the second image supplied from the feature distribution calculation unit 472 and the feature distribution calculation unit 475. Processing is performed, and the feature distribution of the first image after the thinning process and the feature distribution of the second image after the thinning process obtained as a result are supplied to the similarity calculation unit 477.

  In step S434, the similarity calculation unit 477 calculates the similarity between the feature distribution of the first image after the thinning process and the feature distribution of the second image after the thinning process, and the first image and the second image Are output as image similarity.

  As described above, the similarity between images can be easily and reliably obtained even when a slight change in luminance or color occurs.

  An image can be understood as a group of cuts in which a plurality of cuts are combined. Among the plurality of cuts, there may be a plurality of similar cuts such as cuts taken at the same place or at similar angles, such as cuts 1 and 3 or cuts 2 and 5 in FIG. In some cases, it is desirable to automatically group such similar cuts.

  FIG. 17 is a block diagram showing the configuration of an embodiment of an image grouping apparatus to which the present invention is applied for automatically grouping images in this way.

  In FIG. 17, an image grouping apparatus 501 includes a fine histogram generation unit 511, a feature distribution calculation unit 512, a thinning processing unit 513, a cut change detection unit 514, a feature distribution integration unit 515, a storage unit 516, a similarity calculation unit 517, and A grouping unit 518 is configured.

  In FIG. 17, a fine histogram generation unit 511, a feature distribution calculation unit 512, and a thinning processing unit 513 are configured in the same manner as the fine histogram generation unit 401, the feature distribution calculation unit 402, and the thinning processing unit 403 in FIG. The cut change detection unit 514 is configured similarly to the cut change detection device 51 in FIG. The similarity calculation unit 517 is configured similarly to the similarity calculation unit 404 in FIG.

  An image input from the outside is supplied to the fine histogram generator 511.

  The fine histogram generation unit 511 generates a fine histogram of the image supplied thereto and supplies it to the feature distribution calculation unit 512. The feature distribution calculation unit 512 performs filtering on the fine histogram supplied from the fine histogram generation unit 511 and supplies the resulting feature distribution to the thinning processing unit 513.

  The thinning processing unit 513 performs thinning processing for thinning out the feature distribution supplied from the feature distribution calculating unit 512 at a predetermined interval, and the resulting feature distribution after thinning processing is sent to the feature distribution integration unit 515. Supply.

  The cut change detection unit 514 is supplied with an image input from the outside. The cut change detection unit 514 is configured similarly to the cut change detection device 51 in FIG.

  That is, the cut change detection unit 514 performs a predetermined process on the image supplied thereto, thereby calculating the similarity between the first image and the second image to be processed for detecting the cut change. Based on the similarity between the first image and the second image, it is determined whether the boundary between the first image and the second image is a cut change that is a boundary accompanied by temporally discontinuous image changes. Then, determination result information representing the determination result is supplied to the feature distribution integration unit 515.

  The feature distribution integration unit 515 determines that the same cut is continuing when the determination result information supplied from the cut change detection unit 514 indicates that it is not a cut change, and performs the thinning process supplied from the thinning processing unit 513. The subsequent feature distribution is integrated into the stored integrated feature distribution, and the integrated feature distribution after the integration is stored. In addition, when the determination result information supplied from the cut change detection unit 514 indicates a cut change, the feature distribution integration unit 515 determines that the cut change has occurred, and stores the integrated feature distribution obtained by the integration so far. The integrated feature distribution supplied to 516 and held is reset once, for example.

  The storage unit 516 stores the integrated feature distribution supplied from the feature distribution integration unit 515. In this way, the integrated feature distribution for each cut is stored in the storage unit 516.

  The similarity calculation unit 517 reads from the storage unit 516 the integrated feature distributions of the first cut and the second cut that are the target of the image grouping process, and the integrated feature distribution and the second cut of the first cut. And the integrated feature distribution similarity, which is the similarity between the integrated feature distribution of the first cut and the integrated feature distribution of the second cut, is calculated and supplied to the grouping unit 518.

  Based on the integrated feature distribution similarity supplied from the similarity calculation unit 517, the grouping unit 518 determines, for example, whether the first cut and the second cut are in the same group. The grouping unit 518 groups the cuts supplied from the similarity calculation unit 517 in the same manner.

  Next, image grouping processing by the image grouping apparatus 501 in FIG. 17 will be described with reference to the flowcharts in FIGS. 18 and 19.

  The image grouping process is started, for example, when an image is supplied to the image grouping apparatus 501 from the outside.

  In step S471, the fine histogram generation unit 401 of the cut change detection unit 514 generates a fine histogram of the image supplied thereto, and supplies the fine histogram to the feature distribution calculation unit 402 of the cut change detection unit 514.

  In step S472, the feature distribution calculation unit 402 of the cut change detection unit 514 performs a filtering process on the fine histogram supplied from the fine histogram generation unit 401, and supplies the resulting feature distribution of the image to the thinning processing unit 513. To do.

  In step S473, the thinning processing unit 403 of the cut change detection unit 514 performs thinning processing for thinning at a predetermined interval on the feature distribution supplied from the feature distribution calculation unit 402, and after the thinning processing obtained as a result thereof Is supplied to the feature distribution integration unit 515.

  In step S474, the similarity calculation unit 404 configured as shown in FIG. 10 of the cut change detection unit 514 performs the similarity calculation process, that is, the feature distribution of the image after the thinning process and the previous frame after the thinning process. The similarity of the feature distribution of the images is calculated. The processes in steps S471 through S474 are the same as the processes in steps S401 through S404 in FIG.

  In step S475, the determination unit 405 of the cut change detection unit 514 determines whether the similarity supplied from the similarity calculation unit 404 is equal to or less than a predetermined threshold. When it is determined that the similarity is equal to or less than the predetermined threshold value, in step S476, the determination unit 405 determines that the first image (current frame image) and the second image (previous frame) to be processed for detecting cut change. The boundary of (image) is determined to be a cut change that is a boundary accompanied by temporally discontinuous image changes.

  As a result, the cut change detection unit 514 supplies determination result information indicating a cut change to the feature distribution integration unit 515.

  On the other hand, when it is determined in step S475 that the degree of similarity is not equal to or less than the predetermined threshold value, in step S477, the determination unit 405 determines the boundary between the first image and the second image to be processed for detecting a cut change. Is determined not to be a cut change.

  Thereby, the cut change detection unit 514 supplies determination result information indicating that it is not a cut change to the feature distribution integration unit 515.

  In step S478, the fine histogram generation unit 511 generates a fine histogram of the image supplied thereto and supplies the fine histogram to the feature distribution calculation unit 512.

  In step S479, the feature distribution calculation unit 512 performs a filtering process on the fine histogram supplied from the fine histogram generation unit 511, and supplies the resulting feature distribution to the thinning processing unit 513.

  In step S480, the thinning processing unit 513 performs a thinning process for performing thinning at a predetermined interval on the feature distribution supplied from the feature distribution calculating unit 512, and the feature distribution after the thinning process is obtained as a feature distribution. This is supplied to the integration unit 515. The processes in steps S478 to S480 are the same as the processes in steps S401 to S403 in FIG.

  In step S <b> 481, the feature distribution integration unit 515 recognizes that the same cut is continuing when the determination result information supplied from the cut change detection unit 514 indicates that it is not a cut change, and supplies it from the thinning processing unit 513. The feature distribution after the thinning process is added to the integrated feature distribution held, and the integrated feature distribution after the integration is held. In addition, when the determination result information supplied from the cut change detection unit 514 indicates a cut change, the feature distribution integration unit 515 supplies the integrated feature distribution of the cut obtained through the integration up to the storage unit 516. Then, the held integrated feature distribution is once reset and the new cut feature distribution is integrated.

  In step S <b> 482, the storage unit 516 stores the integrated feature distribution for each cut supplied from the feature distribution integration unit 515.

  In step S483, the similarity calculation unit 517 reads from the storage unit 516 the integrated feature distributions of the first cut and the second cut to be subjected to image grouping processing, and the first cut integrated feature distribution and the first cut. The integrated feature distribution of the two cuts is compared, and an integrated feature distribution similarity that is the similarity between the integrated feature distribution of the first cut and the integrated feature distribution of the second cut is calculated and supplied to the grouping unit 518. The similarity is also calculated in the same manner as in step S474.

  In step S484, the grouping unit 518 determines, for example, whether the first cut and the second cut are in the same group based on the integrated feature distribution similarity supplied from the similarity calculation unit 517. The grouping unit 518 groups the cuts supplied from the similarity calculation unit 517 in the same manner.

  As described above, grouping is performed so that similar cuts belong to the same group easily and reliably even when a minute change in luminance or color occurs.

  In each of the above-described processes, it has been described that an image is handled in units of frames. However, an image can be handled in units of fields.

  Further, in this specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

  The present invention can be configured by hardware in addition to software.

  The present invention can be applied to all detection devices that process images, such as broadcast equipment, image editing equipment, camcorders, personal computers for image processing, DVD recorders, and hard disk recorders.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure which shows the change of the histogram when cut change arises. It is a figure which shows the change of the histogram when the micro change of a brightness | luminance arises. It is a figure which shows the change of the histogram when the minute fluctuation | variation of a brightness | luminance arises in each of the left half and the right half of a flame | frame. It is a block diagram which shows the structure of one Embodiment of the personal computer as a detection apparatus to which this invention is applied. It is a block diagram which shows the structure of one Embodiment of a cut change detection apparatus. It is a flowchart explaining a cut change detection process. It is a figure which shows the change of a fine histogram and feature distribution. It is a figure which shows the frequency response characteristic of a low-pass filter. It is a figure explaining the filter process by a non-linear filter. It is a block diagram which shows the structure of one Embodiment of a similarity calculation part. It is a flowchart explaining the similarity calculation process of step S404 of FIG. It is a figure explaining the matching of a sample. It is a figure explaining the similarity of two images. It is a block diagram which shows the structure of one Embodiment of the similarity calculation apparatus to which this invention is applied. It is a flowchart explaining the still image similarity calculation process by the similarity calculation apparatus of FIG. It is a figure explaining the similar cut. It is a block diagram which shows the structure of one Embodiment of the image grouping apparatus to which this invention is applied. It is a flowchart explaining an image grouping process. It is a flowchart explaining an image grouping process.

Explanation of symbols

  1 personal computer, 21 CPU, 22 ROM, 23 RAM, 24 bus, 25 I / O interface, 26 input unit, 27 output unit, 28 storage unit, 29 communication unit, 30 drive, 31 removable media, 51 cut change detection device, 401 fine histogram generation unit 402 feature distribution calculation unit 403 thinning processing unit 404 similarity calculation unit 405 determination unit 431 corresponding sample determination unit 432 similarity determination unit 461 similarity calculation device 471 fine histogram generation unit 472 feature distribution calculation unit, 473 thinning processing unit, 474 fine histogram generation unit, 475 feature distribution calculation unit, 476 thinning processing unit, 477 similarity calculation unit, 501 image grouping device, 511 fine histogram Generation unit, 512 feature distribution calculation unit, 513 thinning processing unit, 514 cut change detection unit, 515 feature distribution integration unit, 516 storage unit, 517 similarity calculation unit, 518 grouping unit

Claims (14)

In a program for causing a computer to execute a detection process for detecting a cut change of an image,
Generating a fine histogram of the luminance or color of each of the first and second images;
Applying a filtering process to the fine histogram to calculate a feature distribution;
Calculating the similarity of the feature distribution of each of the first image and the second image;
A program comprising a step of determining whether a boundary between the first image and the second image is a cut change based on the similarity of the feature distribution. Further comprising performing a thinning process on the feature distribution;
The program according to claim 1, wherein the similarity calculation step calculates a similarity of the feature distribution after the thinning process for each of the first image and the second image. The program according to claim 1, wherein a low-pass filter is used for the filtering process. The program according to claim 1, wherein a filter that nonlinearly compresses the frequency axis of the fine histogram is used for the filtering process. The similarity calculation step includes:
Determining the correspondence of the samples so that the shape of the feature distribution of each of the first image and the second image best matches,
The program according to claim 1, further comprising a step of calculating a similarity of the feature distribution of each of the first image and the second image by comparing samples corresponding to each other in the correspondence relationship. In the detection method for detecting the cut change of the image,
Generating a fine histogram of the luminance or color of each of the first and second images;
Applying a filtering process to the fine histogram to calculate a feature distribution;
Calculating the similarity of the feature distribution of each of the first image and the second image;
A detection method comprising a step of determining whether a boundary between the first image and the second image is a cut change based on the similarity of the feature distribution. In the detection device that detects the cut change of the image,
Fine histogram generating means for generating fine histograms of luminance or color of each of the first image and the second image;
A feature distribution calculating means for performing a filtering process on the fine histogram to calculate a feature distribution;
Similarity calculating means for calculating the similarity of the feature distribution of each of the first image and the second image;
A detection device comprising: determination means for determining whether a boundary between the first image and the second image is a cut change based on a similarity of the feature distribution. In a program for causing a computer to execute image processing for calculating image similarity,
Generating a fine histogram of the luminance or color of each of the first and second images;
Filtering the fine histograms of the first image and the second image to calculate a feature distribution;
A program comprising a step of calculating a similarity of the feature distribution of each of the first image and the second image. In an image processing method for calculating the similarity of images,
Generating a fine histogram of the luminance or color of each of the first and second images;
Filtering the fine histograms of the first image and the second image to calculate a feature distribution;
An image processing method comprising a step of calculating a similarity of the feature distribution of each of the first image and the second image. In an image processing device that calculates the similarity of images,
Fine histogram generating means for generating fine histograms of luminance or color of each of the first image and the second image;
A feature distribution calculating means for calculating a feature distribution by performing a filtering process on the fine histogram of the first image and the second image;
An image processing apparatus comprising: a similarity calculation unit that calculates the similarity of the feature distribution of each of the first image and the second image. In a program for causing a computer to execute image processing for grouping image cuts,
Detect whether the boundary of the input image is a cut change,
Generating a fine histogram of luminance or color of each image constituting each of the detected first cut and second cut;
A feature distribution is calculated by filtering each of the fine histograms of the first cut and the second cut,
Integrating the feature distributions of the images constituting the first cut, respectively, and integrating the feature distributions of the images constituting the second cut,
Calculating the degree of similarity between the feature distribution accumulated in the first cut and the feature distribution accumulated in the second cut;
A program comprising a step of grouping the first cut and the second cut based on the similarity. The detection step of detecting the cut change includes:
Generating a fine histogram of the luminance or color of each of the first and second images;
Filtering the fine histograms of the first image and the second image respectively to calculate a feature distribution;
Calculating the similarity of the feature distribution of each of the first image and the second image;
Determining whether a boundary between the first image and the second image is a cut change based on the similarity of the feature distribution;
The program according to claim 11, wherein the first cut and the second cut are cuts separated by the cut change. In an image processing method for grouping image cuts,
Detect whether the boundary of the input image is a cut change,
Generating a fine histogram of luminance or color of each image constituting each of the detected first cut and second cut;
A feature distribution is calculated by filtering each of the fine histograms of the first cut and the second cut,
Integrating the feature distributions of the images constituting the first cut, respectively, and integrating the feature distributions of the images constituting the second cut,
Calculating the degree of similarity between the feature distribution accumulated in the first cut and the feature distribution accumulated in the second cut;
An image processing method comprising a step of grouping the first cut and the second cut based on the similarity. In an image processing apparatus for grouping image cuts,
Detection means for detecting whether the boundary of the input image is a cut change;
Fine histogram generating means for generating a fine histogram of luminance or color of each image constituting each of the detected first cut and second cut;
A feature distribution calculating unit that performs a filtering process on the fine histograms of the first cut and the second cut, respectively, to calculate a feature distribution;
Feature distribution integrating means for integrating the feature distributions of the images constituting the first cut, respectively, and for integrating the feature distributions of the images constituting the second cut;
Similarity calculating means for calculating a similarity between the feature distribution integrated with the first cut and the feature distribution integrated with the second cut;
An image processing apparatus comprising: a grouping unit that groups the first cut and the second cut based on the similarity.

Images