JP2010176570A - Scene change detecting apparatus, scene change detecting program, and scene change detecting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a scene change position in an image sequence with high accuracy without increasing processing time. <P>SOLUTION: A scene change detecting apparatus includes a motion vector detecting section 52 that detects motion vectors between images in a plurality of positions in the images that configure the image sequence, a motion vector classification processing section 53 that classifies patterns of the images into predetermined motion patterns based on the plurality of detected motion vectors, a classified image change detecting section 54 that detects image change between the images in each pattern classification by applying classified image change detection processing defined per pattern classification in advance to corresponding pattern classification, and a classified scene change detecting section 55 that detects scene change in each pattern classification per pattern classification. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a scene change detection device, a scene change detection program, and a scene change detection method for detecting a scene change position from an image sequence composed of a plurality of images.

  Conventionally, various methods have been proposed for processing a plurality of continuously captured still image sequences or continuous image sequences such as moving images, and detecting scene change positions from these continuous image sequences. . For example, a method of comparing a feature change between adjacent images (between frames) with a predetermined threshold and detecting it as a scene change position when the threshold is exceeded is generally well known. Patent Document 1 discloses a technique for detecting various scene changes of different types using a plurality of feature values.

JP 11-252509 A

  However, the technique disclosed in Patent Document 1 has a problem in that all feature values must be calculated for each image, which increases the processing time required to detect a scene change.

  The present invention has been made in view of the above, and provides a scene change detection device, a scene change detection program, and a scene change detection that can accurately detect a scene change position in an image sequence without increasing processing time. It aims to provide a method.

  In order to solve the above-described problems and achieve the object, a scene change detection apparatus according to the present invention is a scene change detection apparatus that detects a scene change position from an image sequence composed of a plurality of images. A motion vector detecting means for detecting a motion vector between other images at a plurality of positions and pattern classification of the image into a predetermined motion pattern based on the plurality of motion vectors detected in the image A classified image for detecting image changes between the images in each pattern classification by applying pattern classification means and classification-specific image change detection processing defined in advance for each pattern classification to the corresponding pattern classification A change detection unit; and a scene change detection unit for each category that detects a scene change in each pattern classification for each pattern classification. It is intended.

  In addition, a scene change detection program according to the present invention provides a scene change detection device that detects a scene change position from an image sequence including a plurality of images, and moves between other images at a plurality of positions in the image. A motion vector detection procedure for detecting a vector, a pattern classification procedure for classifying the image into a predetermined motion pattern based on a plurality of motion vectors detected in the image, and a pattern classification procedure defined in advance for each pattern classification By applying the classified image change detection process to the corresponding pattern classification, a classified image change detection procedure for detecting an image change between the images in each pattern classification, and a scene change in each pattern classification A scene-specific scene change detection procedure for detecting for each pattern classification.

  A scene change detection method according to the present invention is a scene change detection method for detecting a scene change position from an image sequence composed of a plurality of images, and between the other images at a plurality of positions in the image. A motion vector detection step for detecting a motion vector of the image, a pattern classification step for pattern classification of the image into a predetermined motion pattern based on a plurality of motion vectors detected in the image, and for each pattern classification in advance By applying the defined classification-specific image change detection process to the corresponding pattern classification, a classification-specific image change detection step for detecting an image change between the images in each pattern classification; and A classification-specific scene change detection step of detecting a scene change for each pattern classification.

  According to the present invention, it is possible to detect the image change of each image for each pattern classification by applying the classification-specific image change detection process defined in advance for each pattern classification to the corresponding pattern classification. Therefore, the scene change position in the image sequence can be appropriately detected according to the pattern classification without increasing the processing time, and the detection accuracy is improved.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. Moreover, in description of drawing, the same code | symbol is attached | subjected and shown to the same part.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a functional configuration example of the scene change detection device 1 according to the first embodiment. In the first embodiment, the scene change detection device 1 includes an input unit 2, a display unit 3, a storage unit 4, a control unit that controls each unit of the device, and a processing unit 5 that includes arithmetic functions that perform various arithmetic processes. It consists of

  The input unit 2 is realized by various input devices such as a touch panel and various switches in addition to a keyboard and a mouse, and outputs an input signal corresponding to an operation input to the processing unit 5. The display unit 3 is realized by a display device such as an LCD, an EL display, or a CRT display, and displays various screens based on display signals input from the processing unit 5.

  The storage unit 4 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and stored, a hard disk connected by a built-in or data communication terminal, an information storage medium such as a CD-ROM, and a reading device thereof. It is. The storage unit 4 stores a program for operating the scene change detection device 1 and realizing various functions of the scene change detection device 1, data used during the execution of the program, and the like. . For example, data (image information) of an image sequence composed of a plurality of images captured continuously is stored. In addition, a scene change detection program 41 for detecting a scene change position where a scene changes is stored from the images constituting the image sequence.

  The processing unit 5 is realized by hardware such as a CPU. The processing unit 5 performs instructions and data transfer to each unit constituting the scene change detection device 1 based on an operation signal input from the input unit 2, data stored in the storage unit 4, and the like. The operation of the entire change detection device 1 is controlled in an integrated manner. This processing unit 5 includes an image reading unit 51, a motion vector detection unit 52 as a motion vector detection unit, a motion vector classification processing unit 53 as a pattern classification unit, and an image change by classification as an image change detection unit by classification. It includes a detection unit 54, a classified scene change detection unit 55 as a classified scene change detection unit, and an entire scene change detection unit 56 as a scene change position detection unit.

  The image reading unit 51 reads image sequence data (image information) stored in the storage unit 4. The motion vector detection unit 52 associates the same area shown in each image between the images constituting the image sequence, and detects a plurality of vector data (motion vectors) representing the change in the position. The motion vector classification processing unit 53 classifies the motion of each image constituting the image sequence based on the detected plurality of motion vectors. The classification-specific image change detection unit 54 detects an image change between the images constituting the image sequence using the pattern classification result. Specifically, an image change amount calculation process (classification-specific image change detection process) is defined in advance for each pattern classification based on each motion pattern. The classified image change detection unit 54 applies the classified image change detection process corresponding to the motion pattern to the image to be processed, for example, to change the amount of image change between other images that are adjacent in the imaging order. And the calculated image change amount is detected as an image change between images in each pattern classification. The classification-specific scene change detection unit 55 detects a scene change in each pattern classification for each pattern classification based on an image change between images in each pattern classification. Specifically, weight information and detection conditions are set in advance for each pattern classification based on each motion pattern. Then, the classified scene change detection unit 55 integrates the weight information corresponding to the motion pattern into the image change amount of each image, and then extracts the scene change image that satisfies the detection condition corresponding to the motion pattern. This is performed for each pattern classification and detected as a scene change within each pattern classification. The entire scene change detection unit 56 generates a scene change image sequence composed of the extracted scene change images. The generated scene change image sequence is displayed on the display unit 3 in the order of imaging, for example.

  FIG. 2 is a flowchart illustrating a processing procedure performed by the scene change detection device 1 according to the first embodiment. Note that the processing described here is realized by the processing unit 5 reading and executing the scene change detection program 41 stored in the storage unit 4.

  First, the image reading unit 51 acquires image information of an image sequence stored in the storage unit 4 (step a1).

  Subsequently, the motion vector detection unit 52 detects a motion vector by sequentially processing each image constituting the image sequence (step a3). The calculation of the motion vector can be realized by applying a known method such as a template matching method or an optical flow calculation method. Here, for example, a case where a motion vector is calculated by performing template matching will be described. .

  That is, first, a predetermined number of search target areas are set at predetermined positions in an image in which the processing target image and the imaging order are adjacent. The setting position and the number for setting the search target area can be set as appropriate. Here, the search target area may be set, for example, by dividing the image into a grid pattern and setting the search target area equally. Alternatively, a feature region having a predetermined feature predetermined in the image may be searched, and the searched feature region may be set as a search target region. Specifically, the search target region may be set by searching for a region with a large variance of pixel values or a region with a strong edge strength obtained by edge extraction. Next, template matching is performed using each search target region as a template in order, and a region that most matches each template (high correlation value) is searched from the processing target images. As a result of this template matching, a region most similar to the search target region is searched from the processing target images, and the correlation value is obtained. Then, a change in the center coordinate between each search target area and the area searched for this search target area is detected as a motion vector, and set as a motion vector group in the processing target image. At this time, the motion vector detection unit 52 calculates the reliability of each detected motion vector. For example, if no matching area is found as a result of the search, or if the obtained motion vector has a low correlation value, the reliability is set low, and if the obtained correlation value is high, the reliability is set high.

  Subsequently, the motion vector classification processing unit 53 sequentially processes each image constituting the image sequence, and pattern-classifies the motion of each image based on the detected motion vector group (step a5). Specifically, the motion vector classification processing unit 53 determines the motion pattern of the image by estimating the motion of the field of view from the orientation characteristics of the detected motion vector group, and performs pattern classification. In the present embodiment, each image is classified into one of three movement patterns: “parallel movement”, “movement of the imaging target moving away”, and “movement of the imaging target approaching”.

  FIG. 3 is a diagram illustrating a group of motion vectors in an image in which the motion pattern is classified as “parallel movement”. As shown in FIG. 3, when each motion vector V11 detected in the image to be processed is directed in substantially the same direction and has substantially the same size, the field of view of this image is different from that of other images in which the imaging order is adjacent. It can be estimated that it slides sideways (translation) between the two. In such a case, the motion pattern of the image is classified as “parallel movement”. As a specific processing procedure, for example, the direction of each motion vector is determined. Then, when the directions of the motion vectors are substantially in the same direction, a variance value of the motion vector directions is calculated. Then, the calculated variance value is threshold-processed, and if the calculated variance value is smaller than a preset threshold value, the motion pattern of the image is classified as “parallel movement”.

  FIG. 4 is a diagram illustrating a group of motion vectors in an image in which the motion pattern is classified as “movement away from the imaging target”. As shown in FIG. 4, when each motion vector V13 detected in the image to be processed is oriented so as to converge at a certain point (center) in the image, the field of view of this image is in the order of imaging. It can be estimated that the imaging object within the visual field range moves away from other adjacent images. Here, when the visual field range moves in this way, as shown in FIG. 4, the size of each motion vector changes small as it approaches the center. In such a case, the movement pattern of the image is classified as “movement moving away from the imaging target”. As a specific processing procedure, for example, when the end points of the respective motion vectors converge to one point, the motion pattern of the image is classified as “movement away from the imaging target”.

  FIG. 5 is a diagram illustrating a group of motion vectors in an image in which the motion pattern is classified into “movement toward which an imaging target approaches”. As shown in FIG. 5, when each motion vector V15 detected in the image to be processed is oriented so as to diverge radially from a certain point (center) in the image, the field of view of this image is It can be estimated that the imaging target within the field-of-view range is moving closer to another image in which the imaging order is adjacent. Here, when the field-of-view range moves in this way, the magnitude of each motion vector changes greatly as it approaches the center as shown in FIG. In such a case, the movement pattern of the image is classified as “movement toward which the imaging target approaches”. As a specific processing procedure, for example, when the start points of the respective motion vectors are converged to one point, the motion pattern of the image is classified as “movement toward which the imaging target approaches”.

  Note that the pattern classification method is not limited to this, and the pattern classification may be further finely classified. For example, an image classified as “parallel movement” may be further classified according to the direction of the parallel movement. In addition, the images classified as “movement toward which the imaging target moves away” or “movement toward which the imaging target approaches” may be further classified according to the center position. Moreover, it may be classified into motion patterns other than the illustrated “parallel movement”, “movement of the imaging target moving away”, and “movement of the imaging target approaching”, and the pattern classification condition of each movement pattern may be arbitrarily defined. Can do.

  Subsequently, as shown in FIG. 2, the pattern classification of each motion pattern is sequentially processed, and the process of loop A is performed for each image belonging to each pattern classification (step a7 to step a15). Hereinafter, in the processing in the loop A, the pattern classification to be processed is referred to as “target classification”.

  In loop A, first, the classified image change detection unit 54 applies a classified image change detection process according to the target classification, for example, calculates an image change amount between other images adjacent in the imaging order, It is detected as an image change between images in the target classification (step a9).

  Here, the classified image change detection process defined for each pattern classification will be described. When the motion vector is correctly detected, the magnitude of the motion vector becomes an index representing the image change between images. On the other hand, the tendency of the size of each motion vector in the image differs depending on the motion pattern. In view of this, image-by-category image change detection processing for calculating an image change amount is defined for each pattern classification in accordance with the magnitude tendency of each motion vector determined for each pattern classification using the magnitude of each motion vector. More specifically, for example, a classification-specific image change detection process for calculating an image change amount using a parameter value based on the magnitude of each motion vector is defined.

  For example, as described above, the motion vector groups of the images whose motion pattern is “parallel movement” are directed in substantially the same direction and have substantially the same size. Therefore, a process for calculating the average value of the magnitudes of motion vectors in the image as the image change amount is defined as the “parallel translation” classified image change detection process.

  In addition, as described above, the size of the motion vector group of an image having a motion pattern of “movement that the imaging target moves away” or “movement that the imaging target approaches” changes according to the distance from the center. Therefore, as an image change detection process for each category of “movement that the imaging target moves away” or “movement that the imaging target approaches”, the intermediate value, maximum value, and minimum value of the motion vector in the image are calculated as the image change amount. Define a process to do this. Note that a process for calculating an average value of the magnitudes of motion vectors as an image change amount may be defined.

  Note that the classification-specific image change detection process defined for each pattern classification is not limited to the process using one parameter value based on the magnitude of the motion vector as described above, but the magnitude of the motion vector. It is also possible to use a plurality of parameter values based on the above and integrate these values to calculate the image change amount. Furthermore, not only the parameter value based on the magnitude of the motion vector but also the parameter value based on the direction of the motion vector can be used. For example, the average value, intermediate value, maximum value, minimum value, variance value, variance value of each motion vector direction, each motion vector reliability, etc. It is also possible to define a process for calculating the image change amount by integrating the above as the image change detection process by classification.

  Alternatively, the reliability of each motion vector may be used as a parameter value based on the motion pattern, and the image change amount may be calculated by taking this value into account, so that more reliable image change detection can be realized. Specifically, weighting using the reliability is performed on the magnitude and direction of each motion vector. For example, calculate the weighted average value, weighted intermediate value, weighted maximum value, weighted minimum value, etc. based on the size of the motion vector, or weighted variance based on the direction of the motion vector The image change amount may be calculated by calculating a value such as.

  Subsequently, the classified scene change detection unit 55 detects a scene change in the target classification based on the image change between images in the detected target classification (step a11). Specifically, the classification-specific scene change detection unit 55 first integrates the weight information set in advance for each target class into the image change amount of each image classified into the target class.

  Here, the weight information is, for example, a weighting factor that is individually set for each pattern classification. Compared with the weighting factors that are set for “movement that the imaging target moves away” and “movement that the imaging target approaches”, the “parallel” is set. A large weighting factor is set for “movement”. Compared to each movement pattern, in “parallel movement”, compared to “movement that the imaging target moves away” and “movement that the imaging target approaches”, the imaging target within the field of view is faster than the field of view. This is based on the fact that the change in the appearance of the imaging target between the images to be performed is small.

  FIG. 6 is a schematic diagram illustrating an example of four images (a) to (d) adjacent to each other in the imaging order in which the movement pattern is classified as “parallel movement”, and an image of the imaging target O11 shown in the image. It shows the change in appearance between the two. Further, on each image, a group of motion vectors detected in the image is shown. FIG. 7 is a schematic diagram illustrating an example of four images (a) to (d) adjacent to each other in the imaging order in which the motion pattern is pattern-categorized as “movement that the imaging target moves away”. A change in appearance between images of the imaging target O13 is shown. Further, in each image, a motion vector group detected in the image is shown. Further, FIG. 8 is a schematic diagram showing an example of four images (a) to (d) adjacent to each other in the imaging order in which the motion pattern is pattern-classified as “movement toward which the imaging target approaches”, and is reflected in the image. A change in appearance between images of the imaging target O15 is shown. Further, on each image, a group of motion vectors detected in the image is shown.

  As shown in FIGS. 6 to 8, the motion vector is used when the field of view moves in a “parallel movement” and when the imaging target moves away or the imaging target moves closer. Even when the sizes of the images are substantially equal, the change in the appearance of the imaging target shown in each image is different, and the speed at which the imaging target disappears from the field of view is also different. For example, the change in the appearance of the imaging target O11 when the field of view shown in FIG. 6 moves in “parallel movement” is the imaging target O13 in the case where “the imaging target moves away” shown in FIG. Is faster than the change in the appearance of the image pickup object O15 and the change in the appearance of the image pickup object O15 when the image pickup object is moving as shown in FIG.

  That is, in the example of “parallel movement” shown in FIG. 6, a part of the imaging target O11 is out of the field at the third image (c), and completely disappears from the field of view at the fourth image (d). Yes. On the other hand, in the example of “movement where the imaging target moves away” shown in FIG. 7, the imaging target O13 is included in the fourth image (d) even though the magnitude of the motion vector is almost the same as in FIG. Is reflected and is not out of sight. Therefore, the image change amount of the image classified as “parallel movement” and the image change amount of the image classified as “movement toward which the imaging target moves away” or “movement toward which the imaging target approaches” are comparable. However, the movement of the imaging target is larger in the “parallel movement”. The same applies to the case of FIG. 8, and it can be said that the importance of “parallel movement” is higher than “movement of the imaging target moving away” and “movement of the imaging target approaching”.

  Therefore, in the first embodiment, the weighting factor of “parallel movement” is set to be larger than the weighting factors of “movement that the imaging target moves away” and “movement that the imaging target approaches”. Then, this weight coefficient is integrated into the image change amount. As a result, it is possible to realize image change detection taking into account the importance based on the difference in the change in the appearance of the imaging target for each pattern classification. Also, by integrating the weight information according to the pattern classification of the image into the image change amount, the order of each pattern classification is not determined only by the motion pattern, but is integrated in a mixed manner by the weighting coefficient. A more appropriate ordering can be achieved.

Specifically, the classified scene change detection unit 55 calculates an image change amount in which the weight information is integrated according to the following equation (1), and detects it as an image change between images. Here, ImgMov (j) represents an image change amount calculated for the j-th image in the image sequence, and W _{ImgMov (k)} represents a weight coefficient set as weight information for the k-th pattern classification.

  Then, the classification-specific scene change detection unit 55 extracts an image (scene change image) in which the image change amount integrated with the weight information satisfies a scene change detection condition set in advance for the target class, and the scene change within the target class Detect as. Here, the detection condition is, for example, that an individual threshold value is defined for each pattern classification, and the scene-specific scene change detection unit 55 defines the image change amount of each image classified in the target classification for the target classification. Threshold processing is performed using the threshold, and an image having an image change amount larger than the threshold is extracted as a scene change image. For example, based on the importance of the pattern classification described above, the threshold for the pattern classification with a low importance is set larger than the threshold for the pattern classification with a high importance.

  Note that, here, a case has been described in which a threshold value for each pattern classification is defined as a detection condition, and image change amount threshold processing is performed using this threshold value. However, the present invention is not limited to this. For example, the number of scene changes for each pattern classification may be defined as a detection condition. For example, based on the importance of pattern classification, the number of scene changes is set to be greater than the number of scene changes in the pattern classification with higher importance than the number of scene changes in the pattern classification. Then, the classification-specific scene change detection unit 55 extracts the images with the number of scene changes from the images classified into the target classification as the scene change images in descending order of the image change amount, and detects them as scene changes. Also good.

  If the process of the loop A is executed for the pattern classification of all the motion patterns, then the entire scene change detection unit 56 generates a scene change image sequence from the scene change image detected in step a11 for each pattern classification, The scene change position of the entire image sequence is detected (step a15).

  FIG. 9 is a schematic diagram illustrating an example of an image sequence in which a scene change image is detected. As shown in FIG. 9, the image sequence processed as described above is divided into a plurality of shots with the scene change image as a scene change. Then, when the display is actually performed on the display unit 3, the first scene change image of the shot is sequentially displayed. In this display, an image with a small image change between images is not displayed. That is, the display of images having a high degree of similarity is omitted. Therefore, an efficient display of the images constituting the image sequence can be realized.

  Further, according to the first embodiment, it is possible to detect an image change between images for each pattern classification in which each image in the image sequence is classified in advance by a motion pattern. More specifically, the image change detection process for each classification is defined for each pattern classification, and the image change between the images is detected for each pattern classification by applying the image change detection process for each classification according to the pattern classification. can do. Thus, based on the image change between images in each pattern classification, a scene change in each pattern classification can be detected for each pattern classification. Specifically, the image change amount calculated for each image is integrated with the weight information according to the pattern classification, and the scene change detection image is extracted according to the detection condition according to the pattern classification, thereby obtaining each pattern. Scene changes within the classification can be detected. Therefore, it is possible to detect image changes between images according to the pattern classification, and further to detect scene changes according to the pattern classification, and without increasing the processing time required to detect the scene change, The scene change position in the image sequence can be appropriately detected for each pattern classification, and the detection accuracy is improved.

  Furthermore, even if the “translation” and “movement moving away from the imaging target” or “movement moving closer to the imaging target” seem to have the same magnitude of the motion vector, there is a high possibility that the actual motion amount is different. According to the first embodiment, it is possible to detect a scene change with high accuracy by applying different weight information and detection conditions for each motion vector pattern classification.

  In the above description, an image change between images is detected based on a motion vector detected between adjacent images in the imaging order. However, the present invention is not limited to this. For example, it is possible to detect image changes between images using image features such as correlation between images generally known from the past, SSD (sum of squares of pixel differences), and SAD (sum of absolute values of pixel differences). It is good to do.

  In addition, the detection of the motion vector is not particularly limited to processing between two adjacent images. For example, an image feature between two or more images may be calculated, and an image change between the images may be detected using a statistical calculation based on a combination thereof. For example, for several adjacent images, image features such as motion vectors, normalized cross-correlation, SSD, SAD, etc. between adjacent images are calculated. And it is good also as detecting the image change between images based on the average value of each value.

  In a continuous image sequence, similar images may be adjacent to each other. In such a case, it is also possible to apply the processing result performed on a single image to a plurality of adjacent images. As a method for determining the number of images to which this method is applied, a method of determining a certain number of images in advance may be simply used. Alternatively, the similarity between images may be determined adaptively by determining the comparison with a predetermined threshold. According to this, it is not necessary to process all the images constituting the image sequence, and the processing time can be shortened.

(Embodiment 2)
Next, a second embodiment will be described. In the first embodiment, each image “parallel movement”, “movement in which the imaging target moves away”, and “movement in which the imaging target approaches” are classified into motion patterns, and image changes and scenes between images are classified for each pattern classification. Change was detected. However, an actual image includes an image that cannot be classified into motion patterns such as “parallel movement”, “movement of an imaging target moving away”, and “movement of an imaging target approaching” because the scene changes completely between the images. In the second embodiment, such an image is classified into a new classification (hereinafter referred to as “scene change”), and these images are divided between images based on the motion vector described in the first embodiment. A process different from the image change detection process is applied.

  FIG. 10 is a flowchart illustrating a processing procedure performed by the scene change detection apparatus according to the second embodiment. In FIG. 10, the same reference numerals are assigned to the same processing steps as those in the first embodiment.

  As shown in FIG. 10, after executing the process of Loop A, the motion vector classification processing unit 53 does not classify any of “parallel movement”, “movement of the imaging target away”, or “movement of the imaging target approaching”. Determine the presence or absence. Here, an image that is not classified as a motion pattern is, for example, a case where the number of regions searched as a region corresponding to the search target region from this image at the time of calculating a motion vector is small, or uniformity in the direction of the obtained motion vector This is an image that is considered to have low detection result reliability, for example, when there is no image. Such an image can be estimated as an image in which the image changes greatly between the images and the scene changes completely. If there is no image that is not classified as a motion pattern (step b14: No), the process proceeds to step a15. Then, the entire scene change detection unit 56 generates a scene change image sequence from the scene change image detected for each pattern classification in step a11, and detects a scene change as the entire image sequence.

  On the other hand, when there is an image that is not classified into the motion pattern (step b14: Yes), the motion vector classification processing unit 53 classifies the image that is not classified into the motion pattern into a scene change. Then, the classified image change detection unit 54 detects an image change between images in the scene change classification using a method different from the image change detection method based on the motion vector (step b17). Specifically, image changes between images are detected using correlation between images and image features such as SSD and SAD.

  Subsequently, the classification-specific scene change detection unit 55 detects a scene change in the scene change classification in the same manner as in step a11 (step b19). Specifically, weight information and detection conditions for scene changes are set in advance. The classification-specific scene change detection unit 55 first integrates the scene change weight information into the image change amounts of the images classified as scene changes, and then satisfies the detection condition set for the scene change. The process of extracting the change image is performed for each pattern classification, and detected as a scene change in each pattern classification. For example, as described above, the scene change image is an image that is estimated to have a large image change between the images, and therefore, as the weight information, “parallel movement”, “movement of the imaging target moving away”, and “movement of the imaging target approaching” A larger weight coefficient is set. If a threshold value is defined as the detection condition, the threshold value is set smaller than the threshold values set for “parallel movement”, “movement of the imaging target moving away”, and “movement of the imaging target approaching”. If the number of scene changes is defined as the detection condition, a value larger than the number of scene changes set for “parallel movement”, “movement of the imaging target moving away”, and “movement of the imaging target approaching” is set.

  Then, the entire scene change detection unit 56 generates a scene change image sequence from the scene change image detected for each pattern classification in step b19, and detects a scene change as the entire image sequence (step b21). At this time, as described above, since the motion vector is correctly detected in the image change of the pattern classified image and it is considered that the image change is small compared to the image not classified, the image of the pattern classified image It is also possible to adjust so that the boundary between the maximum value of change and the minimum value of image change of an image group that has not been subjected to pattern classification is continuous.

  According to the second embodiment, the same effect as that of the first embodiment is obtained, and for a scene change image that is not classified into any motion pattern, a method different from the image change detection method using a motion vector. Can be used to detect image changes between images, and scene changes can be detected based on the detected image changes between images. As a result, even when the image sequence includes an image that cannot be classified into patterns, it is possible to appropriately obtain an image change between the images. Therefore, the scene change position in the image sequence can be detected more appropriately, and the detection accuracy can be further improved.

  The above-described scene change detection apparatuses according to the first and second embodiments can be realized by executing a prepared program on a computer system such as a personal computer or a workstation. Hereinafter, a computer system that has the same function as the scene change detection device described in each of the first and second embodiments and executes a scene change detection program will be described.

  FIG. 11 is a system configuration diagram showing a configuration of the computer system 20 to which the embodiment is applied, and FIG. 12 is a block diagram showing a configuration of the main body 210 in the computer system 20. As shown in FIG. 11, the computer system 20 includes a main body 210, a display 220 for displaying information such as an image on a display screen 221 according to instructions from the main body 210, and various information on the computer system 20. A keyboard 230 for inputting and a mouse 240 for designating an arbitrary position on the display screen 221 of the display 220 are provided.

  As shown in FIG. 12, the main body 210 in the computer system 20 includes a CPU 211, a RAM 212, a ROM 213, a hard disk drive (HDD) 214, a CD-ROM drive 215 that accepts a CD-ROM 260, and a USB memory. USB port 216 to which 270 is detachably connected, I / O interface 217 to which display 220, keyboard 230 and mouse 240 are connected, and a LAN interface for connection to a local area network or wide area network (LAN / WAN) N1 218.

  Further, the computer system 20 is connected to a modem 250 for connecting to a public line N3 such as the Internet, and is another computer system via the LAN interface 218 and the local area network or the wide area network N1. A personal computer (PC) 281, a server 282, a printer 283, and the like are connected.

  And this computer system 20 implement | achieves a scene change detection apparatus by reading and executing the scene change detection program memorize | stored in the predetermined storage medium. Here, the predetermined storage medium is a “portable physical medium” including an MO disk, a DVD disk, a flexible disk (FD), a magneto-optical disk, an IC card, etc. in addition to the CD-ROM 260 and the USB memory 270, a computer The “fixed physical medium” such as HDD 214, RAM 212, ROM 213, etc. provided in the body cavity of the system 20, the public line N3 connected via the modem 250, and another computer system (PC) 281 or server 282 are connected. Any storage medium that stores a scene change detection program readable by the computer system 20, such as a “communication medium” that holds the program in a short period of time when the program is transmitted, such as a local area network or a wide area network N1.

  That is, the scene change detection program is stored in a storage medium such as “portable physical medium”, “fixed physical medium”, or “communication medium” so as to be readable by a computer. A scene change detection apparatus is realized by reading and executing a scene change detection program from a storage medium. Note that the scene change detection program is not limited to be executed by the computer system 20, but when the other computer system (PC) 281 or the server 282 executes the scene change detection program, or these cooperate. Thus, the present invention can be similarly applied to the case where the scene change detection program is executed.

  The above-described scene change detection apparatuses according to the first and second embodiments can be suitably applied to, for example, a capsule endoscope system. FIG. 13 is a configuration diagram schematically showing a capsule endoscope system including the scene change detection device 39 to which the first or second embodiment is applied as a general-purpose computer such as a workstation or a personal computer. As shown in FIG. 13, the capsule endoscope system captures an image inside the subject 31 (hereinafter referred to as “in-vivo image”) in addition to the scene change detection device 39. And a receiving device 35 for receiving an intra-body-cavity image wirelessly transmitted from the capsule endoscope 33. For example, a portable recording medium (portable recording medium) 37 is used for transferring image data between the receiving device 35 and the scene change detecting device 39, and corresponds to the storage unit 4 in FIG.

  The capsule endoscope 33 has an imaging function, a wireless function, and the like, and is swallowed from the mouth of the subject 31 and introduced into the subject 31. The capsule endoscope 33 captures about 60000 images every about 0.5 seconds while moving inside the body cavity by a peristaltic motion or the like until the capsule endoscope 33 is discharged to the outside of the body. Send images wirelessly outside the body.

  The receiving device 35 includes receiving antennas A <b> 1 to An that are dispersedly arranged at positions on the body surface corresponding to the passage path of the capsule endoscope 33 in the subject 31. Then, the receiving device 35 receives image data wirelessly transmitted from the capsule endoscope 33 via the receiving antennas A1 to An. The receiving device 35 is configured so that the portable recording medium 37 can be freely attached and detached, and sequentially stores the received image data in the portable recording medium 37. In this way, the in-vivo images inside the subject 31 captured by the capsule endoscope 33 are accumulated in the portable recording medium 37 in the order of imaging by the receiving device 35 and stored as an image sequence.

  The scene change detection device 39 is configured to be detachable from the portable recording medium 37, processes the image sequence of the in-vivo image received by the receiving device 35 and stored in the portable recording medium 37, and The images are sequentially displayed on a display such as an LCD or ELD in the order of imaging. Here, the scene change detection device 39 detects the scene change position from the image sequence of the body cavity image in the same manner as the scene change detection devices of the first and second embodiments, and generates a scene change image sequence.

  Here, the moving speed of the capsule endoscope 33 in the body cavity is not constant, and the time-series change of the captured image varies. For this reason, in the image sequence of the in-vivo images to be processed, a situation in which similar images continue may occur. On the other hand, when all of the enormous number of in-vivo images captured by the capsule endoscope 33 are displayed, an observer such as a doctor takes a long time to observe an enormous number of images of about 60000. Must be spent, and the burden is large. From the above, detecting the scene change position in the image sequence by processing each in-vivo image and generating the scene change image sequence reduces the redundancy of displaying a similar image, Since a rough overall image can be efficiently observed, it is effective from the viewpoint of reducing the burden on the observer.

  According to this capsule endoscope system, since the scene change position can be appropriately detected from the image sequence of the enormous number of in-vivo images captured by the capsule endoscope 33, the observation efficiency by the observer can be improved. It can be improved and efficient diagnosis support can be realized. In particular, the body cavity image captured by the capsule endoscope 33 includes many images that cannot be classified into motion patterns. For this reason, as described in the second embodiment, it is important to detect an image change between images for an image that cannot be classified into patterns, and the scene detection device according to the second embodiment is applied to this capsule endoscope system. Thus, the scene change position can be detected more appropriately.

  In addition to processing an image sequence of an in-vivo image captured by a capsule endoscope, it is suitable for detecting a scene change position from an image sequence with a scene change and displaying an image with a large scene change. Applicable.

3 is a block diagram illustrating a functional configuration example of the scene change detection device according to the first embodiment. FIG. 4 is a flowchart illustrating a processing procedure performed by the scene change detection device according to the first embodiment. It is a figure which shows the motion vector group in the image by which a motion pattern is pattern-classified by "parallel movement". It is a figure which shows the motion vector group by which a motion pattern is pattern-classified into "the movement which moves away from an imaging target." It is a figure which shows the motion vector group by which a motion pattern is pattern-classified into "the movement which an imaging target approaches." It is a schematic diagram which shows an example of the four images with which the imaging order adjacent to which the motion pattern was pattern-classified as "parallel movement". FIG. 10 is a schematic diagram illustrating an example of four images whose movement orders are adjacent to each other in the imaging order in which the pattern classification is “movement of the imaging target moving away”. FIG. 10 is a schematic diagram illustrating an example of four images in which the imaging order is classified in which the motion pattern is “classified as the imaging target approaches”. It is a schematic diagram which shows an example of the image sequence from which the scene change image was detected. 10 is a flowchart illustrating a processing procedure performed by the scene change detection apparatus according to the second embodiment. 1 is a system configuration diagram showing a configuration of a computer system to which an embodiment is applied. It is a block diagram which shows the structure of the main-body part in the computer system of FIG. It is a block diagram which shows typically the capsule type endoscope system provided with the scene change detection apparatus.

DESCRIPTION OF SYMBOLS 1 Scene change detection apparatus 2 Input part 3 Display part 4 Memory | storage part 41 Scene change detection program 5 Processing part 51 Image reading part 52 Motion vector detection part 53 Motion vector classification | category process part 54 Image change detection part classified by classification 55 Scene change detection classified according to classification Unit 56 Whole scene change detection unit

Claims (13)

A scene change detection device that detects a scene change position from an image sequence composed of a plurality of images,
Motion vector detection means for detecting a motion vector between other images at a plurality of positions in the image;
Pattern classification means for pattern-classifying the image into a predetermined motion pattern based on a plurality of motion vectors detected in the image;
A classification-specific image change detection means for detecting an image change between the images in each pattern classification by applying the classification-specific image change detection processing defined in advance for each pattern classification to the corresponding pattern classification,
Classification-specific scene change detection means for detecting scene changes in each pattern classification for each pattern classification;
A scene change detection apparatus comprising:   The classification-specific image change detection process for each pattern classification is a process of calculating an image change amount of the image using one or a plurality of parameter values based on the plurality of motion vectors detected in the image. The scene change detection apparatus according to claim 1, wherein:   The pattern classifying means translates the image based on statistical data or variations of the plurality of motion vectors detected in the image, and generates a motion pattern including a motion in which the imaging target moves away or a motion in which the imaging target approaches. The scene change detection apparatus according to claim 1, wherein pattern classification is performed.   The classification-specific image change detection process applied to an image in which a motion pattern is classified into parallel movement is an image of the image based on at least an average value of the plurality of motion vectors detected in the image. The scene change detection apparatus according to claim 3, wherein the scene change detection process is a process of calculating a change amount.   The image change detection processing classified by classification applied to an image classified into a pattern in which a motion pattern moves away from the imaging target or moves closer to the imaging target has at least the size of the plurality of motion vectors detected in the image. 5. The scene change detection apparatus according to claim 3, wherein the scene change detection apparatus is a process of calculating an image change amount of the image based on an intermediate value, a maximum value, or a minimum value.   6. The classification-based image change detection means calculates an image feature of an image that is not classified into any of the motion patterns, and detects it as an image change between the images. The scene change detection device according to claim 1.   The classification-specific image change detection means detects an image change between the images in consideration of reliability of the plurality of motion vectors detected in the image. The scene change detection device according to one.   The classification-specific scene change detection means detects a scene change in the pattern classification based on an image change between images in the pattern classification. The described scene change detection device.   The classification-specific scene change detection means detects a scene change in each of the pattern classifications in consideration of importance determined in advance for each of the pattern classifications. The scene change detection device according to 1.   The scene change position detecting means for detecting a scene change position of the image sequence based on a scene change detected for each pattern classification, according to any one of claims 1 to 9, Scene change detection device.   The scene according to any one of claims 1 to 10, wherein the image sequence is an image sequence of an in-vivo image captured by a capsule endoscope introduced into a body cavity of a subject. Change detection device. In a scene change detection device that detects a scene change position from an image sequence composed of a plurality of images,
A motion vector detection procedure for detecting motion vectors between other images at a plurality of positions in the image;
A pattern classification procedure for pattern-classifying the image into a predetermined motion pattern based on a plurality of motion vectors detected in the image;
A classification-specific image change detection procedure for detecting an image change between the images in each pattern classification by applying the classification-specific image change detection process defined in advance for each pattern classification to the corresponding pattern classification;
Classification-specific scene change detection procedure for detecting scene changes in each pattern classification for each pattern classification;
A scene change detection program characterized by causing A scene change detection method for detecting a scene change position from an image sequence composed of a plurality of images,
A motion vector detection step of detecting a motion vector between other images at a plurality of positions in the image;
A pattern classification step of pattern classifying the image into a predetermined motion pattern based on a plurality of motion vectors detected in the image;
A classification-specific image change detection step for detecting an image change between the images in each pattern classification by applying a classification-specific image change detection process defined in advance for each pattern classification to the corresponding pattern classification,
A scene-by-class scene change detection step for detecting scene changes in each pattern class for each pattern class;
A scene change detection method comprising:

Images