JP2005080209A - Moving image division method, moving image dividing apparatus, multimedia search index imparting apparatus, and moving image dividing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image division method, moving image dividing apparatus and moving image dividing program which realize division of a moving image file into scenes corresponding to its contents and high efficiency in editing a moving image. <P>SOLUTION: The moving image dividing apparatus 100 is provided with a cut point recognizing section 101, a local variation part recognizing section 102, a high-level sound recognizing section 103, a divided file number adjusting section 104, an original moving image storage section 111, a parameter information storage section 112 and a divided moving image storage section 113. In the order of cut point recognition, local variation part recognition and high-level sound recognition, scene division candidate points of a moving image file are judged and a moving image file 10 that is inputted with the upper limit number of divisions as its upper limit, is divided into the predetermined number of divisions at the scene division candidate points. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving image dividing method, a moving image dividing device, a multimedia search index assigning device, and a moving image dividing program.

  When editing moving image content (moving image file), there is a case where the moving image content is divided at every transition of a video scene or a break of music because there is a semantic unity. For example, in a multimedia search system, a moving image content is divided for providing a search index (a text sentence corresponding to each content is assigned as a search index to a moving image divided for each scene). . In addition, in a system that adds accompanying information such as content related to received broadcast content (moving image content), the broadcast content is divided in order to make the broadcast content into a plurality of pieces of broadcast data.

  With regard to such moving image division, generally, a method of detecting a cut point from the similarity of each frame constituting a moving image file or detecting a silent portion is employed.

The prior art document information related to this application includes the following.
JP 2002-262225 A Japanese Patent Laid-Open No. 9-214879

  However, since there are actually moving image files with various contents, even if scene division is performed uniformly by the conventional method as described above, there are cases where the division is not appropriate for the contents. . That is, there is a problem that flexible scene division according to the content of the moving image file cannot be performed. For example, in the case of a promotion video of a singer, there are many scenes that change scenes, so it is effective to divide the scene by detecting the cut point, but in news programs and martial arts performance videos, Since scenes are not switched so much, scene division by detection of cut points is not effective, meaning that even if all moving image contents are divided based on a single criterion, it is not effective.

  Also, such moving image division is automatically performed according to a predetermined algorithm, but depending on the content of the moving image file, dozens or hundreds of scene division files may be created. Therefore, in this case, the subsequent moving image editing work must be performed based on the large number of scene division files, and there is a problem that the human burden in the moving image editing work is large. In particular, in the multimedia search system described above, when a scene division file is created exceeding the number that can be displayed on one screen, a search index assignment operation for the scene division file becomes complicated.

  The present invention has been made in order to solve the above-described problems, and a moving image dividing method and a moving image dividing method that can perform scene division according to the contents of a moving image file and improve the efficiency of moving image editing work. It is an object to provide a device, a multimedia search index assigning device, and a moving image dividing program.

  In order to achieve the above object, the present invention according to claim 1 is a moving image dividing method for dividing a moving image file into scenes according to the contents of the moving image, wherein the first and second thresholds relating to moving image division, and the scene dividing The computer having storage means for storing the upper limit number of divisions, which is the maximum number of frames, is similar to all the similarities between temporally adjacent frames of the moving image file with respect to the input moving image that is the original moving image before dividing. A cut point detection step of calculating a first numerical value related to the scene and detecting a candidate point for scene division based on a comparison between each of the first numerical values and the first threshold value stored in the storage unit; If the candidate point is not detected in the cut point detection step, all the classes between the temporally adjacent frames in each block obtained by dividing the frame into a plurality of blocks. A second numerical value relating to a degree and a luminance difference, and a local point for detecting a candidate point for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage unit If the candidate point is not detected in the fluctuation part detection step and the local fluctuation part detection step, an average value of the audio levels of all audio data of the moving image file is calculated, and the average value is included in the moving image file. A high-level sound detection step for detecting candidate points for scene division based on a comparison with the sound level of the sound data to be recorded, and the cut point detection step, the local variation point detection step, or the high-level sound detection step. When the number of detected scene division candidate points is equal to or less than the upper limit number of divisions stored in the storage unit, the motion division is performed according to the scene division candidate points. When the file is divided and the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step exceeds the division upper limit number, Based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point, the candidate points for the upper limit number of divisions are selected, and the candidate points are selected according to the selected candidate points. The gist of the present invention is to execute a divided file number adjusting step for dividing a moving image file.

  The present invention according to claim 2 creates a divided video that is divided into scenes according to the content of the video, and adds a search index for multimedia that gives a search index to the divided video in response to a request from a client terminal. The multimedia search index assigning apparatus, comprising: storage means for storing first and second threshold values relating to video division and a division upper limit number that is the maximum number of scene divisions. An original moving image receiving step for receiving a moving image file that is an original moving image selected by a client terminal, and a first numerical value related to all similarities between temporally adjacent frames of the moving image file with respect to the moving image file And based on a comparison between each of the first numerical values and the first threshold value stored in the storage means. A cut point detecting step for detecting candidate points for scene division, and when the candidate points are not detected in the cut point detecting step, in each block obtained by dividing the frame into a plurality of temporally adjacent frames And calculating a second numerical value relating to the similarity and the luminance difference of the image, and detecting candidate points for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage unit When the candidate point is not detected in the local variation location detection step and the local variation location detection step, an average value of audio levels of all audio data of the video file is calculated, and the average value and the video file A high-level sound detection step for detecting candidate points for scene division based on a comparison with the sound level of the sound data included in the sound data, and the cut point detection step. If the number of scene division candidate points detected in the local variation point detection step or the high-level sound detection step is equal to or less than the upper limit number of divisions stored in the storage means, The moving image file is divided according to the candidate points, and the number of scene division candidate points detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step exceeds the division upper limit number. And selecting the candidate points for the upper limit number of divisions based on the first numerical value, the second numerical value, or the level of the audio level calculated at each of the candidate points. The divided file number adjusting step for dividing the moving image file according to the candidate points, and the divided moving image divided in the divided file number adjusting step, respectively. The gist is to execute a display step of transmitting the information to the client terminal and displaying it.

  The present invention described in claim 3 is a moving image dividing apparatus for dividing a moving image file into scenes according to the contents of the moving image, wherein the first and second thresholds relating to moving image division and the division upper limit which is the maximum number of scene divisions Storage means for storing the number, and for the moving image file that is the input original moving image before the division, calculating a first numerical value regarding all similarities between temporally adjacent frames of the moving image file, Cut point detection means for detecting candidate points for scene division based on a comparison between each first numerical value and the first threshold value stored in the storage means, and the candidate points by the cut point detection means Is not detected, in each block obtained by dividing the frame into a plurality, a second numerical value relating to all similarities and luminance differences between the temporally adjacent frames is calculated, and the second number Based on a comparison between each and the second threshold value stored in the storage unit, a local variation point detection unit that detects a candidate point for scene division, and the candidate point is detected by the local variation point detection unit If not, an average value of the audio levels of all the audio data of the video file is calculated, and based on a comparison between the average value and the audio level of the audio data included in the video file, scene division candidates The number of candidate points for scene division detected by the high-level sound detection means for detecting points, the cut point detection means, the local variation point detection means, or the high-level sound detection means is stored in the storage means. In the case where the number is equal to or less than the division upper limit number, the moving image file is divided according to the scene division candidate points, and the cut point detection unit, the local variation point detection unit, or the When the number of scene division candidate points detected by the level sound detection means exceeds the division upper limit number, the first numerical value, the second numerical value calculated at each of the candidate points, Alternatively, the present invention includes a divided file number adjusting unit that selects the candidate points for the upper limit number of divisions based on the level of the audio level and divides the moving image file according to the selected candidate points.

  The present invention according to claim 4 provides a multimedia search index that creates a divided video that is divided into scenes according to the content of the video, and adds a search index to the divided video in response to a request from a client terminal. A storage means for storing first and second threshold values relating to video segmentation and a maximum number of divisions that is the maximum number of scene divisions; and a video file that is an original video selected by the client terminal A first numerical value relating to the similarity between all temporally adjacent frames of the moving image file, the original moving image receiving means, and the moving image file; Cut point detection means for detecting candidate points for scene division based on comparison with the stored first threshold value, and the cut point detection means When the candidate point is not detected, in each block obtained by dividing the frame into a plurality of frames, a second numerical value relating to all similarities and luminance differences between the temporally adjacent frames is calculated, and the second numerical value is calculated. Based on a comparison between each and the second threshold value stored in the storage unit, a local variation point detection unit that detects a candidate point for scene division, and the candidate point is detected by the local variation point detection unit If not, an average value of the audio levels of all the audio data of the video file is calculated, and based on a comparison between the average value and the audio level of the audio data included in the video file, scene division candidates High-level sound detection means for detecting a point, and the candidate for scene division detected by the cut point detection means, the local variation point detection means, or the high-level sound detection means Is less than or equal to the upper limit number of divisions stored in the storage means, the video file is divided according to the scene division candidate points, and the cut point detection means, the local variation location detection means, or the When the number of candidate points for scene division detected by the high-level sound detection means exceeds the upper limit number for division, the first numerical value and the second numerical value calculated at each of the candidate points Alternatively, the number of candidate points corresponding to the upper limit number of divisions is selected based on the level of the audio level, and the divided file number adjusting unit that divides the moving image file according to the selected candidate points and the divided file number adjusting unit And a display means for transmitting and displaying each of the divided divided videos to the client terminal.

  The present invention according to claim 5 is a moving picture dividing program for dividing a moving picture file into scenes according to the contents of the moving picture, wherein the first and second thresholds relating to the moving picture division and the division upper limit which is the maximum number of the scene divisions A first numerical value related to all similarities between temporally adjacent frames of the moving image file is calculated with respect to the moving image file which is the original moving image before division into a computer having storage means for storing the number Then, based on a comparison between each of the first numerical values and the first threshold value stored in the storage means, a cut point detecting step for detecting candidate points for scene division, and the cut point detecting step When the candidate point is not detected, in each block obtained by dividing the frame into a plurality, the similarity and luminance difference between all temporally adjacent frames are related. A local variation point detecting step of detecting a candidate point for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage unit If the candidate point is not detected in the local variation location detection step, an average value of the audio levels of all audio data of the video file is calculated, and the average value and the audio data included in the video file are calculated. A scene detected in the high-level sound detection step for detecting candidate points for scene division based on the comparison with the sound level, the cut point detection step, the local variation point detection step, or the high-level sound detection step When the number of division candidate points is equal to or less than the upper limit number of divisions stored in the storage unit, the moving image file is divided according to the scene division candidate points. If the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step exceeds the division upper limit number, each candidate point Based on the first numerical value, the second numerical value, or the level of the audio level calculated in step 1, the candidate points for the division upper limit number are selected, and the moving image file is divided according to the selected candidate points. The gist is to execute the step of adjusting the number of divided files.

  The present invention according to claim 6 provides a multimedia search index that creates a divided video that is divided into scenes according to the content of the video, and adds a search index to the divided video in response to a request from a client terminal. The multimedia search index assigning apparatus, comprising: a storage means for storing a first and second threshold values related to video division, and a division upper limit number that is the maximum number of scene divisions. An original moving image receiving step for receiving a moving image file that is an original moving image selected by a client terminal, and a first numerical value related to all similarities between temporally adjacent frames of the moving image file with respect to the moving image file And based on a comparison between each of the first numerical values and the first threshold value stored in the storage means. A cut point detecting step for detecting candidate points for scene division, and if the candidate point is not detected in the cut point detecting step, the temporally adjacent frames in each block obtained by dividing the frame into a plurality of blocks Calculating a second numerical value regarding all similarities and luminance differences between them, and comparing each of the second numerical values with the second threshold value stored in the storage means, the candidate points for scene division When the candidate point is not detected in the local variation location detection step and the local variation location detection step, the average value of the audio levels of all audio data of the video file is calculated, and the average value, A high-level audio detection step for detecting candidate points for scene division based on a comparison with the audio level of audio data included in the video file, and the cut inspection If the number of candidate points for scene division detected in the step, the local variation point detection step, or the high-level sound detection step is less than or equal to the upper limit number of divisions stored in the storage means, The moving image file is divided according to candidate points, and the number of scene division candidate points detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step exceeds the division upper limit number. And selecting the candidate points for the upper limit number of divisions based on the first numerical value, the second numerical value, or the level of the audio level calculated at each of the candidate points. A divided file number adjusting step for dividing the moving image file according to the candidate points, and a divided moving image divided by the divided file number adjusting step The gist is to execute a display step of transmitting and displaying each of them to the client terminal.

  According to the present invention, it is possible to divide a scene according to the content of a moving image file and to improve the efficiency of moving image editing work.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of a moving image dividing apparatus 100 according to an embodiment of the present invention. 1 includes a cut point recognition unit 101, a local variation location recognition unit 102, a high-level speech recognition unit 103, a divided file number adjustment unit 104, an original video storage unit 111, a parameter information storage unit 112, and The apparatus includes a divided moving image storage unit 113, and appropriately divides the input moving image file 10 into a predetermined number of divisions according to the contents thereof.

  The moving image dividing apparatus 100 includes a central processing unit (CPU) having at least a calculation function and a control function, and an electronic device having a main storage device (memory) including a RAM having a function of storing programs and data. It is configured.

  A program for executing various processes according to the present embodiment is stored in the main storage device or the hard disk described above.

  Among these, the cut point recognition unit 101, the local variation location recognition unit 102, the high-level speech recognition unit 103, and the divided file number adjustment unit 104 are nothing but a concrete example of the arithmetic control function by the CPU. Further, the original moving image storage unit 111, the parameter information storage unit 112, and the divided moving image storage unit 113 have functions of a main storage device.

  The original moving image storage unit 111 stores the moving image file 10 (moving image file to be divided) input to the moving image dividing device 100, and the parameter information storage unit 112 is necessary when dividing the moving image file 10. For example, specifically, a cut point recognition threshold value P1, a local variation point recognition threshold value P2, an average voice level P3, and a division upper limit number necessary for recognition processing in each unit to be described later are stored. P4 and the like are stored.

  The cut point recognizing unit 101 compares the similarities between adjacent frames among the frames constituting the moving image file 10 and calculates the difference (digitizes the value by a predetermined method), and the difference is a cut point recognition threshold value. If it is larger than P1, the corresponding part is recognized as a cut point and stored as a candidate point for scene division.

  Here, the function of the cut point recognition unit 101 will be specifically described with reference to FIG. In FIG. 2A, scenes are switched at points A and B, but the cut point recognition unit 101 detects points A and B by comparing the similarity between frames described above. As described above, the function of the cut point recognition unit 101 is effective for a moving image in which a plurality of scenes that are very common exist in a moving image file.

  The local variation location recognition unit 102 divides the frame of the moving image file into a plurality of logical blocks, compares the similarity between adjacent frames and the luminance value for each block, and calculates the difference (predetermined) If the difference is larger than the local variation location recognition threshold P2, the corresponding location is stored as a candidate point for scene division.

  Here, the function of the local variation location recognition unit 102 will be specifically described with reference to FIGS. 2B and 3. FIG. 3 shows an example in which one image frame is divided into nine blocks. If at least one of these blocks has a difference in similarity and luminance value greater than the local variation location recognition threshold P2, scene division is performed. Are detected as candidate points. In FIG. 2B, since a part locally varies at points C and D before and after the frame on which the telop display is performed, the local variation location recognition unit 102 performs inter-frame variation in each block described above. Points C and D are detected by comparing the difference in similarity and luminance value. As described above, the function in the local variation location recognition unit 102 is effective for a moving image in which a topic changes due to a telop display or the like, although a scene such as a news program does not change.

  The high level voice recognition unit 103 regards a portion where the voice data of the moving image file is relatively high (determined by quantifying the voice level) as a relatively loud voice or sound. The corresponding portion is stored as a candidate point for scene division. More specifically, the high level sound recognition unit 103 calculates an average sound level P3 of all sound data of the moving image file 10, and if the sound data of the moving image file 10 is larger than the average sound level P3, the corresponding portion is determined. This is stored as a candidate point for scene division.

  Here, the function of the high-level speech recognition unit 103 will be specifically described with reference to FIGS. In FIG. 2C, a shout is uttered at point E, and the voice data is at a relatively high level, so the high level voice recognition unit 103 detects this point E. As described above, the function in the high-level speech recognition unit 103 is effective for a moving image that changes to the next content by voice even if there is little scene variation, such as a martial arts dance video. FIG. 4 is a specific example in which the sound level of the sound data is represented by a waveform graph. In this specific example, if the horizontal portion (no sound level) is the average sound level, the points F1,. Are detected as candidate points.

  The divided file number adjustment unit 104 divides the divided file 10 at the scene division candidate points processed by any one of the cut point recognition unit 101, the local variation location recognition unit 102, and the high level speech recognition unit 103. The divided moving images 20a,..., 20n are stored in the divided moving image storage unit 113.

  Further, the division file number adjustment unit 104 stores, in the parameter information storage unit 112, candidate points for scene division processed by any one of the cut point recognition unit 101, the local variation location recognition unit 102, and the high-level speech recognition unit 103. If it is larger than the division upper limit number P4, the number of division upper limit numbers P4 is selected based on a predetermined rule, and the scene is divided at the selected candidate points. Here, for example, as shown in FIG. 5, the predetermined rule is, for example, selected by the number of division upper limit numbers P4 in descending order according to the number of points assigned to each candidate point (FIG. 5). In this case, the division upper limit number P4 is 3), and this score is numerically assigned to the candidate points when the cut point recognition unit 101, the local variation location recognition unit 102, or the high level speech recognition unit 103 processes, for example. It is a thing.

  Next, the operation of the moving picture dividing apparatus 100 according to the present embodiment will be described with reference to FIGS. Here, FIG. 6 is a flowchart showing the operation of the moving image dividing apparatus 100 when the scene is divided by the cut point recognizing unit 101, and FIG. 7 shows the moving image when the scene is divided by the local variation location recognizing unit 102. FIG. 8 is a flowchart showing the operation of the dividing device 100, and FIG. 8 is a flowchart showing the operation of the moving image dividing device 100 when scenes are divided by the high level speech recognition unit 103.

  First, the operation of the cut point recognition unit 101 will be described. The cut point recognizing unit 101 of the moving image dividing apparatus 100 acquires the moving image data of the moving image file 10 to be divided from the original moving image storage unit 111 frame by frame (step S10), and the similarity with the previous frame is expressed numerically. Calculate (step S20). Next, this numerical value is compared with the cut point recognition threshold value P1 acquired from the parameter storage unit 112 (step S30), and when the difference is larger than the threshold value P1, the difference information (scene division candidate points and numerical value). Is stored in a temporary file (step S40). In the present embodiment, it is assumed that the numerical value decreases as the degree of similarity increases. Then, one frame is acquired for all moving image data of the moving image file 10 (step S10), the similarity is calculated for each frame (step S20), and when the difference is larger than the threshold value P1 (step S20). In step S30, the above process of saving the difference information (step S40) is repeated (step S50).

  Next, when the acquisition of all moving image data of the moving image file 10 is completed and there is one or more difference information (step S60), the divided file number adjustment unit 104 determines that the number of difference information is parameter information. It is checked whether or not the division upper limit number P4 acquired from the storage unit 112 has been exceeded (step S70). If the division upper limit number P4 is exceeded, difference information is selected by the division upper limit number P4 in descending order of numerical values (step S80), and the scene is divided based on the selected difference information, and the divided files 20a, ..., 20n are created and stored in the divided moving image storage unit 113 (step S90). If the division upper limit number P4 is not exceeded, the scene is divided based on the difference information, and divided files 20a,..., 20n are created and stored in the divided moving image storage unit 113 (step S90).

  On the other hand, if there is no difference information, it is determined that there is no scene division point by cut point recognition, and the operation proceeds to the next local variation location recognition unit 102 (step S60).

  The local variation part 102 acquires the moving image data of the moving image file 100 frame by frame (step S100), and calculates the similarity with the previous frame and the difference from the luminance value numerically for each block (step S110). . Next, this numerical value is compared with the local variation location recognition threshold value P2 acquired from the parameter storage unit 112. When the difference is larger than the threshold value P2 (step S120), the difference information (scene division candidate points and numerical values) ) Is saved in a temporary file (step S130). In this embodiment, the numerical value increases as the degree of similarity decreases and the difference in luminance value increases. Then, the entire moving image data of the moving image file 10 is acquired frame by frame (step S100), the similarity and the luminance difference are calculated for each block of the frame (step S110), and the difference is larger than the threshold P2. In such a case (step S120), the above process of saving the difference information (step S130) is repeated (step S140).

  Next, when the acquisition of all moving image data of the moving image file 10 is completed and there is one or more difference information (step S150), the divided file number adjustment unit 104 determines that the number of difference information is parameter information. It is checked whether or not the division upper limit number P4 acquired from the storage unit 112 has been exceeded (step S160). If the division upper limit number P4 is exceeded, difference information is selected by the division upper limit number P4 in descending order of numerical values (step S170), and the scene is divided based on the selected difference information, and the divided files 20a, ..., 20n are created and stored in the divided moving image storage unit 113 (step S180). If the division upper limit number P4 is not exceeded, the scene is divided based on the difference information, and divided files 20a,..., 20n are created and stored in the divided moving image storage unit 113 (step S180).

  On the other hand, if there is no difference information, it is determined that there is no scene division point by local variation location recognition, and the operation proceeds to the next operation of the high level speech recognition unit 103 (step S150).

  The high level voice recognition unit 103 acquires all the audio data of the moving image file 10, calculates the average audio level P3 (step S190), acquires the audio data one by one from the moving image file 10 (step S200), and calculates the average Compared with the audio level P3 (step S210), if the acquired audio data is larger than the average audio level P3, the difference information (scene division candidate points and numerical values) is stored in a temporary file (step S220). In the present embodiment, the numerical value increases as the sound level increases. Then, the audio level is obtained one by one for all audio data of the moving image file 10 (step S200), compared with the average audio level P3 (step S210), and if the audio level is higher than the average audio level P3, the difference is obtained. The above process of storing information (step S220) is repeated (step S230).

  Next, when the acquisition of all moving image data of the moving image file 10 is completed and there is one or more difference information (step S240), the divided file number adjustment unit 104 determines that the number of difference information is parameter information. It is checked whether or not the division upper limit number P4 acquired from the storage unit 112 has been exceeded (step S250). If the division upper limit number P4 is exceeded, difference information is selected by the division upper limit number P4 in descending order of numerical values (step S260), and the scene is divided based on the selected difference information, and the divided files 20a, ..., 20n are created and stored in the divided moving image storage unit 113 (step S270). If the division upper limit number is not exceeded, the scene is divided based on the difference information, and divided files 20a,..., 20n are created and stored in the divided moving image storage unit 113 (step S270).

  On the other hand, when there is no difference information, it is determined that there is no scene division point by high-level speech recognition (after all, it is determined that there is no scene division point), so the moving image file 10 is copied and divided moving image storage is performed. The information is stored in the unit 113 (step S280).

  The priority order of the above-described cut point recognition, local variation location recognition, and high-level speech recognition is mainly due to the following reason based on empirical rules.

  If the scene changes (= camera angle changes) during a movie, it is highly likely that “the content being captured (= the content you want to convey) has changed”. In particular, movies taken with home video cameras tend to have a strong tendency. Therefore, first, cut point recognition is given the highest priority. Next, in the change in the movie, even if the scene has not changed (= the camera angle has not changed), it is considered that there is a high possibility that the content to be communicated has changed. Therefore, the local variation location recognition is set to the second priority. Since the “change in the content to be transmitted” is considered to be largely due to the change in the video rather than the change in the voice, the priority order of the high-level voice recognition is set to the third.

  Therefore, according to the moving image dividing apparatus 100 of the present embodiment, first, scene division is performed by cut point recognition, and then, when cut point recognition is not performed, scene division is performed by local variation location recognition, and finally When the local variation part is not recognized, the scene division is performed by high-level voice recognition to divide the moving image file, so that the scene can be divided according to the content of the moving image file. In addition, since the number of divided files can be controlled by determining the division upper limit number, the efficiency of moving image editing work can be improved.

  Next, the case where the moving image dividing device 100 of the present embodiment is applied to a moving image search index system will be described. FIG. 9 is a schematic configuration diagram of the moving image search index assigning system 1 when the moving image editing server 11 includes the function of the moving image dividing apparatus 100 described above.

  The moving image search index assigning system 1 shown in FIG. 9 assigns a search index to each divided moving image constituting a moving image in response to a request from the client terminal 3 connected via the Internet 5. A video editing server 11 for generating and managing user information, a search index, a split video, user authentication, and the like, a text sentence, for dividing a video into a plurality of divided videos (the above-described video dividing method is used) Application server 13 that performs the generation process, content correlation process, and the like, and the stream server 15 that distributes thumbnail display pages of a plurality of divided moving images. In addition, the moving image search index assignment system 1 of this embodiment is connected to a general WWW server 7 via the Internet 5, and the general WWW server 7 has a home page 71.

  With reference to FIG. 10, the outline | summary of the search index provision method for moving images of this embodiment is demonstrated. As shown in FIG. 10, for example, a text sentence including a character string for a search index to be added to an original video owned by a client terminal is selected as appropriate from the Internet, for example, and displayed on the display of the client terminal 3 by a WWW browser. . In FIG. 10, an example of a text sentence “Sea otter was from Chikajima ... half a half from the northern part of Hokkaido” is displayed as an item of a search index (text sentence).

  Then, the original moving image is transferred to the moving image search index assigning system 1 so that a plurality of divided moving images 1, 2, 3,... N are divided into scenes, and the divided divided moving images 1, 2, 3,. N is received by the client terminal 3 and displayed on the display of the client terminal 3 by the WWW browser.

  Here, FIG. 11 shows the flow of the moving image division. According to FIG. 11, the moving image editing server 11 performs the processing shown in FIGS. 6 to 8 on the original moving image uploaded from the client terminal 3, and displays the divided moving images on the browser screen 300 of the client terminal 3.

  FIG. 12 shows the browser screen 300 in more detail. A plurality of divided moving images 1, 2, 3,..., N divided into scenes are displayed in the frame 305 as indicated by reference numeral 310 by the thumbnail function of the stream server 15. The plurality of divided moving images 1, 2, 3,..., N displayed as thumbnails are not simply still images but are displayed as continuous divided moving images in which images are actually moving.

  The user of the client terminal 3 browses the displayed divided videos 1, 2, 3,... N. For example, when a sea otter image or the like is displayed on the divided video 1, it is displayed on the display of the client terminal 3. As a search index character string that is most suitable to be given to the sea otter segmented moving image 1 from the text strings of the text text, for example, a character string of “sea otter” is inverted with the mouse of the client terminal 3 as shown in FIG. To select.

  When the character string “sea otter” for the search index is selected and reversed in this way, the divided video 1 on which the sea otter image or the like is displayed is grabbed and dragged with the mouse pressed, and the character string “sea otter” is displayed. The mouse is released, and the character string “sea otter” for the search index is given to the divided moving image 1 to associate the two with each other.

  Then, the character string “sea otter” and the divided video in the search index related in this way use the divided video name, and the correspondence between the character string and the divided video name is stored in the video index database as a video search index. Will be registered.

  Therefore, according to the moving image search index assigning system 1, the same effect as the moving image dividing apparatus 100 described above can be obtained. In addition to this, each of the divided moving images can be related to the moving image search index by a simple mouse operation on the client terminal 3. That is, a character string can be associated with each of the divided moving images as a search index, and a correspondence between the character string and the divided moving image name can be efficiently and appropriately generated as a search index for moving images.

  In addition, in this association, a plurality of divided videos are displayed as thumbnails, and text sentences are simultaneously displayed in different frames on the same screen for the plurality of divided videos displayed as thumbnails. The user can visually and easily understand the relationship between the character string in the text sentence and the divided moving image by simple mouse operation while browsing both.

  Furthermore, it is possible to link different contents using a moving image as a key. In addition, the association between the divided video and the character string can be performed from anywhere via the Internet 5 from the client terminal 3, and the user only needs to have only the client terminal 3, and a special device is provided. do not need. Video can be easily incorporated into existing content.

  While the embodiments of the present invention have been described above, various modifications and changes can be made to the embodiments of the present invention without departing from the spirit of the present invention. For example, the moving image dividing apparatus 100 according to the present embodiment can be applied as a stand-alone device or as the moving image search index assignment system 1 described above, and can be applied to various computer systems that involve moving image dividing processing. It is.

It is a schematic block diagram of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a figure explaining the division | segmentation image in the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is an example of the block in the local change location recognition of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is an example of the high level audio | voice recognition of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a figure explaining the adjustment method of the scene division number of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a flowchart explaining operation | movement of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a flowchart explaining operation | movement of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a flowchart explaining operation | movement of the moving image division | segmentation apparatus which concerns on embodiment of this invention. It is a schematic block diagram of the search index provision system for moving images to which the moving image dividing device according to the embodiment of the present invention is applied. It is a figure explaining the outline | summary of the search index provision method for moving images. It is a figure explaining the outline | summary of the moving image division | segmentation process in the search index provision system for moving images. It is a figure which shows the browser screen which displayed the some division | segmentation moving image shown by the client terminal.

Explanation of symbols

1 Video Search Indexing System 3 Client Terminal 5 Internet 7 General WWW Server 10 Video File 11 Video Editing Server 13 Application Server 15 Stream Server 20a-20n Divided Video File 71 Home Page 100 Video Dividing Device 101 Cut Point Recognition Unit 102 Local Fluctuation Location Recognition unit 103 High-level voice recognition unit 104 Divided file number adjustment unit 111 Original moving image storage unit 112 Parameter information storage unit 113 Divided moving image storage unit

Claims (6)

A video splitting method that splits a video file into scenes according to the content of the video,
A computer comprising storage means for storing first and second threshold values relating to video segmentation and a segment upper limit number that is the maximum number of scene segments,
A first numerical value related to all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file that is the original moving image before division, and each of the first numerical values, A cut point detecting step for detecting candidate points for scene division based on the comparison with the first threshold value stored in the storage means;
If the candidate point is not detected in the cut point detection step, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. , A local variation location detecting step for detecting candidate points for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage means;
If the candidate point is not detected in the local variation location detection step, an average value of the audio levels of all audio data of the moving image file is calculated, and the average value and the audio level of the audio data included in the moving image file are calculated. And a high-level sound detection step for detecting candidate points for scene division based on the comparison with
When the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high level audio detection step is equal to or less than the division upper limit number stored in the storage unit, The moving image file is divided according to the candidate points for scene division, and the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting step of selecting and dividing the moving image file according to the selected candidate point;
A video dividing method characterized by executing A method of dividing a video in a multimedia search index adding apparatus that creates a divided video that is divided into scenes according to the content of the video, and adds a search index to the divided video in response to a request from a client terminal,
The multimedia search index assigning device comprising storage means for storing first and second threshold values relating to video segmentation and a segment upper limit number that is the maximum number of scene segments,
An original video reception step of receiving a video file that is an original video selected by the client terminal;
A first numerical value for all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file, and each of the first numerical values and the first value stored in the storage unit are calculated. A cut point detecting step for detecting candidate points for scene division based on the comparison with the threshold;
If the candidate point is not detected in the cut point detection step, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. , A local variation location detecting step for detecting candidate points for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage means;
If the candidate point is not detected in the local variation location detection step, an average value of the audio levels of all audio data of the moving image file is calculated, and the average value and the audio level of the audio data included in the moving image file are calculated. And a high-level sound detection step for detecting candidate points for scene division based on the comparison with
When the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high level audio detection step is equal to or less than the division upper limit number stored in the storage unit, The moving image file is divided according to the candidate points for scene division, and the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting step of selecting and dividing the moving image file according to the selected candidate point;
A display step of transmitting and displaying each of the divided videos divided in the divided file number adjustment step to the client terminal;
A video dividing method characterized by executing A video splitting device that splits a video file into scenes according to the content of the video,
Storage means for storing first and second threshold values relating to video segmentation, and a division upper limit number which is the maximum number of scene divisions;
A first numerical value related to all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file that is the original moving image before division, and each of the first numerical values, Cut point detection means for detecting candidate points for scene division based on comparison with the first threshold value stored in the storage means;
If the candidate point is not detected by the cut point detection means, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. Each of the second numerical values and the second threshold value stored in the storage means, based on a comparison between the local variation location detection means for detecting candidate points for scene division;
If the candidate point is not detected by the local variation location detection means, the average value of the audio levels of all the audio data of the moving file is calculated, and the average value and the audio of the audio data included in the moving image file are calculated. High-level sound detection means for detecting candidate points for scene division based on the comparison with the level;
When the number of scene division candidate points detected by the cut point detection means, the local variation location detection means, or the high-level sound detection means is less than or equal to the division upper limit number stored in the storage means, The moving image file is divided according to the scene division candidate points, and the number of scene division candidate points detected by the cut point detection unit, the local variation point detection unit, or the high-level sound detection unit is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting means for selecting and dividing the moving image file according to the selected candidate point;
A moving picture dividing apparatus comprising: A multimedia search index providing device that creates a divided video that is divided into scenes according to the content of the video, and adds a search index to the divided video in response to a request from a client terminal,
Storage means for storing first and second threshold values relating to video segmentation, and a division upper limit number which is the maximum number of scene divisions;
Original moving image receiving means for receiving a moving image file that is an original moving image selected by the client terminal;
A first numerical value for all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file, and each of the first numerical values and the first value stored in the storage unit are calculated. Cut point detection means for detecting candidate points for scene division based on the comparison with the threshold;
If the candidate point is not detected by the cut point detection means, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. Each of the second numerical values and the second threshold value stored in the storage means, based on a comparison between the local variation location detection means for detecting candidate points for scene division;
If the candidate point is not detected by the local variation location detection means, the average value of the audio levels of all audio data of the video file is calculated, and the average value and the audio level of the audio data included in the video file And high-level sound detection means for detecting candidate points for scene division based on the comparison of
When the number of scene division candidate points detected by the cut point detection means, the local variation location detection means, or the high-level sound detection means is less than or equal to the division upper limit number stored in the storage means, The moving image file is divided according to the scene division candidate points, and the number of scene division candidate points detected by the cut point detection unit, the local variation point detection unit, or the high-level sound detection unit is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting means for selecting and dividing the moving image file according to the selected candidate point;
Display means for transmitting and displaying each of the divided videos divided by the divided file number adjusting means to the client terminal;
A search index assigning apparatus for multimedia, comprising: A video segmentation program that divides a video file into scenes according to the content of the video,
A computer comprising storage means for storing first and second threshold values relating to video segmentation, and a division upper limit number which is the maximum number of scene divisions,
A first numerical value related to all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file that is the original moving image before division, and each of the first numerical values, A cut point detecting step for detecting candidate points for scene division based on the comparison with the first threshold value stored in the storage means;
If the candidate point is not detected in the cut point detection step, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. , A local variation location detecting step for detecting candidate points for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage means;
If the candidate point is not detected in the local variation location detection step, an average value of the audio levels of all audio data of the moving image file is calculated, and the average value and the audio level of the audio data included in the moving image file are calculated. And a high-level sound detection step for detecting candidate points for scene division based on the comparison with
When the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high level audio detection step is equal to or less than the division upper limit number stored in the storage unit, The moving image file is divided according to the candidate points for scene division, and the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting step of selecting and dividing the moving image file according to the selected candidate point;
A video segmentation program characterized by causing A video segmentation program for a multimedia search index assigning device that creates a segmented video segmented according to the content of a video and adds a search index to the segmented video in response to a request from a client terminal,
In the multimedia search index assigning device comprising storage means for storing first and second threshold values relating to video segmentation, and an upper limit number of divisions which is the maximum number of scene divisions
An original video reception step of receiving a video file that is an original video selected by the client terminal;
A first numerical value for all similarities between temporally adjacent frames of the moving image file is calculated for the moving image file, and each of the first numerical values and the first value stored in the storage unit are calculated. A cut point detecting step for detecting candidate points for scene division based on the comparison with the threshold;
If the candidate point is not detected in the cut point detection step, a second numerical value related to all similarities and luminance differences between the temporally adjacent frames is calculated in each block obtained by dividing the frame into a plurality of blocks. , A local variation location detecting step for detecting candidate points for scene division based on a comparison between each of the second numerical values and the second threshold value stored in the storage means;
If the candidate point is not detected in the local variation location detection step, an average value of the audio levels of all audio data of the moving image file is calculated, and the average value and the audio level of the audio data included in the moving image file are calculated. And a high-level sound detection step for detecting candidate points for scene division based on the comparison with
When the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high level audio detection step is equal to or less than the division upper limit number stored in the storage unit, The moving image file is divided according to the candidate points for scene division, and the number of candidate points for scene division detected in the cut point detection step, the local variation point detection step, or the high-level sound detection step is the division upper limit. If the number exceeds the number, the number of candidate points corresponding to the upper limit number of divisions is determined based on the first numerical value, the second numerical value, or the level of the audio level calculated at each candidate point. A dividing file number adjusting step of selecting and dividing the moving image file according to the selected candidate point;
A display step of transmitting and displaying each of the divided videos divided in the divided file number adjustment step to the client terminal;
A video segmentation program characterized by causing

Images