JP3685208B2 - Feature scene detection system - Google Patents

Description

  The present invention relates to a feature scene detection system that detects and displays a feature scene from an image.

  While TV technology and production techniques are rapidly advancing, special effects of video such as flashing light seen in animation programs and sudden changes in scenes are damaging to the health of some viewers. .

  When such a special effect is detected, if a human confirms it while moving forward one frame at a time, even if it can be confirmed at the speed of one frame per second, it takes 48 hours for a two-hour movie. It is physically impossible to investigate all the broadcast videos.

  Therefore, in the conventional method and apparatus, regarding the scene of the flashing light, each frame can be regarded as a short cut. Therefore, the subliminal image detection device described in Patent Document 1 is used as a substitute for detection. I was doing it.

JP-A-9-74575

  With the above-described conventional technology, it is possible to detect a portion exceeding the reference in the video based on the reference value for extracting the video that may impair the health of the viewer. However, since the user does not know the specific feature amount of the extracted scene, there is a problem that it is impossible to compare how much it exceeds the reference value. Further, in the blinking scene, there is a problem that it is difficult to see the range of the blinking scene although the list display is performed in order to detect a large number of still images as characteristic scenes.

  An object of the present invention is to display a feature amount of a feature scene so that a user of this system can easily compare with a reference value. Furthermore, the feature scenes are classified by special effect detection factors and detection times and displayed in a list so that the contents of the feature scenes can be easily grasped.

  In order to achieve the above object, the present invention extracts a still image from a video composed of a plurality of continuous still images, and for the still image, a feature amount relating to a blinking scene of the video, the still image, and the still images before and after the still image. The difference between feature quantities is calculated. Next, a still image that becomes a blinking screen is detected according to the difference between the calculated feature amounts, and registered in the list in the order of detection together with the detection time and the feature amount. Then, the first still image registered at the beginning of the list and the second still image registered after the first still image are extracted. When the interval between the detection time of the first still image and the detection time of the second still image is smaller than a predetermined value, the first still image and the second still image are added to the first group, and the first If the interval between the still image detection time and the second still image detection time is equal to or greater than a predetermined value, the first still image and the second still image are added to the second group. The still image belonging to the first group is displayed in the first area together with the feature amount from which the still image is detected, and the still image belonging to the second group is displayed as the feature amount from which the still image is detected. And the structure displayed on a 2nd area | region was employ | adopted.

  According to the present invention, it is possible to easily grasp the contents of the detected characteristic scenes by detecting the characteristic scenes from the video and displaying the detection results as a list.

  Hereinafter, embodiments of the present invention will be described in detail.

  FIG. 1 is an example of a schematic block diagram of a system configuration for realizing the present invention. Reference numeral 101 denotes a display device such as a CRT, which displays an output result of the computer 110. Commands to the computer 110 can be performed using an input device such as the keyboard 104 and the mouse 103. A video playback device 109 is a tuner device for receiving broadcast programs such as terrestrial broadcasting, satellite broadcasting, and cable television, or a device for playing back moving images stored on an optical disc, a video tape, or the like. An instruction to the computer 110 may be issued from this video reproduction apparatus. A video signal output from the moving image playback device sequentially converts each frame of the video into a digital still image by the video input device 108 such as a capture board, and is sent to the computer 110. Inside the computer, the digital still image is processed by the CPU 106 in accordance with a program that enters the memory 105 via the interface 107 and is stored in the memory 105. Depending on the processing contents of the CPU, various information such as digital still images and CPU processing results can be stored in the auxiliary storage device 102. Next, the processing contents of the CPU will be described in detail.

  FIG. 2 is an example of a flowchart of a program feature scene detection program executed on the system shown in FIG.

  The program is stored in the memory 105 of FIG. 1 and is executed under the following conditions.

  (1) The user instructs the start of detection processing using the mouse 103 or the keyboard 104 of FIG.

  (2) The start of detection processing is instructed from the video reproduction device 109 in FIG.

  When the program is executed, a digital still image fi is taken in from the interface 107 in FIG. 1 (201). Next, the following feature amounts are calculated for the digital still image fi and stored in the memory 105 of FIG. 1 (202).

  202- (1): The average luminance of the digital still image fi is obtained. This result is used to detect a feature value based on the ratio of the luminance difference between the still image and the immediately preceding still image.

  202- (2): The digital still image fi is divided into grid blocks, and the average luminance is obtained for each block. As a result, the average luminance difference between the still image and the immediately preceding still image is small, but when the same dot is compared at the same dot position between the still images, the feature amount due to the ratio of the luminance difference between the still images is large. Used for detection.

  202- (3): Obtain color distribution information of a digital still image fi. This result is used to calculate the area ratio of the red component dots.

  202- (4): The digital still image fi is divided into grid-like blocks, and an average value of RGB values is obtained for each block. This result is used to detect short cuts and logical changes in the video.

  Based on the feature amount calculated in 202, it is determined whether the digital still image fi corresponds to any of the following (203).

  203- (1): When the difference between the feature values of the digital still image fi and fi-1 obtained in 202- (1) is found, and the value exceeds a preset threshold. When this is the case, a feature scene is created by the ratio of the luminance difference between the still image and the immediately preceding still image.

  203- (2): When the following conditions are satisfied based on the feature value obtained in 202- (2). When this is the case, the feature is due to the ratio of the difference in the luminance of the still image where the average luminance difference between the still image and the previous still image is small but the luminance difference is large when compared at the same dot position between the still images. It becomes a scene.

  -When the difference between the features of digital still images fi and fi-1 is obtained and the value exceeds the preset threshold.

  ・ When the number of blocks exceeding the above threshold exceeds a preset threshold.

  203- (3): The difference of the feature value in each combination of digital still image fi and fi-1, fi-1 and fi-2, and fi and fi-2 from the feature value obtained in 202- (4) When it is calculated and the value exceeds the preset threshold value in a certain pattern. When this is the case, a feature scene is created by a logical change point of the video.

  203- (4): When the following conditions are satisfied based on the feature value obtained in 202- (4). When this is the case, the feature scene is a short cut.

  (a) When a difference is obtained from the feature values of digital still images fi and fi-1, and the value exceeds a preset threshold value.

  (b) A still image satisfying the above condition (a) is stored in the memory 105 of FIG. 1, and the time length of the digital still image Sj and the digital still image Sj-1 detected before is obtained. , If its length is shorter than the preset threshold.

If the result of the determination in 203 is that the digital still image fi is determined to be a feature scene, the fi and its feature value are added to the list. This list is managed in the order in which feature scenes are detected. (204). Furthermore, a list of digital still images fi is displayed on the display 101 of FIG. 1 as shown in FIG. (205)
As described above, by performing the processing from 201 to 205, the processing for the digital still image fi is completed. Subsequently, it is determined whether or not to process the next digital still image fi + 1. At this time, if any of the following conditions is met, the detection process ends, and the process 207 is performed.

  (1) The user instructs the end of the detection process with the mouse 103 or the keyboard 104 of FIG.

  (2) The end of the detection process is instructed from the video reproduction device 109 of FIG.

  If the detection process is canceled in the above process 206, the detection time intervals of the feature scenes registered in the list are grouped in a portion that is equal to or smaller than a preset threshold value and classified into groups (207 ). Detailed processing of 207 will be described with reference to FIG.

  In the process of step 207, first, variables are initialized (801). Next, it is determined whether the feature scene Ci is the last element in the list (802). If the determination result is YES, Ci is set as the last group Gj from the first feature scene GFj of the group, and the process ends (810). If the determination result is NO, processing from 803 to 809 is processed as loop iteration (one loop processing). In this part, first, the detection times of the feature scene Ci and the next feature scene Ci + 1 are extracted from the list (803 and 804). Next, it is determined whether the interval between the detection times of Ci and Ci + 1 is equal to or greater than a threshold value (805). If the determination result is YES, group addition processing (806 to 808) is performed. In this process, Ci is added as a group Gj to the group list from the head feature scene GFj of the group (806), the head of the next group is set to GFj + 1 (807), and the group counter j is incremented (808). . If NO, do not add a group. Finally, i which is a counter of the list is incremented (809), and the processing returns to step 802. In this way, the process 207 is performed.

  As a result of this processing, a group list as shown in FIG. 9 is created. Reference numeral 901 denotes a group list that is a list of groups, and reference numeral 902 denotes a group element list that is a list of elements of one group. Reference numeral 903 denotes a group element, which has a structure indicating the group element list 902 from the group list 901. The group element list 902 manages elements (904) including feature scene information (still images, feature amounts, time, and other additional information).

  Next, the feature scenes included in the group list created in the above process 207 are grouped into portions where the number of detections per second is equal to or greater than a preset threshold value, and further classified into groups (208). By this process, the blinking section of the feature scene is searched, and the blinking section can be easily understood when the list is displayed. Details of the processing 208 will be described with reference to FIGS. 10 and 11. The processing in this figure is processing for one group element list 902, and is actually executed as many times as the number of groups.

  However, only one of the processes 207 and 208 can be performed. Therefore, in this case, the processing of 208 can be performed on the list, and the blinking section and other portions can be classified as groups.

  In this process, the blinking range of the feature scene included in one group element list in the process of FIG. 10 is obtained. Then, the frequency of the blinking section is calculated in the process of FIG.

  First, the function of the block in FIG. 10 will be described below. In this process, first, variables are initialized (1001). Next, a preset threshold value n for the number of detections per second is compared with the number of elements in the group element list to determine whether n is larger than the number of elements (1002). As a result, in the case of YES, the process ends because there is no blinking section in this group. In the case of NO, the range of the blinking section is searched by processing from 1003 to 1007. Next, processing in the case of NO will be described. First, it is determined whether the feature scene Ci + n exists in the list of elements (1003). As a result, in the case of YES, since the search for the blinking section is completed, the processing of FIG. 11 is started next. In the case of NO, the determination of the blinking section is continued. Next, the blinking zone determination process will be described. First, the detection time of Ci + n is extracted from the feature scene Ci from the group element list 902 (1004). Next, it is determined whether the detection interval from Ci to Ci + n is within one second (1005). If the result is YES, the blinking image section flag from Ci to Ci + n is turned ON (1006). If NO, do nothing. Finally, the element counter i is incremented (1007). In this way, the blinking section is identified.

  Next, the function of FIG. 11 will be described. First, the group element list counter is initialized (1008). Next, it is determined whether the blinking section flag of the feature scene Ci is ON (1009). If the result is YES, since the head of the blinking section has been found, the tail of the blinking section is searched and the frequency of the blinking section is calculated (1010 to 1016). In this process, first, the search start position of the end position of the blinking section is set to the variable j (1010). Then, Cj does not exist in the group element list (1011), or j is incremented (1013) until the blinking section flag of Cj is turned off (1012), and the tail Cj-1 of the blinking section is obtained. . Based on the result, the number of blinks per second is calculated (1014). Then, the number of blinks is set as additional information from Ci to Cj-1 (1015). Thus, since the search for the number of blinks up to Cj is completed, the start position j at the beginning of the next blinking section is substituted for i (1016). Thereby, the process which calculates the frequency of a blink area is performed.

  When the determination of 1009 is NO, the group element list counter i is incremented (1017). Finally, it is determined whether Ci does not exist in the element list. If the result is YES, all processing 208 has been completed. If NO, the process returns to 1009 to search for the start point of the next blinking section. In this way, the process 208 is performed.

  Finally, the processing results from 207 to 208 are displayed as a list on the display 101 of FIG. 1 as shown in FIG. 5 (209). Next, the user interface of this system will be described.

  FIG. 3 is an example of a user interface of a system that executes the program of FIG.

  This user interface is displayed on the display 101 of FIG. Next, the user interface will be described in detail.

  301 is a mouse pointer. The user uses the mouse 103 or the keyboard 104 of FIG. 1 to operate the mouse pointer and point to an arbitrary position on the user interface.

  Reference numeral 302 denotes a display area of a video to be detected.

  Reference numeral 303 denotes a detection operation start button. By pressing this button with the mouse pointer, the program of FIG. 2 is executed.

  Reference numeral 304 denotes an end button. When this button is pressed with the mouse pointer, the program detection process in FIG. 2 ends.

  Reference numeral 305 denotes a feature scene list area. Here, a list of feature scenes detected by the program of FIG. 2 is displayed. During the detection process, it is displayed as shown in FIG. 4, and after the detection process is completed, it is displayed as shown in FIG.

  Reference numeral 306 denotes a scroll bar. This is used when all the detection results do not enter the area 305 when displaying the detection results after the detection processing is completed.

  The buttons and scroll bars used in the user interface are existing technologies, and will not be described in detail here. Next, the detection result list method will be described.

  FIG. 4 is an example of a detection result list method during the detection process. When a characteristic scene is found, it is displayed in order from the position 401.

  FIG. 5 is an example of a detection result list method after the end of the detection process.

  Reference numeral 501 denotes a group icon display area. In the program of FIG. 2, the group information classified by the feature scene detection interval is displayed.

  Reference numeral 502 denotes an icon display area. This is an area for displaying feature scenes belonging to groups classified by the program of FIG.

  Reference numeral 503 denotes a group-to-group boundary line.

  Reference numeral 504 denotes a boundary line between the group icon display area and the icon display area. Next, icons and group icons will be described.

  FIG. 6 is an example of an icon. The icon displays the feature scene, its feature amount, and detection time.

  Reference numeral 601 displays a digital still image serving as a feature scene. Reference numeral 602 displays a feature scene detection time. Reference numeral 603 denotes an area for displaying the feature amount of the feature scene. When one square represents one feature amount, and the feature amount is detected, the background here is filled with a specific color, and the feature amount of the image is displayed numerically. Thereby, the detection factor of this characteristic scene can be understood at a glance. Reference numeral 604 denotes an area in which the number of detections per second when classified by the process 208 of the program in FIG. 2 is displayed as a frequency. Reference numeral 605 denotes an image thickness indicating that the same image continues when it is detected by the detection factor 203- (4) in the program of FIG. 2. Reference numeral 606 denotes an icon background. When 604 is displayed, the background of the icon is painted with a color such as red. This indicates that this characteristic scene is harmful to the human body at a glance.

  FIG. 7 is a group icon representing group information classified by the feature scene detection interval. Reference numeral 701 denotes a feature scene of the head icon of an icon belonging to a group. Reference numeral 702 denotes a group serial number. Reference numeral 703 denotes the detection time of the first icon of the icons belonging to the group. Reference numeral 704 denotes the detection time of the last icon of the icons belonging to the group. Reference numeral 705 denotes the number of icons belonging to the group. Reference numeral 706 denotes the average number of detections per second in the group obtained from the information of 703, 704, and 705. Reference numeral 707 denotes a group icon background. If the icon belonging to the group includes a feature scene classified by the processing of 208, this area is filled with a color such as red.

  The feature scene is detected by the above processing method and display method.

It is a block schematic diagram of a system configuration in one example to which the present invention is applied. It is a flowchart figure of the characteristic scene detection program in one Example with which this invention is applied. It is a figure showing the user interface of the system in one Example with which this invention is applied. It is a figure showing the detection result list method in the detection process in one Example with which this invention is applied. It is a figure showing the detection result list method after completion | finish of the detection process in one Example with which this invention is applied. It is a figure explaining the icon which displays the feature scene and its feature-value in one Example to which this invention is applied. It is a figure explaining the group icon showing the information of the group classified according to the detection interval of the feature scene in one example to which the present invention is applied. It is a flowchart figure of the program classified into a group by the detection interval of the feature scene in one Example to which the present invention is applied. It is a block diagram of the group list | wrist in one Example with which this invention is applied. It is a flowchart figure of the program classified into a group by the frequency | count of detection of the characteristic scene in 1 second in one Example to which this invention is applied. It is a flowchart figure of the program classified into a group by the frequency | count of detection of the characteristic scene for 1 second following the flowchart of FIG.

Explanation of symbols

201: Processing for capturing a frame image fi, 202: Processing for calculating a feature quantity of fi, 203: Processing for determining whether fi is a feature scene, 204: Processing for adding fi to the list, 205: Fi on the screen Processing to display, 206: Processing to stop processing by a user operation or video playback device, 207: Processing to classify feature scenes registered in the list at the detection interval, 208: Detection of feature scenes registered in the list for 1 second Processing to classify by the number of times, 209: Processing to display the classification result

Claims (3)

In a feature scene detection system that detects a characteristic scene of a video,
Means for extracting a still image from a video composed of a plurality of continuous still images;
Means for calculating a feature amount related to a blinking scene of the video with respect to the extracted still image;
Means for calculating a difference in feature amount between the still image and the still images before and after the still image based on the calculated feature amount;
Means for detecting a still image that becomes a blinking screen according to a difference in feature amount between the calculated still image and the still images before and after the still image;
Means for registering the detected still image in a list in the order of detection together with a detection time and a feature amount;
Means for retrieving a first still image registered at the beginning of the list after detecting the still image;
Means for retrieving a second still image registered after the first still image;
If the interval between the detection time of the first still image and the detection time of the second still image is smaller than a predetermined value, the first still image and the second still image are added to the first group. Means to
If the interval between the detection time of the first still image and the detection time of the second still image is equal to or greater than a predetermined value, the first still image and the second still image are Means to add to the group,
Means for displaying a still image belonging to the first group in a first area together with a feature amount in which the still image is detected;
A feature scene detection system comprising: means for displaying a still image belonging to the second group in a second region together with a feature amount in which the still image is detected.   The feature scene detection system according to claim 1, wherein the feature amount of the still image includes luminance of the still image. The feature scene detection system according to claim 1, wherein the feature amount of the still image includes a ratio of a red component included in the still image.

Images