JP4647551B2 - Slow playback section detection device, slow playback section detection method and program - Google Patents

Description

  The present invention relates to a slow playback section detection device that detects a slow playback section included in image data and related technology.

Patent Documents 1 and 2 include techniques for detecting a playback section (hereinafter referred to as slow playback) using slow motion that appears in image data.
These Patent Documents 1 and 2 disclose techniques that focus on the fact that the same frame image is continuously displayed in a section where slow reproduction is performed.

  The technique described in Patent Document 1 detects a section in which increase / decrease in the amount of change in the frame image between adjacent frames is repeated as a slow playback section.

  The technique described in Patent Document 2 detects a section in which the states of “motion field” and “still field” repeatedly appear at short time intervals as a slow playback section. In the case of a moving image, the field image changes sequentially, so the correlation becomes low. In the case of a still image, the same field image is repeated, so the correlation becomes high. Focusing on the level of this correlation, it is detected as a slow playback section in which the “motion field” and “still field” repeat. For the obtained slow playback section, the slow playback magnification of the section can be obtained from the number of fields.

JP 2002-238030 A JP 2003-333505 A

For example, in a competition that competes for rankings mainly in a time trial format, such as track competition, speed skating, horse racing, bicycle racing, bicycle racing, etc., especially in the slow reproduction of the goal scene (hereinafter referred to as “the athlete”). To provide the viewer with an easy-to-understand video of how many people are wearing (not only humans but also horses and vehicles). The playback speed is intentionally adjusted, such as by slowing down. In such slow playback, the number of frames to be continuously displayed during slow playback is switched from a few frames (about 2 to 5 frames) to a few tens of frames (about 60 to 90 frames) in sequence, depending on the situation of the subject. Is realized.
Also, since the slow playback video of the goal scene is often shot with a dedicated camera installed at the goal point, what is the goal scene included in the video that continuously captures the situation during the race from the start to the goal? , Sometimes taken from a different angle.

In the conventional slow playback section detection method as described above, it is difficult to detect the slow playback section of the goal scene of the time trial competition for the following reason.
In the technique described in Patent Document 1, focusing on the fact that the motion vector is periodically stationary in the slow playback section, the periodic slow playback is detected as slow playback with no change in the playback speed. However, the slow playback of the goal scene of the time trial competition includes, for example, a large number of still images composed of several tens of frames, and does not include a frame in which a motion vector periodically stops. For this reason, it is difficult to detect the slow reproduction in the goal scene of the time trial competition by the technique described in Patent Document 1.

  In the technique described in Patent Document 2, the goal scene of the time trial competition as described above, which includes several tens of frames of still images by increasing the threshold setting for the number of consecutive “motion fields” and “still fields”. In some cases, slow playback can be detected. However, “shoot a stationary object with a fixed camera (for example, 1 second)”, followed by “perform panning of the camera (for example, 2 seconds)”, then continue with “take a stationary object with a fixed camera (for example, 1 second) If the video data obtained by “shooting” is a target to be detected, this video data contains many frames having almost the same image, so it is erroneously detected as video data in the slow playback section. There is. In addition, in the technique described in Patent Document 2, since a section composed of a moving image and a still image each having a number of frames equal to or less than a predetermined threshold is detected as a slow playback section, a slow playback section in which the playback speed changes is detected. There are cases where it is not possible.

  Slow playback is realized by continuously displaying a plurality of frames having substantially the same image. The speed of slow playback can be determined by adjusting the number of times these frames are displayed continuously.

  An object of the present invention is to provide a method for correctly detecting a slow playback section having a large number of frames while suppressing erroneous detection.

In order to solve the above problems, the present invention provides a slow playback section detection device, a slow playback section detection method, and a computer program.
A slow playback section detection device provided by the present invention is a slow playback section detection device that includes a memory and a control unit, and detects a slow playback section included in video data composed of continuous frames. Inter-frame information including a first value indicating that the image state transitions between the frames of the moving image data and a second value indicating that the image state does not transition is registered, and the control means includes the frame A section composed of a plurality of consecutive frames including a change from the first value to the second value in the inter-period information is detected as a candidate section, the last frame of the candidate section, and a frame following the last frame, When the image state between the frames does not transition, the frame following the last frame is added to the candidate section, and the addition of the frame to the candidate section is predetermined. Repeated until matching the matter, when the candidate interval matches the predetermined condition, detecting the candidate section as slow playback section. Thereby, even if it is moving image data with many frames, a slow reproduction area can be detected correctly.

  In one embodiment, the control device includes, in the inter-frame information, a change from a second value to a first value and a change from the first value to the second value that follows the change continuously. A section composed of consecutive frames is detected as a candidate section. As a result, even in the case of moving image data with a large number of frames, including frames in which the transition of the image state between frames continues continuously as “none”, “yes”, “no”, the slow playback section can be detected correctly.

  In one embodiment, the memory further stores direction information indicating a direction of a moving image change between the frames when the transition of the image state between the frames is represented by a first value. Motion vector detection means for detecting a motion vector representing motion of moving image data between frames to be registered, and registration means for registering the direction information generated based on the distribution of the direction of the motion vector detected by the motion vector detection means; Is further provided. For this reason, by using this direction information, it is possible to increase the certainty of the moving direction of the moving image data.

  In one embodiment, the direction of the motion vector is classified into a predetermined number, a histogram creation unit that creates a histogram composed of a frequency distribution in each direction, and an average frequency calculation unit that calculates an average frequency of the frequency distribution of the histogram. The direction information is generated by the control means, and the control means is configured such that a difference between a maximum frequency of the direction distribution of the motion vector in the histogram and the average frequency meets a predetermined difference condition. It is determined that the image state between the frames has changed. For this reason, since the apparatus of such a structure can analyze the tendency of a motion vector, the accuracy of candidate section detection can be improved.

  A slow playback section detection method provided by the present invention is a slow playback section detection method for detecting a slow playback section included in video data composed of continuous frames, and an image between frames of video data composed of continuous frames. Control means accessing the memory in which the interframe information including the first value indicating that the state transitions and the second value indicating that the image state does not transition is registered from the first value to the interframe information. A section composed of a plurality of consecutive frames including a change to the second value is detected as a candidate section, and an image state between the last frame of the candidate section and a frame following the last frame is the first. If it is binary, the frame following the last frame is added to the candidate section, and the addition of the frame to the candidate section is made a predetermined condition. Repeat until match, when the candidate interval matches the predetermined condition, detecting the candidate section as slow playback section.

  The program for detecting a slow playback section provided by the present invention is composed of continuous frames, and the first value indicating that the image state transitions between each frame of moving image data including the slow playback section and the image state does not transition. A computer capable of accessing a memory in which inter-frame information including a second value representing the above is registered is composed of a plurality of consecutive frames including a change from the first value to the second value in the inter-frame information. When a section is detected as a candidate section, and the image state between the last frame of the candidate section and the frame following the last frame is a second value, the frame following the last frame is selected as the candidate When adding to a section and repeating adding a frame to the candidate section until a predetermined condition is met, and the candidate section meets the predetermined condition , Detects the candidate section as slow reproduction section, control means to function as a.

  According to the present invention, it is possible to detect a slow playback section consisting of several tens of frames, and an image having a structure such as a still image / moving picture / still image by panning of a camera is erroneously detected as a slow playback section. Can be suppressed. In addition, since the slow playback section is detected based on the presence or absence of motion in the image state between frames and the number of frames in the entire slow playback candidate section, even an image whose slow playback speed changes can be detected with high accuracy. be able to.

Embodiments of a slow playback section detection device to which the present invention is applied will be described below.
The reproduction of moving images by slow motion (hereinafter, slow reproduction) is realized by presenting the same frame image a plurality of times in succession. The speed of this slow reproduction is determined by adjusting the number of times that frames having substantially the same image are continuously presented.
Here, the phrase “frame having substantially the same image” refers to a frame having an image with a very small image difference to the extent that the movement of the image cannot be recognized with the naked eye.

[Constitution]
As shown in FIG. 1, the slow playback detection apparatus 10 of this embodiment includes a main control unit 11, an image database 12, a motion vector detection unit 13, a histogram creation unit 14, an average power calculation unit 15, and a database management system ( Hereinafter, a DBMS) 16 is provided, and a slow playback section included in moving image data composed of continuous frames is detected.
The main control unit 11 has a function of comprehensively processing the image data registered in the image database 12, and in cooperation with the motion vector detection unit 13, the histogram creation unit 14, the average power calculation unit 15, and the DBMS 16, Processing according to processing procedures shown in FIGS. 4 and 5 described later is executed. The main control unit 11 includes a counter that counts the number of frames to be described later and the number of motion “none” in the image state between frames.

In this embodiment, as an example, the slow playback section of the goal scene of the time trial competition is set as a detection target. Slow playback in the goal scene of time trial competition has the following characteristics.
・ In order to make it easier for viewers to understand the order of passing the goal line, the playback speed of the video is particularly slow, such as slowing down the playback speed at the time of the goal (when the player is on the line before or after the goal line). Adjustments are made irregularly. For this reason, the number of frames having substantially the same image displayed in the slow playback section is not constant, and is adjusted according to the goal scene.
The number of frames having substantially the same image displayed in the slow playback section is very wide, from about several frames (2 to 5 frames) to about several tens of frames (30 to 90 frames).
・ The goal scene is shot with a fixed camera or a camera with a slow panning speed. Often taken from a direction perpendicular to the direction of travel of the competitor (eg, directly above or directly beside). Accordingly, the frequency of the motion vector in the right direction classified by the goal scene image and the histogram is, for example, as shown in FIG.
-The direction of travel of the subject is one direction. For example, as shown in FIG. 2A, the player progresses from the left to the right of the screen (or in the opposite direction).

In the image database 12, whether or not the image state between each frame of the moving image data composed of continuous frames is a cut point, the type of “present” / “none” of the movement of the image state when it is not the cut point, In addition, when the image state has a motion, the direction of the motion is registered as interframe information.
In the image database 12, a slow reproduction candidate section described later is also registered.

FIG. 3 is a diagram illustrating an example of interframe information.
This inter-frame information includes a first value indicating that the image state transitions between each frame of the moving image data, a second value indicating that the image state does not transition, and a first value indicating the transition of the image state between frames. Direction information indicating the direction of change of the moving image between frames when represented. For example, in this embodiment, the first value uses the logic High representing the motion “present”, the second value uses the logic Low representing the motion “none”, and represents the direction of change of the moving image between frames. The direction of the motion vector is used as the direction information, but the direction information is not limited to these examples. For example, a certain numerical value may be used as the first value, and a numerical value different from the first value may be used as the second value.
Note that the inter-frame information does not necessarily include the direction information, and may include only information indicating the presence / absence of image state transition.

  The inter-frame information shown in FIG. 3 is information before and after the goal scene of the time trial competition in which the player moves from the left to the right of the screen. The slow playback section is between the cut point between the frame numbers 1001 and 1002 and the cut point between the frame numbers 1950 and 1951. For this reason, the direction of the motion vector in the case of motion “included” included in this slow playback section is all “right”. Note that the image between the two cut points was taken with a fixed camera or a camera with a very slow panning speed.

  Also, the frames before frame number 1001 and the frames after frame number 1951 are frames that constitute a normal moving image that is not slow playback. Here, by way of example, the image state between these frames is represented by a motion “present”. ", The direction is" up ". Note that this normal movie may be shot with the same camera as the fixed camera that shoots images in the slow playback section, but FIG. 3 shows another camera that is different from the fixed camera that shoots images in the slow playback section. Shows the inter-frame information captured by.

The motion vector detection unit 13 detects a motion vector representing the motion of the moving image data between successive frames and counts the number of motion vectors.
The histogram creation unit 14 classifies the directions of the motion vectors into eight directions, and creates a histogram including a frequency distribution in each direction.
The average frequency calculation unit 15 calculates the average frequency of the histogram frequency distribution created by the histogram creation unit 14.
The DBMS 16 manages registration of data in the image database 12 and reading of data in the image database 12.

[Operation]
The processing contents of the slow reproduction detection apparatus of the present invention will be described below using the processing procedure diagrams shown in FIGS. 4 and 5 and the interframe information shown in FIG.
The main control unit 11 extracts two consecutive frames from the moving image data registered in the image database 12 (Step S1), and the interval between the two consecutive frames extracted in Step S1 is a cut point. It is determined whether or not (step S2). A known technique may be used for determining the cut point. When it is determined that it is not a cut point, the main control unit 11 instructs the motion vector detection unit 13 to detect a motion vector.
The motion vector detection unit 13 detects a motion vector from the image data of each frame sandwiching the frames (step S3). In the present embodiment, a case where a motion vector is detected will be described. For example, when moving image data is compressed data and the frame itself has motion vector information, as in the present embodiment. The motion vector information included in the frame can be used without detecting the motion vector.
The motion vector detection unit 13 counts the number of motion vectors detected in step S3 (step S4), and transmits the number and direction of motion vectors to the main control unit 11.

The main control unit 11 determines whether or not the number of motion vectors is equal to or greater than a predetermined threshold (t1) (step S5). When the number of motion vectors is equal to or greater than the threshold value (t1), the main control unit 11 instructs the histogram creation unit 14 to create a histogram.
The histogram creation unit 14 creates a histogram in which the direction of the motion vector is classified into eight directions (step S6), and transmits the contents of this histogram to the main control unit 11.
The main control unit 11 transmits the contents of the histogram to the average power calculation unit 15.
The average frequency calculation unit 15 calculates the average frequency of the frequencies in the eight directions based on the contents of the histogram (step S7), and transmits the calculation result to the main control unit 11.

The main control unit 11 determines whether the difference between the maximum frequency of the frequencies classified in the eight directions in the histogram and the average frequency calculated by the average frequency calculation unit 15 is equal to or greater than a predetermined threshold (t2). Is determined (step S8).
When the difference between the maximum frequency and the average frequency of the histogram is equal to or greater than a predetermined threshold (t2), the main control unit 11 indicates that there is a motion in the image state between frames in the inter-frame information, The direction is recorded (step S9).
The main control unit 11 determines whether or not the interval between the frames is the last of the image (step S10).
If the interval between the frames is the last of the image data, the process is terminated (END).

If it is determined in step S2 that the interval between frames is a cut point, the main control unit 11 registers that the interval between frames is a cut point in the inter-frame information. Thereafter, the flow proceeds to step S10.
In step S5, when the number of motion vectors is less than the predetermined threshold (t1), the main control unit 11 registers in the inter-frame information that the image state between the frames is “none”. (Step S12). Thereafter, the flow proceeds to step S10.
If the difference between the maximum frequency and the average frequency in the histogram is smaller than the predetermined threshold value (t2) in step S8, the main control unit 11 moves the image state between the frames to “no” in the inter-frame information. Is registered (step S13). Thereafter, the flow proceeds to step S10.
Through the processing as described above, the image state between each frame is determined, and interframe information as shown in FIG. 3 is created.

Next, the processing procedure shown in FIG. 5 will be described.
The main control unit 11 monitors the inter-frame information registered in the image database 12 (step S21).
The main control unit 11 detects, in the inter-frame information, a combination between a plurality of consecutive three frames in which the image state moves and continuously changes as “none”, “present”, and “none” (step S22).
The main control unit 11 registers a section made up of four consecutive frames including the three frames detected in step S22 in the image database 12 as a slow playback candidate section (step S23).

The main control unit 11 reads interframe information between the last frame of the slow playback candidate section detected in step S23 and the next frame following this (step S24).
The main control unit 11 determines whether or not the inter-frame information read in step S24 is the cut point (step S25).
If it is determined that it is not the cut point, the main control unit 11 determines whether or not the interframe information read in step S24 is an image state motion “none” (step S26).

When it is determined that the motion is “none”, the main control unit 11 increments the count value of the counter that counts the number of motions “none” in the image state between the frames in the slow playback candidate section (step S1). S27).
The main control unit 11 adds a frame following the last frame of the slow playback candidate section to the slow playback candidate section (step S28).
After this, the flow returns to step S24. In other words, as long as the state of “none” of the image state between the last frame of the slow playback candidate section and the next frame continues, the subsequent frames are added to the slow playback candidate section one after another. It will be.

If it is determined in step S26 that the interframe information is not the motion “none” in the image state, the main control unit 11 determines whether or not the interframe information is the motion “present” (step S29). ).
If it is determined that the motion is “present”, the main control unit 11 has a predetermined threshold value (t3) where the number of motions “none” in the image state between the frames included in the slow playback candidate section is 1 or more. It is determined whether or not (step S30).
When it is determined that the number of movement “none” is 1 or more and a predetermined threshold (t3) or less, the main control unit 11 sets the count value of the counter that counts the number of movement “none” to 0 ( Step S31).
Thereafter, the flow proceeds to step S28, and the main control unit 11 adds a frame following the last frame of the slow playback candidate section to the slow playback candidate section.

If the main control unit 11 determines in step S30 that the number of motion “none” between frames included in the slow playback candidate section is not greater than or equal to 1 and not greater than or equal to the predetermined threshold (t3), the slow playback candidate It is determined whether or not the number of frames included in the section is equal to or greater than a predetermined threshold (t4) (step S32).
If the number of frames is equal to or greater than the predetermined threshold (t4), the main control unit 11 detects the slow playback candidate section as a slow playback section (step S33).
That is, by repeating step S28 after the four consecutive frames including the combination between the three frames continuously changing as motion “none”, “present”, “none” detected in step S22, When the number of frames included in the slow playback candidate section becomes equal to or greater than the predetermined number (t4), the slow playback candidate section is detected as a slow playback section in step S33.
The main control unit 11 determines whether or not the last frame of the slow playback section is the last frame of the image data (step S34).

If it is determined in step S32 that the number of frames included in the slow playback candidate section is not equal to or greater than the predetermined threshold (t4), the main control unit 11 advances the flow to step S34, and the slow playback candidate section. It is determined whether the last frame is the last frame of the image data.
In step S34, when the main control unit 11 determines that the last frame of the slow playback section is the last frame of the image data, the main control unit 11 ends the flow (END).
If the last frame of the slow playback section is not the last frame of the image data, the main control unit 11 returns the flow to step S21.

  As described above, according to the slow playback section detection device of the present invention, it is possible to detect a slow playback section composed of several tens of frames, and a configuration such as a still image, a moving picture by camera panning, or a still image is configured. It is possible to suppress erroneous detection of an image as a slow playback section. In addition, since the slow playback section is detected based on the presence or absence of motion in the image state between frames and the number of frames in the entire slow playback candidate section, even an image whose slow playback speed changes can be detected with high accuracy. be able to.

In the above, in step S22, the main control unit 11 detects, in the inter-frame information, four consecutive frames in which the image state moves and continuously changes as “none”, “present”, and “none”. Although the form has been described, three consecutive frames in which the image state moves continuously and changes “present” and “none” are detected, and the slow playback section is detected using these three frames as slow playback candidate sections. Also good.

  Further, in the above, when the inter-frame information is “Yes” (step S29) and the number of frames in the slow playback candidate section is equal to or greater than the predetermined threshold (t4) (step S32), the main control unit 11 However, the embodiment has been described in which the slow playback candidate section is determined to be the slow playback section. For example, the directions of the motion vectors in the case of “with” motion in the interframe information included in the slow playback candidate section are all the same. In this case, the slow playback candidate section may be determined to be a slow playback section. That is, for example, the direction of the motion vector between the frame numbers 1008 and 1009 in the interframe information shown in FIG. 3 is the direction of the motion vector that appears before that (the motion vector between the frame numbers 1004 and 1005 in FIG. The direction “right”) may be determined. In such a case, if the direction of the motion vector between frame numbers 1008 and 1009 in FIG. 3 is a direction other than the right, the main control unit 11 determines that the slow playback candidate section is not a slow playback section. To do. Such a determination may be configured such that the main control unit 11 performs the determination together in step S29, or may be configured separately from step S29.

The figure which shows the structure of the slow reproduction area detection apparatus of this invention. The figure which shows the state of the goal scene of a time trial competition, and the frequency distribution of the motion vector in a slow reproduction area and a normal reproduction area. The figure which shows the information between frames. The processing sequence figure of the slow reproduction area detection apparatus and slow reproduction area detection method of this invention. The processing sequence figure of the slow reproduction area detection apparatus and slow reproduction area detection method of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Main control part, 12 ... Image database, 13 ... Motion vector detection part, 14 ... Histogram preparation part, 15 ... Average frequency calculation part, 16 ... DBMS.

Claims (4)

A slow playback section detection device that detects a slow playback section that is included in video data composed of a plurality of frames and presents the same frame image continuously several times,
Storage means for storing the moving image data and inter-frame information indicating the presence or absence of image movement between the frames of the moving image data;
Referring to the inter-frame information, there is detected as three frames candidate section contiguous with changes to the information indicating No information to indicate, between the frame and the frame following the last frame of the said candidate section As long as the information is information indicating that there is no continuous, the processing operation of adding a frame following the last frame of the candidate section to the candidate section is repeated until the interframe information is no longer indicating information, and the frame If the number of frames between information contained in the candidate interval up to the point where no longer the information indicating No is equal to or greater than a predetermined threshold, the section detecting means for detecting the candidate section as said slow reproduction section
A slow playback section detection device comprising: A motion vector detection means for detecting a motion vector representing a motion of an image between successive frames of the moving image data stored in the storage means ;
Histogram creating means for classifying the direction of the motion vector into a predetermined number and creating a histogram composed of frequency distribution in each direction;
Average frequency calculation means for calculating an average frequency of the frequency distribution of the histogram;
When the difference between the maximum frequency of the direction distribution of the motion vector in the histogram and the average frequency satisfies a predetermined difference condition, it is determined that there is an image motion between the frames, and the predetermined difference condition is not satisfied to, judges no movement of the image between the frames, the determination result based on, according to claim 1, further comprising a frame information creation means for storing the inter-frame information in the storage means Slow playback section detection device. Storage means for storing moving image data composed of a plurality of frames and inter-frame information indicating whether or not there is an image motion between each frame of the moving image data is detected, and a slow playback section included in the moving image data is detected. A slow playback section detection method in a slow playback section detection device ,
Referring to the inter-frame information, there is detected as three frames candidate section contiguous with changes to the information indicating No information to indicate, between the frame and the frame following the last frame of the said candidate section As long as the information is information indicating that there is no continuous, the processing operation of adding a frame following the last frame of the candidate section to the candidate section is repeated until the interframe information is no longer indicating information, and the frame If the number of frames between information contained in the candidate interval up to the point where no longer the information indicating No is equal to or greater than a predetermined threshold value, it has a section detecting step of detecting the candidate section as said slow reproduction section A method for detecting a slow playback section , characterized by : And video data ing a plurality of frames, there image motion between each frame of the moving image data, comprising a storage unit for inter-frame information is stored that indicates none, detecting slow playback section included in the moving image data A slow playback section detection program for causing a computer on a slow playback section detection device to perform a slow playback section detection process,
Referring to the inter-frame information, there is detected as three frames candidate section contiguous with changes to the information indicating No information to indicate, between the frame and the frame following the last frame of the said candidate section As long as the information is information indicating that there is no continuous, the processing operation of adding a frame following the last frame of the candidate section to the candidate section is repeated until the interframe information is no longer indicating information, and the frame If the number of frames between information contained in the candidate interval up to the point where no longer the information indicating No is equal to or greater than a predetermined threshold value, the computer section detecting step of detecting the candidate section as said slow reproduction section A slow playback section detection program characterized in that

Images