JP2019528643A - Method, apparatus and system for detecting scene change frames - Google Patents

Abstract

The present invention discloses a method and apparatus and system for detecting scene change frames in a video. When a scene change frame in the video is detected, whether the maximum P frame (Pmax) of all P frames in the video GOP is a scene change frame in order to detect a scene change frame in the P frame How about the relative value of the median or average value of the sizes of multiple I frames between Pmax and the scene change frame closest to Pmax and before Pmax, and the size Pkmax of Pmax, or Pkmax, Judgment is made based on the relative relationship with the median or average value of the sizes of a plurality of P frames in the GOP, thereby effectively reducing the detection omission of the scene change frame.

Description

  The present invention relates to the field of video technology, and in particular, to a method and apparatus and system for detecting scene change frames.

  With the development of communication technology, video services such as IPTV (Internet Protocol Television) and OTT service are widely used commercially. In order to guarantee the quality of the video service, it is necessary to evaluate the video quality, so that timely adjustments are made using corresponding means to ensure the normal operation of the video service. Therefore, how to accurately assess video quality is an important issue that needs to be solved urgently.

  A segment of video includes a plurality of consecutive video frame sequences and generally includes more than one scene. For example, the video segment includes four scenes, scene 1 and scene 3 correspond to a soccer ground shot video, and scene 2 and scene 4 correspond to a spectator seat shot video.

  When video quality is being evaluated, it is first necessary to detect where the scene changes, i.e. the position of the scene change frame, and then the video quality is evaluated based on the scene. For example, to assess video coding loss because video coding loss that occurs during video coding is related not only to video coding type, frame rate, resolution, and bit rate, but also to the complexity of the scene. First, the position where the scene changes needs to be detected. Therefore, it is necessary to accurately detect scene changes when video quality is being evaluated.

  When video frames of a video are being encoded, the video frames are encoded into different types of frames, eg, I frames, P frames, and B frames. The I frame is an intra-frame prediction frame, and only data in the frame is referred to during encoding. The P frame is a prediction frame, in other words, a unidirectional difference frame, and is used to record a difference between this frame and the previous I frame (or P frame). The B frame is a bi-directional interpolation prediction frame, in other words, a bi-directional difference frame, and is used to record the difference between this frame and each of the previous frame and the next frame.

  A method for detecting scene change frames is provided in the standard ITU-T P1201.2 IPTV surveillance solution. However, in the prior art, only the scene change frame in the I frame is detected, but in reality, many scene change frames are P frames. As a result, in the prior art, a detection failure occurs during the scene change frame detection.

  Embodiments of the present invention provide a method and apparatus for detecting scene change frames so as to avoid missing scene change frames in the prior art.

According to a first aspect, a method for detecting a scene change frame is provided. The video includes N picture group GOPs, where N is an integer greater than or equal to 2, and the method
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold, and
P _max is a scene change frame when it is determined that the relative value between is greater than or equal to the second threshold,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of multiple P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1. Includes and.

  In the method provided in the first aspect of the present invention, when a scene change frame in a video is detected, a scene change frame in a P frame can be detected. It can be effectively reduced.

In a first possible embodiment of the first aspect,
May be calculated according to

Can be reflected easily and effectively.

In one embodiment,
Correspondingly, the first threshold is calculated according to the equation
May be calculated according to

In another embodiment,
Correspondingly, the first threshold is calculated according to the equation
May be calculated according to

I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median or average of the sizes of all P frames of the video It is.

I _median is the _median or average of the sizes of all I frames in the video, and P _median is the _median or average of the sizes of all P frames in the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In connection with the first possible embodiment of the first aspect or the first aspect, in a second possible embodiment:
May be calculated according to
Can be reflected easily and effectively.

May in particular be calculated according to the following formula:
, Where P _−m ,..., And P ₋₁ represent P frames prior to P _max in the K th GOP, and P ₁ ,..., And P _n are P _max in the K th GOP. It represents a P frame later than, F is, P _-m, ..., and P _-1 and P _1, ..., and is used to calculate the median or average value of the size of P _n,
m = min (num_before_P_frames, max_num),
n = min (num_after_P_frames, max_num),
num_before_P_frames is the number of P frames before P _max in the Kth GOP, num_after_P_frames is the number of P frames after P _max in the Kth GOP, and max_num needs to be considered Represents the preset number of frames.

Is calculated using the above formula and takes into account some video frames that are closest to the first video frame and within the GOP where the first video frame is located. The accuracy of detecting a frame is further improved.

According to a second aspect, a method for performing video quality assessment is provided. The video includes N picture group GOPs, where N is an integer greater than or equal to 2, and the method
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold,
If it is determined that the relative value between is greater than or equal to the second threshold and there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to the second threshold, there is a B frame in the Kth GOP, and
If it is determined that the relative value between is greater than or equal to the third threshold, P _max is a step of determining that it is a scene change frame,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames in the Kth GOP,
Large but the P _max, a median or average value of the sizes of all B-frames between the nearest and P _max than previous scene change frame into P _max, the first threshold, than 0 And a step in which the second threshold value is greater than 1 and the third threshold value is greater than 1.

  In the method provided in the second aspect of the present invention, when detecting a scene change frame in a video, it is possible to detect a scene change frame in a P frame. It can be effectively reduced. Also, when detecting a scene change frame in the P frame, not only the size of the I frame and P frame but also the size of the B frame are considered, so the accuracy of detecting the scene change frame in the P frame is further increased improves.

In a first possible embodiment of the second aspect,
May be calculated according to

Can be reflected easily and effectively.

In one embodiment,
Correspondingly, the first threshold may be calculated according to the equation
May be calculated according to

In another embodiment,
Correspondingly, the first threshold may be calculated according to the equation
May be calculated according to

I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median or average of the sizes of all P frames of the video It is.

I _median is the _median or average of the sizes of all I frames in the video, and P _median is the _median or average of the sizes of all P frames in the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In connection with the first possible embodiment of the second aspect or the second aspect, in a second possible embodiment:
May be calculated according to
Can be reflected easily and effectively.

May in particular be calculated according to the following formula:
, Where P _−m ,..., And P ₋₁ represent P frames prior to P _max in the K th GOP, and P ₁ ,..., And P _n are P _max in the K th GOP. It represents a P frame later than, F is, P _-m, ..., and P _-1 and P _1, ..., and is used to calculate the median or average value of the size of P _n,
m = min (num_before_P_frames, max_num),
n = min (num_after_P_frames, max_num),
num_before_P_frames is the number of P frames before P _max in the Kth GOP, num_after_P_frames is the number of P frames after P _max in the Kth GOP, and max_num needs to be considered Represents the preset number of frames.

Is calculated using the above formula and takes into account some video frames that are closest to the first video frame and within the GOP where the first video frame is located. The accuracy of detecting a frame is further improved.

In connection with the second aspect or the first or second possible embodiment of the second aspect, in a third possible embodiment:
May be calculated according to

Can be reflected easily and effectively.

In one embodiment,
Correspondingly, the third threshold is calculated according to the equation
May be calculated according to

In another embodiment,
Correspondingly, the third threshold is calculated according to the equation
May be calculated according to

B _threshold is the third threshold, P _median is the _median or average of the sizes of all P frames of the video, and B _median is the _median or average of the sizes of all B frames of the video is there.

P _median is the _median or average of the sizes of all P frames of the video, and B _median is the _median or average of the sizes of all B frames of the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In connection with the third possible embodiment of the second aspect, in a fourth possible embodiment, the scene change frame in the video is determined after P _max in the Kth GOP is determined as the scene change frame. The _median or average size of P frames other than the P frame determined as is used as the new P _median and the new B _threshold is
Calculated according to The new B _threshold is used to determine if P _max in the next GOP is a scene change frame.

In connection with the fourth possible implementation of the second aspect, the third threshold B _threshold can be updated in real time, so the effects of P frames determined as scene change frames are eliminated in a timely manner, This further improves the accuracy of detecting the scene change frame in the P frame.

  According to the third aspect, a detection device for detecting a scene change frame in a video is provided. The video includes N GOPs, where N is an integer equal to or greater than 2, and the detection device includes a first determination unit and a second determination unit.

The first determination unit determines the maximum P frame P _max among all the P frames in the Kth GOP, and the size of P _max is:
And K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N,
The second determination unit
The relative value between is greater than or equal to the first threshold, and
When it is determined that the relative value between is greater than or equal to the second threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1. Configured.

  In the detection apparatus provided in the third aspect of the present invention, when detecting a scene change frame in a video, a scene change frame in a P frame can be detected. Can be effectively reduced.

In the first possible embodiment of the third aspect, the second determining unit is notably the formula
May be calculated.

Can be reflected easily and effectively.

In one embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

In another embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median or average of the sizes of all P frames of the video It is.

I _median is the _median or average of the sizes of all I frames in the video, and P _median is the _median or average of the sizes of all P frames in the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In connection with the first possible embodiment of the third aspect or the third aspect, in the second possible embodiment, the second determining part is in particular the formula
May be calculated.
Can be reflected easily and effectively.

In particular, the second determination unit follows the following formula:
May be calculated.
, Where P _−m ,..., And P ₋₁ represent P frames prior to P _max in the K th GOP, and P ₁ ,..., And P _n are P _max in the K th GOP. It represents a P frame later than, F is, P _-m, ..., and P _-1 and P _1, ..., and is used to calculate the median or average value of the size of P _n,
m = min (num_before_P_frames, max_num),
n = min (num_after_P_frames, max_num),
num_before_P_frames is the number of P frames before P _max in the Kth GOP, num_after_P_frames is the number of P frames after P _max in the Kth GOP, and max_num needs to be considered Represents the preset number of frames.

Is calculated using the above formula and takes into account some video frames that are closest to the first video frame and within the GOP where the first video frame is located. The accuracy of detecting a frame is further improved.

  According to the fourth aspect, there is provided a detection device for detecting a scene change frame in a video. The video includes N picture groups GOP, where N is an integer equal to or greater than 2, and the detection device includes a first determination unit and a second determination unit.

The first determination unit determines the maximum P frame P _max among all the P frames in the Kth GOP, and the size of P _max is:
And K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N,
The second determination unit
The relative value between is greater than or equal to the first threshold,
If it is determined that the relative value between is greater than or equal to the second threshold and there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to the second threshold, there is a B frame in the Kth GOP, and
When it is determined that the relative value between is greater than or equal to the third threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames within the Kth GOP,
It has a P _max, a median or average value of the sizes of all B-frames between the nearest and the scene change frame before P _max to P _max, the first threshold value is greater than 0 And the second threshold is greater than 1 and the third threshold is greater than 1.

  In the detection apparatus provided in the fourth aspect of the present invention, when detecting a scene change frame in a video, a scene change frame in a P frame can be detected. Can be effectively reduced. Also, when detecting a scene change frame in the P frame, not only the size of the I frame and P frame but also the size of the B frame are considered, so the accuracy of detecting the scene change frame in the P frame is further increased improves.

In a first possible embodiment of the fourth aspect, the second decision unit in particular has the formula
May be calculated.

Can be reflected easily and effectively.

In one embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

In another embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median or average of the sizes of all P frames of the video It is.

I _median is the _median or average of the sizes of all I frames in the video, and P _median is the _median or average of the sizes of all P frames in the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In relation to the fourth aspect or the first possible embodiment of the fourth aspect, in the second possible embodiment, the second determining part is in particular the formula
May be calculated.
Can be reflected easily and effectively.

In particular, the second determination unit follows the following formula:
May be calculated.
, Where P _−m ,..., And P ₋₁ represent P frames prior to P _max in the K th GOP, and P ₁ ,..., And P _n are P _max in the K th GOP. It represents a P frame later than, F is, P _-m, ..., and P _-1 and P _1, ..., and is used to calculate the median or average value of the size of P _n,
m = min (num_before_P_frames, max_num),
n = min (num_after_P_frames, max_num),
num_before_P_frames is the number of P frames before P _max in the Kth GOP, num_after_P_frames is the number of P frames after P _max in the Kth GOP, and max_num needs to be considered Represents the preset number of frames.

Is calculated using the above formula and takes into account some video frames that are closest to the first video frame and within the GOP where the first video frame is located. The accuracy of detecting a frame is further improved.

In connection with the first aspect or the second possible embodiment of the fourth aspect or the fourth aspect, in the third possible embodiment, the second determining part is in particular a formula
May be calculated.

Can be reflected easily and effectively.

In one embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

In another embodiment, the second determination unit has the formula
Corresponding to this, the second determination unit calculates
The first threshold may be calculated according to

I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and B _median is the _median or average of the sizes of all B frames of the video It is.

I _median is the _median or average of the sizes of all I frames of the video, and B _median is the _median or average of the sizes of all the B frames of the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

In connection with the third possible embodiment of the fourth aspect, in the fourth possible embodiment, after the P _max in the Kth GOP is determined as the scene change frame, the scene change frame in the video The _median or average size of P frames other than the P frame determined as is used as the new P _median and the new B _threshold is
Calculated according to The new B _threshold is used to determine if P _max in the next GOP is a scene change frame.

  In relation to the fourth possible embodiment of the fourth aspect, the third threshold Bthreshold can be updated in real time, so that the influence of the P frame determined as the scene change frame is eliminated in a timely manner. This further improves the accuracy of detecting the scene change frame in the P frame.

  According to a fifth aspect, a detection device is provided for detecting a scene change frame in a video, the detection device including a processor and a memory.

  The memory is configured to store computer operating instructions.

  The processor, wherein the detection device is provided in any one of the possible embodiments of the first aspect or the first aspect or in any one of the possible embodiments of the second aspect or the second aspect Is configured to execute computer operating instructions stored in memory.

  In the detection apparatus provided in the fifth aspect of the present invention, when detecting a scene change frame in a video, a scene change frame in a P frame can be detected. Can be effectively reduced.

  According to the sixth aspect, a detection device is provided, and the detection apparatus includes a medium unit and a detection apparatus.

  The media portion is configured to acquire a video and send the video to a detection device.

  The detection device acquires the video from the media part and any one of the third aspect or the possible embodiments of the third aspect, the fourth aspect or any of the possible embodiments of the fourth aspect Or the operation performed by the detection device provided in any one of the fifth aspect or any possible embodiment of the fifth aspect.

  In the detection device provided in the sixth aspect of the present invention, when detecting a scene change frame in a video, it is possible to detect a scene change frame in a P frame. Can be effectively reduced.

  According to a seventh aspect, a system for performing video quality assessment is provided, the system including a video server, a transmitting device, and a video terminal. The video stream transmitted by the video server is transmitted to the video terminal via the transmitting device.

  The transmitting device or the video terminal, in particular, any one of the possible embodiments of the third aspect or the third aspect, any one of the possible embodiments of the fourth aspect or the fourth aspect, or The detection device provided in any one of the fifth aspect or possible embodiments of the fifth aspect may be included.

  The system further comprises a first detection device, in particular the first detection device, in particular any one of the third aspect or possible embodiments of the third aspect, the fourth aspect or the fourth aspect. It may be a detection device provided in any one of the possible embodiments, or in any one of the possible embodiments of the fifth aspect or the fifth aspect. The transmission device 2020 or the video terminal 2030 is connected to a first detection device, and the first detection device acquires a video stream using the transmission device or video terminal connected to the first detection device.

  In the system provided in the seventh aspect of the present invention, when a scene change frame in a video is detected, a scene change frame in a P frame can be detected. It can be effectively reduced.

  To describe the technical solutions of the embodiments of the present invention more clearly, the following briefly describes the accompanying drawings required for describing the embodiments or the prior art. Apparently, the accompanying drawings in the following description show only some embodiments of the present invention, and those skilled in the art can further obtain other drawings from these accompanying drawings without creative efforts.

1 is a schematic diagram of a network configuration of a video system 100 according to Embodiment 1 of the present invention. FIG. 1 is a schematic diagram of a network configuration of a video system 100 according to Embodiment 1 of the present invention. FIG. 1 is a schematic diagram of a GOP according to Embodiment 1 of the present invention. 1 is a schematic diagram of a GOP according to Embodiment 1 of the present invention. 1 is a schematic diagram of a GOP according to Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment A of Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment A of Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment A of Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment B of Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment B of Embodiment 1 of the present invention. 2 is a schematic flowchart of a method according to Embodiment B of Embodiment 1 of the present invention. FIG. 3 is a schematic diagram of an example of a GOP according to Embodiment 1 of the present invention. FIG. 3 is a schematic diagram of an example of a GOP according to Embodiment 1 of the present invention. FIG. 3 is a schematic diagram of an example of a GOP according to Embodiment 1 of the present invention. FIG. 5 is a schematic configuration diagram of a detection device 200 according to Embodiment 2 of the present invention. FIG. 5 is a schematic configuration diagram of a detection apparatus 1000 according to Embodiment 3 of the present invention. FIG. 6 is a schematic configuration diagram of a detection device 400 according to Embodiment 4 of the present invention. FIG. 6 is a schematic configuration diagram of a system 2000 according to Embodiment 5 of the present invention. FIG. 6 is a schematic configuration diagram of a system 2000 according to Embodiment 5 of the present invention. FIG. 6 is a schematic configuration diagram of a system 2000 according to Embodiment 5 of the present invention.

  The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are a part of the embodiments of the present invention and not all of the embodiments of the present invention. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

  FIG. 1A is a schematic diagram of a network configuration of a video system 100 according to an embodiment of the present invention. Video system 100 includes a video server 110, one or more transmission devices 120, and a video terminal 130. The video stream transmitted by the video server 110 is transmitted to the video terminal 130 via the transmission device 120.

  The video system 100 may in particular be an IPTV system as shown in FIG. 1B. In the IPTV system, the video server 110 is specifically a video headend (video HE). The transmission device 120 includes, in particular, a network device such as a core router (CR), a broadband network gateway (BNG), or an optical line terminal (OLT). Specifically, the video terminal 130 is a set top box (STB).

  In the video system shown in FIGS. 1A and 1B, when a video stream is transmitted from a video server to a video terminal, abnormal phenomena such as packet loss, delay, jitter, or disturbance are caused in the video stream due to changes in the network state. May occur. These abnormal phenomena may cause a problem that irregular display and frame freezing occur in the video image displayed on the screen of the video terminal, and as a result, the user's video viewing experience is impaired. Therefore, there is a need to monitor a user's video experience by assessing video quality.

  When video quality is being evaluated, it is usually necessary to first detect where the scene changes, i.e. the position of the scene change frame, and then evaluate the video quality based on that scene.

  For example, to assess video coding loss because video coding loss that occurs during video coding is related not only to video coding type, frame rate, resolution, and bit rate, but also to the complexity of the scene. First, the position where the scene changes needs to be detected.

  In another example, if a packet loss occurs during the video transmission process, the video terminal decoder typically retrieves the video content in the corresponding region of the previous frame of the damaged frame to provide error compensation for the damaged frame. As the content of the damaged frame is used as the content of the damaged area, the smaller the content difference between the damaged frame and the previous frame, the better the compensation effect. However, when the damaged frame is a scene change frame, the content of the scene change frame is almost completely different from the content of the frame before the scene change frame, so the compensation effect is the worst. Therefore, if the video quality is affected by packet loss, it is necessary to consider whether the corrupted frame is a scene change frame.

  In another example, the encoded video frame sequence includes a plurality of group of pictures (GOP). As shown in FIG. 2A, each GOP starts with an I frame, followed by several P and B frames, and ends with a frame before the next I frame. The I frame is an intra-frame, the P frame is a forward reference frame, and the B frame is a bidirectional reference frame. When a packet loss occurs in a frame within a GOP, a decoding error due to the packet loss continuously expands to the next video frame, and normally ends with the last frame within the GOP. As shown in FIG. 2B, if the fourth frame in GOP1 is corrupted, the error is usually considered to expand continuously and ends with the last frame of the GOP. However, if there is a scene change frame in the GOP, the content of the scene change frame is almost completely different from the content of the frame before the scene change frame, and intra-frame predictive encoding is usually performed during encoding. (Intraframe predictive coding is performed for most macroblocks of a scene change frame). Therefore, when a frame before the scene change frame in the GOP is damaged, error expansion ends at the scene change frame. As shown in FIG. 2c, the sixth frame of GOP1 is a scene change frame, and if the fourth frame is damaged, error expansion ends at the sixth frame. Therefore, when the video quality is affected by packet loss, it is necessary to detect a scene change frame.

  Note that the sequence of frames described in this embodiment of the invention is a sequence of video frames in time. For example, a video having a duration T (eg, 10 seconds) includes video frame 1 at time t1 and video frame 2 at time t2. If t1 is less than t2, for example, if t1 is 1 second 30 milliseconds and t2 is 5 seconds 40 milliseconds, then video frame 1 is before video frame 2.

  In certain embodiments, a detection device for detecting scene change frames may be located in the video system. The detection apparatus may be located in any device through which the video stream passes (eg, transmitting device 120 or video terminal 130), or any device through which the video stream passes so as to obtain the video stream in a mirrored manner. May be bypassed.

  3A and 3B are schematic flowcharts of a method according to Embodiment 1 of the present invention. The method in Embodiment 1 of the present invention can be applied to the video system 100 shown in FIGS. 1A and 1B, and is executed by a detection apparatus.

  In Embodiment 1 of the present invention, a scene change frame in a video (hereinafter referred to as a detection target video) is detected. The detection target video may be read from the video file or acquired from the acquired video stream. The video to be detected may in particular be a complete video or a video segment of the video. For relatively long videos, a measurement time window is usually set and video segments within the measurement time window are detected. For example, when detecting video, the length of the measurement time window is set to 10 seconds, first the 0-10 second video segment of the video is detected as the video to be detected, then 10-20 seconds Are detected as detection target videos, and such detection is performed by analogy.

  Prior to detection, the detection module may first determine the type (such as I frame, P frame, or B frame) and size of each video frame of the video to be detected.

  For example, the video stream is acquired in real time, information about the video frame is extracted from the packet corresponding to the video stream within a measurement time window (eg 10-20 seconds), and the size of the video frame (in bytes) is calculated Is done. A specific process for calculating the size of the video frame is as follows. For each packet in the measurement time window, first the start identifier of the current video frame is found from the packet header, then the load length of the packet containing that start identifier and the load length of the subsequent packets are It is accumulated until the start identifier of the video frame is found. The cumulative total is the size of the current video frame. See standard ITU-T P1201.2 for a specific implementation of calculating the size of a video frame.

  Next, the types of all video frames within the measurement time window are determined. Specifically, the type of video frame can be determined based on the field random_access_indicator in the packet header of the packet. For I frames, the type of frame can be determined based on random_access_indicator regardless of whether the video is encrypted or not. For non-I frames, if the video is not encrypted, the frame type can be obtained directly from the frame header of the video frame. If the video is encrypted or the frame header of the video frame is lost, first the GOP mode can be estimated based on the size of the frame or the present time stamp (PTS) it can. The GOP mode is usually PBBPBB or PBBBPBBB. The GOP mode can be described using the difference between the current PTS value and the previous PTS value. Once the GOP mode is determined, the mode of all lost or encrypted video frames can be determined. See standard ITU-T P1201.2 for a specific implementation for determining the type of video frame.

  By performing the above two steps, the video to be detected can be divided into several GOPs. One detection target video usually includes a plurality of GOPs. As shown in the example of FIG. 5A, one detection target video includes N GOPs, and N is assumed to be an integer of 2 or more. In the example shown in FIG. 5A, a video frame filled with a single color is an I frame, a video frame filled with diagonal lines is a P frame, and an unfilled video frame is a B frame.

  One video frame is one image. When video frames of a video are being encoded, the video frames are encoded into different types of frames, eg, I frames, P frames, and B frames. An I frame is an intra-frame prediction frame, and only the data in the frame is referenced during encoding, and therefore the I frame contains complete image data. The P frame is a prediction frame, in other words, a unidirectional difference frame, and is used to record a difference between this frame and the previous I frame (or P frame). The B frame is a bi-directional interpolation prediction frame, in other words, a bi-directional difference frame, and is used to record the difference between this frame and each of the previous frame and the next frame.

  An I frame is usually larger than a P frame, and a P frame is usually larger than a B frame. In general, the size of the I frame is 2 to 5 times that of the P frame, and the size of the P frame is 2 to 5 times that of the B frame.

  Since the difference between the content of the scene change frame and the content of the frame before the scene change frame is relatively large, the scene change frame is encoded as a P frame, but intraframe predictive encoding is mostly used in a scene change frame. This macro block is executed with reference to another macro block in the scene change frame. Therefore, the size of the encoded scene change frame is relatively large. If the size of a P frame exceeds half the size of an I frame, the P frame is likely to be a scene change frame. Therefore, when detecting a scene change frame that is a P frame, the relative relationship between the P frame and the I frame can be referred to.

  However, when the video image content changes relatively quickly, for example, in a relatively intensely moving shot scene such as a fierce soccer game, it is not a scene change frame because the correlation between two adjacent video frames is small When a frame is encoded, the frame is encoded into a P frame with reference to the previous video frame, but the compression ratio is relatively low, the size of the P frame is relatively large, and the size of the I frame Even more than half. In this case, the size difference between adjacent P frames that are not scene change frames is not large, but the size difference between a P frame that is a scene change frame and a P frame that is adjacent to this P frame and is not a scene change frame is Relatively large. Therefore, when detecting a scene change frame which is a P frame, it is possible to refer to the ratio of the sizes of adjacent P frames.

  Based on the above analysis, Embodiment A of Embodiment 1 of the present invention for detecting a scene change frame in the detection target video will be described in detail below with reference to FIG. 3A. The video includes N GOPs, where N is an integer greater than or equal to 2.

  The video to be detected may be a video segment of a video file or may be a video segment of a video stream, such as a video segment of a video stream transmitted by a video server to a video terminal. Correspondingly, the detection device may be placed in any device through which the video stream passes (such as the transmitting device 120 or the video terminal 130), and the video stream is acquired so as to obtain the video stream in a mirroring manner. Any device that passes through may be bypassed.

  As shown in FIG. 3A, the method provided in implementation A of embodiment 1 of the present invention includes the following steps.

The detection device determines whether the maximum P frame P _max of all P frames in each GOP starting from the Mth GOP out of N GOPs is a scene change frame. The following operation is performed on each GOP starting from the Mth GOP among the GOPs, where M is 1 or more and N or less.

  Specifically, detection may be initially performed on the first GOP (ie, M = 1). Alternatively, detection may first be performed on a GOP that follows the first GOP. For example, the detected video is the segment of the video at the beginning of the video stream, and the size of the frame in the first two GOPs usually does not have a reference value, so detection is usually the first GOP first (M = 3).

Step 102: Determine the maximum P frame P _max of all P frames in the Kth GOP, where the size of P _max is
And K is a variable ranging from M to N, where 1 ≦ M ≦ N.

Step 103:
The relative value between is greater than or equal to the first threshold, and
If it is determined that the relative value between is greater than or equal to the second threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of the P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1.

Also,
The relative value between is less than the first threshold, or
If it is determined that the relative value between is smaller than the second threshold, P _max is determined not to be a scene change frame.

As shown in FIG. 5B, the second P frame in the Kth GOP is P _max , the I frame in the Kth GOP is not a scene change frame, and the (K−1) th GOP has a scene. There is no change frame, and the third P frame (hereinafter referred to as P ′ _max ) in the (K−2) th GOP is a scene change frame. In this case, the scene change frame closest to P _max and before P _max is P ′ _max .

In certain embodiments, prior to step 102, the I frame in the first GOP may be determined as the scene change frame. If there is no scene change frame between the K-th P _max and the first I-frame in the GOP of the GOP, closest and P _max scene change frame earlier than the P _max, in the first GOP I frame. If the video to be detected is a video segment of a video, and there is a video segment in which a scene change frame is detected before the video to be detected (hereinafter referred to as the previous video segment), the closest to P _max and P Scene change frames prior to _max may be located in the previous video segment.

It can be calculated using all or a median or average value of the size of some of the I frame between P _max and P _max nearest and P _max scene change frame prior to. As shown in FIG. 5B, 2-th P-frame of the K-th in the GOP are P _max, the scene change frame before the nearest and P _max to P _max is P _'max, P _max and P 'There are two I frames I _k and I _k-1 between _max and
Is the average value of the sizes of I _k and I _k−1 .

Can be the median or average of the sizes of all or some of the P frames in the Kth GOP. Preferred embodiments are as follows.
Is calculated according to the following equation:
, Where P _−m ,..., And P ₋₁ represent P frames prior to P _max in the K th GOP, and P ₁ ,..., And P _n are P _max in the K th GOP. It represents a P frame later than, F is, P _-m, ..., and P _-1 and P _1, ..., and is used to calculate the median or average value of the size of P _n,
m = min (num_before_P_frames, max_num),
n = min (num_after_P_frames, max_num),
num_before_P_frames is the number of P frames before P _max in the Kth GOP, num_after_P_frames is the number of P frames after P _max in the Kth GOP, and max_num needs to be considered Represents the preset number of frames. As shown in FIG. 5C, num_before_P_frames is 7, num_after_P_frames is 4, and max_num is set to 6. In this case, m is 6 and n is 4.

Is calculated using the above formula and takes into account some video frames that are closest to the first video frame and within the GOP where the first video frame is located. The accuracy of detecting a frame is further improved.

In certain embodiments,
Can be calculated according to the following equation:

Can be reflected easily and effectively.

The first threshold may be set in advance, and the same first threshold may be used in different GOPs of the same detection target video.
The first threshold is set to 0.53, or
The first threshold is set to 0.47.

  Alternatively, the first threshold may be obtained by calculation or may be dynamically adjusted so that the detection accuracy is further improved. The process of calculating and dynamically adjusting the first threshold is as follows.

Before the Mth GOP of the detection target video is detected, the first threshold value is first calculated according to the following equation.

I _median is the _median value or average value of the sizes of all I frames of the detection target video, and P _median is the _median value or average value of the sizes of all the P frames of the detection target video. The median is used as an example. For example, the video to be detected contains 9 P frames of size 3, 5, 3, 6, 4, 7, 3, 5, and 4, and the sequence is 3, 3, 3, 4, 4, 5, 5, 6, and 7, so P _median is 4, or the detected video contains 3 P frames of size 15, 12, and 18, and the sequence is , 12, 15, and 18 in ascending order, so I _median is 15.

I _median is the _median or average of the sizes of all I frames in the video, and P _median is the _median or average of the sizes of all P frames in the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

Next, whenever a new P frame is determined as a scene change frame, for example, whenever P _max in the Kth GOP is determined to be a scene change frame, it is determined as a scene change frame in the target video. The _median or average size of P frames other than the specified P frame is used as the new P _median , and the new I _threshold is
And the new I _threshold may be used to determine whether P _max in the next GOP (the GOP following the Kth GOP) is a scene change frame.

Since the first threshold I _threshold can be updated in real time, the effects of P frames determined as scene change frames are eliminated in a timely manner, thereby further improving the accuracy of detecting scene change frames in P frames. To do.

Specific embodiments can be as follows. The first threshold is an expression
If obtained according to
Or the first threshold is the expression
If obtained according to
It is.

In certain embodiments,
Can be calculated according to the following equation:

Can be reflected easily and effectively.

  The second threshold value may be normally set in advance, and the same second threshold value may be used in different GOPs of the same detection target video. For example, the second threshold is set to 1.51.

  Implementation A of Embodiment 1 of the present invention can further include Step 101.

  Step 101: A scene change frame is detected from I frames in the Nth GOP from the Mth GOP among the N GOPs.

  In step 101, whether or not the I frame in the Kth GOP is a scene change frame, in particular, determines the size of the I frame in the Kth GOP and the size of the I frame in the (K−1) th GOP. The ratio of the average size of all P frames in the (K−1) th GOP to the average size of all P frames in the Kth GOP, or the (K−1) th The determination can be made based on the ratio between the average value of the sizes of all the B frames in the GOP and the average value of the sizes of all the B frames in the Kth GOP. Specific embodiments are as follows.

1． The ratio r _I between the size of the I frame in the Kth GOP and the size of the I frame in the (K−1) th GOP is calculated.

2． The ratio r _p between the average value of the sizes of all the P frames in the (K−1) th GOP and the average value of the sizes of all the P frames in the Kth GOP is calculated.

3． The ratio r _B between the average value of the sizes of all the B frames in the (K−1) th GOP and the average value of the sizes of all the B frames in the Kth GOP is calculated.

Four. If the ratio r _I is greater than the first threshold or less than the second threshold, then further determination is made for the following condition (1) and condition (2), otherwise I in the Kth GOP It is determined that the frame is not a scene change frame.

Condition (1): r _P is smaller than the third threshold or r _P is larger than the fourth threshold.

Condition (2): r _B is smaller than the fifth threshold or r _B is larger than the sixth threshold.

  If both condition (1) and condition (2) are met, the I frame in the Kth GOP is determined to be a scene change frame; otherwise, the I frame in the Kth GOP is a scene change. It is determined that it is not a frame.

  Refer to standard ITU-T P1201.2 for specific details of the above embodiment.

  In certain embodiments, the I frame in the first GOP can be determined directly as the scene change frame. If K is not 1, the above method can be used to determine if the I frame in the Kth GOP is a scene change frame.

In a particular implementation, as shown in FIG. 3B, embodiment A can be implemented using embodiment J, where step 101 is performed first, then steps 102 and 103 are performed. That is, first, a scene change frame in the I frame in the Nth GOP from the Mth GOP is detected, and then a scene change frame in the P frame in the Nth GOP is detected from the Mth GOP. Is done. For example, first, an I frame in GOP1 is determined as a scene change frame, and then it is determined from GOP M (for example, GOP1) whether an I frame in GOP N is a scene change frame, and GOP M (for example, GOP1) ) To determine whether P _max in GOP N is a scene change frame.

Alternatively, in a particular implementation, implementation A can be implemented using implementation K, and step 101 is performed in combination with steps 102 and 103, as shown in FIG. Is detected based on the sequence of video frames, i.e., based on the sequence of GOPs, starting from the Mth (e.g., first) GOP, the scene change frame in the current GOP is detected and the current When a scene change frame in a GOP is detected, it is first detected whether the I frame in the current GOP is a scene change frame, and then whether P _max in the current GOP is a scene change frame Whether it is detected. For example, first an I frame in GOP1 is determined as a scene change frame, then P _max in GOP1, I frame in GOP2, P _max in GOP2, I frame in GOP3, P _max in GOP3, ..., it is determined in turn whether the I frame in GOP N or P _max in GOP N is a scene change frame. That is, if K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then whether the I frame in the (K + 1) th GOP is a scene change frame Is determined.

When implementation A is implemented using implementation K, the probability that the scene will change continuously in a short time is relatively low, so before it is determined whether the I frame is a scene change frame, First, the distance between the I frame and the previous scene change frame (hereinafter referred to as the first distance) is calculated. If the first distance is less than or equal to the distance threshold, it is determined that the I frame is not a scene change frame, otherwise the I frame is a scene change frame according to the method provided in standard ITU-T P1201.2 It can be further determined whether or not. Specific embodiments are as follows. When K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then the I frame in the (K + 1) th GOP (hereinafter referred to as the current I frame) If the distance between the current I frame and the scene change frame that is closest to and before the current I frame is less than or equal to the distance threshold, the current I frame is not a scene change frame. If not, it can further be determined whether the current I frame is a scene change frame according to the method provided in standard ITU-T P1201.2.

  The distance between two video frames as described in the present invention is the number of video frames between the two video frames. The distance between the Xth video frame and the Yth video frame is YN, and the distance between two adjacent video frames is 1.

  The distance threshold may be set in advance, and the same distance threshold may be used in different GOPs of the same detection target video.

  Alternatively, the distance threshold may be determined or dynamically adjusted so that the detection accuracy is further improved. The process for determining and dynamically adjusting the distance threshold is as follows.

Before the scene change frame in the I frame in the Nth GOP is detected from the Mth GOP of the N GOPs, the initial distance threshold is first determined, and the determined initial distance threshold is: It can be one of the following three lengths:
(1) The length of the longest GOP out of N GOPs,
(2) Average length of N GOPs, and (3) Length L, where the number of GOPs with length L is the largest of N GOPs

  The length of a GOP described in the present invention is the number of video frames included in the GOP.

  For example, the video to be detected includes 8 GOPs of length 10, 6, 8, 7, 8, 7, 9, and 8, and according to method (1), the initial distance threshold is determined to be 10. According to method (2), the initial distance threshold is determined to be 8, and according to method (3), since the number of GOPs having length 8 is the maximum, the initial distance threshold is determined to be 8.

  When encoding is performed using a fixed GOP length, the length of all GOPs is the same, so the initial distance threshold calculated in the above three methods is the same.

  Next, when a new scene change frame is determined, the distance between the new scene change frame and the scene change frame closest to and before the new scene change frame (hereinafter referred to as the first scene change frame). If the distance threshold is smaller than the distance threshold, the distance threshold is updated to the second distance.

In implementation A of embodiment 1 of the present invention, when a scene change frame in a video is detected, a maximum of all P frames in the video GOP is detected to detect a scene change frame in the P frame. whether P-frame P _max is a scene change frame, the median or average value of the size of a plurality of I frames between P _max and P _max nearest and P _max scene change frame prior to, P _max size
Relative to or
And the relative relationship between the median value or the average value of the sizes of a plurality of P frames in the GOP, thereby effectively reducing the detection omission of the scene change frame.

  In video coding, a P frame is usually larger than a B frame, and the P frame is usually 2 to 5 times the size of the B frame. Since the difference between the content of the scene change frame and the content of the frame before the scene change frame is relatively large, the scene change frame is encoded as a P frame, but intraframe predictive encoding is mostly used in a scene change frame. Is executed on the macroblock of Therefore, the size of the encoded scene change frame is relatively large. If the P frame is smaller than twice the size of the B frame, the P frame is likely to be a scene change frame. Therefore, when detecting a scene change frame that is a P frame, the relative relationship between the P frame and the B frame can be referred to.

  Based on the above analysis, Embodiment B of Embodiment 1 of the present invention for detecting a scene change frame in the detection target video will be described in detail below with reference to FIG. 4A. The video includes N GOPs, where N is an integer greater than or equal to 2.

  As shown in FIG. 4A, the method provided in Embodiment B of Embodiment 1 of the present invention includes the following steps.

  Step 202: Step 202 is the same as step 102, and details are not described here again.

Step 203:
The relative value between is greater than or equal to the first threshold,
If it is determined that the relative value between is greater than or equal to the second threshold and there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to the second threshold, there is a B frame in the Kth GOP, and
If it is determined that the relative value between is greater than or equal to the third threshold, P _max is determined to be a scene change frame,
Has a P _max, the average of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames within the Kth GOP,
It has a P _max, a median or average value of the sizes of all B-frames between the nearest and the scene change frame before P _max to P _max, the first threshold value is greater than 0 Less than 1, the second threshold is greater than 1, and the third threshold is greater than 1.

Also,
The relative value between is less than the first threshold, or
The relative value between is less than the second threshold, or
If it is determined that the relative value between is smaller than the third threshold, P _max is determined not to be a scene change frame.

A method for determining that the relative value between is greater than or equal to a first threshold and
The method for determining that the relative value between is greater than or equal to the second threshold is the same as that described for step 102, and details are not described herein again.

In certain embodiments,
Can be calculated according to the following equation:

Can be reflected easily and effectively.

The third threshold may be set in advance, and the same third threshold may be used in different GOPs of the same detection target video.
The first threshold is set to 2.87 if calculated according to
The first threshold is set to 1.87.

  Alternatively, the third threshold may be obtained by calculation or may be dynamically adjusted so that the detection accuracy is further improved. The process of calculating and dynamically adjusting the third threshold is as follows.

Before the Mth GOP of the detection target video is detected, the third threshold value is first calculated according to the following equation.

B _threshold is the third threshold, P _median is the _median or average size of all P frames in the video, and B _median is the _median or average size of all B frames in the detected video Value.

P _median is the _median or average of the sizes of all P frames of the video, and B _median is the _median or average of the sizes of all B frames of the video, so the effective threshold is
Can be accurately calculated, so that the scene change frame in the P frame can be accurately detected.

Next, every time a new P frame is determined as a scene change frame, if it is determined that P _max in the Kth GOP is a scene change frame, the P determined as a scene change frame in the detection target video The _median or average size of non-frame P frames is used as the new P _median and the new B _threshold is
And the new B _threshold may be used to determine if P _max in the next GOP is a scene change frame.

Since the third threshold B _threshold can be updated in real time, the effects of P frames determined as scene change frames are eliminated in a timely manner, thereby further improving the accuracy of detecting scene change frames in P frames. To do.

  Implementation B of embodiment 1 of the present invention can further include step 201, which is the same as step 101, and details are not described here again.

  In certain embodiments, embodiment B is performed using an embodiment similar to the two embodiments of embodiment A (embodiment J and embodiment K), as shown in FIGS. 4B and 4C. You can also.

In implementation B of embodiment 1 of the present invention, when a scene change frame in a video is detected, a maximum of all P frames in the video GOP is detected to detect a scene change frame in the P frame. whether P-frame P _max is a scene change frame, the median or average value of the size of a plurality of I frames between P _max and P _max nearest and P _max scene change frame prior to, P _max size
Relative to or
When the size of all the B frames between a plurality of relative relationships or P _max and the nearest and the scene change frame before P _max to P _max of the median or average value of the size of P frames in the GOP The median or average of
This makes it possible to effectively reduce detection omissions in scene change frames. Also, when detecting a scene change frame in the P frame, not only the size of the I frame and P frame but also the size of the B frame are considered, so the accuracy of detecting the scene change frame in the P frame is further increased improves.

  Based on Embodiment 1 of the present invention, Embodiment 2 of the present invention provides a detection apparatus 200 for detecting a scene change frame in a video. The video includes N GOPs, where N is an integer greater than or equal to 2. As shown in FIG. 6, the detection apparatus 200 includes a first determination unit 210 and a second determination unit 220.

  The first mode of the second embodiment corresponds to the mode A of the first embodiment, and details are as follows.

The first determination unit 210 determines the maximum P frame P _max among all the P frames in the Kth GOP, and the size of P _max is
Where K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N.

The second determination unit 220 is
The relative value between is greater than or equal to the first threshold, and
When it is determined that the relative value between is greater than or equal to the second threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1. Configured.

  Specifically, the first determiner 210 may be specifically configured to perform step 102 of the method of embodiment A of embodiment 1, and the second determiner 220 is the implementation of embodiment 1. It may be specifically configured to perform step 103 of the method of aspect A.

  Further, the second determination unit 220 may be further configured to detect a scene change frame from the I frames in the Mth to Nth GOPs out of the N GOPs. Embodiment A may be specifically configured to perform step 101 of the method.

  The second embodiment of the second embodiment corresponds to the embodiment B of the first embodiment, and details are as follows.

The first determination unit 210 determines the maximum P frame P _max among all the P frames in the Kth GOP, and the size of P _max is
Where K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N.

The second determination unit 220 is
The relative value between is greater than or equal to the first threshold,
If it is determined that the relative value between is greater than or equal to the second threshold and there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to the second threshold, there is a B frame in the Kth GOP, and
When it is determined that the relative value between is greater than or equal to the third threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames within the Kth GOP,
It has a P _max, a median or average value of the sizes of all B-frames between the nearest and the scene change frame before P _max to P _max, the first threshold value is greater than 0 And the second threshold is greater than 1 and the third threshold is greater than 1.

  Specifically, the first determiner 210 may be specifically configured to perform step 202 of the method of embodiment B of embodiment 1, and the second determiner 220 is an implementation of embodiment 1. It may be specifically configured to perform step 203 of the method of aspect B.

  Further, the second determination unit 220 may be further configured to detect a scene change frame from the I frames in the Mth to Nth GOPs out of the N GOPs. Embodiment B may be specifically configured to perform step 201 of the method.

In Embodiment 2 of the present invention, when a scene change frame in the video is detected, a scene change frame in the I frame is detected, and the maximum P frame P _max among all P frames in the video GOP is the scene. Whether or not it is a change frame is the size of multiple I frames between P _max and a scene change frame that is closest to P _max and before P _max to detect a scene change frame in the P frame Median or average and P _max size
Relative to or
And the relative relationship between the median value or the average value of the sizes of a plurality of P frames in the GOP, thereby effectively reducing the detection omission of the scene change frame. In the embodiment B of the embodiment 2 of the present invention, when detecting the scene change frame in the P frame, not only the size of the I frame and the P frame but also the size of the B frame is considered. The accuracy of detecting the scene change frame is further improved.

  In accordance with Embodiment 1 of the present invention, Embodiment 3 of the present invention provides a detection apparatus 1000. As shown in FIG. 7, the detection apparatus 1000 includes a processor 1010 and a memory 1020, and the processor 1010 and the memory 1020 communicate with each other using a bus.

  Memory 1020 is configured to store computer operating instructions. The memory 1020 may include a high speed RAM memory and may further include at least one non-volatile memory such as a magnetic disk memory.

  The processor 1010 is configured to execute computer operating instructions stored in the memory 1020. Specifically, the processor 1010 may be a central processing unit (CPU), and is a core unit of a computer.

  The processor 1010 executes computer operating instructions to enable the detection apparatus 1000 to perform the method of embodiment 1.

  In Embodiment 3 of the present invention, when detecting a scene change frame in a video, it is possible to detect a scene change frame in a P frame, thereby effectively reducing a scene change frame detection failure. Can do.

  In accordance with Embodiments 1-3 of the present invention, Embodiment 4 of the present invention provides a detection device 400. As shown in FIG. 8, the detection device 400 includes a medium part 4010 and a detection device 4020.

  The medium unit 4010 is configured to acquire a video (hereinafter referred to as a detection target video) and transmit the video to the detection device 4020. In particular, the medium unit 4010 may read the detection target video from the video file, or may acquire the detection target video from the reception media stream transmitted by the video server. The video to be detected may in particular be a complete video or a video segment of the video. When the detection target video is a video segment, the medium unit 4010 may transmit the video in which the video segment is arranged (that is, the video including the video segment) to the detection device 4020, and the detection device 4020 In order to detect a scene change frame in the video, a video segment of the received video is detected.

  The detection device 4020 may be, in particular, the detection device 200 provided in the second embodiment or the detection device 1000 provided in the third embodiment, obtains a detection target video from the medium unit 4010, and is provided in the second embodiment. The operation performed by the detection apparatus 200 to be performed or the detection apparatus 1000 provided in the third embodiment is executed.

  The detection device 4020 may further evaluate the quality of the detection target video or the quality of the video in which the detection target video is arranged based on the detected scene change frame.

  In Embodiment 4 of the present invention, when detecting a scene change frame in a video, it is possible to detect a scene change frame in a P frame, thereby effectively reducing a scene change frame detection failure. Can do.

  In accordance with embodiments 1-3 of the present invention, embodiment 5 of the present invention provides a system 2000 for performing video quality assessment. As shown in FIG. 9A, the system 2000 includes a video server 2010, a transmitting device 2020, and a video terminal 2030. The video stream transmitted by the video server 2010 is transmitted to the video terminal 2030 via the transmission device 2020.

  In certain implementations, the transmitting device 2020 or video terminal 2030 may specifically include the detection apparatus 200 provided in embodiment 2 or the detection apparatus 1000 provided in embodiment 3. In certain implementations, both the transmitting device 2020 and the video terminal 2030 may include the detection apparatus 200 provided in Embodiment 2 or the detection apparatus 1000 provided in Embodiment 3. The transmission device 2020 or the video terminal 2030 may in particular be the detection device 400 provided in embodiment 4.

  In another specific embodiment, the system further includes a detection device 2040. As shown in FIGS. 9B and 9C, the detection device 2040 may in particular be the detection device 200 provided in the second embodiment or the detection device 1000 provided in the third embodiment. The transmission device 2020 or the video terminal 2030 is connected to the detection device 2040, and the detection device 2040 acquires the video stream using the transmission device 2020 or the video terminal 2030 connected to the detection device 2040. In certain implementations, the transmitting device 2020 and the video terminal 2030 may be separately connected to one detector 2040.

  In Embodiment 5 of the present invention, when detecting a scene change frame in a video, it is possible to detect a scene change frame in a P frame, thereby effectively reducing a scene change frame detection failure. Can do.

  Those skilled in the art that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Can be recognized. Whether the function is performed by hardware or software depends on the specific application and design constraints of the technical solution. One skilled in the art can use different methods to perform the described functions for each particular application, but that embodiment should not be considered beyond the scope of the present invention.

  For the sake of simplicity and simplicity, the detailed operating processes of the above systems, devices, and units can be referred to the corresponding processes in the above method embodiments, where details are described again. It can be clearly understood by those skilled in the art.

  It should be understood that with respect to some embodiments provided herein, the disclosed systems, devices, and methods may be implemented in other ways. For example, the described apparatus embodiment is merely an example. For example, the unit division is merely logical function division, and may be another division in an actual implementation. For example, multiple units or components may be combined or integrated as separate systems, and some features may be ignored or not performed. Furthermore, the presented or described mutual coupling or direct coupling or communication connection may be implemented using several interfaces. Indirect coupling or communication connections between devices or units may be implemented in electronic form, mechanical form, or other form.

  Units that are described as separate parts may or may not be physically separate, and parts that are presented as units may or may not be physical units, It may be arranged or distributed to a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

  Further, the functional units in the embodiments of the present invention may be integrated into one processing unit, each of these units may physically exist alone, or two or more units may be It may be integrated into one unit.

  If the function is implemented in the form of a software functional unit and sold or used as an independent product, the function may be stored on a computer-readable storage medium. Based on this understanding, in essence, the technical solution of the present invention, or a part that contributes to the prior art, or a part of the technical solution, may be implemented in the form of a software product. The computer software product is stored in a storage medium so that the computer device (which may be a personal computer, server, or network device) performs all or part of the method steps described in the embodiments of the present invention. Includes several instructions for ordering. The above storage medium may store program codes such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Including any medium capable.

  The above descriptions are merely specific embodiments of the present invention, and do not limit the protection scope of the present invention. Modifications or substitutions easily conceived by those skilled in the art within the technical scope disclosed in the present invention shall fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

100 video system
110 Video server
120 sending device
130 video terminals
200 detector
210 1st judgment part
220 Second judgment part
400 detection devices
1000 detector
1010 processor
1020 memory
2000 system
2010 video server
2020 sending device
2030 video terminal
2040 Detector
4010 Media section
4020 Detector

According to a first aspect, a method for detecting a scene change frame is provided. The video contains N picture groups ( GOP ) , where N is an integer greater than or equal to 2,
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold, and
P _max is a scene change frame when it is determined that the relative value between is greater than or equal to the second threshold,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of multiple P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1. Includes and.

According to a second aspect, a method for performing video quality assessment is provided. The video contains N picture groups ( GOP ) , where N is an integer greater than or equal to 2,
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold,
If it is determined that the relative value between is greater than or equal to the second threshold and there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to the second threshold, there is a B frame in the Kth GOP, and
If it is determined that the relative value between is greater than or equal to the third threshold, P _max is a step of determining that it is a scene change frame,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average of the sizes of multiple P frames in the Kth GOP,
Large but the P _max, a median or average value of the sizes of all B-frames between the nearest and P _max than previous scene change frame into P _max, the first threshold, than 0 And a step in which the second threshold value is greater than 1 and the third threshold value is greater than 1.

According to the fourth aspect, there is provided a detection device for detecting a scene change frame in a video. The video includes N picture groups ( GOP ) , where N is an integer equal to or greater than 2, and the detection device includes a first determination unit and a second determination unit.

The detection device 4020 may further evaluate the quality of the video to be detected or the quality of the video in which the video segment is arranged based on the detected scene change frame.

Claims (28)

A method for detecting a scene change frame in a video, wherein the video includes N picture group GOPs, where N is an integer greater than or equal to 2,
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold, and
P _max is a scene change frame when it is determined that the relative value between is greater than or equal to the second threshold,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of a plurality of P frames in the K-th GOP, the first threshold is greater than 0 and less than 1, and the second threshold is greater than 1. A method, including a large step. The method comprises
formula
Calculating the first threshold according to
Further comprising calculating the first threshold according to
I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median of the sizes of all P frames of the video The method according to claim 1, wherein the method is an average value. The first threshold is the formula
If obtained according to
Or the first threshold is the formula
If obtained according to
The method of claim 2, wherein The method comprises
After P _max in the Kth GOP is determined as a scene change frame, the _median or average value of the sizes of P frames other than the P frame determined as a scene change frame in the video is used as a new P _median And the above formula
And calculating a new I _threshold according to claim 2, wherein the new I _threshold is used to determine whether P _max in the next GOP is a scene change frame. Or the method of 3. The method comprises
Detecting a scene change frame from I frames in the Nth GOP from the Mth GOP of the N GOPs;
If K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then whether or not the I frame in the (K + 1) th GOP is a scene change frame. And determining
The step of detecting a scene change frame from an I frame in an Nth GOP from an Mth GOP of the N GOPs,
When it is determined that the first distance is less than or equal to the distance threshold, the I frame in the (K + 1) -th GOP is determined not to be a scene change frame, and the first distance is The I frame in the (K + 1) th GOP; a scene change frame that is closest to the I frame in the (K + 1) th GOP and before the I frame in the (K + 1) th GOP; 5. A method according to any one of the preceding claims, particularly comprising the step of being a distance between. The method comprises
Setting the distance threshold, wherein the distance threshold is one of the following three lengths:
The length of the longest GOP of the N GOPs;
The median or average value of the lengths of the N GOPs, and the length L, the number of GOPs having the length L being the maximum among the N GOPs, the length L
6. The method of claim 5, further comprising the step of being one of:   When a new scene change frame is determined, if the second distance is less than the distance threshold, the distance threshold is updated to the second distance, and the second distance is notably the new scene change frame. 7. The method according to claim 5 or 6, wherein the distance is between a frame and a scene change frame that is closest to and before the new scene change frame. A method for detecting a scene change frame in a video, wherein the video includes N picture group GOPs, where N is an integer greater than or equal to 2,
Determining the maximum P frame P _max of all P frames in the Kth GOP, wherein the size of P _max is
And K is a variable in the range from M to N, 1 ≦ M ≦ N, and
The relative value between is greater than or equal to the first threshold,
If the relative value between is greater than or equal to a second threshold and it is determined that there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to a second threshold, a B frame exists in the Kth GOP, and
If it is determined that the relative value between is greater than or equal to the third threshold, P _max is a step of determining that it is a scene change frame,
But a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of P frames in the Kth GOP,
But a P _max, a median or average value of the sizes of all B-frames between the scene change frames before the nearest and P _max to P _max, the first threshold, from 0 And the second threshold value is greater than 1 and the third threshold value is greater than 1. The method comprises
formula
Calculating the first threshold according to
Calculating the first threshold according to: I _threshold is the first threshold, I _median is the _median or average value of the sizes of all I frames of the video, and P _median 9. The method of claim 8, wherein is the median or average value of the sizes of all P frames of the video. The method comprises
formula
Calculating the threshold value of 3 according to:
Calculating the third threshold according to: B _threshold is the third threshold, P _median is the _median or average of the sizes of all P frames of the video, and B _median 10. A method according to claim 8 or 9, wherein is the median or average value of the sizes of all B frames of the video. The method comprises
After P _max in the Kth GOP is determined as a scene change frame, the _median or average value of the sizes of P frames other than the P frame determined as a scene change frame in the video is used as a new P _median And the above formula
And calculating a new I _{threshold The} following, it said new I _{threshold The} is, P _max in the next GOP is used to determine whether a scene change frame, further comprising the steps of claim 8 The method according to any one of 10 to 10. The method comprises
Detecting a scene change frame from I frames in the Nth GOP from the Mth GOP of the N GOPs;
If K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then whether or not the I frame in the (K + 1) th GOP is a scene change frame. And determining
The step of detecting a scene change frame from an I frame in an Nth GOP from an Mth GOP of the N GOPs,
When it is determined that the first distance is less than or equal to the distance threshold, the I frame in the (K + 1) -th GOP is determined not to be a scene change frame, and the first distance is The I frame in the (K + 1) th GOP; a scene change frame that is closest to the I frame in the (K + 1) th GOP and before the I frame in the (K + 1) th GOP; 12. A method according to any one of claims 8 to 11, comprising in particular a step, which is a distance between. The method comprises
Setting the distance threshold, wherein the distance threshold is one of the following three lengths:
The length of the longest GOP of the N GOPs;
The median or average value of the lengths of the N GOPs, and the length L, the number of GOPs having the length L being the maximum among the N GOPs, the length L
13. The method of claim 12, further comprising the step of being one of:   When a new scene change frame is determined, if the second distance is less than the distance threshold, the distance threshold is updated to the second distance, and the second distance is notably the new scene change frame. 14. The method of claim 13, wherein the distance is between a frame and a scene change frame that is closest to and before the new scene change frame. A detection device for detecting a scene change frame in a video, wherein the video includes N GOPs, N is an integer equal to or greater than 2, and the detection device includes a first determination unit and a first determination unit. Including two decision units,
The first determination unit determines the maximum P frame P _max among all P frames in the Kth GOP, and the size of P _max is:
And K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N,
The second determination unit,
The relative value between is greater than or equal to the first threshold, and
When it is determined that the relative value between is greater than or equal to the second threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of a plurality of P frames in the Kth GOP, the first threshold is greater than 0 and less than 1, and the second threshold is less than 1. A detection device configured to be large. The second determination unit,
formula
Calculating the first threshold according to or
Further configured to calculate the first threshold according to
I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median of the sizes of all P frames of the video 16. The detection device according to claim 15, which is an average value. The first threshold is the formula
If obtained according to
Or the first threshold is the formula
If obtained according to
The detection device according to claim 16, wherein After the second determination unit determines that P _max in the Kth GOP is a scene change frame, the median value of the sizes of P frames other than the P frame determined as the scene change frame in the video Or use the mean as the new P _median and the above formula
The new I _{threshold The} calculated according the new I _{threshold The} is, P _max in the next GOP is used to determine whether a scene change frame, further configured to, according to claim 16 or 17 The method described in 1. The second determination unit,
A scene change frame is detected from I frames in the Nth GOP from the Mth GOP of the N GOPs.
If K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then whether or not the I frame in the (K + 1) th GOP is a scene change frame. Further configured to determine,
The second determination unit is configured to detect a scene change frame from an I frame in an Nth GOP from an Mth GOP of the N GOPs,
When it is determined that the first distance is equal to or less than the distance threshold, it is determined that the I frame in the (K + 1) -th GOP is not a scene change frame, and the first distance is the (K + 1) The distance between the I frame in the GOP and the scene change frame closest to the I frame in the (K + 1) th GOP and before the I frame in the (K + 1) th GOP The detection device according to any one of claims 15 to 18, which specifically includes: The second determination unit sets the distance threshold, and the distance threshold is one of the following three lengths:
The length of the longest GOP of the N GOPs;
The median or average value of the lengths of the N GOPs, and the length L, the number of GOPs having the length L being the maximum among the N GOPs, the length L
20. The detection device of claim 19, further configured to be one of:   When the second distance is smaller than the distance threshold when the second determination unit determines a new scene change frame, the second threshold is updated to the second distance, and the second distance is updated. Is in particular further configured to be a distance between the new scene change frame and a scene change frame that is closest to and before the new scene change frame. Or the detection apparatus of 20. A detection device for detecting a scene change frame in a video, wherein the video includes N picture groups GOP, N is an integer equal to or greater than 2, and the detection device includes a first determination unit. And a second determination unit,
The first determination unit determines the maximum P frame P _max among all P frames in the Kth GOP, and the size of P _max is:
And K is a variable ranging from M to N, and is configured such that 1 ≦ M ≦ N,
The second determination unit,
The relative value between is greater than or equal to the first threshold,
If the relative value between is greater than or equal to a second threshold and it is determined that there is no B frame in the Kth GOP, or
The relative value between is greater than or equal to the first threshold,
The relative value between is greater than or equal to a second threshold, a B frame exists in the Kth GOP, and
When it is determined that the relative value between is greater than or equal to the third threshold, P _max is determined to be a scene change frame,
Has a P _max, a median or average value of the size of a plurality of I frames between the nearest and the scene change frame before P _max to P _max,
Is the median or average value of the sizes of multiple P frames in the Kth GOP,
Has a P _max, a median or average value of the sizes of all B-frames between the scene change frames before the nearest and P _max to P _max, the first threshold value, from 0 And the second threshold value is greater than 1, and the third threshold value is greater than 1. The second determination unit,
formula
Calculating the first threshold according to or
Further configured to calculate the first threshold according to
I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and P _median is the _median of the sizes of all P frames of the video 23. The detection device according to claim 22, which is an average value. The second determination unit,
formula
Calculate the third threshold according to or
And is further configured to calculate the third threshold according to
I _threshold is the first threshold, I _median is the _median or average of the sizes of all I frames of the video, and B _median is the _median of the sizes of all B frames of the video The detection device according to claim 22 or 23, which is an average value. After the second determination unit determines that P _max in the Kth GOP is a scene change frame, the median value of the sizes of P frames other than the P frame determined as the scene change frame in the video Or use the mean as the new P _median and the above formula
25. The method according to claim 22, further comprising: calculating a new I _threshold according to: wherein the new I _threshold is used to determine whether P _max in the next GOP is a scene change frame. The detection device according to any one of the above. The second determination unit,
A scene change frame is detected from I frames in the Nth GOP from the Mth GOP of the N GOPs.
If K is less than N, it is determined whether P _max in the Kth GOP is a scene change frame, and then whether or not the I frame in the (K + 1) th GOP is a scene change frame. Further configured to determine,
The second determination unit is configured to detect a scene change frame from an I frame in an Nth GOP from an Mth GOP of the N GOPs,
When it is determined that the first distance is equal to or less than the distance threshold, it is determined that the I frame in the (K + 1) -th GOP is not a scene change frame, and the first distance is the (K + 1) The distance between the I frame in the GOP and the scene change frame closest to the I frame in the (K + 1) th GOP and before the I frame in the (K + 1) th GOP 26. The detection device according to any one of claims 22 to 25, which specifically includes: The second determination unit sets the distance threshold, and the distance threshold is one of the following three lengths:
The length of the longest GOP of the N GOPs;
The median or average value of the lengths of the N GOPs, and the length L, the number of GOPs having the length L being the maximum among the N GOPs, the length L
27. The detection device of claim 26, further configured to be one of:   When the second distance is smaller than the distance threshold when the second determination unit determines a new scene change frame, the second threshold is updated to the second distance, and the second distance is updated. 28, in particular, is further configured to be a distance between the new scene change frame and a scene change frame that is closest to and before the new scene change frame. The detection device according to 1.