Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
  • H04N19/87—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving scene cut or scene change detection in combination with video compression
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties

Definitions

• the invention relates to a device and a procedure for detecting fades in a sequence of images.
• a shot or sequence shot is an uninterrupted sequence of images filmed by a camera or a camcorder in a single picture capture.
• This shot is the basic unit used to analyse or construct a video content.
• a video sequence generally comprises a succession of shots interconnected with the aid of video editing procedures. These procedures in particular make it possible to create an abrupt transition between two shots or else a progressive transition according to which an image is progressively changed into another image by mixing of the two images (e.g. a fade).
• the detection of these transitions makes it possible in the case of an image sequence coding application to improve the coding quality by taking account of these transitions in the choices (for example structure of a group of images or "GOP") made by the coder. This detection also makes it possible to retrieve the shot boundaries and narrative units so as in particular to allow non-linear navigation around the content.
• the invention is aimed at detecting progressive transitions, more particularly fades, in a sequence of images.
• the invention relates to a method for detecting a fade in a sequence of images comprising pixels or image points with each of which is associated at least one luminance value. It comprises the following steps:
• the luminance level associated with an image is equal to the sum of the luminance values associated with each of the pixels of the image.
• the method furthermore comprises an additional step for validating the presence of a fade in the detection window if a level of motion associated with the detection window is less than a predetermined threshold.
• the level of motion associated with the current image is equal to the ratio between the temporal activity calculated between the current image and the image preceding the current image, termed the preceding image, and the spatial activity calculated for the current image.
• the level of motion associated with the decision window is equal to the level of motion associated with the last image of the decision window.
• the temporal activity between the image and the preceding image is equal to the sum of the absolute values of the pixel-wise differences between the luminance values associated with each pixel of the image and the luminance values associated with each pixel of the preceding image.
• the spatial activity of the image is equal to the sum of the absolute values of the luminance values associated with each pixel of the image.
• the invention also relates to a device for detecting fades in a sequence of images comprising pixels or image points with each of which is associated at least one luminance value. It comprises: - means (32, 33, 34) of calculation for calculating, for each of the images of a window N consecutive images in length, termed the detection window, and for the image preceding the detection window, a luminance level on the basis of the luminance values associated with each of the pixels of each of the images; - means (32, 33, 34) of detection for detecting a fade, for each of the
• N images of the detection window, based on the difference between the luminance level associated with the current image and the luminance level associated with the image preceding the current image; and - means for deducing (32, 33, 34) the presence of a fade for the detection window if for at least k images of the detection window a fade has been detected by the detection means, with 2 ⁇ k ⁇ N.
• the invention relates furthermore to a video coding device which comprises coding means and a device for detecting fades according to the invention.
• the invention relates moreover to a device for structuring a sequence of images characterized in that it comprises indexation means and a device for detecting fades according to the invention.
• FIG. 5 illustrates a device according to the invention
• FIG. 6 illustrates an image sequence coding device according to the invention.
• FIG. 7 illustrates an image sequence structuring device according to the invention.
• the invention is aimed at detecting fades in a sequence of images, the sequence being progressive or interlaced, each image comprising pixels with each of which is associated in particular a luminance value.
• each image comprises an even field and an odd field.
• Each field itself comprises pixels or image points with each of which is associated in particular a luminance value.
• a fade from f to s is called a fade-in and vice versa
• a fade from s to f is called a fade-out.
• - (x,y) are the coordinates of the pixels in the image.
• the factor x(n) is homogeneous inside an image or frame (i.e. independent of x and y) and lies between 0 and 1.
• L e (n) ( ⁇ -x(n))*L s (n) + x(n)L f (n).
• x(n) is split up in the following manner:
• N k ⁇ and k 2 are two coefficients making it possible to express the linear nature of the fading factor.
• the variation in the luminance level between two images (or frames) is dependent on the variation in the luminance level between the start and the end of the fade [L 8 -Lf), and is inversely proportional to the duration (N images or frames) of the fade.
• the method according to the invention is therefore based on the detection of a variation in the luminance level between two successive images. This variation is thereafter compared with an experimentally defined threshold.
• the method is split up into 4 steps referenced 10 to 13.
• the modules represented are functional units, which may or may not correspond to physically distinguishable units.
• Step 10 consists in calculating, over a window N images in length, termed the detection window, a luminance level for each image of the detection window and for the image of the sequence which precedes this window.
• the luminance level of an image is the sum over the whole of the image of the luminance values associated with each of the pixels of this image.
• a window N images in length is a portion of the sequence comprising N successive images.
• Step 11 consists in detecting a fade, for each image of the detection window, by comparing the variation in the luminance level between this image, termed the current image, and the image which precedes it with a threshold.
• This threshold is a multiple of the size of the image as a number of pixels. For example, it is equal to 1.75 multiplied by the width and by the height of the image as a number of pixels. If the variation in the luminance level is greater than this threshold then a fade is detected for the current image. Depending on whether one is seeking to detect fade-in or fade-out, the variation in the luminance level is calculated differently.
• the variation in the luminance level is equal to the difference between the luminance level of the image preceding the current image and the luminance level of the current image.
• the variation in the luminance level is equal to the difference between the luminance level of the current image and the luminance level of the image preceding it.
• this step consists in calculating the number of images of the detection window for which a fade has been detected.
• the results arising from the previous step for fade-in and fade-out are combined thereby making it possible to detect a fade whether it be a fade-in or fade-out.
• a fade-in or fade-out has been detected in step 11 for at least k (2 ⁇ k ⁇ N) images of the detection window, we deduce therefrom the presence of a fade for the detection window.
• the length of the detection window is preferably equal to 8 and k is equal to 6. This step in particular makes it possible to avoid determining false fades due to noise, or else to an instability of the brightness in the course of the sequence.
• the result of the previous step 12 is combined with a local evaluation of the level of motion of the decision window.
• This level of motion can be provided for example by an image level decision module (for example decision of the frame/image mode of coding, decision of the type of the image I 1 P, or B%) of a coding device.
• the level of motion calculated for the last image of the decision window is the level of motion associated with the decision window. If the presence of a fade has been deduced for the detection window at the previous step 12 and if the level of motion for this window is less than a certain threshold, then the presence of a fade for the detection window is validated.
• the decision window can for example have a size of three frames.
• This window shown in Figure 2 can in particular be used, in a coding device, by an image level decision module.
• the window is positioned so as to take an image level decision for the image referenced Pk corresponding to the frames referenced FM and Fj-2- More precisely, the image P k is either an image consisting of the union of the two frames F ⁇ i and F ⁇ 2 ("frame picture"), or an image consisting of two independent frames F ⁇ i and F i-2 ("field picture").
• the image level decisions (for example decision of the "frame pictureTfield picture” mode of coding, decision of the type of the image I, P, or B?) apply at least over two successive frames.
• a frame is repeated. The size of the decision window is therefore three frames.
• the variation in the level of luminance calculated in step 11 is calculated between the frame positioned in Fn in the sliding window and the frame at the same position in the sliding window such as positioned so as to take an image level decision for the preceding image referenced P ⁇ -i.
• Figures 3 and 4 represent the position of the decision window for the following image Pk+i. Specifically, if Pk is composed of the union of F ⁇ i and Fj -2 and if Fj is not a repeated frame or if Pk is composed of two independent frames F ⁇ i and Fj -2 , then the following position of the decision window for the image Pk+i is illustrated by Figure 3.
• the present invention also relates to a device for detecting fades, referenced 30 in Figure 5, implementing the method described previously. Only the essential elements of the device are represented in Figure 5.
• the device 30 comprises: a random access memory 32 (RAM or similar component), a read only memory 33 (hard disk or similar component), a processing unit 34 such as a microprocessor or a similar component and an input /output interface 35. These elements are linked together by an address and data bus 31.
• the read only memory 33 contains the algorithms implementing steps 10 to 12 and optionally step 13 of the method according to the invention.
• the processing unit 34 loads and executes the instructions of these algorithms.
• the random access memory 32 in particular comprises the programs for operating the processing unit 34 loaded on power-up of the appliance, as well as the images to be processed.
• the input /output interface 35 has the function of receiving the input signal (i.e. the source image sequence) and outputs the result of the fade detection according to steps 10 to 12 (or 13) of the method of the invention.
• the device for detecting fades 30 can be used in a coding device referenced 40 in Figure 6.
• This device comprises in particular coding means 41 for coding the source image sequence 400.
• the coding means comprise at least decision means making it possible to select the type of image (e.g. intra (I image), predicted (P images), bidirectional (B images)).
• This module can for example implement the MPEG-2 coding standard defined in particular in the document ISO/IEC 13818-2 (entitled "Information technology -- Generic coding of moving pictures and associated audio information: Video"). More generally, it can implement any standard for coding image sequences. In particular it uses the information provided by the device 30 to dynamically adapt the type of image. This can make it possible to improve the cost of compression and the quality of the decoded images.
• the device for detecting fades 30 can also be integrated with a device for structuring image sequences, referenced 50 in Figure 5.
• This device 50 comprises in particular indexation means 51 making it possible to create a description 501 of the source image sequence 400 (e.g. temporal marker, otherwise known as a "time code", of start and end of fades).
• This device 50 in particular makes it possible to retrieve the shot boundaries and narrative units so as to make it possible to navigate non-linearly through the sequence or to generate digests. More precisely, it makes it possible to retrieve the appropriate shot boundaries when a fade has been introduced during editing by virtue of the device for detecting fades 30 and therefore provides better bases for structuring into narrative units.
• the knowledge of the fades gives high-level information on the structuring of the video.
• the device 50 therefore makes it possible to structure the sequence for example into chapters and sub-chapters in the case of digital sequences of images as on a DVD.
• a fade to black or fade-out followed by a fade from black or fade-in very often indicates a change of "chapter".
• slow motions - which make it possible to detect the interesting phases of a game - are very often delimited by fades.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Studio Circuits (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Image Analysis (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (2)

Family

ID=35717654

Family Applications (1)

Country Status (7)

Cited By (1)

Family Cites Families (17)

• 2005
  • 2005-06-23 FR FR0551728A patent/FR2887731A1/fr active Pending
• 2006
  • 2006-06-20 WO PCT/EP2006/063328 patent/WO2006136544A2/en not_active Ceased
  • 2006-06-20 JP JP2008517478A patent/JP2008544666A/ja not_active Ceased
  • 2006-06-20 US US11/922,439 patent/US8165337B2/en not_active Expired - Fee Related
  • 2006-06-20 KR KR1020077029367A patent/KR20080027255A/ko not_active Abandoned
  • 2006-06-20 EP EP06763780A patent/EP1894417A2/en not_active Withdrawn
  • 2006-06-20 CN CN200680022280XA patent/CN101204093B/zh not_active Expired - Fee Related
• 2012
  • 2012-03-21 US US13/425,645 patent/US20120176548A1/en not_active Abandoned

Non-Patent Citations (5)

Also Published As

Similar Documents

Legal Events