WO2009063172A2 - Détection de pause/arrêt - Google Patents

Détection de pause/arrêt Download PDF

Info

Publication number
WO2009063172A2
WO2009063172A2 PCT/GB2008/003743 GB2008003743W WO2009063172A2 WO 2009063172 A2 WO2009063172 A2 WO 2009063172A2 GB 2008003743 W GB2008003743 W GB 2008003743W WO 2009063172 A2 WO2009063172 A2 WO 2009063172A2
Authority
WO
WIPO (PCT)
Prior art keywords
frames
sequence
static
pause
std
Prior art date
Application number
PCT/GB2008/003743
Other languages
English (en)
Other versions
WO2009063172A3 (fr
Inventor
Roel Peter Geert Cuppen
Winfried Antonius Henricus Berkvens
Ramon Antoine Wiro Clout
Original Assignee
Ambx Uk Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ambx Uk Limited filed Critical Ambx Uk Limited
Priority to GB1007837.6A priority Critical patent/GB2467075B/en
Publication of WO2009063172A2 publication Critical patent/WO2009063172A2/fr
Publication of WO2009063172A3 publication Critical patent/WO2009063172A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4348—Demultiplexing of additional data and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H04N21/8133—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H04N21/814—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts comprising emergency warnings
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus

Definitions

  • This invention relates to a method and system for detecting a pause or stop in a sequence of image frames.
  • the invention provides pause/stop detection during synchronization of content.
  • US2002169817 discloses a real-world representation system which comprises a set of devices, each device being arranged to provide one or more real-world parameters, for example audio and visual characteristics. At least one of the devices is arranged to receive a real-world description in the form of an instruction set of a markup language and the devices are operated according to the description. General terms expressed in the language are interpreted by either a local server or a distributed browser to operate the devices to render the real- world experience to the user. In this way a script is delivered that is used to control other devices alongside the television delivering the original content.
  • synchronization mechanisms cannot detect when video content has been paused or stopped, because they only have access to the video content (which may be analogue or digital) of the video playback system, in which no trick play information is available. This is expected to be the case also in the future, owing to the possibility that the synchronization mechanism for augmentation systems will be implemented in a device separate from the video playback system.
  • the current synchronization mechanism delivers incorrect or no timing information. If the augmentation system and the script playback fully depend on this timing information, then it is to be expected that the augmentation system will provide an incorrect experience for the user, during the non-linear playback situation. This decreases the quality of the augmentation experience, when the user is watching video content. For example, if the synchronization mechanism continues to deliver timing information during a pause or stop situation, then the script playback (which is controlling the augmenting devices such as additional lights, and so on) is continued. This means that augmentation effects are being rendered in a situation in which the script playback should be paused or stopped.
  • the synchronization mechanism still delivers incorrect timing information due to not detecting the pause or stop. This results in an incorrect synchronization for some time after resuming the linear playback. Detecting pause and stop situations in order to pause or stop the augmentation script playback are therefore preferred.
  • a method for detecting a pause or stop in a sequence of image frames comprising receiving a sequence of frames, calculating, for frames in the sequence, a static value, accessing data for comparable frames, comparing the calculated value with the data for the comparable frames, and generating a pause/stop output accordingly.
  • a system for detecting a pause or stop in a sequence of image frames comprising a receiver arranged to receive a sequence of frames, and a processor arranged to calculate, for frames in the sequence, a static value, to access data for comparable frames, to compare the calculated value with the data for the comparable frames, and to generate a pause/stop output accordingly.
  • a computer program product on a computer readable medium for detecting a pause or stop in a sequence of image frames, the product comprising instructions for receiving a sequence of frames, calculating, for frames in the sequence, a static value, accessing data for comparable frames, comparing the calculated value with the data for the comparable frames, and generating a pause/stop output accordingly.
  • the invention provides a detection mechanism, which indicates whether the video content is being played, paused, or stopped, by looking at the actual amount of static and dynamic frames in the video content and by comparing this amount with the expected amount of static and dynamic frames in the video content. This indication can be used in an augmentation system to continue or pause the script playback.
  • This pause/stop detection system makes use of the fingerprint algorithms that are also used in some of the current synchronization mechanisms.
  • the step of receiving a sequence of frames comprises receiving a signal comprising the sequence of frames and the data. Since the information about the frames in the sequence must be compared to reference data, it is logical to include that data in the signal that contains the image frames. This data may be on a frame by frame basis, or may be relevant to a group of frames. If the data is not present in the signal, which will be the case if the sequence of frames is the pure video data output from a device, then the data must be retrieved from a server.
  • the step of calculating, for frames in the sequence, a static value comprises calculating the sum of temporal differences between a frame and a previous frame.
  • sum of temporal differences is shorthand for a calculation of the difference between two frames which is the sum of absolute temporal differences of spatial differences. This is one way in which a static value can be calculated for a frame in the sequence. To illustrate the calculation, a worked example is included in the specification below. In a preferred embodiment, all of the frames in the sequence (except the first) will have a static value calculated. This value may be on a scale of 0 to 16320, or any other suitable scale, as defined by the implementation of the system.
  • the step of calculating, for frames in the sequence, a static value comprises generating a binary output designating frames in the sequence as static or dynamic.
  • each frame is either a static frame or a dynamic frame, which is a relative concept, relative to the previous frame.
  • a frame that has not altered (much) from the previous frame is designated as "static” while a frame that has altered is designated as "dynamic”. If 0 is a static frame and 1 a dynamic frame, then a series often frames could have their static value calculated as 0110010101, each 0 or 1 representing one frame.
  • the data representing the expected value could be a similar binary value, or could simply be a number such as 4, meaning that of the ten frames, four are expected to be static.
  • the step of comparing the calculated value with the data for the comparable frames comprises determining whether the number of static frames in the sequence of frames exceeds the expected number of static frames by a predetermined threshold. In the example above, there are five static frames (five O's), however to limit the likelihood of false outputs, a threshold of "2" could be applied, meaning that a stop/pause output is only generated when the number of detected static frames exceeds the number of expected static frames by two.
  • More complicated schemes are also possible, in which not only the static difference between the actual frames and the reference frames is important, but also the amount of static frames itself in the actual frames and reference frames indicate whether a sequence is static or dynamic, hi this case the step of generating a pause/stop output only generates a pause/stop output if the number of static frames in the sequence of frames exceeds a predetermined threshold and the number of static frames in the comparable frames is below a further predetermined threshold.
  • the step of generating a binary output designating frames in the sequence as static or dynamic comprises applying a threshold to the calculated sum of temporal differences.
  • a threshold can be applied to this scale, with values below designating a frame as static, and values on or above meaning that a frame is designated as dynamic.
  • This process can be configured so that the threshold varies according to at least a portion of the content of the sequence of frames, which means that different thresholds are used at different times to determine whether a frame is static or not.
  • the step of generating a pause/stop output only generates a pause/stop output if the mean static value of a predetermined number of frames is below a predetermined threshold.
  • Alternatives for the "mean static value” are the “maximum static value” or "sum of static values”. This advantageous feature again provides protection against false pause/stop outputs.
  • the absolute value of the static values is used. For example, for a sequence often frames (on a scale of 0 to 16320) the static values for the frames could be, 0, 45, 35, 20, 235, 0, 15, 30, 22, and 108. This will then be compared with the expected values for those ten frames to determine if the sequence is static or dynamic.
  • the method further comprises, following generation of a pause/stop output, continuing to calculate, for frames in the sequence, a static value, and in a predefined circumstance, generating a resume output.
  • a resume output is needed to continue the synchronisation of the external source (the augmentation effects for example) with the video content.
  • the detection of the resume condition may be done as the inverse of the process that detected the pause/stop, but this is not necessarily the case.
  • An entirely different algorithm or routine may be used to determine that the resume should be output.
  • Figure 1 is a schematic diagram of an entertainment system
  • Figure 2 is a screenshot of a display device
  • Figure 3 is a flowchart of a method of detecting a pause or stop in a sequence of image frames
  • Figure 4 is a diagram of two image frames
  • Figure 5 is a diagram of two mean luminance matrices for the image frames of Figure 4
  • Figure 6 is a diagram of two spatial difference matrices, derived from the mean luminance matrices of Figure 5 and
  • Figure 7 is a diagram of a temporal difference matrix, derived from the spatial difference matrices of Figure 6.
  • Figure 1 shows a DVD player 10 connected to a display device 12 through a device 14.
  • a signal 16 is output from the DVD player 10, in the conventional fashion.
  • the device 14 is intercepting the signal 16 in order to monitor that signal 16, which it nevertheless passes on unaltered to the display device 12.
  • the device 14 is shown as an independent device, but the functionality that is provided by that device 14 could be located within the DVD player 10 or within the display device 12.
  • Figure 1 shows a conventional DVD player 10 and a conventional display device 12, with the device 14 located as a separate function in-between the two devices 10 and 12.
  • the device 14 need not be interposed between the player 10 and the display device 12, it could be connected to a separate output of the player 10, or the output of the player 10 could be divided into two, with one going to the display 12 and the other going to the device 14.
  • the signal 16 comprises a sequence 18 of frames 20 and also data 22.
  • the signal 16 includes within it specific data 22, which contains data about each respective frame 10 within the sequence 18.
  • the data 22 need not be present within the signal 16, nor need it be on a frame-by- frame basis. If the data 22 is not present in the signal 16, then the device 14 will obtain it from another location, such as by connecting to a remote server.
  • the device 14 is a system for detecting a pause or stop in the sequence 18 of image frames 20, and comprises a receiver 24, which is arranged to receive the sequence 18 of frames 20, and a processor 26, which is arranged to perform various calculations and processes on the frames 20, in order to detect the pause or stopping of the DVD player 10.
  • a transition from playback to pause or stop implies a transition from a dynamic frame playback to a static frame playback.
  • the video content consists of dynamic frames with significant differences between sequential frames and that during pause or stop the video content consists of static frames with no significant difference between sequential frames.
  • a certain frame of the content is repeated continuously, which results in a static frame playback.
  • a static background frame is showed continuously, which also results in a static frame playback.
  • Figure 2 shows an example of a screenshot 28 that will be a background image which will be shown during a stop sequence (i.e. after a user has pressed the stop button on their remote control device that is controlling the DVD player 10).
  • a dynamic stop namely a Screensaver
  • certain static stop period for example, a few minutes. It could be the case that incorrect timing information is found during this dynamic stop, which is used to synchronize the video content incorrectly.
  • the device 14 detects the pause/stop of the output, before any screensaver is present.
  • the processor 26 of the device 14 is arranged to calculate, for frames 20 in the sequence 18, a static value, to access the data 22 for comparable frames 20, to compare the calculated value with the data 22 for the comparable frames 20, and to generate a pause/stop output accordingly. Effectively the processor 26 is calculating whether a calculated measure of whether a frame or frames is/are static deviates from an expected measure.
  • the expected measure is encapsulated by the data 22.
  • This data 22 may define a static value (on a suitable scale) for each frame 20 or may define a static value that refers to a group of frames. For example, a mean value of a group of frames 20 could be contained within the data 22.
  • the processor 26 is calculating an actual static value of a frame 20 that it receives, which is a relative concept, meaning that its static value is calculated relative to the previously received frame 20. For example, if a user stops the DVD player 10, then an image such as that shown in Figure 2 is repeatedly shown by the player 10. Since the same frame 20 is being continually passed to the device 14, then each frame will have a low static value, it does not differ from the previous frame. It is not enough however, for the device 14 to simply look for a period of static frames 20, as it is common in material shown from a DVD player 10 to have numerous static frames 20 in sequence, since twenty four to thirty frames a second are conventionally used, and a single image displayed for one second (common in films) will generate twenty four to thirty frames 20 that have low static values.
  • Movies are typically recorded at 24 frames/s. In Europe, the frame-rate is typically 25 frames/s and in the US typically 30 frames/s. The solution works for all of them (or any other rate). To provide a working solution, the processor 26 must compare the calculated static value with some reference information, which is embodied in the data 22.
  • Figure 3 summarises the method for detecting a pause or stop in the sequence 18 of image frames 20, which is executed by the device 14.
  • the method comprises the steps of, step Sl, receiving the sequence 18 of frames 20, step S2, calculating, for frames 20 in the sequence 18, a static value, step S3 accessing data 22 for comparable frames 20, step S4 comparing the calculated value with the data 22 for the comparable frames 20, and step S5 generating a pause/stop output accordingly.
  • This generalised methodology can be implemented in a number of different ways.
  • the data 22 can be structured in a frame-by-frame basis, as shown in Figure 1.
  • the device 14 can receive a signal which comprises the sequence 18 of frames 20 and the data 22, or alternatively, the data 22 may be recalled from a remote store, for example via an Internet connection that the device 14 has available, hi the same way, when calculating the static value for a sequence 18 of frames 20, that value may relate to a single frame 20 or to a group of frames 20.
  • the step S2 of calculating, for frames 20 in the sequence 18, a static value comprises calculating the sum of temporal differences between a frame 20 and a previous frame 20. Therefore, a static value is calculated for each frame 20 in the sequence 18.
  • the step of calculating, for frames in the sequence, a static value comprises generating a binary output designating frames 20 in the sequence 18 as static or dynamic.
  • each received frame 20 is designated as either static or dynamic.
  • the step of generating such a binary output can comprise applying a threshold to the calculated sum of temporal differences, hi this case, the step S4 of comparing the calculated value with the data 22 for the comparable frames 20 comprises determining whether the number of static frames 20 in the sequence 18 of frames 20 exceeds the expected number of static frames 20 by a predetermined threshold.
  • the number of actual static frames detected should be above a certain threshold (referred to as SF ⁇ H_actuai below) and the number of expected static frames, as determined form the data 22, is below or equal to a different threshold (referred to below as SFJH expected)-
  • the STD can be used to indicate the extent to which two sequential frames are static or dynamic. Each frame 20 has an STD value relative to the previous frame 20. The same frame can have different STD values, depending upon which frame was the immediate predecessor.
  • One method by which the STD is calculated as follows: 1. Two sequential frames 20 are each divided into a matrix of blocks, for example of nine columns and four rows. Figure 4 shows two sequential frames 20a and 20b with matrices superimposed over the two sequential frames 20a and 20b. The size and shape of the matrix that is used with respect to the frames 20 is a design choice, and the one shown is known to deliver a working solution.
  • the mean luminance (ML) of each block is calculated (with 0 ⁇ ML ⁇ 255).
  • Figure 5 shows the values for the mean luminance in the matrices 30a and 30b of the frames 20a and 20b shown in Figure 4.
  • noise can be created, for example, by the playback system.
  • This noise can influence the mean luminance values of the blocks in the matrices 30. From experiments, it has been shown that the amount of noise and the position of the noise are often frame-dependent. Therefore, to filter out this "frame dependent noise" the differences of neighbouring blocks (columns 'c' and 'c+1 ') in a row are calculated. These spatial differences are used in the next step to calculate the differences of the blocks between two sequential frames 20a and 20b.
  • Figure 6 shows the spatial differences matrices 31, which are derived from the mean luminance matrices 30.
  • the matrices 31 have one fewer column then the matrices 30.
  • the differences of neighbouring blocks in a row are calculated. But it also possible to do this on a column basis or by taking the difference of any two blocks within a frame.
  • the STD is calculated by adding up the absolute values of the temporal differences.
  • This STD indicates how much the two frames 20a and 20b differ from each other.
  • the lower the STD is, the more static two frames are (for example, 'STD 0' means that two frames are identical).
  • the STD of the two sequential frames above is 470. This means that the two frames 20a and 20b are substantially different, which is clearly visible in the screenshots of Figure 4 above.
  • Dynamic frame playback vs. Static frame playback
  • a threshold for the STD is introduced, namely STD TH . If the STD of two frames is lower than STD JH , then the two frames are static, or else the two frames are dynamic. So, the following detection criteria can be used to decide whether two sequential frames are static or dynamic: Static frame:
  • the amount of static frames within a predetermined window determines if the playback of the video content is considered to be static or dynamic; if the amount of static frames of the last 'N' frames exceeds a predefined static frame threshold, namely SF TH , then the playback is static, or else the playback is dynamic. So, the following detection criteria can be used to decide whether the current frame playback is static or dynamic: Static frame playback:
  • the dynamic and static frame playback indication is an actual characteristic of the video content, which can be used to detect static and dynamic video content. However, if only this actual characteristic would be used to detect pause and stop, then static parts of the video content during a playback would also be detected as pause or stop. To detect an actual pause or stop, the characteristics should be compared with the expected characteristic of the video content. If the actual video content is static but the expected video content is dynamic, then the video content is paused or stopped, if the actual video content is dynamic, then the video content is always in a playback situation independent of the expected video content, and if both the actual and expected video content are static, then it is expected that the playback of the video content has reached a static part.
  • the expected characteristics (e.g. 'STD ex pected (f)' or 'STD exp ected (f) ⁇ STD ⁇ H_e ⁇ pe c t ed ') of the corresponding frames cannot be retrieved from the actual video content.
  • One solution is to store (one of) these characteristics in the timetable of the video content that is also used in any augmentation synchronization mechanism. This means that the expected characteristics are linked to the fingerprints of the video frames 20.
  • the 'STDe ⁇ pected(f)' or 'STD expecte d(f) ⁇ STDjH expected' used in the algorithm above can be retrieved from the timetable and can be compared with the corresponding 'STD act uai(f)'.
  • An example of a part of such a timetable (which forms part of the data 22):
  • Each frame 20 has a fingerprint, and a pre-calculated STD and/or a binary value indicating whether the frame 20 is considered to be static or dynamic. Both the STD and the binary value are relative values for the specific frame 20, relative to the previous frame.
  • the expected video content represented in the timetable can differ from the actual video content, while both contents are the same.
  • the step of generating a pause/stop output can be carried out so that there is only generated a pause/stop output if the mean static value of a predetermined number of frames is below a predetermined threshold.
  • This can be considered to be a mean STD constraint. This is to deal with the situation where it could still be possible that video content adhering to the conditions above is actually not paused or stopped but is still being played, but will nevertheless deliver a pause/stop output. Namely, from experiments it has been noticed that video content is unlikely to be paused or stopped when there are one or two strong STDs within the last 'N' actual frames. For example, consider the following STDs of a sequence of actual and expected frames:
  • STD actual (f-n) ⁇ STD mean (e.g. STD mean 4.0)
  • the resume detection follows the hysteresis principle to make the detection system more robust against possible disturbances during pause and stop.
  • the device 14 uses it's own thresholds for the amount of static frames, namely SFxH resumej and the mean STD, namely STD mean to calculate whether to reverse the pause/stop output. Only if the amount of static frames within the actual video content is below the static frame threshold and if the mean STD is above the mean STD threshold, is the video content resumed again:
  • the resume condition does not look at the expected STDs, because during a pause or stop situation the time of the video content is paused. This means that the position of the synchronization mechanism in the timetable stays equal during the pause or stop, and thus the amount of expected low STDs stays equal.
  • the STD values of the frames represented by the timetable are also stored in the timetable, then these values can be used to calculate the difference of the brightness level between the captured video and the reference video in the timetable. The result of this calculation can be used to adapt the actual STD threshold, namely STD ⁇ H_ ac tuai for static video.
  • the principal advantage of the proposed solution is that pause and stop can be detected in a sequence of image frames. This improvement results in a better augmentation experience that can be delivered to the user during playback, pause, and stop situations of the video content.
  • the invention can be used for an improvement of an augmentation experience during trick play.
  • This invention can be used in a suitable decoder, which could be packaged as a separate device 14 or could become part of future TVs and/or disc-player devices in the home cinema domain.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Emergency Management (AREA)
  • Business, Economics & Management (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

L'invention concerne un procédé pour détecter un arrêt ou une pause dans une séquence de trames d'image comprenant la réception d'une séquence de trames, le calcul d'une valeur statique pour des trames de la séquence, l'accès à des données pour des trames comparables, la comparaison de la valeur calculée avec les données des trames comparables, et la production en conséquence d'une sortie pause/arrêt. L'étape de calcul d'une valeur statistique pour des trames de la séquence peut comprendre le calcul de la somme des différences temporelles entre une trame et une trame précédente.
PCT/GB2008/003743 2007-11-14 2008-11-07 Détection de pause/arrêt WO2009063172A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1007837.6A GB2467075B (en) 2007-11-14 2008-11-07 Pause/stop detection

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP07120662 2007-11-14
EP07120662.7 2007-11-14

Publications (2)

Publication Number Publication Date
WO2009063172A2 true WO2009063172A2 (fr) 2009-05-22
WO2009063172A3 WO2009063172A3 (fr) 2009-07-02

Family

ID=40469929

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2008/003743 WO2009063172A2 (fr) 2007-11-14 2008-11-07 Détection de pause/arrêt

Country Status (2)

Country Link
GB (1) GB2467075B (fr)
WO (1) WO2009063172A2 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001033863A1 (fr) * 1999-11-04 2001-05-10 Koninklijke Philips Electronics N.V. Detection de scene importante et filtrage de trame destines a un systeme d'indexation visuelle utilisant un seuil dynamique
US20060133676A1 (en) * 2004-12-21 2006-06-22 Fujitsu Limited Device and method for image processing
EP1729529A1 (fr) * 2005-06-02 2006-12-06 BRITISH TELECOMMUNICATIONS public limited company Détection de perte d'un signal vidéo
GB2428924A (en) * 2005-07-22 2007-02-07 Snell & Wilcox Ltd Automatic detection of still frames in video content
WO2007072326A2 (fr) * 2005-12-23 2007-06-28 Koninklijke Philips Electronics N.V. Synchronisation de script a partir d'empreintes extraites d'un train de donnees de contenu
US20070192782A1 (en) * 2004-08-09 2007-08-16 Arun Ramaswamy Methods and apparatus to monitor audio/visual content from various sources
WO2009022229A1 (fr) * 2007-08-15 2009-02-19 Ati Technologies Ulc Réduction automatique de la consommation électrique d'un dispositif d'affichage vidéo

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001033863A1 (fr) * 1999-11-04 2001-05-10 Koninklijke Philips Electronics N.V. Detection de scene importante et filtrage de trame destines a un systeme d'indexation visuelle utilisant un seuil dynamique
US20070192782A1 (en) * 2004-08-09 2007-08-16 Arun Ramaswamy Methods and apparatus to monitor audio/visual content from various sources
US20060133676A1 (en) * 2004-12-21 2006-06-22 Fujitsu Limited Device and method for image processing
EP1729529A1 (fr) * 2005-06-02 2006-12-06 BRITISH TELECOMMUNICATIONS public limited company Détection de perte d'un signal vidéo
GB2428924A (en) * 2005-07-22 2007-02-07 Snell & Wilcox Ltd Automatic detection of still frames in video content
WO2007072326A2 (fr) * 2005-12-23 2007-06-28 Koninklijke Philips Electronics N.V. Synchronisation de script a partir d'empreintes extraites d'un train de donnees de contenu
WO2009022229A1 (fr) * 2007-08-15 2009-02-19 Ati Technologies Ulc Réduction automatique de la consommation électrique d'un dispositif d'affichage vidéo

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ALAN HANJALIC: "Shot-Boundary Detection: Unraveled and Resolved?" IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 12, no. 2, 1 February 2002 (2002-02-01), XP011014272 ISSN: 1051-8215 *

Also Published As

Publication number Publication date
GB2467075A (en) 2010-07-21
GB2467075B (en) 2012-07-04
GB201007837D0 (en) 2010-06-23
WO2009063172A3 (fr) 2009-07-02

Similar Documents

Publication Publication Date Title
JP5461176B2 (ja) ビデオを識別及び同期させるためのフィンガープリント、装置及び方法
US8559798B2 (en) Image frame processing method and device for displaying moving images to a variety of displays
CA3069853C (fr) Identification de parties precedemment diffusees en continu d'un titre multimedia pour eviter une lecture repetitive
US11282165B2 (en) Dynamically cropping digital content for display in any aspect ratio
TWI345418B (en) Brightness adjusting methods for video frames of video sequence and brightness adjusting devices thereof
US20240089530A1 (en) Temporal placement of a rebuffering event
EP4300970A2 (fr) Systèmes et procédés de commande de jeu vidéo
US20100164975A1 (en) Picture Signal Processing Device, Picture Signal Processing Method, and Picture Display Device
WO2009063172A2 (fr) Détection de pause/arrêt
KR20060086463A (ko) 패런털 레벨을 이용한 시청 제한 제어장치 및 방법
KR20010050617A (ko) 선택된 데이터 세그먼트 내에서 소정의 수의 대표적인데이터를 확인하기 위한 방법
US9848156B2 (en) Television system and multimedia playing method
CN109831703B (zh) 一种针对hdmi信号的运动补偿方法及装置
US20140112646A1 (en) Image display device and image processing device
CN111246280B (zh) 智能电视自动获取应用弹窗图像的方法
KR100996413B1 (ko) 감시용 디지털 비디오 레코더에서의 모션 검출 민감도설정장치 및 방법
JP5403784B2 (ja) 映像処理装置およびノイズ軽減方法
JP2014150314A (ja) 情報処理装置及び情報処理方法
WO2009112803A1 (fr) Traitement d’une séquence de trames
US7932936B2 (en) System that automatically adjusts threshold value of cross color suppression and method thereof
JP2001197425A (ja) 映像信号処理装置
CA3087909A1 (fr) Amelioration de l`amplification de video pour les spectateurs ayant une deficience visuelle
KR20120088260A (ko) 표시장치 및 표시장치의 제어방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08849682

Country of ref document: EP

Kind code of ref document: A2

ENP Entry into the national phase

Ref document number: 1007837

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20081107

WWE Wipo information: entry into national phase

Ref document number: 1007837.6

Country of ref document: GB

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08849682

Country of ref document: EP

Kind code of ref document: A2