Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/85—Assembly of content; Generation of multimedia applications
  • H04N21/854—Content authoring
  • H04N21/8547—Content authoring involving timestamps for synchronizing content
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
  • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
  • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device

Definitions

• the present invention relates to a marked audio-video stream, a method of generating such a stream, a method and an installation for analyzing and synchronizing such a stream.
• the present invention is in the field of visualization of media content, and more particularly audio-visual. State of the art
• the invention relates to the visualization of audiovisual contents including in particular a video signal and / or an audio signal and / or a textual element broadcast superimposed, such as a subtitle for example. It can be seen that supply and demand for such audio-visual content has grown exponentially in the last decade, thus helping to transform the television (and internet) panorama as well as the way in which the public views such content.
• Another aim of the invention is to solve at least one of these problems by a new audio-video stream which makes it possible to check the continuity and / or synchronism of the various textual, audio and video components.
• Another object of the present invention is to provide a device that facilitates the analysis of audio-video streams and to reduce costs. Another goal is to improve the quality and the rendering of broadcast audio-video streams.
• At least one of the above-mentioned objectives is achieved with a marked audio-video stream comprising (i) at least one video signal composed of at least one video frame, and (ii) at least one audio signal composed of at least one audio frame and / or (iii) at least one textual signal composed of at least one subtitle frame, and characterized in that the at least one video frame comprises means for identifying the video synchronism, and the at least one audio frame comprises means for identifying the audio synchronism and / or the at least one subtitle frame comprises means for identifying the text synchronism.
• the audio-video stream marked according to the invention comprises several signals of the same type (for example several subtitles and / or several broadcast languages ...) then the corresponding identifiers make it possible to distinguish each signal of the same category;
• the order in which the frames must be played may be contained in the synchronism identifier itself (using a numerical incrementation, for example) and / or in a database that gathers all the information concerning the audio-video stream Mark. It is therefore possible to know in what order each frame must be broadcast;
• An audio-video stream according to the invention may comprise several instances of each audio and / or text signal.
• an audio-video stream marked according to the invention, in the field of cinema may be composed of a video stream, of two different audio streams, corresponding for example to the dialogues in French and in English. , as well as five text streams corresponding to subtitles in different foreign languages.
• the video stream can also be composed of several video signals.
• the means of identification of the audio synchronism can be transported by a modulated carrier in agreement with said means of identification of the audio synchronism.
• the audio synchronism identification means can be a counter for identifying the instant of broadcasting the corresponding audio frame, thereby determining when - and in what order - the audio frames are broadcast.
• the means for identifying the video synchronism may comprise a unique identifier for indexing a database in order to be able to initiate comparisons with said database and to analyze the corresponding audio-video stream.
• the identifier can be of any type in order to identify - at least - a cell in a database. For example, it may be a pair of integers respectively describing a row and column number of said database, or a QR code which makes it possible to store more information, such as by example a counter.
• the means of identification of the video synchronism may further comprise a two-dimensional structure of predefined images in order to be able to detect the presence of possible artifacts by measuring the distortion of the diffused image compared to the predefined one and whose essential characteristics are recorded in the database. It can be a mosaic with particular images at particular positions. The characteristics concerning the images that constitute the mosaic and their positions are stored elsewhere in a database. Finally, the images may be different for each of the frames that make up the video stream.
• a decoder of a marked audio-video stream comprising (i) means for reading said at least one audio-video stream marked according to the invention, (ii) means for for extracting at least one video synchronism identifier from the at least one video signal of said marked audio-video stream, (iii) means for extracting the at least one audio synchronism identifier from the at least one audio signal of said stream labeled audio-video, (iv) means for extracting the at least one text synchronous identifier from the at least one text signal of said marked audio-video stream, (v) a database comprising on the one hand the decoding information the video synchronism identifier of the at least one video signal of said marked audio-video stream and the audio signal and / or the textual signal of the at least one audio-video stream marked, and (vi) an analysis means for comparing the extracted synchronism identifiers with the contents of the database.
• the analysis means make it possible, on the one hand, to verify that the at least one broadcast audio stream corresponds to the original audio signal, and on the other hand to check the possible presence of video artifacts such as the cuts or macroblocks by verifying that the images present on the identifier of the video synchronism are in conformity with those which have been used in the original signal and which are stored on the database. It is also possible to check that the text broadcast in closed captioning is in line with that of the original signal.
• the invention also relates to a method for generating a marked audio-video stream according to the invention and which can comprise at least one iteration of the following steps:
• a method for analyzing an audio-video stream marked using a processing unit to test (i) the continuity of the at least one video frame of the at least one video signal of said marked audio-video stream; and / or (ii) the continuity of the at least one audio frame of the at least one audio signal of said marked audio-video stream; and / or (iii) the continuity of the at least one subtitle frame of the at least one text signal of said marked audio-video stream.
• the processing unit may comprise a microprocessor or a microcontroller and may be integrated for example in audio-video stream reading equipment, such as for example a "set-top-box", a digital television or any other type of digital decoder.
• the analysis method according to the invention can further test the audio synchronism of the at least one audio frame with respect to the at least one video frame of said marked audio-video stream, and / or the text synchronism of the at least one subtitle frame with respect to the at least one video frame of said marked audio-video stream.
• the analysis method according to the invention can also test the quality of the at least one video frame of said marked audio-video stream.
• the analysis method according to the invention may comprise at least one iteration of the following steps:
• a step of acquiring the synchronism identifiers comprising:
• an artefact detection step comprising:
• the present invention intends to cover the methods of synchronization and / or correction of artifacts of the component signals of the audio-video streams, regardless of the number of signals component said streams.
• the present invention also extends to the synchronization of several audio-video marked streams, one of them being taken as reference.
• the analysis method according to the invention may comprise beforehand at the reading step a step of generating audio-video streams marked from a standard audio-video stream and comprising the following steps:
• FIG. 1 illustrates a principle diagram for the generation of a labeled audio-video stream according to the invention
• FIG. 2 illustrates a particular example of video synchronism identifier
• FIG. 3 illustrates a video stream marked according to the invention
• FIG. 4 illustrates a principle diagram for the analysis of the different artifacts of a labeled audio-video stream according to the invention as well as the detection of synchronization errors
• FIG. 5 illustrates an example of transmission of an audio frame counter by amplitude modulation
• FIGURE 6 illustrates the equivalent spectrum of the generated audio signal.
• FIG. 1 illustrates a principle diagram for the generation 100 of a labeled audio-video stream according to the invention.
• the general principle is to divide each signal composing the original audio-video stream 101 into frames on which are introduced markers which will make it possible to check and / or correct the continuity and / or the synchronism between the signals. different signals and / or correct reading artifacts that may sometimes appear during the broadcast of said signals.
• the method thus delivers a plurality of signals each composed of a plurality of marked frames, said signals comprising for example at least one video signal cut into frames and labeled 104, and / or at least one audio signal cut into frames and labeled 105, and or at least one textual signal cut into frames and marked 103.
• the rasterized, text-cut text signal may represent the subtitle band associated with the video signal, or any other textual information overlaid on the image broadcast by the media.
• the marking of the various signals that make up the audio-video stream marked includes a synchronism identifier, which may, however, contain more information than that necessary to check the continuity of the signals and synchronize the different signals.
• the marking of the video signal cut into frames may comprise a counter to serve as a time reference for the synchronization of the other signals of the marked audio-video stream; it may also contain a unique identifier for indexing a database 110 described below.
• the marking of the audio signal cut into frames can be carried out by amplitude modulation for example or by any other means making it possible to superimpose on the initial audio signal information making it possible to subsequently identify the synchronism of the audio signal with the video signal.
• the code superimposed on the original audio signal and transported on each frame includes a counter or an identifier for accessing such a counter.
• the marking of the textual signal cut into frames further comprises a counter to determine the time of appearance of said text frame.
• a database 110 is filled with all the information that will allow the analysis and subsequent decoding of the audio-video stream marked.
• the database may contain information about the original audio-video stream as well as the full description of the labeled audio-video stream generated by the present method, namely, and not limited to, the content and the relative position of each frame of each signal composing the original audio-video stream.
• the database 110 contains all the information concerning the video, audio and text signals cut into frames and marked. More particularly, the database may contain a video signal counter which makes it possible to identify the synchronism of the other supported signals (audio and textual) and all the information describing all the frames of the audio-video stream marked, to know :
• FIG. 2 illustrates a particular example of a video synchronism identifier comprising a two-dimensional structure 200 and for a given video frame. It is composed of eight predefined images 202 to 209 and different, positioned around an identification code 201 which indexes the database 110.
• the identification code is a QR code, well known to those skilled in the art.
• the present invention is not limited to this type of code but encompasses all the means capable of indexing a database. At a minimum, it may be a pair of integers that describe a row and column number of said database.
• the cell thus pointed in the database 110 contains the information corresponding to the video, audio and / or textual frame currently being broadcast.
• FIG. 3 schematically illustrates a labeled audio-video stream 106 according to the invention. In the example illustrated, the audio-video stream 106 is broken down into three distinct signals which have respectively been subsequently labeled and recombined:
• a video signal 104 composed of a succession of images 341 and 344 at one frequency.
• the image diffusion frequency is 25 frames per second, but this frequency can be higher in the case of high resolution filming, typically 50 to 60 images per second.
• a video synchronism identifier 342 and 345 is associated, comprising in the illustrated example a mosaic of predefined images 343 and 346.
• each frame is associated with a synchronism identifier.
• the audio frame 351 is associated with the identifier 352 and the audio frame 354 is associated with the identifier 355.
• Each identifier furthermore comprises a counter 353 and 356 which makes it possible to determine the instant of broadcast of said frame.
• a text signal 103 for broadcasting the subtitles that correspond to the video and audio frames.
• each frame is associated with a synchronism identifier.
• the text frame 361 is associated with the identifier 362 and the text frame 364 is associated with the identifier 365.
• Each identifier furthermore comprises a counter 363 and
• FIG. 4 illustrates a principle diagram for the analysis of the different artifacts of an audio-video stream marked according to the invention as well as the detection of synchronization errors. It includes the following steps:
• the acquisition of the synchronism identifiers 410 for each of the extracted signals namely at least one video signal 411 cut into frames and marked, at least one audio signal 412 cut into frames and marked, and / or at least one textual signal 413 cut into frames and marked.
• the present invention is not limited to a particular type of means for the acquisition of different identifiers.
• the acquisition of the video synchronism identifier can be performed using a camera or by screen capture; the acquisition of the audio synchronism identifier can be achieved using a microphone or an acquisition card; and the acquisition of the text synchronous identifier can be achieved using an optical character recognition software ...
• - decoding synchronism identifiers 420 for each of the signals composing the audio-video stream labeled 106.
• it may be for the video signal to decode the QR code; for the audio signal, it may be to demodulate the signal carrying the identifier; and for the textual signal, it may be to extract at least one alphanumeric code.
• the extracted video identifier makes it possible to index the database 110 which contains all the information relating to the audio-video stream such that it should ideally be broadcast.
• the information contained, for each frame, thus makes it possible to:
• the audio-video stream marked 450 is thus corrected and / or read optimally.
• the means for analyzing the audio-video streams marked according to the invention may consist of internal means by means of broadcasting said audio-video streams, such as the processor contained in a digital television or a digital decoder, for example, or using outsourced means such as a computer.
• the analysis of the flow obtained after having crossed the processing chain consists first of all in identifying the date of occurrence of each change of identifier of the video signal using the QR code for example and the database and in a second step, to detect the occurrence of the counters of the other signals: for the audio signal, it is for example to detect the beginning of each new audio frame.
• the comparison of dates for identical counter values then gives the offset value between the video and audio and / or textual signals as they are rendered by the broadcasting means and for each of the scanned frames.
• the present invention thus makes it possible to detect synchronism faults greater than the acceptability threshold (located between -185 ms for a delay of the audio signal on the video signal, and +90 ms for an advance of the audio signal on the video signal) with an accuracy of +/- 10 ms in the case of an analysis performed at 50 images per second.
• the audio-video stream is broadcast at 50 frames per second, and the analysis is performed every 500 ms, which means that every 25 frames of a video sequence, a synchronism identifier is inserted.
• the counter of the video frame is extracted during the decoding of the video identifier which marks each video frame. In the example shown, counter # 5204 was not extracted.
• the video clock corresponds to the date of change of the video identifier and represents the time reference for the other streams.
• the counter of the audio frame is extracted by demodulation of the signal contained in each of the audio frames.
• the audio clock corresponds to the detection date of the beginning of the audio frame.
• Audio frame # 5203 is in phase with the video frame.
• synchronization means can be integrated in the analysis means or constitute means in its own right. As non-limiting examples, they may consist of a digital television for example, a digital decoder or a computer.
• FIG. 5 illustrates an example of transmission of an audio frame counter by amplitude modulation
• the abscissa axis 501 representing the time in milliseconds
• the ordinate axis 502 the amplitude of the signal encoding the frame counter 504
• FIG. 6 illustrates the equivalent spectrum 600 of the audio signal generated on four carriers 603-606 and for which the amplitude 602 is represented as a function of the frequency 601.
• the counter or the identifier making it possible to access such a counter is encoded on a modulated carrier in accordance with the latter.
• the transmission of the counter or its identifier within the audio signal must not exceed 1 / Ft seconds, where Ft is the frequency of the video signal.
• Ft is the frequency of the video signal.
• the transmission of the code corresponding to the audio counter or its identifier must not exceed 20 ms.
• each transmission start of the value of the counter or its identifier starts with a specific value indicating a "start of frame". In the example illustrated in FIGURES 5 and 6, the start of the frame is materialized by a maximum in the amplitude of the signal.

Applications Claiming Priority (2)

Publications (1)

Family

ID=53191771

Family Applications (1)

Country Status (3)

Families Citing this family (1)

Citations (2)

Family Cites Families (2)

• 2014
  • 2014-12-19 FR FR1462869A patent/FR3030973B1/fr active Active
• 2015
  • 2015-12-17 WO PCT/EP2015/080253 patent/WO2016097165A1/fr active Application Filing
  • 2015-12-17 EP EP15826139.6A patent/EP3235253A1/de not_active Ceased

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events