EP1894199A1 - Dispositif et procede pour determiner dans un film un emplacement qui presente des informations de film appliquees selon une suite temporelle - Google Patents

Dispositif et procede pour determiner dans un film un emplacement qui presente des informations de film appliquees selon une suite temporelle

Info

Publication number
EP1894199A1
EP1894199A1 EP06754259A EP06754259A EP1894199A1 EP 1894199 A1 EP1894199 A1 EP 1894199A1 EP 06754259 A EP06754259 A EP 06754259A EP 06754259 A EP06754259 A EP 06754259A EP 1894199 A1 EP1894199 A1 EP 1894199A1
Authority
EP
European Patent Office
Prior art keywords
film
signal
time
test
fingerprint representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP06754259A
Other languages
German (de)
English (en)
Inventor
Thomas Sporer
Michael Beckinger
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP1894199A1 publication Critical patent/EP1894199A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B31/00—Associated working of cameras or projectors with sound-recording or sound-reproducing means
    • G03B31/04—Associated working of cameras or projectors with sound-recording or sound-reproducing means in which sound track is not on, but is synchronised with, a moving-picture film
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel

Definitions

  • the present invention relates to an apparatus and a method for determining a location in a film having film information applied in a temporal sequence to synchronize, for example, film events with image reproduction.
  • Audio-video data is on data carriers, e.g. Film or tape, or transmission channels, e.g. Radio or telephone, stored in a fixed format, which can be extended to include new audio formats or other synchronous additional services, such as Subtitles, not allowed. For example, when new audio formats are introduced, new media or movie copies must be produced that have the new audio formats.
  • data carriers e.g. Film or tape
  • transmission channels e.g. Radio or telephone
  • Fig. 8 shows an exemplary film 110.
  • Film information is applied to the film in spatial sequence, or during playback, respectively, in chronological order, e.g. Video information or images 112, which are also referred to as "frames" in the English language, and audio information or one or a plurality of analog or digital audio tracks 114 which are used in the digital case "audio frames".
  • the film 110 has, by way of example, feed perforations 116 with the aid of which the film is played back.
  • the time code is then used to play audio or additional information synchronously from an external data carrier, for example CD in DTS.
  • DTS Digital Theater System
  • mp3 Digital Theater System
  • the time code is then used to play audio or additional information synchronously from an external data carrier, for example CD in DTS.
  • each additional format requires more space on the disk or transmission channel, which may not be available anymore. In the case of film, these are, for example, the tracks for analogue sound, Dolby Digital, DTS, SDDS (
  • the second method is based on the misuse of analog audio tracks for storing time code as described e.g. in a prototype cinema equipped with an IOSONO system.
  • a disadvantage of this method is that the analog track is present in all systems and is often used as a fallback solution in case of disturbances of other systems, that is, an alienation of the analog track prevents the fallback possibility.
  • the automatic switching to the analogue track which is built into most cinemas, causes the timecode to be played back as an analogue signal if no signal is present on the "more modern" Dolby Digital or DTS tracks in a pure wave field synthesis playback, which will be explained below, the redundant analog playback are turned off manually, because otherwise the time code can be heard on the redundant other speakers.
  • WFS Acoustic Wave Field Synthesis
  • the air vibrations of a real situation which make up the sound, are tried over a whole To recreate space.
  • the wave field synthesis is intended to transmit the entire sound field true to the original to the room. This means that the virtual sound sources can be exactly spatially localized, and possibly even seem to exist in the middle of the sounded room, thus they can be bypassed.
  • Systems with up to 200 loudspeakers in cinema systems and up to 900 loudspeakers in theater sound systems have already been implemented.
  • Wave field synthesis is based on the Huygens principle, which states that any point on a wavefront can be considered as the starting point for an elementary spherical wave. By interference of all elementary waves, a new wavefront arises, which is identical to the original wave.
  • the cinema Ilmenau in which the wave field synthesis is operated in two modes.
  • the cinema is operated as a "real" wave field synthesis system, where the analog track of the 35 mm film stores the time code, as explained above with regard to the second "abusive" method, and the WFS sound of an external medium, eg Hard disk or DVD, is leaked.
  • an external medium eg Hard disk or DVD
  • the sound stored on each 35 mm movie is taken from a Dolby processor, alternatively DTS or SDDS could also be used be read, read and decoded, the Dolby processor may also switch automatically to the analog track, and maps the resulting multi-channel signal via WFS on virtual speakers.
  • a Dolby processor alternatively DTS or SDDS could also be used be read, read and decoded, the Dolby processor may also switch automatically to the analog track, and maps the resulting multi-channel signal via WFS on virtual speakers.
  • a disadvantage of the prior art described above is that the synchronization and timing within the film, as described in the publication, is limited to a search window of, for example, 1 minute. Especially in the initial phase of the film, however, it is difficult to define the right window for a successful synchronization. If the portion read or scanned by the film is not in the portion of the stored movie information used for the synchronization, the synchronization will fail or the synchronization will be erroneous. The cinema audience or movie viewer then hears no or a wrong sound to the film.
  • the object of the present invention is to provide an efficient concept for determining a location in a film.
  • This object is achieved by a device for determining a position in a film according to claim 1, a method for determining a position in a film according to claim 20 and by a computer program according to claim 21.
  • the present invention is based on the finding that each location of a film generally has film information specific to that location, so that in a feature extraction different locations of a film have different, specific characteristics of the features. In other words, different places in a movie have different "fingerprints" on. These fingerprints can in turn be used to find a location in a movie.
  • an apparatus for determining a location in a film having film information applied in a temporal sequence comprising: a memory for storing a reference fingerprint representation (FAD) of the movie information, wherein the fingerprint representation is arranged such that a temporal Characteristics of the fingerprint representation depend on a time course of the film information, wherein a stored reference fingerprint representation is associated with a time scale, means for receiving a portion read from the film, means for extracting a test fingerprint representation from the read portion, and means for comparing the test fingerprint representation the reference fingerprint representation to determine the location in the film based on the comparison and time scale.
  • FAD reference fingerprint representation
  • the device and method for determining a location in a film make it possible to determine anywhere in a film at any time without having to prepare or change the film itself.
  • the relevant time information, the time scale is stored along with a saved version of the movie.
  • the film is stored in the form of a reference fingerprint representation, which corresponds to a feature extraction.
  • Preferred embodiments also have the advantage, with a suitable choice of the fingerprint representation, to enable an unambiguous determination of the position.
  • the device and the method for determining a position in a film can be used, for example, in a device for generating a control signal for a film event system be used that synchronizes movie events with a picture playback.
  • movie events include audio, subtitles, and special effects, with special effects such as air currents, jiggling on the cinema chairs, smells, or light effects on the side and back walls.
  • special effects such as air currents, jiggling on the cinema chairs, smells, or light effects on the side and back walls.
  • Different audio languages, such as simultaneous playback of the original version and translations into other languages, as well as different audio techniques, such as the synchronization of digital surround sound, are possible with regard to the audio event.
  • the invention is not limited to movies for a moviegoer, but generally refers to films or audio-video signals, regardless of whether these are films or other data carriers and storage media, eg Magnetic tapes or hard drives, stored movie information.
  • the invention can also be used for pure sound systems without video or, for example, by means of a video ID also for the synchronization of pure video material, i. without sound, to be used with any events.
  • Fig. 1 is a schematic block diagram of a Favor ⁇ th embodiment of an apparatus for Generating a control signal for a movie event system
  • FIG. 2a shows a basic block diagram of an embodiment of an apparatus for performing a correlation
  • Fig. 2b is a schematic block diagram of a preferred embodiment of a device for performing a correlation
  • Fig. 2c.1 an exemplary portion of a film
  • FIG. 2c.2 shows an exemplary course of a tone signal of the section of the film shown in FIG. 2c.1 at a variable, first playback speed and a constant test sample rate;
  • FIG. 2 c shows an exemplary profile of a sound signal of the section of the film shown in FIG. 2 c 1 at a variable, second playback speed and a constant test sampling rate;
  • FIG. 2c shows an exemplary course of a sound signal of the section of the film shown in FIG. 2c.1 at a variable, third playback speed and a constant test sample rate;
  • Fig. 2d.l two exemplary sections of a film
  • Fig. 2d.2 an exemplary course of a reference sound signal of the film
  • FIG. 2d shows an exemplary course of a test tone signal, based on a first playback speed and a constant test scan rate, for a section of the film
  • FIG. 2d.4 an exemplary first correlation result from the correlation of the reference sound signal according to FIG. 2d.2 and the test sound signal according to FIG. 2d.3;
  • Fig. 2d.5 two exemplary sections of a film according to Fig.2d.l;
  • FIG. 2d.6 shows an exemplary course of a reference sound signal of the film according to FIG. 2d.2;
  • Fig. 2d.7 shows an example of a test tone signal based on a second playback speed and a constant test sample rate for a portion of the film
  • 2d.8 shows an exemplary second correlation result from the correlation of the reference sound signal according to FIG. 2d.6 and the test sound signal according to FIG. 2d.7;
  • 3a is a schematic block diagram of a preferred embodiment of a device for determining a location in a film by means of a fingerprint representation
  • Fig. 3b.1 shows two sections of a film
  • 3b.2 shows an exemplary course of the reference sound signal for the two sections according to FIG. 3b.1;
  • Fig. 4 is a schematic block diagram of a preferred embodiment of a device for detecting a location in a film by means of a coarse and a subsequent fine determination of the location;
  • Fig. 5a is a schematic block diagram of a preferred embodiment of an apparatus for Generating a control signal for a movie event system
  • Fig. 5b.1 two sections of a film
  • 5b.2 shows an exemplary course of a reference sound signal for a first section of the film
  • Fig. 5b.3 shows an exemplary course of a test sound signal for a second portion of the film
  • 6a is a schematic block diagram of an exemplary motion picture display system including an apparatus for generating a control signal for a motion picture event system and a motion picture event system;
  • FIG. 6b is a schematic block diagram of an exemplary motion picture display system including an apparatus for generating a control signal with an exemplary audio movie event system;
  • FIG. 7 shows a schematic illustration of an exemplary assignment of a time scale to a film information
  • Fig. 8 is a schematic representation of an exemplary film with applied film information.
  • the device for generating a control signal comprises a device 120 for storing the film information, means for receiving a portion read from the film 140, means for comparing the read portion with the stored movie information 112, 114, and means 180 for determining the control signal based on the comparison and the time scale.
  • One advantage of digitized storage lies in the simple and, above all, error-free duplication of the stored image of the film information.
  • the film remains unchanged as previously described, producing only a stored image of the movie information once, e.g. in the production of the film.
  • a film event system is controlled which, based on the control signal 190, is synchronous with the playing film 110, for example WFS. Sounds or subtitles generated.
  • the means 250 for comparing is adapted to compare the modified test sound signal 272 with the unmodified reference sound signal 274, no reference sampling rate converter 232 is necessary or the apparatus for performing a correlation according to FIG 2b does not include a reference sample rate converter 232.
  • a comparing means 250 configured to compare the unmodified test sound signal 270 with the modified reference sound signal 246 does not include a variable sampler 234.
  • T 2 - Ti (L 2 - L 1 ) / v.
  • the film segment between Li and L 2 or Ti and T 2 is subdivided into n time segments or represented by n + 1 sample values, for example:
  • T 0 defines, for example, the time on the time scale, which is assigned to the point Lo
  • the time Ti defines the time on the time scale that defines the point Li
  • the time T 2 defines the time on the time scale
  • the point L 2 defines the time on the time scale associated with the point L 3 on the film.
  • FIG. 2d.3 represents a currently read film information or test tone signal 270 applied to the film
  • FIG. 2d.2 represents a stored film information or a reference sound signal
  • the memory playback speed and the memory scan rate at which the reference sound signal was generated coincided with the playback speed of the test sound signal and the sampling rate of the test sound signal, and the quotient of memory sampling rate f memory and memory playback speed as previously shown v Spe i ch it with the quotient of the sampling rate f for the test tone signal and the playback speed of the test tone signal v match.
  • the reference sound signal or a portion of the reference sound signal defined by Ti and T 2 , may exactly match the test sound signal representing the portion between Ti and T 2 , more specifically the sample value sequences thereof, and a clear local one by correlation Maximum or a correlation peak are obtained, as shown by way of example in Fig. 2d.4.
  • Fig. 2d.6 unchanged test sample rate f, however, a changed, reduced playback speed v 'of the test sound signal.
  • the playback speed of the test tone signal will not only vary between different movie players, but may vary during a movie. Accurate tracking is essential to ensure synchrony throughout an entire movie.
  • the means for performing a correlation varies the sampling rate of the test sound signal or the sampling rate of the reference sound signal to detect the adverse effect of a variable sampling rate of the test sound signal as described above according to the above-described condition that the quotient of sampling rate and playback speed of the Test sound signal and the reference sound signal must be equal to minimize, to represent the same film section with the same samples.
  • FIG 2d.l -. 2d.8 illustrate simplified examples in which the clarity sake was assumed that the memory playback speed v Spe Icher a normal or usual playback speed of a playback device corresponding to generate a test tone signal.
  • the quotient of sample rate f and playback speed v is the magnitude that must be the same for the reference sound signal and the test sound signal to represent the same portion of the film with the same samples as previously indicated.
  • a double playback speed can also be used if the sampling rate is doubled at the same time.
  • the means 210 for determining may determine a measure of a test playing speed based on the result 278 of the correlation.
  • At least two different reference sound signals are compared to the test sound signal to compare the results of the correlation, for example by means of a quality assessment, which will be explained in more detail with reference to FIG from these a most similar reference sound signal and thus based on the known sampling rate and the known memory playback speed to determine a measure of the playback speed of the test sound signal.
  • the different reference sound signals can be formed one after the other and compared with the test sound signal or simultaneously formed and compared.
  • Apparatus for performing a correlation produces three reference tone signals based on different reference sample rates, the reference tone signal of the middle of the three sample rates being based on the reference sample rate of the reference tone signal having the best quality or maximum match with the test tone signal in a previous comparison, and the two other reference sound signals each have a reference sampling rate higher or lower than the reference sampling rate of the mean reference sound signal and reference sampling rate, respectively.
  • This is controlled by the means 230 for varying on the basis of an output of the means 210 for determining the measure of the test playing speed. This ensures that the reference sampling rate or the reference playback speed of the reference sound signal is matched to the playback speed or reference sampling rate of the test tone signal.
  • the device for determining a position in a film has a memory 320 for storing a reference fingerprint representation of the film information, wherein the fingerprint representation is designed so that a time course of the fingerprint representation depends on a temporal course of the film information, and wherein a stored reference fingerprint representation is a Time scale, having means 340 for receiving a portion read from the film, means 350 for extracting a test fingerprint representation from the read portion, and means 34 for comparing the test fingerprint representation with the reference fingerprint representation, on the basis of the comparison and the time scale to determine the location in the movie.
  • the fingerprint representation comprises a representation in the form of a spectral flatness, wherein a time profile of the fingerprint representation comprises a temporal profile of the spectral flatness.
  • Fig. 3b.1 shows an exemplary film 110 as shown in Fig. 8.
  • a position of the film when playing the film at a given playback speed corresponds to the time Tioo of the time scale, the point L 103 to the time T 103 of the time scale, the point L 113 , the time Tn 3 of the time scale and the Place L1 1 6 the time Tue the time scale.
  • FIG. 3b.2 shows a first section comprising the section from the point L ⁇ O o to Ln 3 and Ti 00 to Ti 13 , respectively, and a second section comprising the section of FIG the point L ⁇ O 3 up to the point Lu 3 or from the time T ⁇ o3 to the time Tn 6 includes. Based on these sections, a fingerprint associated with this section is created based on, for example, spectral analysis, Fourier transformation, or other feature extraction methods.
  • Sections the higher the memory requirement for the reference signal or the request for the processing power in the signal processing.
  • a significant advantage of the fingerprint representation in the form of the spectral flatness is its small memory requirement compared to, for example, a complete storage of the power density spectrum for a same section.
  • a trace of spectral flatness is used as a fingerprint for a portion.
  • Figure 4a shows an exemplary film 110, as shown in Figure 8, and a device for detecting a location in a film having film information applied in a temporal sequence.
  • the embodiment of the device for detecting a position in a film shown in FIG. 4a can be used, for example, in a device for generating a control signal for a film event system, as shown for example in FIG. 1, as device 180 for determining the control signal.
  • the means for determining a location includes a memory 420 for storing film information deposited on a film in sequence, with a time scale associated with the stored movie information, means 440 for receiving a portion read from the film, and synchronization means 460 configured to compare a sequence of samples of the read portion underlying a first sampling rate and a first search window of the stored film information to obtain a coarse result, and a sequence of samples of the read portion; which is based on a second sampling rate and a second search window of the stored film information to obtain a fine result indicative of the location of the film, wherein a position of the second search window in the stored film information depends on the coarse result, and wherein the first Search window time l I is longer than the second search window, and further wherein the first sample rate is lower than the second sample rate.
  • An input of the first means 562 for correlation, an input of a second means 564 for correlation, and an input of the third means 566 for correlation are coupled to an output of a sample rate converter 232, referred to as a sample rate converter (SRC). connected.
  • SRC sample rate converter
  • An output of the first means 562 for correlation, an output of the second means 564 for correlation and an output of the third means 566 for correlation are connected to an input of a first means 568 for quality assessment.
  • the quality assessment device 568 in turn is coupled to the sample rate converter 232 and to a sampler selection means 570, an output of the sampler selection means 570 being connected to an input of a timer 582.
  • the timer 582 in turn is stored with the An audio track device 522 is connected to an input of the sample rate converter 232 for storing the audio track.
  • An output of the first feature extractor 552 is connected to an input of a feature comparison device 554 having, for example, a feature classifier and a database of features, an output of the feature comparison device 554 having an input of the timer 582 is connected.
  • a feature comparison device 554 having, for example, a feature classifier and a database of features
  • An output of the timer 582 is coupled to an input of a time code generation means 584 having a time code database or coupled to a time code database, and an output of the time code generation means 584 is connected to an input of a time code slicer 586, the means 586 is adapted to output a time code 592, and wherein an output of the time code smoothing means 586 is connected to an input of a word clock generator 588, which in turn is adapted to output a word clock signal 594.
  • the apparatus for generating a control signal for a film event system optionally further comprises a second film sound sampler 542 'connected to a second A / D converter 544', the second A / D converter 544 'having a second feature extractor 552', with a fourth means 562 'for correlation with a fourth reference sound signal based on the first sampling rate, with a fifth means 564' for correlation with a fifth reference sound signal based on the second sampling rate, and with a sixth means 566 'for a correlation with a sixth reference sound signal connected at the third sampling rate.
  • An output of the fourth means 562 'for correlation, an output of the fifth means 564' for a correlation and an output of the sixth means 566 'for a correlation are connected to an input of a second means 568' for quality evaluation, wherein an output of the second Further, means 568 'for quality evaluation is connected to an offset compensation 569 and another output is connected to an input of the sample rate converter 232, and further wherein the means for offset compensation 569 is connected to the sample selection 570.
  • the first film tone sampler 542 also referred to as the main sampler, is positioned so that there is enough time for the device to generate a control signal to lock up.
  • the first film tone sampler 542 thus provides a pre-delayed signal.
  • the correlation window width or width of the portion of the test tone signal is added to the synchronization time. Based on the perforations on the film roll, the time difference for the pre-delay can be set exactly. As a first clue, three seconds is recommended.
  • the first film sound sampler 542 reads the sound signal from the soundtrack of the film and samples the sound signal from the soundtrack of the film, and passes this signal on to the first A / D converter 544, the first A / D converter 544 being extended. is formed to generate a digital audio signal or test tone signal based on the sampling rate of the first film sound sampler 542 and the playback speed of the film from which the soundtrack or movie information is read.
  • test fingerprint representation On the basis of the test sound signal 270, one or a plurality of features is extracted or a test fingerprint representation is formed. For example, the spectral flatness is used as a characteristic or fingerprint for the feature extraction or fingerprint representation.
  • the test fingerprint representation is then compared by the feature comparison device 554 with a reference fingerprint representation, as previously noted, wherein the fingerprint representation is such that a time history of the fingerprint representation depends on a temporal history of the movie information, and where a A reference fingerprint representation stored in feature 554 is associated with a time scale, and means 554 for comparing is adapted to determine a location in the film based on the comparison of the test fingerprint representation with the reference fingerprint representation and the time scale respectively a time code signal 554Z produce.
  • the sample rate converter based on the stored reference tone signal 274, generates the same signal at slightly different sample rates, i. modified reference tone signals for the correlations to be calculated in parallel.
  • modified reference tone signals for the correlations to be calculated in parallel.
  • the sampling rate converter 232 generates three reference sound signals 276 or modified reference sound signals. Signals 276, wherein a first reference sound signal based on a first sampling rate and the first means 562 for correlation is supplied, wherein a second reference sound signal 276 based on a second sampling rate and the second means 564 for correlation is supplied, and a third reference sound signal 276 on based on a third sampling rate and supplied to a third means 566 for correlation. Sample rate converter 232 provides low-level, sample rate-different signals to the correlation
  • Means 562, 564, 566 for correlation wherein the sampling rate is always set in response to the previous measured maximum peak-to-noise value from the correlation.
  • a correlation gets a modified reference tone signal with this sampling rate
  • another correlation gets a slightly lower, one level lower, and another correlation gets a slightly higher graduated sampling rate. This ensures that the sample rate converter can, for example, tune or synchronize to a change in the speed of the analog audio signal.
  • the means 522 for storing the soundtrack and the sampling rate converter 232 are preferably designed to use a window width of 2 n, in order to calculate low-cost large correlation windows by means of the fast Fourier transformation (FFT). In parallel, more than three correlations can be calculated to compensate for sudden jumps in the soundtrack.
  • the correlation window is chosen to be large in order to obtain a clear correlation peak. In order to obtain the recognition accuracy of the correlation peak under a sample or a sampling period, oversampling of the input signal or test tone signal can be performed.
  • the means 522 for storing the sound track are in response to the supplied time code signal 582Z of the time bers 582 the reference sound signal in the length of the correlation window, wherein the correlation window is the search window in which the test sound signal is searched.
  • the first means 568 for quality assessment is designed to perform a maximum value search in the cross-correlated signals or the amounts of the signals and to weight the quality of the cross-correlated, depending on the height of the correlation peak compared to other peaks in the cross-correlated or to determine the quality of each individual correlation on the basis of the peak-to-noise distance.
  • the best quality reference tone is determined and the displacement of the peak from the search window is determined based on the position of the peak of the best quality reference tone, for example as a time code difference between the measured and currently valid time code or relative time code issued.
  • the first quality assessment device 568 sends a control signal 568A to the sampling rate converter 232, which for example distinguishes only the three signal values "0", “+1” and "-1", for example "0".
  • the sampling rates of the last sample rate conversion or correlation are maintained, since the correlation result from the modified reference tone signal with the average sampling rate has been determined to be the highest quality, at "+1” the sampling rates are increased by one more level than the last sample rate conversion or correlation because the correlation result from the modified reference tone signal having the highest sampling rate was determined to be the highest quality, and at -1 the sample rates are reduced from the previous sample rate conversion by one step, since the correlation is from the test tone signal and the modified reference sound signal with the lowest reference sample rate had the best correlation result or peak-to-noise ratio.
  • the sample rate converter will be e.g. is increased or decreased by one sample rate delta value, or so driven that it does not sample rate conversion.
  • the correlation serves to address two important aspects. First, determining the location in the film or determining the point in time in the film on the basis of the time code difference from the correlation. Second, determining the measure of the playback speed to determine the optimum reference sample rate or sample rate conversion of the reference sample rate, respectively.
  • the adaptation of the sampling rates or the recapture of adapted playback speeds in turn enables better correlation results and thus in turn improves the timing or determination of the location in the film and thus in turn improves the synchronization and the prediction.
  • a preferred exemplary embodiment according to FIG. 5 is designed, by means of a signal analysis, to detect signal parts with specific characteristics in order to then hide them during synchronization and thus to prevent false detections or synchronizations or to avoid random fluctuations of the time axis.
  • Such characteristics may be, for example, the loudness of the signal part or the "problem" of a signal and the signal analysis or detection of problematic parts on the basis of SNR (signal-to-noise ratio), PNR (peak-to-noise), spectral power or power density spectrum, spectral flatness or averaging a time ⁇ union based sequence.
  • SNR signal-to-noise ratio
  • PNR peak-to-noise
  • spectral power or power density spectrum spectral flatness or averaging a time ⁇ union based sequence.
  • the quality of correlations with quiet signal parts is lower than with correlations with loud signals because of the higher quantization noise in the digital sampling, therefore quiet signal parts are suppressed by means of threshold values or adaptively, to avoid random fluctuations of the time axis.
  • the signal energy can be another quality feature.
  • Another example is the hiding of problematic, because recurrent signal parts in order to avoid ambiguity and thus, for example, incorrect synchronization.
  • Problematic signal parts or sections can also be signaled as metadata, for example, in order to hide these signal parts, regardless of the quality of the current correlation.
  • the time code generation means 584 is designed to convert based on the time code signal 582Z of the timer 582, which may for example be based on an internal or proprietary time code, into a standardized time code or a time code signal based on a standardized time code, for example.
  • the timer 582 is controlled by an internal clock
  • a coarse audio ID fingerprint or fingerprint representation for example, the time code signal 554Z from the feature determination tion or fingerprint representation
  • the determined correlation difference for example, the time-code difference signal 570Z determined from the correlation of the device 570 for picker selection.
  • the timer must prioritize correlation signal (highest priority), time code from feature determination, and internal clock (lowest priority).
  • the time code smoothing means 586 is arranged to smooth the time code signal 584Z so as to avoid, for example, a hopping time code or, if there are time codes from the correlation, to find meaningful intermediate values, e.g. Compensate for pauses in the analog tone.
  • the time code signal 592 generated by the time code slicer 586 is preferably a standard time code with which the movie event system is synchronized. However, the time code signal 592 can also be used to generate a corresponding sample clock or sample clock via a very slowly regulating phase-locked loop (PLL) if the enclosed audio reproduction system is of a digital type. Such phase locked loops are available as finished devices and are not the subject of this patent.
  • PLL phase-locked loop
  • more than one telecine with time varying offset from the projection lens may be used to improve the robustness of film damage or portions that are poorly tuned for synchronization.
  • a second film tone sampler 542 ' may then be used, for example, since the second film tone sampler 542' is already present in conventional cinema systems. Breaks in the analogue tone can hereby be bridged by the film tone samplers 542, 542 'attached at different locations on the motion picture film, since the probability increases with short pauses in the film tone, the at least one scanner, the first film sound scanner 542 or the second film sound scanner 542', provides enough signal for a correlation and the associated synchronization.
  • different scanners e.g. for analogue sound, Dolby Digital sound (including decoder), DTS digital sound (including DTS decoder) or another sound and a combination of the above may be used as the reference soundtrack and / or test soundtrack.
  • Dolby Digital sound including decoder
  • DTS digital sound including DTS decoder
  • another sound and a combination of the above may be used as the reference soundtrack and / or test soundtrack.
  • individual tracks can be used for comparison using averaging, majority decision or prioritization, automatically or via metadata, the time information generated therefrom, as well as a downmix to mono.
  • different scanners may be used for different audio formats and / or different film scanners with different timing offsets.
  • Using a downmix on mono has the advantage that when the monaural track is used as a stored audio track, it saves less than storing five channels, for example.
  • the storage of different, that is, more than one soundtrack, ie no downmix means that all channels are stored independently of each other and then, for example, as explained above, corresponding comparisons or majority decisions are performed, then the synchronization using a particular channel, the actual soundtrack and a corresponding channel of the stored soundtrack.
  • the initialization phase or the first synchronization and the resynchronization after a recording pause form two critical phases during a film projection or a synchronization of a film event system.
  • Preferred embodiments therefore initially calculate more than three parallel correlations, since no synchronization has yet occurred, that is, more than three reference tone signals of different sampling rates are compared with the test tone signal in order to obtain the correct sampling rate or playback speed of the test tone signal as quickly as possible determine. It is also possible to try different sampling rates in succession until one of the correlations has the best signal-to-noise ratio.
  • the first feature extractor 552 and the feature classifier 554 provide a coarse absolute in conjunction with the database
  • a preferred embodiment is therefore designed, in a first step, to determine a rough determination of a location in a film by displaying a longer section of the film by means of a reference sampling signal with a lower sampling rate, and also a test tone signal by sampling. obtained at a lower sampling rate. Based on the coarse location in the film, in a second step, a higher sample rate reference tone signal and a higher sample rate test tone signal are used to finely determine the location in the film.
  • the window length is adjusted when correlated.
  • time-long windows but a reduced sampling rate of the signals are used, if a time should be approximately found and only tracked, short windows may even be used with oversampling of the signals to achieve a higher temporal accuracy.
  • a "compatible reproduction” of the "old" audio format can take place until the exact position is determined.
  • a "compatible" playback of the "old” audio format can be done if the synchronization has been lost significantly until the exact position is determined again.
  • the scanner selection means 570 and the offset compensation means 569 are necessary only in embodiments with more than one film sound scanner.
  • the scanner selection means 570 decides whether the result or time code difference of the first quality judging means 568 (568Z) or the result or time code difference 568Z 'of the second quality judging means 568' is sent to the job setting determiner 582 in the movie or a time code 582Z. Since the second film tone sampler 542 'scans the test tone signal at another location on the film, the offset between the location where the first film tone sampler 542 scans the film to where the second film tone sampler 542' scans the film by means 569 for offset compensation. so that the timer 582 obtains the correct time code difference 570Z regardless of whether the time code difference 568Z or the time code difference 568Z 'is selected with respect to the most recently stored or last stored location of the movie stored in the timer.
  • the different reference sound signals of different reference sampling rates can also be generated one after the other and compared with the test sound signal in order to determine the measure of the playback speed of the test sound signal or the optimum reference sampling rate.
  • more than three modified reference sound signals may be compared to the test sound signal, in parallel or serially, to allow not only early synchronization fast sync but also during movie screening, the film event system for large cracks in the film, e.g. caused by cuts or in the film missing sections, faster to resynchronize to the current location in the film.
  • a synchronization of a film event system can also take place on the basis of the images applied to the film, both for an evaluation of features or fingerprints and for a correlation of a test image signal with one or a plurality of references - picture signals.
  • the correlation of audio and / or video signals for determining the temporal location in an audio and / or video stream can be used, and a synchronous playback can be controlled on the basis of this timing determination.
  • an audio and / or video signature from the raw material in the form of an audio ID / Video ID Identification
  • ID Identification
  • the basic idea of the invention is, for example, to digitally store the already existing analog tone, in order to then synchronize it with the analog audio track on the motion picture film by means of correlation and other feature determination.
  • the output signal or control signal of the device for generating a control signal or of the synchronizing device can be any time code format.
  • a data set for the device for generating a control signal or for the synchronization device must be created during production.
  • the data carrier includes the digitized analog audio track, e.g. in dolby stereo format as found on the roll of film, feature data about the soundtrack and matching timecodes.
  • a reference sound signal 274 is read from the device 522 for storing a sound track and a modified reference sound signal according to FIG. 5b.2 is generated by the device for sample rate conversion 232, which contains a film section from the point L 0 up to the point L 3 or the point of time To assigned to the point L 0 or a corresponding representing the time code T and the time L 3 associated with the point L 3 or time code.
  • FIG. 5b.4 shows the result of the correlation of the modified reference sound signal according to FIG. 5b.2 and the section of the test sound signal FIG. 5b.3.
  • Time code difference or the relative time code is formed.
  • the timer 582 knows the last absolute time or absolute time code and only requires the time code difference 570Z to obtain the updated absolute time or time code to determine.
  • the difference can be represented, for example, from the position of the peak with respect to the time of the beginning of the search window.
  • the peak is, for example, the fourth sample, ie the test signal from FIG. 5b.3 is shifted by "3 • ⁇ t" relative to the reference sound signal from FIG. 5b.2, where ⁇ t is the Ab ⁇ corresponding to the modified sampling rate is the tasting period.
  • the advantage of the adapted to the variable playback speed of the test sound signal sampling rate or playback speed of the reference tone signal advantageous to wear, since the .DELTA.t is adapted to the playback speed, a more accurate determination of the location in the film or displacement relative to the search window is possible than at a fixed sampling rate of the reference sound signal, since then only multiples of this sampling rate are generated for a determination of the location in the film.
  • the time To of the search window or reference sound signal can be equal to the Ti of the previous correlation, since the film is played only forward.
  • FIG. 6a shows an embodiment of a film system in which a device 100 for generating a control signal 190 is coupled to a movie event system 600, thereby generating the device 100 for generating a control signal based on the film 110, as shown in FIG that the control signal 190, for example a time code, is synchronized with the movie event system 600.
  • FIG. 6b shows a film system comprising a device 100 for generating a control signal 100 and a wave field synthesis system 610 as an exemplary film event system, the exemplary embodiment of the wave field synthesis system 610 comprising a device 620 for controlling the wave field synthesis system, a digital memory 622 for the wave field synthesis audio signals and a plurality of loud speakers 624 for the wave field synthesis system.
  • the control signal generating device 100 Based on the film 110 or, for example, an analog movie soundtrack 114, the control signal generating device 100 generates the control signal 190 to lip-sync a wave analog audio audio experience to an originally analog-converted movie.
  • other audio systems for example digital audio systems or digital surround audio systems, can of course also be synchronized lip-synchronously by means of the device 100 for generating a control signal.
  • FIG. 7 shows an exemplary film as shown in FIG. 8, an exemplary digitally stored reference sound signal 720, and an assignment of a time scale.
  • the analog audio signal is sampled at a given playback speed and rate, for example, 44.1 kHz, and audio portions of, for example, 10 ms are stored as a so-called audio frame, that is, the digital reference sound signal is present on the memory as a result of audio frames.
  • the assigned time of a time scale can then be, for example, to number the audio frames from 0 or 1 in ascending order as time code or time scale, time code TCl corresponds to audio frame AFI in FIG. 7, or, for example, the start time or end time of one To find audio frames as time code, such as for the first audio frame either 0 ms or 10 ms if an audio frame has a duration of 10 ms.
  • Time codes usually have formats such as hour: minute: second: frame, whereby the frame usually refers to video frames with, for example, 24 frames per second (cinema film).
  • a time scale or time code can therefore, for example, assign a plurality of audio frames to a video frame or define an audio frame as the smallest time scale unit. Accordingly, the time code or the time scale can then for example assign 4 audio frames to a time code, see TCl 'in FIG. 7, which comprises four audio frames AFI -AF4, or assign a single Audi frame to a time code, see TCl in Fig. 7, which is associated with an audio frame AFI.
  • the audio Frames also represent temporally overlapping sections of the audio signal.
  • the control signal 190 may be formed, for example, as a time code, but also as a sequence of pulses, where, for example, each pulse corresponds to a time scale unit and, similar to a relative time code, the film event system accumulates the pulses to synchronize with the film.
  • a further exemplary embodiment in order, furthermore, to have available, for example, an analog audio signal as a fallback, but at the same time also to realize a time code for synchronous additional services, offers the approach of embedding a watermark in the audio and / or video signal.
  • Advantage of this solution is that even with "difficult" audio signals, such as very quiet passages or even similar "monotonous" noises, a clean clock recovery is possible.
  • the complete set of the relevant Watermark claims in particular in the field of searching for the correct clock rate or the readjustment of the sampling rate, makes sense.
  • the decisive disadvantage of this approach is that the actual film has to be changed or a new version or copy of the film has to be created in order to be able to embed the watermarks in the audio and / or video signal.
  • the method according to the invention can be implemented in hardware or in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmable computer system such that the method is performed.
  • the inventions thus fertil in a computer program product with a program stored on a machine-readable carrier, the program code for performing the inventive procedural ⁇ Rens, when the computer program product on a computer expires.
  • the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

La présente invention concerne un dispositif pour déterminer dans un film (110) un emplacement qui présente des informations de film (112, 114) appliquées selon une suite temporelle. Le dispositif de l'invention comprend: une mémoire (320) pour enregistrer une représentation d'empreinte digitale de référence des informations de film (112, 114), la représentation d'empreinte digitale étant telle qu'une allure dans le temps de la représentation d'empreinte digitale, dépend d'une allure dans le temps des informations de film, et une échelle de temps étant associée à une représentation d'empreinte digitale de référence enregistrée; un système (340) pour recevoir une section lue du film (110); un système (350) pour extraire une représentation d'empreinte digitale d'essai issue de la section lue; et un système (360) pour comparer la représentation d'empreinte digitale d'essai avec la représentation d'empreinte digitale de référence, afin de déterminer l'emplacement dans le film (110) à partir de la comparaison et de l'échelle de temps.
EP06754259A 2005-06-22 2006-06-09 Dispositif et procede pour determiner dans un film un emplacement qui presente des informations de film appliquees selon une suite temporelle Withdrawn EP1894199A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE102005028978 2005-06-22
DE102005045628A DE102005045628B3 (de) 2005-06-22 2005-09-23 Vorrichtung und Verfahren zum Ermitteln einer Stelle in einem Film, der in einer zeitlichen Folge aufgebrachte Filminformationen aufweist
PCT/EP2006/005553 WO2006136300A1 (fr) 2005-06-22 2006-06-09 Dispositif et procede pour determiner dans un film un emplacement qui presente des informations de film appliquees selon une suite temporelle

Publications (1)

Publication Number Publication Date
EP1894199A1 true EP1894199A1 (fr) 2008-03-05

Family

ID=36716607

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06754259A Withdrawn EP1894199A1 (fr) 2005-06-22 2006-06-09 Dispositif et procede pour determiner dans un film un emplacement qui presente des informations de film appliquees selon une suite temporelle

Country Status (4)

Country Link
EP (1) EP1894199A1 (fr)
JP (1) JP5137826B2 (fr)
DE (1) DE102005045628B3 (fr)
WO (1) WO2006136300A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101290621B (zh) * 2007-04-17 2011-06-15 上海申瑞电力科技股份有限公司 安全数字卡的存储检索方法
WO2012078142A1 (fr) 2010-12-07 2012-06-14 Empire Technology Development Llc Différences d'empreintes digitales audio pour la mesure de la qualité d'expérience de bout en bout
JP2013178216A (ja) * 2012-02-28 2013-09-09 Koichi Ono タイムコード履歴更新型ラウドネスメータ

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1519363A1 (fr) * 2001-05-25 2005-03-30 Dolby Laboratories Licensing Corporation Méthode d'alignement temporel de signaux utilisant des caractérisations basées sur des évenements auditifs

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5040081A (en) * 1986-09-23 1991-08-13 Mccutchen David Audiovisual synchronization signal generator using audio signature comparison
US5055939A (en) * 1987-12-15 1991-10-08 Karamon John J Method system & apparatus for synchronizing an auxiliary sound source containing multiple language channels with motion picture film video tape or other picture source containing a sound track
EP0693215B1 (fr) * 1993-01-08 1998-11-25 Arthur D. Little Enterprises, Inc. Procede et appareil permettant l'elimination de sequences publicitaires televisees
IL109649A (en) * 1994-05-12 1997-03-18 Electro Optics Ind Ltd Movie processing system
JPH1020420A (ja) * 1996-06-25 1998-01-23 Sony Cinema Prod Corp 映画フィルム
DE10134471C2 (de) * 2001-02-28 2003-05-22 Fraunhofer Ges Forschung Verfahren und Vorrichtung zum Charakterisieren eines Signals und Verfahren und Vorrichtung zum Erzeugen eines indexierten Signals
US20050022252A1 (en) * 2002-06-04 2005-01-27 Tong Shen System for multimedia recognition, analysis, and indexing, using text, audio, and digital video
EP2782337A3 (fr) * 2002-10-15 2014-11-26 Verance Corporation Système de suivi de media, de gestion et d'information
EP1593272B1 (fr) * 2003-02-14 2017-08-30 Thomson Licensing DTV Synchronisation automatique de services media de contenus media fondes sur des fichiers audio et des fichier video
DE10322722B4 (de) * 2003-05-20 2005-11-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Synchronisieren eines Audiossignals mit einem Film
ATE373389T1 (de) * 2003-07-25 2007-09-15 Koninkl Philips Electronics Nv Verfahren und einrichtung zur erzeugung und erkennung von fingerabdrücken zur synchronisierung von audio und video

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1519363A1 (fr) * 2001-05-25 2005-03-30 Dolby Laboratories Licensing Corporation Méthode d'alignement temporel de signaux utilisant des caractérisations basées sur des évenements auditifs

Also Published As

Publication number Publication date
WO2006136300A1 (fr) 2006-12-28
DE102005045628B3 (de) 2007-01-11
JP2008547145A (ja) 2008-12-25
JP5137826B2 (ja) 2013-02-06

Similar Documents

Publication Publication Date Title
EP1627391B1 (fr) Systeme et procede pour synchroniser un signal audio avec un film
EP1794564B1 (fr) Dispositif et procede pour synchroniser des donnees supplementaires et des donnees de base
US5040081A (en) Audiovisual synchronization signal generator using audio signature comparison
CN101218648B (zh) 用于确定具有胶片信息的胶片上的位置的设备和方法
EP1968043B1 (fr) Méthode de détection de section de composition musicale et dispositif correspondant, méthode d enregistrement de données et dispositif correspondant
US8205148B1 (en) Methods and apparatus for temporal alignment of media
DE69734430T2 (de) Informationsaufzeichnung und -wiedergabe
DE69924765T2 (de) Vorrichtung zur Erzeugung von Daten zur Wiederfindung von Videodaten und Vorrichtung zur Wiederfindung von Videodaten
DE3783602T2 (de) Ersatztonband.
EP2573773A2 (fr) Alignement de temps des signaux audio enregistrés
EP1894198B1 (fr) Dispositif et procede pour realiser une correlation entre un signal de son d'essai qui peut etre produit avec une vitesse variable, et un signal de son de reference
DE69528091T2 (de) Verfahren und Vorrichtung zum Aufzeichnen und Wiedergeben von komprimierten Videodaten
DE102005045628B3 (de) Vorrichtung und Verfahren zum Ermitteln einer Stelle in einem Film, der in einer zeitlichen Folge aufgebrachte Filminformationen aufweist
DE69422974T2 (de) System und methode zur wiedergabe von mehreren digitalen videoprogrammen
DE69922193T2 (de) Verfahren und vorrichtung zur dynamischen synchronisation einer digitalen tonspur mit einem film in einem filmmedium
EP1872368B1 (fr) Dispositif et procede pour detecter un emplacement dans un film
DE102005035216A1 (de) Playback-Vorrichtung zur Synchronisation eines Playback eines Video-Signalanteils und eines Audio-Signalanteils eines AV-Signals und Verfahren dafür
WO2000021290A2 (fr) Reduction d'une partie indesirable d'un programme diffuse
DE102017131266A1 (de) Verfahren zum Einspielen von Zusatzinformationen zu einer Liveübertragung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20071214

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20080312

RBV Designated contracting states (corrected)

Designated state(s): DE FR GB

RIN1 Information on inventor provided before grant (corrected)

Inventor name: BECKINGER, MICHAEL

Inventor name: SPORER, THOMAS

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20121128