FR2740897A1 - Audio and video processing method with copy protection - Google Patents

Abstract

The method involves dividing the audio and video data into a number of sample blocks. Each block of samples is subject to a time or frequency transformation to determine the spectral energy of the block. The transformed spectrum is processed to eliminate spectral lines according to predetermined psycho-sensorial criteria. Bits are allocated to the lines which have not been eliminated. Certain spectral lines are replaced by marking codes which are chosen so they are masked according to psycho-sensorial criteria. The spectral lines chosen for marking are determined according to a predetermined algorithm which is based on a secret pseudo-random sequence.

Description

Method and device for identifying audio and / or video data, without introducing Perceptible disturbances
The invention relates to the identification or marking of audio and / or video data.

It finds general application in securing audio and / or video data, and more particularly against their fraudulent reproduction or re-broadcasting.

Currently, for example using a tape recorder or a video recorder, it is relatively easy for an indelicate user to reproduce and / or repeat fraudulently (for example commercially) audio and / or video data for which he only paid a private use fee.

The Applicant has posed the problem of providing a simple means capable of effectively combating this practice, without causing perceptible disturbances in the audio and / or video data.

The present invention provides a solution to this problem.

It relates to a method for processing audio and / or video data in which said data is broken down into the form of a plurality of blocks of samples, said method comprising, for at least one block of samples, the following steps - a) performing a time / frequency transformation on the block of samples to be processed to determine the spectral energy of said block; -b) process the spectrum of the sample block thus transformed to eliminate the masked lines according to predetermined psychosensory criteria; and -c) allocate bits to the lines not eliminated.

According to a general definition of the invention, step b) further comprises the following step -bl) for at least some lines not eliminated, insert data forming fingerprints, chosen so that they are masked, according to said psychosensory criteria, in the lines not eliminated, which makes it possible to mark said audio and / or video data thus processed without introducing perceptible disturbances.

Such marking makes it possible to unambiguously identify the source of the data fraudulently recorded and / or rebroadcast, in particular that which simply paid the rights to acquire these data and which reproduced and / or rebroadcast these data fraudulently.

According to a preferred embodiment of the invention, step b1) comprises the following steps -bl1) choose at least one non-eliminated line from the non-eliminated lines, according to a predetermined selection algorithm, and -b12) for the line not eliminated thus chosen, insert a data item forming an imprint, chosen so that it is masked according to psychosensory criteria.

Preferably, the selection algorithm uses a predetermined pseudo-random sequence;
In practice, steps b11 and b12 are applied to each block of samples, which makes it possible to obtain an identification of all of the audio and / or video data to be processed.

According to a particular embodiment of the invention, the insertion of the data forming fingerprints corresponds to a power modulation of the line not eliminated and selected.

For example, the psychosensory criteria are of the acoustic and / or visual type.

The present invention also relates to a data processing device for the implementation of the method according to the invention.

Other characteristics and advantages of the invention will appear in the light of the detailed description below, and of the drawings in which - FIG. 1 schematically represents the essential and constitutive means of the device for marking audio and / or video data according to the invention; - Figure 2 is a flowchart illustrating the determination of the frame for inserting imprinted data into the signal to be processed according to the invention; - Figure 3 is a flowchart illustrating the assembly of data forming fingerprints in the signal to be processed according to the insertion frame according to the invention; - Figures 4A and 4B are schematic representations of the fingerprint signal and the insertion frame according to the invention; and - Figure 5 is a flowchart illustrating the extraction of fingerprint data according to the invention.

The device of FIG. 1 allows the marking of audio and / or video data, according to the flow diagram described with reference to FIG. 2.

The remainder of the description is based, without limitation, on the processing of digital data according to the ISO standard MPEG 1 or 2 with ISO for "International standard organization", MPEG for "Moving Picture Expert Group". Obviously, the data can be processed in the form of another digital standard. They can also be analog.

Under these conditions, the analog signal (step E3) is initially digitally converted according to an appropriate standard (step E4).

An input signal SE is applied to input 2 of a time / frequency transformation module 4.

For example, the input signal SE is sampled at a frequency of 48 kHz (ISO MPEG 1 standard). This sampling frequency here corresponds to that used for high fidelity sound.

The signal SE is cut beforehand (step E1) into N blocks of 1152 samples each.

The transformation module 4 applies a time / frequency transformation to the input signal (step E6).

For example, the transformation is of the transformation type of
Discrete Fourier (TFD) or even a modified discrete cosine transformation (TCDM). These transformations are well known to those skilled in the art.

The signal leaving the output 6 of the transformation module 4 is applied to the input 8 of a spectral line elimination module 10 (step E8). This module 10 is capable of calculating, according to selected psychosensory criteria, here the psycho-acoustic criteria of the ISO MPEG 1 standard, the parts of the masked spectrum which it is therefore unnecessary to code, and the parts of the spectrum to be coded.

More precisely, module 10 eliminates the inaudible spectral components in the transformed signal from module 4.

The elimination module 10 processes the spectrum of the block of samples thus transformed by the module 4 to eliminate the masked lines according to psychosensory or similar criteria.

The non-eliminated lines exiting from the output 12 of the module 10 are applied to the input 14 of a bit allocation module 16 which will encode these data while respecting the psychosensory criteria mentioned above.

It is in this processing chain that the means of the invention combine.

According to the invention, a data insertion module forming fingerprints 100 is provided for the lines not eliminated, that is to say those leaving the output 12 of the module 10.

Generally, the data forming fingerprints are chosen so that they are masked according to the psychosensory criteria mentioned above. This insertion must not introduce any disturbance into the output signal.

This result is obtained by precisely checking the psychosensory criteria to mask the fingerprints thus inserted.

Thus, the marking of the data is made almost inaudible, non-disturbing, that is to say that it does not statistically degrade the signal by using the psycho-acoustic properties mentioned above.

It should be noted that the fact of marking only the lines not eliminated, that is to say the lines having a level higher than the coding threshold, makes it possible to guarantee that this information can be extracted on yet another copy, on the condition Obviously, the copier does not modify the initial signal with degradation of the initial signal (for example deletion of part of the initial signal).

Referring to Figure 2, there is shown a flowchart illustrating the selection of non-eliminated lines capable of receiving the data forming fingerprints. This selection corresponds to the determination of a data insertion frame forming fingerprints.

For example, the selection algorithm includes the following criteria
If the block N of samples processed does not have any non-eliminated line that meets the psychosensory criteria for inserting a fingerprint (step E8), no data forming an imprint is inserted in this block N (step E9).

In the presence of a single non-eliminated line having or meeting the psycho-acoustic criteria (step E10), this block N is capable of being coded later with imprint data having the value of an "O" or a "1" (step E10).

On the other hand, in the presence of at least two non-eliminated lines responding favorably to the psycho-acoustic criteria (step E13), provision is made, for example, to make a selection of a single line among these non-eliminated lines (step E14). This single line is chosen according to a pseudo-random sequence (step E1S).

The value 0 or 1 of the fingerprint data can be chosen according to the following coding (step E72).

When the psycho-acoustic criterion is such that a noise less than or equal to 6 dB (for example) is masked, that is to say not coded, it is possible to use this characteristic to achieve a power modulation corresponding to " 1 "and" 0 "of the fingerprint data.

Thus, compared to the original value of the power of a line not eliminated to be marked, for example of a power equal to +72 dB compared to the low reference level, it is possible to assign a footprint datum of a value "1" by modulating the power of the line not eliminated at t78 dB (step E12), or of a value "O", by modulating the power of the line not eliminated at +66 dB (step E11 ).

Thus, this power modulation extends over a difference of 12 dB which is perfectly detectable, for those who know where to look for said modulation (i.e. the line not eliminated and selected according to the algorithm of the invention), but undetectable for those who do not know and above all inaudible since the "error" added or removed is masked (according to psychosensory criteria).

In summary, the marking process therefore comprises two main steps.

The first step called "determination of the insertion frame" (FIG. 2) consists first of all in determining for each block of the audio and / or video signal to be processed (steps E2 and E21), the positions eligible to contain the data forming an imprint taking into account the psychosensory properties of the signal to be processed (FIG. 4B).

It then consists in electing, among the eligible positions, the position chosen according to the criterion linked to the secret pseudo-random sequence when several lines are eligible in a block.

Finally, it consists in pre-calculating, for each block, the power modulation which corresponds, for the line not eliminated and selected, to an imprint of a value "0" or an imprint of value "1", in accordance with coding principle described above.

The second step called "assembly" consists in assembling the data thus forming a pre-calculated imprint later on to the original signal.

The fingerprint data (FIG. 4A) can correspond to the identity of the person viewing the audio and / or video data. This identity can be brought by any means, for example the identity stored in a smart card memory in the case of a television data decoder.

It should be noted that the operation for determining the insertion frame is carried out only once. Advantageously, this operation is carried out at the time of application of the psycho-acoustic model. With reference to FIG. 4B, the ineligible positions, that is to say the blocks having no non-eliminated lines, are hatched while the blocks having at least one non-eliminated line are left in white.

The imprint of value "0" or "1" is here materialized by a modulation of the power ae the spectral line not eliminated according to the psychosensory criteria and elected according to the criterion related to the pseudo-random sequence. This power modulation is performed "in addition" to symbolize the value "1" or "less" to symbolize the value "0", relative to the power of the original signal. This power modulation, which results in "noise" linked to the intentionally committed error, is such that it is no longer audible and / or visible (masked noise).

If necessary, synchronization information is contained in the fingerprint data so that the entire signal constituted by the useful signal and the fingerprint data is perfectly synchronous with a view in particular to simplifying the decoding by a user authorized to extract the data forming fingerprints.

Advantageously, the calculation of the insertion frame (FIG. 2) and the insertion of the data forming fingerprints (FIGS. 3, 4A and 4B) are dissociated, since the first operation is more complex and has the advantage of not being carried out. only once, preferably at the same time as the application of the psycho-acoustic model, that is to say at the coding level. On the other hand, the second operation can be carried out at the time of the consultation of the data, or more exactly at the level of the restitution of the data, which makes it possible to characterize the recipient.

It should be noted that it is to confer an additional degree of security on securing the data that it is provided according to the invention to choose, for each block, at least one non-eliminated line from the non-eliminated lines (step E71).

For the non-eliminated line thus chosen, a datum forming the imprint chosen is inserted (in accordance with the modulation described above) so that it is masked according to psychosensory criteria.

Such an insertion, extended to all the blocks of samples, thus makes it possible to make the marking undetectable for an unauthorized person insofar as the selection algorithm (insertion frame according to the pseudo-random sequence) forms a sort of secret key for inserting fingerprint data.

In the absence of this secret key (pseudo-random sequence) and by not knowing the modulation of the fingerprints, an unauthorized person is unable to detect the marking of the data.

Advantageously, the data forming fingerprints is moreover overcoded, for example so as to generate a spread spectrum or its equivalent in the time domain in order to make these fingerprints resistant to errors and to malicious scrambling attempts.

Overcoding results in coding of 1 bit per 24 ms frame, i.e. a useful bit rate of 40 bits / s (in the case of an ISO frame
MPEG 1 at 48 kHz). Over one or more minutes of a piece of music, the spreading factor can thus be 10 for approximately 50 useful alphanumeric characters.

It should be noted that the operations mentioned above apply perfectly to an encoder using a psycho-acoustic model.

In the case of an encoder not using a psychoacoustic model, it is also necessary to carry out a preprocessing which will inject in analog or digital the signal corresponding to the imprint to be inserted. This insertion is developed by following exactly the same principles as those described with reference to Figures 1 to 4B, with a calculation in accordance with the rules of a psycho-acoustic model included in the pre-treatment so as not to disturb, audibly and / or visual, the signal to be coded. The fingerprint data insertion frame is also determined according to the principles described above.

It should be noted that an encoder not using a psycho-acoustic model, used downstream of the pre-processing means in accordance with the invention, will not eliminate these data forming fingerprints thus inserted.

As a variant, it may be advantageous to insert the data forming fingerprints at the level of a data decoder.

Advantageously, this fingerprint data can be associated with identity information that cannot be falsified by the user (from a chip card for example or from a memory associated with a processor that cannot be read from the outside).

This variant thus makes it possible to identify the recipient of the legal copy transmitted. Any hacking from this copy thus makes it possible to go back to this user.

In this case, identification information is added to the decoded signal according to a process similar to that used when the coder does not use a psycho-acoustic model.

This method is distinguished, however, by the fact that the assembly operation is carried out this time after decoding, before transmission to the user.

It should be noted that the signal transmitted to the user is thus modified without causing any appreciable discomfort in terms of listening and / or sight.

For example, the fingerprint data thus inserted can characterize the identity of the source of the decoded data and the identity of the authorized decoder.

It should be noted that the fingerprint data cannot be detected if the original of the coded piece is not available and the secret frame (pseudo-random sequence) for inserting the fingerprint data.

A particular device can allow the extraction of the data thus inserted.

With reference to FIG. 5, this extraction device is capable of extracting the data forming fingerprints according to the following algorithm.

First, the original audio (and / or video) signal 50 is applied to the input 52 of a synchronization data extraction module 80.

The output 54 of the extraction module 80 delivers the original audio (and / or video) signal synchronized by processing the synchronization information to be applied to a digital processing device 90. When the original audio (and / or video) signal 50 has no synchronization, the output 56 is directly connected to the processing device 90.

Then, the audio (and / or video) signal to be identified 60 from the output 66 is applied to the processing device 90, when it does not include synchronization bits. Otherwise, in the presence of synchronization bits, these are processed by the extraction module 80 to synchronize the audio signal to be identified with the original signal.

Finally, the signal (or pseudo-random sequence) of fingerprint data insertion frame 70, from the output 76 is applied to the processing device 90, when it does not include synchronization bits. Otherwise, in the presence of synchronization bits, these are processed by the extraction module 80 to synchronize them with the original audio signal and the audio signal to be identified.

On the basis of the knowledge of the secret frame (pseudo-random sequence) 70, of the modulation of the power of the lines and of the original 50 of the signal (advantageously also secret), the processing device 90 is capable of extracting from the signal to identify 60, the data forming fingerprints. This extraction is obtained by conventional processing operations such as correlation, alignment, and time / frequency transformation.

In the presence of an over-encoding of the data to be processed, the output 92 of the processing device 90 is advantageously applied to a spectrum despreading module 97 which delivers at its output 98, the data forming fingerprints 99.

This fingerprint data 99 can be compared to presumed fingerprint data 95 according to a predetermined authentication algorithm.

It should be noted that the marking of the audio and / or video data according to the method according to the invention does not only modify in any way the listening or the view of said data but above all constitutes an electronic signature resistant to attempts at falsification or erasure insofar as the energy characterizing the imprint is "spread out and drowned" in the energy of the information to be protected, and that any falsification leads to a degradation of this audio and / or video information as it distorts its characteristics. Consequently, this signature can be advantageously applied to the protection of digitized audio and / or video works. This signature can thus be a solution to be adopted to reconcile the digitalization of works and the protective rules of copyright.

Claims (13)

Claims 1. A method for processing audio and / or video data in which the data is decomposed in the form of a plurality of blocks of samples, said method comprising, for at least one block of samples, the steps following: -a) perform a time / frequency transformation on the block of samples to be processed to determine the spectral energy of said block; -b) process the spectrum of the sample block thus transformed to eliminate the masked lines according to predetermined psychosensory criteria; and -c) allocate bits to the lines not eliminated; characterized in that step b) further comprises the following step: -bl) for at least certain non-eliminated lines, insert data forming imprints, chosen so that they are masked according to said psycho-sensory criteria in the lines not eliminated, which makes it possible to mark said audio and / or video data thus processed without introducing perceptible disturbances. 2. Method according to claim 1, characterized in that step b1) comprises the following steps: -bll) choosing at least one non-eliminated line from the non-eliminated lines, according to a predetermined selection algorithm, and -b12) for the non-eliminated line thus chosen, insert a datum forming the imprint chosen so that it is masked according to psychosensory criteria. 3. Method according to claim 2, characterized in that the selection algorithm uses a predetermined pseudo-random sequence. 4. Method according to claim 3, characterized in that the pseudo-random sequence is secret. 5. Method according to one of the preceding claims, characterized in that the psychosensory criteria are of acoustic and / or visual type. 6. Method according to claim 2, characterized in that steps bl1 and b12 are applied to each block of samples. 7. Method according to one of the preceding claims, characterized in that the insertion of the data forming fingerprints corresponds to a power modulation of the lines not eliminated and selected. 8. A device for processing audio and / or video data in which the data is broken down into the form of a plurality of N sample blocks, said device comprising, for each sample block: -a) transformation means (4) suitable for carrying out a time / frequency transformation on the block of samples to be processed in order to determine the spectral energy of said block; -b) processing means (10) capable of processing the spectrum of the block of samples thus transformed to eliminate the masked lines according to predetermined psychosensory criteria; and -c) allocation means (16) suitable for allocating bits to the non-eliminated lines; characterized in that it further comprises: - insertion means (iO0) suitable, for at least certain non-eliminated lines, for inserting data forming imprints, chosen so that they are masked according to said psycho-criteria sensory in the lines not eliminated, which makes it possible to mark said audio and / or video data thus processed without introducing perceptible disturbances. 9. Device according to claim 8, characterized in that the insertion means (100) comprise: - selection means suitable for choosing at least one non-eliminated line from the r.on lines eliminated according to a predetermined selection algorithm, and - own means, for the non-eliminated line thus chosen, to insert a datum forming the imprint chosen so that it is masked according to psychosensory criteria. 10. Device according to claim 9, characterized in that the selection means comprise a generator of a predetermined pseudo-random sequence. 11. Device according to claim 10, characterized in that the pseudo-random sequence is secret. 12. Device according to one of claims 8 to 11, characterized in that the psychosensory criteria are of acoustic and / or visual type. 13. Device according to one of claims 8 to 12, characterized in that it comprises means for modulating the power of the lines not eliminated and selected in order to insert therein the data forming fingerprints.