JP2000330579A - Method and device for inserting watermark into music information and its program recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make noise caused by inserting a watermark less conspicuous. SOLUTION: In the watermark insertion method, a watermark inserting position is randomly determined with respect to musical sound signals. If one of the binary watermark information is '1', the musical sounds from an insertion position Pn to Xmsec are duplicated and made into as insertion signals. If the information is '0', the musical sounds from Pn+X to Xmsec are duplicated and inserted to an insertion position Pn as insertion signals. Note that before inserting the signals, the musical sounds from the position Pn to Xmsec are duplicated, the sound pressure is increased for 5 dB to produce relative signals, the signals are delayed for Ymsec, that is an allowable delay for the perception characteristic, and the signals are inserted into the musical sounds. Thus, the insertion signals are suppressed hearingwise by the relative signals.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for distributing music information via a communication network such as a wide area computer network (Internet), for example, information for protecting the copyright of music information, a so-called digital watermark. And an apparatus for superimposing music information on music information.

[0002]

2. Description of the Related Art Conventionally, in a system for distributing music information via a communication network such as the Internet, there is a problem that music information is easily copied without permission, and various security information embedding methods have been devised. ing.
With respect to encoded music information, security data has been added as digital data to read the security data to prevent duplication and the like. However, if the encoded music information is decoded into audio waveform data, the added information disappears, and information without security data can be copied. Therefore, a method of embedding security data in audio waveform data without deteriorating the S / N ratio has been studied so that the security data is not lost even if the data is decoded.

[0003]

When embedding security data including a great deal of information in audio waveform data, the data is embedded in a frequency region that cannot be heard by human ears in order to reduce noise. That technique is taken. However, in most encodings, a method of reducing the amount of data by removing portions that cannot be heard by the human ear is employed, and thus embedded data may disappear.

[0004]

Therefore, according to the present invention, information to be embedded is embedded as noise, and a signal for suppressing the noise as sound is inserted to make the presence of the embedded data inaudible and prevent duplication. Therefore, when the music information is changed, it is possible to leave noise when the data is changed, and furthermore, by embedding the data into the left and right channels, the data is embedded so that it is difficult to decipher the correlation between the data embedding positions.

[0005]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The method of inserting a watermark into a multi-channel tone according to the present invention and the configuration of the apparatus are as follows. As shown in FIG. 1, the insertion information generation unit 1 receives as input the tone data converted into digital data and the attributes of the tone data, and generates information to be inserted into the tone data as a binary signal. The watermark information insertion unit 2 determines a position where the insertion information in the musical sound data is to be embedded, and inserts the insertion information of the binary signal as a watermark. The relative signal insertion unit 3 generates a signal that suppresses noise generated by the inserted signal as a sound, and inserts the signal into the musical sound data. The insertion signal is read by the watermark information decoding unit 4, and the read signal is restored to the attribute information by the attribute information reproducing unit 5.

More specifically, the attribute information generation unit 11 in the insertion information generation unit 1 rarely changes, for example, the right information, etc. from the input attribute information. The information Ie that is unique to music data, such as a composer and a copyright holder, that needs to be prevented as much as possible, and a high-speed readout process that requires a high update frequency, such as a distribution source, a user's usage status, and a distribution route, are required. And information Im that fluctuates depending on the use situation of the distributor, the user, and the like.

The insert code generator 12 converts the insert information generated by the attribute information generator 11 into binary data that can be inserted into musical tone data. Since the universal information Ie needs to be in a form that is difficult to decipher even if it takes time to generate information,
For example, binarization is performed using a unique binarization dictionary, a unique encoding method, or the like. On the other hand, the changing information Im
As for (2), it is necessary to reduce the amount of information and to be able to read data at high speed in consideration of adding a path (a path indicating the type of duplication performed) and quickly specifying the path. For this reason, for example, binarization is performed using a widely-common, high-compression rate, and low-processing amount such as Huffman coding.

[0008] As shown in FIG.
If the information insertion position has regularity, it is easy to extract and decode the information. Therefore, in order to find the difficulty, it is necessary to disperse the insertion position. Let it. Random numbers must maintain uniqueness with respect to this musical tone data, and in order to maintain the difficulty of decoding, the seed value and width from which random numbers are generated, the random number generation algorithm, and the information provider and information user Both share.

The time-series position determining unit 22 includes a random number generating unit 2
From the first value of the random numbers generated in step 1, it is determined whether the random numbers are arranged in ascending or descending order. To determine the time series point to insert, the second and subsequent generated random numbers are
A list is generated by using the total number of bits of the insertion data and arranging the random numbers in the order determined from the first value. Each value represents each time-series point representing the music data in units of seconds, and information is inserted at the corresponding time.

The insertion channel determining unit 23 determines whether to assign an even number or an odd number to the left (right) channel depending on whether the first value of the random number generated by the random number generating unit 21 is an even number or an odd number. The second and subsequent random numbers for determining the channel to be inserted are as follows for the total number of bits of the insertion data.
The even and odd numbers of the values are determined in the order of occurrence, and the channel to be inserted is determined.

The insertion frequency determination unit 24 selects a part which is predicted not to be deleted by encoding and which is difficult to perceptually determine. As an example, a part that does not enter the mask of the psychoacoustic model is considered as a part that is not deleted by encoding, and therefore a part that is not subjected to masking is selected. Furthermore, since a slight change is perceived in the part where the sound is continuously sustained, a frequency with a short duration is selected, and even if a small amount of reverberation remains after canceling noise, it is hard to perceive Processing such as selecting a portion where sound is concentrated on the frequency is performed, and a frequency band satisfying these three conditions is determined as the information insertion frequency.

In the insertion signal generation unit 25, if completely different types of signals such as genres are inserted as watermarks, it is difficult to sufficiently suppress the signals. Therefore, the time series position determination unit 22 is used as the insertion signal ISn. And a copy of the tone signal MS (n) near the insertion position Pn determined by the insertion channel determination unit 23. Insertion code generator 1
When one value of the insertion information In binarized by 2 is binarized to, for example, 1 and 0, and 1 is inserted at the insertion position, the insertion is canceled from the insertion position Pn as shown in FIG. A length X (unit: ms) that does not cause a difficult click sound
ec) Duplicate the previous tone signal. The minimum copy length X that is not perceived as a click sound is the average of the tone signals MS (Pn-S),..., MS (Pn) from the insertion position Pn to the reference signal length S (unit: msec) given as an initial value. The frequency Sp is calculated (251, FIG. 3) and approximated by the following equation (1).

[0013] In other words, the reference signal length S is a length necessary for analyzing the frequency of the musical tone signal. For example, the reference signal length S is set to 200 msec.
(252), if 0 < Sp < 1 KHz, X =
1875 / Sp + 10 msec (253), where Sp is 1 KH
If it exceeds z, X = 12 msec (254). It is checked whether or not the insertion information In to be inserted into the insertion position Pn is 1 (255). When 0 is inserted into the insertion position Pn, an insertion signal is embedded from the insertion position Pn as shown in FIG. After a time X, the signal ISn = M for the time X from the position Pn + X to the position Pn + 2X after that.
S (Pn + X),..., MS (Pn + 2X) are duplicated (256, see FIG. 3).

When the insertion information In is 1, as shown in FIG. 4, the signal ISn = MS (Pn-X),..., MS for the time X from the position Pn-X before the time X with respect to the insertion position Pn.
(Pn) is copied (257). Note that the above equation (1)
Is a numerical value selected according to the well-known auditory characteristic that, for example, at 1 kHz or more, when a sound arrives at one ear earlier by 12 milliseconds, the sound delayed later cannot be heard by the ear.

As described above, a duplicate signal corresponding to the inserted signal 1, 0 is obtained, and this is inserted as watermark information into the insertion position Pn of the tone signal as described later. , Ie, noise. In order to suppress this noise, a relative signal is created and inserted into a tone signal as described below. If the sound from one of the left and right channels precedes by about 1 to 30 msec, the sound of the other corresponding subsequent channel cannot be perceived, and the sound pressure of the other following channel becomes 5 dB.
The relative signal insertion unit uses the perceptual characteristics of the left and right ears of the human to hear the same phase, and the tone at the insertion position of the channel in which the insertion signal is inserted is converted to the subsequent data of the same channel. The noise is canceled by a method in which the sound pressure is increased and the synthesized sound is changed so as to be perceived. That is, as shown in FIG. 5A, when "1" is inserted, the musical tone of Xmsec immediately before the insertion position Pn is copied and inserted into the portion of Xmsec from the insertion position Pn as indicated by an arrow, but before the insertion. Duplicate the tone from Pn to Xmsec (taken out as indicated by the arrow), increase the sound pressure and insert it into the portion from Pn + Y to Xmsec as indicated by the arrow to suppress noise (synthesis) based on the insertion. I do.
Similarly, as shown in FIG. 5B, when inserting “0”,
A copy of the musical tone from Pn + X to Pn + 2X is inserted into the part from Pn to Xmsec as shown by the arrow, the part from Pn to Xmsec before the insertion is taken out as shown by the arrow, and the sound pressure is increased to increase the sound pressure from Pn + Y to Xmsec as shown by the arrow. Insert into the part.

The antiphase position determining section 31 acquires the time-series positions of the insertion positions determined by the watermark information inserting section 2 as shown in FIG. 6, generates a list of insertion channels, and perceives from the insertion positions. The position Yn shifted backward by the delay time Y (unit: msec) allowed by the characteristic is determined as a position where a relative signal for suppressing noise based on the inserted copy information is inserted. At this time, when the sound signal uses various sound sources having various frequencies and when the number of sound sources is small, the latter needs to have a shorter delay time to make it difficult to perceive. The time Y is obtained by the following equation.
The frequency band is divided in advance into H sub-bands Sbn having input values, and tone signals MS (Pn),..., MS from the insertion position Pn to the reference signal length S given as an initial value.
Fourier transform of order N is performed on (Pn + S) (3
1-1), when the result is made to correspond to the sub-band SBn, the number of sub-bands having the corresponding value is set to Hs. That is, i = 1, 2,..., N (Nth order) is initialized to 0, and the number Hs of subbands in which the spectrum exists is initialized to 0 (31-2), and it is determined whether i <N (31-
3) If i <N, it is determined whether the spectrum Fn (i) of i is in any of the subbands. If (31-4), Hs is incremented by 1 (31-5), and i is further increased. The value returns to step 31-3, and if it is not in the sub-band in step 31-4, the process proceeds to step 31-6. If i is equal to N in step 31-3, Hs / H is 1/10 or less. Is determined (31-7).

The minimum delay Lmin is 1 ≦ Lmin, for example, 2 ms
ec, the maximum allowable delay Lmax is Lmax ≦ 30 and Lmin <L
max, for example, set as an initial value at 30 msec.
If s / H is 1/10 or less, Y = Lmin (31−
8) If Hs / H is not less than 1/10, Y = Lmax ×
(Hs / H) (31-9). Let Pn + Y be Yn (31-10). That is, Becomes

The relative signal generator 32 first converts the tone signal at the position determined by the time-sequence position determiner 22 and the insertion channel determiner 23 into an MS (P
n),..., MS (Pn + X). Furthermore, sound pressure α sufficient for the delayed signal to be perceived in the same phase as the preceding signal
(Unit: dB), and increase the sound pressure of the duplicated signal by α. From the perceptual characteristics, the sound pressure level α has a correlation with the delay time Y, but in this embodiment, the delay time Y is 30 msec or less from the equation (2), so α is a constant 5. Thus, the relative signal DS 'is generated.

The insertion signal insertion unit 33 converts the signal ISn generated by the insertion signal generation unit 25 into the time series position determination unit 22.
, And MS (Pn + X) at the positions determined by the insertion channel determination unit 23. The relative signal insertion unit 34 converts the relative signal DS ′ generated by the relative signal generation unit 32 into the tone signals MS (Yn),..., MS (Yn + X) at the position Yn determined by the antiphase position determination unit 31.
To be synthesized.

As shown in FIG. 7, the watermark information decrypting section 4
The random number generation unit 41 generates a random number using the same method as the random number generation unit 21 that generates a random number when inserting watermark information.
A time-series read position detector 42, a read channel detector 43,
The read frequency detector 44 similarly performs the same processing as the time series position determiner 22, the insertion channel determiner 23, and the insert frequency determiner 24 when watermark information is inserted, and specifies the information insertion position. The information reading unit 45 reads the insertion information from the insertion position of the information specified by the processing up to the reading frequency detection unit 44. The read insertion information is converted by the binary information reproducing unit 46 into binary information. For example, in the above example, when the information is "1", the tone signal between Pn-X and Pn is duplicated and inserted into the specified information insertion position Pn. When a correlation between the signal and the read insertion information is obtained, a large correlation value is obtained when the information is "1", and a small correlation value is obtained when the information is "0". Using this, the read insertion information is converted into binary information.

As shown in FIG. 8, the attribute information reproducing unit 5 uses the universal information Ie and the variation information Im,
It determines which information is requested, and issues a reproduction instruction to the attribute information decoding unit 52. The attribute information decoding unit 52 decodes the information requested by the request information selection unit 51. When the universal information Ie is requested and the encoding is performed using the unique dictionary, the encoding is performed using the decoding dictionary. When the unique encoding is used, the encoding is decoded and the attribute information is reproduced.

In the above description, the present invention is applied to a case where watermark information is inserted into a stereo tone signal. However, the present invention can also be applied to a case where watermark information is inserted into a monaural music signal. Similarly, the present invention is applied to a tone signal having three or more channels. It is also applicable when inserting. In short, the present invention is to make noise based on the insertion of watermark information suppressed as sound by inserting a relative signal.

The functions of the respective parts described above can also be applied by causing a computer to decode and execute a program.

[0024]

According to the present invention, for example, it is possible to embed right information for preventing illegal duplication or the like into music information, and to insert a relative signal for suppressing noise due to the embedded information, thereby deteriorating music information. To prevent duplication due to noise caused by the relative signal inserted to suppress when music information is changed,
Since the embedded data can be left even after the copy, the security of the distribution of the music information can be ensured.

Furthermore, since the amount of information does not change by embedding data and it is difficult to falsify, it is possible to embed and distribute various information at the same download time, and to provide additional information to the user. It becomes possible.

[Brief description of the drawings]

FIG. 1A is a block diagram showing the overall configuration of a method for inserting a watermark into musical sound information according to the present invention, and FIG.
It is a figure showing the flow of.

FIG. 2 is a diagram showing a flow of processing of a watermark information insertion unit 2 of FIG. 1A.

FIG. 3 is a view showing a flow of processing of an insertion signal generation unit 25 in FIG. 2;

FIG. 4 is a diagram showing how an insertion signal is generated.

FIG. 5 is a diagram showing a state of each insertion of an insertion signal and a relative signal.

FIG. 6 is a diagram showing a flow of processing of a relative signal insertion unit 3 in FIG. 1A.

FIG. 7 is a diagram showing a flow of processing of a watermark information decryption unit 4 in FIG. 1A.

FIG. 8 is a diagram showing a flow of processing of an attribute information reproducing unit 5 in FIG. 1A.

Continuing on the front page (72) Inventor Junki Tomioka 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Japan Telegraph and Telephone Corporation (72) Inventor Kazuhiro Sugiyama 3-192-2 Nishishinjuku, Shinjuku-ku, Tokyo Japan F-term in Telegraph and Telephone Corporation (reference) 5B017 AA06 BA07 BB03 CA16 5B082 GA01 GA02 GC05 5D044 AB05 BC01 BC04 CC04 DE50 GK17 5J064 AA01 CA01 CC07

Claims (15)

[Claims] A step of generating watermark information from information to be inserted; a step of determining a watermark information insertion position for music information; a step of inserting the watermark information at the watermark information insertion position; Generating a relative signal that suppresses noise generated in the music information as sound based on the insertion of the watermark information; determining a relative signal insertion position at which the relative signal is inserted into the music information; Inserting the relative signal into a position. 2. The step of generating the relative signal includes the step of extracting the music information at the watermark information insertion position, and the step of obtaining the relative signal by increasing the sound pressure level of the extracted music information. The method of determining the relative signal insertion position is a process of finding a position delayed by a delay time allowed by perceptual characteristics with respect to the watermark information insertion position and setting the position as the relative signal insertion position. 1. A method for inserting a watermark into the music information described in 1. 3. The step of obtaining the position delayed by the delay time includes the step of frequency-analyzing the music information at the watermark information insertion position, and the ratio of the frequency present as a result of the analysis to the entire frequency band of the music information. 3. A method for inserting a watermark into music information according to claim 2, comprising the step of determining the delay time according to the following. 4. The step of generating the watermark information includes the step of converting the information to be inserted into binary information, and the step of converting the music immediately before the watermark information insertion position into one of the binary information. 4. The method according to claim 1, further comprising extracting information, extracting the music information slightly after the watermark information insertion position as the other of the binary information, and using the music information as the watermark information. Watermark insertion method to music information described in. 5. The music information includes a plurality of channels, and the watermark information insertion position determining step is a step of determining a watermark information insertion position by randomly distributing the watermark information insertion positions to the plurality of channels. Item 6. A method for inserting a watermark into music information according to any one of Items 1 to 4. 6. A means for generating watermark information from information to be inserted, a means for determining a watermark information insertion position for music information, a means for inserting the watermark information at the watermark information insertion position, Means for generating a relative signal for suppressing noise generated in the music information as sound based on the insertion of the watermark information; means for determining a relative signal insertion position for inserting the relative signal with respect to the music information; Means for inserting the relative signal at the insertion position. 7. The means for generating the relative signal comprises means for extracting the music information at the watermark information insertion position, and means for obtaining the relative signal by increasing the sound pressure level of the extracted music information. The relative signal insertion position determining means is a means for obtaining a position delayed by a delay time allowed by perceptual characteristics with respect to the watermark information insertion position and setting the position as the relative signal insertion position. 6. A watermark insertion device for music information described in 6. 8. The means for obtaining the position delayed by the delay time comprises: means for frequency-analyzing the music information at the watermark information insertion position; and the ratio of the frequency present as a result of the analysis to the entire frequency band of the music information. 8. The apparatus for inserting a watermark into music information according to claim 7, further comprising means for determining the delay time according to the following. 9. The watermark information generating means includes means for converting the information to be inserted into binary information, and as one of the binary information, the music information immediately before the watermark information insertion position. 9. A means for extracting information and extracting the music information slightly after the watermark information insertion position as the other of the binary information and using the extracted music information as the watermark information. A watermark insertion device for music information described in 1. 10. The music information comprises a plurality of channels, and the watermark information insertion position determining means is means for randomly determining the watermark information insertion position among the plurality of channels. Item 10. An apparatus for inserting a watermark into music information according to any one of Items 6 to 9. 11. A process for generating watermark information from information to be inserted, a process for determining a watermark information insertion position for music information, a process for inserting the watermark information at the watermark information insertion position, A process of generating a relative signal for suppressing noise generated in the music information as a sound based on the insertion of the watermark information, a process of determining a relative signal insertion position at which the relative signal is inserted into the music information, and a process of inserting the relative signal A recording medium recording a program for causing a computer of a device for inserting a watermark into music information to perform a process of inserting the relative signal into a position. 12. The relative signal generation process includes a process of extracting the music information at the watermark information insertion position, and a process of obtaining the relative signal by increasing the extracted music information by a predetermined amount. 12. The relative signal insertion position determining process is a process of obtaining a position delayed by a delay time allowed by perceptual characteristics with respect to the watermark information insertion position and setting the position as the relative signal insertion position. Recording medium. 13. A process for obtaining the position delayed by the delay time includes a process for frequency-analyzing the music information at the watermark information insertion position, and a ratio of a frequency present as a result of the analysis to the entire frequency band of the music information. 13. The recording medium according to claim 12, further comprising a process of determining the delay time according to the following. 14. A process for generating the watermark information, the process of converting the information to be inserted into binary information, and the process of converting the music immediately before the watermark information insertion position into one of the binary information. 14. A process according to claim 11, further comprising extracting information, extracting the music information just after the watermark information insertion position as the other of the binary information, and setting the music information as the watermark information. A recording medium according to claim 1. 15. The music information includes a plurality of channels, and the watermark information insertion position determination process is a process of randomly distributing and determining the watermark information insertion position among the plurality of channels. Items 11 to 14
The recording medium according to any one of the above.