JP3835370B2 - Watermark data embedding device and computer program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a watermark data embedding device that embeds watermark data in sound source waveform data that represents a sound source waveform and is used to generate a musical tone signal, and a computer program used to embed the watermark data.
[0002]
[Prior art]
Conventionally, watermark data such as identification information has been embedded in all or part of the waveform data for the sound source in order to find an illegal copy of the waveform data for the sound source representing the waveform for the sound source. In this conventional watermark data embedding, the watermark data is embedded in the same manner with respect to the waveform data for the sound source of the embedded portion, that is, the sampling data of the embedded portion.
[0003]
[Problems to be solved by the invention]
Embedding watermark data is effective for finding unauthorized duplication of waveform data for sound sources as described above, but embedding watermark data in waveform data for sound sources means changing the contents of waveform data for sound sources. To do. Due to the nature of the waveform data for sound source, there are a part that is greatly affected by the musical sound signal that is generated when the amount of watermark data embedded is increased, and a part that is not so. If embedding is used, there is a problem that the sound quality of the generated musical sound signal deteriorates.
[0004]
SUMMARY OF THE INVENTION
The present invention has been made to address the above-described problems, and an object of the present invention is to provide a watermark that can efficiently embed watermark data in sound source waveform data without impairing the sound quality of the generated musical sound signal. A data embedding device and a computer program used for embedding the watermark data are provided.
[0005]
  In order to achieve the above object of the present invention, a feature of the present invention resides in a watermark data embedding apparatus that embeds watermark data in sound source waveform data that represents a sound source waveform and is used to generate a musical tone signal.SoundPosition on the time axis of the sound source waveform represented by the source waveform dataA data amount determining means for determining a data amount of watermark data to be embedded in the waveform data for sound source, which is different depending on the characteristics of the musical sound by the waveform data for sound source at each position, and on the time axis Of the amount of data determined for each locationWatermark data, Corresponding to the position on the time axisEmbed in sound source waveform dataWith embedding meansThere is.
[0006]
  The data embedding device of the present invention isFor example,further,Dividing the sound source waveform represented by the sound source waveform data into multiple sections on the time axisDividing means for performing the data amount determining means,The divided sectionEach can be configured to determine the data amount of the watermark data to be embedded in the sound source waveform data, which is a different data amount according to the characteristics of the musical sound by the sound source waveform data in each section. Also,For example, waveform data for sound sourceTimeDivided into multiple sections on theThe data amount determining means isThe divided sectionEach time, the data amount of the watermark data to be embedded in the sound source waveform data may be determined, which is a different data amount according to the characteristics of the musical sound by the sound source waveform data in each section. Furthermore, the present invention can be implemented as a computer program for data embedding.
[0007]
  Also,the aboveIn the present invention,In order to vary the amount of watermark data embedded in the waveform data for sound sources,For example, the frequency of embedding watermark data in a series of sampling data constituting the sound source waveform data is varied, or the amount (number of bits) of watermark data embedded in each sampling data is varied.Good.Alternatively, a plurality of types of watermark information consisting of a plurality of types of character strings having different lengths are prepared, and watermark data representing a long character string or watermark data representing a short character string is selectively employed. You may do it. In short, any aspect may be adopted as long as the amount of watermark data to be embedded is appropriately set according to the section.
[0008]
  According to the present invention configured as described above, the position on the time axis of the waveform for the sound source.RestAndA different amount of data according to the characteristics of the musical sound by the waveform data for the sound source at each position.Watermark data can be embedded in sound source waveform data. Specifically, characteristic parts of the sound source waveform (for example, the peak part of the attack, the decay part immediately after the attack), the part of the low volume level, etc., with large changes in tone (overtone structure), volume level, etc. In, the frequency of embedding watermark data is lowered or the amount of embedding of watermark data is reduced. In addition, in a stable portion of a sound source waveform with a medium volume level and small changes in timbre (overtone structure), volume level, etc., the watermark data embedding frequency is increased or the watermark data embedding amount is increased. To do. As a result, according to the present invention, the watermark data can be efficiently embedded in the sound source waveform data without deteriorating the sound quality of the generated musical sound signal.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an electronic apparatus to which an apparatus and a program according to the present invention are applied. As this electronic device, various devices including an electronic musical instrument, a sequencer, a personal computer, etc. can be considered.
[0010]
This electronic device includes a control group 11 and a display 12. The operator group 11 is arranged on the operation panel, and is operated by the user to instruct various operations of the electronic device. The operation of the operator group 11 is detected by turning on / off an operator switch provided corresponding to each operator in the detection circuit 13 connected to the bus 20. The display device 12 is arranged on the operation panel or independently of the operation panel, and is composed of a liquid crystal display, a cathode ray tube device, or the like, and displays characters, symbols, figures, and the like. The display mode of the display 12 is controlled by a display control circuit 14 connected to the bus 20.
[0011]
A CPU 31, a timer 32, a ROM 33, a RAM 34, an external storage device 35, and a sound source circuit 41 are connected to the bus 20. The CPU 31, the timer 32, the ROM 33, and the RAM 34 constitute a microcomputer body, and control various operations of the electronic device by executing programs. The external storage device 35 includes an external recording medium such as a hard disk HD incorporated in advance in the electronic device, a flexible disk FD removable from the electronic device, and a compact disk CD. The external storage device 35 also includes a drive unit that writes and reads data and programs to and from the external recording medium. In particular, in the present embodiment, the watermark data embedding program of FIG. 3 and the watermark data extracting program of FIG. 4 are stored in these external recording media such as the hard disk HD, flexible disk FD, and compact disk CD.
[0012]
The tone generator circuit 41 generates (that is, reproduces) a tone signal using the tone generator waveform data stored in the built-in waveform memory 41 a under the control of the CPU 31. The generated musical sound signal is supplied to the sound system 42. The sound system 42 includes an amplifier, a speaker, and the like, and generates a musical sound corresponding to the supplied musical sound signal. As shown in FIG. 2, the waveform memory 41a stores waveform data WD1, WD2,. Each waveform data is formed by sampling a waveform for a sound source as shown in FIG. 5 at a predetermined rate, and includes a plurality of sampling data representing a series of instantaneous values (that is, level values) of a musical sound waveform.
[0013]
In addition, a MIDI interface circuit 51 and a communication interface circuit 52 are also connected to the bus 20. The MIDI interface circuit 51 communicates MIDI data with the MIDI device 53 connected to the circuit 51. The MIDI device 53 is a generic name for various electronic devices that comply with the MIDI standard, such as other electronic musical instruments and personal computers. The communication interface circuit 52 can be connected to the outside via the communication network 54, and can exchange various programs and data between the electronic device and the outside.
[0014]
Next, the operation of the embodiment configured as described above will be described. The user causes the CPU 31 to execute the watermark data embedding program stored in the external recording medium of the external storage device 35. In executing this program, the watermark data embedding program stored in the external recording medium is transferred to the RAM 34, and the CPU 31 executes the watermark data embedding program transferred to the RAM 34. When this watermark data embedding program is not recorded on the external recording medium, the watermark data embedding program is downloaded from the MIDI device 53 via the MIDI interface circuit 51, or the communication interface circuit 52 and the communication network 54 are connected. Alternatively, the watermark data embedding program may be downloaded from the outside, and the downloaded watermark data embedding program may be executed. Furthermore, the watermark data embedding program may be stored in the ROM 33 in advance, and the watermark data embedding program may be executed.
[0015]
The execution of this watermark data embedding program is started in step 100 of FIG. 3, and in step 102, the waveform data to be embedded is read out. In this waveform data reading process, the user operates the operator group 11 to display the types and names of the plurality of waveform data WD1, WD2,... WDn stored in the waveform memory 41a on the display unit 12. One waveform data is selected from the plurality of waveform data. The CPU 31 reads out the selected waveform data (a series of sampling data) from the waveform memory 41 a and stores it in the RAM 34.
[0016]
Next, in step 104, the CPU 31 divides the read waveform data into a plurality of sections on the time axis according to a predetermined section division rule. In the present embodiment, the section division rule is divided into a plurality of sections according to the amplitude of the waveform represented by the waveform data and the passage of time. Specifically, as shown in FIG. 5, the musical sound signal is divided into a plurality of sections B1, B2, B3, B4, and B5 according to the magnitude of the amplitude from the start of generation.
[0017]
Next, in step 106, an embedded section in which the watermark data is embedded and a non-embedded section in which the watermark data is not embedded are determined according to a predetermined embedding rule among the plurality of divided sections. In the present embodiment, as shown in FIG. 5, the section B2 including the peak value of the attack portion having a very large amplitude and the section B5 having a small amplitude are set as non-embedded sections, and the other sections B1, B3, and B4 are set as embedded sections. .
[0018]
Next, in step 108, an embedding mode of each determined embedding section is determined according to a predetermined embedding mode rule. In the present embodiment, as shown in FIG. 5, an embedding mode for embedding a small amount of watermark data is determined for sections B1 and B3 with relatively large amplitudes (in other words, sections with large amplitude changes). The An embedding mode for embedding a large amount of watermark data is determined for a section B4 in which the amplitude is medium (in other words, a stable section in which the amplitude is not small but the amplitude change is small).
[0019]
  In this case, the embedding mode of a small amount of watermark data reduces the frequency of embedding the watermark data in a series of sampling data (see the left column in FIG. 6), or only applies the watermark data to a small number of bits for each sampling data. It is realized by embedding (see the left column in FIG. 7). Conversely, a large amount of watermark dataofIn the embedding mode, the frequency of embedding watermark data into a series of sampling data is increased (see the right column in FIG. 6), or watermark data is embedded in a large number of bits for each sampling data (right in FIG. 7). Column reference). In FIGS. 6 and 7, one square of each sampling data indicates one bit, and a bit of sampling data in which a square with a pattern is replaced with watermark data is illustrated. In addition to these embedding modes, an embedding mode is adopted in which the total embedding amount of watermark data is determined by combining both the embedding frequency of the watermark data and the embedding amount of the watermark data with respect to the one sampling data. You can also
[0020]
After the process of step 108, the CPU 31 executes a cyclic process including steps 110 to 120 to embed watermark data in the waveform data. In step 110, one section is extracted in order from the head section of the waveform data stored in the RAM and divided into a plurality of sections, and set as a target section. In step 112, it is determined whether the set target section is the watermark data embedding section determined in step 106. If the target section is a watermark data non-embedded section, the process proceeds to step 120. If the target section is the watermark data embedding section, in step 114, an embedding process corresponding to the embedding mode determined for the target section by the processing in step 108 is selected. This embedding process is a process prepared in the watermark data embedding program or as a separate program corresponding to the type of embedding mode.
[0021]
After the selection of the embedding process, the watermark data is added to each sampling data from the head of the target section until the embedding of the watermark data for all the sampling data in the target section is completed by the cyclic processing in steps 116 and 118. It will be embedded sequentially. In the embedding of the watermark data, in a manner defined by the selected embedding process, one or more bits on the lower side in the sampling data are composed of a data string representing a watermark character, Replaced with part. For example, regarding the character string “XYZ123”, “X” serial data converted from a plurality of bits representing “X” to 1-bit serial data, and a plurality of bits representing “Y” converted to 1-bit serial data “ The character string “XYZ123” is converted into serial data byte by byte so that serial data of “Y” is connected, and “X”, “Y”,... One bit or a plurality of bits on the lower side are sequentially replaced bit by bit with the prepared serial data.
[0022]
When the embedding of the watermark data is completed for all the sampling data in the target section, “Yes” is determined in step 118, and the embedding of the watermark data is completed for all the sections in step 120. Determine whether. If the embedding of the watermark data for all the sections has not been completed, the process returns to step 110, and a new target section is set, a new embedding process is selected, and a watermark data embedding process for the new target section is executed. When the watermark data embedding is completed for all the sections, the execution of the watermark data embedding program is terminated in step 122.
[0023]
In this way, the waveform data in which the watermark data is written is returned to the waveform memory 41 a or stored in the external recording medium of the external storage device 35. Further, the data is transferred to the MIDI device 53 via the MIDI interface circuit 51 or output to the outside via the communication interface circuit 52 and the communication network 54.
[0024]
As can be understood from the above description, according to the embodiment, watermark data is not embedded in the sampling data included in the sections B2 and B5. In addition, regarding the sections B1, B3, and B4 in which the watermark data is embedded, a different amount of watermark data is embedded in the sampling data depending on the section. Therefore, it is possible not to impair the sound quality of the generated musical sound signal by not including much watermark data in the portion that better represents the characteristics of the sound source waveform. Watermark data can be embedded efficiently. Therefore, when this waveform data is reproduced, natural musical sounds having remarkable characteristics are reproduced.
[0025]
Next, an operation for extracting watermark data embedded in waveform data as described above will be described. In this case, the user causes the CPU 31 to execute the watermark data extraction program stored in the external recording medium of the external storage device 35, as in the case of the watermark data embedding program. Also in this case, if the watermark data extraction program is not recorded on the external recording medium, the watermark data extraction program is downloaded from the MIDI device 53 or from the outside via the communication network 54, or stored in the ROM 33. A watermark data extracting program stored in advance may be used.
[0026]
The execution of this watermark data extraction program is started at step 200 in FIG. 4, and the waveform data from which the watermark data is to be extracted is obtained from the waveform memory 41a by the processing of step 202 similar to steps 102 to 108 of the watermark data embedding program. Read out and store in RAM 34. Thereafter, the waveform data is divided into a plurality of sections in accordance with the section division rule described above, and the watermark data embedding section and the non-embedding section are determined for each section, and the watermark data extraction method is determined for the embedding section. To do. In this case, the same rule as the above-described embedding presence / absence rule is applied to the determination of the watermark data embedding section and the non-embedding section. The rule corresponding to the embedding mode rule described above is also applied to the watermark data extraction method.
[0027]
After the process of step 202, the CPU 31 executes a cyclic process consisting of steps 204 to 210 to take out watermark data embedded in the waveform data. In step 204, one section is taken out in order from the head section of the waveform data stored in the RAM and divided into a plurality of sections, and set as a target section. If the set target section is determined to be a watermark data non-embedded section by the process of step 202, “No” is determined in step 206 and the process proceeds to step 210.
[0028]
If the target section is a watermark data embedding section, in step 208, the extraction process corresponding to the extraction method determined for the target section by the process of step 202 is selected and the extraction process is executed. The watermark data embedded in each sampling data is sequentially extracted from the beginning of the target section. This extraction process is a process prepared in the watermark data extraction program or as a separate program corresponding to the various extraction methods. Further, in this extraction process, a bit string (serial data) is created by sequentially connecting 1-bit or multiple-bit data that is embedded in each sampling data and forms the bit pattern of the watermark data from the head.
[0029]
When the extraction of the watermark data for all the sampling data in the target section in step 208 is completed or, as described above, it is determined in step 206 that the target section is a non-embedded section of the watermark data, step 210 It is determined whether or not the watermark data extraction process for all the sections has been completed. If the watermark data extraction process for all the sections has not been completed, “No” is determined in step 210, and the watermark data extraction process in steps 204 to 208 is continued.
[0030]
When the watermark data extraction process for all the sections is completed, “Yes” is determined in step 210, the generated bit string is verified in step 212, and the watermark data extraction program of the watermark data extraction program is determined in step 212. End execution. In this verification, the created bit string is cut out by a predetermined number of bits, and the data of the cut out predetermined bit number is converted into a character string and displayed on the display 12. Thereby, the user can visually confirm the information represented by the watermark data embedded in the waveform data. In this verification, the data representing the converted character string is compared with the watermark information (character string) prepared in advance, and watermark data embedding is automatically confirmed. As described above, since the watermark data (identification information) embedded in the waveform data is extracted and confirmed by executing the watermark data extraction program, an illegal copy of the waveform data can be easily found.
[0031]
In carrying out the present invention, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the object of the present invention.
[0032]
For example, in the above-described embodiment, the sound source waveform data is divided into a plurality of sections over time according to the magnitude of the amplitude, as the section division rule. However, the present invention is not limited to this, and the sound source waveform data may be divided into a plurality of sections as time passes according to the magnitude of the temporal change rate of the amplitude. In this case, since the portion with a large change rate has a large change in the waveform for the sound source and often expresses the characteristics of the musical tone, a small amount of watermark data is embedded in the portion. A large amount of watermark data is embedded in the portion where the change rate is small.
[0033]
In addition, frequency analysis of the sound source waveform represented by the sound source waveform data shows that the portion with a lot of harmonic components or the portion where the harmonic components change drastically often represents the timbre characteristics of the musical tone. Embed a small amount of watermark data. Then, a large amount of watermark data is embedded in a portion with less harmonic components or a portion where the harmonic component changes slowly. Even if these division division rules and embedding mode rules are applied, the sound quality of the generated musical sound signal can be kept intact, and watermark data can be embedded efficiently in the waveform data for the sound source.
[0034]
In the above embodiment, the watermark data is not embedded in the sampling data in the sections B2 and B5 in FIG. However, a small amount of watermark data may be embedded in the sampling data of the sections B2 and B5. However, with respect to the section B2, in order not to impair the characteristics of the musical sound to be reproduced, the section B5 is less than the sections B1 and B3 in order to prevent the waveform data having a small amplitude from being greatly affected by the watermark data. An amount of watermark data may be embedded.
[0035]
Also, in the watermark data embedding mode rule, the amount of watermark data embedded in the sound source waveform data can be changed as follows. That is, a plurality of types of watermark information including a plurality of types of character strings having different lengths are prepared, and watermark data representing a long character string (for example, the character string “XYZ123”) is included in a portion where the amount of watermark data to be embedded is large. The watermark data representing a short character string (for example, the simplified character string “XYZ”) may be employed in a portion where the amount of watermark data embedded is small. In the first place, when embedding watermark data, the same data is usually embedded repeatedly. When only a small amount of watermark data can be embedded, only a small number of watermark data can be embedded over a wide range. Embedding a small number of watermark data is undesirable because of the requirement to embed watermark data repeatedly. Therefore, the change of the embedding mode for selecting any one of a plurality of types of watermark data is effective in avoiding the undesirable situation.
[0036]
In the above embodiment, the watermark data embedding amount according to the embedding mode for each section is divided into only two stages (see FIG. 5). However, more types of embedding modes can be prepared, and the embedding amount of the watermark data can be set to three or more levels. Also, an embedding mode that gradually increases or decreases embedding of watermark data may be adopted in one embedding mode.
[0037]
In the above embodiment, the process of dividing the waveform data for sound source in step 104 into a plurality of sections, the determination of the watermark data embedding section and the non-embedding section in step 106, and the embedding mode of each embedding section in step 108 are as follows. The process of determining and the process of actually embedding watermark data in the waveform data for sound source consisting of steps 110 to 120 are simultaneously performed by a series of processes. However, instead of this, the processing of step 104 and the processing of steps 106 to 120 may be performed independently and separately. In this case, the data indicating the division position may be added to the waveform data and stored by the process of step 104. Further, the processing in steps 104 to 108 and the processing in steps 110 to 120 may be performed independently and separately. In this case, the data indicating the division position by the process of step 104, the data indicating the embedding section and the non-embedding section by the process of step 106, and the data indicating the embedding mode of step 108 are added to the waveform data and stored. It is good to.
[0038]
In these cases, the processing in step 104 and the processing in steps 106 to 120, or the processing in steps 104 to 108 and the processing in steps 110 to 120 are performed in different processes, or different workers are performed in different places. May be. In other words, the embedding process of the watermark data in different processes or places is prepared in advance if the sound source waveform data divided into a plurality of sections is prepared, and the watermark data is added to the prepared sound source waveform data. It means that it may be embedded.
[0039]
In the above embodiment, the waveform memory 41a stores waveform data composed of sampling data that directly represents the instantaneous value (level value) of the musical sound waveform. However, various types of compressed data are replaced with sampling data. It may be stored in the waveform memory 41a. For example, a plurality of difference data representing the difference between each sample point of the musical sound waveform may be stored in the waveform memory 41a instead of the sampling data.
[0040]
Further, in the above embodiment, the watermark data embedding and extracting process is applied to the waveform data stored in the waveform memory 41a of the sound source circuit 41. However, the external recording of the ROM 33 and the external storage device 35 is applied. The watermark data embedding and extracting process may be applied to the waveform data stored in the medium. Further, the waveform data stored in the MIDI device 53 may be taken into the RAM 34 via the MIDI interface circuit 51, and the watermark data embedding and taking-out process may be applied to the taken waveform data. . Furthermore, the waveform data may be taken into the RAM 34 from the outside via the communication network 54 and the communication interface circuit 52, and the watermark data embedding and taking-out process may be applied to the fetched waveform data.
[Brief description of the drawings]
FIG. 1 is a block diagram of an electronic apparatus to which the present invention is applied.
FIG. 2 is a data format of sound source waveform data.
FIG. 3 is a flowchart of a watermark data embedding program executed by the CPU of FIG. 1;
FIG. 4 is a flowchart of a watermark data extraction program executed by the CPU of FIG.
FIG. 5 is a musical sound waveform diagram represented by waveform data.
FIG. 6 is an explanatory diagram for explaining an example of an embedding mode of watermark data with respect to sound source waveform data;
FIG. 7 is an explanatory diagram for explaining another example of an embedding manner of watermark data into waveform data for sound source.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Operator group, 12 ... Display, 31 ... CPU, 33 ... ROM, 34 ... RAM, 35 ... External storage device, 41 ... Sound source circuit, 52 ... Communication interface circuit.

Claims (4)

In a watermark data embedding device that embeds watermark data in waveform data for a sound source that represents a waveform for the sound source and is used to generate a musical sound signal,
Watermark data to be embedded in sound source waveform data with different amounts of data according to the characteristics of the musical sound by the sound source waveform data at each position for each position on the time axis of the sound source waveform represented by the sound source waveform data A data amount determining means for determining the data amount of
Embedding means for embedding watermark data having a data amount determined for each position on the time axis in waveform data for a sound source corresponding to the position on the time axis
A watermark data embedding device. A dividing means for dividing the sound source waveform represented by the sound source waveform data into a plurality of sections on the time axis ,
The data amount determination means has a data amount that differs depending on the characteristics of the musical sound of the waveform data for sound source in each section for each of the divided sections , and the amount of watermark data to be embedded in the waveform data for sound source. The watermark data embedding device according to claim 1, wherein the watermark data embedding device is determined. Sound source waveform data is divided into multiple sections on the time axis .
The data amount determining means is a data amount of watermark data to be embedded in the sound source waveform data, which is different for each of the divided sections according to the characteristics of the musical sound by the sound source waveform data in each section. The watermark data embedding device according to claim 1, wherein the watermark data embedding device is determined. In a computer program that embeds watermark data in waveform data for a sound source that represents the waveform for the sound source and is used to generate a musical sound signal,
On the computer,
Watermark data to be embedded in sound source waveform data with different amounts of data according to the characteristics of the musical sound by the sound source waveform data at each position for each position on the time axis of the sound source waveform represented by the sound source waveform data A data amount determination procedure for determining the data amount of
A computer program for executing an embedding procedure for embedding watermark data having a data amount determined for each position on the time axis in waveform data for sound source in correspondence with the position on the time axis .

Images