TWI661421B - System and method with audio watermark - Google Patents

Abstract

The invention discloses a system and method with audio watermark. The method includes: selecting an audio watermark transmission configuration parameter according to a user preference setting and a background noise sample; converting the action code script into an embedded symbol sequence according to the audio watermark transmission configuration parameter; embedding the embedded symbol sequence into the original audio to Become an embedded watermark audio; decode the embedded watermark audio to restore the symbol and time series; and interpret the restored symbol and time series into a restore action code script and the phase shift information of the recording and the original audio start time .

Description

System and method with audio watermark

The invention relates to a technology of audio watermarking, in particular to a system and method with audio watermarking.

Audio watermarking technology is a method of information dissemination. The transmitting device can embed the desired message into the audio in advance through the signal space conversion and the encoding technology of the message frame, and the receiving device can decode it through the message code frame. Decoding technology restores the message.

Audio watermarking technology has two main application purposes: one is to detect the integrity or source verification of audio files; the other is to embed messages in the audio and send them through the air channel for the receiving device to interpret the messages. The method of spatial conversion and embedding of audio signals in audio watermarking technology includes low-bit encoding, phase encoding, echo hiding, spread spectrum and other methods.

In the prior art, a framework using time-varying watermarks for information integrity confirmation, tampering position detection, tampering method judgment, and restoration of damaged areas is proposed, which can solve the problem of digital recording data's judgment in court. In another prior art, a watermark loading device is proposed. First calculate the volume and pitch of the original audio, and filter the volume and pitch thresholds of the watermark loading target segment, and then compare the volume and pitch thresholds to obtain the watermark loading target segment and loading watermark. Information.

However, in the above prior art, different audio watermark transmission configuration parameters cannot be selected according to different user preferences and background noise samples to adjust the transmission speed or accuracy of the embedded watermark audio; The audio watermark synchronizes the timestamp and information of the sending device and the receiving device.

The invention provides a system and method with an audio watermark, which can be used to adjust the transmission speed or accuracy of the embedded watermark audio, or to synchronize the time stamp and information of the sending device and the receiving device with the audio watermark.

The system with an audio watermark in the present invention includes: an audio watermark transmission configuration selection module that selects audio watermark transmission configuration parameters according to user preference settings and background noise samples; a symbol sequence generation module that The audio watermark transmission configuration parameter selected by the watermark transmission configuration selection module converts the action code script to be embedded into the embedded symbol sequence; the audio watermark embedded module is from the symbol sequence generation module The embedded symbol sequence is embedded in the original audio to become the audio of the embedded watermark; the audio watermark decoding module decodes the audio of the embedded watermark from the audio watermark embedded module to restore the symbol and time series; Symbol sequence An interpretation module, which interprets the restored symbols and time series from the audio watermark decoding module into the restored action code script and the time-shift information of the recording and the original audio start time.

The method with audio watermark in the present invention includes the following steps: selecting audio watermark transmission configuration parameters according to user preference settings and background noise samples; according to the selected audio watermark transmission configuration parameters, scripting action codes to be embedded Convert to embedded symbol sequence; embed the embedded symbol sequence into the original audio to become the embedded watermark audio; decode the embedded watermark audio to restore the symbol and time series; and interpret the restored symbol and time series as Restore the action code script and the phase shift information of the recording and the start time of the original audio.

In order to make the above features and advantages of the present invention more comprehensible, embodiments are described below in detail with reference to the accompanying drawings. Additional features and advantages of the present invention will be partially explained in the following description, and these features and advantages will be partially obvious from the description, or may be learned through practice of the present invention. The features and advantages of the invention are realized and achieved by means of elements and combinations specifically pointed out in the scope of the patent application. It should be understood that both the foregoing general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the scope of the invention as claimed.

01‧‧‧System with audio watermark

010‧‧‧User Preferences

011‧‧‧Noise tolerance

012‧‧‧Reliability

020‧‧‧ background noise

021‧‧‧Background Noise Sample

030‧‧‧Action code definition table

040‧‧‧Action code script

042‧‧‧ Time difference between the starting point of the action and the original audio

050‧‧‧Original Audio

051‧‧‧Text Information

060‧‧‧air channel

100‧‧‧Audio watermark transmission configuration selection module

110‧‧‧Background Noise Attribute Analysis Submodule

119‧‧‧Background Noise Properties

120‧‧‧ Audio Watermarking Protocol Decision Sub-module

130‧‧‧ Base Byte Generation Submodule

190‧‧‧Audio watermark transmission configuration parameters

191‧‧‧Sync Symbol Group Definition

192‧‧‧ packet symbol group definition

193‧‧‧ length of sound box

194‧‧‧Error correction mechanism

195‧‧‧Symbol set definition

196‧‧‧basic byte length

197‧‧‧Estimated decoding time

198‧‧‧ Symbol and base byte correspondence table

199‧‧‧Audio sampling rate

200‧‧‧Symbol sequence generation module

210‧‧‧ Action code time axis adjustment submodule

211‧‧‧packet number

212‧‧‧Time difference between packet start point and original audio start point

213‧‧‧action code

214‧‧‧ Time difference between the execution of the action and the start of the packet

219‧‧‧Action code time axis adjustment script

220‧‧‧Text Conversion Symbol Group Submodule

221‧‧‧packet number

222‧‧‧Symbol set of time difference between packet start point and original audio

223‧‧‧Action code symbol group

224‧‧‧Symbol set of time difference between execution and packet start point

229‧‧‧Action Symbol Group Script

230‧‧‧ Error Correction Code Operator Module

231‧‧‧packet number

232‧‧‧Symbol set of time difference between packet start point and original audio start point

233‧‧‧ Action code symbol set with error correction

234‧‧‧Symbol set with time difference between execution action with error correction and packet start

239‧‧‧ Action Symbol Group Script with Error Correction

240‧‧‧Symbol sequence combination submodule

290‧‧‧ Embedded Symbol Sequence

300‧‧‧ Audio Watermark Embedded Module

310‧‧‧ Sound frame cutting submodule

319‧‧‧Audio frame

320‧‧‧Signal Conversion Submodule

329‧‧‧Audio parameters

330‧‧‧Audio Watermark Gain Value Decision Sub-module

338‧‧‧ Audio watermark gain value of previous frame

339‧‧‧Audio watermark gain value

340‧‧‧Symbol Frame Cutting Submodule

349‧‧‧inline symbol box

350‧‧‧Symbol Box Embedded Module

359‧‧‧Embedded watermark audio parameters

360‧‧‧Anti-signal conversion sub-module

369‧‧‧Embedded watermark audio frame

370‧‧‧Sound frame combination submodule

390‧‧‧Embedded Watermark Audio

391‧‧‧ Embedded watermark audio samples with background noise and unknown starting point

400‧‧‧Audio Watermark Decoding Module

410‧‧‧ symbol starting position search submodule

419‧‧‧The closest starting position of the symbol in the future

420‧‧‧ Sound frame cutting submodule

428‧‧‧ decoded end position

429‧‧‧Audio frame

430‧‧‧Signal conversion sub-module

439‧‧‧Audio parameters

440‧‧‧Symbol Decoding Submodule

448‧‧‧ Decoding Strength Score

449‧‧‧ decoded symbol

450‧‧‧Symbol sequence combination submodule

490‧‧‧ Restored symbols and time series

500‧‧‧Symbol sequence interpretation module

510‧‧‧Symbol sequence cutting submodule

511‧‧‧The symbol group of the time difference between the beginning of the packet and the beginning of the original file

512‧‧‧ Action code symbol set with error correction

513‧‧‧Symbol set with time difference between execution action with error correction and original audio start point

514‧‧‧Time difference between packet start and recording start

519‧‧‧Restoring Action Symbol Group Script with Error Correction

520‧‧‧ Error Correction Submodule

521‧‧‧Action code symbol set

522‧‧‧Symbol set of time difference between starting action and original audio start point

529‧‧‧ Restore Action Symbol Group Script

530‧‧‧Restore action code script generation submodule

531‧‧‧action code

532‧‧‧ Execution time

580‧‧‧Restore Action Code Script

590‧‧‧ Phase shift information from the start time of recording and original audio

A‧‧‧Transmission device

A1‧‧‧Speaker

B‧‧‧ Receiving Device

B1‧‧‧Microphone

Steps S1 to S5‧‧‧‧

FIG. 1 is a schematic block diagram of a system with an audio watermark in the present invention; FIG. 2 is a schematic block diagram of an audio watermark transmission configuration selection module in FIG. Fig. 3 is a schematic block diagram of a symbol sequence generating module in Fig. 1 of the present invention; Fig. 4 is a schematic block diagram of an audio watermark embedded module in Fig. 1 of the present invention; FIG. 6 is a schematic block diagram of an audio watermark decoding module in FIG. 1 of the present invention; FIG. 6 is a schematic block diagram of a symbol sequence interpretation module in FIG. 1 of the present invention; FIG. 7 is a view illustrating the present invention The schematic diagram of the definition of the medium packet symbol group; FIG. 8 is a schematic diagram showing the correspondence table of symbols and base bytes in the present invention; FIG. 9 is a schematic diagram showing the action code definition table in the present invention; and FIG. 10 is a diagram showing The schematic diagram of the action code script in the present invention; FIG. 11 is a schematic diagram showing the action code time axis adjustment sub-module in the third diagram of the present invention; FIG. 12 is the schematic diagram of the action code time axis adjustment script in the present invention; FIG. 13 is a schematic diagram of an action symbol group script in the present invention; FIG. 14 is a schematic diagram of an action symbol group script with error correction in the present invention; FIG. 15 is a schematic diagram of an embedded symbol table in the present invention. Schematic diagram; FIG. 16 shows the embedded symbols in the present invention A schematic diagram of a column; Fig. 17 illustrates a system schematic of the present invention, reduction of time series symbols; FIG. 18 is a schematic diagram showing a data table of a synchronization symbol group and a load symbol group in the present invention; FIG. 19 is a schematic diagram showing a script of a restoration action symbol group with error correction in the present invention; Schematic diagram of the script for restoring action symbols in the invention; Fig. 21 is a diagram showing the error correction method in the present invention; Fig. 22 is a diagram showing the restoration action code and time difference sequence in the present invention; and Fig. 23 is a drawing Schematic diagram of restoring action code script in the invention; Figure 24 is a schematic diagram showing the phase shift information of the recording and original audio start time in the invention; Figure 25 is a schematic diagram showing the method with audio watermark in the invention The flowchart and FIG. 26 are schematic diagrams showing the synchronization of the clock and information of the transmitting device and the receiving device by the system and method with audio watermark of the present invention.

The following describes the embodiments of the present invention with specific specific implementation forms. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this description, and can also be implemented by other different specific implementation forms. Or apply.

FIG. 1 is a schematic architecture diagram of an audio watermarking system 01 in the present invention. As shown in the figure, the system 01 with audio watermark can be composed of five main modules, which are the audio watermark transmission configuration selection module 100 and the symbol. The number sequence generating module 200, the audio watermark embedded module 300, the audio watermark decoding module 400, and the symbol sequence interpretation module 500.

The audio watermark transmission configuration selection module 100 can select or determine the transmission protocol and configuration parameters of the overall audio watermark transmission. The user preference setting 010 and the background noise sample 021 are used as the input of the audio watermark transmission configuration selection module 100. , And then output as the audio watermark transmission configuration parameter 190 for the subsequent modules (such as the symbol sequence generation module 200, the audio watermark embedded module 300, the audio watermark decoding module 400, or the symbol sequence interpretation Module 500) embed and decode information.

The symbol sequence generating module 200 can convert the action code script 040 that the user wants to embed into the embedded symbol sequence 290 according to the transmission configuration specified by the audio watermark transmission configuration parameter 190.

The audio watermark embedding module 300 may embed the embedded symbol sequence 290 in the original audio 050 according to the transmission configuration specified by the sound frame length 193 (see FIG. 2) of the audio watermark transmission configuration parameter 190. Audio 390 with embedded watermark.

The audio watermark decoding module 400 can randomly record the audio 390 embedded in the watermark through the air channel 060 and the background noise 020 according to the transmission configuration specified by the audio watermark transmission configuration parameter 190. The "embedded watermark audio sample 391 with background noise and unknown starting point" decodes the "restoration symbol and time series 490" contained in the audio.

The symbol sequence interpretation module 500 can interpret the "restored symbols and time series 490" into "restored action code script 580" and "recording and original audio from the transmission configuration specified by the audio watermark transmission configuration parameter 190" Phase shift information at the beginning of time 590 ".

FIG. 2 is a schematic block diagram of the audio watermark transmission configuration selection module 100 in FIG. 1 of the present invention. As shown in the figure, the audio watermark transmission configuration selection module 100 has a background noise attribute analysis submodule 110, an audio watermark communication protocol decision submodule 120, and a base byte generation submodule 130, and the audio watermark The transmission configuration selection module 100 takes user preference settings 010 and background noise samples 021 as its inputs. The user preference setting 010 includes a noise tolerance 011 and a reliability 012. The noise tolerance 011 is a user-specified tolerance for noise caused by embedded information, and the reliability 012 is a user-specified description of a transmission reliability requirement. The background noise sample 021 is an audio sample simulated or actually recorded by the user based on the background noise 020 of the future application environment.

The background noise attribute analysis sub-module 110 can analyze the background noise attribute 119 from the background noise sample 021 through an audio conversion algorithm, and the background noise attribute 119 includes the volume attribute and the dynamic range attribute of the background noise sample 021.

The audio watermark communication protocol decision sub-module 120 may determine the audio watermark transmission configuration parameter 190 according to the background noise attribute 119 and the user preference setting 010, and the audio watermark transmission configuration parameter 190 includes a synchronization symbol group definition 191, a packet symbol Group definition 192, sound frame length 193, error correction mechanism 194, symbol set definition 195, base byte length 196, estimated decoding time 197, and audio sampling rate 199.

The base byte generation sub-module 130 may generate a base byte of the specified number according to the number of the symbol set specified in the symbol set definition 195 and the base byte length 196 to record the base byte of the specified number of bytes. Floating in audio "Symbol and base byte correspondence table 198" specified by watermark transmission configuration parameter 190.

FIG. 3 is a schematic block diagram of the symbol sequence generating module 200 in FIG. 1 of the present invention. As shown in the figure, the symbol sequence generation module 200 has an action code time axis adjustment sub-module 210, a text conversion symbol group sub-module 220, an error correction code calculation sub-module 230, and a symbol sequence combination sub-module 240. The sequence generating module 200 generates the embedded symbol sequence 290 by using the audio watermark transmission configuration parameter 190 and the action code script 040 as inputs, so as to determine the symbol sequence of embedded audio in the next stage according to the embedded symbol sequence 290.

The user must provide an action code definition table 030 and an action code script 040 to specify the content to be embedded in the audio. The action code definition table 030 contains both an action code and a description of the action content (see FIG. 9), and the two can be used to restore the action code script 580 (see FIG. 23) to perform the specified action on the application side. The action code script 040 is a file, and the content of the action code script 040 describes one or more sets of action code script information. Each set of the action code script information includes an action code and a time difference between execution of the action and the beginning of the original audio (see FIG. 10).

The action code time axis adjustment sub-module 210 can determine the position of the actual embedded information packet according to the packet symbol group definition 192, the sound frame length 193, and the estimated decoding time 197 of the audio watermark transmission configuration parameter 190. Axis adjustment script 219. The content of the action code time axis adjustment script 219 includes "packet number 211", "time difference between the start point of the packet and the original audio start point 212", "action code 213", and "the time difference between the execution of the action and the start point of the packet 214 ". In addition, the action code time axis adjustment sub-module 210 can be calculated according to the packet symbol group definition 192, the audio sampling rate 199, and the sound frame length 193. Out the packet time length, cut the audio from the beginning to the packet frame according to the packet time length, and then arrange the packet frame number in sequence. At the same time, the action code time axis adjustment sub-module 210 can reduce the execution time specified by the action code script 040 minus the estimated decoding time 197 and the packet time length to become the latest start point of the packet, and obtain the start position of the packet frame less than The latest packet at the latest starting point of the packet is regarded as the embedded packet, and the packet number 211 of the embedded packet is recorded.

The text conversion symbol group sub-module 220 can convert the text of the action code time axis adjustment script 219 into a symbol group sequence according to the symbol set definition 195 of the audio watermark transmission configuration parameter 190. In addition, the text conversion symbol group submodule 220 can set the "packet number 211" of the action code time axis adjustment script 219, "the time difference 212 between the start point of the packet and the original audio start point", "action code 213", and "execute The time difference 214 between the action and the start point of the packet is converted into "packet number 221", "the symbol set 222 of the time difference between the start point of the packet and the original audio start point" in the action symbol set script 229, and the "action code symbol set 223 "and" Symbol set 224 of time difference between execution action and packet start point ".

The error correction code operator module 230 may perform the operation of the error correction code according to the error correction mechanism 194 on the "action code symbol group 223" in the action symbol group script 229 and "the symbol group 224 that performs the time difference between the action and the start point of the packet" To increase the credibility of the content being delivered. After the operation of the error correction code calculation module module 230, an action symbol group script 239 with error correction can be generated, and the content of the action symbol group script 239 with error correction includes "packet number 231", "packet start point and Symbol set 232 of the time difference of the original audio starting point ", "Action code symbol group 233 with error correction" and "Symbol group 234 with time difference between execution action and packet start point with error correction".

The symbol sequence combination sub-module 240 can combine the action symbol group script 239 with error correction according to the audio watermark transmission configuration parameter 190, the symbol set definition 195, the packet symbol group definition 192, and the synchronization symbol group definition 191 into an embedded symbol sequence. 290. The embedded symbol sequence 290 is a sequence formed by the symbols defined in the symbol set definition 195. This sequence is successively arranged by the packet. The symbol groups in the packet are arranged by the packet symbol group definition 192, and the packet symbol group definition 192 A synchronization symbol group including a synchronization symbol group definition 191 is provided for a decoding end (or a receiving device) to decode a symbol sequence.

FIG. 4 is a schematic block diagram of the audio watermark embedded module 300 in the first image of the present invention. As shown in the figure, the audio watermark embedded module 300 has a sound frame cutting submodule 310, a signal conversion submodule 320, an audio watermark gain value decision submodule 330, a symbol frame cutting submodule 340, and a symbol frame. Embedded sub-module 350, anti-signal conversion sub-module 360 and sound frame combination sub-module 370, and the audio watermark embedded module 300 transmits the "sound frame length 193" specified by the audio watermark transmission configuration parameter 190 Corresponding to the "Symbol and Base Byte Mapping Table 198", the embedded symbol sequence 290 is embedded in the original audio 050 to become the embedded audio 390 watermark.

The sound frame cutting sub-module 310 may sequentially cut the original audio 050 into the audio sound frame 319 according to the sound frame length 193, so that the signal conversion sub-module 320 performs signal processing on all the cut out audio sound frames 319. The signal conversion sub-module 320 may convert the specified signal of the audio frame 319 to obtain the audio parameter 329. Audio watermark gain value decision sub-module 330 can be calculated according to audio parameter 329 The average volume, and referring to the audio watermark gain value 338 of the previous frame, determines the audio watermark gain value 339 of the current frame to be embedded.

The symbol frame cutting sub-module 340 may sequentially extract the embedded symbol frame 349 from the embedded symbol sequence 290 to specify the symbol to be embedded in the current sound frame. The sub-module 350 embedded in the symbol frame can hide the symbols in the embedded symbol frame 349 into the audio according to the corresponding symbol and base byte correspondence table 198 and the audio watermark gain value 339. Parameter 329 to obtain the audio parameter 359 of the embedded watermark. The anti-signal conversion sub-module 360 can convert the audio parameter 359 of the embedded watermark through the inverse signal conversion to convert the audio parameter back to the sound box to form the audio sound box 369 of the embedded watermark. The sound frame combination sub-module 370 may sequentially combine the audio sound frames 369 with embedded watermarks to restore the audio watermark 390 with embedded watermarks.

FIG. 5 is a schematic block diagram of the audio watermark decoding module 400 in FIG. 1 of the present invention. As shown in the figure, the audio watermark decoding module 400 has a symbol starting position search submodule 410, a sound frame cutting submodule 420, a signal conversion submodule 430, a symbol decoding submodule 440, and a symbol sequence combination submodule. Group 450, and the audio watermark decoding module 400 uses the "sound frame length 193" and "symbol and base byte correspondence table 198" specified by the audio watermark transmission configuration parameter 190 to contain "unknown starting point" The embedded watermark audio sample 391 "of the background noise is used to decode the audio watermark, and finally the restored symbol and time series 490 are decoded.

The symbol starting position search sub-module 410 can select the "decoded end position 428" from "embedded watermark audio sample 391 with unknown background and background noise", and based on "sound frame length 193" and "symbol Basal byte pairs "Table 198" is searched for "future closest symbol starting position 419". In the initial state, the decoded end position 428 is the starting point of the "embedded watermark audio sample 391 with background noise and unknown starting point". When the decoding intensity score 448 is higher than the threshold value, the closest starting position of the symbol 419 in the future can be directly based on the previous closest starting position of the symbol 419 plus the sound box length 193, thereby saving the calculation time. time.

The sound frame cutting sub-module 420 can cut an audio frame 429 with a length of 193 from the closest starting position of the symbol in the future, and then update the decoded end position 428 to the closest starting position of the symbol in the future. 419 plus the number of sound box length 193.

The signal conversion sub-module 430 may convert the audio frame 429 into an audio parameter 439 in a specified signal conversion method. The symbol decoding sub-module 440 may calculate the distance of the audio parameter 439 by using the base byte of the "symbol and base byte correspondence table 198" one by one to obtain the distance as the decoding intensity score 448, and the symbol corresponding to the shortest distance That is, the decoded symbol 449.

The symbol sequence combination sub-module 450 may temporarily store all the decoded symbols 449 calculated by the symbol decoding sub-module 440, and infer the time difference between the decoded symbol 449 and the recording start point from the closest starting position of the symbol 419 in the future. The two pieces of information of the decoded symbol 449 and the time difference are concatenated as "restored symbol and time series 490".

FIG. 6 is a schematic block diagram of the symbol sequence interpretation module 500 in FIG. 1 of the present invention. As shown in the figure, the symbol sequence interpretation module 500 includes a symbol sequence cutting sub-module 510, an error correction sub-module 520, and a restoration action code script generation sub-module 530. The symbol sequence interpretation module 500 is based on audio floating water. Print the synchronization symbol group definition 191, the packet symbol group definition 192, the error correction mechanism 194, and the symbol set definition 195 specified by the transmission configuration parameter 190, and perform symbol sequence interpretation on the restored symbols and time series 490 to obtain the "restore action code Script 580 "and" Phase shift information 590 of the start time of recording and original audio ".

The symbol sequence cutting sub-module 510 can compare the synchronization symbol group with the time series 490 according to the synchronization symbol group definition 191, and interpret the "recovery action symbol group script 519 with error correction" according to the packet symbol group definition 192. , And a set of recovery action symbol set script 519 with error correction includes "symbol set 511 of the time difference between the start of the packet and the original audio file", "action code symbol set 512 with error correction", and "executive action with error correction" Symbol set 513 ″ from the time difference from the original audio start point. The symbol sequence cutting submodule 510 may also record the time difference between the recorded symbol and the recording start point in the time series 490 according to the first symbol of the packet, and record it as "the time difference 514 between the start of the packet and the recording start point".

The error correction sub-module 520 may, according to the designation method of the error correction mechanism 194, restore the "action code symbol group 512 with error correction" and the "execution action with error correction and original" The symbol group 513 of the time difference of the audio start point is restored to the "action code symbol group 521" and "the symbol group 522 of the time difference between the action and the original audio start point", plus the "packet start and original audio file start The time difference symbol group 511 "and" the time difference between the start of the packet and the recording start point 514 "become the recovery action symbol group script 529.

Restore action code script generation submodule 530 can be determined based on the symbol set Yi Yi 195 interprets the symbol group in the restoration action symbol group script 529 back to the text, and generates "restore action code script 580" and "phase shift information 590 of the recording and original audio start time". The restoration action code script 580 may include an action code 531 and an execution action time 532. The time shift information 590 between the recording and the original audio start time can be divided by the median of "the symbol set 511 of the time difference between the start of the packet and the original audio file" and "the time difference 514 of the start of the packet and the start of the recording" Presumably available.

Aiming at the above-mentioned preferred embodiments of each module in FIGS. 2 to 6, it is further illustrated with reference to FIGS. 7 to 24 as follows. It should be noted that the following is for illustration only, and the present invention is not limited thereto.

As shown in FIG. 2, the audio watermark transmission configuration selection module 100 takes the user preference setting 010 and the background noise sample 021 as its inputs. The user preference setting 010 includes a noise tolerance 011 and a reliability 012, and the noise tolerance 011 can specify a tolerance for noise caused by embedded information for a user. For example, the noise tolerance 011 (such as Tlr) for noise caused by embedded information can be set to 3 (high), 2 (medium), 1 (low), and the user-selectable reliability 012 (such as Conf) Set to 3 (safer), 2 (moderate), 1 (faster). The background noise sample 021 is an audio sample simulated or actually recorded by the user based on the background noise 020 of the future application environment.

The background noise attribute analysis sub-module 110 in FIG. 2 can calculate the average volume difference between the embedded audio and the background noise sample 021, and set the volume attribute (such as V _noise ) of the background noise attribute 119 to 3 (strong), 2 (medium) ), 1 (weak). At the same time, the background noise attribute analysis sub-module 110 can linearly scale the background noise sample 021 to the interval of 0 ~ 1, and calculate the standard deviation of the volume variation of the background noise sample 021, and the dynamic range attribute of the background noise attribute 119 (such as DR _noise is set to 3 (large), 2 (medium), and 1 (small).

The audio watermark communication protocol decision sub-module 120 in FIG. 2 can determine the audio watermark transmission configuration parameter 190 according to the background noise attribute 119 and the user preference setting 010. For example, [1] set the sound frame length 193 as L _window = 1024 (this number can be adjusted according to the required reliability); [2] set the base byte length 196 as L _c = 512 (this number must not be greater than Exceeds the length of the sound box); [3] defines the error correction mechanism 194 as a voting mechanism for every six sets of repeated information (this mechanism can be adjusted according to the required reliability); [4] sets the estimated decoding time 197 as D _decode = 3 seconds, L _decode = D _decode * SR = 132300 samples (this number must be obtained through experience or actual experiments); [5] set the audio sampling rate to 199 to SR = 44100 samples / second; [6] to define 195 according to the symbol set Define the number of symbols N _S = 6, and the set of symbols that can be used is S = {S ₁ = '1', S ₂ = '2', S ₃ = '3', S ₄ = '4', S ₅ = ' a ', S ₆ =' b '}, the symbol set of the available synchronization symbol group is S _sync = {S ₅ =' a ', S ₆ =' b '}, the symbol set of the usable load symbol group is S _payload = {S ₁ = '0', S ₂ = '1', S ₃ = '2', S ₄ = '3'}; [7] Define the synchronization symbol group SS _SW = {according to the definition of synchronization symbol group 191 SS _SW1 = S ₅ S ₅ S ₆ S ₆ , SS _SW2 = S ₅ S ₆ S ₅ S ₆ , SS _SW3 = S ₆ S ₆ S ₅ S ₅ }.

FIG. 7 is a schematic diagram illustrating a packet symbol group definition 192 in the present invention. As shown in FIG. 2 and FIG. 7, the audio watermark communication protocol decision sub-module 120 can use the packet symbol group definition 192 to define a synchronization symbol group, a load symbol group, and a combination manner between the symbol groups. In this embodiment, a packet is divided into six parts, which sequentially include a first synchronization symbol group (such as SS _SW1 , consisting of 4 symbols), a symbol group 232 (such as the time difference between the packet start point and the original audio start point) (such as SS _start , consisting of 8 symbols), second synchronization symbol group (such as SS _SW2 , consisting of 4 symbols), action code symbol group with error correction 233 (such as ECSSIX _action , consisting of 24 symbols), third Synchronous symbol group (such as SS _SW3 , consisting of 4 symbols), and symbol group 234 (such as ECSS _offset , consisting of 36 symbols) with the time difference between the execution action of the error correction and the packet start point. The symbol length of a packet is SSL _packet = 80 symbol length.

FIG. 8 is a schematic diagram showing a symbol and base byte correspondence table 198 in the present invention. As shown in FIG. 2 and FIG. 8, the base byte generation submodule 130 may be based on the number of symbols N _S = 6 and the base byte length 196 (L _c = 512) specified in the symbol set definition 195. , Randomly select six sets of base bytes C ₁ to C _{6 of} length L _c corresponding to six symbols S ₁ to S _{6 respectively} , among which, six signs S ₁ to S ₆ and six sets of bases of length Lc The correspondence table of the bytes C ₁ to C ₆ is the symbol and base byte correspondence table 198 of FIG. 8. In addition, the orthogonality between two pairs of base bytes can be limited. The absolute distance of the cosine distance can be used to calculate the vector distance between the two base bytes. If the vector distance is less than a specific threshold (such as 0.99) (this specific threshold can be According to the reliability Conf, adjust between 0 ~ 1), then the combination of base bytes must be regenerated.

The symbol sequence generating module 200 in FIG. 3 takes the audio watermark transmission configuration parameter 190 and the action code script 040 as input to generate an embedded symbol sequence 290, and then determines the embedded audio sequence in the next stage according to the embedded symbol sequence 290. Symbol sequence. The user must provide an action code definition table 030 and an action code script 040 to specify the content to be embedded in the audio accordingly.

FIG. 9 is a schematic diagram showing an action code definition table 030 in the present invention. As shown in the figure, the action code definition table 030 is a file. For example, action The description of the action content corresponding to code 3 is "close the browser".

FIG. 10 is a schematic diagram showing an action code script 040 in the present invention. As shown in the figure, the action code script 040 is a file, and the content of the action code script 040 describes one or more sets of action codes and their execution time. For example, the action code corresponding to action number ASN = 2 is IX _action = 3, and the time difference between the execution of the action and the original audio start point 042 is T _action = 20000 (milliseconds); if the time length is calculated by the number of samples, it is SI _action = T _action * SR / 1000 = 882000 (number of samples). For example, this action number specifies that the action with the action code of 3 is performed at the 20,000 milliseconds (that is, the 20th second) or 882000th sample from the beginning of the original audio, which is the "close browser" shown in Figure 9.

FIG. 11 is a schematic diagram of the time code adjustment sub-module 210 of the action code in FIG. 3 of the present invention. As shown in FIG. 3 and FIG. 11, the parameter calculation of the axis adjustment sub-module 210 for the action code, the cutting of the packet frame, and the time axis adjustment are illustrated as examples below.

In terms of parameter calculation, the action code time axis adjustment sub-module 210 can calculate according to the sound frame length 193 (such as L _window = 1024), the audio sampling rate 199 (such as SR = 44100), and the packet symbol group definition 192 in FIG. 7. The length of the symbol samples (such as L _symbol = L _window = 1024), the packet symbol length (such as SSL _packet = 80), and the packet sample length (such as L _packet = L _symbol * SSL _packet = 1024 * 80 = 81920). In addition, the estimated decoding time 197 is L _decode = 132300.

In packet frame cutting, the action code time axis adjustment sub-module 210 can cut the audio from the beginning to the packet frame according to the packet sample length L _packet , and sequentially sequence the packet number PSN from 0, starting at the kth packet The point is SI _packet (k) = k * L _packet .

On the time axis adjustment, "the time difference between the latest start point of the embedded packet executing the action number m and the starting point of the original audio SI _deadline (m)" is taken as "the execution time of the action number m and the starting point of the original audio The time difference (number of samples) SI _action (m) ”is obtained by subtracting L _decode and L _packet . "Time difference SI _embed (m) between the starting point of the embedded packet of the action number m and the starting point of the original audio" is the SI _packet (k) of the starting point of the most recent packet whose packet frame start position is less than the SI _deadline (m) , That is, SI _embed (m) = SI _packet (k). The solution of the "time _offset SI _offset (m) between the execution action number m and the start point of its embedded packet" is SI _offset (m) = SI _action (m)-SI _embed (m). At the same time, the SI _deadline (m), SI _embed (m), and SI _offset (m) are recorded in the action code time axis adjustment script 219.

FIG. 12 is a schematic diagram showing an action code time axis adjustment script 219 in the present invention. For example, in the above time axis adjustment, SI _action (2) = 882000 ₍₁₎ ; SI _deadline (2) = SI _action (2) -L _packet -L _decode = 882000-81920-132300 = 667780 ₍₂₎ ; The most recent packet whose packet start position is less than 667780 is the packet number 8 ₍₃₎ (ie PSN = floor (667780/81920) = 8), so the action number ASN = 2 the starting point of the embedded watermark SI _embed (2) is placed SI _packet (8) at the starting position of packet number 8, SI _embed (2) = SI _packet (8) = L _packet * 8 = 655360 ₍₄₎ ; SI _offset (2) = SI _action (2) -SI _embed ( 2) = 882000-655360 = 226640 ₍₅₎ .

Furthermore, the action code time axis adjustment script 219 in FIG. 12 may include "action number ASN", "action code IX _action ", "execution time of action number m and time difference (sample number) from the original audio start point SI _action (m) "," Time difference between the latest start point of the embedded packet in action number m and the original audio start point SI _deadline (m) "," packet number PSN "," embedded packet start point in action number m "Time difference SI _embed (m) from the original audio start point" and "Time _offset SI _offset (m) between the execution time of the action number m and the start point of its embedded packet".

FIG. 13 is a schematic diagram showing an action symbol group script 229 in the present invention. As shown in FIG. 3 and FIG. 13, the text conversion symbol group submodule 220 can convert the text in the action code time axis adjustment script 219 to a symbol group sequence according to the symbol set definition 195, and the action code time axis adjustment script The "packet number PSN", "action code IX _action ", and "the time difference SI _offset (m) between the execution time of the action number m and its embedded packet start point" in 219 are respectively converted into the action symbol group script 229. "Packet number symbol group SSPSN", "action code symbol group SSIX _action " and "execution time of action number m and its embedded packet start point time difference symbol group SSSI _offset (m)".

For example, the text conversion symbol group sub-module 220 may sequentially add data rows without a specified packet number, and the time difference SI between the execution time of "action code IX _action " and "action number m and the start point of its embedded packet" _offset (m) ”are all preset to 0. The action code symbol group SSIX _action is an action code. The IX _action is represented by a quartet and is filled with four digits. The number of full digits varies depending on the total number of action codes. The packet number symbol group SSPSN represents the packet number PSN in quaternary and is filled with 0 to 8 digits. The number of digits to be filled may vary depending on the embedded audio length. "The execution time of action number m and its embedded packet start point time difference symbol set SSSI _offset (m)" is the "time difference between the execution time of action number m and its embedded packet start point SI _offset (m)" divided by After 1000, it is rounded up, expressed as a quartet, and filled with 6 digits. The number of full digits can be changed according to the embedded audio length.

FIG. 14 is a schematic diagram showing the action symbol group script 239 with error correction in the present invention. As shown in FIG. 3 and FIG. 14, the error correction code operation module 230 may perform an operation of the error correction code to generate an action symbol group script 239 with error correction, thereby improving the credibility of the transmitted content. In this embodiment, the error correction mechanism 194 uses a repeat information voting mechanism every six groups. For example, the original symbol group is expressed as '0003'. After being calculated by the error correction code operator module 230, the symbol group with error correction is set to '000300030003000300030003'. At the same time, after the operation of the error correction code operator module 230, "action symbol group script 239 with error correction" can be generated, and the content of the action symbol group script 239 with error correction includes action number ASN, packet number PSN, and packet Numbered symbol set SSPSN and error correction code operator module 230 generated "action code symbol set with error correction ECSSIX _action " and "action time with error corrected action number m and its embedded packet start point time difference Symbol group ECSSSI _offset (m) ".

FIG. 15 is a schematic diagram showing an embedded symbol table in the present invention. As shown in FIG. 3 and FIG. 15, the symbol sequence combination submodule 240 can sort the packet symbol group definition 192 according to the packet symbol group definition 192 according to the action symbol group script 239 with error correction, and sort the “packet number PSN”, “ First synchronization symbol group SS _SW1 "," Packet number symbol group SSPSN "," Second synchronization symbol group SS _SW2 "," Action code symbol group with error correction ECSSIX _action "," Third synchronization symbol group SS _SW3 "" The execution time of the action number m with the error correction and the time difference symbol group ECSSSI _offset (m) "of its embedded packet start point are combined into an embedded symbol table.

FIG. 16 is a schematic diagram showing the embedded symbol sequence 290 in the present invention. As shown in Figures 3 and 16, in addition to the packet number PSN, the symbol sequence combination sub-module 240 can sequentially define the embedded symbol table from left to right and top to bottom, and define the embedded symbol table in The symbols are arranged in a sequence to become an embedded symbol sequence 290 (such as sbl ).

In the audio watermark embedded module 300 in FIG. 4, the sound frame cutting sub-module 310 can sequentially cut the original audio 050 (such as x ) according to the sound frame length 193 (such as L _window = 1024) into a sample length of The audio frame 319 of 1024 (such as x _i ) is then subjected to subsequent signal processing for all cut-out audio frames 319.

The signal conversion sub-module 320 in FIG. 4 can convert the specified signal of the audio frame 319 (such as x _i ). In this embodiment, a discrete cosine transform (DCT) of 1024 points is used to obtain audio parameters of 1024 dimensions. 329 (such as f _i ). The i-th frame (such as x _i ) contains audio samples (such as x _{i , 0} ~ x _{i , 1023} ). The audio parameter 329 is , m = 0 ~ 1023.

The audio watermark gain decision sub-module 330 in FIG. 4 can calculate the average volume (such as v _i ) according to the audio parameter 329, and refer to the audio watermark gain value 338 (such as g _i-1 ) of the previous sound frame. To embed the current audio frame watermark gain value 339 (such as g _i ), where v _i = Σ _m | f _{i, m} | ² , g ₁ = α v ₁ , g _i = β (α v _i ) + (1-β) g _{i -1} . α, β can be set according to auditory experiment or experience. The symbol frame cutting sub-module 340 in FIG. 4 can sequentially take out an embedded symbol sequence 290 (such as sbl ) from an embedded symbol frame 349 (such as sbl _i ) to specify a symbol to be embedded in the current sound frame.

FIG 4 symbols of the embedded block submodule 350 may be based on symbols, and the base of FIG. 8 byte correspondence table 198 to find the corresponding byte sbl _i of the substrate, e.g., sbl _i = a, a substrate corresponding bit The tuple is C _{5, k} , k = 0 ~ 511. Moreover, the sub-module 350 embedded in the symbol frame of FIG. 4 can hide the base byte into the audio parameter 329 (such as f _{i, m} ) to become the audio parameter 359 (such as ),among them, The following is a simpler implementation of the awm_encode function:

The anti-signal conversion sub-module 360 in FIG. 4 can convert the audio parameters of the embedded watermark 359 (such as ) After anti-signal conversion, and convert the audio parameters back to the sound box to become the audio sound box with embedded watermark 369 (such as ), And , k = 0 ~ 1023.

According to the above algorithm, an audio frame 369 with embedded watermark (such as ), The sound box combination sub-module 370 in Figure 4 Combine them in order to restore the audio 390 (such as x ' ) with embedded watermark.

In the audio watermark decoding module 400 in FIG. 5, the symbol starting position search sub-module 410 can decode the decoded end point of the embedded watermark audio sample 391 (such as y ) with background noise that has an unknown starting point. Position 428 (such as offset ), and search for the closest symbol starting position 419 (such as next_sbl_start ) in the future according to the sound frame length 193 (such as L _window = 1024), and the corresponding table of symbols and base bytes 198 (such as C) . In the initial state, the decoded end position 428 is offset = L _window / 2, and the decoding strength score 448 is score = 0. The following (a) to (c) describe how to find next_sbl_start with a brief example:

(1) If the decoding strength score 448 (such as score ) of the previous decoding is greater than the decoding strength score threshold (such as γ), the next closest symbol starting position 419 is next_sbl_start = offset +1, and the next step is directly performed. Frame cutting sub-module 420; if score γ, then continue the process of searching for the starting position of the symbol sub-module 410.

(B) starting from the position of y offset -L _window / 2 acquires a sound of a length L _window frame z _0; y is from of Starting from the position of, to obtain a sound box z _{1 of} length L _window ; and so on, to obtain z ₀ ~ z _p , where p = L _window -1 = 1023.

(3) Convert the audio frame 429 (such as z ₀ ~ z _p ) into the audio parameter 439 (such as h ₀ ~ h _p ) in the same audio conversion method as the signal conversion sub-module 430, and use the symbol decoding sub-module 440 obtains decoding symbols 449 (such as sync_sbl ₀ ~ sync_sbl _p ) and decoding strength scores 448 (such as sync_score ₀ ~ sync_score _p ) for h ₀ ~ h _p . If sync_score _n is the maximum value, next_sbl_start should be updated to , And record the resolved symbol sync_sbl _n to "restore the symbol and time sequence 490", and the decoding intensity score 448 is score = sync_score _n .

The sound frame cutting sub-module 420 in FIG. 5 can cut an audio frame 429 (such as z _{next_sbl_start} ) with a length of 193 (such as L _window = 1024) from the nearest starting position of the symbol 419 (such as next_sbl_start ). ), And then the decoded end position 428 is updated to offset = next_sbl_start + L _window -1.

The signal conversion sub-module 430 in FIG. 5 can convert the audio sound box 429 (such as z _{next_sbl_start} ) into the audio parameter 439 (such as h _{next_sbl_start} ) in a specified signal conversion method, and the signal conversion method of the signal conversion sub-module 430 must The signal conversion method is the same as the signal conversion sub-module 320 of the audio watermark embedded module 300.

FIG symbol decoding submodule 5440 may be the first audio parameter 439 (e.g., _h next_sbl_start) element with an embedded watermark in accordance with the arrangement order of embedding h _ordered, as in this embodiment _{h ordered = h next_sbl_start [512:} 1023] . The symbol decoding sub-module 440 may calculate the similarity similarity _i one by one by using h _ordered and the base byte C _i in the "Symbol and Base Byte Correspondence Table 198" one by one, and the similarity may be cosine similarity. , The symbol corresponding to the maximum similarity obtained according to this is the decoded symbol 449 (such as Sdec _n ), and the decoded symbol 449 is stored as the last symbol of the temporary restoration symbol sequence (such as t mp_symbol_seq ). The time difference between the starting points is tdec = next_sbl_start / SR , and tdec is stored as the last number of the temporary restoration time series (such as tmp_time_seq ). The decoding intensity score 448 is score = similarity _i , and the decoded ending position 428 is updated to offset. ' = offset + L _window .

FIG. 17 is a schematic diagram showing a reduction symbol and a time series 490 in the present invention. As shown in FIG. 5 and FIG. 17, the symbol sequence combination sub-module 450 can integrate t mp_symbol_seq and tmp_time_seq into a “reduction symbol and time series 490”, and the reduction symbol and time series 490 includes “reduction symbol number DEC_SIX "," Restore Symbol DEC_S "and" Time Difference DEC_T Between Restore Symbol and Recording Start Point ".

FIG. 18 is a schematic diagram showing a data table of the restored synchronization symbol group and the restored load symbol group in the present invention. As shown in FIG. 6 and FIG. 18, the symbol sequence cutting sub-module 510 of the symbol sequence interpretation module 500 can perform a synchronization symbol group comparison of “restored symbols and time series 490” according to the synchronization symbol group definition 191 and Cut to generate a data table of the synchronization symbol group and the load symbol group.

FIG. 19 is a schematic diagram showing a script 519 of a restoration action symbol group with error correction in the present invention. As shown in FIG. 6 and FIG. 19, the symbol sequence cutting sub-module 510 can decode the restoration action symbol group script 519 with error correction and the restoration action symbol group script 519 with error correction according to the packet symbol group definition 192 combination. Includes "Decoded Packet Number DPSN", "Restored Packet Number Symbol Set DEC_SSPSN", "Restoration Action Code Symbol Set with Error Correction DEC_ECSSIX _action ", and "Run Time of Recovery with Error Correction and Its Embedded Packet Start Point Time Difference Symbol Group DEC_ECSSSI _offset ". In addition, the symbol sequence cutting sub-module 510 may record the time difference between the recording and recording start points in the "data table of the synchronization symbol group and the load symbol group" according to the first symbol of the packet, and record it as "the packet start and Time difference REC_T _{packet from the} recording start point ”.

FIG. 20 is a schematic diagram showing a script 529 for restoring action symbols in the present invention. As shown in FIG. 6 and FIG. 20, the error correction submodule 520 may, according to the designation method of the error correction mechanism 194, convert the “recovery action code with error correction” in the “recovery action symbol set script with error correction 519” Symbol group DEC_ECSSIX _action "and" Reduction execution time with error correction and its embedded packet start point time difference symbol group DEC_ECSSSI _offset "are restored to" Restore action code symbol group DEC_SSIX _action "and" Restore execution time and its Embedded packet start point time difference symbol group DEC_SSSI _offset ”, plus“ time difference between _packet start point and recording start point REC_T _packet ”to become the recovery action symbol group script 529.

FIG. 21 is a schematic diagram showing an error correction method in the present invention. In the embodiment in FIG. 20 above, the restored packet number is 2 as an example, which has an error. The correct action code symbol set is "001200100010001000103010". The designated error correction method of the error correction mechanism 194 is to repeat six groups of votes with four symbols. As shown in Figure 21, the voting process of the error correction method is restored. (Voting result) is "0010".

The restoration action code script generation submodule 530 in FIG. 5 can interpret the symbol group in the restoration action symbol group script 529 back to text according to the symbol set definition 195, and generate “restore action code script 580” and “recording and Phase shift information 590 "for the original audio start time.

FIG. 22 is a schematic diagram showing a restoration action code and a time difference sequence in the present invention. As shown in FIG. 5 and FIG. 22, the way in which the recovery action code script generation submodule 530 can interpret the symbol group back to the text needs to correspond to the text conversion symbol group submodule 220. For example, FIG. 22 and the following (A) to (c):

(1) Reduction action code DEC_IX _action can be obtained by reverting the DEC_SSIX _{action of the} set of reduction action code symbols represented by quaternary to decimal. For example, the DEC_SSIX _action of the restoration action code symbol group is "0010", and the DEC_IX _{action of} the restoration action code is "4".

(2) The restored packet number DEC_PSN can be obtained by converting the restored packet number symbol group DEC_SSPSN indicated by quaternion back to decimal. For example, the restored packet number symbol group DEC_SSPSN is "00000103", and the restored packet number DEC_PSN is "19".

(C) "execution time and the reduction of its embedded packet starting point of the time difference symbol group DEC_SI _offset" by ary representation of "reducing the execution time of the packet and its built-in symbol set the starting point of the time difference DEC_SSSI _offset" back to decimal Multiply by 1000 to get it. For example, "the set of time difference between the execution time of the restoration and its embedded packet start point DEC_SSSI _offset " is "010201", and the decimal place is "289", "the time difference of the execution time of the restoration and the start time of its embedded packet The group "DEC_SI _offset " is "289000".

FIG. 23 is a schematic diagram showing the restoration action code script 580 in the present invention. As shown in the figure, the restoration action code script 580 includes "reduction action code DEC_IX _action ", "the difference between the execution time of the restoration and the original audio start point time (sample number) DEC_SI _action ", and "the execution time of the restoration and the original audio start point Time difference (msec) DEC_T _action "," Time difference between the execution time of the restore and the original audio start point (sample number) DEC_SI _action "consists of" restore packet number DEC_PSN "," packet sample length SL _packet "and" recovery execution time and its Calculate the embedded packet start point time difference symbol group DEC_SI _offset ", DEC_SI _action = DEC_PSN * L _packet + DEC_SI _offset ," the time difference between the execution time of the restore and the original audio start point (ms) DEC_T _action "is calculated as DEC_T _action = DEC_SI _action / SR.

FIG. 24 is a schematic diagram showing the phase shift information 590 of the recording and original audio start time in the present invention. As shown in the figure, the time shift information 590 between the recording and the original audio start time can be calculated from the "time difference (ms) of the packet start point and the original audio start point DEC_T _packet " and "the time difference between the start of the packet and the start of recording The median of the gap between REC_T _packet ”can be estimated, where DEC_T _packet = DEC_PSN * L _packet / SR.

FIG. 25 is a schematic flowchart of a method with an audio watermark in the present invention. As shown in Figure 1 and Figure 25, the main technical content of this method is as follows, The rest of the technical content is as shown in the above-mentioned Figures 1 to 24, and will not be repeated here.

In step S1 of FIG. 25, the audio watermark transmission configuration selection module 100 selects the audio watermark transmission configuration parameter 190 according to the user preference setting 010 and the background noise sample 021.

In step S2 in FIG. 25, the symbol sequence generation module 200 converts the action code script 040 that the user wants to embed into the audio watermark transmission configuration parameter 190 selected by the audio watermark transmission configuration selection module 100. Embedded symbol sequence 290.

In step S3 of FIG. 25, the embedded audio sequence 290 from the symbol sequence generating module 200 is embedded in the original audio 050 by the audio watermark embedded module 300 to become the embedded watermark 390.

In step S4 of FIG. 25, the audio watermark decoding module 400 decodes the audio signal 390 embedded in the watermark from the audio watermark embedding module 300 into a restored symbol and a time series 490.

In step S5 of FIG. 25, the symbol sequence interpretation module 500 interprets the restored symbols and time series 490 from the audio watermark decoding module 400 into the restored action code script 580 and the recording and original audio start time. Phase shift information 590.

FIG. 26 is a schematic diagram showing the synchronization of the clock and information of the transmitting device A and the receiving device B by the system and method with audio watermark of the present invention.

As shown in Fig. 1 and Fig. 26, the system and method with audio watermarking of the present invention can be used for a transmitting device A having a speaker A1 and a receiving device B having a microphone B1. This transmission device A can pass through the system (such as the audio watermark (Embedded module 300) or method, embed an embedded symbol sequence (such as text information 051) into the original audio 050 (such as multimedia audio) to become an embedded watermarked audio 390, and use the speaker A1 of the transmission device A to insert the internal The embedded watermarked audio 390 is transmitted to the receiving device B through the microphone B1 for the receiving device B to decode the restoration action code from the embedded watermarked audio 390 through the system (such as the audio watermark decoding module 400) or method. Scripts (such as text information 051) and phase shift information 590 of the recording and original audio start time.

In addition, in the above-mentioned FIG. 1 and FIG. 26, the system and method with audio watermarking of the present invention can provide a two-layer structure of a symbol encoding layer and an audio encoding layer, and perform encoding of an action symbol group script on the symbol encoding layer. , Serialization or error correction, and embed the embedded symbol sequence to be transmitted in the audio coding layer with the base byte of the array in the original audio.

In summary, the system and method with audio watermarking of the present invention may have at least one of the following technical effects or advantages.

(1) The present invention can select different audio watermark transmission configuration parameters according to different user preference settings and background noise samples, so as to adjust the transmission speed or accuracy of the embedded watermark audio, or customize the Select the fastest transmission speed with acceptable accuracy.

(2) The present invention can adjust the time stamp transmission mechanism through the time axis, so that the receiving device can obtain the playback time stamp information of the transmitting device to achieve the effect of multi-screen synchronization. In addition, the present invention can enable the receiving device to know the current multimedia playback time point of the transmitting device and the action code to be completed at a specified time through the special sequence arrangement and packet design of the transmitting device.

(3) The transmission device with speakers can be transmitted through the system or method of the present invention. Method, embed the embedded symbol sequence (such as text information) into the original audio (such as multimedia audio) to become the embedded watermark audio, and send the embedded watermark audio to the receiving device with a microphone through the speaker. In addition, the receiving device can decode the restoration action code script (such as text information) and the phase shift information of the recording and the original audio start time from the embedded watermark audio through the system or method of the present invention.

(4) The present invention proposes a two-layer structure of a symbol encoding layer and an audio encoding layer, which is applicable to various audio watermark embedding methods and error protection mechanisms, and provides better application flexibility for application and development ends. At the same time, the present invention performs the encoding, serialization, or error correction protection mechanism of the action symbol group script in the symbol encoding layer, and embeds or hides the embedded symbol sequence to be transmitted in the audio encoding layer with the array base byte in the original audio Through the two-layered coding structure and clear processing flow, the system or method is more flexible to adapt to different application scenarios.

For example, the application scenarios of the audio watermarking system and method of the present invention can be described as follows, but not limited thereto.

(1) Multi-screen real-time interactive games or programs: Synchronized multi-screen with audio watermarks can achieve interactive puzzle programs for multiple people to answer, and can also broadcast different ads according to different users.

(2) Electronic signboards, concerts or large-scale events: Information broadcast with audio watermarks to establish synchronization among the masses.

(3) Copyright protection: Provide media copyright protection mechanism with audio watermark.

(IV) Offline communication between electronic goods: use audio watermark for information Pass the exchange to achieve the effect of the Internet of Things.

The above-mentioned embodiments merely exemplify the principles, features, and effects of the present invention, and are not intended to limit the implementable scope of the present invention. Anyone who is familiar with this technology can perform the above operations without departing from the spirit and scope of the present invention. Modifications and changes to the implementation form. Any equivalent changes and modifications made by using the disclosure of the present invention should still be covered by the scope of patent application. Therefore, the scope of protection of the rights of the present invention should be as listed in the scope of patent application.

Claims (26)

A system with audio watermark includes: audio watermark transmission configuration selection module, which selects audio watermark transmission configuration parameters according to user preference settings and background noise samples; a symbol sequence generation module, which is based on the audio watermark The audio watermark transmission configuration parameter selected by the watermark transmission configuration selection module converts the action code script to be embedded into an embedded symbol sequence; the audio watermark embedded module will be derived from the symbol sequence generation module The embedded symbol sequence is embedded in the original audio to become the embedded watermark audio; the audio watermark decoding module decodes the embedded watermark audio from the audio watermark embedded module into a restored symbol and Time sequence; and a symbol sequence interpretation module, which interprets the restored symbol and time sequence from the audio watermark decoding module into a restoration action code script and time shift information of the recording and the original audio start time. The system according to item 1 of the scope of patent application, wherein the audio watermark transmission configuration selection module has a background noise attribute analysis sub-module, which analyzes the background noise from the background noise sample through an audio conversion algorithm. Attribute, and the background noise attribute includes a volume attribute and a dynamic range attribute of the background noise sample. The system according to item 2 of the scope of patent application, wherein the audio watermark transmission configuration selection module further has an audio watermark communication protocol decision sub-module, which determines the audio watermark based on the background noise attribute and the user preference setting. Audio watermark transmission configuration parameters. The system according to item 3 of the scope of patent application, wherein the audio watermark transmission configuration selection module further has a base byte generation sub-module, which is defined in the symbol set definition of the audio watermark transmission configuration parameters. The specified number of symbol sets and the length of the base byte generate the base byte of the specified number of records to record the base byte of the specified number of bits in the symbol and base bit specified by the audio watermark transmission configuration parameter Tuple correspondence table. The system according to item 1 of the scope of patent application, wherein the symbol sequence generating module has an action code time axis adjustment sub-module, which is based on the definition of the packet symbol group and the sound frame length of the audio watermark transmission configuration parameter. And the estimated decoding time determine the position of the embedded information packet and generate an action code time axis adjustment script. The system according to item 5 of the scope of patent application, wherein the symbol sequence generating module further has a text conversion symbol group submodule, which is based on the definition of the symbol set of the audio watermark transmission configuration parameter. The text of the axis adjustment script is converted into a sequence of symbol groups. The system according to item 6 of the scope of patent application, wherein the symbol sequence generating module further has an error correction code operator module, and the error correction code operator module is specified according to the audio watermark transmission configuration parameter. The error correction mechanism performs the operation of the error correction code on the action code symbol group in the action symbol group script and the symbol group that executes the time difference between the action and the packet start point to generate an action symbol group script with error correction. The system according to item 7 of the scope of patent application, wherein the symbol sequence generating module further has a symbol sequence combination sub-module, which transmits the action symbol group script with error correction according to the audio watermark transmission configuration parameter The symbol set definition, the packet symbol group definition, and the synchronization symbol group definition are combined into the embedded symbol sequence. The system described in item 1 of the scope of patent application, wherein the audio watermark embedded module is embedded in the embedded symbol sequence according to the transmission configuration specified by the sound box length of the audio watermark transmission configuration parameter. The original audio becomes the audio of the embedded watermark. The system described in item 1 of the scope of patent application, wherein the audio watermark embedded module has a sound frame cutting sub-module and a signal conversion sub-module, and the sound frame cutting sub-module is based on the original audio according to the The length of the sound frame specified by the audio watermark transmission configuration parameter is sequentially cut into audio frames, and the signal conversion sub-module performs signal conversion on the cut out audio frame to obtain audio parameters. The system according to item 10 of the scope of patent application, wherein the audio watermark embedded module further includes an audio watermark gain value decision submodule and a symbol frame cutting submodule, and the audio watermark gain value decision submodule. The group calculates the average volume according to the audio parameter, and determines the audio watermark gain value to be embedded in the current audio frame by referring to the audio watermark gain value of the previous audio frame, and the symbol frame cutting submodule is derived from the embedded symbol sequence. Remove the inline symbol box to specify the symbol embedded in the current sound box. The system according to item 11 of the scope of patent application, wherein the audio watermark embedded module further has a symbol frame embedded sub-module, which is based on the symbol and base bit specified by the audio watermark transmission configuration parameter. Group the correspondence table with the audio watermark gain value of the current sound frame, and hide the symbol in the embedded symbol frame with the corresponding base byte into the audio parameter to obtain the audio parameter of the embedded watermark. The system according to item 12 of the scope of patent application, wherein the audio watermark embedded module further has an anti-signal conversion sub-module and a sound frame combination sub-module, and the anti-signal conversion sub-module converts the embedded floating The watermark audio parameters are converted by the inverse signal to convert the audio parameters back to the audio frame to become the embedded audio watermark audio frame, and the audio frame combination sub-module sequentially combines the embedded audio watermark audio frame to restore Into the embedded watermark. The system described in item 1 of the scope of patent application, wherein the audio watermark decoding module is based on the transmission configuration specified by the audio watermark transmission configuration parameter, and randomly starts recording through the air channel and adds background noise. Embed the audio of the watermark to decode the restored symbol and time series from the embedded watermark audio samples containing the background noise. The system according to item 1 of the scope of patent application, wherein the audio watermark decoding module has a symbol starting position searching sub-module, which is obtained from the embedded watermark audio samples containing background noise whose starting point is unknown. The decoded end position is searched for the closest future start position of the symbol according to the sound box length, symbol and base byte correspondence table specified by the audio watermark transmission configuration parameter. The system according to item 15 of the scope of patent application, wherein the audio watermark decoding module further has a sound frame cutting sub-module, which cuts a length from the future closest symbol starting position to a length of the sound frame length The audio frame is further updated with the decoded end position to the future closest symbol start position plus the number of the frame length. The system according to item 16 of the scope of patent application, wherein the audio watermark decoding module further has a signal conversion sub-module and a symbol decoding sub-module, and the signal conversion sub-module converts the audio frame into a signal. Converted into audio parameters, and the symbol decoding sub-module calculates the distance of the audio parameters one by one according to the symbols and the base byte of the base byte correspondence table to obtain the distance as the decoding intensity score, and the shortest distance corresponds to The symbol is a decoding symbol. The system according to item 17 of the scope of patent application, wherein the audio watermark decoding module further has a symbol sequence combination submodule, which will decode the decoding symbols from the symbol decoding submodule and the closest to the future. The start position of the symbol is presumed that the time difference between the decoded symbol and the recording start point is concatenated into a restored symbol and a time series. The system according to item 1 of the scope of patent application, wherein the symbol sequence interpretation module has a symbol sequence cutting sub-module, which, according to the definition of the synchronization symbol group specified by the audio watermark transmission configuration parameter, The restoration symbol is compared with the time series to synchronize the symbol group, and the script of the restoration action symbol group with error correction is decoded according to the packet symbol group definition specified by the audio watermark transmission configuration parameter. The system according to item 19 of the scope of patent application, wherein the symbol sequence interpretation module further has an error correction sub-module, which includes the error correction action code symbol group in the script of the error correction restoration action symbol group script And the symbol group with the time difference between the execution action and the original audio start point with error correction is restored to the action code symbol group and the symbol group with the time difference between the execution action and the original audio start point, respectively. The system according to item 20 of the scope of patent application, wherein the symbol sequence interpretation module further includes a recovery action code script generation submodule, which reads back the symbol group in the error correction recovery action symbol group script. Text, and based on it, restore action code script and phase shift information of recording and original audio start time. The system described in item 1 of the scope of the patent application is for a transmitting device and a receiving device, and the transmitting device uses the system to embed the embedded symbol sequence into the original audio to become the audio of the embedded watermark. The audio of the embedded watermark is transmitted to the receiving device through the transmitting device, so that the receiving device decodes the restoration action code script and the recording and original audio start from the embedded watermark audio through the system. Phase shift information of time. The system described in item 1 of the scope of patent application further includes a two-layered structure of a symbol encoding layer and an audio encoding layer to perform encoding, serialization, or error correction of the action symbol group script at the symbol encoding layer, and At the audio coding layer, the embedded symbol sequence to be transmitted is embedded or hidden in the original audio with the base bytes of the array. A method with an audio watermark includes the following steps: selecting an audio watermark transmission configuration parameter according to a user preference setting and a background noise sample; according to the selected audio watermark transmission configuration parameter, an action code script to be embedded Convert into embedded symbol sequence; embed the embedded symbol sequence into the original audio to become embedded watermark audio; decode the embedded watermark audio to restore the symbol and time series; and restore the restored symbol and time series Interpreted as a script to restore the action code and time shift information from the start of the recording and the original audio. The method according to item 24 of the scope of the patent application is for a transmitting device and a receiving device, and the transmitting device uses the method to embed the embedded symbol sequence into the original audio to become the embedded watermark audio, and then The audio of the embedded watermark is transmitted to the receiving device through the transmitting device, so that the receiving device decodes the restoration action code script and the recording and original audio start from the embedded watermark audio through the method. Phase shift information of time. The method according to item 24 of the scope of patent application, further comprising providing a two-layered structure of a symbol encoding layer and an audio encoding layer to perform encoding, serialization, or error correction of the action symbol group script at the symbol encoding layer, and At the audio coding layer, the embedded symbol sequence to be transmitted is embedded or hidden in the original audio with the base bytes of the array.