CN104320719B - Television program interaction participatory approaches based on audio frequency watermark and system - Google Patents
Television program interaction participatory approaches based on audio frequency watermark and system Download PDFInfo
- Publication number
- CN104320719B CN104320719B CN201410647192.5A CN201410647192A CN104320719B CN 104320719 B CN104320719 B CN 104320719B CN 201410647192 A CN201410647192 A CN 201410647192A CN 104320719 B CN104320719 B CN 104320719B
- Authority
- CN
- China
- Prior art keywords
- frame
- watermark
- signal
- frequency
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/835—Generation of protective data, e.g. certificates
- H04N21/8358—Generation of protective data, e.g. certificates involving watermark
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Television Systems (AREA)
Abstract
The invention discloses a kind of television program interaction participatory approaches and system based on audio frequency watermark, including：（1）Embedded step includes the watermark signal of interactive information to the insertion of television programme audio signal；（2）Step is played, that is, utilizes playing device（Television set or player）Play the TV programme of embedded watermark signal；（3）Recording step uses mobile terminal device to record the television programme audio signal of the lower insertion watermark signal played；（4）Extraction step utilizes mobile terminal device to extract watermark signal from the television programme audio signal of embedded watermark signal, as long as television program interaction can be immediately engaged in by opening network using mobile terminal.The present invention is convenient and efficient, and does not influence appreciation and viewing of the spectators to TV programme.
Description
Technical field
The present invention relates to Information Hiding Techniques fields, more specifically, being related to a kind of TV programme based on audio frequency watermark
Interactive participatory approaches and system.
Background technology
With the continuous development of modern science and technology technology and communication technology constantly improve, there is one kind in television media industry
The completely new method for attracting spectators with interaction mode, improving program audience rating, spectators are also ready to participate in TV by various modes
Program interaction.In the epoch that current mobile terminal device is popularized, what spectators pursued is the operating method of more convenient and quicker, it would be desirable to
More directly and quickly participate in program interaction.Term " mobile terminal device " used herein above refers to mobile phone, notebook, tablet electricity
Brain etc..
The happy meeting of Lantern Festival happiness of Hunan Satellite TV, the lines of what Gui of host are such：It is broadcast by authentic herbal tea king Lao Ji using names
The happy meeting of Hunan Satellite TV Lantern Festival happiness gone out, you can please remember ours by sending New Year wish/New Year blessing to SMS platform
Clearance is talked secretly：Happy to get home noisy Lantern Festival, have good luck Wang Laoji.Smart phone user can pass through 360 mobile phone-downloaded lakes simultaneously
First social software -- the sound of flapping of southern satellite TV, scan the two-dimensional code into Hunan Satellite TV Lantern Festival happiness pleasure can be opened up in the sound of flapping it is non-
Often interesting game of guessing lantern riddles.
One critical aspects of above-mentioned lines are exactly to participate in guess lantern riddles game, i.e. program interaction.Compared with traditional mode, ginseng
The interest that program can be enhanced with interaction, can also improve the enthusiasm of spectators.Currently, spectators can be joined by following several method
With program interaction：A) it is participated in by short message mode.Such method interactivity is poor, and program can not accomplish immediate interactive with spectators, and
And it will produce expense.B) it is participated in by the remote controler of interactive television interactive.Program interaction is participated in by TV remote controller undoubtedly
Be easily, but the set-top box of common TV because its technological means restrict, can not TV Festival be participated in using this method
Common TV set-top box only interactive digital TV or is upgraded to interactive digital TV and could participate in electricity by mesh interaction link
Depending on program interaction.Few using the user volume of interactive digital TV at present, interactive digital TV is not popularized also, is used simultaneously
Which can not participate in commenting on, and can cause page jump while opening webpage, influence viewing effect.C) by scanning the two-dimensional code
Into website.This method needs two-dimensional code scanning software, and scans inconvenience.Therefore, above several television program interaction ginsengs
Television program interaction all cannot be directly and quickly participated in method.
Invention content
In view of the deficiencies of the prior art, the present invention provides it is a kind of it is convenient, fast, reliably based on audio frequency watermark
Television program interaction participatory approaches and system.
In order to solve the above technical problems, the present invention adopts the following technical scheme that：
A kind of television program interaction participatory approaches based on audio frequency watermark, including step：
Step 1, it is embedded in the watermark signal containing interactive information to television programme audio signal；
Step 2, the television programme audio signal of embedded watermark signal is played using playing device；
Step 3, the television programme audio signal of the lower insertion watermark signal played is recorded using mobile terminal device；
Step 4, the extraction watermark letter from the television programme audio signal of the insertion watermark signal under mobile terminal device record
Number, television program interaction can be participated in by connecting the interactive network address in watermark signal using mobile terminal device.
Step 1 further comprises sub-step：
Step 1.1, gammatone analysis filtering is carried out to television programme audio signal and obtains N number of subband signal, and from N
Intermediate frequency subband signal is chosen in a subband signal, i.e., a subband signals of m to n in N number of subband signal；
Step 1.2, the even number bit sign of watermark signal is stored to CCI, odd number bit sign is stored to LOAD, and uses two
System indicates symbol in CCI and LOAD；
Step 1.3, the matrix chip that size is N ╳ M is generated according to watermark signal, element initial value is all provided in matrix chip
It is equal to the number of bits of symbol and the product of lines in watermark signal for 0, M：
1.3a defines the spreading sequence matrix lookup of size rows ╳ lines, is made of 0 and 1, per a line and every
0 and 1 number is equal in one row, and rows and lines are even number；
1.3b, it is m to enable zidai initial values, and frame initial values are 1；
1.3c, if (frame-1) can be divided exactly by lines, generates the random integers in [1, rows], adopts to current frame
With random integers update spread spectrum sequence matrix lookup current line numbers p_lut；If (frame-1) cannot be divided exactly by lines, protect
It is constant to hold the current p_lut of spread spectrum sequence matrix lookup；
1.3d, by the element value of pth in spreading sequence matrix lookup _ lut row (frame-1) %lines+1 row with
The binary number of the position [(frame-1)/lines+1] is different or, and assigning the of matrix chip by exclusive or result in LOAD
Zidai rows frame row, [] indicates rounding；
1.3e enables frame=frame+1, cycle execute step 1.3c~1.3d, until frame is more than M；
1.3f enables zidai=zidai+1, enables frame=1, and cycle executes step 1.3c~1.3e, until zidai>n；
Step 1.4, intermediate frequency subband signal amplitudes are modulated using matrix chip, obtain the N number of of embedded watermark signal
Subband signal synthesizes through gammatone and obtains the time-domain audio signal containing interactive information.
The generation of random integers in sub-step 1.3c in [1, rows] is real by calling random number generator in matlab
It is existing, specially：
RandStream functions are called to initialize random number generator kind according to current sign in current window serial number and CCI
Son.
Water is extracted in the television programme audio signal of the insertion watermark signal under the record of slave mobile terminal device described in step 4
Official seal number further comprises sub-step：
Step 4.1, structure includes the three-dimensional matrice watermark of all possible watermark signal combination, specially：
4.1a defines the spreading sequence matrix lookup of size rows ╳ lines, is made of 0 and 1, per a line and every
0 and 1 number is equal in one row, and rows and lines are even number；
4.1b, it be 1, frame initial values is 1 that enable i initial values, which be 0, bit initial values,；
4.1c, if (frame-1) can be divided exactly by lines, generates the random integers in [1, rows], adopts to current frame
With random integers update spread spectrum sequence matrix lookup current line numbers p_lut；If (frame-1) cannot be divided exactly by lines, protect
It is constant to hold the current p_lut of spread spectrum sequence matrix lookup；By spreading sequence matrix lookup pths _ lut row (frame-1) %
The element value of lines+1 row is assigned to watermark (bit, frame, i)；
4.1d enables frame=frame+1, cycle execute step 4.1c, until frame is more than M, M is to be accorded in watermark signal
Number number of bits and lines product, then execute step 4.1e；
4.1e, enables bit=bit+1, frame=1, and cycle executes step 4.1c~4.1d, until bit is more than (n-m+1),
Then, step 4.1f is executed；
4.1f enables i=i+1, bit=1, frame=1, cycle execute step 4.1c~4.1e, until i is equal to 2a, a is
Number of bits；
Step 4.2, frequency domain index matrix decoder, frequency domain index matrix decoder of the definition for controlling frequency domain
In respectively row represent a kind of frequency domain index range, the starts frequency of frequency domain index range and terminate frequency and set according to intermediate frequency subband number
It is fixed；
Step 4.3, in time index matrix pointe of the definition for controlling time domain scale, time index matrix pointe
Each row represent a kind of corresponding time index values of time-scaling scale；
Step 4.4, according to the principle of general covariance, matrix under each frequency domain index range and time-scaling scale is obtained
The correlation of element in watermark respective columns takes the corresponding i of maximum related value, time index pointer timeindex, frequency
Index point freqindex normalizes maximum related value with maximum related value；
Step 4.5, according to the corresponding i of maximum related value, time index pointer timeindex, frequency indices pointer
Freqindex calculates watermark value usage to be selected, when the sum of continuous three normalized maximum related values with covariance formula
More than threshold value, and when continuously the corresponding i values of maximum related value are equal three times, then it is assumed that detect watermark；Otherwise, it re-executes
Step 4.4.
The generation of random integers p_lu in step 4.1c in [1, rows] is by calling random number generator in matlab
It realizes, specially：
RandStream functions are called to initialize random number generator kind according to current i and current window serial number number window
Son.
Step 4.2 is specially：
According to intermediate frequency subband signal head numbers m and last number n settings maximum frequency domain index range [startfre,
Endfreq], startfre=m-b indexes starts frequency for frequency domain；Endfreq=n+c terminates frequency for frequency domain index；b、c
For the natural number being arranged according to experience；The frequency domain index range of representative is respectively arranged in frequency domain index matrix decoder no more than maximum
Frequency domain index range.
Step 4.3 is specially：
Time-scaling factor scalefactor is defined, when being calculated according to the time-scaling factor and current sample point serial number number
Between index matrix pointer element values, wherein frame rows, timeindex column element values pointer (frame,
Timeindex) it is：Pointer (frame, timeindex)=((1+ (0.2+frame) * scalefactor*
blocksperwindow/FRAMESPERWINDOW)/searchstep
Wherein, blocksperwindow, FRAMESPERWINDOW are constant, and blocksperwindow is zoom scale
The product of quantity, spread spectrum sequence matrix lookup columns and continuous sample points, FRAMESPERWINDOW are zoom scale quantity
With the product of spread spectrum sequence matrix lookup columns.
The above-mentioned corresponding system of television program interaction participatory approaches based on audio frequency watermark, including：
Watermark signal is embedded in module, is used for being embedded in the watermark signal containing interactive information to television programme audio signal；
Playing module is used for playing the television programme audio signal of embedded watermark signal using playing device；
Recording module, the television programme audio for being used for being recorded the lower insertion watermark signal played using mobile terminal device are believed
Number；
Watermark signal extraction module is used for the television programme audio letter from the insertion watermark signal under mobile terminal device record
Watermark signal is extracted in number, it is mutual to participate in TV programme using the interactive network address in mobile terminal device connection watermark signal
It is dynamic.
The object of the present invention is to provide a kind of methods participating in television program interaction based on audio frequency watermark, can facilitate
Related interactive information is quickly obtained, and the information content for including can be very big, you can to greatly improve watermark information embedded quantity
And there is very high reliability.
The method for the participation television program interaction based on audio frequency watermark that therefore, it is necessary to a kind of, the method neither influence spectators
TV programme are watched, and can automatically participate in interaction, without time-consuming and cumbersome input network address, have not only been facilitated but also quick.
Description of the drawings
Fig. 1 is the particular flow sheet of the method for the present invention；
Fig. 2 is Embedded step flow chart；
Fig. 3 is extraction step flow chart.
Specific implementation mode
The method of the present invention includes four steps：(1) Embedded step includes interactive letter to the insertion of television programme audio signal
The watermark signal of breath, the present invention use the embedded mobile GIS modulated based on amplitude that watermark signal is embedded in television programme audio signal；
The interactive information, which refers to, converts each symbol with the relevant interactive information of TV programme, generally a group code, such as network address
For corresponding ASCII character, that is, include the watermark signal of interactive information.(2) step is played, that is, utilizes playing device (TV
Machine or player) play the TV programme for being embedded in watermark signal；(3) recording step uses mobile terminal device to record lower play
Insertion watermark signal television programme audio signal, since the television programme audio signal comprising watermark signal only needs a few minutes
Even several seconds, thus, it is only required to arbitrarily enroll a part for television programme audio signal；(4) extraction step utilizes
Mobile terminal device extracts watermark signal from the television programme audio signal of embedded watermark signal, as long as being beaten using mobile terminal
Television program interaction can be immediately engaged in by opening network.
The detailed process of above-mentioned Embedded step is as follows：
1.1 pairs of television programme audio signals carry out gammatone analysis filtering, by television programme audio signal decomposition at N
A subband signal.
Intermediate frequency subband letter is chosen in the 1.2 N number of subband signals obtained from sub-step 1.1 according to the bark bands of human auditory system
Number, to avoid the more sensitive low frequency part of human ear.13~20 bark can generally be chosen with corresponding frequency.
1.3 pre-process the watermark signal containing interactive information, so that watermark signal can be suitble to subsequent arithmetic.
1.4 generate chip matrixes, for pretreated watermark signal to be randomly assigned, are detected so as not to the person of being destroyed
Out.
1.5 are modulated the intermediate frequency subband signal amplitudes of selection according to element value in chip matrixes.
N number of intermediate frequency subband signal synthesis of embedded watermark signal is contained interactive information by 1.6 using gammatone synthetic filterings
Time-domain audio signal, i.e., the television programme audio signal of embedded watermark signal.
The detailed process of said extracted step is as follows：
4.1 according to the television programme audio signal of embedded watermark signal generate by 0 and 1 for element watermark (bit,
Frame, i) matrix, to contain all possible interactive information situation.Wherein, 1~n-m+1 of bit value ranges, frame take
The product of the number of bits of symbol in value range 1~42,42 expression frequency expansion sequence digit and watermark signal, i value ranges 0~
27, in specific implementation, take m=13, n=22.
4.2 generate decoder matrixes, are used for the control of frequency domain.Element maximum value endfreq in decoder matrixes
It is m-3 for n+3, minimum value startfre, the decoder matrixes obtained in specific implementation are the matrix of 10 rows 7 row, and each column corresponds to
A kind of frequency indices are as follows：
10 11 12 13 14 15 16
11 12 13 14 15 16 17
12 13 14 15 16 17 18
13 14 15 16 17 18 19
14 15 16 17 18 19 20
15 16 17 18 19 20 21
16 17 18 19 20 21 22
17 18 19 20 21 22 23
18 19 20 21 22 23 24
19 20 21 22 23 24 25
4.3 generate pointer matrixes, are used for the control of time domain scale.
4.4 loader cycle buffering area, for storing pending parameter.
4.5 seek the normalization maximum correlation of pending parameter.
4.6 detect watermark data CCI and LOAD, the invention according to maximum normalized correlation using detection formula
Correlation function used corrects on the basis of original correlation function, largely improves accuracy rate.
Technical solution is further illustrated the present invention below in conjunction with the drawings and specific embodiments.
See that the watermark signal comprising interactive information is embedded in television programme audio signal by Fig. 1 using embedded mobile GIS first,
The television programme audio signal of embedded watermark signal is obtained, embedded watermark letter is played using playing devices such as television set or players
Number television programme audio signal, play and believed simultaneously using the television programme audio of the embedded watermark signal of mobile terminal device admission
Number；Recording finishes, and immediately extracts watermark signal from the television programme audio signal of admission by mobile terminal device, such as interactive
Network address etc.；Finally, network address is opened using mobile terminal device and may participate in television program interaction.
Fig. 2 shows the flows for being embedded in the watermark signal containing interactive information in the present embodiment to television programme audio signal
Figure, detailed process are as follows：
Step 1.1, using gammatone analysis filters group by television programme audio signal decomposition at N number of subband signal.
The frequency dividing characteristic of the fine simulation human ear basilar memebrane of gammatone analysis filters group energy, the present invention utilize
Gammatone analysis filters simulate human auditory system frequency response, and time domain expression-form is as follows：
gi(t)=AtN-1 exp(-2πbit)cos(2πfit+φi) (1)
In formula (1)：
gi(t) amplitude of i-th of the subband signal of expression in t moment；T indicates time, t >=0；A is filter gain；N is
Filter order, i number for filter order, 1≤i≤N；F is centre frequency, φiIt is phase；biIt is decay factor, the factor
Determine the bandwidth of response filter；bi=1.019ERB (fi), wherein ERB (fi) it is equivalent rectangular bandwidth, ERB (fi)=24.7
(4.37fi/1000+1)。
Step 1.2, the intermediate frequency subband signal in N number of subband signal that selecting step 1.1 obtains, such as m to n sons are taken a message
Number, m ＜ n, it is to avoid the low frequency signal of human ear sensitivity to select intermediate frequency subband signal.
Step 1.3, the pretreatment of watermark signal.
By even number bit sign storage in watermark signal in CCI, by the storage to LOAD of odd number bit sign, and using two into
Symbol in CCI and LOAD is shown in tabulation.Assuming that watermark signal be " 1234567890ABCDEF ", then CCI=1,3,5,7,9, A, C,
E }, LOAD={ 2,4,6,8,0, B, D, F }, digit is remembered since 0.In order to which all symbols in ASCII character table are all represented
Come, each symbol in CCI and LOAD is indicated with 7 bits, for example, the corresponding ASCII character of symbol " h " is 104, then adopts
It is indicated " h " with 1101000.
Step 1.4, generator matrix chip.
Build the matrix chip that size is N ╳ 42, wherein N is the subband signal number that step 1.1 obtains, m rows to n-th
Row all elements are binary number, remaining element is 0.
This step is specific as follows：
(1) the spread spectrum sequence matrix lookup that structure size is 12 ╳ 6, element is two-stage system number, and often row is
The spread spectrum sequence that one 6 bit is constituted.
Element in self-defined spread spectrum sequence matrix lookup, element are 0 or 1, ensure that spread spectrum sequence matrix lookup is each
Row is equal with 0 and 1 number in each row.Spread spectrum sequence matrix lookup sizes are not limited to 12 ╳ 6, and size can be according to reality
Situation sets itself.
In this specific implementation, indicate that watermark signal symbol, each bit use spread spectrum sequence using 7 bits
Matrix lookup carries out spread spectrum, and since spread spectrum sequence matrix lookup has 6 row, then matrix chip columns should be set as 7 ╳ 6=42 row.
(2) value of current window serial number window known to and the CCI and LOAD corresponding to current window, current window serial number
Window initial values are 1, call RandStream functions to initialize using current window serial number and the corresponding CCI values of current window random
Matlab realizations specifically can be used in number generator seed.The embedded required sample points of watermark symbol are a window.
Watermark can correspond to time counter when being embedded in, and when the time, counter counts number reached a window size, then add current window serial number
1。
Assuming that watermark signal is " 12345678 ", watermark signal even number bit sign is stored in CCI, the storage of odd number bit sign
In LOAD, then window serial number window and the correspondence of CCI, LOAD value are shown in Table 1.
The correspondence of table 1 window serial number window and CCI, LOAD value
| window | 1 | 2 | 3 | 4 |
| CCI | 1 | 3 | 5 | 7 |
| LOAD | 2 | 4 | 6 | 8 |
(3) it is m to enable zidai initial values, and m is the intermediate frequency subband signal number minimum value chosen；Frame representing matrixes chip
Column number, it is 1 to enable frame initial values.
(4) to current frame, if (frame-1) can be divided exactly by 6, call random number generator function generate [1,
Rows] random integers in range, rows is spread spectrum sequence matrix lookup line numbers, in this specific implementation, rows=12.
Using the random number as spread spectrum sequence matrix lookup current line numbers p_lut；Otherwise, keep current line number p_lut constant.
It (5) will be in the element value and LOAD of pth _ lut rows, (frame-1) %6+1 row in spread spectrum sequence matrix lookup
The (frame-1)/6+1 binary numeral is different or, to realize the spread spectrum of watermark signal, and exclusive or result is assigned to matrix
Zidai rows, the frame row of chip.
(6) it enables frame=frame+1, cycle execute step (4)~(5), until when frame is more than 7*6=42, terminates
Cycle executes step (7), and 7 be the number of bits of symbol in watermark signal in this specific implementation, and 6 indicate frequency expansion sequence digits.
(7) zidai=zidai+1 is enabled, frame=1 is enabled, cycle executes step (4)~(6), until zidai>N terminates
Cycle obtains matrix chip, wherein effective line number is n-m+1, i.e. m~n rows of matrix chip, columns 7*6, remaining row member
Element value is 0.
Step 1.5, (n-m+1) of selection a intermediate frequency subband signal amplitudes are adjusted according to element value in matrix chip
System obtains N number of subband signal of embedded watermark signal.
To resist desynchronization attack, signal amplitude modulation is carried out using repeated encoding thought in this specific implementation.Work as matrix
When element value is 1 in chip, the amplitude of continuous 10 sample point signals is multiplied by the same gain controlling elements mul, works as matrix
When element value is 0 in chip, the amplitude of continuous 10 sample point signals is multiplied by 1/mul, thus obtains being embedded in watermark letter
Number and N number of subband signal through ovennodulation.Gain controlling elements mul passes through to extracting accuracy and sentience progress
Tradeoff progress value, generally 0.5~2.5.
Step 1.6, N number of subband signal that embedded watermark signal is synthesized using gammatone composite filters, is embedded in
The time-domain audio signal of watermark signal.
Fig. 3 shows that the flow chart for extracting watermark signal in the present embodiment from time-domain audio signal, detailed process are as follows：
Step 4.1, three-dimensional matrice watermark (bit, frame, i) is generated.
Watermark matrixes are used for combining comprising all possible watermark signal, and specific implementation step is as follows：
(1) spread spectrum sequence matrix lookup is generated, size is 12 × 6, and element is binary number 0 or 1.Here
Spread spectrum sequence matrix lookup defined in spread spectrum sequence matrix lookup fully synchronized rapid 1.4.
(2) RandStream functions are called to initialize random number generator seed according to i and current window serial number number window,
The initial value of i is that 0, window initial values are 1；Watermark can correspond to time counter when being embedded in, when time counter counts number reaches
When one window size, then current window serial number is added 1.
(3) it is the intermediate frequency subband signal number that 1, frame initial values are 1, bit correspondence selections to enable bit initial values；
(4) to current frame, if (frame-1) can be divided exactly by 6, random number generation function is called to generate random whole
Spread spectrum sequence matrix lookup matrixes pth _ lut rows, (frame-1) %6+1 element value arranged are assigned to matrix by number p_lut
Watermark (bit, frame, i).
(5) frame=frame+1, cycle is enabled to execute step (4), until frame is more than 7*6, end loop executes step
Suddenly (6).
(6) bit=bit+1, frame=1, cycle is enabled to execute step (4)~(5), until bit is more than (n-m+1), end
Cycle executes step (7).
(7) i=i=+1, bit=1, frame=1, cycle is enabled to execute step (4)~(6), until i is equal to 27, to obtain
Obtain three-dimensional matrice watermark.
Step 4.2, frequency domain index matrix decoder is generated.
Respectively row respectively represent a kind of subband signal frequency domain index range, i.e. subband signal frequency domain in index matrix decoder
The starts frequency and end frequency of index.In this specific implementation index matrix decoder be (endfreq-startfre+1) row,
The matrix of 7 row.
Matrix decoder can be used for controlling the frequency domain index range of watermark signal detection.It is accurate to improve watermark signal extraction
True rate, according to the maximum frequency domain index range [startfre, endfreq] of intermediate frequency subband signal number m~n settings, startfre
=m-3 indexes starts frequency for frequency domain, and endfreq=n+3 terminates frequency for frequency domain index.The son of each watermark signal detection
Band signal number is maintained as n-m+1, i.e., so that matrix decoder be decoder (：, 1)=[m-3 n-3]T, decoder
(：, 2)=[m-2 n-2]T, decoder (：, 3)=[m-1 n-1]T, decoder (：, 4)=[m n]T, decoder (：, 5)=
[m+1 n+1]T, decoder (：, 6)=[m+2 n+2]T, decoder (：, 7)=[m+3 n+3]T, therefore watermark signal detects
Subband signal sum be (n+3)-(m-3)+1=n-m+7.
Startfre and endfreq can be with self-defining, however it is not limited to which the above-mentioned setting value provided, general startfre are wanted
It asks and is less than m, endfreq requires to be more than n, can expand frequency domain search range in this way.
Step 4.3, generated time index matrix pointe.
In this specific implementation, time index matrix pointe is the matrix of 42 rows, 7 row, for controlling watermark signal detection
Time index range.Anti- desynchronization attack to improve audio signal is equivalent to using repeated encoding thought by 10 × 6
Continuous sample point is embedded in a binary bits value, then an embedded watermark symbol needs (10 × 6) × 7 sample point.Cause
This, when extracting watermark signal, first, defines zoom factor scalefactor, corresponds to seven kinds of different zoom scale, be with 1
Center takes three values respectively on 1 left side and the right, such as 0.9,0.93,0.97 and 1.03,1.07,1.1,0.9,0.93,
0.97,1,1.03,1.07,1.1 constitutes zoom factor corresponding with 7 kinds of zoom scale；Then, according to zoom factor
Scalefactor and current sample point serial number number calculate time index matrix pointer element values, each in time index matrix
Row represent a kind of corresponding index value of zoom scale.Zoom scale quantity sets itself, general zoom scale quantity set are got over
More, the accuracy of extraction is higher, but can increase calculate time and complexity simultaneously, and when specific implementation can set according to actual demand
Suitable zoom scale quantity.
Frame rows in time index matrix pointer, timeindex column element values pointer (frame,
Timeindex) it is：
Pointer (frame, timeindex)=((1+ (0.2+frame) * scalefactor*
blocksperwindow/FRAMESPERWINDOW)/searchstep
Wherein, timeindex values in [1,7] range, corresponding 7 kinds of zoom factor scalefactor；Frame [1,
42] value in range；Blocksperwindow, FRAMESPERWINDOW are constant, and blocksperwindow is zoom scale
The product of quantity, spread spectrum sequence matrix lookup columns and continuous sample points, i.e. 7 × 6 × 10=420；
FRAMESPERWINDOW is the product of zoom scale quantity and spread spectrum sequence matrix lookup columns, i.e. 7 × 6=42；
Searchstep is step-size in search, for controlling computation complexity, generally takes the integer in 1~4.
Frequency indices matrix and time index matrix are set, is on the one hand to resist desynchronization attack, is on the other hand
In order to resist the Compression and Expansion attack of signal.
Step 4.4, loader cycle buffering area load_buffer, for storing pending parameter.
Above-mentioned pending parameter refers to the energy accumulation of every 6 sample points in the intermediate frequency subband signal of embedded watermark signal
With, for example, be respectively the energy accumulation of 1,2,3,4,5, the 6 corresponding intermediate frequency subband signal of sample point by serial number, i.e., it is pending
Parameter.
Step 4.5, pending parameter normalization maximum correlation is obtained.
The specific implementation mode of this step is as follows：
(1) correlation is calculated, it is former using covariance to the value of each time scale, frequency domain and watermark
It manages to calculate the correlation of every case.
Element is 1 or 0 in watermark matrixes, right under each time scale and each frequency domain
The element that watermark matrixes i-th arrange, by the quantity of wherein the sum of 1 corresponding ability value of element divided by element 1, meanwhile, it will
The quantity of the sum of 0 corresponding ability value of element divided by element 0, the two is subtracted each other, that is, obtains each case in watermark matrixes
Under corresponding correlation.
(2) the corresponding i of maximum related value, time index pointer timeindex, frequency indices pointer freqindex are obtained
With maximum related value maximum.
(3) maximum related value is normalized, i.e., maximum related value is subtracted after correlation average value divided by normalized
Standard deviation.
Step 4.6, according to the corresponding i of maximum related value, time index pointer timeindex, frequency indices pointer
Freqindex calculates watermark value usage to be selected with covariance formula, when putting down for continuous three normalized maximum related values
Mean value is more than threshold value, then judges there is watermark at this time；And when continuously the corresponding i values of maximum related value are equal three times, then it is assumed that inspection
Watermark is measured, at this point, current i, which is assigned to CCI, usage, is assigned to LOAD.If it is not detected that watermark, is returned to step (4),
It reloads buffering area.
Threshold value is the compromise of false alarm rate and omission factor, and threshold value is bigger, and omission factor is bigger, and false alarm rate is smaller；Threshold value is smaller, leakage
Inspection rate is smaller, and false alarm rate is bigger；Therefore need that suitable threshold value is arranged according to actual conditions and simulated experiment so that false alarm rate and
Omission factor is smaller.So-called false alarm rate is exactly that the probability of watermark is detected in the case of no-watermark；Omission factor is exactly to have watermark
Situation detects the probability for watermark of haunting.
Claims (5)
1. the television program interaction participatory approaches based on audio frequency watermark, which is characterized in that including step：
Step 1, it is embedded in the watermark signal containing interactive information to television programme audio signal；
Step 1 further comprises sub-step：
Step 1.1, gammatone analysis filtering is carried out to television programme audio signal and obtains N number of subband signal, and from N number of son
Intermediate frequency subband signal is chosen in band signal, i.e., a subband signals of m to n in N number of subband signal；
Step 1.2, the even number bit sign of watermark signal is stored to CCI, odd number bit sign is stored to LOAD, and uses binary system
Indicate symbol in CCI and LOAD；
Step 1.3, the matrix chip that size is N ╳ M being generated according to watermark signal, element initial value is set as 0 in matrix chip,
M is equal to the number of bits of symbol and the product of lines in watermark signal, and lines indicates the row number of lookup：
1.3a defines the lookup of size rows ╳ lines, is made of 0 and 1,0 and 1 number in every a line and each row
Equal, rows and lines are even number；
1.3b, it is m to enable zidai initial values, and frame initial values are 1；M is the intermediate frequency subband signal number minimum value chosen,
The row number and column number of zidai and frame difference representing matrixes chip；
1.3c, if (frame-1) can be divided exactly by lines, generates the random integers in [1, rows], using this to current frame
Random integers update spread spectrum sequence matrix lookup current line numbers p_lut；If (frame-1) cannot be divided exactly by lines, keep expanding
The current p_lut of spectral sequence matrix lookup are constant；
1.3d, by the element value and LOAD of pth in spreading sequence matrix lookup _ lut row (frame-1) %lines+1 row
In the position [(frame-1)/lines+1] binary number it is different or, and exclusive or result to be assigned to the zidai rows of matrix chip
Frame is arranged, and [] indicates rounding；
1.3e enables frame=frame+1, cycle execute step 1.3c~1.3d, until frame is more than M；
1.3f enables zidai=zidai+1, enables frame=1, and cycle executes step 1.3c~1.3e, until zidai>n；
Step 1.4, intermediate frequency subband signal amplitudes are modulated using matrix chip, obtain N number of subband of embedded watermark signal
Signal synthesizes through gammatone and obtains the time-domain audio signal containing interactive information；
Step 2, the television programme audio signal of embedded watermark signal is played using playing device；
Step 3, the television programme audio signal of the lower insertion watermark signal played is recorded using mobile terminal device；
Step 4, watermark signal is extracted from the television programme audio signal of the insertion watermark signal under mobile terminal device record, adopted
Television program interaction can be participated in by connecting the interactive network address in watermark signal with mobile terminal device；
Extraction watermark letter in the television programme audio signal of the insertion watermark signal under the record of slave mobile terminal device described in step 4
Number, further comprise sub-step：
Step 4.1, structure includes the three-dimensional matrice watermark (bit, frame, i) of all possible watermark signal combination, tool
Body is：
4.1a defines the spreading sequence matrix lookup of size rows ╳ lines, is made of 0 and 1, per a line and each row
In 0 and 1 number it is equal, rows and lines are even number；
4.1b, it be 1, frame initial values is 1 that enable i initial values, which be 0, bit initial values,；
4.1c, if (frame-1) can be divided exactly by lines, generates the random integers in [1, rows], using this to current frame
Random integers update spread spectrum sequence matrix lookup current line numbers p_lut；If (frame-1) cannot be divided exactly by lines, keep expanding
The current p_lut of spectral sequence matrix lookup are constant；By spreading sequence matrix lookup pths _ lut row (frame-1) %lines
The element value of+1 row is assigned to watermark (bit, frame, i)；
4.1d enables frame=frame+1, cycle execute step 4.1c, until frame is more than M, M is symbol in watermark signal
Then the product of number of bits and lines executes step 4.1e；
4.1e, enables bit=bit+1, frame=1, and cycle executes step 4.1c~4.1d, until bit is more than (n-m+1), so
Afterwards, step 4.1f is executed；
4.1f enables i=i+1, bit=1, frame=1, cycle execute step 4.1c~4.1e, until i is equal to 2a, a is binary system
Digit；
Step 4.2, each in frequency domain index matrix decoder of the definition for controlling frequency domain, frequency domain index matrix decoder
Row represent a kind of frequency domain index range, and the starts frequency and end frequency of frequency domain index range are numbered according to intermediate frequency subband to be set；
Step 4.3, it is respectively arranged in time index matrix pointe of the definition for controlling time domain scale, time index matrix pointe
Represent a kind of corresponding time index values of time-scaling scale；
Step 4.4, according to the principle of general covariance, matrix under each frequency domain index range and time-scaling scale is obtained
The correlation of element in watermark respective columns takes the corresponding i of maximum related value, time index pointer timeindex, frequency
Index point freqindex normalizes maximum related value with maximum related value；
Step 4.5, according to the corresponding i of maximum related value, time index pointer timeindex, frequency indices pointer
Freqindex calculates watermark value usage to be selected, when the sum of continuous three normalized maximum related values with covariance formula
More than threshold value, and when continuously the corresponding i values of maximum related value are equal three times, then it is assumed that detect watermark；Otherwise, it re-executes
Step 4.4.
2. the television program interaction participatory approaches based on audio frequency watermark as described in claim 1, it is characterised in that：
The generation of random integers in sub-step 1.3c in [1, rows] is by calling random number generator in matlab to realize, tool
Body is：
RandStream functions are called to initialize random number generator seed according to current sign in current window serial number and CCI.
3. the television program interaction participatory approaches based on audio frequency watermark as described in claim 1, it is characterised in that：
The generation of random integers p_lut in step 4.1c in [1, rows] is real by calling random number generator in matlab
It is existing, specially：
RandStream functions are called to initialize random number generator seed according to current i and current window serial number number window.
4. the television program interaction participatory approaches based on audio frequency watermark as described in claim 1, it is characterised in that：
Step 4.2 is specially：
According to intermediate frequency subband signal head numbers m and last number n, maximum frequency domain index range [startfre, endfreq] is set,
Startfre=m-b indexes starts frequency for frequency domain；Endfreq=n+c terminates frequency for frequency domain index；B, c is according to warp
Test the natural number of setting；The frequency domain index range that representative is respectively arranged in frequency domain index matrix decoder is not more than maximum frequency domain index
Range.
5. the television program interaction participatory approaches based on audio frequency watermark as described in claim 1, it is characterised in that：
Step 4.3 is specially：
Time-scaling factor scalefactor is defined, time rope is calculated according to the time-scaling factor and current sample point serial number number
Draw matrix pointer element values, wherein frame rows, timeindex column element values pointer (frame, timeindex)
For：
Pointer (frame, timeindex)=(1+ (0.2+frame) * scalefactor*blocksperwindow/
FRAMESPERWINDOW)/searchstep
Wherein, blocksperwindow, FRAMESPERWINDOW be constant, blocksperwindow be zoom scale quantity,
The product of spread spectrum sequence matrix lookup columns and continuous sample points, FRAMESPERWINDOW are zoom scale quantity and expansion
The product of spectral sequence matrix lookup columns；Searchstep is step-size in search.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410647192.5A CN104320719B (en) | 2014-11-14 | 2014-11-14 | Television program interaction participatory approaches based on audio frequency watermark and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410647192.5A CN104320719B (en) | 2014-11-14 | 2014-11-14 | Television program interaction participatory approaches based on audio frequency watermark and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104320719A CN104320719A (en) | 2015-01-28 |
| CN104320719B true CN104320719B (en) | 2018-09-07 |
Family
ID=52375876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410647192.5A Active CN104320719B (en) | 2014-11-14 | 2014-11-14 | Television program interaction participatory approaches based on audio frequency watermark and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104320719B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105392022B (en) * | 2015-11-04 | 2019-01-18 | 北京符景数据服务有限公司 | Information interacting method and device based on audio frequency watermark |
| CN105374360B (en) * | 2015-11-25 | 2018-12-14 | 武汉大学 | Intersect additivity spread spectrum audio frequency watermark embedding grammar, detection method and system |
| CN105635841A (en) * | 2015-12-28 | 2016-06-01 | 北京正奇联讯科技有限公司 | Interaction broadcast control method and system |
| CN105791973A (en) * | 2016-03-07 | 2016-07-20 | 大连乐云信息技术有限公司 | Resolving method and resolving device based on sound wave watermark |
| CN105916040A (en) * | 2016-05-18 | 2016-08-31 | 北京正奇联讯科技有限公司 | Trigger method and system of secondary event in television broadcasting |
| CN108712666B (en) * | 2018-04-04 | 2021-07-09 | 聆刻互动（北京）网络科技有限公司 | Interactive audio watermark-based mobile terminal and television interaction method and system |
| CN111190518B (en) * | 2019-12-30 | 2022-05-17 | 中央电视台 | Interaction method and device between first screen and second screen, terminal and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103428538A (en) * | 2013-08-12 | 2013-12-04 | 广州信为信息科技有限公司 | Method, device and system for interaction of interactive broadcast televisions |
| CN103763578A (en) * | 2014-01-10 | 2014-04-30 | 北京酷云互动科技有限公司 | Method and device for pushing program associated information |
| CN103985387A (en) * | 2014-04-17 | 2014-08-13 | 苏州乐聚一堂电子科技有限公司 | Audio instruction and intelligent control method thereof |
-
2014
- 2014-11-14 CN CN201410647192.5A patent/CN104320719B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103428538A (en) * | 2013-08-12 | 2013-12-04 | 广州信为信息科技有限公司 | Method, device and system for interaction of interactive broadcast televisions |
| CN103763578A (en) * | 2014-01-10 | 2014-04-30 | 北京酷云互动科技有限公司 | Method and device for pushing program associated information |
| CN103985387A (en) * | 2014-04-17 | 2014-08-13 | 苏州乐聚一堂电子科技有限公司 | Audio instruction and intelligent control method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104320719A (en) | 2015-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104320719B (en) | Television program interaction participatory approaches based on audio frequency watermark and system | |
| Serizel et al. | Sound event detection in synthetic domestic environments | |
| CN108922518A (en) | voice data amplification method and system | |
| CN102265536B (en) | Methods and apparatus to perform audio watermarking and watermark detection and extraction | |
| CN106340291A (en) | Bilingual subtitle production method and system | |
| US9641363B2 (en) | System for triggering actions on computing devices via audio signals | |
| CN104981869A (en) | Signaling audio rendering information in a bitstream | |
| US10166472B2 (en) | Methods and systems for determining a reaction time for a response and synchronizing user interface(s) with content being rendered | |
| CN110797038B (en) | Audio processing method and device, computer equipment and storage medium | |
| CN108712666B (en) | Interactive audio watermark-based mobile terminal and television interaction method and system | |
| CN103763578A (en) | Method and device for pushing program associated information | |
| CN103246837B (en) | Method and the subscriber equipment of identifying code are provided | |
| CN107578777A (en) | Word-information display method, apparatus and system, audio recognition method and device | |
| CN103618986A (en) | Sound source acoustic image body extracting method and device in 3D space | |
| CN106571145A (en) | Voice simulating method and apparatus | |
| CN108737884A (en) | A kind of content recordal method and its equipment, storage medium, electronic equipment | |
| Fagerström et al. | One-to-many conversion for percussive samples | |
| CN104216612A (en) | Information processing method and electronic equipment | |
| CN107396178A (en) | A kind of method and apparatus for editing video | |
| US10820133B2 (en) | Methods and systems for extracting location-diffused sound | |
| CN109040778B (en) | Video cover determining method, user equipment, storage medium and device | |
| CN104135668B (en) | There is provided and obtain the method and device of digital information | |
| CN107580264A (en) | Multimedia resource play handling method and device | |
| CN104270676B (en) | A kind of information processing method and electronic equipment | |
| JP6367748B2 (en) | Recognition device, video content presentation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |