EP2362388A1

EP2362388A1 - Method and apparatus for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile receiver

Info

Publication number: EP2362388A1
Application number: EP10305181A
Authority: EP
Inventors: Ulrich Gries; Peter Georg Baum; Walter Voessing; Michael Arnold
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2010-02-24
Filing date: 2010-02-24
Publication date: 2011-08-31
Anticipated expiration: 2030-02-24
Also published as: EP2362388B1

Abstract

For determining the audience figures of a TV or radio program, audio watermarking can be used. Panelists are carrying a mobile receiver including a microphone (MIC), whereby the received acoustic signal is checked (WMD) for watermark data. Because movement of the mobile receiver causes a Doppler shift in the received acoustic signal (AWMS), for detecting an embedded watermark a frequency search (FRS) within a given frequency range must be carried out. According to the invention, the current maximum motion speed of the mobile receiver is determined (GSR, ACC, MSPC) and is used for decreasing the frequency range within which the correct frequency needs to be searched.

Description

The invention relates to a method and to an apparatus for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile watermark data receiver, wherein motion of the mobile receiver causing a Doppler shift of the received signal is taken into account.

Background

For determining the audience figures of a TV or radio program, audio watermarking (WM) can be used. In such case a TV or radio broadcaster will embed a unique station ID inside the audio signal transmitted by the transmitter stations. Panelists use a device including a microphone, an audio WM detector and a storage and/or transmission unit for monitoring via the microphone the environmental sound, i.e. the acoustic sound signal carrying the WM and being transmitted by one or more loudspeakers, plus any kind of 'noise' signal occurring in the shop or airport hall etc. If a TV set or radio is turned on, the embedded WM is detected and the station ID is decoded and stored in the storage unit, or is directly transmitted by the WM signal receiver device transmission unit to the audience measurement company.

Invention

For detecting the embedded watermark (WM), either a full frequency search must be carried out (which is time and CPU processing power consuming), or an inherently robust frequency shift WM system is used, but such system often does not meet the requirements concerning allowable data rate for the WM feature.
Today's trend is to make use of mobile panelist devices, so that audience measurements can also take place e.g. in shops and airports. But a movement of such mobile device will result in a Doppler shift being imposed on the received sound signal, resulting in a frequency de-synchronisation. Unfortunately, most WM systems are inherently not robust against such frequency de-synchronisation. Therefore the correct frequency shift needs to be found, which normally can be achieved only by carrying out a time consuming and CPU processing power consuming search. This will result in missed WM detection and shorter battery life time, which both are not in the interest of the audience measurement company.
A problem to be solved by the invention is to achieve faster and/or more reliable watermark detection for a mobile WM signal receiver. This problem is solved by the method disclosed in claim 1. An apparatus that utilises this method is disclosed in claim 2. According to the invention, one or more accelerometers and/or a Galileo or GPS system signal receiver are integrated into the mobile WM signal detection device. Inside the WM signal detection device the accelerometer data signals and/or the Galileo or GPS system location data signals are used for calculating an approximation of the device speed and the real Doppler shift caused by the motion of the device. The motion speed approximation value or values are used as an a-priori knowledge in the WM detection processing for reducing the number of candidate frequencies searched and thereby the search time and/or for improving the WM signal detection results and/or for reducing the CPU power consumption.
I.e., advantages of the inventive watermark signal detection are improved WM detection rate and increased battery life time.
Mobile detectors will probably include a GPS or Galileo device anyway for measuring the location of the content exposure. Then advantageously the cost of the WM receiver device will increase only by adding accelerometers and corresponding control software, which are quite cheap.
In principle, the inventive method is suited for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation of said signal a current decoder or demodulation frequency needs to be determined, said method including the steps:

a) based on a current candidate frequency out of a given frequency range, determining from a correlation of a current signal section with one or more reference bit sequences whether or not at least one watermark bit of said signal is present in said current signal section;
b) if true, said current candidate frequency is assumed to be the correct frequency,
and if not true, performing step a) using a different candidate frequency out of said frequency range;
c) if no correct frequency is found after steps a) and b) have been carried out for all candidate frequencies out of said frequency range, assuming that said current signal section does not carry a watermark,
wherein the range of said frequency searching is limited according to current maximum device motion speed data representing a correspondingly limited Doppler shift, which current maximum device motion speed data are derived for said mobile receiver device from Galileo or GPS data and/or from accelerometer data.

In principle the inventive apparatus is suited for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation of said signal a current decoder or demodulation frequency needs to be determined, said apparatus including:

means for receiving said acoustic path signal and for providing a corresponding data signal;
means being adapted for determining, based on a current candidate frequency out of a given frequency range, from a correlation of a current signal section with one or more reference bit sequences whether or not at least one watermark bit of said signal is present in said current signal section, wherein
if true, said current candidate frequency is assumed to be the correct frequency,
and if not true, said determining means perform said watermark bit determination using a different candidate frequency out of said frequency range;
and wherein, if no correct frequency is found in said determining means for all candidate frequencies out of said frequency range, assuming that said current signal section does not carry a watermark;
a Galileo or GPS receiver and/or one or more accelerometers;
means for calculating for said mobile receiver device from corresponding data of said Galileo or GPS receiver and/or said accelerometers a limited range to be applied in the frequency searching in said determining means, according to current maximum device motion speed data representing a correspondingly limited Doppler shift.

Advantageous additional embodiments of the invention are disclosed in the respective dependent claims.

Drawing

Exemplary embodiments of the invention are described with reference to the accompanying drawing, which shows in:

Fig. 1: block diagram of a WM signal receiver device according to the invention.

Exemplary embodiments

In known WM signal receiver devices the whole possible frequency range has to be searched in order to determine the correct frequency shift introduced by the Doppler shift. Therefore the maximum allowable speed or acceleration of the device needs to be specified. If the current speed is outside that specification, the WM will not be detected.
As an example, the maximum speed allowed in the direction to or from the acoustic sound signal transmitter (i.e. loudspeaker) is 3m/s. With 300m/s being the approximated speed of acoustic waves, this will result in a Doppler shift of about ±1%. Assuming that the WM system is robust against a frequency deviation of 0.05%, 2/0.05 +1 = 41 candidate frequencies are to be tested in order to cover the full range from moving with 3m/s towards the loudspeaker via no motion to moving with 3m/s away from the loudspeaker.
This means that in the worst case the CPU power consumption during this frequency search period is 41 times higher than during normal mode in which the correct frequency has been found already. However, for a mobile device this is not acceptable due to battery life time constraints.
Furthermore, the speed of the device with respect to the loudspeaker may change quickly, for example if the person carrying the WM device is walking by a loudspeaker (first towards, then away) or if the person suddenly stops. This means that the resulting frequency search operation will take place quite frequently.
In the inventive device shown in Fig. 1, a microphone MIC receives acoustic signals from a loudspeaker LSP. The microphone output signal passes through an A/D converter and an optional spectral whitening step or stage SPW to a watermark demodulator or decoder stage or step WMD, which carries out - for a given frequency - a correlation with one or more reference bit sequences, and which is controlled by a frequency search step or stage FRS that receives motion speed or Doppler shift data from motion speed calculator or step MSPC. In case a correlation result shows one or more suitable correlation result peaks, a watermark signal bit corresponding to the applied reference bit sequence has been determined. The demodulator/decoder starts operation with a first candidate frequency, tries to decode a watermark. If successful, it is assumed that that candidate frequency is the correct frequency. If not successful, the following or a different candidate frequency is checked, and so on. In case no candidate frequency out of the frequency search range was successful it is assumed that at present, i.e. for a current WM frame or signal section of the received input signal, no watermark is present within the received input signal.
The watermark demodulator or decoder stage or step WMD sends a frequency correct signal FCS to frequency search step or stage FRS for signalling that the current frequency is the correct one.
From the output signals of the embedded accelerometers ACC1 to ACC3 and/or an embedded GPS or Galileo signal receiver GSR, the current speed of the WM signal receiver device is calculated in the motion speed calculator or step MSPC. From the acceleration values the current speed can be calculated by integrating them over a short time period, and based on the previous speed value. When switching on the mobile device the initial speed value can be assumed to be zero. In particular, if no Galileo or GPS signal is evaluated and no positive or negative acceleration is determined for a predetermined time period, the current speed value can be assumed to be zero because otherwise in the meantime the person carrying the mobile device would have left the sound field of the loudspeaker LSP.
The Galileo/GPS receiver can also take into account the current spatial orientation of the WM device and thereby automatically determine the share of the motion speed that is directed to, or away from, the loudspeaker LSP, following a corresponding initialisation taking into account the loudspeaker position. For such initialisation the loudspeaker coordinates can be stored upon placing the WM device nearby the loudspeaker for a short time period, or by automatic transmission of the current loudspeaker coordinates, or by pre-stored loudspeaker coordinates, depending on the current application of the WM signal receiver device.
In case accelerometers only are used (three one-dimensional accelerometers or two two-dimensional accelerometers or a single three-dimensional accelerometer), the position of the loudspeaker is unknown. By using the motion speed data in calculator MSPC, the current maximum possible Doppler shift can be calculated for the frequency search in frequency search step or stage FRS. This estimated Doppler shift data will normally not correspond to the real Doppler shift data because the maximum shift is reached only if the movement is directly towards or away from the loudspeaker. If the movement is more tangential to the loudspeaker position, the real Doppler shift is lower, depending on the distance to the loudspeaker.
However, the knowledge of the current maximum Doppler shift will already significantly reduce the frequency range to be searched because the maximum allowable frequency shift (i.e. motion speed) according to the device specification is rarely reached in real life. If, for example for the person carrying the WM signal receiver device a walking speed of 1m/s is detected, a maximum Doppler shift of 0.3% is calculated and 13 candidate frequencies only instead of 41 candidate frequencies need to be tested.
As an alternative, the spatial orientation of the WM signal receiver device with respect to the loudspeaker can be determined (in a spatial orientation detector SPOD that provides the motion speed calculator MSPC with corresponding data. Such determination of the spatial orientation of the WM signal receiver device (which is comparable to the initialisation mentioned above) can be achieved by using an additional optical signal received from (the location of) the loudspeaker LSP, or by using several (directional) microphones in the WM device for determining from which average or main direction the sound is received by these microphones. Instead of a single corresponding initialisation, a repeated or periodic or continuous determination of the spatial orientation of the WM signal receiver device can be carried out.
With or without loudspeaker location determination, even more can be gained in case the WM signal receiver device is not moving and only non-watermarked sound is received. This will happen very frequently in real life, for example if the person carrying the device is sitting somewhere and no TV or radio broadcast signal is playing from loudspeaker LSP, but only some ambient noise is received by microphone MIC. With the known technology all 41 candidate frequencies are to be tested because the WM detector cannot differentiate between non-detection due to no WM signal being embedded and non-detection due to wrong candidate frequency selection. But in the inventive WM signal receiver device a single frequency only is to be tested, which represents an improvement by a factor of 41.
The station or program ID and/or the watermark data can be stored in a memory MEM for a future upload to the audience measurement data company. As an alternative, these data can be transmitted via a transmitter TR to a corresponding audience measurement data receiver that is operated for that company.
The calculated speed and acceleration data can also be used for adapting other parameters of the WM detection processing, for example the correlation length.

Claims

Method for evaluating a possibly watermarked signal (AWMS) that is received (MIC) via an acoustic path (LSP, MIC) in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation (WDM) of said signal a current decoder or demodulation frequency needs to be determined (FRS), said method including the steps:
a) based on a current candidate frequency out of a given frequency range, determining (WMD) from a correlation of a current signal section with one or more reference bit sequences whether or not at least one watermark bit of said signal (AWMS) is present in said current signal section;

b) if true, said current candidate frequency is assumed to be the correct frequency,
and if not true, performing step a) using a different candidate frequency out of said frequency range;

c) if no correct frequency is found after steps a) and b) have been carried out for all candidate frequencies out of said frequency range, assuming that said current signal section does not carry a watermark,
characterised in that the range of said frequency searching (FRS) is limited according to current maximum device motion speed data representing a correspondingly limited Doppler shift, which current maximum device motion speed data are derived (MSPC) for said mobile receiver device from Galileo or GPS data (GSR) and/or from accelerometer data (ACC1, ACC2, ACC3).
Apparatus for evaluating a possibly watermarked signal (AWMS) that is received (MIC) via an acoustic path (LSP, MIC) in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation (WDM) of said signal a current decoder or demodulation frequency needs to be determined (FRS), said apparatus including:
- means (MIC, A/D) for receiving said acoustic path signal (AWMS) and for providing a corresponding data signal;

- means (WMD) being adapted for determining, based on a current candidate frequency out of a given frequency range, from a correlation of a current signal section with one or more reference bit sequences whether or not at least one watermark bit of said signal (AWMS) is present in said current signal section, wherein
if true, said current candidate frequency is assumed to be the correct frequency,
and if not true, said determining means (WMD) perform said watermark bit determination using a different candidate frequency out of said frequency range;
and wherein, if no correct frequency is found in said determining means (WMD) for all candidate frequencies out of said frequency range, assuming that said current signal section does not carry a watermark;
characterised by:
- a Galileo or GPS receiver (GSR) and/or one or more accelerometers (ACC1, ACC2, ACC3);

- means (MSPC, FRS) for calculating for said mobile receiver device from corresponding data of said Galileo or GPS receiver and/or said accelerometers a limited range to be applied in the frequency searching in said determining means (WMD), according to current maximum device motion speed data representing a correspondingly limited Doppler shift.
Method according to claim 1, or apparatus according to claim 2, wherein prior to said decoding or demodulation (WDM) a spectral whitening step or stage (SPW) is carried out or arranged, respectively.
Method according to claim 1 or 3, or apparatus according to claim 2 or 3, wherein a single frequency only out of said frequency range is checked in case said mobile receiver device is not moving according to the data from said Galileo or GPS receiver (GSR) and/or said one or more accelerometers (ACC1, ACC2, ACC3).
Method according to claim 1, 3 or 4, or apparatus according to one of claims 2 to 4, wherein, when determining (MSPC) said current maximum device motion speed data, said Galileo or GPS receiver (GSR) also takes into account the current spatial orientation of said mobile device and thereby automatically determines the share of the motion speed that is directed to - or away from - a source (LSP) of said acoustic path, following a corresponding initialisation taking into account the position of said source.
Method according to claim 5, or apparatus according to claim 5, wherein for such initialisation the coordinates of said source (LSP) are stored upon placing said mobile device nearby said source for a short time period, or by automatic transmission of the current coordinates of said source, or by pre-stored source coordinates.
Method according to claim 5 or 6, or apparatus according to claim 5 or 6, wherein a determination of the spatial orientation of said mobile device is carried out repeatedly or in a periodic or continuous manner.
Method according to claim 1, 3 or 4, or apparatus according to one of claims 2 to 4, wherein, when determining (MSPC) said current maximum device motion speed data, the current spatial orientation of said mobile device is also taken into account and the share of the motion speed that is directed to - or away from - a source (LSP) of said acoustic path is thereby automatically determined, wherein said current spatial orientation of said mobile device with respect to said source is determined (SPOD) by using an additional optical signal received in said mobile device from the location of - or a location near to - said source (LSP), or by using several microphones in said mobile device for determining from which average or main direction the acoustic signal is received by these microphones.
Method according to one of claims 1 and 2 to 8, or apparatus according to one of claims 2 to 8, wherein station or program ID data and/or watermark data received by said mobile device are stored in a memory (MEM).
Method according to one of claims 1 and 2 to 9, or apparatus according to one of claims 2 to 9, wherein station or program ID data and/or watermark data received by said mobile device are transmitted via a transmitter (TR) to a corresponding audience measurement data receiver that is operated for an audience measurement data collecting company.