JP6056260B2 - Sound processing apparatus, sound processing method and program - Google Patents

Description

  The present invention relates to a sound processing apparatus, a sound processing method, and a program for processing an acoustic signal in which a watermark signal is embedded.

  An information watermarking technique for embedding information in an area that cannot be perceived by humans with respect to an acoustic signal is known. Information watermarking technology for acoustic signals is expected to be applied to copyright information and data distribution, and various methods have been proposed. For example, Echo-Hiding is widely known (see, for example, Patent Document 1).

  Echo hiding is a technique that divides an acoustic signal into unit time, superimposes an artificial echo that is difficult for humans to perceive in each section, and expresses information using the delay time of the echo. FIG. 1 is a diagram illustrating an example of an information watermark by echo hiding.

  For example, as shown in FIG. 1, the delay times corresponding to the symbols “00”, “01”, “10”, and “11” are 0.1 msec, 0.2 msec, 0.3 msec, and 0.4 msec, respectively. . At this time, when one of the four echoes is superimposed on each section, each section in which the artificial echo is embedded expresses 2-bit information.

When extracting data from an acoustic signal, a feature value called a cepstrum coefficient sequence is used. The cepstrum coefficient sequence is given by the following equation (1).
Cepstrum = IFFT (log (abs (FFT (X)))) (1)
X: vector of digitally recorded audio samples FFT: function for performing Fourier transform abs: absolute value acquisition log: logarithm (base of logarithm is arbitrary)
IFFT: Function for performing inverse Fourier transform A vector sequence obtained by equation (1) is called a cepstrum coefficient sequence, and is used to analyze whether an input sound source includes superimposed sound.

  FIG. 2 is a diagram illustrating an example of a cepstrum analysis result. It is assumed that an echo sound delayed by time T is superimposed on the input sound source. In this case, as a result of the cepstrum analysis, a sharp peak can be observed at the position T in the cepstrum space. The horizontal axis of the graph shown in FIG. 2 is referred to as quefrency with respect to the time axis.

  The extraction process on the reception side uses the premise that the delay time of an artificial echo that can be embedded in advance is known. The quefrency having the maximum value is selected from the cepstrum coefficients at the position corresponding to the artificial echo group, and it is determined that the symbol corresponding to the quefrency (delay time) is embedded. In the case shown in FIG. 1, the positions corresponding to the artificial echo group are 0.1 msec, 0.2 msec, 0.3 msec, and 0.4 msec.

  In recent years, the power of mobile phones, tablet PCs (Personal Computers), PCs, and the like has become lower and lower due to MEMS (Micro Electro Mechanical Systems) sensors and low-power microcontrollers. On the other hand, since the digital data of the acoustic signal has a higher sampling rate than other sensors such as an acceleration sensor and a temperature sensor, the processing load is relatively high. Here, there is a technique for controlling the sampling rate by comparing the received signal power and the threshold for each sampling (see, for example, Patent Document 2).

JP 2012-37701 A JP 2004-165929 A

  In order to calculate a cepstrum for watermark extraction, high load processing such as FFT is repeatedly executed. Therefore, there is a problem that the power consumption of the apparatus that executes the extraction process increases, and the operation time of the apparatus decreases.

  Therefore, in order to perform watermark extraction processing such as echo hiding in real time on devices that require low power consumption such as mobile terminal devices, the processing amount can be reduced without reducing the extraction accuracy, and power can be saved. It is good to plan. Generally, when the sampling rate is increased, the extraction accuracy is increased but the power consumption is increased. When the sampling rate is decreased, the power consumption is decreased but the extraction accuracy is decreased.

  However, in the prior art, it was not possible to save power without lowering the accuracy of watermark signal extraction.

  Therefore, the disclosed technique aims to provide a sound processing device, a sound processing method, and a program capable of suppressing power consumption while preventing deterioration of watermark signal extraction accuracy.

  The sound processing device according to one aspect of the disclosure includes an AD conversion unit that converts an acoustic signal on which a watermark signal is superimposed from an analog signal into a digital signal at a predetermined sampling rate, and a predetermined section of the divided digital signal, A determination unit for determining success or failure of extraction of the watermark signal, a storage unit for storing a determination result by the determination unit, and a rate control for controlling the predetermined sampling rate based on the determination result stored in the storage unit A section.

  According to the disclosed technology, it is possible to suppress power consumption while preventing deterioration of the watermark signal extraction accuracy.

The figure which shows an example of the information watermark by echo hiding. The figure which shows an example of a cepstrum analysis result. The block diagram which shows an example of the hardware of the sound processing apparatus in an Example. The block diagram which shows an example of the function of the processor in an Example. The figure explaining the success or failure determination of the extraction of a watermark signal. The figure which shows an example of the state transition of a sampling rate. The figure explaining a synchronous process. The figure which shows an example of the change timing of a sampling rate. 6 is a flowchart illustrating an example of a watermark extraction process in the embodiment. The flowchart which shows an example of the synchronous process in an Example. The figure which shows an example of schematic structure of the sound processing apparatus in a modification.

  Embodiments will be described below with reference to the accompanying drawings.

[Example]
<Configuration>
FIG. 3 is a block diagram illustrating an example of hardware of the sound processing device 10 according to the embodiment. The sound processing apparatus 10 in the embodiment is, for example, a microcontroller for acoustic watermark extraction processing. Note that in the sound processing device 10, an information processing device such as a mobile terminal device including a microcontroller illustrated in FIG. 3, a tablet terminal, a PC, or a game device may be referred to as a sound processing device.

  The sound processing apparatus 10 illustrated in FIG. 3 includes a microphone sensor 102, an ADC (analog / digital converter) 104, a bus 106, a timer 108, a processor 110, and a memory 112. The memory 112 is directly connected to the processor 110, but may be connected via the bus 106.

  As shown in FIG. 3, the processor 110 is connected to the bus 106. A timer 108 and an ADC 104 are connected to the bus 106 as peripheral devices. The microphone sensor 102 is connected to the ADC 104.

  The microphone sensor 102 functions as a receiving unit and receives an analog signal of an acoustic signal on which a watermark signal is superimposed. The acoustic signal is a signal such as voice or music, and data on which a watermark signal can be extracted in real time is superimposed. In the following, echo hiding is used as a technique capable of extracting a watermark signal in real time. The microphone sensor 102 outputs the received acoustic signal to the ADC 104.

  The ADC 104 functions as an AD (analog / digital) converter, for example. The ADC 104 converts the acoustic signal acquired from the microphone sensor 102 from an analog signal to a digital signal at a predetermined sampling rate under the control of the processor 110.

  The timer 108 instructs the processor 110 by sampling the sampling timing. The sampling rate is set in the timer 108. For example, the predetermined sampling rate set in the timer 108 is rewritten by the processor 110.

  The processor 110 acquires the digital signal of the acoustic signal, issues a conversion instruction to the ADC 104 at the interrupt timing by the timer 108, and executes watermark extraction processing. The watermark extraction process can be realized by loading and executing a watermark extraction process program stored in the instruction memory 112a of the memory 112.

  The processor 110 stores the result of the success or failure of the watermark extraction process in the data memory 112b of the memory 112. Detailed processing of the processor 110 will be described later with reference to FIG.

  The memory 112 is a storage unit such as a read only memory (ROM) or a random access memory (RAM), and includes an instruction memory 112a and a data memory 112b. The instruction memory 112a and the data memory 112b have different storage areas in the memory 112, for example.

  The instruction memory 112 a stores a watermark extraction processing program executed by the processor 110.

  The data memory 112b includes a sample buffer for storing a digital data string of the acoustic signal and a success / failure buffer for storing data indicating success / failure information of the watermark extraction process.

  The sample buffer is an area in which, for example, a maximum of 512 samples can be stored. The success / failure buffer is an area for storing, for example, 1-bit information indicating success / failure of extraction processing of 6 sections.

《Processor》
Here, the function of the processor will be described. FIG. 4 is a block diagram illustrating an example of functions of the processor 110 in the embodiment. The processor 110 illustrated in FIG. 4 functions as the acquisition unit 202, the extraction unit 204, and the rate control unit 206 by executing the watermark extraction processing program.

  The acquisition unit 202 acquires a digital signal of an acoustic signal from the ADC 104 via the bus 106 at a predetermined sampling rate. At this time, the acquisition unit 202 samples the digital signal at the interrupt timing by the timer 108. The acquisition unit 202 outputs the acquired digital signal to the extraction unit 204.

  The extraction unit 204 divides this digital signal into predetermined intervals for the digital signal of the acoustic signal acquired at a predetermined sampling rate, and performs watermark extraction processing for each predetermined interval. As illustrated in FIG. 2, the extraction unit 204 performs cepstrum analysis for each predetermined section and extracts a watermark signal corresponding to the delay time.

  At this time, the extraction unit 204 includes a determination unit 242, and the determination unit 242 determines whether a watermark signal is extracted using a cepstrum analysis result for each predetermined section.

  FIG. 5 is a diagram for explaining success / failure determination of watermark signal extraction. As shown in FIG. 5, the success or failure of the extraction is determined by comparing the maximum cepstrum coefficient at the quefrency corresponding to the delay time with a threshold value.

  For example, if the maximum cepstrum coefficient is equal to or greater than the threshold, the determination unit 242 indicates “1” as extraction success, and indicates “0” as extraction failure if the maximum cepstrum coefficient is less than the threshold. The threshold value may be a fixed value set in advance, or may be a value based on the average value and standard deviation of cepstrum coefficients in the section, for example, average value + standard deviation.

  In FIG. 5A, the quefrency corresponding to the delay time, and the quefrency with the maximum cepstrum coefficient is 0.2 msec. Since the cepstrum coefficient of the quefrency is equal to or greater than the threshold value, the determination unit 242 determines that extraction is successful.

  In FIG. 5B, the quefrency corresponding to the delay time, and the quefrency with the maximum cepstrum coefficient is 0.3 msec. Since the cepstrum coefficient of the quefrency is less than the threshold value, the determination unit 242 determines that the extraction has failed.

  As illustrated in FIG. 5, the determination unit 242 determines whether the extraction is successful for each section, and stores the determination result in the success / failure buffer.

  Returning to FIG. 4, the rate control unit 206 controls the sampling rate for sampling the digital signal of the acoustic signal based on the determination result stored in the memory 112. For example, the rate control unit 206 prepares a plurality of sampling rates, selects one of the sampling rates based on the determination results of a predetermined number of sections in the past, and uses the selected sampling rate to sample the timer 108. Rewrite the rate.

  Further, the rate control unit 206 counts the number of successes / failures of the first predetermined number of sections stored in the memory 112. If the number of successes is equal to or greater than the threshold, the rate control unit 206 decreases the predetermined sampling rate, and the number of failures exceeds the threshold. If so, the predetermined sampling rate may be increased. The first predetermined number is, for example, 6, but can be appropriately changed, and the threshold can also be appropriately changed.

  Further, the rate control unit 206 lowers the predetermined sampling rate if the determination results of the most recent second predetermined number of consecutive sections stored in the memory 112 are all successful, and sets the predetermined sampling rate if all are unsuccessful. You may make it raise. The second predetermined number is, for example, 3, but can be appropriately changed.

  Note that the rate control unit 206 does not increase the sampling rate beyond the upper limit when the sampling rate reaches the upper limit. Similarly, when the sampling rate reaches the lower limit, the rate control unit 206 does not lower the lower limit value.

  Accordingly, the rate control unit 206 can adaptively control the sampling rate based on the latest watermark signal extraction result.

(Concrete example)
Next, a specific example of the watermark extraction process in the embodiment will be described. The rate control unit 206 prepares, for example, 32 kHz, 16 kHz, and 8 kHz as the sampling rate of the ADC 104.

  Further, the extraction unit 204 sets a section for extracting a watermark signal as 16 msec. The sampling rate of the ADC 104 is set to 32 kHz, 16 kHz, and 8 kHz as described above. The number of digital samples included in each section is 512, 256, and 128, respectively.

  In addition, the determination unit 242 according to the embodiment dynamically determines a threshold value for the extraction success / failure determination of each section for each section. The determination unit 242 calculates the average value avg and standard deviation sd of the cepstrum coefficient sequence in each section, and sets the threshold value as avg + sd.

  When the value of the cepstrum coefficient corresponding to the delay time of the artificial echo is equal to or greater than this threshold, the determination unit 242 determines that the determination target section is successfully extracted. When the value of the cepstrum coefficient corresponding to the delay time is lower than the threshold value, the determination unit 242 determines that extraction has failed.

  In the embodiment, the success / failure information stored in the success / failure buffer is for six sections, and the sampling rate of the ADC 104 is determined based on the success / failure of extraction of the past six sections as shown in FIG.

  Further, the rate control unit 206 may determine the sampling rate of the ADC 104 without waiting for the accumulation of the success / failure results for six sections if the success is made in three consecutive sections and the failure is made in three consecutive sections. .

  FIG. 6 is a diagram illustrating an example of state transition of the sampling rate. As shown in FIG. 6, the sampling rate is three (32 kHz, 16 kHz, 8 kHz), for example, and starts at 32 kHz. Note that the number of sampling rates is not limited to three but may be plural.

(When the current sampling rate is 32 kHz)
The rate control unit 206 determines that the change condition has been reached when four or more of the six determination results stored in the success / failure buffer indicate failure, or when the last three consecutive determination results indicate failure. If not, leave the sampling rate as is. The fact that the change condition has not been reached means that 3 or 6 determination results are not stored in the success / failure buffer, or that the above conditions for 3 or 6 consecutive determination results are not satisfied.

  Further, the rate control unit 206 sets the sampling rate to 16 kHz when four or more of the six determination results stored in the success / failure buffer indicate success, or when the latest three consecutive determination results indicate success. To decide.

(When the current sampling rate is 16 kHz)
The rate control unit 206 determines the sampling rate to be 32 kHz when four or more of the six determination results stored in the success / failure buffer indicate failure, or when the latest three consecutive determination results indicate failure. To do.

  Further, the rate control unit 206 keeps the sampling rate as it is when the change condition is not reached.

  Further, the rate control unit 206 sets the sampling rate to 8 kHz when four or more of the six determination results stored in the success / failure buffer indicate success, or when the latest three consecutive determination results indicate success. To decide.

(When the current sampling rate is 8 kHz)
The rate control unit 206 determines the sampling rate to be 16 kHz when four or more of the six determination results stored in the success / failure buffer indicate failure, or when the latest three consecutive determination results indicate failure. .

  In addition, the rate control unit 206 determines that when four or more of the six determination results stored in the success / failure buffer indicate success, or when the latest three consecutive determination results indicate success, or the change condition If not, leave the sampling rate as it is.

  As a result, if the watermark signal can be extracted well, the sampling rate can be reduced to reduce power consumption. If the watermark signal cannot be extracted properly, the sampling rate can be increased to increase the extraction accuracy. . Thus, with the above-described configuration, the disclosed technology can achieve power saving.

<Synchronous processing>
In the watermark extraction process, the processor 110 performs a synchronization process in order to increase the extraction accuracy. FIG. 7 is a diagram for explaining the synchronization processing. FIG. 7A shows the original signal and the attenuated artificial echo. As shown in FIG. 7A, on the transmitting side, in addition to echoes (0.1 msec, 0.2 msec, 0.3 msec, 0.4 msec) indicating the watermark signal (00, 01, 10, 11), An echo having a delay time (0.5 msec) corresponding to the dedicated symbol 'S' is superimposed. The dedicated symbol for synchronization is also referred to as a synchronization symbol. The transmitting side periodically inserts a section in which the synchronization dedicated symbol 'S' is superimposed.

  FIG. 7B shows an input sound source. Assume that the input sound source shown in FIG. 7B is received by the receiving side (sound processing apparatus 10).

  FIG. 7C shows processor synchronization processing. For example, as shown in FIG. 7C, the receiving side executes watermark extraction processing in a small section having a width (for example, 4 msec) narrower than the width of each 16 msec section. The receiving side detects a change point between the small interval of the synchronization symbol 'S' and the small interval of the data symbol ('00', '01', '10', '11'), and the aircraft point that is the boundary of the interval Recognize.

  FIG. 7D shows the starting point of the processor watermark extraction process. As shown in FIG. 7D, the reception side divides the received acoustic signal in units of 16 msec with reference to the synchronization point.

  Thereby, the watermark extraction process of the processor 110 can be synchronized with the acquisition of the sound source. The synchronization process may be performed periodically or may be performed only once.

<Change timing>
When it is necessary to change the sampling rate of the ADC 104, the processor 110 acquires all the sample data of the section currently being stored and then changes the rate of the ADC 104.

  This is because an interrupt is generated from the timer 108 even during the cepstrum extraction process, so if the sampling rate of the ADC 104 is changed immediately, sample data acquired at different sampling rates within one section will be mixed. Therefore, normal watermark extraction cannot be performed.

  FIG. 8 is a diagram illustrating an example of the change timing of the sampling rate. As shown in FIG. 8, the timing for changing the sampling rate of the ADC 104 is the head of the next section 7 of the section immediately following the section 5 for which success / failure has been determined (the section 6 currently being sampled).

  In the example illustrated in FIG. 8, since four of the sections 0 to 5 have been successfully extracted, the rate control unit 206 controls to downsample the sampling rate of the ADC 104. At this time, since the section 6 is already being sampled, the rate control unit 206 rewrites the sampling rate of the timer 108 so that the sampling rate is changed from the section 7.

  Thereby, the rate control unit 206 can control the sampling rate at an appropriate timing.

<Restoring the watermark signal>
In actual processing, there is a high possibility that accurate data is not extracted from the frame determined as NG in the success / failure determination. Therefore, it is preferable to introduce a mechanism for restoring the superimposed data string from the watermark extraction result.

  The watermark extraction accuracy may be improved by increasing the frequency resolution by taking a large number of samples in one frame, but the data transfer rate is inversely proportional to the number of samples in one frame.

  Therefore, it is desirable to embed a watermark signal to which an error correction signal is added in an acoustic signal after performing an error correction process on the watermark signal sequence itself to be superimposed. On the receiving side, by processing the error correction signal of the extracted watermark signal sequence, it is possible to completely restore the original watermark signal with respect to missing bits up to a certain level.

  By using a technique close to the Shannon limit, such as LDPC (low density parity check code), it is possible to suppress a decrease in data transfer rate as much as possible. On the other hand, in a microcontroller that operates at a level of several mW and requires low power, the correction capability is inferior to that of LDPC, but it is desirable to apply an error correction algorithm such as a convolutional code with high speed and a small amount of processing.

<Operation>
Next, the operation of the sound processing apparatus 10 in the embodiment will be described. FIG. 9 is a flowchart illustrating an example of a watermark extraction process in the embodiment. In step S101, the processor 110 performs a synchronization process. The synchronization process will be described later with reference to FIG.

  In step S102, the processor 110 performs an initialization process. For example, the processor 110 sets Rate (sampling rate), which is a variable of the timer rate, to 32 kHz, and sets Sample, which is a variable of the number of one-frame samples, to 512.

  In step S103, the processor 110 operates the timer 108. In step S <b> 104, the processor 110 issues an AD conversion request to the ADC 104 when an interrupt occurs periodically according to the Rate (32 kHz) set in the timer 108. The processor 110 reads out and acquires the conversion result from the ADC 104, and stores the acquired data in the sample buffer.

  The processor 110 counts the number of interruptions and determines whether or not one frame of sample data has been stored. For example, one frame is also referred to as one section. When sample data for one frame is stored (step S104—YES), the process proceeds to step S105. When sample data for one frame is not stored (step S104—NO), the process returns to step S104.

  In step S105, the processor 110 (extraction unit 204) performs watermark extraction processing on the sample data for one frame in accordance with the sampling rate set in Rate. The processor 110 performs data watermark extraction processing using the time of the first sample point of the last frame extracted in the synchronization processing (the first frame from which symbols other than 'S' are extracted) as the synchronization start point.

  In step S106, when the watermark extraction for one frame (one section) is completed, the processor 110 discards the sample data string in the sample buffer and empties the buffer.

  In step S107, the processor 110 determines whether there is an instruction to end watermark extraction. The end instruction may be given by the user or embedded as a watermark signal. If there is an end instruction (step S107—YES), the watermark extraction process is ended, and if there is no end instruction (step S107—NO), the process proceeds to step S108.

  In step S108, the processor 110 (determination unit 242) compares the maximum peak of the cepstrum coefficient corresponding to the delay time with the threshold value for the extracted data.

  In step S109, the processor 110 (determination unit 242) stores a success / failure result in the success / failure buffer indicating success if the maximum peak is equal to or greater than the threshold and failure if the maximum peak is equal to or less than the threshold. The threshold is a value that is changed for each section to be determined, for example.

  In step S110, the processor 110 (rate control unit 206) determines whether or not three success / failure results have been stored. If three are stored (step S110-YES), the process proceeds to step S111. If three are not stored (step S110-NO), the process returns to step S104.

  In step S111, the processor 110 (rate control unit 206) determines whether or not all three consecutive success / failure results stored in the success / failure buffer are successful. If all are successful (step S111-YES), the process proceeds to step S117, and if all is not successful (step S111-NO), the process proceeds to step S112.

  In step S112, the processor 110 (rate control unit 206) determines whether or not all three consecutive success / failure results stored in the success / failure buffer have failed. If all are unsuccessful (step S112—YES), the process proceeds to step S120. If all are unsuccessful (step S112—NO), the process proceeds to step S113.

  In step S113, the processor 110 (rate control unit 206) determines whether or not six success / failure results have been stored. If six have been stored (step S113—YES), the process proceeds to step S114, and if not six (step S113—NO), the process returns to step S104.

  In step S114, the processor 110 (rate control unit 206) determines whether or not four or more of the six success / failure results stored in the success / failure buffer are successful. If four or more are successful (step S114-YES), the process proceeds to step S117, and if four or more is not successful (step S114-NO), the process proceeds to step S115.

  In step S115, the processor 110 (rate control unit 206) determines whether or not four or more of the six success / failure results stored in the success / failure buffer have failed. If four or more pieces have failed (step S115—YES), the process proceeds to step S120, and if four or more pieces have not failed (step S115—NO), the process proceeds to step S116.

  In step S116, the processor 110 discards the oldest data (success / failure result) from the success / failure buffer.

  In step S117, the processor 110 (rate control unit 206) determines whether or not Rate is 8 kHz. If Rate == 8 kHz (step S117—YES), the process proceeds to step S116, and if Rate == 8 kHz is not satisfied (step S117—NO), the process proceeds to step S118.

  In step S118, the processor 110 (rate control unit 206) sets Rate = Rate / 2 and Sample = Sample / 2.

  In step S119, the processor 110 (rate control unit 206) discards all data in the success / failure buffer.

  In step S120, the processor 110 (rate control unit 206) determines whether or not Rate is 32 kHz. If Rate == 32 kHz (step S120—YES), the process proceeds to step S116, and if Rate == 32 kHz is not satisfied (step S120—NO), the process proceeds to step S121.

  In step S121, the processor 110 (rate control unit 206) sets Rate = Rate × 2 and Sample = Sample × 2. After step S121, the process proceeds to step S119.

  As a result, power consumption can be efficiently suppressed while observing the success or failure of watermark extraction.

  FIG. 10 is a flowchart illustrating an example of the synchronization process in the embodiment. In step S201 illustrated in FIG. 10, the processor 110 sets synchronization processing. For example, the processor 110 sets Rate to 32 kHz and sets Sample to 128. As shown in FIG. 7C, one frame of the synchronization process is set to, for example, 1/4 of the watermark extraction process.

  In step S202, the processor 110 operates the timer 108. In step S <b> 203, the processor 110 issues an AD conversion request to the ADC 104 when an interruption occurs periodically according to the sampling rate (32 kHz) set in the timer 108. The processor 110 reads out and acquires the conversion result from the ADC 104, and stores the acquired data in the sample buffer.

  The processor 110 counts the number of interruptions and determines whether or not one frame of sample data has been stored. When sample data for one frame is stored (step S203—YES), the process proceeds to step S204, and when sample data for one frame is not stored (step S203—NO), the process returns to step S203.

  In step S204, the processor 110 (extraction unit 204) performs a watermark extraction process on the sample data for one frame.

  In step S205, the processor 110 discards one frame of sample data from the sample buffer.

  In step S206, the processor 110 determines whether or not the result of watermark extraction is 'S', which is a synchronization symbol. If a synchronization symbol is extracted (step S206—YES), the process proceeds to step S207. If a synchronization symbol is not extracted (step S206—NO), the process returns to step S203.

  In step S207, the processor 110 stores the sampling data acquired according to the sampling rate (32 kHz) set in the timer 108 in the sample buffer, and determines whether or not sample data for one frame has been stored. If sample data for one frame is stored (step S207—YES), the process proceeds to step S208. If sample data for one frame is not stored (step S207—NO), the process returns to step S207.

  In step S208, the processor 110 (extraction unit 204) performs a watermark extraction process on the sample data for one frame.

  In step S209, the processor 110 determines whether or not the result of watermark extraction is 'S', which is a synchronization symbol. If a synchronization symbol is extracted (step S209—YES), the process proceeds to step S210. If a synchronization symbol is not extracted (step S209—NO), the synchronization process is terminated.

  In step S210, the processor 110 discards one frame of sample data from the sample buffer.

  That is, when the synchronization symbol is extracted once, it means that it is a synchronization frame, and therefore the processor 110 executes the extraction process until the extraction of the synchronization symbol 'S' is completed. If a symbol other than the synchronization symbol 'S' is extracted, it means that the synchronization frame has ended and the data frame has started. Therefore, after the synchronization process is completed, the process proceeds to step S102 in FIG.

  As described above, the sound processing apparatus 10 can perform watermark extraction while dynamically changing the sampling rate of the ADC 104 based on the success / failure determination of the past extraction results. Thus, the power consumption of the ADC 104 and the processor 110 can be reduced by reducing the sampling rate and the operation rate of the processor while suppressing the deterioration of the watermark extraction accuracy.

<Effect>
In the acoustic processing, the higher the frequency resolution, the higher the accuracy of processing is possible. Therefore, in the watermark extraction processing, the ADC sampling rate greatly affects the extraction accuracy. However, since the power consumption of the ADC is generally proportional to the sampling rate, the power consumption increases as the high extraction accuracy is required.

  Further, in the load of cepstrum calculation, the calculation time of FFT increases as the number of samples increases. For example, the case where the number of samples is 2048 samples is compared with the case where the number of samples in each section of the acoustic signal is 1024 samples. In this case, the FFT calculation time is simply estimated to be 2048 × log 2 (2048) / 1024 × log 2 (1024) = 2.2 times.

  Since the processing performance of the sound processing device is proportional to the operating frequency, 2.2 times more time is required even in the cepstrum calculation processing in which the FFT calculation time is dominant. For example, it is assumed that the cepstrum extraction time of the sound processing apparatus is 20 msec when the number of samples in the section is 1024 samples in a 16 kHz acoustic signal. Under this assumption, if the ADC sampling rate is doubled to 32 kHz in order to increase the extraction accuracy, an approximate extraction time of 44 msec is required.

  In the case of 16 kHz / 1024 samples or 32 kHz / 2048 samples, since the width of each section is 64 msec, sound processing is performed for approximately 68 (44/64)% at 16 kHz and approximately 31 (20/64)% at 32 kHz. The device is idle.

  When the sound processing apparatus is in an idle state, the operating power can be extremely reduced by cutting off the clock supply and power supply of the sound processing apparatus. Therefore, in order to reduce the operating power as much as possible, it is desirable to reduce the ADC sampling rate as much as possible without reducing the extraction accuracy.

  However, it cannot be determined whether the current ADC sampling rate is valid until the watermark is actually extracted from the input acoustic signal. Therefore, in the disclosed technique, it is possible to reduce power consumption while maintaining the extraction accuracy by dynamically changing the ADC from past extraction results that are close in time.

  In particular, the success / failure of watermark extraction varies depending on the tune, and the tune has the property that the same tendency continues for a certain length of time, such as introduction and interlude, so by referring to the extraction results of past sections that are close in time, An appropriate ADC sampling rate can be determined.

  In addition, various noise components such as natural reverberation and noise are superimposed on the recorded acoustic signal. Therefore, it can be said that the extraction result in a state including a noise component has low reliability. Therefore, when the value of the cepstrum peak corresponding to the position of the artificial echo is smaller than the threshold value, the determination unit 242 determines that the strength of the artificial echo has become relatively small due to the influence of noise. . In this case, the determination unit 242 determines that the extraction result is unreliable and can determine failure.

  The extraction accuracy of the watermark may change depending on the tune. The tunes tend to continue for a long period of time, such as introduction, rust, and interlude. Therefore, an appropriate ADC sampling rate can be determined by referring to the extraction results of past intervals that are close in time.

  Therefore, the sound processing device stores the success / failure values of the extraction results from the past N intervals that are close in time, and the current ADC rate is desired from the number of failure and success intervals included therein. It can be determined whether or not the extraction result is valid.

  If there are many successful sections, the current sampling rate of the ADC may be too high, so reducing the sampling rate of the ADC reduces the processing time of the sound processing device and reduces the operating power.

  When there are many failed sections, the current sampling rate of the ADC is too low and the desired extraction accuracy may not be reached. Therefore, the sound processing apparatus 10 increases the sampling accuracy by increasing the sampling rate of the ADC. To improve.

  In this way, the sound processing apparatus 10 dynamically changes the sampling rate of the ADC to optimize the calculation time and operating power of the sound processing apparatus within the limit of the extraction accuracy target, and is unnecessarily high. Processing at the sampling rate can be suppressed.

  Further, when using information on the past N sections, it is desirable to change the ADC sampling rate in response to changes in extraction accuracy as soon as possible. In the disclosed technique, when L success or failure frames are consecutively smaller than N, the ADC sampling rate is immediately changed in order to reduce unnecessary operating power or improve extraction accuracy. Therefore, the ADC sampling rate can be changed in a shorter time without waiting for the success / failure information of N sections to be accumulated. In the above example, N is 6 and L is 3. However, the present invention is not limited to this example.

  It is also conceivable to set N to 1 and change the ADC sampling rate for each section. In this case, the data memory 112b is not an essential configuration, and the rate control unit 206 may perform rate control each time the determination unit 242 determines.

<Modification>
Next, a sound processing apparatus according to a modification will be described.

<Configuration>
FIG. 11 is a diagram illustrating an example of a schematic configuration of a sound processing device 30 according to a modification. The sound processing device 30 illustrated in FIG. 11 includes at least a control unit 302, a main storage unit 304, an auxiliary storage unit 306, a communication unit 308, and a recording medium I / F unit 310. Each unit is connected via a bus so that data can be transmitted / received to / from each other.

  The control unit 302 is a CPU (Central Processing Unit) that performs control of each device, calculation of data, and processing in the computer. The control unit 302 is an arithmetic unit that executes a program stored in the main storage unit 304 or the auxiliary storage unit 306. The control unit 302 receives data from the communication unit 308 or each storage unit, calculates, processes, and outputs the data. And output to each storage unit.

  In addition, the control unit 302 can execute the above-described processing by executing a program for performing any of the above-described processes stored in the auxiliary storage unit 306, for example.

  The main storage unit 304 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like, and a storage device that stores or temporarily stores programs and data such as an OS and application software that are basic software executed by the control unit 302 It is.

  The auxiliary storage unit 308 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software or the like. The auxiliary storage unit 306 may store a program acquired from the recording medium 312 or the like.

  The communication unit 308 performs wired or wireless communication. The communication unit 308 may store, for example, a program transmitted from a server or the like in the auxiliary storage unit 306.

  A recording medium I / F (interface) unit 310 is an interface between the sound processing apparatus 30 and a recording medium 312 (for example, a flash memory) connected via a data transmission path such as a USB (Universal Serial Bus).

  In addition, a predetermined program is stored in the recording medium 312, and the program stored in the recording medium 312 is installed in the sound processing apparatus 30 via the recording medium I / F unit 310. The installed predetermined program can be executed by the sound processing device 30.

  The processor 110 illustrated in FIG. 3 is realized by, for example, the control unit 302, and the memory 112 illustrated in FIG. 3 can be realized by, for example, the main storage unit 304, the auxiliary storage unit 306, and the like.

  It is also possible to record the watermark extraction program on the recording medium 312 and cause the computer or portable terminal to read the recording medium 312 on which the program is recorded, thereby realizing the processing described above.

  The recording medium 312 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, and information is electrically stored such as a ROM or flash memory. Various types of recording media such as a semiconductor memory for recording can be used. The recording medium 312 does not include a carrier wave.

  In addition, the sound processing apparatus described in the above-described embodiments and modifications, for example, superimposes the same content as the announcement on the in-car announcement of a train with a watermark signal, and a person with hearing disabilities displays the announcement indicated by the watermark signal. It is used when displaying on the screen. Thereby, the sound processing device can be applied to knowing the contents of the announcement embedded in the sound, store information, and the like.

  The sound processing device, the sound processing method, and the program disclosed above have been described in detail. However, the present invention is not limited to the above-described embodiments and modification examples. It can be changed. It is also possible to combine all or a plurality of the constituent elements of the above-described embodiments and modifications.

In addition, the following additional remarks are disclosed regarding the above Example.
(Appendix)
An AD converter that converts the audio signal on which the watermark signal is superimposed, from an analog signal to a digital signal at a predetermined sampling rate;
A determination unit that determines success or failure of extraction of the watermark signal for each predetermined section of the divided digital signal;
A storage unit for storing a determination result by the determination unit;
A rate control unit that controls the predetermined sampling rate based on the determination result stored in the storage unit;
A sound processing apparatus comprising:
(Appendix 2)
The rate control unit
The number of successes / failures in the first predetermined number of sections stored in the storage unit is counted. If the number of successes is equal to or greater than a threshold value, the predetermined sampling rate is decreased, and if the number of failures is equal to or greater than the threshold value, the predetermined number is determined. The sound processing apparatus according to appendix 1, wherein the sampling rate is increased.
(Appendix 3)
The rate control unit
Additional remark 1 or 2 if the determination results of the second predetermined number of the most recent consecutive sections stored in the storage unit are all successful, the predetermined sampling rate is lowered, and if all are unsuccessful, the predetermined sampling rate is increased. The sound processing apparatus as described.
(Appendix 4)
The acoustic signal is embedded with the watermark signal by the delay time of the echo to be superimposed,
The determination unit
The success or failure of the extraction is determined by comparing a cepstrum coefficient corresponding to the delay time with a threshold value based on an average value and a standard deviation of the cepstrum coefficient sequence of the predetermined section based on the cepstrum analysis result of the predetermined section. The sound processing apparatus according to any one of appendices 1 to 3.
(Appendix 5)
The audio signal on which the watermark signal is superimposed is converted from an analog signal to a digital signal by an AD converter at a predetermined sampling rate,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
A sound processing method in which a computer executes a process of controlling the predetermined sampling rate based on a determination result stored in the storage unit.
(Appendix 6)
Acquire a digital signal converted at a predetermined sampling rate for an analog acoustic signal with a watermark signal superimposed on it,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
A program for causing a computer to execute processing for controlling the predetermined sampling rate based on a determination result stored in the storage unit.

10 sound processing apparatus 102 microphone sensor 104 ADC
106 Bus 108 Timer 110 Processor 112 Memory 202 Acquisition Unit 204 Extraction Unit 206 Rate Control Unit 242 Determination Unit

Claims (7)

An AD converter that converts the audio signal on which the watermark signal is superimposed, from an analog signal to a digital signal at a predetermined sampling rate;
A determination unit that determines success or failure of extraction of the watermark signal for each predetermined section of the divided digital signal;
A storage unit for storing a determination result by the determination unit;
A rate control unit that controls the predetermined sampling rate based on the determination result stored in the storage unit;
Equipped with a,
The rate control unit
The number of successes / failures in the first predetermined number of sections stored in the storage unit is counted. If the number of successes is equal to or greater than a threshold value, the predetermined sampling rate is decreased, and if the number of failures is equal to or greater than the threshold value, the predetermined number is determined. A sound processing device that increases the sampling rate . An AD converter that converts the audio signal on which the watermark signal is superimposed, from an analog signal to a digital signal at a predetermined sampling rate;
A determination unit that determines success or failure of extraction of the watermark signal for each predetermined section of the divided digital signal;
A storage unit for storing a determination result by the determination unit;
A rate control unit that controls the predetermined sampling rate based on the determination result stored in the storage unit;
With
The rate control unit
The determination result of the continuous interval of the most recent second predetermined number stored in the storage unit, if all successful lowering the predetermined sampling rate, if all failed to increase the predetermined sampling rate Ruoto processor . The acoustic signal is embedded with the watermark signal by the delay time of the echo to be superimposed,
The determination unit
The success or failure of the extraction is determined based on a cepstrum coefficient corresponding to the delay time and a threshold value based on an average value and a standard deviation of the cepstrum coefficient sequence of the predetermined section based on a cepstrum analysis result of the predetermined section. Item 3. The sound processing apparatus according to Item 1 or 2 . The audio signal on which the watermark signal is superimposed is converted from an analog signal to a digital signal by an AD converter at a predetermined sampling rate,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
Based on the determination result stored in the storage unit, the predetermined sampling rate is controlled ,
The number of successes / failures in the first predetermined number of sections stored in the storage unit is counted. If the number of successes is equal to or greater than a threshold value, the predetermined sampling rate is decreased, and if the number of failures is equal to or greater than the threshold value, the predetermined number is determined. A sound processing method in which the computer executes a process of increasing the sampling rate of the sound. The audio signal on which the watermark signal is superimposed is converted from an analog signal to a digital signal by an AD converter at a predetermined sampling rate,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
Based on the determination result stored in the storage unit, the predetermined sampling rate is controlled,
If the determination results of the most recent second predetermined number of consecutive sections stored in the storage unit are all successful, the computer lowers the predetermined sampling rate, and if all the determinations fail, the computer increases the predetermined sampling rate. The sound processing method to perform. Acquire a digital signal converted at a predetermined sampling rate for an analog acoustic signal with a watermark signal superimposed on it,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
Based on the determination result stored in the storage unit, the predetermined sampling rate is controlled ,
The number of successes / failures in the first predetermined number of sections stored in the storage unit is counted. If the number of successes is equal to or greater than a threshold value, the predetermined sampling rate is decreased, and if the number of failures is equal to or greater than the threshold value, the predetermined number is determined. A program that causes a computer to execute a process to increase the sampling rate of a computer. Acquire a digital signal converted at a predetermined sampling rate for an analog acoustic signal with a watermark signal superimposed on it,
For each predetermined section of the divided digital signal, determine the success or failure of extraction of the watermark signal,
The determination result for each predetermined section is stored in a storage unit,
Based on the determination result stored in the storage unit, the predetermined sampling rate is controlled,
If the determination results of the most recent second predetermined number of consecutive sections stored in the storage unit are all successful, the predetermined sampling rate is lowered, and if all are unsuccessful, the predetermined sampling rate is increased. The program to be executed.

Images