TWI740460B - Voice watermark processing method, voice system and voice watermark processing device - Google Patents

Abstract

A voice watermark processing method, a voice system and a voice watermark processing device are provided. The voice watermark processing method includes the following steps. A first voice signal is received during a first time period. A predicted voice signal corresponding a second time period is predicted based on the first voice signal. A voice strength information of the predicted voice signal over time and frequency is analyzed. Based on the voice strength information of the predicted voice signal, a superimposed watermark signal is generated. A second voice signal is received during the second time period. The superimposed watermark signal is played so that the superimposed watermark signal and the second voice signal are superimposed.

Description

Voice floating watermark processing method, voice system and voice floating water Print processing device

The present invention relates to a signal processing method, an electronic system and a processing device, and more particularly to a voice watermark processing method, a voice system and a voice watermark processing device.

For speakers or people who want to protect their own conversation work, it is often forbidden to sniff out, so as to avoid unfriendly sharing. However, the volume of recording devices has been shrinking, and even a variety of camouflage shapes have evolved. In actual situations, it is difficult to completely eliminate the case of skimming.

Furthermore, the conversations of public figures in public may be maliciously edited by interested people, distorting the original meaning of the conversation. The lighter ones destroy the reputation of the interviewer, and the more serious ones may severely disrupt social order.

Therefore, researchers are striving to develop a voice watermark technology, hoping to track the source of the distribution through the voice watermark, and hope to check whether the voice has been maliciously spliced through the voice watermark.

In addition, the voice watermark needs to be added immediately when talking to achieve the actual protection effect. How to achieve real-time processing is also one of the current technical bottlenecks.

The present invention relates to a voice watermark processing method, a voice system and a voice watermark processing device, which can generate an adaptive superimposed watermark signal in real time when the talker or lecturer talks, so that the superimposed watermark signal and the voice signal can be superimposed Overlay. In this way, the other party will record the superimposed watermark signal and voice signal when recording. The superimposed watermark signal contains information such as the standard time or location. If it is spread in the future, the superimposed watermark signal can be analyzed and the source of the spread can be known. And, you can check whether the voice signal has been spliced by superimposing the watermark signal.

According to the first aspect of the present invention, a voice watermark processing method is proposed. The voice watermark processing method includes the following steps. A first voice signal is received in a first time interval. Predicting a predicted voice signal in one of the second time intervals based on the first voice signal. Analyze and predict the speech intensity information of the speech signal over time and frequency. According to the voice intensity information of the predicted voice signal, a superimposed watermark signal is generated. Receive a second voice message in the second time interval No. Play the superimposed watermark signal so that the superimposed watermark signal and the second voice signal are superimposed.

According to the second aspect of the present invention, a speech system is provided. The voice system includes a radio device, a voice watermark processing device and a playback device. The radio device is used for receiving a first voice signal in a first time interval. The voice watermark processing device includes a prediction unit, an analysis unit and a watermark generation unit. The prediction unit is used for predicting a predicted voice signal in a second time interval based on the first voice signal. The analysis unit is used for analyzing and predicting the speech intensity information of the speech signal over time and frequency. The watermark generating unit is used for generating a superimposed watermark signal based on the voice intensity information of the predicted voice signal. The radio device further receives a second voice signal in the second time interval. The playback device is used for playing the superimposed watermark signal so that the superimposed watermark signal and the second voice signal are superimposed.

According to the third aspect of the present invention, a voice watermark processing device is provided. The voice watermark processing device includes a prediction unit, an analysis unit and a watermark generation unit. The prediction unit is used for predicting a predicted voice signal in a second time interval according to a first voice signal received in a first time interval. The analysis unit is used for analyzing and predicting the speech intensity information of the speech signal over time and frequency. The watermark generating unit is used for generating a superimposed watermark signal based on the voice intensity information of the predicted voice signal. The superimposed watermark signal is used for superimposing a second voice signal received in the second time interval.

In order to have a better understanding of the above and other aspects of the present invention, the following specific examples are given in conjunction with the accompanying drawings to describe in detail as follows:

100,100’: Radio device

200,200’: Voice watermark processing device

210: prediction unit

220: Analysis unit

230: watermark generation unit

240, 250: transmission unit

300,300’: playback device

900: Internet

1000,1000’: Voice system

MS, MS2, MS3, MS4: superimposed watermark signal

MS0: Template watermark signal

S110, S120, S130, S140, S141, S142, S143, S144, S150, S160: steps

S1: Voice intensity information

S2: Template strength information

S3: Overlay intensity information

TP1, TP2, TP3, TP4: time interval

VS,VS’,VS1,VS2,VS3,VS4: Voice signal

VS2’, VS3’, VS4’: Predict the voice signal

α : superimposed value

β : predetermined value

Figure 1 is a schematic diagram of a speech system according to an embodiment.

Figure 2 is a schematic diagram of a voice system according to another embodiment.

Figure 3 illustrates the relationship between the voice signal and the superimposed watermark signal.

Figure 4 shows a block diagram of a speech system according to an embodiment.

Figure 5 shows a flowchart of a voice watermark processing method according to an embodiment.

FIG. 6 shows a detailed flowchart of step S140 according to an embodiment.

Figure 7 is a schematic diagram of a template watermark signal according to an embodiment.

Figure 8 shows a schematic diagram of a method for continuously executing voice watermark processing.

Please refer to FIG. 1, which shows a schematic diagram of a speech system 1000 according to an embodiment. In one embodiment, the voice system 1000 is, for example, a system composed of a microphone, a desktop computer, and an amplifier. The voice system 1000 includes a radio device 100, a voice watermark processing device 200, and a playback device 300. In the embodiment of FIG. 1, for example, the speaker is holding the radio device 100 (for example, a microphone) to give a speech. After the voice signal VS is transmitted to the voice watermark processing device 200, the superimposed watermark signal MS is generated. Play the superimposed watermark signal MS and the voice signal VS at the same time through the playback device 300 (for example, an amplifier) to Make the two superimpose. In this way, the listener will record the superimposed watermark signal MS and the voice signal VS when recording. The superimposed watermark signal MS contains information such as standard time or location. If it is dispersed in the future, the superimposed watermark signal MS can be analyzed and the source of the dispersion can be known. In addition, it is possible to check whether the voice signal VS has been spliced by superimposing the watermark signal MS.

Please refer to Figure 2, which shows a schematic diagram of a voice system 1000' according to another embodiment. In one embodiment, the voice system 1000 is, for example, a system composed of a notebook computer and a server. The voice system 1000' includes a radio device 100', a voice watermark processing device 200', and a playback device 300'. In the embodiment shown in Figure 2, for example, employees use laptop computers to conduct video conferences. During the meeting, the employee speaks to the radio device 100' (such as the microphone of a notebook computer), and the voice signal VS' is transmitted to the voice watermark processing device 200' (such as a server) through the network 900, and then superimposed The watermark signal MS'. Then, the superimposed watermark signal MS' and the voice signal VS' are transmitted to the playback device 300' (for example, a speaker of another laptop) through the network to simultaneously play the superimposed watermark signal MS' and the voice signal VS' , Make the two superimpose. In this way, the other party will record the superimposed watermark signal MS’ and the voice signal VS’ when recording. The superimposed watermark signal MS’ contains information such as standard time or location. If it is spread in the future, the superimposed watermark signal MS' can be analyzed and the source of the spread can be known. In addition, it is possible to check whether the voice signal VS' has been spliced by superimposing the watermark signal MS'.

Please refer to Figure 3, which illustrates the relationship between the voice signal VS and the superimposed watermark signal MS. The voice signal VS is continuously captured by the radio device 100. The superimposed watermark signal MS in this embodiment is adaptively fine-tuned according to the content of the voice signal VS (for example, with the voice The intensity of the signal VS is increased/decreased in proportion), rather than a fixed signal, so that noise that conflicts with the voice signal VS can be avoided.

However, since the generation of the superimposed watermark signal MS requires processing time, when the voice system 1000 receives the voice signal VS1 in the time interval TP1, it uses the voice signal VS1 to predict the predicted voice signal VS2' corresponding to the time interval TP2, and then combines it with the template The superimposed watermark signal MS2 is generated after the intersection of the watermark signal MS0. In this way, when the voice signal VS2 is received in the time interval TP2, the real voice signal VS2 and the superimposed watermark signal MS2 can be broadcast at the same time.

Please refer to FIG. 4, which shows a block diagram of a speech system 1000 according to an embodiment. The voice watermark processing device 200 includes a prediction unit 210, an analysis unit 220, a watermark generation unit 230, a transmission unit 240, and a transmission unit 250. The voice watermark processing device 200 is, for example, a desktop computer, a notebook computer, or a remote server. The prediction unit 210, the analysis unit 220, and the watermark generation unit 230 are, for example, a circuit, a chip, a circuit board, a program module, or a memory device storing program codes. The transmission unit 240 and the transmission unit 250 are, for example, a 3.5 mm audio source port, a 6.3 mm audio source port, a wired network transmission unit, or a wireless network transmission module. The voice watermark processing device 200 obtains the predicted voice signal VS2' through the prediction technology, and then intersects the template watermark signal MS0 to generate the superimposed watermark signal MS2. In this way, when the voice signal VS2 is received in the time interval TP2, the voice signal VS2 and the superimposed watermark signal MS2 can be simultaneously broadcast. The following is a flowchart to explain the operation of the above components in detail.

Please refer to Figures 3 to 5. Figure 5 shows a flowchart of a voice watermark processing method according to an embodiment. In step S110, the radio device 100 operates in a first time interval Receive a first voice signal (as shown in FIG. 3, for example, the voice signal VS1 is received in the time interval TP1).

Then, in step S120, the prediction unit 210 predicts a predicted voice signal in a second time interval based on the first voice signal (as shown in Figure 3, for example, it predicts the predicted voice signal corresponding to the time interval TP2 based on the voice signal VS1 VS2').

Then, in step S130, the analysis unit 220 analyzes the voice intensity information of the predicted voice signal over time and frequency (as shown in FIG. 3, for example, analyzes the voice intensity information S1 of the predicted voice signal VS2' over time and frequency). In the time-frequency diagram (the horizontal axis is time, the vertical axis is frequency), different frequencies and different times correspond to different intensities. The darker gray scale represents high intensity, and the lighter gray scale represents low intensity. In this step, the prediction unit 210 obtains the predicted voice signal VS2' through a Long Short Term Memory network (LSTM) algorithm.

Then, in step S140, the watermark generating unit 230 generates a superimposed watermark signal based on the voice intensity information of the predicted voice signal (as shown in Figure 3, for example, based on the voice strength information S1 of the predicted voice signal VS2', the superimposed watermark signal is generated. Watermark signal MS2). Step S140 includes several sub-steps. Please refer to FIG. 6, which shows a detailed flowchart of step S140 according to an embodiment. Step S140 includes steps S141 to S144. In step S141, the watermark generating unit 230 provides a template watermark signal MS0. For example, please refer to FIG. 7, which shows a schematic diagram of a template watermark signal MS0 according to an embodiment. The template strength information S2 of the template watermark signal MS0 presents a text or a pattern in the time-frequency diagram. The text or drawing can be related to the standard time or place point. The template intensity information S2 of the template watermark signal MS0 is a binary pattern (only all black or all white).

In step S142, the watermark generating unit 230 determines whether the voice strength information S1 is higher than a predetermined value β and the template strength information S2 is greater than zero. If the voice strength information S1 is higher than the predetermined value β and the template strength information S2 is greater than 0, then step S143 is entered; if the voice strength information S1 is not higher than the predetermined value or the template strength information S2 is not greater than 0, then step S144 is entered.

In step S143, the watermark generating unit 230 sets the superimposition strength information S3 as a superimposition value α . In one embodiment, the superimposed value α is, for example, a fixed value. In another embodiment, the superimposed value α can also be a proportional value of the speech intensity information S1 (for example, 0.1 times the speech intensity information S1).

In step S144, the watermark generating unit 230 sets the superimposition strength information S3 to 0.

The above steps are, for example, the following formula (1):

Among them, f is the frequency, t is the time, S 3( f,t ) is the superimposed intensity information S3 under each frequency and each time, and S 1( f,t ) is the speech intensity information under each frequency and each time S1, S 2 ( f, t ) are template intensity information S2 under each frequency and each time.

After setting the superimposed intensity information S3 for each frequency and time, the superimposed watermark signal MS2 can be obtained. As shown in Figure 3, the original "t" pattern displayed on the template watermark signal MS0, after the template watermark signal MS0 and the predicted voice signal VS2' have intersection, the superimposed watermark signal MS2 can still roughly see "t "pattern, This pattern is generated in conjunction with the predicted voice signal VS2', and no noise other than the voice signal VS2 will be generated.

Next, in step S150, the radio device 100 receives the second voice signal in the second time interval (as shown in FIG. 3, for example, the voice signal VS2 is received in the time interval TP2).

Then, in step S160, the playback device 300 plays the superimposed watermark signal so that the superimposed watermark signal is superimposed with the second voice signal. As shown in Figure 3, the playback device 300 simultaneously plays the superimposed watermark signal MS2 and the voice signal VS2 corresponding to the time interval TP2. The skimmer will simultaneously record the superimposed watermark signal MS2 and the voice signal VS2 during the skimming.

The above steps are performed repeatedly to continuously play the superimposed watermark signal MS and the voice signal VS. Please refer to Figure 8, which illustrates a schematic diagram of a method for continuously executing voice watermark processing. When the voice watermark processing method is executed for the first time (the first voice signal and the second voice signal are voice signal VS1 and voice signal VS2, respectively), the voice signal VS1 in the time interval TP1 is used to predict the predicted voice signal in the time interval TP2 VS2', and then intersect with the template watermark signal MS0 to generate a superimposed watermark signal MS2.

When the voice watermark processing method is executed again (the first voice signal and the second voice signal are voice signal VS2 and voice signal VS3, respectively), the voice signal VS2 in the time interval TP2 is used to predict the predicted voice signal VS3' in the time interval TP3, Then, after intersecting with the template watermark signal MS0, a superimposed watermark signal MS3 is generated.

Similarly, when the voice watermark processing method is executed again (the first voice signal and the second voice signal are voice signal VS3 and voice signal VS4, respectively), the time interval is used The voice signal VS3 of TP3 is used to predict the predicted voice signal VS4' of the time interval TP4, which is then intersected with the template watermark signal MS0 to generate the superimposed watermark signal MS4, and so on.

According to the above-mentioned embodiment, when the talker or the lecturer is talking, the adaptive superimposed watermark signal MS can be generated in real time, so that the superimposed watermark signal MS and the voice signal VS are superimposed. In this way, the other party will record the superimposed watermark signal MS and the voice signal VS when recording. The superimposed watermark signal MS contains information such as standard time or location. If it is dispersed in the future, the superimposed watermark signal MS can be analyzed and the source of the dispersion can be known. In addition, it is possible to check whether the voice signal VS has been spliced by superimposing the watermark signal MS.

In summary, although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to those defined by the attached patent application scope.

100: Radio device

200: Voice watermark processing device

210: prediction unit

220: Analysis unit

230: watermark generation unit

240, 250: transmission unit

300: playback device

1000: Voice system

MS2: superimposed watermark signal

MS0: Template watermark signal

S1: Voice intensity information

S2: Template strength information

S3: Overlay intensity information

VS1, VS2: Voice signal

VS2’: Predict the voice signal

α: superimposed value

β: predetermined value

Claims (11)

A voice watermark processing method includes: receiving a first voice signal in a first time interval; predicting a predicted voice signal in a second time interval based on the first voice signal; analyzing the predicted voice signal with time and frequency A voice strength information; generate a superimposed watermark signal based on the voice strength information of the predicted voice signal; receive a second voice signal in the second time interval; and play the superimposed watermark signal to make the superimposed watermark The signal is superimposed on the second voice signal; wherein in the step of analyzing the voice intensity information of the predicted voice signal over time and frequency, the predicted voice signal is calculated through a Long Short Term Memory network (LSTM) By the law. The voice watermark processing method according to claim 1, wherein the superimposed watermark signal has superimposed intensity information over time and frequency, and the step of generating the superimposed watermark signal includes: providing a template watermark signal, the template floating The watermark signal has template intensity information with time and frequency; if the voice intensity information is higher than a predetermined value and the template intensity information is greater than 0, then the superimposed intensity information is set as a superimposed value, where the superimposed value is the speech intensity A ratio of information. The voice watermark processing method according to claim 2, wherein the step of generating the superimposed watermark signal further includes: if the voice intensity information is not higher than the predetermined value and or the template intensity information is not greater than 0, then setting the superimposition The intensity information is 0. The voice watermark processing method according to claim 2, wherein the template strength information of the template watermark signal presents a text or a pattern in a time-frequency image. The voice watermark processing method according to claim 4, wherein the text or the pattern is related to a standard time or a place. A voice system includes: a radio device for receiving a first voice signal in a first time interval; a voice watermark processing device including: a prediction unit for predicting a second voice signal based on the first voice signal A predicted voice signal in a time interval; an analysis unit for analyzing the voice intensity information of the predicted voice signal over time and frequency; and a watermark generating unit for generating based on the voice intensity information of the predicted voice signal A superimposed watermark signal; and a playback device, and the radio device further receives a second time interval Two voice signals. The playback device is used to play the superimposed watermark signal so that the superimposed watermark signal and the second voice signal are superimposed; wherein the analysis unit analyzes the voice intensity information of the predicted voice signal over time and frequency The prediction unit obtains the predicted voice signal through a Long Short Term Memory network (LSTM) algorithm. The voice system according to claim 1, wherein the superimposed watermark signal has superimposed intensity information with one of time and frequency, and the watermark generating unit provides a template watermark signal, and the template watermark signal has a ratio of time and frequency A template strength information; if the voice strength information is higher than a predetermined value and the template strength information is greater than 0, the watermark generating unit sets the superimposed strength information as a superimposed value, wherein the superimposed value is one of the voice strength information Proportion value. The voice system according to claim 7, wherein if the voice strength information is not higher than the predetermined value and or the template strength information is not greater than 0, the watermark generating unit sets the superimposed strength information to 0. The voice system according to claim 7, wherein the template strength information of the template watermark signal presents a text or a pattern in a time-frequency image. The speech system according to claim 9, wherein the text Or the pattern is related to a standard time or a place. A voice watermark processing device includes: a prediction unit for receiving a first voice signal in a first time interval to predict a predicted voice signal in a second time interval; and an analysis unit for analyzing the predicted voice The signal is one of voice strength information with time and frequency; and a watermark generating unit for generating a superimposed watermark signal based on the voice strength information of the predicted voice signal, and the superimposed watermark signal is used to communicate with the second time A second voice signal is superimposed in the interval received; wherein when the analysis unit analyzes the voice intensity information of the predicted voice signal with time and frequency, the prediction unit uses a Long Short Term Memory network (LSTM) to calculate Method to obtain the predicted voice signal.

Images