CN115035903A

CN115035903A - Physical voice watermark injection method, voice tracing method and device

Info

Publication number: CN115035903A
Application number: CN202210952992.2A
Authority: CN
Inventors: 王滨; 李超豪; 王星; 陈加栋; 张峰
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2022-08-10
Filing date: 2022-08-10
Publication date: 2022-09-09
Anticipated expiration: 2042-08-10
Also published as: CN115035903B

Abstract

The embodiment of the invention provides a physical voice watermark injection method, a voice tracing method and a device, and relates to the technical field of voice privacy security, wherein the physical voice watermark injection method comprises the following steps: determining a sound signal matched with the physical voice of the target scene as a physical voice watermark signal; playing a physical voice watermark signal in a target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is the voice obtained by superposing the physical voice and the voice obtained by playing the physical voice watermark signal; the specifying information of the physical voice watermark signal is recorded. According to the technical scheme, a realization basis is provided for tracing the source of the voice embedded with the physical voice watermark signal, and the tracing of the voice recorded by the recording equipment in the target scene is further realized.

Description

Physical voice watermark injection method, voice tracing method and device

Technical Field

The invention relates to the technical field of voice privacy security, in particular to a physical voice watermark injection method, a voice tracing method and a device.

Background

In order to protect the copyright of the voice, a voice copyright side can inject a watermark into the voice, and a watermark injection method in the related art is to write the watermark into the audio data corresponding to the voice in the transmission process of the audio data after the audio data corresponding to the voice is generated.

However, for the voice obtained by malicious eavesdropping and recording of the illegal molecules through the recording device, it is often difficult for the eavesdropped party to inject the watermark into the illegally recorded voice, so that the source tracing of the illegally recorded voice cannot be performed, and therefore, how to trace the source of the voice recorded by the recording device is an urgent problem to be solved.

Disclosure of Invention

The embodiment of the invention aims to provide a physical voice watermark injection method, a voice tracing method and a voice tracing device, so as to realize the injection of a physical voice watermark into voice recorded by a recording device and further realize the voice tracing based on the injected physical voice watermark. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for injecting a physical voice watermark, where the method includes:

determining a sound signal matched with the physical voice of the target scene as a physical voice watermark signal;

playing the physical voice watermark signal in the target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is voice obtained by superposing the physical voice and voice played by the physical voice watermark signal;

recording the specified information of the physical voice watermark signal; wherein, the specified information includes the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

Optionally, before the playing the physical voice watermark signal in the target scene according to the preset playing policy, the method further includes:

modulating the physical voice watermark signal to a specified frequency band to obtain the modulated physical voice watermark signal;

the playing the physical voice watermark signal in the target scene according to a preset playing strategy includes:

and playing the modulated physical voice watermark signal in the target scene according to a preset playing strategy.

Optionally, the designated frequency band is an ultrasonic frequency band.

Optionally, the playing the physical voice watermark signal in the target scene according to a preset playing policy includes:

playing the physical voice watermark signal for multiple times in the target scene;

at this time, the playback information includes: and the interval duration sequence of the physical voice watermark signal is a duration sequence formed by the playing interval duration of the physical voice watermark signal.

Optionally, the playing the physical voice watermark signal multiple times in the target scene includes:

playing the physical voice watermark signal once in the target scene;

determining the playing interval duration after the playing is finished;

and after the playing of the physical voice watermark signal is finished, waiting for the determined playing interval duration, and returning to the step of playing the physical voice watermark signal once in the target scene.

Optionally, the determining the playing interval duration after the playing is completed includes:

randomly generating an interval duration as the playing interval duration after the playing is finished; or the like, or, alternatively,

and selecting an interval duration from the specified interval duration sequence as the playing interval duration after the playing is finished.

Optionally, the selecting an interval duration from the specified interval duration sequence as the playing interval duration after the playing is completed includes:

selecting an interval duration from the interval durations according to the sequence of the interval durations in the specified interval duration sequence, and taking the interval duration as the playing interval duration after the playing is completed; alternatively, the first and second electrodes may be,

randomly selecting an interval duration from each interval duration in the specified interval duration sequence as the playing interval duration after the playing is completed.

Optionally, the specified interval duration sequence is a randomly generated interval duration sequence.

Optionally, the determining a sound signal matched with the physical speech of the target scene as the physical speech watermark signal includes:

and when the target scene meets the preset sound watermark injection requirement, determining a sound signal matched with the physical voice of the target scene as a physical voice watermark signal.

Optionally, the target scene meets a preset sound watermark injection requirement, including:

receiving a sound watermark injection instruction aiming at the target scene; and/or the presence of a gas in the gas,

and detecting that the physical voice in the target scene meets a specified voice condition.

Optionally, the detecting that the physical voice in the target scene meets a specified voice condition includes:

detecting a sound signal containing a specified object in physical voice in the target scene; and/or the presence of a gas in the gas,

and detecting that the target scene is a specified scene based on the physical voice in the target scene.

Optionally, determining whether the target scene is a designated scene by using the following method includes:

inputting the physical voice in the target scene into a pre-trained scene classification model to obtain a scene type output by the scene classification model; determining whether the obtained scene type is a designated type to which a designated scene belongs, if so, determining that the target scene is the designated scene, otherwise, determining that the target scene is not the designated scene; or the like, or a combination thereof,

detecting whether the physical voice in the target scene contains appointed vocabularies or not, if so, determining that the target scene is the appointed scene, otherwise, determining that the target scene is not the appointed scene.

Optionally, the determining a sound signal matched with the physical voice of the target scene includes:

determining scene information of the target scene based on the physical voice of the target scene; wherein the determined scene information comprises a scene type of the target scene and/or a signal content of the physical voice;

and selecting the sound signal corresponding to the determined scene information from the preset sound signals based on the corresponding relation between the preset scene information and the sound signals, and taking the sound signal as the sound signal matched with the physical voice of the target scene.

In a second aspect, an embodiment of the present invention further provides a voice tracing method, where the method includes:

acquiring a voice to be traced;

extracting a watermark signal from the voice to be traced to obtain a watermark signal to be traced;

determining whether the watermark signal to be traced is matched with the physical voice watermark signal or not based on the pre-recorded specified information of the physical voice watermark signal; the physical voice watermark signal is a sound signal which is played in a target scene and is matched with physical voice of the target scene; the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal;

and if so, determining that the voice to be traced is the voice recorded from the target scene.

Optionally, the determining, based on the pre-recorded specific information of the physical voice watermark signal, whether the watermark signal to be traced back matches the physical voice watermark signal includes:

under the condition that the specified information comprises playing information, obtaining the playing information of the watermark signal to be traced in the voice to be traced; determining whether the playing information of the watermark signal to be traced is matched with the playing information of the physical voice watermark signal to obtain a first matching result;

under the condition that the specified information comprises signal characteristics, determining whether the signal characteristics of the watermark signal to be traced are matched with the signal characteristics of the physical voice watermark signal or not to obtain a second matching result;

and determining whether the watermark signal to be traced matches with the physical voice watermark signal or not based on the first matching result and/or the second matching result.

Optionally, the playing information of the physical voice watermark signal includes: an interval duration sequence of the physical voice watermark signal, wherein the interval duration sequence is a duration sequence composed of playing interval durations when the physical voice watermark signal is played for multiple times in the target scene;

the obtaining of the playing information of the watermark signal to be traced in the voice to be traced includes:

and determining the distribution interval duration of the watermark signal to be traced in the voice to be traced to obtain a distribution interval sequence as the playing information of the watermark signal to be traced.

Optionally, the determining whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal includes:

calculating whether a first similarity between the distribution interval sequence and the interval duration sequence is greater than a first preset threshold or determining whether the distribution interval sequence is a similar subset of the interval duration sequence;

if the first similarity is greater than the first preset threshold, or the distribution interval sequence is a similar subset of the interval duration sequence, determining that the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, and if the first similarity is not greater than the first preset threshold, or the distribution interval sequence is not the similar subset of the interval duration sequence, determining that the playing information of the watermark signal to be traced does not match with the playing information of the physical voice watermark signal.

Optionally, the determining whether the signal characteristic of the watermark signal to be traced matches the signal characteristic of the physical voice watermark signal includes:

calculating the similarity between the signal characteristics of the watermark signal to be traced and the signal characteristics of the physical voice watermark signal as a second similarity;

determining whether the second similarity is greater than a second preset threshold;

if the second similarity is larger than the second preset threshold, determining the signal characteristics of the watermark signal to be traced to be matched with the signal characteristics of the physical voice watermark signal, otherwise, determining the signal characteristics of the watermark signal to be traced to be not matched with the signal characteristics of the physical voice watermark signal.

Optionally, the determining whether the watermark signal to be traced and the physical voice watermark signal are matched based on the first matching result and/or the second matching result includes:

and if the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal and/or the signal characteristic of the watermark signal to be traced matches with the signal characteristic of the physical voice watermark signal, determining that the watermark signal to be traced matches with the physical voice watermark signal.

Optionally, the extracting the watermark signal from the voice to be traced to obtain the watermark signal to be traced includes:

extracting a sound signal with amplitude larger than a preset amplitude threshold value in the voice to be traced as the watermark signal to be traced; and/or the presence of a gas in the gas,

extracting a sound signal of a preset frequency band in the voice to be traced as the watermark signal to be traced; and/or the presence of a gas in the gas,

and extracting the watermark signal of the voice to be traced by utilizing a pre-trained watermark extraction model to obtain the watermark signal to be traced.

In a third aspect, an embodiment of the present invention provides an apparatus for injecting a physical voice watermark, where the apparatus includes:

the determining module is used for determining a sound signal matched with the physical voice of the target scene as a physical voice watermark signal;

the playing module is used for playing the physical voice watermark signal in the target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is the voice obtained by superposing the physical voice and the voice played by the physical voice watermark signal;

the recording module is used for recording the specified information of the physical voice watermark signal; wherein, the specified information includes the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

In a fourth aspect, an embodiment of the present invention provides a voice tracing apparatus, where the apparatus includes:

the obtaining module is used for obtaining the voice to be traced;

the extraction module is used for extracting the watermark signal of the voice to be traced to obtain the watermark signal to be traced;

the matching module is used for determining whether the watermark signal to be traced is matched with the physical voice watermark signal or not based on the pre-recorded specified information of the physical voice watermark signal; the physical voice watermark signal is a sound signal which is played in a target scene and is matched with physical voice of the target scene; the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal;

and the source tracing module is used for determining the voice to be traced as the voice recorded from the target scene if the voice to be traced is matched with the voice to be traced.

In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the steps of the physical voice watermark injection method or the steps of the voice tracing method when executing the program stored in the memory.

In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for injecting a physical voice watermark or the steps of the method for tracing to source of voice are implemented.

The embodiment of the invention has the following beneficial effects:

the method for injecting the physical voice watermark provided by the embodiment of the invention determines a sound signal matched with the physical voice of a target scene as a physical voice watermark signal; playing a physical voice watermark signal in a target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; recording the specified information of the physical voice watermark signal; wherein, the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal. The physical voice watermark signal is played in the target scene, so that the voice played by the physical voice watermark signal is superposed with the physical voice in the target environment at an empty port, which means that the watermark information is embedded in the superposed voice after superposition, at this time, if the target environment has a recording device, the recording device can only record the superposed voice, so that the recorded voice contains the physical voice watermark signal, further, after the physical voice watermark signal is played, the specified information of the physical voice watermark signal is recorded, so that the source tracing can be carried out subsequently by using the specified information, and thus, the method provides a realization basis for the source tracing of the voice embedded with the physical voice watermark signal.

In addition, the voice tracing method provided by the embodiment of the invention obtains the voice to be traced; extracting a watermark signal of the voice to be traced to obtain a watermark signal to be traced; determining whether the watermark signal to be traced is matched with the physical voice watermark signal or not based on the pre-recorded specified information of the physical voice watermark signal; and if the voice to be traced is matched with the voice to be traced, determining that the voice to be traced is the voice recorded from the target scene. Since the physical voice watermark signal is a sound signal which is played in the target scene and is matched with the physical voice of the target scene, it means that if a recording device exists in the target scene, the voice recorded in the recording device necessarily includes the physical voice watermark signal, and therefore, if it is determined that the watermark signal to be traced is matched with the physical voice watermark signal based on the pre-recorded specified information of the physical voice watermark signal, it can be determined that the voice to be traced is recorded by the recording device in the target scene. Therefore, the source tracing of the voice recorded by the recording equipment in the target scene can be realized through the scheme.

Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.

Fig. 1 is a first flowchart of a method for injecting a physical voice watermark according to an embodiment of the present invention;

fig. 2 is a second flowchart of a method for injecting a physical voice watermark according to an embodiment of the present invention;

fig. 3 is a third flowchart of a method for injecting a physical voice watermark according to an embodiment of the present invention;

fig. 4 is a first flowchart of a voice tracing method according to an embodiment of the present invention;

fig. 5 is a fourth flowchart of a method for injecting a physical voice watermark according to an embodiment of the present invention;

fig. 6 is a second flowchart of a voice tracing method according to an embodiment of the present invention;

fig. 7 is a block diagram of an apparatus for injecting a physical voice watermark according to an embodiment of the present invention;

fig. 8 is a structural diagram of a voice tracing apparatus according to an embodiment of the present invention;

fig. 9 is a block diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.

With the rapid development of technologies such as recording transcription and voice recognition and the popularization and application of tools such as intelligent voice assistants, the efficiency and the precision of sound signal acquisition, storage and recognition are greatly improved. However, acquisition of sound signals is dependent on the deployment of large numbers of sound recording devices, which poses an increasingly serious security threat to sound privacy. For example, if someone carries a sound recording device or maliciously manipulates a sound recording device in a specific scene, and eavesdrops an important conference or a privacy conversation, adverse effects may be caused. In the watermark injection method in the related art, the watermark can be written into the audio data corresponding to the voice only in the data transmission process after the audio data corresponding to the voice is generated, and the watermark cannot be injected into the voice obtained by eavesdropping, so that the source of the illegally recorded voice cannot be traced, and the eavesdropping behavior is attacked. Therefore, it is an urgent problem to be solved how to trace the source of the voice recorded by the recording device.

In order to solve the above problem, embodiments of the present invention provide a method for injecting a physical voice watermark, a method for tracing a source of a voice, and an apparatus. It should be noted that, in a specific application, the embodiments of the present invention can be applied to various electronic devices, such as a personal computer, a server, a mobile phone, and other devices with data processing capability. In addition, the injection method and the voice tracing method of the physical voice watermark provided by the embodiment of the invention can be realized by software, hardware or a combination of software and hardware.

In an embodiment, the method for injecting a physical voice watermark provided in the embodiment of the present invention may be applied to a sound playing device, such as a speaker, optionally, the sound playing device may play a sound signal in a specified frequency band, such as playing a sound signal in an ultrasonic frequency band, or the embodiment of the present invention may be applied to an electronic device with sound recording and playing functions, where the electronic device may receive and play a sound signal, such as a computer system with sound recording and playing functions, and the present invention is not limited to this.

First, a method for injecting a physical voice watermark provided in an embodiment of the present invention is described below, where the method may include:

playing a physical voice watermark signal in a target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is the voice obtained by superposing the physical voice and the voice obtained by playing the physical voice watermark signal;

recording the specified information of the physical voice watermark signal; wherein, the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

In the method for injecting the physical voice watermark provided by the embodiment of the invention, the physical voice watermark signal is played in the target scene, so that the voice after the playing of the physical voice watermark signal is overlapped with the physical voice in the target environment at an empty port, which means that the watermark information is embedded in the overlapped voice, at this time, if a recording device exists in the target environment, the recording device can only record the overlapped voice, so that the recorded voice contains the physical voice watermark signal, further, after the playing of the physical voice watermark signal, the specified information of the physical voice watermark signal is recorded, and thus, the specified information can be used for tracing the source subsequently, and thus, the scheme provides a basis for tracing the voice in which the physical voice watermark signal is embedded.

The following describes an exemplary method for injecting a physical voice watermark according to an embodiment of the present invention with reference to the accompanying drawings.

As shown in fig. 1, a method for injecting a physical voice watermark provided by an embodiment of the present invention may include steps S101 to S103:

s101, determining a sound signal matched with physical voice of a target scene as a physical voice watermark signal;

the target scene may be a scene that generates sound at will, for example, distinguished by scene type, and may be a privacy dialog scene, a copyright audio recording scene, a privacy dialog scene, and the like; the target scenes can be conference rooms, recording rooms, classrooms and the like according to scene position distinction. The physical voice may be any sound signal generated in the target scene, such as a sound signal generated by human talking, a sound signal generated by playing music, and the like, and may also be a superposition of a plurality of sound signals.

The sound signal matched with the physical voice of the target scene can be any one selected as the sound signal matched with the physical voice of the target scene, and the selected sound signal can be a noise signal, a song sound signal, a conversation sound signal and the like.

Optionally, the sound signal matched with the physical voice of the target scene may also be a sound signal determined based on the physical voice of the target scene, for example, based on the physical voice of the target scene, scene information of the target scene is determined, and then based on a corresponding relationship between preset scene information and the sound signal, a sound signal corresponding to the determined scene information is selected from preset sound signals to serve as the sound signal matched with the physical voice of the target scene.

The determined scene information may include a scene type of the target scene and/or a signal content of the physical voice. The scene type may be a privacy conversation scene, a copyright audio recording scene, a privacy conversation scene, and the like, and the signal content of the physical voice may be a text content of the physical voice. The signal content can be obtained by extracting the physical speech of the target scene by using a pre-trained artificial intelligence model. Optionally, when the content of the signal to be extracted is the text content of a conversation, the artificial intelligence model may be deep speech (an open source speech-to-text model), Kaldi (an open source speech-to-text model), or the like. Through the corresponding relation between the preset scene information and the sound signals, the appropriate sound signals can be flexibly selected for different scenes.

In an example, the scene information includes a scene type, and the scene type of the target scene is a conference type, a privacy conversation type, and the like that require voice privacy, a large-amplitude noise signal or a voice signal of a person speaking may be selected to interfere recording of the physical voice in the target scene by the recording device, so as to achieve privacy of the physical voice in the target scene. In another example, if the scene information includes signal content and the signal content of the physical voice in the target scene is music, a sound signal of a music type may be selected to interfere with the recording of the physical voice in the target scene by the recording device.

In order to increase the uniqueness of the sound signal, the sound signal matched with the physical voice of the target scene may further include identification information of the target scene, a digital password, a text password, and the like. Of course, the sound signal matched with the physical voice of the target scene may not be limited to the above. The identification information of the target scene may be information representing the location, time, occurrence event, and other content of the target scene, and may be, for example, a piece of voice content, such as XX year, X month, X day, XX company in XX city, XX meeting, and the like. The digital password and the text password may be a string of randomly generated numbers and texts, or may be a preset string of numbers and texts, and the embodiment of the present invention is not particularly limited.

For convenience of expression, in the embodiment of the present invention, a sound signal matched with a physical voice of a target scene is used as a physical voice watermark signal.

In this step, when the target scene meets the preset sound watermark injection condition, the step of determining the sound signal matched with the physical voice of the target scene may be performed, where when the target scene meets the preset sound watermark injection condition, it indicates that the target scene needs to perform sound watermark injection to protect the physical voice in the target scene. Optionally, in the embodiment of the present invention, whether a target scene meets a preset sound watermark injection condition may be determined in multiple ways, which will be specifically described in detail in the subsequent embodiments and will not be described herein again. Of course, it should be emphasized that this step may be performed in any case, which means that this step does not need to be performed until a certain condition is satisfied, but may be performed at any time.

S102, playing a physical voice watermark signal in a target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is the voice obtained by superposing the physical voice and the voice obtained by playing the physical voice watermark signal;

in order to enable the voice recorded by the recording device in the target scene to carry the watermark information, in this step, the physical voice watermark signal may be played in the target scene according to a preset playing strategy. After the physical voice watermark is played in the target scene, the target scene comprises the physical voice and the voice after the physical voice watermark signal is played, which means that the played physical voice watermark signal is overlapped with the physical voice in the target environment at an air interface, so that the watermark information is embedded in the overlapped voice, at this time, if a recording device exists in the target environment, the voice recorded by the recording device comprises the physical voice watermark signal.

The preset playing policy may be determined according to requirements and experience, and may be any playing policy, such as continuous playing, single playing, multiple playing, and the like. This is all right, and in the case that the preset playing strategy is continuous playing, the physical voice watermark signal can be continuously played in the target scene, which means that the physical voice watermark signal is cyclically played, so that the physical voice watermark signal always exists in the target scene. And under the condition that the preset playing strategy is single playing, playing the physical voice watermark signal once in the target scene, which means that the physical voice watermark signal is played once and then the playing is stopped. In the case that the preset playing strategy is multiple playing, the physical voice watermark signal can be played in the target scene for a preset number of times, which means that the physical voice watermark signal is played for the preset number of times after being played for the preset number of times and then is stopped.

S103, recording the specified information of the physical voice watermark signal; wherein, the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

The signal characteristic of the physical voice watermark signal may be a baseband signal of the physical voice watermark signal, or a pre-trained special extraction network may be used to perform characteristic extraction on the physical voice watermark signal, and the extracted characteristic information is used as the signal characteristic of the physical voice watermark signal. In the case that the physical voice watermark signal is played in the target scene once, the playing information of the physical voice watermark signal may be the playing duration of the physical voice watermark signal. In the case that the physical voice watermark signal is played multiple times in the target scene, the playing information may further include: playing times, an interval duration sequence of the physical voice watermark signal and the like, wherein the interval duration sequence is a duration sequence formed by playing interval durations of the physical voice watermark signal. For example, after the first play, the physical voice watermark signal waits for 8 seconds to start the second play, after the second play, the physical voice watermark signal waits for 10 seconds to start the third play, and after the third play, the physical voice watermark signal waits for 9 seconds to start the fourth play, so that the sequence of durations formed by the durations of the play intervals of the physical voice watermark signal can be represented as [8,10,9 ].

Since the final purpose of the scheme is to trace the source of the voice, for example, if the continuous playing or the single playing is performed, the specific information may be the signal characteristics of the physical voice watermark signal, which means that if the signal characteristics of the watermark signal in the voice to be traced are matched with the signal characteristics of the physical voice watermark signal, the voice to be traced is considered to be the voice collected by the sound recording device in the target scene. When the preset playing strategy is playing for multiple times, the interval duration between every two times of playing of the physical voice watermark signal can be used as the playing information of the physical voice watermark signal, which means that if the interval duration of the watermark signal in the voice to be traced is matched with the interval duration of playing the physical voice watermark signal, the voice to be traced is considered as the voice collected by the recording device in the target scene. In this case, of course, the feature information of the watermark signal may be further combined for confirmation to improve the accuracy.

In addition, the tracing information of the physical voice watermark signal can be recorded, so that the related information of the physical voice watermark signal can be further known during tracing. The tracing information may include: generating time, generating place, generating object, copyright side and other information.

In order to trace the source of the voice subsequently, after determining the specific information of the physical voice watermark signal, the specific information of the physical voice watermark signal may be recorded, and the specific information may be recorded in any manner, for example, an electronic form, a document, and the like.

In this embodiment, because the physical voice watermark signal is played in the target scene, the voice after the playing of the physical voice watermark signal is superimposed with the physical voice in the target environment at the air interface, which means that the watermark information has been embedded in the superimposed voice after the superimposition, at this time, if a recording device exists in the target environment, the recording device can only record the superimposed voice, so that the recorded voice contains the physical voice watermark signal, further, after the playing of the physical voice watermark signal, the specified information of the physical voice watermark signal is recorded, so that the tracing can be performed subsequently by using the specified information, which can be seen that this scheme provides a basis for performing the tracing on the voice in which the physical voice watermark signal has been embedded.

Optionally, in another embodiment of the present invention, the target scene meets a preset sound watermark injection requirement, which may be multiple, for example, in an implementation manner meeting the requirement, when a sound watermark injection instruction for the target scene is received, it indicates that the target scene meets the preset sound watermark injection requirement, where the sound watermark injection instruction may be sent by a worker by clicking a specified button in a User Interface (UI) or inputting a specified field, and certainly is not limited thereto. When a sound watermark injection instruction for a target scene is received, which means that the target scene meets a preset sound watermark injection requirement, a sound signal matched with physical voice of the target scene can be determined as a physical voice watermark signal.

In another implementation manner meeting the requirement, when it is detected that the physical voice of the target scene meets the specified voice condition, it indicates that the target scene meets the preset voice watermark injection requirement. In this case, the environmental sound in the target scene may be acquired first, and then the acquired environmental sound may be detected to determine whether it satisfies the specified speech condition. The specified speech conditions may be determined on demand and empirically.

The above-mentioned manner for detecting that the physical voice of the target scene meets the specified voice condition may be various, and for example, at least one of the following manners may be included:

the first way to satisfy the specified speech conditions: detecting that the physical voice of the target scene contains a sound signal of a specified object;

the specified object can be an object which can make sound such as a person, a musical instrument and the like, voiceprint information of the specified object can be extracted and stored in advance through a voiceprint technology, then, voiceprints of various sounds in the physical voice in the target environment are detected, whether the detected voiceprints are the voiceprints of the specified object is judged, and whether the sound signals of the specified object are contained in the physical voice is determined.

The second way to satisfy the specified speech conditions: and detecting the target scene as a specified scene based on the physical voice of the target scene.

The designated scene may be any scene, such as a privacy dialog scene, a copyright audio recording scene, a privacy dialog scene, and the like, and may be set according to actual requirements, which is not specifically limited in the embodiment of the present invention.

In this way, whether the target scene is the designated scene may be detected based on the physical voice of the target scene. Optionally, a plurality of ways may be adopted to detect whether the target scene is a specified scene, for example, the following may be included:

the first scene detection mode: detecting whether the physical voice in the target scene contains appointed vocabularies or not, if so, determining that the target scene is the appointed scene, otherwise, determining that the target scene is not the appointed scene;

it can be understood that when a person in the target scene speaks, the physical voice may include a voice signal generated by the person speaking, and the voice signal includes information such as sentences, words, and the like. The specified vocabulary can be preset, and for example, the specified vocabulary is as follows: "start injection", "start running", etc. When the physical voice is detected to contain the specified vocabulary, the detected target scene is determined to be the specified scene.

For example, in one implementation, detecting whether the specified vocabulary is contained in the physical speech of the target scene may include the following processes: firstly, a pre-trained artificial intelligence model, such as DeepSpeech, Kaldi and the like, is utilized to process physical voice to obtain the text content generated by speaking of people in a target scene, and then whether the obtained text content contains appointed vocabularies or not is judged.

The second scene detection mode: the scene type of the target scene can be determined based on the physical voice of the target scene, whether the obtained scene type is a specified type to which the specified scene belongs or not is further determined, if yes, the target scene is determined to be the specified scene, and if not, the target scene is determined not to be the specified scene.

In this way, the pre-trained scene classification model may be used to process the physical speech of the target scene to determine whether the target scene is the type to which the specified scene belongs. Specifically, the method can comprise the following steps:

step A1, inputting the physical voice of the target scene into a pre-trained scene classification model to obtain the scene type output by the scene classification model;

the scene types to which the different scenes belong may be set according to requirements and experience, and may include at least one of a type of a concert, a type of a conference, a type of a private conversation, a type of a music recording, and the like, for example.

In order to more accurately determine the scene type of the target scene, a pre-trained scene classification model may be used to classify the target scene, and the scene classification model may be obtained by training in any training manner, such as supervised, semi-supervised, unsupervised, and so on.

In order to train the scene classification model, a plurality of sound signals may be prepared in advance as a sample sound signal, for example, a sound signal recorded in a conference type scene, a sound signal recorded in a privacy conversation type scene, and the like, and the scene type recorded in the sample sound signal may be set to a true value of the scene type corresponding to the sample sound signal.

In a training mode of a scene classification model, a plurality of sample sound signals can be input into the scene classification model to be trained, so that the scene classification model to be trained outputs a scene type recognition result about each sample sound signal, a true value of the scene type recognition result corresponding to each sample sound signal and the scene type corresponding to each sample sound signal is utilized to calculate a model loss of the scene classification model, and then model parameters of the scene classification model to be trained are adjusted based on the model loss until the loss of the scene classification model to be trained is converged, so that the trained scene classification model is obtained.

Step a2, determining whether the obtained scene type is a specified type to which the specified scene belongs, if so, determining that the target scene is the specified scene, otherwise, determining that the target scene is not the specified scene.

Various scene types may be set in advance as a specified type to which a specified scene belongs and a non-specified type to which a non-specified scene belongs, for example, a conference type and a privacy conversation type may be set as the specified types, and other scene types may be set as the non-specified types. Therefore, after the scene type of the physical voice of the target scene is obtained, whether the scene type is the specified type to which the specified scene belongs can be judged. The scene type of the target scene can be automatically identified by utilizing the pre-trained scene classification model without manual identification, so that the labor cost can be reduced, and the identification efficiency can be improved.

Before inputting the physical speech of the target scene into the pre-trained scene classification model, it may also be possible to first recognize the text content generated by speaking of a person included in the physical speech of the target scene by using a pre-trained artificial intelligence model, such as deep speech, Kaldi, etc., and then input the recognized text content into the pre-trained scene classification model to obtain the scene type output by the scene classification model, or to first extract the scene features of the physical speech of the target scene by using a feature extraction network and then input the scene features into the pre-trained scene classification model to obtain the scene type output by the scene classification model. Compared with the scene in which the physical voice of the target scene is directly input to the pre-trained scene classification model, the scene classification model under the scene has a similar training process, and is not repeated here.

In this embodiment, the manner of detecting that the physical voice of the target scene satisfies the specified voice condition may be one of the two manners, for example, detecting that the physical voice of the target scene includes a sound signal of the specified object, and at this time, if only the one manner is satisfied, it is determined that the physical voice of the target scene satisfies the specified voice condition. Or, the above two modes may be used simultaneously in the above-mentioned mode of detecting that the physical voice of the target scene satisfies the specified voice condition, and at this time, when the two modes are satisfied simultaneously, it means that the physical voice of the target scene satisfies the specified voice condition.

In this embodiment, a physical voice watermark signal may be embedded in the voice recorded by the recording device, thereby providing a basis for performing voice tracing. Furthermore, the sound signal matched with the physical voice of the target scene can be determined when the target scene meets the preset sound watermark injection requirement, so that the physical voice watermark signal can be prevented from being frequently played.

In another embodiment of the present invention, in order to make the playing information have more characteristics, so as to improve the accuracy of subsequent tracing, in the embodiment of the present invention, the preset playing policy may be played for multiple times, in a simple manner, after the physical voice watermark signal is determined, the physical voice watermark signal may be played for multiple times in a target scene, and at this time, the playing information may include: and the interval duration sequence of the physical voice watermark signal is a duration sequence formed by playing interval durations of the physical voice watermark signal.

In one implementation, as shown in fig. 2, the playing the physical voice watermark signal multiple times may include the following steps S201 to S203:

s201, in a target scene, playing a physical voice watermark signal once;

in this step, the physical voice watermark signal may be played only once, that is, after the determined physical voice watermark signal is played, step S202 or S203 (fig. 2 only takes the execution of S202 as an example) may be executed, instead of continuously playing the physical voice watermark signal.

S202, determining the playing interval duration after the playing is finished;

the step S201 may be performed synchronously, that is, the playing interval duration after the playing is completed is determined while the physical voice watermark signal is played. This step may also be performed asynchronously, and for example, it may be possible to perform step S201 first and then perform this step, or perform this step first and then perform step S201.

The above manners for determining the playing interval duration after the playing is completed may be various, and exemplarily, the method may at least include one of the following two manners:

the first method for determining the duration of the playing interval: randomly generating an interval duration as the playing interval duration after the playing is finished;

in this way, an interval duration may be randomly generated each time the play interval duration after the completion of the play of this time is determined, and the interval duration may be used as the play interval duration after the completion of the play of this time.

The second play interval duration determination method: and selecting an interval duration from the specified interval duration sequence as the playing interval duration after the playing is finished.

The specified interval duration sequence may be preset or generated according to a specified rule, where the specified rule may be generated randomly or generated according to a corresponding relationship between time, a scene and the interval duration sequence generated by the specified interval duration sequence, for example, current time is encoded to obtain an encoding sequence, and the encoding sequence is converted into the interval duration sequence.

After the specified interval duration sequence is obtained, an interval duration can be selected from the interval durations according to the sequence of the interval durations in the specified interval duration sequence, and the interval duration can be used as the playing interval duration after the playing is completed. For example, the time interval sequence may be represented as [8,10,9], where 8 represents waiting for 8 seconds after the first play ends, starting the second play, 10 represents waiting for 10 seconds after the second play ends, starting the third play, and 9 represents waiting for 9 seconds after the third play ends, starting the fourth play, and if the present play is the first play, then 8 seconds are selected as the play interval duration after the present play is completed.

Or, a manner of selecting an interval duration from the specified interval duration sequence may also be to randomly select an interval duration from each interval duration in the specified interval duration sequence, as the playing interval duration after the playing is completed. For example, the time interval sequence may be represented as [8,10,9], and if the playing of this time is the first playing, a time length may be randomly selected from 8 seconds, 9 seconds, or 10 seconds as the playing interval time length after the playing of this time is completed.

S203, after the playing of the physical voice watermark signal is finished, waiting for the determined playing interval duration, and returning to execute the step S201.

The process of playing the physical voice watermark signal and waiting for the determined playing interval duration may be repeated uninterruptedly until the voice watermark is no longer required to be injected in the target scene, and for example, the process of playing the physical voice watermark signal and waiting for the determined playing interval duration may be stopped when an instruction for stopping the injection of the voice watermark is received or when it is detected that the target scene does not meet the preset voice watermark injection requirement, so as to complete the injection of the watermark this time.

When the physical voice in the target scene does not contain the specified vocabulary after exceeding the specified duration, or the physical voice in the target scene contains the sound signal of the specified object is detected, and the target scene is detected to be converted into an insecure scene, the target scene can be considered not to meet the preset sound watermark injection requirement, namely the watermark injection is not required to be carried out on the physical voice signal of the target scene.

In this embodiment, a physical voice watermark signal may be embedded in the voice recorded by the recording device, thereby providing a basis for performing voice tracing. Furthermore, when the physical voice watermark signal is played for multiple times, an interval duration sequence formed by interval durations between the physical voice watermark signals is played for every two times and can be used as playing information of the physical voice watermark signal, so that the playing information has more characteristics, and the accuracy of tracing the voice is improved.

Optionally, in another embodiment of the present invention, as shown in fig. 3, the method for injecting a physical voice watermark provided in the embodiment of the present invention may include steps S301 to S304:

s301, determining a sound signal matched with the physical voice of the target scene as a physical voice watermark signal;

the step is the same as or similar to the step S101, and is not described herein again in the embodiments of the present invention.

Before the physical voice watermark signal is played in the target scene according to the preset playing strategy, the method for injecting the physical voice watermark provided by the embodiment of the invention further comprises the following steps:

s302, modulating the physical voice watermark signal to a specified frequency band to obtain a modulated physical voice watermark signal;

the designated frequency band may be any frequency band, or a specific frequency band, such as a sound wave frequency band imperceptible to human ears, i.e., an ultrasonic wave frequency band or an infrasonic wave frequency band.

In one implementation, the specified frequency band may be an ultrasonic frequency band. Wherein, the ultrasonic frequency range is the frequency range larger than 20 KHz. Illustratively, the physical voice watermark signal can be modulated to the ultrasonic frequency band by using AM (Amplitude Modulation) Modulation. The modulation method is as follows:

wherein the content of the first and second substances,

representing the audio signal after modulation of the audio signal,

and

belonging to the amplitude of the carrier signal and the baseband signal,

representing time。

Is the frequency of the carrier signal and,

the frequency of the baseband signal. The frequency and amplitude of the carrier signal and the baseband signal may be set to fixed values or time-varying values as desired and empirically.

By way of example, in one implementation,

it may be set to a frequency of 25kHz,

it may be set to a frequency of 1kHz,

can be equal to

。

Because the working principle of the recording device is to convert mechanical sound waves into electric signals, and the input/output signal transmission characteristic of the recording device has a nonlinear effect, the nonlinear effect can be formalized as follows:

wherein the content of the first and second substances,

which represents the input sound signal, is,

representing the output audio signal, a and B are coefficient parameters, respectively.

The recording device nonlinear demodulation process can be expressed as follows:

wherein the content of the first and second substances,

indicating a demodulated signal of the sound recording apparatus for the received sound signal,

is a constant vector, i.e. a non-linear coefficient of the recording device. The non-linear demodulation process of the recording device is automatically performed when the sound signal is received. Due to the non-linear effect of the recording equipment, the physical voice watermark signal modulated to the ultrasonic frequency band can be recorded, namely

Will contain the physical voice watermark signal.

Therefore, the physical voice watermark signal modulated to the ultrasonic frequency band can realize the concealed injection of the watermark under the condition of not influencing the normal conversation of people in a target scene. Meanwhile, due to the nonlinear effect of the recording equipment, under the condition that the physical voice watermark signal is an interference signal, the recording equipment can record the interference signal, so that the recording equipment can be interfered under the condition that normal conversation of personnel in a target scene is not influenced, physical voice generated by normal conversation of the personnel in the target scene is shielded, and physical voice protection of the target scene is realized.

In the case of modulating the physical voice watermark signal to a specific frequency band to obtain a modulated physical voice watermark signal, playing the physical voice watermark signal in the target scene according to a preset playing strategy may include:

and S303, playing the modulated physical voice watermark signal in the target scene according to a preset playing strategy.

After the modulated physical voice watermark signal is obtained, the modulated physical voice watermark signal can be played in the target scene, and the specific playing strategy of the modulated physical voice watermark signal is the same as or similar to that in step S102, which is not described herein again in the embodiments of the present invention.

S304, recording the specified information of the physical voice watermark signal; wherein, the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

This step is the same as or similar to step S103, and is not described again in this embodiment of the present invention.

In this embodiment, a physical voice watermark signal may be embedded in the voice recorded by the recording device, thereby providing a basis for performing voice tracing. Furthermore, the implementation manner of the method for injecting the physical voice watermark provided by the embodiment of the invention is further enriched by modulating the physical voice watermark signal to the specified frequency band to obtain the modulated physical voice watermark signal, and when the specified frequency band is the ultrasonic frequency band, the watermark can be injected into the sound signal of the physical world or the recording of the sound by the recording equipment can be interfered under the condition that the normal conversation of the personnel in the target scene is not influenced.

Corresponding to the above-mentioned method for injecting physical voice watermark, the embodiment of the present invention further provides a voice tracing method,

the method can be applied to electronic equipment with a data processing function, such as a computer, a mobile phone and the like. As shown in fig. 4, a voice tracing method provided in the embodiment of the present invention includes steps S401 to S405:

s401, obtaining a voice to be traced;

the voice to be traced can be any sound signal which needs to be traced. Optionally, the voice to be traced can be acquired from the specified storage location, for example, the storage locations such as the usb disk, the specified database, and the cloud end, and in this implementation, the voice to be traced can be uploaded to the specified storage location in advance by the staff. Alternatively, the voice to be traced may also be acquired in real time, which is all possible, and this is not limited in this embodiment of the present invention.

S402, extracting a watermark signal of the voice to be traced to obtain the watermark signal to be traced;

in this step, the watermark signal extraction can be performed on the voice to be traced by adopting the following three ways:

the first extraction method comprises the following steps: extracting a sound signal with amplitude larger than a preset amplitude threshold value in the voice to be traced as a watermark signal to be traced;

in this implementation manner, after obtaining the voice to be traced, it may be determined whether the amplitude of each sampling point in the voice to be traced is greater than the preset amplitude threshold, if so, the sound signal of the sampling point is retained, otherwise, the sound signal of the sampling point is deleted, and finally, the sound signal whose amplitude is greater than the preset amplitude threshold in the voice to be traced is obtained, which of course is not limited thereto. The preset amplitude threshold value can be determined according to actual conditions and experience.

The second extraction method comprises the following steps: extracting a sound signal of a preset frequency band in the voice to be traced as a watermark signal to be traced;

in this implementation manner, the band-pass filter may be used to extract the sound signal of the preset frequency band in the speech to be traced, which of course is not limited thereto. The preset frequency range can be determined according to actual conditions and experience.

The third extraction mode is as follows: and (4) extracting the watermark signal of the voice to be traced by utilizing a pre-trained watermark extraction model to obtain the watermark signal to be traced.

In this implementation manner, the voice to be traced can be input into the pre-trained watermark extraction model, so that the watermark extraction model processes the voice to be traced to obtain the watermark signal to be traced. The watermark extraction model can be obtained by training in any training mode, and the embodiment of the invention is not particularly limited.

In an implementation manner, in order to more comprehensively obtain a watermark signal to be traced from the voice to be traced, a variety of manners may be used to extract the watermark signal from the voice to be traced, for example, a sound signal with an amplitude larger than a preset amplitude threshold value in the voice to be traced, a sound signal with a preset frequency band in the voice to be traced, and a watermark extraction model trained in advance may be used to extract the watermark signal from the voice to be traced, so as to obtain the sound signal, which is all used as the watermark signal to be traced.

In another implementation manner, in order to more accurately obtain a watermark signal to be traced in a voice to be traced, a sound signal with an amplitude greater than a preset amplitude threshold value in the voice to be traced may be extracted first to obtain a first extraction signal, a sound signal in a preset frequency band in the first extraction signal is extracted to obtain a second extraction signal, and finally, a pre-trained watermark extraction model is used to extract a watermark signal from the second extraction signal to obtain the watermark signal to be traced.

S403, determining whether the watermark signal to be traced matches with the physical voice watermark signal based on the pre-recorded specified information of the physical voice watermark signal; if yes, executing step S404, otherwise, executing step S405;

the physical voice watermark signal is a sound signal which is played in a target scene and is matched with the physical voice of the target scene; the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal; the above-mentioned physical voice watermark signal is a sound signal which is played in the target scene and is matched with the physical voice of the target scene, and the determination mode of the physical voice watermark signal has been introduced above, and is not described herein again.

The manner of determining whether the watermark signal to be traced matches the physical voice watermark signal may include:

step B1, under the condition that the designated information includes the playing information, obtaining the playing information of the watermark signal to be traced in the voice to be traced; determining whether the playing information of the watermark signal to be traced is matched with the playing information of the physical voice watermark signal to obtain a first matching result;

the playing information of the physical voice watermark signal comprises: the time length sequence of the physical voice watermark signal is a time length sequence formed by playing time lengths of the physical voice watermark signal when the physical voice watermark signal is played for multiple times in a target scene;

the obtaining of the playing information of the watermark signal to be traced in the voice to be traced may include:

and determining the distribution interval duration of the watermark signal to be traced in the voice to be traced to obtain a distribution interval sequence as the playing information of the watermark signal to be traced to the source.

The distribution interval duration of the watermark signal to be traced in the voice to be traced is the interval duration between every two times of playing the watermark signal to be traced in the voice to be traced. And recording the distribution interval duration in the voice to be traced to obtain a distribution interval sequence. And obtaining a first matching result by determining whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal.

The determining whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal may be: and determining whether the values in the distribution interval sequence and the interval duration sequence can be in one-to-one correspondence, if so, determining that the distribution interval sequence is matched with the interval duration sequence, otherwise, determining that the distribution interval sequence is not matched with the interval duration sequence.

Step B2, under the condition that the specified information includes the signal characteristic, determining whether the signal characteristic of the watermark signal to be traced matches with the signal characteristic of the physical voice watermark signal, and obtaining a second matching result;

the signal feature of the physical voice watermark signal may be a baseband signal of the physical voice watermark signal, or feature information obtained by performing feature extraction on the physical voice watermark signal by using a pre-trained feature extraction network. And determining whether the signal characteristics of the watermark signal to be traced are matched with the signal characteristics of the physical voice watermark signal or not to obtain a second matching result.

The above-mentioned method for determining whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, and determining whether the signal characteristic of the watermark signal to be traced matches with the signal characteristic of the physical voice watermark signal will be described in detail in the following embodiments, and will not be described herein again.

And step B3, determining whether the watermark signal to be traced matches the physical voice watermark signal based on the first matching result and/or the second matching result.

In this embodiment, the following situations may exist in the manner of determining whether the watermark signal to be traced matches the physical voice watermark signal:

in the first case: in the case where the designation information includes playback information, it may be determined whether the watermark signal to be traced matches the physical voice watermark signal only by the first matching result. In this case, if the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, it is determined that the watermark signal to be traced matches with the physical voice watermark signal, otherwise, it is determined that the watermark signal to be traced does not match with the physical voice watermark signal.

In the second case: in the case where the specifying information includes the signal feature, only the second matching result determines whether the watermark signal to be traced matches the physical voice watermark signal. In this case, if the signal characteristics of the watermark signal to be traced match the signal characteristics of the physical voice watermark signal, it is determined that the watermark signal to be traced matches the physical voice watermark signal, otherwise, it is determined that the watermark signal to be traced does not match the physical voice watermark signal.

In the third case: under the condition that the designated information comprises playing information and signal characteristics, whether the watermark signal to be traced is matched with the physical voice watermark signal or not can be determined by combining a first matching result and a second matching result, when the condition that the playing information of the watermark signal to be traced is matched with the playing information of the physical voice watermark signal or the signal characteristics of the watermark signal to be traced is matched with the signal characteristics of the physical voice watermark signal is met, in any one of the two conditions, the watermark signal to be traced is determined to be matched with the physical voice watermark signal, and if the two conditions are not met, the watermark signal to be traced is determined to be not matched with the physical voice watermark signal. Or when the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal and the signal characteristic of the watermark signal to be traced matches with the signal characteristic of the physical voice watermark signal, determining that the watermark signal to be traced matches with the physical voice watermark signal, and if any one of the situations is not met, determining that the watermark signal to be traced does not match with the physical voice watermark signal.

S404, determining the voice to be traced as the voice recorded from the target scene.

The physical voice watermark signal is a sound signal played in a target scene, and if a recording device exists in the target scene, the voice recorded in the recording device necessarily contains the physical voice watermark signal, so that if the watermark signal to be traced in the voice to be traced is matched with the physical voice watermark signal, the voice to be traced can be determined to be recorded by the recording device in the target scene, and tracing of the voice to be traced can be completed.

S405, determining that the voice to be traced is not the voice recorded from the target scene.

In this embodiment, because the physical voice watermark signal is a sound signal that is played in the target scene and matches the physical voice of the target scene, it means that, if a recording device exists in the target scene, the voice recorded in the recording device necessarily includes the physical voice watermark signal, and therefore, if it is determined that the watermark signal to be traced back matches the physical voice watermark signal based on the pre-recorded specifying information of the physical voice watermark signal, it may be determined that the voice to be traced back is recorded by the recording device in the target scene. Therefore, the source tracing of the voice recorded by the recording equipment in the target scene can be realized through the scheme.

Optionally, in another embodiment of the present invention, the determining whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal may include:

calculating a first similarity between the distribution interval sequence and the interval duration sequence, and determining whether the distribution interval sequence is greater than a first preset threshold or not, or determining whether the distribution interval sequence is a similar subset of the interval duration sequence or not;

if the first similarity is greater than a first preset threshold value or the distribution interval sequence is a similar subset of the interval duration sequence, it is determined that the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, and if the first similarity is not greater than the first preset threshold value or the distribution interval sequence is not the similar subset of the interval duration sequence, it is determined that the playing information of the watermark signal to be traced does not match with the playing information of the physical voice watermark signal.

In this implementation, the following two cases need to be considered:

in the first case, the obtained watermark signal to be traced is a complete signal, and in this case, the similarity between the distribution interval sequence and the interval duration sequence may be calculated as a first similarity; then determining whether the first similarity is larger than a first preset threshold value; and if the first similarity is greater than a first preset threshold value, determining that the distribution interval sequence is matched with the interval duration sequence, otherwise, determining that the distribution interval sequence is not matched with the interval duration sequence.

Considering that the distribution interval duration of the to-be-traced watermark signal in the to-be-traced voice can be different from the originally generated to-be-traced voice in the transmission process of the to-be-traced voice, whether the distribution interval sequence is matched with the interval duration sequence is determined by comparing whether the distribution interval sequence and the interval duration sequence can be in one-to-one correspondence or not, and therefore, whether the distribution interval sequence is matched with the interval duration sequence is determined by calculating the similarity between the distribution interval sequence and the interval duration sequence, and the tracing accuracy of the to-be-traced voice is improved.

In the second case: if the obtained watermark signal to be traced is an incomplete signal, a Time Wrap (Time warping) technique may be used to determine whether the distribution interval sequence is a similar subset of the interval duration sequence, and if so, it may be determined that the distribution interval sequence matches the interval duration sequence. In this case, for example, a Time Wrap technique may be used to extract a sequence segment similar to the distribution interval sequence in the interval duration sequence, and then calculate whether a cosine distance or a Hamming distance between the distribution interval sequence and the extracted sequence segment is greater than a preset distance threshold, if so, it is determined that the distribution interval sequence is a similar subset of the interval duration sequence.

Similarly, in an implementation manner, the determining whether the signal characteristic of the watermark signal to be traced matches the signal characteristic of the physical voice watermark signal may include:

and if the second similarity is larger than a second preset threshold, determining the signal characteristics of the watermark signal to be traced and matched with the signal characteristics of the physical voice watermark signal, otherwise, determining the signal characteristics of the watermark signal to be traced and not matched with the signal characteristics of the physical voice watermark signal.

The similarity between the feature information of the watermark signal to be traced and the feature information of the physical voice watermark signal may be, but is not limited to, calculating a cosine distance or a Hamming distance between the feature information of the watermark signal to be traced and the feature information of the physical voice watermark signal.

In this embodiment, when it is determined that the watermark signal to be traced is matched with the physical voice watermark signal based on the pre-recorded specifying information of the physical voice watermark signal, it may be determined that the voice to be traced is recorded by the recording device in the target scene, so as to trace the source of the voice recorded by the recording device in the target scene. Furthermore, whether the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal is determined by calculating the first similarity of the distribution interval sequence and the interval duration sequence, so that the problem of source tracing error caused by the fact that the values in the distribution interval sequence and the interval duration sequence cannot correspond to each other can be solved, and the accuracy of tracing the voice recorded by the recording equipment in the target scene is further improved.

For convenience of understanding, the following describes an example of a method for injecting a physical voice watermark and a method for tracing a voice source according to an embodiment of the present invention with reference to the drawings.

As shown in fig. 5, in an actual application process, a process of the method for injecting a physical voice watermark provided by the embodiment of the present invention may include:

step 1: and performing content identification, analysis and prediction on the sound signals in the target scene in real time to obtain the characteristics of the target scene and the content carried by the sound signals.

In this step, the artificial intelligence model identification can be used for identifying the sound signals in the target scene in real time, and then feature extraction and scene classification are carried out based on the pre-trained scene classification model, so as to obtain the features of the target scene and the content carried by the sound signals.

Step 2: and determining the type and the content of the sound signal matched with the physical voice of the target scene as a physical voice watermark signal according to a preset rule and based on the characteristics of the target scene and the content carried by the sound signal.

The type of the sound signal may include noise, song, talk, etc., and the content of the sound signal may include identification information of the target scene, a random number password, a random word password, etc.

And step 3: and modulating the physical voice watermark signal to an ultrasonic frequency band.

In this step, the physical voice watermark signal may be modulated to the ultrasonic frequency band by using AM modulation.

And 4, step 4: and playing the modulated physical voice watermark signal according to the randomly generated interval duration sequence, and recording the randomly generated interval duration sequence.

The specified interval duration sequence may be preset by a worker, or may be generated according to a specified rule, where the specified rule may be randomly generated, or generated according to a corresponding relationship between time, a scene, and the interval duration sequence generated by the specified interval duration sequence.

And 5: and the played physical voice watermark signal is naturally superposed with the sound signal in the target scene, so that the physical voice watermark signal is embedded in the sound signal recorded by the recording equipment.

It can be seen that, in this embodiment, since the physical voice watermark signal is played in the target scene, the voice after the playing of the physical voice watermark signal is superimposed on the physical voice in the target environment at the air interface, which means that the watermark information has been embedded in the superimposed voice after the playing of the physical voice watermark signal, at this time, if a recording device exists in the target environment, the recording device can only record the superimposed voice, so that the recorded voice contains the physical voice watermark signal, further, after the playing of the physical voice watermark signal, the specified information of the physical voice watermark signal is recorded, so that the tracing can be performed subsequently by using the specified information, which can provide a basis for performing the tracing on the voice in which the physical voice watermark signal has been embedded.

As shown in fig. 6, in an actual application process, a process of the voice tracing method provided by the embodiment of the present invention may include:

step 1: and acquiring the voice to be traced.

Step 2: and extracting the watermark signal to be traced after the nonlinear demodulation of the recording equipment and the distribution interval sequence of the watermark signal to be traced.

And step 3: and calculating the similarity between the extracted watermark signal to be traced and the physical voice watermark signal as a first similarity.

Wherein the watermark signal F to be traced ₁ The baseband signal which can be a watermark signal to be traced, a physical voice watermark signal F ₀ The first similarity S can be calculated by calculating cosine distance, Hamming distance and other modes ₁ 。

And 4, step 4: and judging whether the extracted distribution interval sequence of the watermark signal to be traced is in the interval duration sequence of the physical voice watermark signal or not, and calculating a second similarity between the distribution interval sequence and the interval duration sequence.

In this step, the distribution interval sequence TI of the extracted watermark signal to be traced can be determined by using the Time Wrap technique ₁ Whether or not in a physical speech watermark signalInterval duration sequence TI ₀ In the interior, sequence segments similar to the distribution interval sequence in the interval time length sequence are extracted, and the cosine distance or the Hamming distance between the distribution interval sequence and the extracted sequence segments is calculated to obtain a second similarity S ₂ 。

And 5: and judging whether the first similarity is greater than a first preset threshold value and whether the second similarity is greater than a second preset threshold value, and if so, determining that the extracted watermark signal to be traced matches with the physical voice watermark signal.

In this step, a first similarity S is detected ₁ And a second degree of similarity S ₂ Whether or not to simultaneously satisfy the preset threshold thr ₁ And thr ₂ (ii) a If yes, determining that the physical voice watermark signal F is embedded in the voice to be traced ₁ 。

As can be seen, in this embodiment, since the physical voice watermark signal is a sound signal that is played in the target scene and matches the physical voice of the target scene, it means that if a recording device exists in the target scene, the voice recorded in the recording device necessarily includes the physical voice watermark signal, and therefore, if it is determined that the watermark signal to be traced back matches the physical voice watermark signal based on the pre-recorded specifying information of the physical voice watermark signal, it may be determined that the voice to be traced back is recorded by the recording device in the target scene. Therefore, the source tracing of the voice recorded by the recording equipment in the target scene can be realized through the scheme.

An embodiment of the present invention further provides an apparatus for injecting a physical voice watermark, as shown in fig. 7, the apparatus includes:

a determining module 710, configured to determine a sound signal matching the physical speech of the target scene as a physical speech watermark signal;

a playing module 720, configured to play the physical voice watermark signal in the target scene according to a preset playing policy, so that a voice recorded by the recording device is a superimposed voice when the recording device exists in the target scene; the superposed voice is a sound signal obtained by superposing the physical voice and the physical voice watermark signal;

a recording module 730, configured to record the specific information of the physical voice watermark signal; wherein, the specified information includes the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal.

Optionally, the apparatus further comprises:

the modulation module is used for modulating the physical voice watermark signal to a specified frequency band to obtain the modulated physical voice watermark signal;

the playing module is specifically configured to play the modulated physical voice watermark signal in the target scene according to a preset playing strategy.

Optionally, the designated frequency band is an ultrasonic frequency band.

Optionally, the playing module is specifically configured to:

playing the physical voice watermark signal for multiple times in the target scene; the playing information includes: and the interval duration sequence of the physical voice watermark signal is a duration sequence formed by the playing interval duration of the physical voice watermark signal.

Optionally, the playing module plays the physical voice watermark signal for multiple times, including:

playing the physical voice watermark signal once in the target scene;

determining the playing interval duration after the playing is finished;

Optionally, the determining, by the playing module, the playing interval duration after the playing is completed includes:

Optionally, the selecting, by the play module, an interval duration from the specified interval duration sequence as the play interval duration after the playing is completed includes:

Optionally, the determining module is specifically configured to determine, when a target scene meets a preset sound watermark injection requirement, a sound signal matched with a physical voice of the target scene as a physical voice watermark signal.

Optionally, the determining module detects that the physical voice in the target scene meets a specified voice condition, including:

detecting a sound signal containing a specified object in physical voice in the target scene; and/or the presence of a gas in the atmosphere,

Optionally, the determining module determines whether the target scene is a designated scene, including:

inputting the physical voice in the target scene into a pre-trained scene classification model to obtain a scene type output by the scene classification model;

determining whether the obtained scene type is a designated type to which a designated scene belongs, if so, determining that the target scene is the designated scene, otherwise, determining that the target scene is not the designated scene; or the like, or, alternatively,

Optionally, the determining module includes:

the first determining submodule is used for determining scene information of the target scene based on the physical voice of the target scene; wherein the determined scene information comprises a scene type of the target scene and/or a signal content of the physical voice;

and the selecting submodule is used for selecting the sound signal corresponding to the determined scene information from the preset sound signals based on the corresponding relation between the preset scene information and the sound signals, and taking the sound signal as the sound signal matched with the physical voice of the target scene.

An embodiment of the present invention further provides a voice tracing apparatus, as shown in fig. 8, the apparatus includes:

an obtaining module 810, configured to obtain a voice to be traced;

an extracting module 820, configured to perform watermark signal extraction on the voice to be traced to obtain a watermark signal to be traced;

a matching module 830, configured to determine whether the watermark signal to be traced is matched with the physical voice watermark signal based on pre-recorded specific information of the physical voice watermark signal; the physical voice watermark signal is a sound signal which is played in a target scene and is matched with the physical voice of the target scene; the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal;

and the tracing module 840 is configured to determine that the voice to be traced is the voice recorded from the target scene if the voice to be traced is matched with the target scene.

Optionally, the matching module includes:

the first obtaining submodule is used for obtaining the playing information of the watermark signal to be traced in the voice to be traced under the condition that the specified information comprises playing information; determining whether the playing information of the watermark signal to be traced is matched with the playing information of the physical voice watermark signal to obtain a first matching result;

the second obtaining submodule is used for determining whether the signal characteristics of the watermark signal to be traced are matched with the signal characteristics of the physical voice watermark signal or not under the condition that the specified information comprises the signal characteristics, and obtaining a second matching result;

and the matching sub-module is used for determining whether the watermark signal to be traced matches with the physical voice watermark signal or not based on the first matching result and/or the second matching result.

Optionally, the playing information of the physical voice watermark signal includes: the interval duration sequence of the physical voice watermark signal is a duration sequence formed by playing interval durations of the physical voice watermark signal when the physical voice watermark signal is played for multiple times in the target scene;

the first obtaining sub-module obtains the playing information of the watermark signal to be traced in the voice to be traced, and the playing information comprises:

Optionally, the first obtaining sub-module includes:

a first calculating unit, configured to calculate a first similarity between the distribution interval sequence and the interval duration sequence, whether the first similarity is greater than a first preset threshold, or determine whether the distribution interval sequence is a similar subset of the interval duration sequence;

a first determining unit, configured to determine that, if the first similarity is greater than the first preset threshold, or the distribution interval sequence is a similar subset of the interval duration sequence, the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, and if the first similarity is not greater than the first preset threshold, or the distribution interval sequence is not the similar subset of the interval duration sequence, determine that the playing information of the watermark signal to be traced does not match with the playing information of the physical voice watermark signal.

Optionally, the second obtaining sub-module includes:

a second calculating unit, configured to calculate a similarity between the signal feature of the watermark signal to be traced and the signal feature of the physical voice watermark signal, as a second similarity;

a second determining unit, configured to determine whether the second similarity is greater than a second preset threshold;

and the matching unit is used for determining the signal characteristics of the watermark signal to be traced to be matched with the signal characteristics of the physical voice watermark signal if the second similarity is greater than the second preset threshold, or else determining the signal characteristics of the watermark signal to be traced to be not matched with the signal characteristics of the physical voice watermark signal.

Optionally, the matching sub-module is specifically configured to:

Optionally, the extracting module is specifically configured to:

extracting a sound signal of a preset frequency band in the voice to be traced as the watermark signal to be traced; and/or the presence of a gas in the atmosphere,

An embodiment of the present invention further provides an electronic device, as shown in fig. 9, which includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904,

a memory 903 for storing computer programs;

the processor 901 is configured to implement the steps of the above-mentioned physical voice watermark injecting method or voice tracing method when executing the program stored in the memory 903.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the above-mentioned physical voice watermark injection method or voice tracing method.

In yet another embodiment of the present invention, a computer program product containing instructions is further provided, which when run on a computer, causes the computer to perform the steps of the above-mentioned method for injecting a physical voice watermark or the method for voice tracing.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for injecting a physical voice watermark, the method comprising:

playing the physical voice watermark signal in the target scene according to a preset playing strategy, so that the voice recorded by the recording equipment is overlapped voice under the condition that the recording equipment exists in the target scene; the superposed voice is the voice obtained by superposing the physical voice and the voice played by the physical voice watermark signal;

2. The method according to claim 1, wherein before playing the physical voice watermark signal in the target scene according to a preset playing strategy, the method further comprises:

the playing the physical voice watermark signal in the target scene according to a preset playing strategy comprises:

3. The method of claim 2, wherein the specified frequency band is an ultrasonic frequency band.

4. The method according to any one of claims 1 to 3, wherein the playing the physical voice watermark signal in the target scene according to a preset playing strategy comprises:

5. The method according to claim 4, wherein playing the physical voice watermark signal multiple times in the target scene comprises:

playing the physical voice watermark signal once in the target scene;

determining the playing interval duration after the playing is finished;

6. The method according to claim 5, wherein the determining the playing interval duration after the playing is completed comprises:

7. The method according to claim 6, wherein the selecting an interval duration from the specified interval duration sequence as the playing interval duration after the playing is completed comprises:

8. The method of claim 6, wherein the specified interval duration sequence is a randomly generated interval duration sequence.

9. The method of claim 1, wherein determining the sound signal matching the physical speech of the target scene as the physical speech watermark signal comprises:

10. The method of claim 9, wherein the target scene satisfies a predetermined sound watermark injection requirement, comprising:

11. The method of claim 10, wherein the detecting that the physical speech in the target scene satisfies a specified speech condition comprises:

detecting that a sound signal of a specified object is contained in physical voice in the target scene; and/or the presence of a gas in the gas,

12. The method of claim 11, wherein determining whether the target scene is a designated scene comprises:

inputting the physical voice in the target scene into a pre-trained scene classification model to obtain a scene type output by the scene classification model; determining whether the obtained scene type is a designated type to which a designated scene belongs, if so, determining that the target scene is the designated scene, otherwise, determining that the target scene is not the designated scene; or the like, or, alternatively,

13. The method of claim 1, wherein determining the sound signal that matches the physical speech of the target scene comprises:

14. A voice tracing method, characterized in that the method comprises:

acquiring a voice to be traced;

determining whether the watermark signal to be traced is matched with the physical voice watermark signal or not based on the pre-recorded specified information of the physical voice watermark signal; the physical voice watermark signal is a sound signal which is played in a target scene and is matched with the physical voice of the target scene; the specified information comprises the signal characteristics of the physical voice watermark signal and/or the playing information of the physical voice watermark signal;

15. The method according to claim 14, wherein the determining whether the watermark signal to be traced matches the physical voice watermark signal based on the pre-recorded specification information of the physical voice watermark signal comprises:

16. The method according to claim 15, wherein the playback information of the physical voice watermark signal comprises: the interval duration sequence of the physical voice watermark signal is a duration sequence formed by playing interval durations of the physical voice watermark signal when the physical voice watermark signal is played for multiple times in the target scene;

17. The method according to claim 16, wherein the determining whether the playback information of the watermark signal to be traced matches the playback information of the physical voice watermark signal comprises:

if the first similarity is greater than the first preset threshold value or the distribution interval sequence is a similar subset of the interval duration sequence, determining that the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal;

and if the first similarity is not greater than the first preset threshold value or the distribution interval sequence is not the similar subset of the interval duration sequence, determining that the playing information of the watermark signal to be traced does not match with the playing information of the physical voice watermark signal.

18. The method according to claim 15, wherein the determining whether the signal characteristics of the watermark signal to be traced matches the signal characteristics of the physical voice watermark signal comprises:

19. The method according to claim 15, wherein the determining whether the watermark signal to be traced matches the physical voice watermark signal based on the first matching result and/or the second matching result comprises:

and if the playing information of the watermark signal to be traced matches with the playing information of the physical voice watermark signal, and/or the signal characteristic of the watermark signal to be traced matches with the signal characteristic of the physical voice watermark signal, determining that the watermark signal to be traced matches with the physical voice watermark signal.

20. The method according to claim 14, wherein the extracting the watermark signal from the speech to be traced to obtain the watermark signal to be traced comprises:

extracting a sound signal with amplitude larger than a preset amplitude threshold value in the voice to be traced as the watermark signal to be traced; and/or the presence of a gas in the atmosphere,

and utilizing a pre-trained watermark extraction model to extract the watermark signal of the voice to be traced to obtain the watermark signal to be traced.

21. An apparatus for injecting a physical voice watermark, the apparatus comprising:

22. A voice tracing apparatus, characterized in that the apparatus comprises:

the obtaining module is used for obtaining the voice to be traced;

23. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

a processor arranged to perform the method steps of any of claims 1 to 13, or the method steps of any of claims 14 to 20, when executing a program stored in a memory.

24. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-13 or the method steps of any one of the claims 14-20.