CN115547343A - Voiceprint recognition method and device, edge device and voiceprint recognition system - Google Patents

Abstract

The scheme is that voice data of a user is obtained at a user terminal, voice print feature extraction is carried out on the voice data at the edge terminal to obtain voice print data, and the voice print data and stored voice print samples are compared at a cloud terminal device to determine the identity of the user. The sound content of the user can not be output from the edge side, so that the sound content is protected more safely; moreover, by transmitting the voiceprint data, the voice content is not transmitted, the pressure of network bandwidth is reduced, and the network delay is reduced.

Description

Voiceprint recognition method, device, edge device and voiceprint recognition system

technical field

The present application relates to the technical field of artificial intelligence, and in particular to a voiceprint recognition method, a voiceprint recognition device, a computer-readable storage medium, an edge device, and a voiceprint recognition system.

Background technique

Voiceprint is a non-contact biometric feature, which is the same as human biometric features such as face, iris, fingerprints, finger veins, and palm prints. It has the uniqueness of identity, that is, an identity can be uniquely determined by speaking a sentence.

Compared with other authentication technologies, voiceprint recognition is more convenient and safer. Therefore, voiceprint-based multi-modal intelligent authentication is currently widely used in many fields such as intelligent dialogue. However, in the prior art, the user's voice data is often directly sent through the network to a device with powerful data computing capabilities, such as a cloud server, to extract the user's voiceprint features and perform voiceprint comparison.

However, in practical applications, the actual needs of data privacy protection proposed by users and regulatory agencies are often encountered. For example, voiceprint-based identity authentication in the financial field does not allow the user's voice data content to be directly submitted to the verification system for non-inductive verification. Voiceprint identity registration and verification.

Therefore, there is an urgent need to provide a voiceprint recognition scheme with better privacy protection of voice data to solve the above-mentioned problems in the prior art.

Contents of the invention

In order to solve the problem of voice data privacy protection, the present application provides a voiceprint recognition method, a voiceprint recognition device, a computer-readable storage medium, an edge device, and a voiceprint recognition system.

According to an aspect of the embodiments of the present application, a voiceprint recognition method is disclosed, which is used for an edge device, and the edge device is used for connecting with a user terminal and a cloud device. The voiceprint recognition method includes:

Receive user voice data sent by the user terminal;

performing voiceprint feature extraction on the voice data to obtain voiceprint data;

Sending the voiceprint data to the cloud device, so that the cloud device can compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

According to an aspect of the embodiment of the present application, a voiceprint recognition device is disclosed, which is used for an edge device, and the edge device is used for connecting with a user terminal and a cloud device. The voiceprint recognition device includes a receiving module, a feature extraction module, and a sending module, wherein the receiving module is used to receive user voice data sent by a user terminal; the feature extraction module is used to perform voiceprint features on the voice data The voiceprint data is extracted; the sending module is used to send the voiceprint data to the cloud device, so that the cloud device can compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

According to an aspect of the embodiments of the present application, an edge device is disclosed, which is used for connecting with a user terminal and a cloud device. The edge device includes one or more processors and memory, the memory is used to store one or more programs, when the one or more programs are executed by the one or more processors, the The edge device implements the voiceprint recognition method described above.

According to an aspect of the embodiments of the present application, a computer-readable storage medium is disclosed, the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor of a computer, all The computer executes the voiceprint recognition method as described above.

According to an aspect of the embodiment of the present application, a voiceprint recognition method is disclosed, which is used in a voiceprint recognition system, and the voiceprint recognition system includes a user terminal, an edge device connected to the user terminal, and an edge device connected to the edge The cloud device connected to the end device. The voiceprint recognition method includes:

Obtaining the voice data of the user through the user terminal, and sending the voice data to the edge device;

performing voiceprint feature extraction on the voice data by the edge device to obtain voiceprint data, and sending the voiceprint data to a cloud device;

The user identity is determined by comparing the voiceprint data with the stored voiceprint samples by the cloud device.

In an exemplary embodiment, the acquiring the voice data of the user through the user terminal includes:

Obtain the voice stream that contains the voice data of both parties in the call through the user terminal; the two parties include the user and the verification customer service, the user's voice data includes the user's IP address, and the verification customer service The voice data contains the IP address of the verification customer service, and the IP address of the user is different from the IP address of the verification customer service;

Acquire the user's voice data from the voice stream according to the IP address.

In an exemplary embodiment, the cloud device establishes multiple voiceprint libraries according to geographic regions, and at least one of the multiple voiceprint libraries stores voiceprint samples of users corresponding to the geographic regions.

In an exemplary embodiment, the comparing the voiceprint data with the stored voiceprint samples by the cloud device to determine the identity of the user includes:

The cloud device determines the geographical area where the user is located according to the IP address corresponding to the voiceprint data;

According to the geographical area where the user is located, select the voiceprint library corresponding to the language of the geographical area;

The identity of the user is determined by comparing the voiceprint data with the voiceprint samples of the voiceprint library.

In an exemplary embodiment, before comparing the voiceprint data with the stored voiceprint samples by the cloud device, the voiceprint recognition method further includes:

Collect multiple pieces of sound data of each user through the user terminal, and send the multiple pieces of sound data to the edge device;

performing voiceprint feature extraction on multiple pieces of said voice data through said edge device, and sending voiceprint data corresponding to multiple pieces of said voice data to a cloud device;

The voiceprint samples corresponding to each user are determined by the cloud device according to multiple pieces of voiceprint data corresponding to the voice data.

In an exemplary embodiment, before performing voiceprint feature extraction on the voice data by the edge device to obtain the voiceprint data, the voiceprint recognition method further includes:

Training a neural network model on the cloud device to obtain a neural network model whose loss function meets preset requirements;

Sending the neural network model whose loss function meets the preset requirements to each of the edge devices, so that the edge device extracts voiceprint features from the voice data based on the neural network model to obtain voiceprint data.

According to an aspect of the embodiment of the present application, a voiceprint recognition system is disclosed, including a user terminal, an edge device connected to the user terminal, and a cloud device connected to the edge device; wherein the user terminal It is used to acquire the voice data of the user, and send the voice data to the edge device; the edge device is used to perform voiceprint feature extraction on the voice data to obtain voiceprint data, and send the voiceprint The data is sent to the cloud device; the cloud device is used to compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

The technical solutions provided by the embodiments of the present application at least include the following beneficial effects:

In the technical solution provided by this application, the user's voice data is obtained at the user terminal, the voiceprint feature is extracted from the voice data at the edge device, and the voiceprint data is obtained, and the voiceprint data and the stored voiceprint samples are compared on the cloud device, thereby Identify the user. The user's voice content does not go out of the edge side, which has a more secure protection for the voice content; moreover, through the transmission of voiceprint data, the voice content is not transmitted, reducing the pressure on network bandwidth and network delay.

It is to be understood that both the foregoing general description and the following detailed description are exemplary only and are not restrictive of the application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

Fig. 1 is a schematic diagram of a voiceprint recognition system according to an exemplary embodiment;

Fig. 2 is a flowchart of a voiceprint recognition method according to an exemplary embodiment;

Fig. 3 is a detailed flowchart of step S102 in the embodiment corresponding to Fig. 2;

Fig. 4 is a detailed logical block diagram of step S102 in the embodiment corresponding to Fig. 2;

Fig. 5 is a block diagram of a voiceprint recognition device according to an exemplary embodiment;

Fig. 6 is a block diagram of an edge device according to an exemplary embodiment;

Fig. 7 is a block diagram of a computer system shown according to an exemplary embodiment;

Fig. 8 is a flowchart of a voiceprint recognition method according to an exemplary embodiment;

Fig. 9 is a detailed flowchart of step S201 in the embodiment corresponding to Fig. 8;

Fig. 10 is a detailed flowchart of step S2012 in the embodiment corresponding to Fig. 9;

Fig. 11 is a detailed flowchart of step S203 in the embodiment corresponding to Fig. 8;

Fig. 12 is a flow chart showing a voiceprint sample collection according to an exemplary embodiment;

Fig. 13 is a flowchart of a voiceprint recognition method according to another exemplary embodiment.

The reference signs are explained as follows:

100. Voiceprint recognition system; 101. User terminal; 102. Edge device; 103. Cloud device; 104. UPF; 105. Edge device; 106. Network; 200. Voiceprint recognition device; 201. Receiving module; 202 , feature extraction module; 203, sending module; 300, edge device; 301, processor; 302, memory; 400, computer system; 401, CPU; 402, ROM; 403, storage part; 404, RAM; 405, bus ; 406, I/O interface; 407, input part; 408, output part; 409, communication part; 410, driver; 411, removable medium.

detailed description

Although the present application may readily take the form of embodiments, only some of them are shown in the drawings and will be described in detail in this specification, and it is to be understood that this specification should be considered as the It is an exemplary illustration of the principles of the application and is not intended to limit the application to what is described herein.

In addition, the terms "comprising", "comprising", "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or modules is not limited to the listed steps or modules, but optionally also includes other unlisted steps or modules, or optional Other steps or modules inherent to these processes, methods, products or devices are also included.

In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined as "first", "second", and "third" may explicitly or implicitly include one or more features.

In the description of the present application, unless otherwise specified, the meaning of "plurality" refers to two or more.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "example" are used to represent examples, illustrations or illustrations. Any embodiment or design described as "exemplary" or "example" in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner.

Exemplary embodiments will be described in detail below. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as detailed in this Summary.

Exemplary embodiments of the present application provide a voiceprint recognition method, a voiceprint recognition device, an edge device, a computer-readable storage medium, and a voiceprint recognition system. Fig. 1 shows a schematic diagram of a voiceprint recognition system that can implement the voiceprint recognition method of the exemplary embodiment of the present application. As shown in FIG. 1 , the voiceprint recognition system 100 may include a user terminal 101, an edge device 102, and a cloud device 103, wherein the user terminal 101 and the edge device 102 are connected in communication to realize communication between the user terminal 101 and the edge device 102. The cloud device 103 communicates with the edge device 102 to realize information exchange between the cloud device 103 and the edge device 102 .

In detail, the user terminal 101 and the edge device 102 may be connected in a wired manner, for example, connected through an RS485 bus, a USB cable, or an Ethernet. The user terminal 101 and the edge device 102 may also be connected wirelessly, for example, through WI-FI, Bluetooth, ZigBee, LoRa (Long Range Radio, long-distance radio) and the like.

In detail, the edge device 102 and the cloud device 103 can be communicated and connected through 4G communication, 5G communication and other means.

Understandably, the user terminal 101 may be, for example, a smart phone, a phone watch, a tablet computer supporting a phone function, and the like.

It can be understood that the edge device 102 may be any electronic device capable of data interaction and data processing, such as a computer installed with a voiceprint feature extraction program. Likewise, the cloud device 103 may be any electronic device with information interaction capability and data processing capability, such as a computer installed with a voiceprint library and a voiceprint comparison program.

Embodiments of the present application provide a voiceprint recognition method, a voiceprint recognition device, an edge device, a computer-readable storage medium, and a voiceprint recognition system, which can be used in scenarios such as smart identity verification. Since the voice content of the user does not go out of the edge side in this application, the voice content is more securely protected and the privacy protection is better. The voiceprint recognition method, voiceprint recognition device, edge device, computer-readable storage medium, and voiceprint recognition system provided in the embodiments of the present application are specifically described through the following embodiments.

Referring to FIG. 2 , this embodiment provides a voiceprint recognition method, which is used for an edge device. The voiceprint recognition method includes the following steps S101-S103.

S101. Receive user voice data sent by a user terminal.

S102, performing voiceprint feature extraction on the received voice data to obtain voiceprint data.

In detail, the voiceprint feature extraction is performed on the received voice data, including decoding the received voice data, and further performing operations such as discrete Fourier transform on the basis of decoding, and then extracting voiceprint features.

In more detail, as shown in FIG. 3, step S102 includes the following steps S1021-S1025.

S1021. Decode the received sound data.

It can be understood that during the process of transmitting voice data, the voice data will be encoded. Therefore, after receiving the voice data sent by the user terminal, the voice data needs to be decoded first, and then the following voiceprint feature extraction operation is performed.

S1022. Perform preprocessing on the decoded sound data to obtain first data.

In detail, in step S1022, the audio data is preprocessed, including pre-emphasis, framing, and windowing. As shown in FIG. 4 , first, pre-emphasize the decoded sound data; then, perform frame division on the pre-emphasized sound data; and then perform windowing operation on the frame-divided sound data. Since many features of the sound data are concentrated in the high-frequency band, and the higher the frequency, the smaller the spectral value. Therefore, the high-frequency part of the sound data is pre-emphasized first, so as to improve the high-frequency resolution of the sound data. Framing can obtain the short-term characteristics of the sound data, and further performing a windowing operation on the framed sound data can reduce the discontinuity between frames.

It can be understood that preprocessing the sound data is not limited to performing pre-emphasis, framing, and windowing operations.

S1023. Perform discrete Fourier transform (Discrete Fourier Transform, DFT for short) on the first data obtained in step S1022 to obtain second data.

For example, the discrete Fourier transform is performed layer by layer on the sound data after the windowing operation, and bit-permuted complex data, that is, the second data is output.

S1024. Perform Mel filtering on the second data obtained in step S1023 to obtain third data.

For example, odd-even separation processing is performed on the energy value of the second data, and multi-order Mel filtering is performed on the energy value of the odd time index or even time index to obtain the multi-order Mel value of the current frame, that is, the third data.

S1025. Perform a logarithmic operation on the third data obtained in step S1024 to obtain voiceprint data.

S103. Send the voiceprint data to the cloud device, so that the cloud device can compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

Referring to FIG. 5 , this embodiment provides a voiceprint recognition device 200 , which is used for an edge device, and includes a receiving module 201 , a feature extraction module 202 and a sending module 203 . Wherein, the receiving module 201 is used to receive the user's voice data sent by the user terminal; the feature extraction module 202 is used to perform voiceprint feature extraction on the voice data to obtain the voiceprint data; the sending module 203 is used to send the voiceprint data to the cloud device for The cloud device can compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

In detail, the feature extraction module 202 first decodes the received sound data; then preprocesses the decoded sound data to obtain the first data, and the preprocessing includes pre-emphasis, framing, and windowing; Discrete Fourier transform is performed on the data to obtain second data; then Mel filtering is performed on the second data to obtain third data; finally, logarithmic operation is performed on the third data to obtain voiceprint data.

For the implementation process of the functions and functions of each module in the above-mentioned voiceprint recognition device 200, please refer to the implementation process of the corresponding steps in the above-mentioned voiceprint recognition method for details, and will not be repeated here.

Referring to FIG. 6, this embodiment provides an edge device 300, the edge device 300 includes one or more processors 301 and a memory 302, the memory 302 is used to store one or more programs, when one or more When the program is executed by one or more processors 301, the edge device 300 implements the voiceprint recognition method of the present application.

Fig. 7 schematically shows a structural block diagram of a computer system for implementing some embodiments of the present application. It should be noted that the computer system shown in FIG. 7 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in Figure 7, the computer system 400 includes a central processing unit 401 (Central Processing Unit, CPU), which can be stored in a program in a read-only memory 402 (Read-Only Memory, ROM) or loaded from a storage part 403 to a random Various appropriate actions and processes are executed by accessing programs in the memory 404 (Random Access Memory, RAM). In the random access memory 404, various programs and data necessary for device operation are also stored. The CPU 401 , the read only memory 402 and the random access memory 404 are connected to each other through a bus 405 . An input/output interface 406 (Input/Output interface, ie, I/O interface) is also connected to the bus 405 .

The following components are connected to the input/output interface 406: an input section 407 including a keyboard, a mouse, etc.; an output section 408 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker ; a storage section 403 including a hard disk or the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the Internet. A driver 410 is also connected to the input/output interface 406 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 410 as necessary so that a computer program read therefrom is installed into the storage section 403 as necessary.

In particular, according to the embodiments of the present application, the processes described in the respective method flowcharts can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 409 and/or installed from removable media 411 . When the computer program is executed by the central processing unit 401, various functions defined in the device of the present application are performed.

It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable The combination. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution device, device, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

Referring next to FIG. 8 , the voiceprint recognition method provided by an exemplary embodiment of the present application includes the following steps S201-S203.

S201. Obtain voice data of a user through a user terminal, and send the voice data to an edge device.

In a call scenario, since the voice data of the two parties in the call will be mixed together, role separation is required to distinguish the voice data of the two parties in the call. For example, if the user is the listener and the customer service is verified as the caller, the voice data of the user (the listener) needs to be obtained for further processing to verify the identity of the user.

In an exemplary embodiment, the user's voice data includes the user's IP address, and the voice data of the verification customer service includes the IP address of the verification customer service, and the user's IP address is different from the IP address of the verification customer service. to identify users. This method does not need to separate roles through artificial intelligence algorithms, which is more efficient; and distinguishing roles through IP addresses is more accurate.

In an exemplary embodiment, as shown in FIG. 9, step S201 includes the following steps S2011-S2013.

S2011. Obtain, through the user terminal, a voice stream that includes voice data of the user and the verification customer service during the call.

Wherein, the voice data of the user includes the IP address of the user, and the voice data of the verification customer service includes the IP address of the verification customer service, and the user's IP address is different from the IP address of the verification customer service.

S2012. According to the IP address, acquire the user's voice data from the voice stream.

Specifically, each piece of voice data includes an IP address and voice content, and by identifying the IP address, users can be distinguished and customer service verified.

For example, socket (socket), scapy, etc. may be used to obtain user voice data.

In more detail, the IP address of the verification customer service is relatively fixed, while the users are constantly changing, and different users have different IP addresses. For example, the IP address includes four sections, and the first three sections of the IP address of the verification customer service are the same, and only the fourth section is different because of different verification customer service seats. In an exemplary embodiment, as shown in Figure 10, Acquiring the user's voice data from the voice stream according to the IP address includes the following steps S20121-S20122.

S20121. Identify the sound data whose IP address in the sound stream is consistent with the first three segments of the IP address of the verified customer service.

S20122, taking the sound data other than the sound data whose IP address is consistent with the first three segments of the IP address of the verification customer service as the sound data of the current user.

By identifying the first three segments of the IP address of the verification customer service, the role separation between the user and the verification customer service can be quickly realized, and the user's voice data can be obtained from the voice stream; moreover, the role separation is more accurate.

S2013, sending the voice data of the user to the edge device.

In detail, a voiceprint recognition system may include multiple edge devices, and the same user terminal may establish communication connections with multiple edge devices. In step S2013, the user's voice data may be sent to multiple edge devices Among them, an edge device that is geographically closest to the current user terminal can reduce the transmission distance of voice data, thereby improving the transmission efficiency of voice data and ensuring the transmission quality.

Understandably, it is not limited to distinguishing users and verifying customer service through IP addresses, and further obtaining user voice data based on IP addresses. In other embodiments, other methods can also be used to obtain user voice data from voice streams containing voice data of both parties. sound data.

The above-mentioned embodiments are described in the scenario of a two-party conversation. It can be understood that the present application is not only applicable to the scenario of a two-party conversation.

S202. Using the edge device to extract voiceprint features from the voice data to obtain voiceprint data, and send the voiceprint data to the cloud device.

In detail, the edge device first receives the sound data sent by the user terminal, and decodes the sound data; then, preprocesses the decoded sound data to obtain the first data; and then performs discrete Fourier analysis on the obtained first data leaf transformation to obtain the second data; then Mel filtering is performed on the obtained second data to obtain the third data; finally, logarithmic operation is performed on the obtained third data to obtain the voiceprint data.

S203. Determine the identity of the user by comparing the voiceprint data with the stored voiceprint samples through the cloud device.

Since the voiceprint characteristics may be different when the user speaks in different languages, in an exemplary embodiment, multiple voiceprint libraries are established on the cloud device according to geographical regions. In this exemplary embodiment, at least one of the plurality of voiceprint databases stores voiceprint samples corresponding to geographic regions of the user.

It can be understood that the same voiceprint library may store voiceprint samples of users speaking in different languages, that is, the same voiceprint library may contain voiceprint samples in multiple languages. Similarly, voiceprint samples of the same user may be stored in multiple voiceprint libraries.

For example, for a region where Mandarin is the language of daily communication, the voiceprint database of the region stores the corresponding voiceprint features when users speak Mandarin; for a region where Mandarin is the language of daily communication, the voiceprint database of the region stores the user When speaking in a dialect, the corresponding voiceprint features; for areas where English is used as a daily communication language, the voiceprint database in this area stores the corresponding voiceprint features when users speak in English; for areas where Mandarin and dialects are used as daily communication languages region, the voiceprint library in this region stores the corresponding voiceprint features when the user speaks in Mandarin and when speaking in a dialect.

In an exemplary embodiment, as shown in FIG. 11 , step S203 includes the following steps S2031-S2033.

S2031. The cloud device determines the geographical area where the user is located according to the IP address corresponding to the voiceprint data.

S2032. According to the geographic region where the user resides, select a voiceprint library corresponding to the language of the geographic region.

It can be understood that the language of the geographical area can include common language and dialects other than common language, wherein the common language includes Chinese common language (such as Mandarin), foreign common language (such as English) and the like; dialects can be, for example, Cantonese, Hakka, etc.

S2033, comparing the voiceprint data with the voiceprint samples in the voiceprint database to determine the identity of the user.

Understandably, if the same voiceprint database stores voiceprint samples of the same user when speaking in different languages, in step S2033, it is also possible to combine the voiceprint data of the user with the corresponding voiceprint samples when speaking in different languages Compare the results to determine the identity of the user.

Next, please refer to FIG. 12 , which shows a flow chart of voiceprint sample collection in an exemplary embodiment. Before performing step S203 to compare the voiceprint data with the stored voiceprint samples by the cloud device to determine the identity of the user, the voiceprint samples need to be collected first. As shown in Fig. 12, in an exemplary embodiment, collecting voiceprint samples includes the following steps S301-S303.

S301. Collect multiple pieces of voice data of each user through a user terminal, and send the multiple pieces of voice data to an edge device.

S302. Perform voiceprint feature extraction on multiple pieces of voice data through the edge device, and send the voiceprint data corresponding to the multiple pieces of voice data to the cloud device.

S303. Determine the voiceprint samples corresponding to each user through the cloud device according to the voiceprint data corresponding to the pieces of voice data.

Usually, multiple pieces of sound data of the user are collected during the user registration stage, and the multiple pieces of sound data are used as registration sound data, which is used as the basis for identity verification during the user's subsequent operations. Generally, the duration of each piece of sound data in the registration phase is 6-8 seconds.

Since the voiceprint characteristics of users speaking in different time periods or in different states are different, the voiceprint samples obtained by collecting multiple pieces of voice data of each user through the user terminal can more accurately reflect the voiceprint characteristics of the user. The generalization ability of recognition is also stronger.

For example, in an exemplary embodiment, in step S303, the average value of the voiceprint data corresponding to multiple pieces of voice data of each user is acquired through the cloud device, and the average value is used as the voiceprint sample corresponding to each user. Of course, the method is not limited to averaging voiceprint data corresponding to multiple pieces of voice data to obtain voiceprint samples corresponding to each user.

Understandably, before step S202 is performed to obtain voiceprint data through the voiceprint feature extraction of the voice data by the edge device, a neural network model needs to be set on the edge device, and the neural network model needs to be trained first, so that the neural network The model can realize the voiceprint feature extraction in this application. Referring to FIG. 13 , in an example embodiment, the voiceprint recognition method includes the following steps S401-S405.

S401. Train a neural network model on a cloud device, and obtain a neural network model whose loss function meets a preset requirement.

In detail, the loss function adopts AM-softmax function, which has stronger robustness and generalization.

S402. Send the neural network model whose loss function meets the preset requirements to each edge device.

S403. Obtain voice data of the user through the user terminal, and send the voice data to the edge device.

S404. Using the neural network model to extract voiceprint features from the voice data through the edge device to obtain voiceprint data, and send the voiceprint data to the cloud device.

S405. Using the cloud device to compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

By uniformly training the neural network model on cloud devices and then distributing updates to each edge device, the uniformity and effectiveness of the neural network model can be maintained.

In summary, this application adopts the voiceprint feature extraction and verification method based on cloud-edge-device collaboration, and realizes voiceprint verification without the user's voice content coming out of the edge side; at the same time, there is no need to install a plug-in monitoring network card to capture all Voice streaming, without monitoring the network card and all the traffic flowing through the network card, can be applied to application scenarios with high requirements for information privacy; moreover, the intensive voiceprint library and the comparison of voiceprint features are done on the cloud device, and the voiceprint There is unified security control; in addition, because the edge device transmits the user's voiceprint data, it does not transmit the voice content, which reduces the pressure on the network bandwidth and reduces the network delay.

Next, please refer to FIG. 1 , which is a schematic diagram of a voiceprint recognition system according to an exemplary embodiment. As shown in FIG. 1 , the voiceprint recognition system 100 includes a user terminal 101 , an edge device 102 and a cloud device 103 , the user terminal 101 is connected to the edge device 102 , and the edge device 102 is connected to the cloud device 103 .

In detail, the user terminal 101 is used to obtain voice data of the user, and send the voice data to the edge device 102 . The edge device 102 is used to extract voiceprint features from the voice data to obtain voiceprint data, and send the voiceprint data to the cloud device 103 . The cloud device 103 is used to compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

It can be understood that in addition to voice verification services, the voiceprint recognition system 100 may also involve other services, such as normal call services. In the voiceprint recognition system 100, there is also a UPF (User Plane Function) 104. In addition to the edge device 102 for voiceprint feature extraction, there is also an edge device 105 for other business processing. Each user The terminal 101 is connected to the UPF104 in communication, and the data sent by each user terminal 101 is sent to the UPF104, and the UPF104 splits the data, forwards the voice data of the voice verification service to the corresponding edge device 102, and forwards other data to the edge device 105, the edge device The end device 102 then forwards the data to the cloud device 103 and the edge end device 105 through the network 106 .

In detail, the user terminal 101 acquires the voice stream that includes the voice data of the user and the verification customer service during the call, the user's voice data includes the user's IP address, and the voice data of the verification customer service includes the IP address of the verification customer service, and the user The IP address of the user is different from the IP address of the verification customer service; the user terminal 101 obtains the user's voice data from the voice stream according to the IP address.

In detail, the edge device 102 receives the voice data sent by the user terminal 101, and decodes the voice data; performs preprocessing on the decoded voice data to obtain the first data, and the preprocessing includes pre-emphasis, framing, and windowing; Discrete Fourier transform is performed on the first data to obtain second data; Mel filtering is performed on the second data to obtain third data; logarithmic operation is performed on the third data to obtain voiceprint data.

In detail, the cloud device 103 establishes multiple voiceprint libraries according to geographic regions, and at least one of the multiple voiceprint libraries stores voiceprint samples of users corresponding to geographic regions.

In detail, the cloud device 103 determines the geographical area where the user is located according to the IP address corresponding to the voiceprint data; selects the voiceprint library corresponding to the language of the geographical area according to the geographical area where the user is located; compares the voiceprint data with the voiceprint data. The voiceprint sample of the fingerprint database is used to determine the user's identity.

In detail, the user terminal 101 collects multiple pieces of voice data of each user, and sends the multiple pieces of voice data to the edge device 102; the edge device 102 performs voiceprint feature extraction on the multiple pieces of voice data, and maps the multiple pieces of voice data to The voiceprint data of each user is sent to the cloud device 103; the cloud device 103 determines the voiceprint samples corresponding to each user according to the voiceprint data corresponding to the multiple pieces of voice data.

In more detail, the cloud device 103 obtains an average value of voiceprint data corresponding to multiple pieces of voice data of each user, and uses the average value as a voiceprint sample corresponding to each user.

In detail, the cloud device 103 is also used to train the neural network model to obtain the neural network model whose loss function meets the preset requirements; and sends the neural network model whose loss function meets the preset requirements to each edge device 102, so that the edge The terminal device 102 performs voiceprint feature extraction on the voice data based on the neural network model to obtain voiceprint data.

For a more detailed implementation process of the functions and functions of each component in the above-mentioned voiceprint recognition system 100, please refer to the implementation process of the corresponding steps in the above-mentioned voiceprint recognition method for details, and will not be repeated here.

In the technical solution provided by this application, the user's voice data is obtained at the user terminal, the voiceprint feature is extracted from the voice data at the edge device, and the voiceprint data is obtained, and the voiceprint data and the stored voiceprint samples are compared on the cloud device, thereby Identify the user. The user's voice content does not go out of the edge side, which has a more secure protection for the voice content; moreover, through the transmission of voiceprint data, the voice content is not transmitted, reducing the pressure on network bandwidth and network delay.

Those skilled in the art should be aware that, in the above one or more examples, the functions described in this application may be implemented by hardware, software, firmware or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable storage medium.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation. For example a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted, or not implemented.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A voiceprint recognition method is used for an edge device, and the edge device is used for being connected with a user terminal and a cloud device, and is characterized by comprising the following steps:

receiving user voice data sent by a user terminal;

extracting voice print characteristics of the voice data to obtain voice print data;

and sending the voiceprint data to cloud equipment so that the cloud equipment can compare the voiceprint data with the stored voiceprint samples to determine the identity of the user.

2. The utility model provides a voiceprint recognition device for edge end equipment, edge end equipment is used for being connected with user terminal and high in the clouds equipment, its characterized in that, voiceprint recognition device includes:

the receiving module is used for receiving user voice data sent by a user terminal;

the characteristic extraction module is used for extracting the voiceprint characteristics of the voice data to obtain voiceprint data;

and the sending module is used for sending the voiceprint data to the cloud end equipment so that the cloud end equipment can compare the voiceprint data with the stored voiceprint samples to determine the user identity.

3. The utility model provides an edge end equipment for be connected with user terminal and high in the clouds equipment, its characterized in that, edge end equipment includes:

one or more processors;

memory storing one or more programs that, when executed by the one or more processors, cause the edge-end device to implement the voiceprint recognition method of claim 1.

4. A computer-readable storage medium storing computer-readable instructions which, when executed by a processor of a computer, cause the computer to perform the voiceprint recognition method of claim 1.

5. A voiceprint recognition method is used for a voiceprint recognition system, and the voiceprint recognition system comprises a user terminal, edge end equipment connected with the user terminal and cloud end equipment connected with the edge end equipment; the voiceprint recognition method is characterized by comprising the following steps:

acquiring sound data of a user through a user terminal, and sending the sound data to edge terminal equipment;

performing voiceprint feature extraction on the voice data through the edge terminal equipment to obtain voiceprint data, and sending the voiceprint data to cloud equipment;

and comparing the voiceprint data with the stored voiceprint sample through the cloud equipment to determine the user identity.

6. The voiceprint recognition method according to claim 5, wherein the obtaining of the voice data of the user by the user terminal comprises:

acquiring a sound stream containing sound data of both parties of a call in the call process through a user terminal; the two parties of the call comprise the user and a verification customer service, the voice data of the user comprises the IP address of the user, the voice data of the verification customer service comprises the IP address of the verification customer service, and the IP address of the user is different from the IP address of the verification customer service;

and acquiring the voice data of the user from the voice stream according to the IP address.

7. The voiceprint recognition method according to claim 5, wherein the cloud device establishes a plurality of voiceprint libraries according to a geographic area, at least one of the voiceprint libraries storing voiceprint samples corresponding to the geographic area for a user;

the comparing of the voiceprint data and the stored voiceprint sample through the cloud device determines the user identity, and includes:

the cloud device determines the geographic area where the user is located according to the IP address corresponding to the voiceprint data;

selecting a voiceprint library of a language corresponding to a geographical area according to the geographical area where a user is located;

and comparing the voiceprint data with the voiceprint sample of the voiceprint library to determine the identity of the user.

8. The voiceprint recognition method of claim 5, wherein prior to comparing the voiceprint data with the stored voiceprint samples by the cloud device, the voiceprint recognition method further comprises:

acquiring a plurality of pieces of sound data of each user through a user terminal, and sending the plurality of pieces of sound data to edge terminal equipment;

performing voiceprint feature extraction on the plurality of pieces of sound data through the edge terminal equipment, and sending the voiceprint data corresponding to the plurality of pieces of sound data to cloud end equipment;

and determining the voiceprint samples corresponding to the users according to the voiceprint data corresponding to the plurality of pieces of voice data through the cloud equipment.

9. The voiceprint recognition method according to claim 5, wherein before obtaining the voiceprint data by the voiceprint feature extraction of the sound data by the edge device, the voiceprint recognition method further comprises:

training a neural network model on the cloud equipment to obtain the neural network model with a loss function meeting preset requirements;

and issuing a neural network model with a loss function meeting preset requirements to each edge terminal device so that the edge terminal devices perform voiceprint feature extraction on the sound data based on the neural network model to obtain voiceprint data.

10. A voiceprint recognition system is characterized by comprising a user terminal, edge end equipment connected with the user terminal and cloud end equipment connected with the edge end equipment;

the user terminal is used for acquiring sound data of a user and sending the sound data to the edge terminal equipment;

the edge terminal equipment is used for extracting voiceprint features of the voice data to obtain voiceprint data and sending the voiceprint data to the cloud terminal equipment;

and the cloud equipment is used for comparing the voiceprint data with the stored voiceprint samples to determine the user identity.

Images