CN116013264A

CN116013264A - Method, device, electronic equipment and medium for obtaining voice training data

Info

Publication number: CN116013264A
Application number: CN202211678278.5A
Authority: CN
Inventors: 周毅
Original assignee: Apollo Zhilian Beijing Technology Co Ltd; Apollo Zhixing Technology Guangzhou Co Ltd
Current assignee: Apollo Zhilian Beijing Technology Co Ltd; Apollo Zhixing Technology Guangzhou Co Ltd
Priority date: 2022-12-26
Filing date: 2022-12-26
Publication date: 2023-04-25

Abstract

The disclosure provides a method, a device, electronic equipment and a medium for obtaining voice training data, and relates to the field of data processing, in particular to the field of audio processing. The method for obtaining speech training data may include: the method comprises the steps of obtaining an audio stream containing at least one voice instruction by carrying out audio acquisition operation on the at least one voice instruction which is played in sequence; identifying at least one instruction data from the audio stream, each instruction data of the at least one instruction data comprising an instruction text, an audio start point, and an audio end point of a respective voice instruction of the at least one voice instruction; and storing the audio stream and the at least one instruction data as speech training data.

Description

Method, device, electronic equipment and medium for obtaining voice training data

Technical Field

The present disclosure relates to the field of data processing technology, and in particular to audio processing, and more particularly to a method, apparatus, electronic device, computer readable storage medium and computer program product for obtaining speech training data.

Background

More and more intelligent devices have audio acquisition and voice interaction capabilities. In order to train models such as speech recognition, a large amount of audio is often required, such as recognition audio, wake-up audio, etc. In order to ensure the accuracy of the functions of identifying, waking up and the like of the required model, the audio needs to be marked.

The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, the problems mentioned in this section should not be considered as having been recognized in any prior art unless otherwise indicated.

Disclosure of Invention

The present disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for obtaining speech training data.

According to an aspect of the present disclosure, there is provided a method for obtaining speech training data, comprising: the method comprises the steps of obtaining an audio stream containing at least one voice instruction by carrying out audio acquisition operation on the at least one voice instruction which is played in sequence; identifying at least one instruction data from the audio stream, each instruction data of the at least one instruction data comprising an instruction text, an audio start point, and an audio end point of a respective voice instruction of the at least one voice instruction; and storing the audio stream and the at least one instruction data as speech training data.

According to another aspect of the present disclosure, there is provided an apparatus for obtaining speech training data, comprising: an audio stream obtaining unit, configured to obtain an audio stream containing at least one voice instruction by performing an audio acquisition operation on at least one voice instruction that is played in sequence; an instruction data identifying unit for identifying at least one instruction data from the audio stream, each of the at least one instruction data including an instruction text, an audio start point, and an audio end point of a corresponding one of the at least one voice instruction; and a training data storage unit for storing the audio stream and the at least one instruction data as speech training data.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method for obtaining speech training data in accordance with one or more embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform a method for obtaining speech training data according to one or more embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a method for obtaining speech training data according to one or more embodiments of the present disclosure.

In accordance with one or more embodiments of the present disclosure, automatically labeled speech training data may be obtained.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The accompanying drawings illustrate exemplary embodiments and, together with the description, serve to explain exemplary implementations of the embodiments. The illustrated embodiments are for exemplary purposes only and do not limit the scope of the claims. Throughout the drawings, identical reference numerals designate similar, but not necessarily identical, elements.

FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, in accordance with an embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of a method for obtaining speech training data according to an embodiment of the present disclosure;

3A-3B illustrate various examples of speech training data according to embodiments of the present disclosure;

FIG. 4 shows a data flow diagram according to an embodiment of the present disclosure;

FIG. 5 shows a block diagram of an apparatus for obtaining speech training data according to an embodiment of the present disclosure;

fig. 6 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the present disclosure, the use of the terms "first," "second," and the like to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of the elements, unless otherwise indicated, and such terms are merely used to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, they may also refer to different instances based on the description of the context.

The terminology used in the description of the various illustrated examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, the elements may be one or more if the number of the elements is not specifically limited. Furthermore, the term "and/or" as used in this disclosure encompasses any and all possible combinations of the listed items.

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented, in accordance with an embodiment of the present disclosure. Referring to fig. 1, the system 100 includes one or

more client devices

101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120.

Client devices

101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the method for obtaining speech training data according to the present disclosure.

In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of

client devices

101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.

In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof that are executable by one or more processors. A user

operating client devices

101, 102, 103, 104, 105, and/or 106 may in turn utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be appreciated that a variety of different system configurations are possible, which may differ from system 100. Accordingly, FIG. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

The user may use

client devices

101, 102, 103, 104, 105, and/or 106 to collect audio data, play audio data, obtain voice training data, train models with voice training data, and so forth. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that the present disclosure may support any number of client devices.

Client devices

101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS); or include various mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, windows Phone, android. Portable handheld devices may include cellular telephones, smart phones, tablet computers, personal Digital Assistants (PDAs), and the like. Wearable devices may include head mounted displays (such as smart glasses) and other devices. The gaming system may include various handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), short Message Service (SMS) applications, and may use a variety of communication protocols.

Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a number of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. For example only, the one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a blockchain network, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.

The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture that involves virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain virtual storage devices of the server). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

The computing units in server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. Server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of

client devices

101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of

client devices

101, 102, 103, 104, 105, and 106.

In some implementations, the server 120 may be a server of a distributed system or a server that incorporates a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and virtual private server (VPS, virtual Private Server) service.

The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of databases 130 may be used to store information such as audio files and video files. Database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. Database 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve the databases and data from the databases in response to the commands.

In some embodiments, one or more of databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key value stores, object stores, or conventional stores supported by the file system.

The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.

A method 200 for obtaining speech training data according to an exemplary embodiment of the present disclosure is described below with reference to fig. 2.

At step S201, an audio stream containing at least one voice instruction is obtained by performing an audio acquisition operation on the sequentially played at least one voice instruction.

At step S202, at least one instruction data is identified from the audio stream, each of the at least one instruction data comprising an instruction text, an audio start point, and an audio end point of a respective one of the at least one voice instruction.

At step S203, the audio stream and the at least one instruction data are stored as speech training data.

According to the method disclosed by the embodiment of the invention, the voice training data with automatic labeling can be obtained. Specifically, by automatically collecting, identifying, and summarizing the audio streams, the audio streams and the identified text are used as speech training data. It is to be appreciated that the speech training data thus obtained may include audio and corresponding annotation text, start time, end time, may be used in speech recognition models, wake models, etc. models as would be understood by one skilled in the art for training, testing, etc., and the present disclosure is not limited thereto.

According to some embodiments, wherein the audio acquisition operation is performed by audio acquisition means of a first type of device, and wherein the speech training data is speech training data for the first type of device.

The speech training data thus obtained may be hardware device specific, i.e. the training data obtained is related to the audio acquisition capabilities of the device. For example, different hardware devices may have different collection capabilities, and may have the phenomenon of dropping consonants, vowels, prefixes, air-supplied tones, and the like.

According to some embodiments, further comprising, after obtaining an audio stream containing the at least one speech instruction, performing a noise reduction process on the audio stream, and wherein identifying at least one instruction data from the audio stream comprises identifying at least one instruction data from the noise-reduced audio stream.

For example, the noise reduction process is based on a noise reduction processing algorithm specific to the first type of device, and the speech training data thus obtained may further correspond to the first type of device. For example, noise reduction algorithms used by vehicle equipment, home speakers, etc. are not the same.

According to some embodiments, the first type of device has at least one typical acoustic environment, and wherein audio acquisition of the sequentially played at least one voice command comprises: an audio acquisition operation is performed in an acoustic environment of the at least one typical acoustic environment.

The method has the advantages that the method is carried out under a typical acoustic environment, so that the audio which can be acquired under the actual condition of the audio equipment can be better simulated; the obtained voice training data can train the engine to obtain better effect

In other words, the obtained audio and time annotation data (e.g., and without limitation, wav files and textgrid files as described below) may be multiplexed and trained based on the same scene and/or type. The same scene or type may include the same hardware type, such as a radio type, a noise reduction feature type, and/or the same acoustic space, such as speed of a car, volume, etc.

According to some embodiments, wherein the first type of device is an in-vehicle device and the at least one typical acoustic environment includes engine noise at least one vehicle speed.

The audio acquisition is carried out under the engine noise of the vehicle-mounted equipment, so that the audio which can be acquired under the actual condition of the audio equipment can be better simulated; the voice training data obtained in this way trains the engine, and can obtain better effects.

According to some embodiments, wherein the first type of device is a music playing device and the at least one typical acoustic environment comprises playing sound at least one volume level of the music playing device.

Music playback apparatuses such as televisions, speakers, and the like often can play out music, programs, and the like, and also have a function of collecting sound. The audio device is collected in the acoustic environment, so that the audio which can be collected under the actual condition of the audio device can be better simulated; the voice training data obtained in this way trains the engine, and can obtain better effects.

According to some embodiments, wherein the at least one voice command is played through a manual mouth.

A human mouth (simulator mouth) can simulate human voice to play so as to better simulate the effect of audio collection.

According to some embodiments, the at least one audio piece is a cut-out processed audio piece, the cut-out processing being used to remove blank portions of the audio piece that do not contain human voice.

For example, the at least one voice command is played by: obtaining at least one audio clip, each of the at least one audio clip comprising a speech instruction; and sequentially playing the at least one audio clip by inserting a gap between adjacent ones of the at least one audio clip.

Therefore, the audio frequency fragments with uniform interval, starting time and ending time and convenient processing can be obtained.

According to some embodiments, wherein the played at least one voice instruction is at least one marked audio clip, and wherein the method further comprises: the identified at least one instruction data is compared to the corresponding tag.

The audio recognition capability of the device may also be verified by comparing the recognized instruction data with the corresponding indicia; if the identification is wrong, the audio identification model of the equipment can be adjusted

According to some embodiments, each of the at least one voice instruction is a wake word or a query instruction.

Wake words and query instructions (queries) may be identified. Specifically, a generated voice training data file may contain the same instruction or may be a set of different instructions. For example, one voice training data file may include multiple "small" wake words (e.g., from different speakers, using the same or different dialects), or other wake words, one voice training data file may include multiple query instructions of the same type (e.g., "play music") or different types (e.g., "play music," "turn on air conditioner," "please search … …," etc.), or one voice training data file may include both query instructions and wake words, and the disclosure is not limited thereto.

In the delivery service of a vehicle enterprise or other intelligent devices, a large amount of audio needs to be recorded, for example, recognition audio and wake-up audio, the length of one audio may be several hours, and thousands of command lists (Command Lists) are included, so that in order to ensure the recognition and wake-up accuracy of the Command Lists, the audio needs to be cut and marked, but because the related words are relatively many and the corpus is relatively long, a long time is required for manpower and material resources, and errors may also occur.

Therefore, a technical means is needed to automatically cut and label the audio. The input of manpower is reduced, and the work efficiency is improved.

Automatic labeling of wake-up speech. For example, in the field of smart devices, it is often necessary to process a large amount of recorded audio, and for one instruction (e.g., a wake-up word or a specific instruction query), several, hundreds or even tens of thousands of pieces of audio may need to be recorded. These audio signals then need to be manually annotated when to begin speaking a sentence, when to complete it, what is said, etc., which is time consuming and potentially subject to error.

The user of the disclosed aspects may include an AI practitioner, or the like. The input of the method can be several, tens, thousands of recorded audio (such as wav audio), such as audio read by multiple persons for the same instruction "small scale", or audio for different instructions. The output may be an audio stream and corresponding annotations, such as an audio wav file and a textgrid file as described below, and so forth. It will be appreciated that the annotation requires two things to do, the first annotating the content of the audio, e.g. what the current audio says, and converting to text. And secondly, the audio starting and ending points are required to be marked, so that the statistics and recognition of the wake-up rate, the recognition accuracy rate and the like are facilitated.

As an example of one specific training sample, as shown in FIG. 3A, the required training data may include two files, namely, an audio file "small-scale Guangdong. Wav" and a markup file "small-scale Guangdong. Textgrid". It is to be understood that the above file names, language types, file formats are merely examples, and the present disclosure is not limited thereto.

Continuing with the example above, the Textgrid file may describe what time to start speaking what time to end, and the wav file is a specific audio training file.

The Textgrid file content may include a total time length, a start time of each segment, an end time, text content, and the like. As an example, there may be a first start time x1min=0, a first end time x1max=0.97, the corresponding text 1= "small degree"; a second start time x2min=0.97 and a second end time x2max=2.25, the corresponding text2 being blank (or indicated as "") to indicate that the piece of audio is the interval between two voice indications; a third start time x3min=2.25, a third end time x3max=3.02, the corresponding text 3= "small degree" … …, and so on. It is to be understood that the above text, length of time, number of tones, representation of intervals, etc. are merely examples and the present disclosure is not limited thereto.

Fig. 3B shows an example of audio file content. It can be seen that one audio file content may include a plurality of voice instructions, in this example the wake word "small" and it is understood that this is only an example and the present disclosure is not limited thereto.

A data flow diagram 400 according to one specific, non-limiting embodiment of the present disclosure is described below in connection with fig. 4.

Referring to fig. 4, a recorded audio file 410 may be obtained first, according to an embodiment of the present disclosure. The audio file 410 may be audio cut, for example, using a ffmpeg based vad algorithm, or in other ways known to those skilled in the art. The section is a file after the audio cutting is finished. In some examples, because the content of the audio is known, such as what the entire piece of audio reads in what order, the file may be named based on this principle, e.g., "small scale. Wav". Thus, a processed audio file, here for the sake of example, denoted as an audio cut text annotation file 420, is obtained.

The corresponding audio file 420 may be sent to the audio playback system 430 or may otherwise enable the audio playback system to read and play the audio file.

Based on the cut file, a section of script for playing audio can be written. When training audio needs to be collected for the vehicle-mounted equipment, the mobile phone or some other intelligent equipment, the corresponding audio can be played by utilizing a play script. The corresponding hardware device may be preloaded with a voice assistant 440. The voice may be recorded based on a hardware-based microphone and fed to a voice assistant for wake-up identification. In addition, the audio file may be saved, for example, simultaneously, for subsequent training use, such as by audio recording system 450. It will be appreciated that in some examples, the textgrid file may be written dependent on the recognition point in time of the voice assistant. The wav file may be recorded by a voice assistant, i.e. a file obtained after noise reduction of the device.

According to some embodiments, the audio start point and audio end point are obtained by: starting timing when the audio is played, and recording the current point as a voice starting point; and recording the voice tail point through log grabbing. Illustratively, timing may begin at the time of audio playback, recording the current point as the speech start point, and through a log (log) grabbing system 460, when captured by a locator, the speech end point. Illustratively, the dotting file may be written by the audio time dotting system 470. According to some embodiments, the method may further comprise capturing the recognition start time and the recognition end time as the audio start and audio end points by capturing a script. As a non-limiting specific example, the device where the voice assistant is located may be connected to the computer through a protocol (USB, ADB, etc.), by capturing a script, capturing the point in the log (log) where recognition begins and ends, writing the log file. Alternatively, the smart device may write, for example, store under the corresponding directory, and export after the playback is completed.

Illustratively, the corresponding wake-up recognition results may also be counted by the result counting system 480. The statistics may be used to further train the engine, and so on. For example, if the point in time/content of the speech handwriting aid is inconsistent with the original file, it may be used to further train the recognition capabilities of the engine.

According to one or more embodiments of the present disclosure, the voice file may be cut by ffmpeg and text-tagged according to the order of speaking; writing a corresponding audio playing system, and playing the cut audio when the training corpus is required to be acquired; and when the audio is played, the recording of the corresponding equipment is started at the same time, and the audio recording system stores the audio acquired by the current equipment. And (3) starting and ending the playing and simultaneously recording the time points of the audio, recording by the audio time dotting system, and recording when the log grabbing system grabs the result. The person cutting and marking the audio may have errors; the large amount of corpus requires a large amount of manpower. According to one or more embodiments of the present disclosure, audio recording and related labeling is performed automatically, without human intervention, improving efficiency, and saving costs.

Because different devices have different noise reduction effects, ambient sounds, etc., according to one or more embodiments of the present disclosure, the noted audio may be specific to a particular device, played out through a manual mouth, collected by the device, noise reduced, etc. For example, with an in-vehicle device, there may be noise, music, car launch noise, etc. when the sentence is played out on the car. For intelligent sound boxes, televisions and the like, music and sound of programs played by the intelligent sound boxes and the televisions also exist. The sound reception and noise reduction algorithms are different for different devices. The device will process the audio by noise reduction into noise-free audio of music based on the audio with the noise-mixed music. The audio frequency received by the voice assistant and processed by the device is different from the audio frequency played by the voice assistant, because some tail sounds can be restrained after noise reduction, the whole audio frequency length can be shortened after noise reduction, and the voice assistant needs to re-label after the noise reduction processing after playing. This is the process of noise reduction that results in a change in the length of the audio. In addition, after the device's microphone collects data and while delivering audio to the recognition engine, frames are lost because the CPU processing the data rate slows. The length of the audio is also changed. Therefore, the training files collected in this way can better simulate the audio files received by the engine in actual use, so that the engine can be better trained.

As described above, the obtained training files may be used to further train the engine. The trained engine may include speech recognition, wake-up, etc. engines that may be used for wake-up and recognition operations, without including noise reduction, etc. functionality, and thus is suitable for processed and denoised samples recorded by a voice assistant.

In the art, to enhance the wake-up recognition rate, after audio is collected, the audio is often subjected to noise reduction treatment and then transmitted to an engine for engine recognition for training. However, because the noise reduction algorithm and the CPU computation power of each hardware are different, it is more beneficial to annotate the corpus processed for each device.

An apparatus 500 for obtaining speech training data according to an embodiment of the present disclosure is now described with reference to fig. 5. The apparatus 500 for obtaining speech training data may include an audio stream obtaining unit 501, an instruction data identifying unit 502, and a training data storing unit 503.

The audio stream obtaining unit 501 may be configured to obtain an audio stream containing at least one voice instruction by performing an audio acquisition operation on the at least one voice instruction played in sequence. The instruction data identifying unit 502 may be configured to identify at least one instruction data from the audio stream, each of the at least one instruction data comprising an instruction text, an audio start point and an audio end point of a respective one of the at least one voice instruction. The training data storage unit 503 may be configured to store the audio stream and the at least one instruction data as voice training data.

According to the device disclosed by the embodiment of the invention, the voice training data with automatic labeling can be obtained.

According to some embodiments, the audio acquisition operation is performed by an audio acquisition means of a first type of device, and wherein the speech training data is speech training data for the first type of device.

According to some embodiments, the method further comprises performing a noise reduction process on the audio stream after obtaining the audio stream containing the at least one speech instruction, and wherein identifying at least one instruction data from the audio stream comprises identifying at least one instruction data from the noise-reduced audio stream.

According to some embodiments, the at least one voice command is played through a manual mouth.

In the technical scheme of the disclosure, the related processes of collecting, acquiring, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order is not violated.

According to embodiments of the present disclosure, there is also provided an electronic device, a readable storage medium and a computer program product.

Referring to fig. 6, a block diagram of an electronic device 600 that may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the electronic device 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, ROM 602, and RAM603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

A number of components in the electronic device 600 are connected to the I/O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 may be any type of device capable of inputting information to the electronic device 600, the input unit 606 may receive input numeric or character information and generate key signal inputs related to user settings and/or function control of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and/or a remote control. The output unit 607 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 608 may include, but is not limited to, magnetic disks, optical disks. The communication unit 609 allows the electronic device 600 to exchange information/data with other devices through a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth devices, 802.11 devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.

The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the method 200 and variations thereof, and the like. For example, in some embodiments, the method 200, variations thereof, and the like may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of method 200 and variants thereof, etc., described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method 200 and variants thereof, etc., in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the foregoing methods, systems, and apparatus are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples but only by the claims following the grant and their equivalents. Various elements of the embodiments or examples may be omitted or replaced with equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in the present disclosure. Further, various elements of the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the disclosure.

Claims

1. A method for obtaining speech training data, comprising:

The method comprises the steps of obtaining an audio stream containing at least one voice instruction by carrying out audio acquisition operation on the at least one voice instruction which is played in sequence;

identifying at least one instruction data from the audio stream, each instruction data of the at least one instruction data comprising an instruction text, an audio start point, and an audio end point of a respective voice instruction of the at least one voice instruction; and

the audio stream and the at least one instruction data are stored as speech training data.

2. The method of claim 1, wherein the audio acquisition operation is performed by an audio acquisition device of a first type of device, and wherein the speech training data is speech training data for the first type of device.

3. The method of claim 2, further comprising, after obtaining an audio stream containing the at least one voice instruction, performing a noise reduction process on the audio stream, and wherein identifying at least one instruction data from the audio stream comprises identifying at least one instruction data from the noise-reduced audio stream.

4. A method according to claim 2 or 3, wherein the first type of device has at least one typical acoustic environment, and wherein audio acquisition of the sequentially played at least one voice command comprises: an audio acquisition operation is performed in an acoustic environment of the at least one typical acoustic environment.

5. The method of claim 4, wherein the first type of device is an in-vehicle device and the at least one typical acoustic environment includes engine noise at least one vehicle speed.

6. The method of claim 4, wherein the first type of device is a music playing device and the at least one typical acoustic environment includes a playing sound at least one volume level of the music playing device.

7. The method of any of claims 1-6, wherein the at least one voice command is played through a manual mouth.

8. The method of any of claims 1-7, the at least one audio clip being a cut-out audio clip, the cut-out being to remove blank portions of the audio clip that do not contain human voice.

9. The method of any of claims 1-8, wherein the played at least one voice instruction is at least one tagged audio clip, and wherein the method further comprises: the identified at least one instruction data is compared to the corresponding tag.

10. The method of any of claims 1-9, wherein each of the at least one voice instruction is a wake word or a query instruction.

11. The method of any of claims 1-10, wherein the audio start point and audio end point are obtained by:

starting timing when the audio is played, and recording the current point as a voice starting point; and

and recording the voice tail point through log grabbing.

12. The method of claim 11, further comprising capturing an identification start time and an identification end time as the audio start and audio end points by capturing scripts.

13. An apparatus for obtaining speech training data, comprising:

an audio stream obtaining unit, configured to obtain an audio stream containing at least one voice instruction by performing an audio acquisition operation on at least one voice instruction that is played in sequence;

an instruction data identifying unit for identifying at least one instruction data from the audio stream, each of the at least one instruction data including an instruction text, an audio start point, and an audio end point of a corresponding one of the at least one voice instruction; and

and the training data storage unit is used for storing the audio stream and the at least one instruction data as voice training data.

14. The apparatus of claim 13, wherein the audio acquisition operation is performed by an audio acquisition apparatus of a first type of device, and wherein the speech training data is speech training data for the first type of device.

15. The apparatus of claim 14, further comprising means for noise reduction processing an audio stream containing the at least one voice instruction after the audio stream is obtained, and wherein identifying at least one instruction data from the audio stream comprises identifying at least one instruction data from the noise-reduced audio stream.

16. The apparatus of claim 14 or 15, wherein the first type of device has at least one typical acoustic environment, and wherein audio acquisition of the sequentially played at least one voice command comprises: an audio acquisition operation is performed in an acoustic environment of the at least one typical acoustic environment.

17. The apparatus of claim 16, wherein the first type of device is an in-vehicle device and the at least one typical acoustic environment comprises engine noise at least one vehicle speed.

18. The apparatus of claim 16, wherein the first type of device is a music playing device and the at least one typical acoustic environment comprises a playing sound at least one volume level of the music playing device.

19. The apparatus of any of claims 13-18, wherein the at least one voice instruction is played through a manual mouth.

20. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the method comprises the steps of

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

21. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-12.

22. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-12.