CN111341297B - Voice wake-up rate test system and method - Google Patents
Voice wake-up rate test system and method Download PDFInfo
- Publication number
- CN111341297B CN111341297B CN202010144278.1A CN202010144278A CN111341297B CN 111341297 B CN111341297 B CN 111341297B CN 202010144278 A CN202010144278 A CN 202010144278A CN 111341297 B CN111341297 B CN 111341297B
- Authority
- CN
- China
- Prior art keywords
- tested
- electronic device
- test
- awakened
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 160
- 238000000034 method Methods 0.000 title claims description 22
- 238000010998 test method Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 abstract description 2
- 238000012827 research and development Methods 0.000 abstract description 2
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000007790 scraping Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/01—Assessment or evaluation of speech recognition systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
Abstract
The invention provides a test system and a test method for testing the awakening rate of a voice model, which support the test of a plurality of noise scenes at multiple distances, and comprise a device to be tested, a plurality of control devices and a plurality of audio players, wherein the plurality of control devices comprise a main control electronic device which is used for connecting the device to be tested and running an automation program (python), the audio players are controlled to be started in sequence to play test corpora at each distance, meanwhile, the audio players can be controlled to play specific noise, and the device to be tested is awakened through the test corpora. By using the scheme of the invention, the testing efficiency of the voice awakening model can be improved, the manual workload is reduced, the test data is convenient to manage, the test data is more efficiently provided for research and development, and the model optimization is promoted.
Description
Technical Field
The invention relates to the technical field of voice recognition, in particular to a voice awakening rate testing system and a voice awakening rate testing method.
Background
With the progress of data processing technology and the rapid spread of mobile internet, computer technology is widely applied to various fields of society, and then mass data is generated. Among them, voice data is receiving more and more attention. Speech recognition is a cross discipline. The fields to which speech recognition technology relates include: signal processing, pattern recognition, probability and information theory, sound and hearing mechanisms, artificial intelligence, and the like.
In 2006, hinton proposed a deep learning network, which indicated that the problem of deep neural networks with many training parameters due to too many layers could be solved by using layer-by-layer initialization. The wave of deep learning has been raised in the industry and academia and has enjoyed great success in the fields of speech recognition and image processing.
The speech recognition model based on deep learning needs to continuously improve the speech recognition rate of the model by changing the model structure, training data and the like, so that a quick iteration version is needed for test verification, and great workload is brought to testers.
The performance of the voice recognition model is generally measured by the awakening rate of the device to be tested under different scenes and different distances, the testing method is to use a high-fidelity sound box to play the corpus to be tested, the device to be tested is placed under different distances from the sound box, and meanwhile, the noise of different scenes is played by another sound box, so that the aim of verifying the awakening rate under different scenes is fulfilled.
Disclosure of Invention
Therefore, the invention aims to provide an automatic multi-distance and multi-noise scene voice recognition model testing method for the same hardware device to be tested.
The invention relates to a test method for a voice model wake-up rate, which supports continuous multi-distance test on the same device to be tested. The system and the method for testing the voice wake-up rate support a plurality of noise scenes at multiple distances. The purpose of the invention is realized by the following technical scheme.
A voice wake-up rate test system, comprising:
each control device is correspondingly connected with an audio player and is respectively controlled to start the correspondingly connected audio player to play a preset test corpus, and the test corpus is used for performing wake-up test on a device to be tested running with a voice recognition model;
the control devices comprise a first electronic device, and the first electronic device is connected with the device to be tested and used for calculating the total awakening times of the device to be tested;
the relative position relation between the audio player and the equipment to be tested is different.
The system is characterized in that the plurality of control devices comprise at least two second electronic devices, and the first electronic device is respectively connected with each second electronic device;
the first electronic device includes:
the first control module is used for respectively sending a starting instruction to the first electronic equipment and each second electronic equipment so as to start each audio player correspondingly connected to the first electronic equipment to play the test corpus according to a certain sequence;
the log acquisition module is used for capturing and storing logs of the equipment to be tested;
and the evaluation module is respectively connected with the log acquisition module and the first control module and used for reading and processing the logs so as to obtain the total awakening times of the equipment to be tested.
The system is characterized in that the first electronic device and each second electronic device respectively comprise a second control module, the second control module is correspondingly connected with the audio player and is used for receiving a starting instruction sent by the first control module to start the correspondingly connected audio player to play a test corpus;
the log acquisition module respectively captures and stores logs of the equipment to be tested before starting testing, before playing the testing corpora each time, after playing the testing corpora each time and after finishing testing;
the evaluation module specifically comprises:
the frequency acquisition unit is connected with the log acquisition module and used for acquiring the initial awakening frequency of the device to be tested before the test is started, the first awakening frequency of the device to be tested before the test corpus is played each time, the second awakening frequency of the device to be tested after the test corpus is played each time and the final awakening frequency of the device to be tested after the test is finished according to the read log;
the storage unit is connected with the frequency acquisition unit and used for storing the awakened frequency acquired by the frequency acquisition unit;
the comparison and judgment unit is connected with the storage unit and used for comparing whether the corresponding first awakened times and the corresponding second awakened times are equal or not according to each played test corpus and outputting a comparison result;
the recording unit is connected with the comparison and judgment unit and used for receiving the comparison result and recording the corresponding test corpus into the failure item list when the comparison result shows that the first awakened times and the second awakened times are equal;
the loop judgment unit is respectively connected with the first control module and the operation module and is used for recording the frequency of sending the starting instruction by the first control module and judging whether all the tested corpora are played completely, if not, the incomplete information is fed back to the first control module so that the first control module continues to send the starting instruction; if the completion is finished, sending the completion information to an operation module;
and the computing unit is connected with the storage unit and used for processing the received information according to the final awakened times and the initial awakened times to obtain the total awakened times of the equipment to be tested.
The control system is characterized in that the control devices further comprise at least one third electronic device, and the first electronic device is respectively connected with each third electronic device;
each third electronic device comprises a third control module used for starting the audio player connected correspondingly and playing specific noise;
the second control module in the first electronic device is further configured to: when a noise scene needs to be simulated, the second electronic device is controlled to start the audio player correspondingly connected to the second electronic device to play the test corpus, and meanwhile, each third control module is controlled to start the audio player correspondingly connected to the third electronic device and play specific noise.
A voice wake-up rate testing method, characterized in that, using the above voice wake-up rate testing system, includes the following steps:
s1, running a voice recognition model by equipment to be tested;
s2, respectively controlling each control device to start the correspondingly connected audio player to play the preset test corpus;
and S3, calculating the total awakening times of the equipment to be tested by the first electronic equipment.
The method is characterized in that the step S2 specifically comprises the following steps:
step S21, setting at least two of the plurality of control devices as second electronic devices;
step S22, a first control module of the first electronic device sends a starting instruction to the first electronic device and each second electronic device respectively so as to start each audio player correspondingly connected to the first electronic device to play the test corpus according to a certain sequence;
the step S3 specifically includes: the first electronic device captures and stores the log of the device to be tested, reads and processes the log, and obtains the total awakening times of the device to be tested.
The method is characterized in that a step S101 is further included between the step S1 and the step S2: before testing, capturing and storing a log of the equipment to be tested, reading and processing the log, and acquiring and storing the initial awakening times of the equipment to be tested;
step S22 specifically includes the following steps:
step S221, capturing and storing the log of the equipment to be tested, reading the log to obtain and store the first awakened times of the equipment to be tested before playing the test corpus;
step S222, the first control module controls the first electronic device and each second electronic device to start the audio player correspondingly connected to the first electronic device to play a test corpus according to a certain sequence;
step S223, capturing and storing the log of the device to be tested, reading the log to obtain and store the second awakened times of the device to be tested after the current test corpus is played;
step S224, comparing whether the corresponding first awakened times and the second awakened times are equal to each other for each played test corpus, and outputting a comparison result;
step S225, when the comparison result shows that the first awakened times are equal to the second awakened times, recording the corresponding test corpus into a failure item list;
step S226, judging whether all the test corpora are played completely according to the number of times of sending the starting instruction by the first control module, and if not, continuing to step S221; if yes, continuing to step S3;
the step S3 specifically includes the following steps:
step S31, capturing and storing the log of the device to be tested in a fixed path, and reading the log to obtain the last awakening times of the device to be tested after the test;
and step S32, processing according to the final awakened times and the initial awakened times to obtain the total awakened times of the device to be tested.
Characterized in that step S21 further comprises: setting at least one of the plurality of control devices as a third electronic device;
step S22 further includes: when a noise scene needs to be simulated, the first control module controls each third electronic device to start the audio player correspondingly connected to play the specific noise while controlling the audio player correspondingly connected to play the test corpus.
The beneficial technical effects of the invention are as follows: the method supports automatic testing of the voice awakening rate of multiple distance and multiple noise scenes, improves the testing efficiency of the voice awakening model, reduces the manual workload, and facilitates management of test data. Test data are provided for research and development more efficiently, and model optimization is promoted.
Drawings
FIG. 1 is a schematic diagram of a scenario of an embodiment of the system of the present invention;
FIGS. 2-6 are schematic structural diagrams of embodiments of the system of the present invention;
fig. 6-9 are schematic diagrams of the steps of an embodiment of the method of the present invention.
The system comprises 1-first electronic equipment, 2b, 2 c-second electronic equipment, 3-equipment to be tested, 4a, 4b, 4c,4 d-audio players, 5-third electronic equipment, 6-control equipment, 10-an evaluation module, 11-a second control module, 12-a first control module, 13-a log acquisition module, 14-a frequency acquisition unit, 15-a storage unit, 16-a comparison and judgment unit, 17-a recording unit, 18-a circulation judgment unit, 19-a calculation unit and 51-a third control module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
Referring to fig. 1-5, the present invention provides a voice wake-up rate testing system capable of supporting voice wake-up rates of multiple distance and multiple noise scenes, which has a plurality of control devices 6, wherein each control device 6 is correspondingly connected to an audio player 4. Preferably, the music player 4 may be a hi-fi stereo,
each control device 6 is controlled to start the correspondingly connected audio player 4 to play the test corpus.
The device under test 3 is used for running the voice recognition model, and the device under test 3 is awakened through the test corpus played by the audio player 4. The DUT shown in fig. 1 represents a device under test, on which a model is running, called a device under test 3, and the music player 4 plays voice information, such as keywords, to wake up the device under test 3.
One of the control devices 6 serves as a first electronic device 1, and the first electronic device 1 is connected with the device to be tested 3 and used for calculating the awakening times of the device to be tested 3.
The relative position relationship between the audio player 4 and the device under test 3 is different.
As an embodiment of the present invention, fig. 1 shows four device control devices 6 of the Linux system, where each of the control devices 6 is correspondingly connected to an audio player 4, and is configured to start the correspondingly connected audio player 4 to play voice information. And the relative position relations between the four audio players 4 and the equipment to be tested 3 are different, wherein the three music players 4 and the equipment to be tested 3 are on the same straight line, and the distances between the three music players and the equipment to be tested 3 are respectively 1 meter, 2 meters and 3 meters. While the other music player 4 is not in the line and is at a distance of, for example, 2 meters from the device under test 3. As a preferred embodiment, the control device 6 connected to the music player at a distance of 1 meter from the device under test is used as the first electronic device 1, and is connected to the device under test 3 in a serial port manner, so as to calculate the number of times the device under test 3 is awakened.
Further, the plurality of control devices 6 includes at least one second electronic device 2; the first electronic device 1 is respectively connected with each second electronic device 2, and is used for sequentially controlling the first electronic device 1 and the second electronic devices 2 to start the audio players 4 correspondingly connected to play a test corpus according to a certain sequence; the device to be tested 3 is awakened through the test corpus played by the audio player 4; the first electronic device 1 is configured to capture and store a log of the device to be tested 3, for example, store the log in a fixed path, and obtain the number of awakened times of the device to be tested 3 by reading the log in the fixed path, so as to calculate the total number of awakened times of the device to be tested 3. The distances between the audio player 4 correspondingly connected to the first electronic device 1, the audio player 4 correspondingly connected to each second electronic device 2, and the device under test 3 are different.
Specifically, the first electronic device 1 includes a first control module 12, configured to send a start instruction to the first electronic device 1 and each second electronic device 2, respectively, so as to start each correspondingly connected audio player to play 4 the test corpus according to a certain sequence. The first electronic device 1 includes a log obtaining module 13, configured to capture and store a log of the device under test, for example, under a fixed path. The first electronic device 1 includes an evaluation module 10, which is respectively connected to the log obtaining module 13 and the first control module 12, and configured to read and process the log, so as to obtain the total number of awakening times of the device under test 3.
Fig. 3 to 5 show a first electronic device 1 connected to a device under test 3 and a second electronic device 2, respectively, and fig. 3 shows a case with two second electronic devices 2, as an embodiment of the present invention. With reference to fig. 1-4, three music players 4a-4c are in the same line with the device under test 3 and are at distances of 1 meter, 2 meters and 3 meters from the device under test 3, respectively. The control device 6 connected to the music player 4a is taken as the first electronic device 1, and the control devices 6 connected to the music players 4b, 4c are taken as the second electronic devices 2, 2b and 2c, respectively.
The first electronic device 1 is connected with the second electronic devices 2b and 2c in an ssh manner, and the first electronic device 1 sequentially controls the first electronic device 1, the second electronic devices 2b and 2c to start the audio players 2a, 2b and 2c connected correspondingly to play a test corpus according to a certain sequence. The device under test 3 runs a speech recognition model, and is awakened by the test corpus played by the audio players 2a, 2b, and 2c, and specifically, by recognizing the keywords/words in the test corpus. The first electronic device 1 captures a log of the device to be tested 3 and stores the log to the fixed path, and obtains the awakening times of the device to be tested 3 by reading the log in the fixed path, so that the total awakening times of the device to be tested 3 is calculated. Specifically, the playing sequence of the audio player may be determined according to the distance to the device to be tested, for example, the music player 4a is started to play a test corpus first, after the playing is finished, the music player 4b is started to play a test corpus, and finally the music player 4c is started to play a test corpus.
Further, referring to fig. 5, the first electronic device 1 and the second electronic device 2 respectively have a second control module 11, and the second control module 11 is correspondingly connected to the audio player 4 and configured to receive a start instruction sent by the first control module 12 to start the audio player 4 correspondingly connected to play a test corpus.
The log obtaining module 13 is configured to capture and store a log of the device to be tested 3 before starting the test, before playing the test corpus each time, after playing the test corpus each time, and after finishing the test, for example, store the log in a fixed path;
the evaluation module 10 specifically includes: the frequency obtaining unit 14 is connected to the log obtaining module 13, and configured to obtain, according to the read log, an initial awakening frequency of the device to be tested before starting the test, a first awakening frequency of the device to be tested before playing the test corpus each time, a second awakening frequency of the device to be tested after playing the test corpus each time, and a final awakening frequency of the device to be tested after finishing the test. And a storage unit 15 connected to the number of times acquisition unit 14 and configured to store the number of times of awakening acquired by the number of times acquisition unit 14. And the comparison and judgment unit 16 is connected to the storage unit 15, and is configured to compare, for each played test corpus, whether the corresponding first awakened time and the corresponding second awakened time are equal, and output a comparison result. And the recording unit 17 is connected to the comparison and judgment unit 16, and is configured to receive the comparison result, and record the corresponding test corpus into the failure item list when the comparison result indicates that the first awakened number is equal to the second awakened number. The circulation judging unit 18 is respectively connected to the first control module 12 and the operation module 19, and is configured to record the number of times that the first control module 12 sends the start instruction and judge whether all the test corpora have been played completely, and if not, feed back incomplete information to the first control module 12, so that the first control module 12 continues to send the start instruction; if the completion is finished, the completion information is sent to the operation module 19;
and the calculating unit 19 is connected to the storage unit 15, and is configured to process the final awakened times and the initial awakened times to obtain the total awakened times of the device to be tested after receiving the completion information.
Setting at least one of the plurality of control devices 6 as the third electronic device 5; when a noise scene needs to be simulated, the first control module 12 controls each third electronic device 5 to start the corresponding connected audio player to play a specific noise while controlling the corresponding connected audio player to play the test corpus.
Specifically, the plurality of control devices 6 further include at least one third electronic device 5, and each third electronic device 5 includes a third control module 51, configured to start the correspondingly connected audio player 4 to play a specific noise; the first electronic device 1 is connected to each of the third electronic devices 5, and the first control module 12 is configured to: when a noise scene needs to be simulated, the second control module 11 is controlled to start the correspondingly connected audio player 4 to play the test corpus, and at the same time, each third control module 51 is controlled to start the correspondingly connected audio player 4 to play the specific noise.
Referring to fig. 1, 5 and 6, as an embodiment of the present invention, the music players 4a to 4c are in the same line with the device under test 3 and are at distances of 1 meter, 2 meters and 3 meters from the device under test 3, respectively. The control device 6 connected to the music player 4a is taken as the first electronic device 1, and the control devices 6 connected to the music players 4b, 4c are taken as the second electronic devices 2, 2b and 2c, respectively. The music player 4d which is not on the same straight line with the music players 4a-4c and the device to be tested 3 is used for playing specific noise, the control device 6 connected with the music player 4d is used as a third electronic device 5, the third electronic device 5 is connected with the first electronic device 1 in a ssh mode, the third electronic device 5 is provided with a third control module 51, and when the music players 4a-4c play a test corpus, the first control module 12 of the first electronic device 1 controls the third control module 51 to start the music player 4d which is correspondingly connected to play the specific noise, so that the voice wake-up rate of the device to be tested in a noise scene is tested. Specifically, a straight line formed by connecting the music player 4d and the device to be tested 3 forms a 60-degree included angle with a straight line formed by connecting the music player 4a-4c and the device to be tested 3, the distance between the music player 4d and the device to be tested 3 is 2 meters, and the music players 4a-4d are all on the same side of the device to be tested 3.
In a preferred embodiment of the present invention, the number of the second electronic devices 2 may be 3 or more, specifically 4 or 5.
Further, the music player 4 correspondingly connected with the second electronic device 2 and the music player 4 connected with the first electronic device 1 are arranged at equal intervals and in a collinear manner with the device under test 3.
And the number of the third electronic devices 5 may be more than 2. Further, the specific noise played by the music player 4 connected to each third electronic device 5 is different.
As the control devices 6, each control device 6 includes at least one processor that controls the operation process. In particular the first electronic device 1, the processor may coordinate the operations of the various modules and units, such as the coordination evaluation module 10, the log acquisition module 13, the first control module 12, etc. Specifically, for example, before the test is started, and before and after each test corpus is played, the log obtaining module 13 is controlled to start log scraping, and after the log obtaining module 13 scrapes the log, the log obtaining module 13 is controlled to start the working process by sending an instruction to the first control module 12.
Referring to fig. 6-9, the present invention further provides a voice wake-up rate testing method, which uses the above voice wake-up rate testing system, and includes the following steps.
Step S1, the equipment to be tested 3 runs a voice recognition model.
And S2, respectively controlling each control device 6 to start the correspondingly connected audio player to play the preset test corpus.
And S3, the first electronic device 1 calculates the total awakening times of the device to be tested 3.
As a specific embodiment of the present invention, the step S2 specifically includes the following steps:
at step S21, at least two of the plurality of control apparatuses 6 are set as the second electronic apparatus 2.
Preferably, distances between the audio player 4 correspondingly connected to the first electronic device 1, the audio player 4 correspondingly connected to each second electronic device 2, and the device under test 3 may be set to be different.
Step S22, the first control module 12 of the first electronic device 1 sends a start instruction to the first electronic device 1 and each second electronic device 2, so as to start each of the correspondingly connected audio players 4 to play the test corpus according to a certain sequence.
The step S3 specifically comprises the following steps: the first electronic device 1 captures and stores the log of the device to be tested 3, reads and processes the log, and obtains the total awakening times of the device to be tested 3. The first electronic device 1 captures a log of the device to be tested 3 and stores the log to the fixed path, and obtains the awakening times of the device to be tested 3 by reading the log in the fixed path.
As a specific embodiment of the present invention, a step S101 is further included between step S1 and step S2: before the test, the log obtaining module 13 of the first electronic device 1 captures and stores the log of the device to be tested 2, for example, the log is stored in a fixed path, and the frequency obtaining unit 14 of the first electronic device 1 reads and processes the log, obtains and stores the initial awakened frequency of the device to be tested.
Further, step S22 specifically includes the following steps:
step S221, capturing and storing the log of the device 3 to be tested, and reading the log to obtain and store the first awakened number of times of the device 3 to be tested before playing the test corpus. The log obtaining module 13 captures a log of the device 3 to be tested and stores the log in the fixed path, and the frequency obtaining unit 14 reads the log in the fixed path to obtain the first awakened frequency of the device 3 to be tested before playing the test corpus.
In step S222, the first control module 12 controls the first electronic device 1 and each second electronic device 2 to start the audio player 4 connected correspondingly to play a test corpus according to a certain sequence. The first control module 12 controls the first electronic device 1 and the second control module 11 on each second electronic device 2 in a certain order to start the audio player 4 connected correspondingly to play a test corpus.
Step S223, capturing and storing the log of the device to be tested 3, and reading the log to obtain and store the second awakened times of the device to be tested 3 after the playing of the current test corpus. The log obtaining module 13 captures a log of the device to be tested 3 and stores the log in the fixed path, and the frequency obtaining unit 14 reads the log in the fixed path to obtain the awakened frequency of the device to be tested 3 after the test corpus is played. The storage unit 15 stores the number of awakenings acquired by the number-of-times acquisition unit 14 each time.
Step S224, for each played test corpus, comparing whether the corresponding first awakened times and the corresponding second awakened times are equal, and outputting a comparison result. The comparison and determination unit 16 compares whether the corresponding first awakened times and the second awakened times are equal to each other for each played test corpus, and outputs a comparison result.
Step S225, when the comparison result indicates that the first awakened number is equal to the second awakened number, recording the corresponding test corpus into the failure item list. The recording unit 17 records the corresponding test corpus into the failure item list when the comparison result indicates that the first awakened number is equal to the second awakened number.
Step S226, determining whether all the tested corpora have been played completely according to the number of times the first control module 12 sends the start instruction, and if not, continuing to step S221; if so, continue to step S3. The loop judgment unit 18 judges whether all the corpus has been played according to the number of times of sending the start instruction by the first control module 12, if not, continues to step S221; if so, continue to step S3.
Further, step S3 specifically includes:
step S31, capturing and saving the log of the device under test 3, for example, saving the log in a fixed path, and reading the log to obtain the last wake-up time of the device under test 3 after the test. The log obtaining module 13 captures a log of the device to be tested 3 and stores the log in the fixed path, and the frequency obtaining unit 14 reads the log in the fixed path to obtain the final wake-up frequency of the device to be tested 3 after the test;
and step S32, processing according to the final awakened times and the initial awakened times to obtain the total awakened times of the device to be tested. The calculating unit 19 obtains the total number of awakened times of the device under test 3 according to the number of awakened times subtracted by the initial number of awakened times.
Further, step S21 specifically includes: at least one of the plurality of control devices 6 is set as the third electronic device 5.
Step S22 further includes: when a noise scene needs to be simulated, the first control module 12 controls to start the audio player 4 connected correspondingly to play the test corpus and simultaneously controls each third electronic device 5 to start the audio player 4 connected correspondingly to play the specific noise.
Specifically, each third electronic device 5 is provided with a third control module 51; step S222 specifically includes, when a noise scene needs to be simulated, the first control module 12 controls the second control module 11 to start the correspondingly connected audio player 4 to play the test corpus, and at the same time, controls each third control module 51 to start the correspondingly connected audio player 4 to play the specific noise.
Further, there are two second electronic devices 2.
Further, the first electronic device 1 is connected with the device to be tested 3 in a serial port mode; the first electronic device 1 is connected with the second electronic device 2 and the third electronic device 5 in a serial port mode respectively.
Further, the audio player 4 is a hi-fi stereo.
As a specific embodiment of the present invention, the music players 4a-4c are in the same line with the device under test 3, and are at distances of 1 meter, 2 meters, and 3 meters from the device under test 3, respectively. The control device 6 connected to the music player 4a is defined as a first electronic device 1 (pc 1), and the control devices 6 connected to the music players 4b and 4c are defined as second electronic devices 2b (pc 2) and 2c (pc 3), respectively. While the music player 4d, which is not on the same line with the music players 4a-4c and the device under test 3, is used to play a specific noise, and the control device 6 connected to the music player 4d is used as the third electronic device 5 (denoted as pc 4).
The method comprises the following steps that a pc1 is connected with a device to be tested and runs an automation program (python), wherein the pc1 is used for running an automation code, is connected with the device to be tested 3 through a serial port, grabs log of the device to be tested 3 and stores the log to the local, and is directly connected with a sound 1 and is used for playing test corpora with a distance of 1 meter;
the log capture method here is as follows:
the serial port grabs log to run in the sub-thread and simultaneously plays the audio.
t＝threading.Thread(target＝get_log,args＝(rate,))
t.setDaemon(True)
t.start()
And (2) running an automation code by the pc1, and controlling the pc to play the test corpus at a distance of 2 meters through ssh connection.
shell＝citizenshell.SecureShell(hostname＝pc_ip,username＝‵username′,password＝‵123456′)
shell(‵mplayer/home/Lenovo/play_list[i]′)
shell.disconnect()
pc3, operating an automatic code by the pc1, and controlling the pc to play the test corpus at a distance of 3 meters through ssh connection; the method is the same as pc2.
pc4, running an automatic code by the pc1, controlling the pc to play noise through ssh connection, and simulating various noise scenes;
the pc4 is controlled by the pc1, and a subprogram is independently established for the pc4 by the program and used for playing noise and simultaneously running with the log capturing subprocess and the corpus playing main process.
Shell＝citizenshell.SecureShell(hostname＝pc4_ip,username＝‵usrname′,password＝‵123456′)
t1＝threading.Thread(target＝util.play_noise,args＝(shell,))
t1.setDaemon(True)
t1.start()。
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (6)
1. A voice wake-up rate test system, comprising:
each control device is correspondingly connected with an audio player, each control device is respectively controlled to start the correspondingly connected audio player to play a preset test corpus, and the test corpus is used for performing wake-up test on a device to be tested running with a voice recognition model;
the control devices comprise a first electronic device, and the first electronic device is connected with the device to be tested and used for calculating the total awakening times of the device to be tested;
the relative position relations between the audio player and the equipment to be tested are different;
the control devices comprise at least two second electronic devices, and the first electronic device is connected with each second electronic device respectively;
the first electronic device includes:
the first control module is configured to send a start instruction to the first electronic device and each of the second electronic devices, so as to start each of the audio players connected correspondingly to play the test corpus according to a certain sequence;
the log acquisition module is used for capturing and storing the log of the equipment to be tested;
the evaluation module is respectively connected with the log acquisition module and the first control module and is used for reading and processing the logs so as to obtain the total awakening times of the equipment to be tested;
the first electronic device and each second electronic device respectively comprise a second control module, and the second control module is correspondingly connected with the audio player and is used for receiving the starting instruction sent by the first control module to start the correspondingly connected audio player to play one piece of test corpus;
the log acquisition module is used for respectively capturing and storing logs of the equipment to be tested before starting testing, before playing the testing corpus each time, after playing the testing corpus each time and after finishing testing;
the evaluation module specifically comprises:
the frequency acquisition unit is connected with the log acquisition module and used for acquiring the initial awakening frequency of the device to be tested before the test is started, the first awakening frequency of the device to be tested before the test corpus is played each time, the second awakening frequency of the device to be tested after the test corpus is played each time and the final awakening frequency of the device to be tested after the test is finished according to the read log;
the storage unit is connected with the times acquisition unit and is used for storing the awakened times acquired by the times acquisition unit;
the comparison and judgment unit is connected with the storage unit and used for comparing whether the corresponding first awakened times and the second awakened times are equal or not and outputting a comparison result aiming at each played test corpus;
the recording unit is connected with the comparison and judgment unit and used for receiving the comparison result and recording the corresponding test corpus into a failure item list when the comparison result shows that the first awakened times and the second awakened times are equal;
the circulation judging unit is respectively connected with the first control module and the operation module and is used for recording the times of sending the starting instruction by the first control module and judging whether the playing of all the test corpus is finished or not, if not, the unfinished information is fed back to the first control module so that the first control module continues to send the starting instruction; if the operation is finished, the finishing information is sent to the operation module;
and the computing unit is connected with the storage unit and used for processing the completion information according to the final awakened times and the initial awakened times to obtain the total awakened times of the equipment to be tested.
2. The system according to claim 1, wherein the plurality of control devices further comprises at least one third electronic device, and the first electronic device is connected to each of the third electronic devices;
each third electronic device comprises a third control module used for starting the audio player connected correspondingly and playing specific noise;
the second control module in the first electronic device is further configured to: when a noise scene needs to be simulated, the second electronic device is controlled to start the audio player correspondingly connected to the second electronic device to play the test corpus, and meanwhile, each third control module is controlled to start the audio player correspondingly connected to the second electronic device to play a specific noise.
3. A voice wake-up rate test method, using the voice wake-up rate test system of claim 1, comprising the steps of:
step S1, the equipment to be tested runs a voice recognition model;
s2, respectively controlling each control device to start the audio player correspondingly connected to play preset test corpora;
and S3, calculating the total awakening times of the equipment to be tested by the first electronic equipment.
4. The method for testing a voice wake-up rate according to claim 3, wherein the step S2 specifically comprises the following steps:
step S21 of setting at least two of the plurality of control devices as second electronic devices;
step S22, a first control module of the first electronic device sends a starting instruction to the first electronic device and each second electronic device respectively so as to start each audio player correspondingly connected to the first electronic device to play the test corpus according to a certain sequence;
the step S3 specifically comprises the following steps: and the first electronic equipment captures and stores the log of the equipment to be tested, reads and processes the log to obtain the total awakening times of the equipment to be tested.
5. The method of claim 4, wherein the voice wake-up rate test method,
a step S101 is further included between the step S1 and the step S2: before testing, capturing and storing the log of the equipment to be tested, reading and processing the log, and acquiring and storing the initial awakening times of the equipment to be tested;
the step S22 specifically includes the following steps:
step S221, capturing and storing the log of the device to be tested, reading the log to obtain and store the first awakened times of the device to be tested before playing the test corpus;
step S222, the first control module controls the first electronic device and each of the second electronic devices in a certain order to start the audio player correspondingly connected to the first electronic device to play one test corpus;
step S223, capturing and storing the log of the device to be tested, reading the log to obtain and store a second awakened frequency of the device to be tested after the current test corpus is played;
step S224, for each played test corpus, comparing whether the corresponding first awakened times and the corresponding second awakened times are equal, and outputting a comparison result;
step S225, when the comparison result shows that the first awakened times are equal to the second awakened times, recording the corresponding test corpus into a failure item list;
step S226, determining whether all the test corpora have been played completely according to the number of times the first control module sends the start instruction, and if not, continuing the step S221; if the step is finished, continuing to the step S3;
the step S3 specifically includes the following steps:
step S31, capturing and storing the log of the equipment to be tested in a fixed path, and reading the log to obtain the last awakening times of the equipment to be tested after the test;
and step S32, processing according to the final awakened times and the initial awakened times to obtain the total awakened times of the equipment to be tested.
6. The method of claim 4, wherein the voice wake-up rate test method,
the step S21 further includes: setting at least one of the plurality of control devices as a third electronic device;
the step S22 further includes: when a noise scene needs to be simulated, the first control module controls each third electronic device to start the audio player correspondingly connected to play the specific noise while controlling the audio player correspondingly connected to be started to play the test corpus.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010144278.1A CN111341297B (en) | 2020-03-04 | 2020-03-04 | Voice wake-up rate test system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010144278.1A CN111341297B (en) | 2020-03-04 | 2020-03-04 | Voice wake-up rate test system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111341297A CN111341297A (en) | 2020-06-26 |
| CN111341297B true CN111341297B (en) | 2023-04-07 |
Family
ID=71187165
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010144278.1A Active CN111341297B (en) | 2020-03-04 | 2020-03-04 | Voice wake-up rate test system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111341297B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111933137B (en) * | 2020-08-19 | 2024-04-16 | Oppo广东移动通信有限公司 | Voice wake-up test method and device, computer readable medium and electronic equipment |
| CN112102814A (en) * | 2020-11-09 | 2020-12-18 | 深圳市友杰智新科技有限公司 | Method and device for evaluating wake-up model, computer equipment and storage medium |
| CN113093691B (en) * | 2021-03-16 | 2024-02-02 | 通山星火原实业有限公司 | Testing equipment and testing method for intelligent household Bluetooth voice remote controller |
| CN113593549A (en) * | 2021-06-29 | 2021-11-02 | 青岛海尔科技有限公司 | Method and device for determining awakening rate of voice equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107613447A (en) * | 2017-10-27 | 2018-01-19 | 深圳市传测科技有限公司 | A kind of intelligent terminal audio test device, system and method for testing |
| CN108877770A (en) * | 2018-05-31 | 2018-11-23 | 北京百度网讯科技有限公司 | For testing the methods, devices and systems of intelligent sound equipment |
| CN109243426A (en) * | 2018-09-19 | 2019-01-18 | 易诚博睿(南京)科技有限公司 | A kind of automatization judgement voice false wake-up system and its judgment method |
| CN110265020A (en) * | 2019-07-12 | 2019-09-20 | 大象声科（深圳）科技有限公司 | Voice awakening method, device and electronic equipment, storage medium |
| CN110415698A (en) * | 2018-11-15 | 2019-11-05 | 腾讯科技（深圳）有限公司 | Artificial intelligence data detection method and device and storage medium |
| CN110808029A (en) * | 2019-11-20 | 2020-02-18 | 斑马网络技术有限公司 | Vehicle-mounted machine voice test system and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107221326B (en) * | 2017-05-16 | 2021-05-28 | 百度在线网络技术（北京）有限公司 | Voice awakening method and device based on artificial intelligence and computer equipment |
| CN109391528A (en) * | 2018-08-31 | 2019-02-26 | 百度在线网络技术（北京）有限公司 | Awakening method, device, equipment and the storage medium of speech-sound intelligent equipment |
-
2020
- 2020-03-04 CN CN202010144278.1A patent/CN111341297B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107613447A (en) * | 2017-10-27 | 2018-01-19 | 深圳市传测科技有限公司 | A kind of intelligent terminal audio test device, system and method for testing |
| CN108877770A (en) * | 2018-05-31 | 2018-11-23 | 北京百度网讯科技有限公司 | For testing the methods, devices and systems of intelligent sound equipment |
| CN109243426A (en) * | 2018-09-19 | 2019-01-18 | 易诚博睿(南京)科技有限公司 | A kind of automatization judgement voice false wake-up system and its judgment method |
| CN110415698A (en) * | 2018-11-15 | 2019-11-05 | 腾讯科技（深圳）有限公司 | Artificial intelligence data detection method and device and storage medium |
| CN110517679A (en) * | 2018-11-15 | 2019-11-29 | 腾讯科技（深圳）有限公司 | A kind of audio data processing method and device, storage medium of artificial intelligence |
| CN110265020A (en) * | 2019-07-12 | 2019-09-20 | 大象声科（深圳）科技有限公司 | Voice awakening method, device and electronic equipment, storage medium |
| CN110808029A (en) * | 2019-11-20 | 2020-02-18 | 斑马网络技术有限公司 | Vehicle-mounted machine voice test system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111341297A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111341297B (en) | Voice wake-up rate test system and method | |
| CN109817219A (en) | Voice wake-up test method and system | |
| CN108511000B (en) | Method and system for testing identification rate of awakening words of intelligent sound box | |
| CN110288997B (en) | Device wake-up method and system for acoustic networking | |
| CN109192193B (en) | Voice recognition product testing method and testing device | |
| CN101452696B (en) | Signal processing device, signal processing method and program | |
| CN101427314B (en) | Method and apparatus for automatic adjustment of play speed of audio data | |
| CN110602624B (en) | Audio testing method and device, storage medium and electronic equipment | |
| US20230018853A1 (en) | Creating a cinematic storytelling experience using network-addressable devices | |
| US11511200B2 (en) | Game playing method and system based on a multimedia file | |
| CN110245679A (en) | Image clustering method, device, electronic equipment and computer readable storage medium | |
| CN104698884B (en) | A kind of method of control packet, device and system | |
| CN108711415B (en) | Method, apparatus and storage medium for correcting time delay between accompaniment and dry sound | |
| CN110581915A (en) | Stability testing method and device, storage medium and electronic equipment | |
| CN109979485A (en) | Audio evaluation method and device | |
| CN107896334A (en) | Carry out live method and apparatus | |
| CN103366074A (en) | Situational command system and operation method | |
| CN105828254A (en) | Audio adjusting method and device | |
| CN113270112A (en) | Electronic camouflage voice automatic distinguishing and restoring method and system | |
| CN109410972B (en) | Method, device and storage medium for generating sound effect parameters | |
| JP2023526285A (en) | Test method and apparatus for full-duplex voice interaction system | |
| CN111429949B (en) | Pitch line generation method, device, equipment and storage medium | |
| CN114071127A (en) | Live video delay testing method and device, storage medium and electronic equipment | |
| CN103576642B (en) | Nuclear power plant analog machine acousto-optic environment system and its implementation | |
| CN101247111A (en) | Method and apparatus for controlling audio signal output level of portable audio device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |