CN115223588B - Child voice phrase matching method based on pinyin distance and sliding window - Google Patents

Abstract

The invention discloses a child voice phrase matching method based on pinyin distance and a sliding window, which comprises the following steps: collecting phrase audios of children, obtaining audio transcription texts through a voice recognition model, and marking given target text phrases; converting the target text phrase and the transcribed text into corresponding pinyin sequences, and searching the minimum pinyin distance between the target text phrase and the pinyin sequences of the transcribed text by using a sliding window; and calculating an optimal judgment section according to the data labeling result and the minimum distance set, wherein the phrase matching is successful when the minimum distance is smaller than the left end point of the section, the matching is failed when the minimum distance is larger than or equal to the right end point of the section, and the matching is submitted to manual judgment when the minimum distance is within the section. The invention considers the ambiguity of the pronunciation of the children and the uncertainty of the sentence length, combines the ideas of the pinyin distance and the sliding window, and uses the artificial auxiliary judgment, thereby being beneficial to improving the accuracy of target text phrase matching, more accurately judging the cognition level of the children and having practicability.

Description

A children's speech phrase matching method based on pinyin distance and sliding window

Technical Field

The invention relates to the field of natural language processing, and in particular to a children's speech phrase matching method based on pinyin distance and sliding window.

Background Art

Nowadays, the assessment of children's cognitive abilities is a direction of brain science research. One of the solutions is to ask children to give short descriptions of pictures or scenes, and match them with target text phrases to determine the correctness of cognition. However, due to the lack of literacy skills of young children, it is often necessary to evaluate based on the content of the speech, which involves the collection, transcription and judgment of audio, which greatly increases the workload of volunteers. To address this problem, machines can participate in the transcription and judgment links to save labor costs. With the development of speech recognition technology, the current recognition accuracy of adult speech can reach more than 95%, and related products are widely used. However, children may speak unclearly, and existing speech recognition models find it difficult to correct the ambiguous parts of the expression, resulting in difficulty in matching the target text phrases and increasing the number of audios misjudged as cognitive errors.

From the perspective of pinyin, if two completely different Chinese characters have similar pronunciations, the corresponding pinyins also have certain similarities. By measuring the pinyin distance, it is allowed to match Chinese characters with similar pronunciations within a certain range, which can better solve the above problem. At present, the pinyin distance is often expressed by the edit distance between the English letter strings corresponding to two pinyins, which has certain feasibility, but ignores the similarity of the pronunciation between the initial consonants or finals of the pinyin.

In the audio collection process, considering the cognitive ability of young children, it is difficult to limit the length of children's speech content, and there will often be many redundant words, which will affect the matching of target text phrases. Therefore, in the longer children's speech transcription text, it is necessary to find possible matching target text phrases, and the sliding window strategy is feasible.

Summary of the invention

In view of this, the purpose of the present invention is to provide a children's speech phrase matching method based on pinyin distance and sliding window, so as to find possible matching target text phrases in the content of children's speech and reduce the adverse effects of children's ambiguous pronunciation.

In order to achieve the above object, the present invention adopts the following technical solution:

A children's speech phrase matching method based on pinyin distance and sliding window comprises the following steps:

Step 1: Given a target text phrase, collect audio of the child's phrase, obtain the transcribed text of the child's phrase audio through the speech recognition model, and annotate it according to whether the content expressed in the audio includes the target text phrase;

Step 2: Convert the target text phrase and the transcribed text into corresponding pinyin sequences. In the pinyin sequence corresponding to the transcribed text, use the sliding window algorithm to find the subsequence with the smallest pinyin distance to the target text phrase and record the minimum distance, including:

2.1) Ignoring the pinyin tones, the target text phrases and the transcribed text are converted into corresponding pinyin sequences;

2.2) Use a sliding window algorithm, with the window size being the same as the number of characters in the target text phrase. The window slides rightward one character at a time, traversing the pinyin sequence corresponding to the transcribed text, and finding the subsequence with the minimum pinyin distance to the target text phrase. The subsequence length = window size, and record the minimum distance. If there are multiple target text phrases, perform this operation on each target text phrase to obtain a set of minimum distances. The number of set elements is the number of target text phrases. Finally, find the minimum value in the set as the minimum distance between the transcribed text and multiple target text phrases.

2.3) For two pinyin sequences S = {s ₁ , s ₂ , ..., s _n }, Q = {q ₁ , q ₂ , ..., q _n }, we have:

d(S, Q)=[d(s ₁ , q ₁ )+d(s ₂ , q ₂ )+……+d(s _n , q _n )]÷n

d is the phonetic distance. For the phonetic symbols s _i and q _i of two independent characters, s _i and q _i are split into the initial part and the final part respectively, then:

d( _si , _qi ) = initial consonant distance ( _si , _qi ) + final vowel distance ( _si , _qi )

Initial consonant distance (s _i , q _i ) = initial consonant edit distance (s _i , q _i ) × initial consonant weight (s _i , q _i )

The initial consonant weights (s _i , q _i ) are manually designed based on the pronunciation similarity of the initial consonants of s _i and q _i , with a weight range of [0.5, 1.5]. The calculation method of the final distance (s _i , q _i ) is consistent with the initial consonant distance.

Step 3: For all the labeled data in step 1, the minimum distance is calculated using the method described in step 2 to obtain a set of minimum distances, and a judgment interval is obtained according to the set ratio of manual participation. For each minimum distance, if it is less than the left endpoint of the interval, the target text phrase matches successfully; if it is greater than or equal to the right endpoint of the interval, the target text phrase matches unsuccessfully; if it is within the interval, that is, including the left endpoint of the interval but not including the right endpoint of the interval, it is manually determined whether the target text phrase matches; according to the data labeling results, for each set manual participation ratio, a sliding window algorithm is used to find the corresponding judgment interval when the accuracy is maximized, specifically including:

3.1) Let the judgment interval be [left, right). If the minimum distance is less than left, the target text phrase matches successfully. If the minimum distance is greater than or equal to right, the target text phrase matches unsuccessfully. If left is less than or equal to the minimum distance and less than right, it is manually determined whether the target text phrase matches.

3.2) The set ratio of the degree of manual participation is the sequence {0, k ₁ %, k ₂ %, ..., k _t %}. The m minimum distances of all the labeled data calculated in step 2 are sorted in ascending order to obtain an ordered array a = {d ₁ , d ₂ , ..., d _i , ..., d _j , ..., d _m }. When the manual ratio is k _r %, the sliding window algorithm is used on the ordered array a, with m × k _r % as the window size. Let the current window be (d _i , d _j ), then j-i+1 = m × k _r %, and the determination method of the judgment interval [left, right) is:

For each judgment interval, all data are matched using the rules described in step 3.1) to filter the data that require manual judgment and compare them with the labeled data to calculate the accuracy of the current judgment result. When using the sliding window algorithm, initially i=0, the window moves to the right by 1 unit each time, and the judgment interval that maximizes the accuracy of the judgment result is found as the optimal judgment interval when the manual ratio is k _r %.

Compared with the prior art, the present invention has the following technical effects:

The present invention is a children's speech phrase matching method based on pinyin distance and sliding window. Compared with the previous method of calculating pinyin similarity using only the edit distance of pinyin, the pronunciation similarity of initials and finals is considered, and a weight matrix of the edit distance between initials and finals is constructed, which further optimizes the calculation method of pinyin distance. At the same time, the judgment interval is determined based on a large amount of data and has statistical significance.

The present invention takes into account the ambiguity of children's pronunciation and the redundancy of speech content, improves the accuracy of target text phrase matching, more accurately determines the cognitive level of children, and is feasible.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a flow chart of an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention is further described below in conjunction with specific embodiments and drawings.

Example

Referring to FIG. 1 , the present invention is a method for matching children's speech phrases based on pinyin distance and sliding window, comprising the following steps:

Step 1: Obtain the transcribed text {this is tooth this} of the child's speech with ambiguous pronunciation through the speech recognition model, and give the target text phrase {teeth};

Step 2: Convert the target text phrase and the transcribed text into corresponding pinyin sequences. In the pinyin sequence corresponding to the transcribed text, use the sliding window algorithm to find the subsequence with the smallest pinyin distance to the target text phrase and record the minimum distance, including:

2.1) Without considering the pinyin tones, the target text phrase and the transcribed text are converted into the corresponding pinyin sequences {ya, chi} and {zhe, shi, ya, ci} respectively;

2.2) Use a sliding window algorithm, with the window size being the same as the number of characters in the target text phrase (the number of characters is 2). The window slides to the right one character at a time, traversing the pinyin sequence {zhe, shi, ya, ci} corresponding to the transcribed text, and finding the subsequence with the minimum pinyin distance from the target text phrase. The subsequence length = window size, and record the minimum distance d _min = min{d({ya, chi}, {zhe, shi}), d({ya, chi}, {shi, ya}), d({ya, chi}, {ya, ci})}, where d is the pinyin distance. If there are multiple target text phrases, perform this operation on each target text phrase separately to obtain a set of minimum distances. The number of set elements is the number of target text phrases. Finally, find the minimum value in the set as the minimum distance between the transcribed text and multiple target text phrases.

2.3) The distance between two pinyin sequences S = {ya, chi} and Q = {ya, ci} is:

d(S,Q)=[d(ya,ya)+d(chi,ci)]÷2 (d is the phonetic distance)

For the pinyin of two independent characters chi and ci, split chi and ci into the initial consonant part ch and c and the final part i and i respectively, then we have:

d(chi,ci)=d(ch,c)+d(i,i)

d(ch, c) = edit distance (ch, c) × weight (ch, c)

The weight (ch, c) is designed manually based on the pronunciation similarity of ch and c, and its value is 0.5, so d(ch, c)=1×0.5=0.5, d(i, i)=0×1.0=0.

Step 3: For all the labeled data in step 1, the minimum distance is calculated using the method described in step 2 to obtain a set of minimum distances, and a judgment interval is obtained according to the set ratio of manual participation. For each minimum distance, if it is less than the left endpoint of the interval, the target text phrase matches successfully; if it is greater than or equal to the right endpoint of the interval, the target text phrase matches unsuccessfully; if it is within the interval, that is, including the left endpoint of the interval but not including the right endpoint of the interval, it is manually determined whether the target text phrase matches; according to the data labeling results, for each set manual participation ratio, a sliding window algorithm is used to find the corresponding judgment interval when the accuracy is maximized, specifically including:

3.1) Let the judgment interval be [left, right). If the minimum distance is less than left, the target text phrase matches successfully. If the minimum distance is greater than or equal to right, the target text phrase matches unsuccessfully. If left is less than or equal to the minimum distance and less than right, it is manually determined whether the target text phrase matches.

3.2) The set ratio of the degree of manual participation is a sequence of {0, 5%, 10%, ..., 50%}. The m = 5000 minimum distances of all the labeled data calculated in step 2 are sorted in ascending order to obtain an ordered array a = {d ₁ , d ₂ , ..., d _i-1 , d _i , d _i+1 , ..., d _j-1 , d _j , d _j+1 , ..., d _m } = {0, 0, ..., 1.4, 1.5, 1.5, ..., 1.9, 1.9, 1.9, ..., 4.0}. When the manual ratio is 5%, the sliding window algorithm is used for the ordered array a, with 5000 × 5% = 250 as the window size, then i and j satisfy j-i+1 = 250, there are i and j such that the window is (d _i , d _j ) = (1.5, 1.9), and the determination method of the decision interval [left, right) is:

left＝(1.4+1.5+1.5)÷3≈1.47

right＝(1.9+1.9+1.9)÷3＝1.9

For each judgment interval, all data are matched using the rules described in step 3.1) to determine the results, filter the data that require manual judgment, and compare with the labeled data to calculate the accuracy of the current judgment result; when using the sliding window algorithm, initially i=0, the window moves to the right by 1 unit each time, and finds the judgment interval [1.5, 1.9) that maximizes the accuracy of the judgment result, i.e. 89.29%, as the optimal judgment interval when the manual ratio is 5%.

The above description is only a preferred embodiment of the present invention, and certain modifications may be made thereto within the scope defined by the claims of the present invention, but all modifications will fall within the protection scope of the present invention.

Claims (3)

1. A children's voice phrase matching method based on spelling distance and sliding window is characterized in that the method comprises the following steps:

Step 1: giving a target text phrase, collecting audio of the phrase of the child, obtaining a transcription text of the audio of the phrase of the child through a voice recognition model, and marking according to whether the content of the audio expression comprises the target text phrase;

Step 2: converting the target text phrase and the transcription text into corresponding pinyin sequences, searching a subsequence with the smallest pinyin distance with the target text phrase in the pinyin sequences corresponding to the transcription text by using a sliding window algorithm, and recording the minimum distance;

Step 3: calculating the minimum distance of all marked data in the step 1 by using the method in the step 2 to obtain a set of minimum distances, obtaining a judging section according to the set proportion of the artificial participation degree, and for each minimum distance, if the minimum distance is smaller than the left end point of the section, matching the target text phrase successfully, if the minimum distance is larger than or equal to the right end point of the section, matching the target text phrase fails, and if the minimum distance is within the section, namely comprises the left end point of the section but does not comprise the right end point of the section, manually judging whether the target text phrase is matched; and according to the data labeling result, for each set manual participation proportion, searching a corresponding judgment section when the accuracy reaches the maximum by using a sliding window algorithm.

2. The method for matching children's voice phrase according to claim 1, wherein the step 2 specifically comprises:

2.1 Regardless of pinyin tones, converting the target text phrase and the transcribed text into a corresponding pinyin sequence;

2.2 Using a sliding window algorithm, wherein the window size is the same as the word number of the target text phrases, sliding the window rightwards by 1 word each time, traversing the pinyin sequence corresponding to the transcription text, searching the subsequence with the smallest pinyin distance with the target text phrases, wherein the subsequence length=window size, recording the minimum distance, if a plurality of target text phrases exist, respectively operating each target text phrase to obtain a set with the minimum distance, wherein the number of elements in the set is the number of the target text phrases, and finally searching the minimum value in the set to be used as the minimum distance between the transcription text and the plurality of target text phrases;

2.3 For two pinyin sequences s= { S ₁,s₂,……,s_n}、Q＝{q₁,q₂,……,q_n }, there are:

d(S，Q)＝[d(s₁,q₁)+d(s₂,q₂)+……+d(s_n,q_n)]÷n

d is the pinyin distance, and for pinyin s _i、q_i of two independent words, s _i、q_i is split into an initial part and a final part respectively, then there are:

d (s _i,q_i) =initial distance (s _i,q_i) +final distance (s _i,q_i)

Initial consonant distance (s _i,q_i) =initial consonant edit distance (s _i,q_i) ×initial weight (s _i,q_i)

Wherein the sound mother right value (s _i,q_i) is designed manually according to the similarity of the pronunciation of the initials of s _i、q_i, the weight range is 0.5 and 1.5, and the calculation mode of the vowel distance (s _i,q_i) is consistent with the distance of the initials.

3. The method for matching children's voice phrases according to claim 1, wherein the step 3 specifically comprises:

3.1 If the minimum distance is less than or equal to right, the matching of the target text phrase is successful, if the minimum distance is less than or equal to right, the matching of the target text phrase is failed, and if the minimum distance is less than or equal to right, the matching of the target text phrase is manually judged;

3.2 The set proportion of the artificial participation degree is a sequence {0, k ₁％,k₂％,……,k_t% }, m minimum distances of all marked data calculated in the step 2 are subjected to ascending order to obtain an ordered array a= { d ₁,d₂,……,d_i,……,d_j,……,d_m }, when the artificial proportion is k _r%, a sliding window algorithm is used for the ordered array a, m x k _r% is used as a window size, the current window is made to be (d _i,d_j), and j-i+1=m x k _r%, and the determination mode of a determination interval [ left, right ] is as follows:

For each judging section, all data are subjected to matching result judgment by using the rule in the step 3.1), data needing manual judgment are filtered, and compared with marked data, and the accuracy of the current judging result is calculated; when the sliding window algorithm is used, the window is moved rightward by 1 unit each time, and a determination section where the accuracy of the determination result is maximized is found as an optimal determination section where the manual ratio is k _r%.