KR102455709B1 - Method and apparatus for automated evaluation of synthetic speech based on artificial intelligence - Google Patents

Abstract

The present invention relates to a method and apparatus for automating the evaluation of synthetic voices based on artificial intelligence, the steps of separating the synthesized voice and the original voice into chord and percussion, respectively, and the separated chord and percussion parts as an evaluation index, the average value , and calculating the evaluation score by calculating the standard deviation, the average value of the mel spectrogram, and the standard deviation of the mel spectrogram.

Description

Method and apparatus for automated evaluation of synthetic speech based on artificial intelligence}

Embodiments of the present invention relate to a method and apparatus for automating the evaluation of artificial intelligence-based synthesized speech capable of automating the evaluation of speech generated through artificial intelligence or a computer.

Artificial intelligence (AI) refers to a technology that realizes human learning ability, reasoning ability, perceptual ability, and natural language understanding ability through computer programs. Artificial intelligence, which is currently being developed, is mainly used in technologies for implementing a Conversational User Interface (CUI). Technologies used here include speech to text (STT), natural language understanding (NLU), natural language generation (NLG), and text to speech (TTS). .

Text-to-speech synthesis is a 　speech synthesis application used to convert text into speech, and it can convert arbitrary character strings, words, and sentences into the sound of a person speaking the same content and output it. Through this, the result text corresponding to the command input through voice recognition, such as a smartphone, TV, speaker, navigation, etc., is generated and output as reproducible audio by naturally synthesized human voice in a text-to-speech synthesis application.

This text-to-speech synthesis method shows natural speech characteristics at the level of conversation with a human compared to the related art through text-to-speech synthesis based on an artificial neural network or using a computer.

However, although natural speech characteristics are derived through artificial neural networks or computer-based speech synthesis methods, the method of evaluating synthesized speech through various methods depends on the evaluation of questionnaires and the judgment of tone by individuals, so it is subjective of the individual. There was a problem that the evaluation had to proceed accordingly.

The above-mentioned background art is technical information possessed by the inventor for the derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be a known technique disclosed to the general public prior to the filing of the present invention.

Republic of Korea Patent Publication No. 2009-0026504

Until now, the evaluation of voices generated by artificial intelligence or computers is a form of estimating the performance of the voice module and the quality of the voice uttered through the module by scoring scores through several third parties who have not heard the voice. Since progress has been made, an embodiment of the present invention for solving the problems of the prior art provides an artificial intelligence-based synthetic voice evaluation automation method and apparatus capable of automating the evaluation of a voice generated through artificial intelligence or a computer do.

Another embodiment of the present invention provides an artificial intelligence-based synthetic voice evaluation automation method and apparatus that can automate and objectify the evaluation of a machine-made voice based on detailed indicators for the synthesized voice evaluation.

An aspect of the present invention provides a step of separating a synthesized voice and an original voice into harmonic and percussive, respectively, and the separated harmonic and percussive parts as evaluation indicators, average value, standard deviation, mel and calculating an evaluation score by calculating a spectrogram average value and a mel spectrogram standard deviation.

In addition, the step of separating each chordally and percussion instrument is characterized in that the synthesized voice and the original voice are separated through a short-time Fourier transform (STFT), respectively.

The calculating of the evaluation score further includes converting the separated chordal and percussion parts into a melspectrogram.

In addition, the converting into the mel spectrogram may further include converting the two-dimensional information converted into the mel spectrogram into a one-dimensional one.

The calculating of the evaluation score further includes outputting a comparison result by comparing the calculated evaluation score and the evaluation score with a threshold value.

Another aspect of the present invention is a chord/percussion source separation unit that separates the synthesized voice and the original voice chordically and percussion, respectively, and calculates the average value and the melspectrogram average value with the separated chordal and percussion parts. and a standard deviation module for calculating standard deviation and melspectrogram standard deviation with the separated chord and percussion parts.

In addition, the chordal/percussion source separating unit separates the synthesized voice and the original voice through a short-time Fourier transform (STFT), respectively.

In addition, the evaluation automation apparatus further includes a mel spectrogram conversion module unit for converting the separated chord and percussion parts into a mel spectrogram.

In addition, the evaluation automation apparatus further includes a Mel 1D transform unit for converting the 2D information converted into the Mel spectrogram into 1D.

The evaluation automation apparatus further includes an output unit for outputting a comparison result by comparing the calculated evaluation score with a threshold value and the evaluation score.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

The method and apparatus for automating the evaluation of artificial intelligence-based synthesized speech according to an embodiment of the present invention may enable automation of the evaluation of speech generated through artificial intelligence or a computer.

In addition, for the evaluation of the voice generated by artificial intelligence or computer, several third parties who have not heard the voice are hired or asked to score and evaluate the performance of the voice module and the quality of the voice uttered through the module, or In the evaluation method, it has the effect of enabling automation and objectification of the evaluation of synthesized speech based on artificial intelligence-based detailed indicators.

1 is a diagram illustrating an apparatus for evaluating synthesized speech according to an embodiment of the present invention.
2 is a diagram illustrating a formula for each detailed indicator according to an embodiment of the present invention.
3 is a flowchart illustrating a method for evaluating a synthesized voice according to an embodiment of the present invention.
4 is a flowchart illustrating an evaluation score reflection method in the synthesized speech evaluation apparatus according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense.

In the following examples, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have means that the features or components described in the specification are present, and the possibility that one or more other features or components may be added is not excluded in advance.

In the following embodiments, when it is said that a part such as a film, region, or component is on or on another part, not only when it is directly on the other part, but also another film, region, component, etc. is interposed therebetween. Including cases where there is

In the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

The present invention is for automating the evaluation of speech generated by artificial intelligence or a computer, and is a technology for automating and objectifying the evaluation of machine-made speech based on detailed indicators.

For the evaluation of the synthesized voice, questionnaire evaluation has been conducted through a plurality of third parties who have not heard the synthesized voice to be evaluated.

Specifically, a questionnaire evaluation is conducted on the synthesized voice through a large number of people who have not heard the synthesized voice and the original voice created through a specific program. Thereafter, the lowest and highest points are determined, and the score for each voice is selected as a value between them according to the individual's thoughts. The questionnaire for this evaluation can simply give a score for the voice, and by several indicators, for example, an indicator that evaluates whether a voice is distinguishable from a human intonation, and an indicator that evaluates the presence of noise generated by a machine. It is possible to obtain the average of the scores or a method of summing the scores by giving scores to the voice as an indicator, an indicator for determining whether all syllables are expressed, or the like.

One problem here is the problem of timbre judgment. The fact that the timbre differs depending on the instrument or the person making the sound is obvious to anyone who can hear the sound, but the timbre is a difficult element to define. Therefore, experiments have been performed to express these tones with three primary colors, such as colors, or with 13 primary colors, but complete sound classification is not possible.

As described above, in the synthetic voice evaluation method using a third party, the evaluation score is consistent depending on the individual's subjective tendency, and it is difficult to evaluate objectively. Accordingly, an objective and automated synthetic voice evaluation device will be described with reference to the drawings below.

1 is a diagram illustrating an apparatus for evaluating synthesized speech according to an embodiment of the present invention.

1, the synthesized voice evaluation apparatus 100 includes a synthesized voice input unit 110, an original voice input unit 120, a harmonic/percussive source separation (HPSS) module unit 130, It may include a Mel-spectrogram transformation module unit 140 , a Mel one-dimensional transformation unit 150 , an average value module unit 160 , a standard deviation module unit 170 , an output unit 180 , and the like. .

The synthesized voice evaluation apparatus 100 is an artificial intelligence-based automatic synthesized voice evaluation module, controls components in the apparatus, and may perform evaluation with scores calculated from a plurality of indices. Accordingly, the synthesized voice input unit 110 may receive synthesized voice data output from any one voice synthesizing apparatus. In addition, the synthesized voice input unit 110 may transmit the input voice data to the HPSS module unit 130 .

The original voice input unit 120 may receive voice data as an original voice corresponding to the synthesized voice as a file or record and store the speaker's voice. In addition, the original voice input unit 120 may transmit the input original voice to the HPSS module unit 130 .

는 단시간 푸리에 변환(STFT: Short-time Fourier transform)을 통해 다음 (수학식 1)과 같이 변환될 수 있다.The HPSS module unit 130 may separate the synthesized voice, that is, the mechanical sound, into chord/percussion source parts. specifically a specific sound wave

can be transformed as follows (Equation 1) through a short-time Fourier transform (STFT).

[Equation 1]

,

,

은 프레임 수,

은 프레임 크기 (즉, 이산 STFT의 길이),

은 윈도우 함수이며

는 hop(이하 홉)의 크기다. 이 변환을 거친 후 음파

의 파워 스펙트럼은 다음 (수학식 2)와 같이 정의될 수 있다.here,

,

,

number of frames,

is the frame size (i.e. the length of the discrete STFT),

is a window function

is the size of hops (hereafter hops). After this transformation, the sound wave

The power spectrum of can be defined as follows (Equation 2).

[Equation 2]

에 대한 유한집합

이 고려되어야한다. 이 경우 중간값,

는

일 때 다음 (수학식 3)과 같이 정의된다.Here, any finite exponential set

finite set for

This should be taken into account. In this case, the median,

Is

When , it is defined as follows (Equation 3).

[Equation 3]

에 대한 화음적, 타악기적 중간값 필터는 다음 (수학식 4)와 같이 정의될 수 있다.specific matrix

A chordal and percussion median filter for ? can be defined as follows (Equation 4).

[Equation 4]

와

는 타악기적, 화음적 중간값 필터의 크기이다. (수학식 2)에서 언급한

에 (수학식 4)의 두 함수를 적용하여

와

를 얻은 후 각각의 결과에 다음 (수학식 5)의 바이너리 마스크(binary mask) 함수들을 적용한다.here,

Wow

is the size of the percussion and chord median filter. (Equation 2) mentioned in

By applying the two functions of (Equation 4) to

Wow

After obtaining , apply the binary mask functions of the following (Equation 5) to each result.

[Equation 5]

에 위의 binary mask 함수들을 적용하면, 원음의 화음적 부분과 타악기적 부분에 대한 푸리에 변환결과들

과

을 얻을 수 있으며, 이들을 역푸리에 변환하여 얻은 결과,

와

가 원음

에 대한 화음적 및 타악기적 부분들이다. 이 과정을 화음적/타악기적 소스 분리(Harmonic/Percussive Source Separation)라 한다. 원음

에 대해 다음 (수학식 6)은 자명할 수 있다.The result obtained by transforming the original note

Applying the above binary mask functions to , the Fourier transform results for the chord and percussion

class

can be obtained, and the result obtained by inverse Fourier transform

Wow

the original sound

chordal and percussion parts for This process is called Harmonic/Percussive Source Separation. original sound

For , the following (Equation 6) may be self-evident.

[Equation 6]

평면에서 그의 역함수

를 분석하는 것이 도움이 된다. 이 역함수는 주파수가 가로축에 놓여있으며, 세로 축에 시간에 해당되는 함수가 있다. 그리고

와

는 확률분포로 다루어 분석을 수행할 수 있다.Most voices have both chordal and percussion parts. The chordal part takes the form of a function whose frequency changes continuously with time, whereas the percussion part contains several frequencies at a specific time. Therefore, when analyzing the percussion part,

its inverse in the plane

It is helpful to analyze In this inverse function, frequency lies on the horizontal axis, and the vertical axis has a function corresponding to time. and

Wow

can be analyzed as a probability distribution.

The original voice and the synthesized voice are separated into chord/percussion parts through the HPSS module unit 130 and input to the melspectrogram conversion module unit 140, the average value module unit 160 and the standard deviation module unit 170, respectively. can be

The mel spectrogram conversion module unit 140 generates a spectrogram in a manner that expresses voice data as a picture on a two-dimensional plane. A spectrogram is a tool for visualizing and understanding sound or waves, and combines the characteristics of a waveform and a spectrum. The mel-scale refers to reflecting the human ear in the spectrogram, which is a color map. In general, as the frequency increases, the frequency interval that humans distinguish becomes wider, and the Mel scale converts the scale unit by using a filter based on this principle. The generated mel spectrograms are generated in different shapes when different speakers speak the same sentence or when the same speaker speaks different sentences.

As described above, the mel spectrogram generated from the mel spectrogram transformation module unit 140 is transmitted to the mel one-dimensional transformation unit 150, and the mel one-dimensional transformation unit 150 converts the mel spectrogram transferred in two dimensions into one dimension. conversion can be performed. In addition, mel spectrograms for the original speech and the synthesized speech converted into one dimension may be input to the average value module unit 160 and the standard deviation module unit 170 , respectively.

The average value module unit 160 may calculate an average value for each chord/percussion part of the original voice, and may calculate an average value for each chord/percussion part of the synthesized voice. In addition, an average value of the mel spectrogram of the chord/percussion part of the original voice may be calculated, and the average value of the mel spectrogram of the chord/percussion part of the synthesized voice may be calculated, respectively. The calculated value may be transmitted to the output unit 180 .

The standard deviation module unit 170 may calculate a standard deviation value for the chord/percussion part of the original voice, respectively, and may calculate a standard deviation value for the chord/percussion part of the synthesized voice, respectively. In addition, standard deviation values of the mel spectrogram of the chord/percussion part of the original voice may be calculated, respectively, and standard deviation values of the mel spectrogram of the chord/percussion part of the synthesized voice may be calculated, respectively. The calculated value may be transmitted to the output unit 180 .

와

가 얻어졌다고 가정하면 다음의 값들을 구할 수 있다.Expressing this in a specific formula, it is possible to express

Wow

Assuming that is obtained, the following values can be obtained.

의 평균(

), 표준편차(

), MEL스펙트로그램의 평균(

)과 표준편차(

)-

average of (

), Standard Deviation(

), the mean of the MEL spectrogram (

) and standard deviation (

)

의 평균(

), 표준편차(

), MEL스펙트로그램의 평균(

)과 표준편차(

).-

average of (

), Standard Deviation(

), the mean of the MEL spectrogram (

) and standard deviation (

).

If the human voice used to obtain these values is separated through values corresponding to the machine sound (ie, synthesized voice) generated through the corresponding script, the following values can also be obtained.

의 평균(

), 표준편차(

), MEL스펙트로그램의 평균(

)과 표준편차(

).-

average of (

), Standard Deviation(

), the mean of the MEL spectrogram (

) and standard deviation (

).

의 평균(

), 표준편차(

), MEL스펙트로그램의 평균(

)과 표준편차(

).-

average of (

), Standard Deviation(

), the mean of the MEL spectrogram (

) and standard deviation (

).

As shown in the table of FIG. 2, eight physical property indicators can be obtained.

2 is a diagram illustrating a formula for each detailed indicator according to an embodiment of the present invention.

), 화음부의 상대 표준편차(

, 타악기부의 상대 평균(

), 타악기부의 상대 표준편차(

), 화음부의 상대 MEL 평균(

), 화음부의 상대 MEL 표준편차(

), 타악기부의 상대 MEL 평균(

및 타악기부의 상대 MEL 표준편차(

이며, 각각의 수식을 토대로 값을 산출할 수 있다. 각각의 지표들은 기 설정된 계수들(

)을 찾아 다음 (수학식 7)을 계산할 수 있다.Referring to FIG. 2 , as detailed indicators, the eight physical property indicators are the relative average (

), the relative standard deviation of the chord (

, the relative mean of percussion (

), the relative standard deviation of percussion instruments (

), the relative MEL average of the chords (

), the relative MEL standard deviation of the chord (

), the relative MEL mean of percussion (

and the relative MEL standard deviation of the percussion part (

, and a value can be calculated based on each formula. Each index is a preset coefficient (

) to calculate the following (Equation 7).

[Equation 7]

값들은 경험적으로 세워질 수 있는 값이며, 원하는 점수의 의미에 따라 임의로 세울 수도 있다.here,

The values are values that can be established empirically, and can be set arbitrarily according to the meaning of the desired score.

The output unit 160 receives and outputs the evaluation scores generated through the average value module unit 160 and the standard deviation module unit 170 , and may be, for example, a display device. In addition, the calculated evaluation score data may be transmitted to the speech synthesis device to which the synthesized voice is input, and when the evaluation score is lower than a threshold value, a re-learning performance and redesign request message may be transmitted to the speech synthesis device.

Through this, the synthesized voice evaluation apparatus 100 may automatically calculate and output evaluation scores for 8 indicators compared to the original voice for the input synthesized voice.

3 is a flowchart illustrating a method for evaluating a synthesized voice according to an embodiment of the present invention.

Referring to FIG. 3 , in step S300 , a synthesized voice is input from the synthesized voice input unit 110 of the synthesized voice evaluation apparatus 100 , and the original voice is received from the original voice input unit 120 and transmitted to the HPSS module unit 130 . will do

In step S310, the HPSS module unit 130 separates the received synthesized voice and the original voice into chord/percussion parts, respectively. In step S320, the mel spectrogram conversion module unit 140 converts the chord/percussion parts of the synthesized voice and the original voice into mel spectrograms, respectively.

In step S330, the average value module unit 160 calculates the average value for the chord/percussion part of the original voice and the average value for the mel spectrogram, respectively, and the average value for the chord/percussion part of the synthesized voice and the mel An average value for each spectrogram is calculated.

In step S340, the standard deviation module unit 170 calculates the average value for the chord/percussion part of the original voice and the standard deviation value for the mel spectrogram, respectively, and the average value for the chord/percussion part of the synthesized voice. and standard deviation values for the Mel spectrogram are respectively calculated.

Thereafter, in step S350 , the output unit 160 receives and outputs the evaluation score generated through the average value module unit 160 and the standard deviation module unit 170 .

4 is a flowchart illustrating an evaluation score reflection method in the synthesized speech evaluation apparatus according to an embodiment of the present invention.

Referring to FIG. 4 , in step S400, a sentence or voice is input as an input value to a speech synthesis device (not shown), and in step S410, the input sentence or voice is applied to the pre-learned text-to-speech synthesis model in the speech synthesis device. passed, and the synthesized voice is output.

In step S420, the synthesized voice evaluation apparatus 100 evaluates the scores calculated from the new indicators, and specifically acquires chordal and percussion partial values for the synthesized voice and the original voice, and obtains the acquired chordal and percussion instruments. After calculating the mean, standard deviation, MEL mean, and MEL standard deviation as partial values, the calculated values are used as evaluation scores for the input synthetic voice. Thereafter, in step S430, the synthesized speech evaluation apparatus 100 compares and determines whether the calculated score is equal to or greater than the threshold value, and when the calculated score is equal to or greater than the threshold value, in step S440, as the evaluation result of the speech synthesis apparatus, the score for each index is output, and such The data is transmitted to the speech synthesizer. This can be determined as a state in which the speech synthesizer can be utilized as a complete module to a certain extent.

However, in step 430, the synthesized speech evaluation apparatus 100 compares and determines whether the calculated score is equal to or greater than the threshold value, and when it is less than or equal to the threshold value, in step 450, as a result of the comparison evaluation of the speech synthesis apparatus, the score for each index is output, The score of the index below the threshold is output. In addition, information or a message suggesting re-learning or module redesign of the speech synthesis device is transmitted to the speech synthesis device.

Through this, automated evaluation of the synthesized voice from the designed voice synthesizer can be performed and objective information can be provided to a certain extent. can be checked

An artificial intelligence-based synthetic voice evaluation automation method according to an embodiment of the present invention is for automating the evaluation of a voice generated through artificial intelligence or a computer, and it automates the evaluation of a voice made by a machine based on detailed indicators, There are advantages to being objective.

Meanwhile, the present invention can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device.

In addition, the computer-readable recording medium may be distributed in a network-connected computer system, and the computer-readable code may be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

The steps constituting the method according to the present invention may be performed in an appropriate order, unless the order is explicitly stated or there is no description to the contrary. The present invention is not necessarily limited to the order in which the steps are described.

The use of all examples or exemplary terminology (eg, etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by the examples or exemplary terms unless limited by the appended claims. It is not limited. In addition, those skilled in the art can appreciate that various modifications, combinations and changes can be made according to design conditions and factors within the scope of the appended claims or equivalents thereof.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below, but also all ranges equivalent to or changed from these claims are of the spirit of the present invention. would be said to belong to the category.

As such, the present invention has been described with reference to one embodiment shown in the drawings, but this is merely exemplary, and those skilled in the art will understand that various modifications and variations of the embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

100: synthesized speech evaluation device
110: synthesized voice input unit
120: original voice input unit
130: HPSS module unit
140: Mel spectrogram conversion module unit
150: Mel one-dimensional transformation unit
160: average value module unit
170: standard deviation module part
180: output unit

Claims (10)

A step in which a chord/percussion source separation unit performs short-time Fourier transform (STFT) on the received synthesized voice and the original voice, and then inverse Fourier transforms the transform results to obtain the following (1) to (4), respectively
(1) a chordal source of the synthesized speech;
(2) a percussion source of the synthesized voice;
(3) a chordal source of the original voice;
(4) a percussion source of the original voice;
converting, by a mel spectrogram conversion module unit, the above (1) to (4) into two-dimensional mel spectrograms, respectively;
Step, by the Mel one-dimensional transformation module unit, converting the two-dimensional Mel spectrogram of (1) to (4) into one dimension to obtain the following (5) to (8), respectively
(5) one-dimensional Mel spectrogram of (1) above,
(6) one-dimensional Mel spectrogram of (2) above,
(7) one-dimensional Mel spectrogram of (3) above,
(8) the one-dimensional Mel spectrogram of (4);
After the average value module unit calculates the average values of (1) to (8) above, respectively, the average values of (1), (2), (5) and (6) above (3), (4), (7) ) and calculating an evaluation score compared to the average value of (8); and
After the standard deviation module unit calculates the standard deviations of (1) to (8) above, respectively, the standard deviations of (1), (2), (5) and (6) above (3), (4) , (7) and (8) compared to the standard deviation of the evaluation score is calculated for an AI-based synthetic voice evaluation automation method.
After short-time Fourier transform (STFT) of the received synthesized voice and the original voice, the chordal/percussion source separation unit obtains the following (1) to (4) by inverse Fourier transforming the transform results, respectively
(1) a chordal source of the synthesized speech;
(2) a percussion source of the synthesized voice;
(3) a chordal source of the original voice;
(4) a percussion source of the original voice;
a mel spectrogram conversion module unit for converting (1) to (4) into a two-dimensional mel spectrogram, respectively;
Mel one-dimensional transformation module unit that converts the two-dimensional Mel spectrogram of (1) to (4) into one dimension to obtain the following (5) to (8), respectively
(5) one-dimensional Mel spectrogram of (1) above,
(6) one-dimensional Mel spectrogram of (2) above,
(7) one-dimensional Mel spectrogram of (3) above,
(8) the one-dimensional Mel spectrogram of (4);
Calculate the average value of (1) to (8) above, respectively, and calculate the average value of (1), (2), (5) and (6) above (3), (4), (7) and (8) an average value module unit for calculating an evaluation score in comparison with an average value; and
Calculate the standard deviations of (1) to (8), respectively, and calculate the standard deviations of (1), (2), (5) and (6) above (3), (4), (7) and (8) ), an artificial intelligence-based synthetic voice evaluation automation device including a standard deviation module unit that calculates an evaluation score in comparison with the standard deviation of . delete delete delete delete delete delete delete delete

Images