DE102021103221A1 - Method of recognizing speech and translating it into written text - Google Patents

Abstract

The invention relates to a method for recognizing speech and translating it into written text using a speech text translation system, with each syllable of the language being represented in writing by at least one letter according to a specification, with the language being spoken by a person (4), with each At least one acoustic parameter is provided for each syllable with regard to its pronunciation, which is assigned to the written specification, with at least one syllable currently being spoken by the person (4) being compared with a reference value for the at least one acoustic parameter provided for this purpose, with the case that for the at least one currently spoken syllable of the speech text translation system an acoustic deviation from the respectively provided acoustic reference value for the at least one acoustic parameter is detected, this at least one syllable from the speech text translation system according to the acoustic deviation as Te xt is varied in writing.

Description

The invention relates to a method for recognizing speech and translating it into written text and a speech text translation system.

For example, dictating devices are known with which it is possible to transform spoken words into written words.

The pamphlet EP 2 385 520 A2 describes a method and apparatus for generating text from spoken words.

A text correction system is from the reference EP 2 391 105 A1 known.

From the article "'Good Luck with That!': Teaching Machines to Understand Sarcasm" by Xinyi Ou, a procedure for machine understanding of sarcasm is known.

Against this background, it was an object to improve speech recognition.

This object is achieved by a method and a language text translation system having the features of the independent patent claims. Embodiments of the method and the language text translation system are evident from the dependent patent claims.

The method according to the invention is intended for recognizing spoken language and translating or transforming it into written text using a language text translation system.

It is taken into account that each syllable of the language is represented in writing by at least one letter according to a specification. Here it is taken into account that each syllable comprises at least one sound, which is represented by at least one letter, i. H. represented in writing by at least one vowel and/or at least one consonant. According to the specification, it is possible for each commonly known syllable to be assigned a character string as text according to the specification. The language is spoken by a specific person. In addition, at least one acoustic parameter is provided for each commonly known syllable with regard to its pronunciation or oral articulation by the person, with which it is possible to define a standardized pronunciation for the respective syllable. In this case, at least one acoustic parameter is assigned to the respective written specification.

In one implementation of the method, the language spoken by the person is recorded or recorded with a microphone. At least one syllable currently spoken by the person and recorded further is compared with a reference value for the at least one acoustic parameter provided for this purpose. In the event that the voice text translation system detects an acoustic deviation from a respectively provided acoustic reference value for the at least one currently spoken syllable, this at least one syllable is provided and/or represented in writing by the voice text translation system according to the acoustic deviation as text that varies accordingly . In the method, the text to be written is written or written down on the basis of the spoken language by the language text translation system, taking into account the acoustic deviation.

In one embodiment, a tolerance value can be provided and/or taken into account for the acoustic deviation of the at least one acoustic parameter from the reference value, with the acoustic deviation from the reference value being compared with the tolerance value. If the deviation is at most as large as the tolerance value, the text will not be varied in writing. If the acoustic deviation is greater than the tolerance value, the text is varied in writing according to the acoustic deviation.

In the process it is u. a. possible that a duration of the at least one syllable of the at least one letter, i. H. of the vowel and/or the consonant that forms this syllable is determined as an acoustic parameter. The duration of the syllable is taken into account as an acoustic parameter. In addition, a deviation in the duration of the syllable from a reference duration provided for this purpose is taken into account as a reference value. It is possible here for the duration of the spoken syllable to deviate from the reference duration provided for this purpose by more than one tolerance duration as a tolerance value. In this case, to vary the written text, the number of letters is increased if the duration is greater than the reference duration plus a first tolerance duration, or decreased if the duration is shorter than the reference duration minus a second tolerance duration. If the duration deviates from the reference duration by no more than a respective tolerance duration, the text provided for the written representation of the syllable remains the same according to the specification.

It is also possible that a frequency of the at least one syllable comprising the at least one letter and thus a corresponding pitch of the at least one syllable is determined. In this case, a deviation in the frequency of the spoken syllable is taken into account as an acoustic parameter. Furthermore, a deviation in the frequency of the spoken syllable from a reference frequency provided for this purpose is taken into account as a reference value. It is possible here for the frequency of the spoken syllable to deviate by more than one tolerance frequency as a tolerance value from the reference frequency provided for this purpose. The frequency can be higher than the reference frequency plus a first tolerance frequency, in which case the number of letters to represent the syllable is reduced. Alternatively, it is possible that the frequency is lower than the reference frequency provided for this purpose minus a second tolerance frequency, in which case the number of letters is increased.

It is also possible for a volume or a level of the at least one spoken syllable to be determined. A volume is taken into account as an acoustic parameter. In addition, a deviation of the volume from a reference volume provided for this purpose is taken into account as a reference value. It is possible here for the volume of the spoken syllable to deviate by more than a tolerance volume as a tolerance value from the reference volume provided for this purpose. It is possible here that at least one capital letter is used to represent the respective syllable if the volume of the spoken syllable is louder than the reference volume plus a first tolerance volume. Alternatively, at least one small letter is used to represent the syllable as text if the volume should be lower than the reference volume minus a second tolerance volume.

Typically, the at least one currently spoken syllable is part of an oral communication and/or speech by the person. The reference value for the at least one acoustic parameter is determined from the message or speech as spoken language, usually always, against which the at least one spoken syllable is then later compared, which alternatively or additionally means that the at least one spoken syllable is compared with the reference value, on the basis of such a comparison for the at least one currently spoken syllable of the speech text translation system the acoustic deviation from the reference value for the acoustic parameter provided in each case is detected, this at least one syllable from the speech text translation system according to the acoustic Deviation is provided and/or illustrated as text varies accordingly in writing.

In a further refinement, it is possible for the person's speech to be recorded in advance or earlier and for the reference value for the acoustic parameter for a respective syllable to be based on the previously recorded and/or recorded speech of this person which includes the respective syllable. It is conceivable here that training is carried out for the voice text translation system, in which this voice text translation system is automatically taught or learned to the language or pronunciation of the respective person, usually using artificial intelligence. On the basis of such training and/or learning, it is also possible to recognize syllables and to provide them as known and to assign the written specification provided for this purpose to a respective syllable according to the reference value. It is also possible here for previously unknown syllables to be learned by the language text translation system, with the reference value for the at least one acoustic parameter described above being determined as a specification for a new or previously unknown syllable.

Furthermore, it is alternatively or additionally possible for the language text translation system to record, record and take into account a language currently spoken by the person and thus the corresponding current speech or message over a time interval that can span a few minutes. Such a person's speech comprises a plurality of syllables. For this speech and the currently spoken language of the person, a mean value, i. H. a mean value for a duration for all spoken syllables, a frequency for all spoken syllables and/or a volume for all spoken syllables is determined, with a respective mean value being taken into account as a reference value for the respective acoustic parameter for at least one syllable currently spoken during the speech. It is possible to take into account fluctuations in the speech in terms of duration, frequency and/or volume with regard to the total number of syllables spoken within the speech and to derive the text from this speech or to record it in writing according to the fluctuations and thus according to corresponding deviations from to vary the respective mean values as reference values. It is thus possible to define and/or determine the reference value for the at least one acoustic parameter while speaking.

It is provided that the speech spoken by the person is picked up by a microphone. In addition, a distance from the person's mouth to this microphone, with which the speech is recorded, is determined. This remover When comparing the at least one spoken syllable with the reference value of the at least one acoustic parameter provided for this purpose, in particular a deviation from the reference volume is taken into account. In order to assess the volume, a corresponding distance correction is made by the voice text translation system, since the volume with which the spoken language of the person is finally recorded by the microphone depends on the distance or distance of the person's mouth from the microphone.

The language text translation system according to the invention is designed to recognize speech and to translate it or convert it into written text or writing. In this case, this voice text translation system has a microphone and a computing unit. It is taken into account here that each syllable of the language can be represented or represented in writing or as text by at least one letter according to a specification. In addition, the language is spoken by a specific person to whom the language text translation system is usually taught and/or trained. The processing unit is designed to provide the respective reference value for the at least one acoustic parameter for each commonly known syllable with regard to its pronunciation and to assign it to the written or textual specification. The microphone is designed to record at least one syllable currently being spoken by the person. The processing unit is also designed to compare the at least one recorded syllable with the reference value for the at least one acoustic parameter provided for this purpose and, in the event that the processing unit detects an acoustic deviation from the acoustic reference value provided for this purpose for the at least one syllable currently spoken detected, this at least one syllable according to the detected acoustic deviation as text varies in writing.

This speech text translation system is designed to carry out at least one embodiment of the method described above, usually with software support, with artificial intelligence being used by the speech text translation system to detect and/or recognize the acoustic deviation and to vary the text on the basis of the acoustic deviation.

In one embodiment of the method, the speech text translation system is designed to take into account and/or provide a tolerance value for the acoustic deviation from the respectively provided acoustic reference value for the at least one acoustic parameter, with the at least one syllable being interpreted as text by the speech text translation system according to the acoustic deviation is varied in writing if the acoustic deviation from the reference value is greater than the tolerance value to be taken into account.

In a further refinement, the language text translation system can have at least one text output module, for example a display field, for displaying the text on the basis of the language recorded in each case and/or a text editor. Furthermore, the speech text translation system can also have a camera as an optical sensor, with which it is possible to detect the distance or the distance between the person's mouth and the microphone, the processing unit being designed to perform the distance correction described above on the basis of the detected distance to perform.

The method and the language text translation system are i.a. an emotionalization and individualization of speech-to-text messages, e.g. for social media and/or for news, is possible.

With the speech text translation system or a corresponding speech recognition system of naturally spoken language, certain criteria or properties of the language, e.g. B. a speech volume and a duration of a sustained vowel of a syllable taken into account.

In a further refinement of the method, it is provided that the acoustic deviation is to be and/or is displayed by a respective formatting as a variation of the text, with the acoustic deviation from the speaking person being indicated by a respective code word. According to the definition, the acoustic deviation of the at least one acoustic parameter from the reference value is indicated and/or signaled by the code word, the code word defining that the acoustic deviation is greater than the tolerance value. In this case, the text is formatted and/or varied in writing, e.g. with regard to the spelling of letters, with a definable or to be defined and/or defined formatting of the text, e.g. is deactivated by another code word that unformats the text. It is possible to set a mode for the written text via or with the formatting. In this case, such formatting can also be used, for example, to introduce a new style of language, such as an "ironic spelling", such as is already used in social media in some circles today and continues to be used in pop culture as so-called "SpongeBob meme language". In doing so, e.g. For example, if irony spelling is activated, the spoken message "Caution, that's irony" is displayed as text with "WARNING THAT IS IRONY". This feature could be activated, for example, by naming the code word "irony mode" beforehand. Accordingly, when the irony mode is implemented, the written text is formatted based on the spoken language or the at least one spoken syllable in such a way that the letters are written alternately in upper and lower case, with a small letter on each large letter and a small letter on each a capital letter follows. As a further formatting and/or as a further mode, the written text can be written in phonetic transcription based on the spoken language according to a phonetic transcription mode. It is also possible for the text to be intentionally misspelled in a misspelling mode based on the spoken language. In addition, a cursive mode is also possible, in which the text is written in italics based on the spoken language. Provision is usually made for a particular mode and the code word assigned to it to be stored in a database with the associated functions, ie a particular formatting, a particular language, etc., with the associated function being stored, for example, when the code word is entered verbally or formatting is retrieved from the database.

A dialect mode is also conceivable, according to which the text is written in a selectable dialect, for example Bavarian, based on the spoken language. Alternatively or additionally, a language text translation mode can be activated or activated based on a first language currently spoken by the person, e.g . In this case, the acoustic deviation of the at least one acoustic parameter from the reference value, which is greater than the tolerance value, is indicated and/or signaled by this code word by definition. Translations between languages of different countries and/or peoples are possible. Such a translation between different languages can be activated by a translation or translation mode. In this case, it is provided that the acoustic deviation is to be represented or is represented by a language translation as a variation of the text, with the at least one syllable being spoken in a first spoken language and translated into a second language by the language text translation system and translated into the second language is written as text writing varied.

In one embodiment, it is possible for the speech text translation system to determine a reference volume level as a reference value and to translate upper-case letters in the text if this reference volume level is exceeded. In this case, the reference value for an acoustic parameter described above can be learned by the speech text translation system for the respective person or a corresponding customer. Here, an adaptation to a quiet or a loud speech or language type of the respective person is possible. The same is correspondingly also possible for the frequency and/or duration of the language spoken by the respective person.

In one embodiment, the method takes into account that the volume of the spoken language also depends on the distance between the person's mouth and the microphone, which is why the corresponding distance correction is provided. In an application of the method and the voice text translation system in a motor vehicle or in a car, the position of the microphone relative to the head of the person as a passenger or occupant is relatively fixed.

In an embodiment of the method, a holding of vowels is also detected and recorded in a multiple version as writing or text. This is possible by detecting and/or measuring the duration of each syllable comprising that vowel, lengthening a number of letters representing each vowel in the text to be written to represent each syllable in writing.

With the method and the language text translation system, individualization and emotionalization of voice inputs for social media in spoken and written form is possible. Thus it is still u. a. possible to eliminate any misunderstandings due to otherwise misinterpreted text messages.

If the person says energetically: "WHY DO YOU WANT THAT?" and would also like to convey that this sentence was not spoken or said in the resting heart rate, this can be recorded via the recorded volume of the speech. In comparison to conventional voice recording, the method can thus be used to avoid the text that was entered orally as voice being changed to “Why do you want that?”.

According to another example, the person says "Haaaaaaallo?" and wants to express a completely different meaning than a mere greeting. Possibly it should be expressed that the addressee or addressee should report to the sender. At least a certain emphasis should be expressed. Such stress is detected by the length or duration of the spoken vowel "a" of the syllable. The speech is recorded differently than by a normal speech recognition unit, which only makes a "Hello" out of it.

The presented method and the presented language text translation system can be used in the age of increasing digitization and maintenance of social networks, since increasing individualization and emotionalization is important here. An application of the method and the language text translation system can also be used by a driver of a vehicle as a person, since the driver must primarily take on driving tasks when driving the vehicle and cannot write messages in a conventional way, for example typing. With the method and the language text translation system, the writing of texts is made easier for the driver.

In contrast to conventional speech input, the method can now also be used to transfer or translate speech into text, which goes beyond the usual standard didactics. In one implementation of the method, an algorithm, typically software-based, is used by the voice text translation system to translate spoken language into written text, with the ability to transform the person's speech into moody text rather than normal mood-neutral text. The method goes beyond conventional speech recognition measures, which are only designed to recognize dictated punctuation marks, such as e.g. B. an exclamation mark to be recognized as such. The method can thus be used to translate and/or convert a person's voice to articulate speech into a differentiated representation as text. With the method, an acoustic reference value for the volume or the level and/or a tone or an emphasis is determined for the language spoken by the person. If deviations from this acoustic reference value are detected that are greater than the tolerance value, it is possible, for example, to influence capitalization and punctuation of the text to be provided.

Taking into account the tone or emphasis, e.g. a voice pitch, of the spoken language, punctuation marks for the text to be written are automatically recognized by the language text translation system due to the emphasis and no longer have to be dictated, with the punctuation marks in the text being set accordingly. Furthermore, paragraphs with a specific speech pause length between two syllables in the spoken language can be recognized as such. Furthermore, expressions such as "hmmh" or "eh" are also recognized as such and written or written down in the text in order to convey not only a pure text content but also a "resonating" emotional state of the speaking person, e.g. approving, thoughtful, doubting, mocking or rejecting , to fix.

The language text translation system is designed to automatically learn when the method and/or the language text translation system is used by the person, e.g. by a driver, his or her language characteristics and/or way of speaking, e.g. using artificial intelligence. Furthermore, the voice text translation system is designed to get to know his or her individual way of speaking by means of targeted queries to the person, for example the driver. Thereby, the speech text translation system can learn a natural speech flow of the person. In this case, reference values for acoustic parameters for the language spoken regularly and/or individually by the person are learned and/or trained.

In a further refinement, the language text translation system recognizes whether the person is singing their language and thus their syllables. In this case, the written text is provided with special characters, e.g. with music notes that indicate a melody that has been sung.

It goes without saying that the features mentioned above and those still to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

• 1 zeigt in schematischer Darstellung ein Kraftfahrzeug, das eine Ausführungsform des erfindungsgemäßen Sprachtextübersetzungssystems zur Durchführung einer Ausführungsform des erfindungsgemäßen Verfahrens aufweist.

The invention is shown schematically on the basis of embodiments in the drawing and is described schematically and in detail with reference to the drawing.

• 1 shows a schematic representation of a motor vehicle which has an embodiment of the speech text translation system according to the invention for carrying out an embodiment of the method according to the invention.

In the 1 Schematically illustrated embodiment of the speech text translation system according to the invention is arranged in a motor vehicle 2 and has a microphone 6, a computing unit 8, a text output module 10 and a camera 12 as components. Also shows 1 that there is a person 4 in an interior of the motor vehicle 2 .

In the embodiment of the method according to the invention, it is provided that the language text translation system is or has been taught and/or trained to a common language or pronunciation of syllables by the person 4 . Taking this into account, at least one acoustic parameter is provided for each syllable of the speech of the person 4 with regard to a duration of a respective syllable, a loudness of a respective syllable and a frequency of a respective syllable. In addition, the respective syllable is assigned a written specification for representing the syllable as text, with a corresponding assignment of a syllable to its written specification being trained and/or learned by the language text translation system. In this case, a reference value is provided and/or taken into account for each acoustic parameter, which is assigned to the specification for the display as text. In addition, a tolerance value is provided for each acoustic parameter, which is taken into account for assessing the deviation from the reference value.

In the method, the currently spoken speech of the person 4 is recorded by the microphone 6 and analyzed and/or evaluated by the computing unit 8 using an algorithm with software support, each syllable currently being spoken and recorded by the microphone 6 being matched with the reference value provided for this purpose for the at least an acoustic parameter is compared. If the computing unit 8 detects an acoustic deviation from the reference value of the acoustic parameter provided for this purpose for at least one currently spoken syllable that is greater than the tolerance value, this at least one syllable is written as text by the text output module 10 according to the acoustic deviation varied and/or recorded.

The camera 12 detects the distance between the speaking mouth of the person 4 and the microphone 6 in order to be able to compare the volume of the spoken language and thus of spoken syllables, taking this distance or a corresponding distance into account, with the acoustic reference value provided for this purpose.

Reference List

2

vehicle

4

person

6

microphone

8th

unit of account

10

text output module

12

camera

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• EP 2385520 A2 [0003]
• EP 2391105 A1 [0004]

Claims (10)

A method of recognizing speech and translating it into written text using a speech text translation system, each syllable of the language being written by at least one letter according to a specification, the language being spoken by a person (4), each syllable being as to its pronunciation at least one acoustic parameter is provided, which is assigned to the written specification, with at least one syllable currently being spoken by the person (4) being compared with a reference value for the at least one acoustic parameter provided for this purpose, with the case that for the at least one currently spoken syllable of the voice text translation system, an acoustic deviation from the respectively provided acoustic reference value for the at least one acoustic parameter is detected, this at least one syllable is varied by the voice text translation system according to the acoustic deviation as text in writing i.e. procedure after claim 1 , in which a duration of the at least one syllable is determined and taken into account as an acoustic parameter, a deviation of the duration from a reference duration provided for this purpose being taken into account. procedure after claim 1 or 2 , in which a frequency of the at least one syllable is determined and taken into account as an acoustic parameter, a deviation of the frequency from a reference frequency provided for this purpose being taken into account. Method according to one of the preceding claims, in which a volume of the at least one syllable is determined and taken into account as an acoustic parameter, a deviation of the volume from a reference volume provided for this purpose as a reference value being taken into account. Method according to one of the preceding claims, in which the speech of the person (4) is detected in advance, the acoustic reference value for the at least one acoustic parameter for a respective syllable being based on an average of previously detected speech comprising the respective syllable. Method according to one of the preceding claims, in which speech spoken by the person (4) is taken into account to determine the acoustic reference value for the at least one acoustic parameter, a mean value being determined for all spoken syllables for the spoken language and used as the respective acoustic reference value for the at least one acoustic parameter for the at least one syllable is taken into account. Method according to one of the preceding claims, in which the acoustic deviation from the respectively provided acoustic reference value for the at least one acoustic parameter is compared with a tolerance value, the at least one syllable being varied as text in writing by the speech text translation system according to the acoustic deviation if the acoustic deviation from the reference value is greater than the tolerance value. Method according to one of the preceding claims, in which the acoustic deviation is to be represented by a respective formatting as a variation of the text, the acoustic deviation being indicated by a respective code word, the respective formatting of the text being activated by the respective code word. Method according to one of the preceding claims, in which the acoustic deviation is to be represented by a language translation as a variation of the text, the at least one syllable being spoken in a first language and being varied by the language text translation system by a translation into a second language as text in writing. Speech text translation system for recognizing speech and translating it into written text, the speech text translation system being adapted to write each syllable of the speech by at least one letter according to a specification, the speech being spoken by a person (4), the speech text translation system to is designed to provide at least one acoustic parameter for each syllable with regard to its pronunciation and to assign it to the written specification, the voice text translation system being designed to assign at least one syllable currently spoken by the person (4) with a reference value for the at least one acoustic parameter provided for this purpose compare and in the event that the speech text translation system for the at least one currently spoken syllable of the speech text translation system an acoustic deviation from the respectively provided acoustic reference value for the at least one acoustic parameters is detected, the speech text translation system is adapted to vary this at least one syllable according to the acoustic deviation as text in writing.

Images