CN112820293A - Voice recognition method and related device - Google Patents

Abstract

The present application discloses a speech recognition method and a related device. The method includes: acquiring to-be-processed audio generated by at least one target person and physiological parameters of the target person; transcribing the to-be-processed audio into speech to obtain a speech transcription result ; Display the result of voice transcription, and prompt the physiological parameters of the target person when the physiological parameters of the target person meet the warning conditions. Through the technical solution provided by the present application, it can be realized that the user can directly and intuitively locate the position of the physiological parameter warning of the target person according to the prompt, and more intuitively display the physiological parameters that meet the warning conditions.

Description

A kind of speech recognition method and related device

technical field

The present application relates to the technical field of speech recognition, and in particular, to a speech recognition method and related apparatus.

Background technique

When making breakthroughs in the field of conversation, the experience of the interviewer is extremely high, and the interviewer needs to be able to accurately "observe words and colors", adjust the direction of the question in a timely manner according to the confession of the interviewee, point directly to acute problems, remove the false and preserve the truth, so as to Quickly defeat the psychological bottom line of the person being talked to. For staff who are not very experienced in work, if there is a set of auxiliary conversation analysis system, which can judge the authenticity and sensitivity of the confession of the conversation object, it will undoubtedly add a sharp tool to the work.

The existing solution needs to prepare fixed topics for the conversation process, and cannot provide early warning for open-ended questions. Although it can provide early warning prompts for abnormal emotional states, it cannot intuitively locate the specific conversation content. The speaker must interrupt the conversation process for playback analysis, or Judging the key points based on the experience and memory of the conversation can easily cause the interruption of the conversation, and it is impossible to review the response content that has generated warnings during the conversation without interruption, which affects the continuity of the conversation and the key timing of breakthroughs. Therefore, There is a need for a technical solution that can solve the above technical problems.

SUMMARY OF THE INVENTION

The main technical problem to be solved by this application is to provide a voice recognition method and related device, which can enable the user to directly and intuitively locate the position of the target person's physiological parameter warning according to the prompt, and more intuitively display the physiological parameters that meet the warning conditions.

In order to solve the above-mentioned technical problems, a technical solution adopted in this application is to provide a speech recognition method, the method comprising:

Obtain the audio to be processed generated by at least one target person and the physiological parameters of the target person;

The audio to be processed is transcribed into speech to obtain a transcribed result;

The voice transcription result is displayed, and the physiological parameters of the target person are prompted when the physiological parameters of the target person meet the warning condition.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a speech recognition device, the speech recognition device includes a processor, a memory and a communication interface coupled to the processor; wherein,

the communication interface is used to communicate with other electronic devices;

the memory is used to store computer programs;

The processor is adapted to run the computer program to perform the method of speech recognition as described above.

In order to solve the above technical problem, another technical solution adopted in this application is to provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program that can be run by a processor, and the computer program is used to realize The method of speech recognition as described above.

The beneficial effects of the present application are: different from the situation in the prior art, the technical solution provided by the present application, by acquiring the audio to be processed generated by at least one target person and the physiological parameters of the target person in the audio to be processed, and then The to-be-processed audio is transcribed according to the set speech transcription rules, and then a speech transcription result is obtained, and the speech transcription result is displayed. Wherein, when the voice transcription result is displayed, the physiological parameters of the target person will be prompted when the physiological parameters of the target person meet the warning condition. That is, in the technical solution provided by the present application, by analyzing the physiological parameters of the target person in the audio to be processed, early warning can be realized for the physiological parameters of the target person who answers the open-ended questions; , prompts the physiological parameters of the target person, and more intuitively displays the physiological parameters that meet the warning conditions, so that the user can directly and intuitively locate the position of the target person's physiological parameter warning according to the prompt, thereby improving speech recognition and more adaptable to open question and answer. needs for speech recognition.

Description of drawings

1 is a schematic flowchart of an embodiment of a speech recognition method of the present application;

2 is a schematic flowchart of another embodiment of a speech recognition method of the present application;

3 is a schematic flowchart of another embodiment of a speech recognition method of the present application;

FIG. 4 is a schematic flowchart of still another embodiment of a speech recognition method of the present application;

5 is a schematic flowchart of another embodiment of a speech recognition method of the present application;

6 is a schematic flowchart of another embodiment of a speech recognition method of the present application;

FIG. 7 is a schematic structural diagram of an embodiment of a speech recognition device according to the present application;

FIG. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the description of the present application, "a plurality of" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an embodiment of a speech recognition method of the present application. In the current embodiment, the method provided by this application includes:

S110: Acquire the audio to be processed generated by at least one target person and the physiological parameters of the target person.

When the speech recognition device performs speech recognition, the to-be-processed audio generated by at least one target person is first acquired. Wherein, the audio to be processed includes audio data generated by at least one target person and needs to be recognized by speech. The audio data includes voice data and video data, that is, the to-be-processed audio data includes the voice generated by the target person and the video obtained by recording the target person when the target person generates the above-mentioned voice, and the target person is a person who needs voice recognition.

Among them, it should be noted that the audio data to be processed includes audio data of two or more characters, and the target characters are included in the audio to be processed and need to be subjected to speech recognition and physiological parameter analysis. The audio data to be processed The number of characters included in is greater than or equal to the number of target characters included in it. In one embodiment, a target person is included in the audio to be processed. In another embodiment, speech recognition and physiological parameter analysis may be performed simultaneously on two characters in the audio to be processed, that is, the audio to be processed may include two or more target characters, which is not specifically limited here. The actual setting shall prevail.

In the current embodiment, the audio to be processed may be acquired by an external module connected to the speech recognition device that collects and obtains the target person in real time, and is input to the speech recognition device for speech recognition. Wherein, the external module for acquiring the audio to be processed includes at least a microphone and a camera. Further, the camera includes a USB camera. It can be understood that, in other embodiments, the external module for acquiring the audio to be processed may also include other types of modules, which are not listed one by one here.

In another embodiment, the audio to be processed may also be pre-acquired audio data that is not collected in real time. Specifically, the audio to be processed may be existing audio data directly input by other devices, or may be audio data directly acquired by a speech recognition device through a network.

The to-be-processed audio is acquired by using a microphone array. Further, when the audio to be processed includes voice data and video data, the voice data in the audio to be processed is acquired by a microphone array, and the video data in the audio to be processed is acquired synchronously by a camera.

After acquiring the audio to be processed generated by the at least one target person, the physiological parameters of the target person are further acquired from the audio to be processed. Wherein, the physiological parameters of the target person include at least one physiological parameter of the target person's stress value, heart rate, blood oxygen, respiratory rate, blood pressure, and heart rate variability. It can be understood that, in other embodiments, the physiological parameters of the target person may also include other types of parameters, which are not listed one by one here.

Further, in an embodiment, the physiological parameters of the target person are obtained by analyzing the audio to be processed by using the physiological parameter obtaining module.

In another embodiment, other acquisition devices can also be assisted to acquire the physiological parameters of the target person. Among them, other collection devices include: bracelets.

Further, in an embodiment, when the audio to be processed is acquired in real time, the audio to be processed is analyzed in real time to obtain the physiological parameters of the target person. Further, the to-be-processed audio is analyzed to obtain the physiological parameters of the target person at each collection moment.

Further, in another embodiment, when the audio to be processed is acquired in real time, the processed audio may be analyzed according to a preset time period to obtain the physiological parameters of the target person within the preset time period. For example, according to the preset, when the audio to be processed is acquired in real time, according to the preset period of 10s, the latest 10s of the collected audio to be processed can be analyzed, and then the target person in the latest 10s can be obtained. Physiological parameters in . For example, since the real-time collection of the audio to be processed is started, according to the settings, after the audio to be processed for the first 10s is collected, the audio to be processed for the first 10s collected before is analyzed to obtain the target character in the first audio. Physiological parameters in 10s of audio to be processed, and so on. In the current embodiment, by analyzing the audio to be processed according to a preset time period to obtain the physiological parameters of the target person, it is possible to obtain the physiological parameters of the target person more accurately.

Further, when the audio to be processed is not acquired in real time, the corresponding audio to be processed may be analyzed according to the set analysis rules, and then the physiological parameters of the target person at each collection moment are acquired. For example, according to the preset, every two adjacent collection moments are separated by 1s. Correspondingly, set 1s, 2s, 3s, 4s, 5s... , 2s, 3s, 4s, 5s...ns and other physiological parameters at each acquisition time.

S120: Perform voice transcription on the audio to be processed to obtain a voice transcription result.

After the audio to be processed and the physiological parameters of the target person are acquired, the audio to be processed is further transcribed into speech, and a corresponding speech transcribed result is obtained. Among them, the speech transcription is to recognize and output the text included in the audio to be processed, and the speech transcription result is text data.

Further, in the current embodiment, step S120 is to recognize all the texts included in the audio to be processed, and output it as a result of speech transcription.

Further, if the audio to be processed is collected in real time, correspondingly, the collected audio to be processed can also be transcribed in real time.

In another embodiment, a set cache period may also be reserved for collecting to-be-processed audio for a certain period of time, and then combined with the to-be-processed audio within the set cache period to perform voice transcription, thereby obtaining a voice transcription result.

Further, the above-mentioned step S120 performs voice transcription on the audio to be processed to obtain a voice transcription result, further comprising: segmenting the audio to be processed, and performing voice transcription on several segments of the audio to be processed obtained from the segmentation to obtain a number of Transcribe text segments as a result of speech transcription.

Further, segmenting the audio to be processed in the above steps includes at least one of the following ① to ③:

①. Segment the audio belonging to different characters in the audio to be processed. Specifically, audios belonging to different characters in the audio to be processed may be segmented according to the timbre. Alternatively, when the audio to be processed includes only two characters and they are located in different orientations of different microphone arrays, the audio belonging to different characters can be segmented according to the orientation of each part of the audio in the audio to be processed.

②. If there is a preset pause in the first audio belonging to the same person in the audio to be processed, the first audio is segmented according to the preset pause. The preset pause is a speech pause that exceeds a preset time, and the preset time is a preset time, which is not particularly limited here, and the first audio is segmented audio that belongs to the same character.

③. If the number of words of the second audio belonging to the same person in the audio to be processed is greater than the preset number of words, the second audio is segmented. Wherein, the second audio is the audio obtained by segmentation and belonging to the same character, and the number of words is greater than the set number of words, and the preset number of words can be set according to actual needs, which is not limited here.

Further, when an embodiment includes the above ① to ③ at the same time, correspondingly, the above ① to ③ may be executed in sequence. That is, the audios belonging to different characters in the audio to be processed are firstly segmented, and then it is determined whether the first audio belonging to the same character has a preset pause, and if so, the first audio is segmented again according to the preset pause; then It is further determined whether the number of words of the second audio belonging to the same person is greater than the preset number of words, and if so, the second audio is further segmented.

Further, it should be noted here that the execution order of acquiring the physiological parameters of the target person included in the audio to be processed and performing voice transcription of the audio to be processed is not limited here. The steps may be executed simultaneously, or may be executed in succession one after the other, depending on the actual setting.

S130: Display the voice transcription result, and prompt the physiological parameters of the target person when the physiological parameters of the target person meet the warning condition.

After voice transcription is performed on the audio to be processed, and a voice transcription result is obtained, the voice transcription result is further displayed. If it is judged that the physiological parameters of the target person meet the warning conditions, the physiological parameters that meet the warning conditions and the corresponding voice transcription results will be superimposed and output when the voice transcription result is displayed, so as to realize the prompting of the physiological parameters of the target person. .

The speech transcription result includes several transcribed text segments, and the several transcribed text segments are obtained by transcribing several audio parts in the audio to be processed. Correspondingly, the displaying of the speech transcription result in the above step S130 further includes: displaying several transcribed text segments.

Further, when voice transcription of the audio to be processed is performed to obtain a voice transcription result, text segments corresponding to different characters will be output and displayed in the form of dialogues using shading of different colors. For example, when the audio to be processed includes the audio data of two persons, A and B, correspondingly, when the audio to be processed is transcribed, shading of different colors will be used to output the text segments corresponding to each of A and B. For example, using The text segment corresponding to A is output in blue shading, and the text segment corresponding to B is output in yellow.

In the current embodiment, the pre-warning condition is: the values collected by the physiological parameter at the first number of consecutive collection moments are all greater than or equal to the first threshold. The first number is a preset number of collection moments used to determine whether the physiological parameter meets the warning condition, and the first threshold is a preset comparison value used to determine whether the physiological parameter meets the warning condition. The specific type of parameter determines the specific value of the first threshold.

Specifically, since the types of parameters included in the physiological parameters are different, the corresponding first threshold may not be a value. For example, when the physiological parameter includes multiple different types (types) of parameters, the corresponding first threshold value may also include comparison values corresponding to multiple different types of parameters, for example, when the physiological parameter includes stress value and heart rate, Correspondingly, the first threshold value includes a first comparison value set corresponding to the stress value and a second comparison value set corresponding to the heart rate, wherein the first comparison value is a stress value used for judging the target person. The second comparison value is a value used to judge whether the heart rate of the target person is too fast, that is, whether the heart rate of the target person meets the warning condition.

In another embodiment, the early warning condition may also be: among the values collected by the physiological parameter at the first consecutive collection moments, there are a second number of values greater than or equal to the second threshold. The second number is a preset comparison value of the number of collection moments that meet the warning condition, the second number is less than or equal to the first number, and the second threshold is a comparison value used to determine whether the physiological parameter meets the warning condition.

Further, in an embodiment, the second threshold may be equal to the above-mentioned first threshold, which is not specifically limited herein.

In the embodiment corresponding to FIG. 1 of the present application, the audio to be processed generated by at least one target person and the physiological parameters of the target person in the audio to be processed are obtained, and then the audio to be processed is transcribed according to the set voice transcription rules. Phonetic transcription, and then get the phonetic transcription result, and display the phonetic transcription result. Wherein, when the voice transcription result is displayed, the physiological parameters of the target person satisfying the warning condition and the corresponding voice transcription result will be superimposed and displayed to prompt the physiological parameters of the target person. That is, in the technical solution provided by the present application, by analyzing the physiological parameters of the target person in the audio to be processed, early warning can be realized for the physiological parameters of the target person who answers the open-ended questions; , prompting the physiological parameters of the target person, so that the user can directly and intuitively locate the position of the target person's physiological parameter warning according to the prompt, thereby improving the voice recognition and more suitable for the needs of voice recognition in open question and answer.

Please refer to FIG. 2 , which is a schematic flowchart of another embodiment of a speech recognition method of the present application. In the current embodiment, the method provided by this application includes:

S201: Acquire audio to be processed generated by at least one target person and physiological parameters of the target person.

Wherein, in the current embodiment, the audio to be processed is acquired by using a microphone array, and the microphone array includes a certain number of acoustic sensors, which are respectively used to collect the audio to be processed generated by characters in different directions.

Specifically, when the audio to be processed includes voice data and video data, the microphone array is used to collect voice data, and when acquiring the voice data generated by the target person, the orientation of the target person relative to the microphone array is stationary, Therefore, in the current embodiment, the position of the audio part of the audio to be processed can be directly determined based on the microphone array.

Therefore, in the current embodiment, before the audio to be processed is segmented, the method provided by the present application further includes steps S202 to S203.

S202: Determine, based on the microphone array, the position where each part of the audio to be processed is located.

After acquiring the to-be-processed audio generated by at least one target person and the physiological parameters of the target person, and the voice data in the to-be-processed audio is collected and acquired by the microphone array, then further determine the orientation of each part of the audio in the to-be-processed audio based on the microphone array . Wherein, the orientation of each part of the audio is relative to the orientation of the microphone array.

Wherein, in the current embodiment, each partial audio is at least a piece of audio corresponding to different characters in the audio to be processed. Furthermore, since the voices between multiple different characters are in the form of dialogues, the audio corresponding to each character may include at least one partial audio. It should be noted that when a certain audio to be processed is generated by a conversation between two people, A and B, the audio to be processed correspondingly includes a plurality of partial audios generated by A, as well as a plurality of audios generated by B. audio.

Each part of the audio further includes each part of the voice data, and the above step S202 further includes: determining the position where each part of the voice data in the audio to be processed is located based on the microphone array.

In another embodiment, the audio to be processed may also be divided in a fixed manner to obtain each part of the audio. For example, the audio of a continuously set number of frames may be regarded as a partial audio, or each partial audio may be obtained by dividing according to the timbre or the pause time, which is specifically set according to the actual setting, which is not limited here.

Further, when the audio to be processed includes voice data, each part of the audio includes each part of the voice, and correspondingly, the voice data included in the to-be-processed audio is divided according to a set time interval to obtain each part of the voice. If preset, the speech of every 1s is divided into a partial speech. Alternatively, the voices with different timbres may be divided into one partial voice according to the setting. Alternatively, the voice data may be divided into a plurality of partial voices according to the setting according to the pause time included in the voice data.

S203: Determine the person to which each part of the audio belongs based on the position of each part of the audio.

After determining the location of each part of the audio in the audio to be processed based on the microphone array, further determine the person to which each part of the audio belongs based on the location of each part of the audio, and after determining the person to which each part of the audio belongs, further execute the following Step S204.

In another embodiment, when the audio to be processed includes voice data, each part of the audio includes each part of the voice, that is, when it is determined in step S203 that the position of each part of the voice is located, then step S203 is based on the location of each part of the voice. Orientation, to determine the person to which each part of the voice belongs.

S204: Segment the audio to be processed, and perform voice transcription on several segments of the audio to be processed obtained from the segmentation to obtain a number of transcribed text segments, which are used as a result of speech transcription. In the current embodiment, a part of the audio for which the person to which the person belongs is determined is segmented, and then the segmented audio to be processed is voice transcribed to obtain several transcribed text segments.

Further, in some embodiments, when the audio to be processed includes voice data and video data, the step S204 may be to divide the voice data to obtain several pieces of voice data, and then only perform the segmentation on the several pieces of voice data obtained by segmentation. Speech transcription, and then several transcribed text segments are obtained. It should be noted that, in these embodiments, the video data is not divided, and the video data is used to obtain the physiological parameters of the target person.

S205: Display the voice transcription result, and prompt the physiological parameters of the target person when the physiological parameters of the target person meet the warning condition.

Step S204 is the same as the steps in the embodiment of the above-mentioned step S120, and step S205 is the same as the above-mentioned step S130. For details, please refer to the description of the corresponding part of FIG. 1 above, which will not be described in detail here.

In the embodiment shown in FIG. 2 , by using the microphone array to collect the audio to be processed, different characters can be better distinguished, and the early warning for the physiological parameters of different characters is more targeted. For example, in the question answering system used in the field of interrogation, by using the microphone array to collect the audio to be processed, it can better realize the early warning of physiological parameters only in the segment answered by the respondent.

At the same time, using the microphone array can achieve better and more accurate segmentation of the audio to be processed, and then realize the physiological parameters that meet the warning conditions in units of sentences.

Please refer to FIG. 3 , which is a schematic flowchart of another embodiment of speech recognition according to the present application.

In the current embodiment, the voice transcription result includes several transcribed text segments. Correspondingly, in step S120, the audio to be processed is voice-transcribed, and after the voice transcription result is obtained, the method provided by the present application further includes: displaying several Transcribe text segments.

Correspondingly, in the current embodiment, in the above step S130, when the physiological parameters of the target person meet the warning condition, prompting the physiological parameters of the target person further includes:

S301: In the case that the physiological parameter satisfies the warning condition, determine the warning time based on the collection time of the physiological parameter, and determine the warning parameter value based on the value of the physiological parameter.

Further, the early warning time is a time point. The warning time may be determined based on the collection moments corresponding to a plurality of physiological parameters that satisfy the warning condition.

In one embodiment, the midpoint time of the collection time corresponding to the physiological parameter satisfying the warning condition may be determined as the warning time. For example, the collection times of the physiological parameters that meet the warning conditions are 6s, 7s, 8s, 9, and 10s, respectively. Correspondingly, the midpoint time 8s of the collection times corresponding to the physiological parameters that meet the warning conditions is determined as the warning time.

In another embodiment, it can also be understood as outputting the acquisition time corresponding to the parameter in the central position among the continuous physiological parameters that meet the warning condition as the warning time, for example, the values of the physiological parameters that meet the warning condition are sequentially They are: 32, 33, 49, 31, and 46, respectively, then output the collection time corresponding to 49 as the early warning time.

Further, the warning time may also be a time period. In one embodiment, the time range included in the collection moment corresponding to the physiological parameter satisfying the warning condition may be determined as the warning time. For example, the collection times of the physiological parameters that meet the warning conditions are 11s, 13s, 15s, 17s, 19s, and 21s, respectively. According to the setting, the time range included between 11s and 21s can be determined as the warning time.

In another embodiment, a part of the time range included in the collection moment corresponding to the physiological parameter satisfying the warning condition may also be determined as the warning time. Following the above example, the time range between 15s and 17s may also be determined as the early warning time according to the preset.

Wherein, the early warning parameter value is determined based on the physiological parameters at each collection moment satisfying the early warning condition.

In one embodiment, the early warning parameter value is the average value of the physiological parameters at each collection moment that meets the early warning condition. When the pre-warning condition is: when the values collected by the physiological parameters at the first consecutive number of collection moments are all greater than or equal to the first threshold, if it is determined that the physiological parameters of the target person satisfy the pre-warning condition, the corresponding first consecutive number of The average value of the values of the physiological parameters is collected at each collection time, and the obtained average value is output as the warning parameter value. Correspondingly, if the physiological parameter includes multiple types of parameter values, the mean value of each type of parameter collected at the first number of consecutive collection moments is obtained respectively, and the obtained mean value of each type is correspondingly output as each type of physiological parameter. The warning parameter value corresponding to the parameter.

When the pre-warning condition is: among the values collected by the physiological parameters at the first consecutive number of acquisition moments, when there is a second number of values greater than or equal to the second threshold, if it is determined that the physiological parameters of the target person meet the pre-warning conditions, the corresponding The average value of the physiological parameters collected at the first number of collection moments is obtained, and the obtained average value is output as the warning parameter value.

In another embodiment, the early warning parameter value may also be an intermediate value of the physiological parameter at each collection moment that satisfies the early warning condition. Correspondingly, in the current embodiment, the physiological parameters at each collection time that satisfy the warning condition are sorted, and then the intermediate value obtained by sorting is output as the warning parameter value.

Further, when the physiological parameter includes a variety of different types of parameters, different calculation methods can be set for the warning parameter values corresponding to the different types of parameters. For example, when the physiological parameters include stress value and heart rate, correspondingly, set the median value of the stress value at the collection time that meets the warning condition as the stress value warning parameter value, and obtain the average value of the heart rate that meets the warning condition as the heart rate Warning parameter value.

S302: Determine the transcribed text segment that matches the warning time as the target transcribed text segment.

After the early warning time is determined based on the collection time of the physiological parameter, and the early warning parameter value is determined based on the value of the physiological parameter, the transcribed text segment matching the early warning time is further determined as the target transcribed text segment. The target transcribed text is the finally determined transcribed text segment that is superimposed and displayed with the warning parameter value.

In one embodiment, it is first judged whether there is a transcribed text segment whose time range covers the warning time. If there is a transcribing text segment whose time range covers the warning time, the existing transcribing text segment is used as the target The transcribed text segment whose scope covers the warning time is output as the target transcribed text segment.

In another embodiment, if it is determined that there is no transcribed text segment with a time range covering the warning time, the transcribed text segment located after the early warning time and closest to the warning time is used as the target transcribed text segment, which is about to be located in the early warning time. The transcribed text segment after the time and immediately adjacent to the warning time is used as the target transcribed text segment.

S303: Display the warning parameter value within the preset area range of the target transcription text segment.

After the target transcription text segment is determined, the warning parameter value is displayed within the range of the preset area of the target transcription text segment. Wherein, the preset area range of the target transcribed text segment is a preset area that is close to the target transcribed text segment. For example, according to the preset area range, it is above or below the target transcribed text segment. Alternatively, the preset area range is set as the right side or the left side of the target transcribed text segment, which is based on the actual setting, which is not limited here.

Please refer to FIG. 4 . FIG. 4 is a schematic flowchart of still another embodiment of a speech recognition method of the present application. In the current embodiment, the method provided by this application includes:

S401: Acquire audio to be processed generated by at least one target person and physiological parameters of the target person. Step S401 is the same as the above-mentioned step S110, and reference may be made to the description of the corresponding part of S110 above.

S402: Determine, based on the microphone array, the location where each part of the audio to be processed is located.

S403: Determine the person to which each part of the audio belongs based on the position of each part of the audio. Steps S402 and S403 are the same as the above-mentioned steps S202 and S203 respectively, and details are not repeated here.

S404: Segment the audio to be processed, and perform voice transcription on several segments of the audio to be processed obtained from the segmentation to obtain several transcribed text segments, which are used as a result of the speech transcription. For the segmentation rule in step S404, reference may be made to the description in step S120 above, which will not be repeated here.

S405: Display several transcribed text segments.

After the audio to be processed is segmented, and several segments of the audio to be processed obtained from the segment are voice transcribed to obtain several transcribed text segments, the several transcribed text segments are further displayed. Further, in another embodiment, step S405 is executed synchronously with the following steps S406 to S409, and the specific execution sequence is not limited herein.

In the current embodiment, the pre-warning condition is that the values collected by the physiological parameters at multiple collection times meet the preset requirements, then in the current embodiment, determining the pre-warning time based on the collection time of the physiological parameters in the above step S301 further includes step S406 The central time among multiple collection moments is the early warning time.

S406: In the case that the physiological parameter satisfies the warning condition, the central time among the multiple collection moments is the warning time. Among them, the earliest collection time and the last collection time among the multiple collection times that satisfy the warning condition are added together, and then divided by 2 to obtain the central time.

In the current embodiment, determining the warning parameter value in the above step S301 and based on the value of the physiological parameter further includes step S407.

S407: Take the mean value of the values collected by the physiological parameter at multiple collection times as the warning parameter value. Among them, the warning parameter value is a parameter that is finally displayed superimposed with the target transcribed text segment and used to display the physiological state of the target person to the user. The values of the physiological parameters collected at the first consecutive collection moments satisfying the warning condition are added, and the mean value is obtained to obtain the warning parameter value.

S408: Determine the transcribed text segment that matches the warning time as the target transcribed text segment.

S409: Display the warning parameter value within the preset area range of the target transcription text segment. Steps S408 and S409 are the same as the above-mentioned steps S302 and S303, and for details, please refer to the descriptions in the corresponding parts above.

In one embodiment, if there are many kinds of physiological parameters of the target person, and some kinds of physiological parameters meet the warning condition, and some kinds of physiological parameters do not meet the warning condition, the above-mentioned warning parameter value is displayed in the target transcription text segment. Within the preset area, it further includes:

Display the warning parameter value corresponding to the physiological parameter that meets the warning condition within the preset area range of the target transcription text segment. For example, the physiological parameters also include the target person's stress value, heart rate, blood oxygen, respiratory rate, blood pressure, and heart rate variability, but after judgment, only the stress value and heart rate value meet the warning conditions. Correspondingly, only the stress value corresponds to The warning parameter value of , and the warning parameter value corresponding to the heart rate are displayed within the preset area of the target transcription text segment.

Further, the warning parameter value is superimposed on the target transcribed text segment for warning in units of sentences. In the current embodiment, by superimposing the warning parameter value on the target transcription text segment in units of sentences, the problem of accurate display during real-time warning can be better solved, which is more conducive to real-time positioning of sensitive problems and their early warning level.

In another embodiment, please refer to FIG. 5 , which is a schematic flowchart of another embodiment of a speech recognition method of the present application. If there are multiple physiological parameters of the target person, the above step S409 further includes steps S501 to S502.

S501: It is determined that the priority of the physiological parameter that meets the warning condition is higher than the priority of the physiological parameter that does not meet the warning condition.

That is, it is judged that there are some kinds of physiological parameters that meet the warning conditions, and some kinds of physiological parameters do not meet the warning conditions, then the priority of the physiological parameters that meet the warning conditions is set to be higher than the priority of the physiological parameters that do not meet the warning conditions. , and correspondingly display the physiological parameters that meet the warning conditions first or display the physiological parameters that meet the warning conditions first.

S502: Sort the warning parameter values corresponding to each physiological parameter according to the priority to form a parameter set to be displayed, and display the parameter set to be displayed within the preset area range of the target transcription text segment.

After step S501, the priority of the physiological parameter that meets the warning condition is set higher than the priority of the physiological parameter that does not meet the warning condition, then the warning parameter value corresponding to each physiological parameter is further sorted according to the priority to form a to-be-displayed parameter set, and then display the parameter set to be displayed in the preset area range of the target transcription text segment. The parameter set to be displayed is the parameters displayed superimposed with the target transcribed text segment.

In other embodiments, processing priorities may also be set for different types of physiological parameters in advance. For example, when the physiological parameter includes the stress value and heart rate of the target person, the priority of processing the stress value can be set higher than that of the heart rate. Correspondingly, it will first judge whether the physiological parameters of the target person meet the warning conditions based on the stress value. Similarly, if the stress value and heart rate meet the warning conditions at the same time, the corresponding stress value will be displayed first.

Further, in some embodiments, the display order of each physiological parameter is determined by further combining the ratio of the warning parameter value corresponding to each physiological parameter exceeding the comparison value. The physiological parameter that is about to exceed the comparison value by the largest proportion will be displayed first. Wherein, the ratio of exceeding the comparison value is the ratio between the difference obtained by subtracting the comparison value of the physiological parameter from the warning parameter value corresponding to the physiological parameter and the comparison value. Furthermore, the method provided by the present application further includes: magnifying and displaying the physiological parameter that exceeds the maximum proportion of the comparison value, so as to better prompt the user to pay attention to the physiological parameter first.

Further, in another embodiment, the warning level (warning type) can also be preset, and the warning level and the warning parameter value are continuously compared, and only the physiological parameter that meets the warning condition and has the highest warning level is displayed.

Please refer to FIG. 6, which is a schematic flowchart of another embodiment of a speech recognition method of the present application. The methods provided in this application include:

S601: Acquire the audio to be processed generated by at least one target person and the physiological parameters of the target person.

S602: Perform voice transcription on the audio to be processed to obtain a voice transcription result.

S603: Display the voice transcription result, and prompt the physiological parameters of the target person when the physiological parameters of the target person meet the warning condition.

Steps S601 to S603 in the current embodiment are the same as the above-mentioned steps S110 and S130. For details, please refer to the descriptions in the corresponding parts above.

In the current embodiment, the physiological parameter includes a stress value, the warning condition includes that the stress value meets a preset requirement, and the speech transcription result includes several transcribed text segments, and the several transcribed text segments are transcribed from several audio parts in the audio to be processed get.

In the current embodiment, the voice transcription result is displayed in the step, and when the physiological parameters of the target person meet the warning condition, after the physiological parameters of the target person are prompted, that is, the method provided by the present application further includes steps after step S603 S604 and step S605.

S604: Use the first transcribed text segment corresponding to the tension value meeting the preset requirement, the second transcribed text segment associated with the first transcribed text segment, and the pre-warning level corresponding to the tension value as a set of pre-warning information.

Wherein, the first transcribed text segment is a speech transcription result corresponding to when the target person's physiological parameters within the time corresponding to the speech segment meet the warning condition obtained by transcribing the target person's speech. The second transcribed text segment is the voice transcription result associated with the first transcribed text segment. For example, the second transcribed text segment is the voice transcribed result corresponding to the question asked by the questioner, and the corresponding first transcribed text segment Transcription of the speech produced for the answering character to answer the question posed by the questioning character.

S605: Generate an early warning report based on the early warning information.

After obtaining the warning information, further generate the warning report and output it. The early warning report may at least be in the form of a report. For example, in an embodiment, the audio to be processed is generated by both parties to the question and answer, and correspondingly, an early warning report as shown in the following table can be generated.

Further, when the early warning report is generated, the early warning information may be arranged according to the sequence of the early warning time, and the early warning report is generated.

In another embodiment, the warning report may also be generated according to the severity program of the warning type, that is, arranging the warning information with the more serious warning type first. For example, the warning information of high stress can be displayed first, followed by the warning information of moderate stress, then the warning information of mild stress, and finally the warning information of abnormal heart rate.

In the current embodiment, an early warning report is generated based on the early warning information, and the early warning question and answer pairs are displayed in the early warning report, and at the same time, sorting according to the warning level (alarm type) and threshold is further supported, which is more conducive to the analysis and summary of the audio content to be processed. , pointing directly to the key problem points, which is convenient for classifying and backtracking the problem after the conversation is over or suspended, assisting efficient analysis to find suspicious problems, and providing report reference for detailed analysis.

The technical solution provided by this application was used in the field of conversation at that time, and the high-performance all-in-one workstation was used as the operating platform to run the speech recognition engine for speech transcription and the video analysis engine for sentiment analysis locally. The USB camera collects the face video data of the person being questioned, the voice is collected through the microphone array, and the question and answer roles are distinguished based on the collected voice data based on the microphone array. The specific speech recognition process can include the following: First, use The emotion recognition engine obtains the physiological parameters of the person being questioned by analyzing the video data collected in real time, wherein the physiological parameters include: stress value, heart rate and other physiological parameter information as mentioned above, which can be specified as 1 data per second. Form real-time collection.

Then, based on the voice transcription engine, the voice is transcribed in real time through the microphone array, and the voices of the two sides of the question and answer are distinguished by the orientation, and the audio segments to be processed are automatically segmented. The segmentation logic includes at least one of the following three points: ①. Segment audio belonging to different characters in the audio to be processed. ②. If there is a preset pause in the first audio belonging to the same person in the audio to be processed, the first audio is segmented according to the preset pause. ③. If the number of words of the second audio belonging to the same person in the audio to be processed is greater than the preset number of words, the second audio is segmented.

Emotion recognition alerts can include two types: heart rate alerts and stress alerts. For example, in one embodiment, the threshold of the heart rate warning is 120, and the stress warning is divided into three levels according to low, middle and high, and the thresholds are 30/45/70 respectively. When both the heart rate and the stress have warnings, they are superimposed on the target transcription text. The priority of the tension value warning of the segment is greater than the heart rate warning. When both of them generate an early warning, only the tension value warning can be superimposed according to the settings.

It should be noted that each time an early warning condition is met, an early warning is generated. The early warning condition for satisfying the heart rate warning is that one piece of heart rate data is sent every second, and the set number of consecutive collection times are greater than the threshold, which is called once the warning condition is met. On the contrary, the pre-warning condition is not satisfied. The center time of the warning is the warning time. The early warning of the tension value is that the tension value of a set number of consecutive collection moments is greater than the corresponding level threshold. In other embodiments, it can also be set that if the second set number or more of the first set number in a row is greater than the corresponding level threshold, it is determined that the early warning condition of the current alarm level is satisfied. For example, when the stress warning is divided into three levels according to low, medium and high, and the thresholds are 30/45/70, the first set number is 5, the second set number is 3, and the stress value of 5 consecutive points is obtained after collection. If they are 32/33/49/31/46 respectively, it is judged to be a mild early warning, the average early warning value is taken, then the level is a mild early warning, and the early warning time is the time when the tension value generates data 49.

Among them, in the early warning, it will further judge whether the warning time falls within the time interval of the transcribed text corresponding to the answerer's answering voice, and if it falls within the time interval, the transcribed text segment is an early warning event, such as an early warning. The time is 16:19:20, and the time interval of the transcribed text corresponding to the answerer’s answer is 16:19:13—16:19:25, then the system judges that this paragraph generates an alert event, alert type and corresponding alert Parameter values can be superimposed on the transcribed text.

It should be noted that, in addition to referring to the time interval of the transcribed text corresponding to the answering voice of the respondent, the target transcribed text may also be determined according to the above three segments. For example, when the roles are switched as described in ① above: only need to judge the matching according to the time interval of the transcribed text. When there is a preset pause as described in ②: when an early warning is generated during the pause, the superimposition of the early warning is still classified into the next transcribed text, and the early warning time is subject to the actual early warning time point. Or, when the content of the transcribed text is larger than the preset number of words: when the actual range of the warning is just at the critical point of segmenting by the preset number of words, the warning is classified into the transliterated text of the next paragraph.

Wherein, as described above, when multiple alarm events are generated within a time range corresponding to one transcribed text, they are finally superimposed and displayed according to the set priority. For example, according to the settings, the stress value will be superimposed on the target transcribed text segment first, and the stress warning level will be superimposed in the order of high, middle and low priority. For the same level of stress value, the data with the larger warning value will be superimposed.

Please refer to FIG. 7 , which is a schematic structural diagram of an embodiment of a speech recognition device of the present application.

In the current embodiment, the speech recognition device 700 provided by the present application includes a processor 701 , a memory 702 and a communication interface 703 coupled to the processor 701 , that is, the memory 702 and the communication interface 703 are respectively connected to the processor 701 . The speech recognition device 700 may execute the method in any one of FIGS. 1 to 6 and its corresponding embodiments.

The processor 701 is connected to the memory 702 and the communication interface 703 respectively.

The communication interface 703 is used to communicate with other external electronic devices under the control of the processor 701 to transmit physiological parameters and instructions. Wherein, other electronic devices at least include: an external module for collecting audio to be processed.

The memory 702 includes local storage and stores a computer program. When the computer program is executed by the processor 701, the method of any one of the above embodiments can be implemented.

The processor 701 is configured to run the computer program stored in the memory 702 to execute the method of any one of the embodiments of FIG. 1 to FIG. 6 and its corresponding embodiments.

Referring to FIG. 8, FIG. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 800 stores a computer program 801 that can be executed by the processor, and the computer program 801 is used to implement the speech recognition method described in any one of the above embodiments in FIG. 1 to FIG. 6 and its corresponding embodiments. Specifically, the above-mentioned computer-readable storage medium 800 may be one of a memory, a personal computer, a server, a network device, or a USB flash drive, which is not specifically limited herein.

The above description is only an embodiment of the present application, and is not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied to other related technologies Fields are similarly included within the scope of patent protection of this application.

Claims (10)

1. A method of speech recognition, the method comprising:

acquiring audio to be processed generated by at least one target person and physiological parameters of the target person;

performing voice transcription on the audio to be processed to obtain a voice transcription result;

and displaying the voice transcription result, and prompting the physiological parameters of the target person under the condition that the physiological parameters of the target person meet early warning conditions.

2. The method according to claim 1, wherein the voice transcribing the audio to be processed to obtain a voice transcription result comprises:

and segmenting the audio to be processed, and performing voice transcription on a plurality of segments of audio to be processed obtained by segmentation to obtain a plurality of transcribed text segments to be used as the voice transcription result.

3. The method of claim 2, wherein the segmenting the audio to be processed comprises at least one of:

segmenting audio belonging to different characters in the audio to be processed;

if a preset pause exists in a first audio belonging to the same character in the audio to be processed, segmenting the first audio according to the preset pause; wherein the preset pause is a voice pause exceeding a preset time;

and if the word number of a second audio belonging to the same figure in the audio to be processed is greater than a preset word number, segmenting the second audio.

4. The method of claim 3, wherein the audio to be processed is captured using a microphone array; prior to the segmenting the audio to be processed, the method further comprises:

determining the directions of the parts of the audio to be processed based on the microphone array;

and determining the character to which each part of the audio belongs based on the position of each part of the audio.

5. The method of claim 1, wherein the voice transcription result comprises a plurality of transcription text segments, and the plurality of transcription text segments are obtained by transcription of a plurality of partial audios in the audio to be processed;

the displaying the voice transcription result comprises:

displaying the plurality of transcribed text segments;

under the condition that the physiological parameters of the target person meet the early warning condition, prompting the physiological parameters of the target person comprises the following steps:

determining early warning time based on the acquisition time of the physiological parameters under the condition that the physiological parameters meet early warning conditions, and determining early warning parameter values based on the values of the physiological parameters;

determining the transcription text segment matched with the early warning time as a target transcription text segment;

and displaying the early warning parameter value in a preset area range of the target transcription text segment.

6. The method of claim 5, wherein determining the transcribed text segment that matches the pre-warning time as a target transcribed text segment comprises:

if the transfer text segment with the time range covering the early warning time exists, taking the existing transfer text segment as the target transfer text segment;

and if the transcribed text segment of which the time range covers the early warning time does not exist, taking the transcribed text segment which is positioned after the early warning time and is closest to the early warning time as the target transcribed text segment.

7. The method according to claim 5, wherein the early warning condition is that the values acquired by the physiological parameters at a plurality of acquisition moments meet a preset requirement; the determining of the early warning time based on the acquisition time of the physiological parameter comprises:

taking the central time of the multiple acquisition moments as the early warning time;

the determining an early warning parameter value based on the value of the physiological parameter includes:

taking the average value of the values of the physiological parameters acquired at the plurality of acquisition moments as the early warning parameter value;

and/or the target person has multiple physiological parameters, wherein some types of the physiological parameters meet the early warning condition, and some types of the physiological parameters do not meet the early warning condition; the displaying the early warning parameter value in the preset area range of the target transcription text segment includes:

displaying the early warning parameter value corresponding to the physiological parameter meeting the early warning condition in a preset area range of the target transcription text segment; or,

determining that the physiological parameter meeting the early warning condition has higher priority than the physiological parameter not meeting the early warning condition; and sequencing the early warning parameter values corresponding to each physiological parameter according to the priority to form a parameter set to be displayed, and displaying the parameter set to be displayed in a preset area range of the target transcription text segment.

8. The method of claim 1, wherein the physiological parameter comprises a stress value, the warning condition comprises that the stress value meets a preset requirement, and the voice transcription result comprises a plurality of transcription text segments which are obtained by transcribing a plurality of parts of audio in the audio to be processed;

the method further comprises the following steps that after the voice transcription result is displayed and the physiological parameters of the target person are prompted under the condition that the physiological parameters of the target person meet early warning conditions, the method further comprises the following steps:

taking a first transcription text segment corresponding to the tension value meeting a preset requirement, a second transcription text segment associated with the first transcription text segment and an early warning level corresponding to the tension value as a group of early warning information;

and generating an early warning report based on the early warning information.

9. A speech recognition device comprising a processor and a memory and a communication interface coupled to the processor; wherein,

the communication interface is used for communicating with other electronic equipment;

the memory is used for storing a computer program;

the processor is configured to run the computer program to perform the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that it stores a computer program executable by a processor for implementing the method of any one of claims 1 to 9.

Images