JP2023174514A - Information processing device, information processing method, and program - Google Patents

Abstract

To assist selection of an utterance to be targeted for training data.SOLUTION: In an information processing system 100, an information processing device includes: an acquisition unit that acquires voice data; a voice recognition unit that detects, from a voice related to the acquired voice data, an utterance section which is a section in which an utterance has been made; a determination unit that determines whether the utterance in the detected utterance section satisfies one or more conditions previously set for outputting candidates for training data; and an output unit that outputs content of a first utterance in the utterance section determined as satisfying the conditions as a candidate for the training data.SELECTED DRAWING: Figure 14

Description

The present invention relates to an information processing device, an information processing method, and a program.

Conventionally, a spoken language corpus is generated using the program audio (audio data) of a specific program, subtitle text added to the program in advance, and a transcription of the program audio, and is used for speech recognition. Techniques for learning acoustic models are known.

The above-mentioned conventional technology discloses supporting the generation of teacher data using audio data of a program. However, when a worker checks voice data and generates training data, it may be burdensome for the operator to select utterances to be used as training data.

The disclosed technology has been developed in view of the above-mentioned circumstances, and aims to support the selection of utterances to be used as training data.

The disclosed technology includes an acquisition unit that acquires voice data, a voice recognition unit that detects a speech section in which an utterance is made from the speech related to the voice data, and a speech recognition section that detects an utterance in the detected speech section. a determination unit that determines whether one or more preset conditions are met for outputting data candidates; and a content of the first utterance in the utterance section that is determined by the determination unit to satisfy the condition. The information processing apparatus includes an output unit that outputs the teacher data as a candidate.

According to an embodiment of the present invention, it is possible to support selection of utterances to be used as training data.

1 is a diagram illustrating an example of a system configuration of an information processing system. FIG. 3 is a first diagram illustrating duplicate utterances. FIG. 7 is a second diagram illustrating duplicate utterances. FIG. 2 is a first diagram illustrating agitation and filler. FIG. 2 is a second diagram illustrating agitation and filler. FIG. 3 is a first diagram illustrating the exclusion of duplicate utterances from audio data. It is a figure showing a comparative example. FIG. 7 is a second diagram illustrating the exclusion of duplicate utterances from audio data. FIG. 3 is a diagram illustrating the exclusion of compliments and fillers from audio data. FIG. 3 is a diagram illustrating an isolated match and an isolated filler. FIG. 3 is a diagram showing the relationship between audio data and speech sections that satisfy a specific condition. FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device. It is a diagram showing an example of the hardware configuration of a terminal device. FIG. 2 is a diagram illustrating the functional configuration of each device included in the information processing system. It is a figure showing an example of a recognition result data storage part. 3 is a flowchart illustrating processing of the information processing device. FIG. 6 is a diagram illustrating a display example of training data candidates. FIG. 7 is another diagram illustrating a display example of training data candidates. FIG. 3 is a diagram illustrating a method of acquiring audio data.

Embodiments will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of a system configuration of an information processing system.

The information processing system 100 of this embodiment includes an information processing device 200 and a terminal device 400, both of which are connected via a network or the like.

The information processing device 200 of this embodiment may be a general computer, and includes a speech recognition section 250 and a generation support section 260. The speech recognition unit 250 obtains speech data, performs speech recognition processing on the obtained speech data, and obtains a character string converted from the speech data.

Note that in the following description, data obtained by performing voice recognition processing on voice data may be expressed as recognition result data. The recognition result data includes, for example, the utterance ID which is identification information for identifying the utterance included in the audio data, the start time of the utterance section in which the utterance is being made, the end time of the utterance section, and the audio of the utterance section. The character string (text) converted from the data is associated with the data.

Furthermore, in the following description, the utterance content of the utterance section includes at least one of the audio data of the utterance section and the character string converted from the audio data of the utterance section. In other words, the utterance content of the utterance section is included in recognition result data that is the result of voice recognition processing performed on the voice data of the utterance section.

The generation support unit 260 uses the recognition result data to support generation of teacher data for improving the accuracy of speech recognition processing. Note that the teacher data in this embodiment is teacher data for causing a speech recognition model, which will be described later, to perform machine learning.

Specifically, the generation support unit 260 selects recognition result data in which the utterance in the utterance section satisfies one or more preset conditions from among the recognition result data for each utterance section as training data candidates. Identify. Then, the generation support unit 260 causes the terminal device 400 to display a list of recognition result data specified as teacher data candidates.

The terminal device 400 may be, for example, a tablet terminal, a smartphone, etc., or may be a general computer similar to the information processing device 200, and is mainly used by a worker who generates teaching data. It's okay to be.

When recognition result data is selected in the list of recognition result data displayed on the terminal device 400, the information processing device 200 generates teacher data based on the selected recognition result data.

Therefore, in this embodiment, the operator who generates the teacher data can improve the accuracy of the speech recognition process by simply selecting the recognition result data to be used for generating the teacher data from among the recognition result data that are candidates for the teacher data. It is possible to generate training data for

Note that in the example of FIG. 1, the information processing device 200 includes the speech recognition section 250 and the generation support section 260, but the present invention is not limited to this. The speech recognition unit 250 and the generation support unit 260 may be realized by separate devices. Specifically, for example, the speech recognition unit 250 may be realized by a speech recognition device different from the information processing device 200.

Here, the points of interest in this embodiment will be explained.

FIG. 2 is a first diagram illustrating duplicate utterances. FIG. 3 is a second diagram illustrating duplicate utterances. FIG. 2 shows, for example, a case where parts of the utterances of speaker 1 and speaker 2 overlap.

In FIG. 2, the speech section of speaker 1 is the section from timing T1 to timing T3, and the speech section of speaker 2 is the section from timing T2 to timing T4. Furthermore, in the section from timing T2 to timing T3, the utterances of speaker 1 and utterances of speaker 2 overlap.

In this embodiment, such overlapping utterances of multiple speakers may be expressed as overlapping utterances.

Such overlapping sections frequently occur, for example, in meetings where multiple people participate. FIG. 3 is a diagram illustrating an example of the ratio between the meeting time and the time during which overlapping utterances were made (time during which overlapping utterances were made) in various conferences. Note that the conference time refers to the period from the start to the end of audio data recording.

As shown in FIG. 3, in a typical conference, the time ratio of overlapping utterances to the total utterances is less than 5% of the total utterance time, as shown in FIG.

However, since the training data for speech recognition is in units of sentences (single meaningful utterances), for example, even if repeated utterances occur for a very short time, that portion will have a negative impact on the learning of the entire sentence. The entire sentence may not be suitable for study.

FIG. 4 is a first diagram illustrating the match and filler. FIG. 5 is a second diagram illustrating the match and filler.

Fillers and compliments often occur in conversations between people at meetings and the like. As shown in FIG. 4, fillers are words that are used as fillers such as "um" and "um", or interjections that are uttered when surprised or disappointed.

Although the filler itself has almost negligible negative effects on speech recognition learning, it is unnecessary information in teacher data and the like.

On the other hand, since apologies indicate the intention of the recipient, they are meaningful in teacher data, etc., but since they occur more often than necessary, it becomes a heavy burden for the operator to check all the pouts.

FIG. 5 is an example of a histogram of utterance lengths in multiple types of conferences. From FIG. 5, it can be seen that in all the meetings shown in histograms 1 to 6, the frequency of utterances with an utterance length of less than 1 second is highest. An utterance with an utterance length of less than 1 second strongly indicates that it is an utterance that includes agreement.

As described above, in conversations between people, many monotonous utterances like mutual exchanges occur, so it can be seen that it is a heavy burden for the worker to check all of this voice data.

In this embodiment, focusing on these points, among the recognition result data for each utterance section included in audio data of recorded conversations of multiple people, recognition result data for utterance sections where the utterance is a duplicate utterance, a compliment, or a filler. are excluded from training data candidates. In this embodiment, by selecting candidates for teacher data in this way, it is possible to reduce the burden on the worker involved in generating the teacher data. In other words, in this embodiment, teacher data can be efficiently generated.

Note that in this embodiment, the audio data obtained by recording a conversation between multiple people may be audio data recorded while the distance between the speaker's mouth and the microphone is a certain distance or more.

In the following description, a state in which the distance between the speaker's mouth and the microphone is a certain distance or more may be expressed as a "Far Field." In addition, in the following explanation, audio data acquired when the distance between the speaker's mouth and the microphone that acquires the audio data is a certain distance or more is expressed as audio data acquired in the Far Field. There is. Details of how to obtain audio data will be described later.

Next, a case where duplicate utterances are excluded from audio data and a comparative example will be described with reference to FIGS. 6 to 8.

FIG. 6 is a first diagram illustrating the removal of duplicate utterances from audio data. In FIG. 6, exclusion of duplicate utterances will be explained with reference to the example of FIG.

In the example of FIG. 2, the speech section of speaker 1 is the section from timing T1 to timing T3, and the speech section of speaker 2 is the section from timing T2 to timing T4. Furthermore, the section of overlapping utterances between speaker 1 and speaker 2 is the section from timing T2 to timing T3.

Therefore, in the present embodiment, recognition result data for utterance sections in which utterances do not overlap among the recognition result data for each utterance section in the audio data from timing T1 to timing T4 is extracted and used as training data candidates. Note that in the following explanation, an utterance that does not overlap with the utterances of other speakers will be expressed as an independent utterance.

FIG. 7 is a diagram showing a comparative example. In FIG. 7, voice recognition processing is performed on audio data in an interval from timing T1 to timing T3 (speaker 1's utterance interval) and audio data in an interval from timing T2 to timing T4 (speaker 2's utterance interval). This shows the case where

In this case, the audio data in the interval from timing T1 to timing T3 (speaker 1's utterance interval) and the audio data in the interval from timing T2 to timing T4 (speaker 2's utterance interval) are overlapped. This is the audio data that includes.

Speech data that includes repeated utterances has ambiguous phonemes, and character strings obtained through speech recognition may be inaccurate. Furthermore, when voice recognition processing is performed on voice data that includes repeated utterances, the utterance content included in the recognition result data is likely to be inaccurate because the phonemes are unclear. Therefore, even if a speech recognition model is trained using this recognition result data as training data, there is a possibility that it will not contribute to improving the accuracy of speech recognition. Furthermore, the technology for separating duplicate utterances into single utterances is also highly difficult, making it difficult to ensure high accuracy.

FIG. 8 is a second diagram illustrating the exclusion of duplicate utterances from audio data. In this embodiment, the recognition result data of the utterance section from timing T1 to timing T2, which is a single utterance, and the recognition result data of the utterance section from timing T3 to timing T4 are used as training data candidates.

Furthermore, in this embodiment, the recognition result data of the utterance section from timing T2 to timing T3, which is the overlapping utterance of speaker 1 and speaker 2, is excluded from the training data candidates.

By doing so, in this embodiment, recognition result data in which speech data with clear phonemes and character strings obtained through highly accurate speech recognition processing are associated are candidates for training data, and the training data is It can be presented to the worker who generates it.

Next, with reference to FIGS. 9 and 10, the exclusion of compliments and fillers from audio data will be described.

FIG. 9 is a diagram illustrating the exclusion of compliments and fillers from audio data. Figure 9(A) shows a case where speaker 2 sporadically makes concurrence or short utterances (filler) while speaker 1 is speaking, and Figure 9(B) shows a case where speaker 2 sporadically makes short utterances (filler) while speaker 1 is speaking. This shows a case where a speaker 2 responds with a comment or a short utterance (filler) to a certain speaker 1.

In the example of FIG. 9A, speaker 2's compliments and fillers overlap with speaker 1's utterance. However, as mentioned above, since agitation and fillers occur frequently during a conversation, if the utterance sections in which agitation and fillers overlap are treated as duplicate utterances and excluded from the training data candidates, the recognition result becomes a training data candidate. The amount of data is significantly reduced. Further, compliments and fillers that overlap with the main speaker's utterance can also be regarded as noise.

Therefore, in this embodiment, when an utterance of speaker 1 and a compliment or filler of speaker 2 overlap, the recognition is not treated as an overlapping utterance, but is obtained by performing voice recognition processing on the voice data of only the compliment or filler. Exclude result data from training data candidates.

Specifically, in this embodiment, only the recognition result data corresponding to the voice data indicating the utterance of speaker 1 (the main speaker) in FIG. Recognition result data corresponding to voice data indicating a compliment or filler is excluded from the training data candidates.

Therefore, in the example of FIG. 9A, only the recognition result data of the utterance section of speaker 1 is a candidate for teacher data.

In addition, in this embodiment, as shown in FIG. 9B, when speaker 2 continues to give compliments or fillers to speaker 1, who is the main speaker, the voice indicating the compliment or isolated filler Exclude recognition result data corresponding to the data from training data candidates.

Specifically, in this embodiment, only the recognition result data of the utterance section of speaker 1, who is the main speaker in FIG. The data is excluded from the training data candidates.

Furthermore, in this embodiment, recognition result data corresponding to voice data corresponding to isolated exchanges and isolated fillers during a conversation are also excluded from the training data candidates.

Now, with reference to FIG. 10, isolated fillers and isolated fillers will be explained. FIG. 10 is a diagram illustrating an isolated match and an isolated filler. FIG. 10(A) is a diagram illustrating an isolated filler, and FIG. 10(B) is a diagram illustrating an isolated filler.

In this embodiment, an isolated compliment or an isolated filler refers to a state in which there is no utterance other than a filler or a compliment within an utterance section in which utterances are continuous.

As shown in FIG. 10(A), the content of the utterance in the utterance section K1 is "Ah, that's right," and includes the utterance "Ah, that's right" as well as the utterance "Ah, right." Therefore, the recognition result data of the utterance section K1 is a candidate for teacher data.

Further, the utterance content in the utterance section K2 is "Ah", and the utterance content in the utterance section K3 is "I see." In this case, the utterance section K2 contains only a mutual comment and does not include any other utterances, so it becomes an isolated compliment. Therefore, the recognition result data of the utterance section K2 is excluded from the training data candidates.

Furthermore, in FIG. 10B, the content of the utterance in the utterance section K4 is "Um, is this okay?" and includes the filler "Um," as well as the utterance "Is this okay?" Therefore, the recognition result data of the utterance section K4 is a candidate for teacher data.

Further, the utterance content in the utterance section K5 is "umm", and the utterance content in the utterance section K6 is "Is this okay?". In this case, the utterance section K5 contains only filler utterances and does not include other utterances, so it becomes an isolated filler. Therefore, the recognition result data of the utterance section K5 is excluded from the training data candidates.

In this manner, in the present embodiment, recognition result data of a utterance section in which the voice data satisfies a specific condition, out of voice data recorded as utterances by a plurality of people, is used as a candidate for training data. In other words, in this embodiment, among the utterances by a plurality of people, the content of utterances in a utterance section that satisfies one or more preset conditions (specific conditions) is used as a candidate for training data. Note that the specific conditions may be set in advance by the user of the information processing system 100. The user of the information processing system 100 may be, for example, an administrator of the information processing device 200, or a user of the terminal device 400 (a worker who generates teacher data).

The specific condition is any one of the following.

-The utterance is from the main speaker and is a single utterance without overlapping utterances from multiple people (hereinafter referred to as condition 1).
・It is not an utterance by the main speaker, nor is it an utterance consisting only of isolated compliments or isolated fillers (hereinafter referred to as condition 2).
・In the main speaker's utterance, which includes some overlapping utterances, there is a portion that is a single utterance (hereinafter referred to as condition 3).
In this embodiment, recognition result data of an utterance section that satisfies any one of these conditions is used as a candidate for training data.

Below, with reference to FIG. 11, the relationship between audio data recorded from multiple people's utterances and utterance sections that satisfy a specific condition will be specifically described.

FIG. 11 is a diagram showing the relationship between audio data and utterance sections that satisfy a specific condition.

In FIG. 11, an audio waveform 10 represented by audio data obtained by recording the utterances of a plurality of people is shown in association with a character string converted from the audio data for each utterance section. Further, in FIG. 11, area 11 shows the utterances of speaker 1 who is the main speaker, and area 12 shows the utterances of speaker 2 who is not the main speaker.

In FIG. 11, utterance sections K10 and K12 by speaker 1 satisfy condition 1 among the specific conditions. Therefore, the recognition result data of the utterance sections K10 and K12 are considered as training data candidates. Further, the speech section K14 of speaker 1 partially overlaps with the speech section K15 of speaker 2. Therefore, in this embodiment, only the portion of the utterance section K14 that does not overlap with the utterance section K15 becomes the utterance section that satisfies condition 3. Furthermore, the recognition result data of this utterance section is used as a candidate for teacher data.

Further, the content of the utterance of speaker 1 in utterance section K16 is a mutual exchange, which satisfies condition 1. Therefore, the recognition result data of the utterance section K16 is taken as a candidate for teacher data.

Furthermore, in FIG. 11, utterance sections K11, K13, and K17 are isolated responses by speaker 2, and do not satisfy any of the specific conditions 1 to 3. Therefore, the recognition result data of the utterance sections K11, K13, and K17 are excluded from the training data candidates.

Furthermore, the utterance section K15 is a duplicate utterance and does not satisfy any of the specific conditions 1 to 3. Therefore, the recognition result data of the utterance section K15 is excluded from the training data candidates.

In the information processing system 100 of the present embodiment, the information processing device 200 identifies recognition result data that are candidates for teacher data in this way, and causes the terminal device 400 to display the candidates for the teacher data. Then, the information processing device 200 of this embodiment generates teacher data using the recognition result data selected by the user of the terminal device 400.

Each device included in the information processing system 100 of this embodiment will be described below. FIG. 12 is a diagram illustrating an example of the hardware configuration of the information processing device.

The information processing device 200 is constructed by a computer, and as shown in FIG. interface) 208, a network I/F 209, a bus line B1, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I/F 216.

Among these, the CPU 201 controls the operation of the information processing device 200 as a whole. The ROM 202 stores programs used to drive the CPU 201, such as IPL. RAM 203 is used as a work area for CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls reading and writing of various data to the HD 204 under the control of the CPU 201.

A display (display device) 206 displays various information such as a cursor, menu, window, characters, or images. External device connection I/F 208 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. The network I/F 209 is an interface for data communication using a communication network. The bus line B1 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 201 shown in FIG. 12.

Further, the keyboard 211 is a type of input means that includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 214 controls reading and writing of various data on the DVD-RW 213, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like. The media I/F 216 controls reading or writing (storage) of data to a recording medium 215 such as a flash memory.

FIG. 13 is a diagram illustrating an example of the hardware configuration of a terminal device. FIG. 13 shows a hardware configuration when the terminal device 400 is a smartphone.

As shown in FIG. 13, the terminal device 400 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a CMOS sensor 405, an image sensor I/F 406, an acceleration/direction sensor 407, a media I/F 409, and a GPS receiving section 411. ing.

Among these, the CPU 401 controls the operation of the entire terminal device 400. The ROM 402 stores the CPU 401 and programs used to drive the CPU 401 such as IPL. RAM 403 is used as a work area for CPU 401. The EEPROM 404 reads or writes various data such as smartphone programs under the control of the CPU 401.

A CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a type of built-in imaging means that images a subject (mainly a self-portrait) and obtains image data under the control of the CPU 401. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge Coupled Device) sensor may be used. The image sensor I/F 406 is a circuit that controls driving of the CMOS sensor 405. The acceleration/direction sensor 407 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect geomagnetism. A media I/F 409 controls reading or writing (storage) of data to a recording medium 408 such as a flash memory. GPS receiving section 411 receives GPS signals from GPS satellites.

The terminal device 400 also includes a long distance communication circuit 412, a CMOS sensor 413, an image sensor I/F 414, a microphone 415, a speaker 416, a sound input/output I/F 417, a display 418, an external device connection I/F (Interface) 419, It includes a short-range communication circuit 420, an antenna 420a of the short-range communication circuit 420, and a touch panel 421.

Among these, the long-distance communication circuit 412 is a circuit that communicates with other devices via a communication network using an antenna 412a. The CMOS sensor 413 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 401. The image sensor I/F 414 is a circuit that controls driving of the CMOS sensor 413. Microphone 415 is a built-in circuit that converts sound into electrical signals. The speaker 416 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice.

The sound input/output I/F 417 is a circuit that processes input/output of sound signals between the microphone 415 and the speaker 416 under the control of the CPU 401 . The display 418 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays images of the subject, various icons, and the like. The external device connection I/F 419 is an interface for connecting various external devices. The near field communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a type of input means by which the user operates the terminal device 400 by pressing the display 418.

The terminal device 400 also includes a bus line 410. The bus line 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401 shown in FIG. 13.

Next, with reference to FIG. 14, the functions of each device included in the information processing system 100 of this embodiment will be described. FIG. 14 is a diagram illustrating the functional configuration of each device included in the information processing system.

First, the functional configuration of the information processing device 200 will be explained. The information processing device 200 of this embodiment includes a speech recognition section 250, a generation support section 260, a communication control section 265, a speech data storage section 270, a recognition result data storage section 280, and a teacher data storage section 290. The speech recognition unit 250 and the generation support unit 260 are realized by the CPU 201 of the information processing device 200 reading and executing a program stored in the HD 204 or the like. The voice data storage section 270, the recognition result data storage section 280, and the teacher data storage section 290 are realized by storage areas of the HD 204 and the like.

In the information processing device 200 of this embodiment, the audio data storage unit 270 stores audio data acquired by the information processing device 200. The recognition result data storage section 280 stores recognition result data that is the result of the speech recognition process by the speech recognition section 250. Details of the recognition result data storage section 280 will be described later.

Note that in the recognition result data storage unit 280, the recognition result data may be stored in association with the voice data specified by the utterance ID of each utterance section. Furthermore, in the present embodiment, the speech data stored in the speech data storage section 270 may be given an utterance ID for each utterance section. In this embodiment, it is sufficient that the recognition result data for each utterance section and the audio data for each utterance section are correlated by the utterance ID.

Further, in the recognition result data storage unit 280, the recognition result data may be stored in association with information indicating whether or not the recognition result data is a candidate for teacher data. In other words, in the recognition result data storage unit 280, the recognition result data may be stored in association with information indicating a determination result by the determination unit 261, which will be described later.

The teacher data storage unit 290 stores teacher data generated by a generation unit 263, which will be described later. The teacher data may be data in which audio data is associated with a character string converted from the audio data.

The speech recognition section 250 includes an acquisition section 251, a section detection section 252, a speech recognition model 253, and a learning section 254. The acquisition unit 251 acquires audio data. The audio data acquired by the acquisition unit 251 may be audio data read from the audio data storage unit 270, or may be audio data acquired from an external device of the information processing system 100. When acquiring audio data from an external device of the information processing system 100, the acquisition unit 251 may store the acquired audio data in the audio data storage unit 270.

The section detection section 252 detects an utterance section from the audio related to the acquired audio data. The utterance section indicates the section in which utterances are being made. When the section detection unit 252 of this embodiment detects a speech section in the audio data, it assigns a speech ID to the specified speech section, and associates the start time and end time of the speech section with the speech ID. It's fine.

The voice recognition model 253 may be a voice recognizer that performs voice recognition processing on voice data acquired when the distance between the speaker's mouth and a sound collection device such as a microphone is a certain distance or more. , obtain a character string (text) as a result of speech recognition processing. In other words, the speech recognition model 253 of this embodiment may be a speech recognizer that performs speech recognition on speech data acquired in the Far Field and converts the speech data into a character string. Details of the speech recognition model 253 will be described later.

The character string acquired by the speech recognition model 253 may be stored in the recognition result data storage unit 280 as recognition result data associated with the utterance ID and the start time and end time of the utterance section.

Specifically, the audio data acquired in the Far Field is, for example, audio data collected using a tabletop microphone such as a boundary microphone.

The learning unit 254 causes the speech recognition model 253 to learn when the teacher data stored in the teacher data storage unit 290 is input.

The generation support section 260 includes a determination section 261, an output section 262, and a generation section 263. The determining unit 261 determines whether the utterance in the utterance section specified by the utterance ID included in the recognition result data satisfies any one of the specific conditions. In other words, the determination unit 261 determines whether or not the recognition result data is to be a candidate for teacher data, based on information indicating a specific condition. Note that information indicating a specific condition may be held in the determination unit 261.

The output unit 262 outputs the recognition result data to the terminal device 400 along with information indicating the result of the determination by the determination unit 261. In other words, the determination unit 261 outputs the recognition result data that is a candidate for teacher data and the recognition result data that is excluded from the candidates for teacher data to the terminal device 400.

The generation unit 263 generates teacher data from the recognition result data in response to an operation on the terminal device 400, and stores it in the teacher data storage unit 290. Specifically, when recognition result data is selected in the terminal device 400, the generation unit 263 creates a correspondence between a character string converted from voice data and included in the selected recognition result data, and the recognition result data. Generate teacher data that is associated with audio data.

The communication control unit 265 controls communication between the information processing device 200 and external devices. Specifically, the communication control unit 265 controls communication between the information processing device 200 and the terminal device 400.

Next, the functional configuration of the terminal device 400 will be explained. Terminal device 400 includes an input reception section 450, a communication control section 460, and a display control section 470.

The input receiving unit 450 receives various inputs to the terminal device 400. Specifically, the input accepting unit 450 accepts selections from teacher data candidates displayed on the terminal device 400. Communication control unit 460 controls communication between terminal device 400 and external devices. The display control unit 470 controls various displays on the display 418 of the terminal device 400. Specifically, the display control unit 470 causes the display 418 to display a list screen including the recognition result data that are candidates for teacher data and the recognition result data that are excluded from the candidates for teacher data.

Next, with reference to FIG. 15, the recognition result data storage unit 280 of this embodiment will be described. FIG. 15 is a diagram showing an example of a recognition result data storage section.

The recognition result data stored in the recognition result data storage unit 280 of this embodiment includes utterance ID, start time, end time, and utterance content as information items, including the item "utterance ID" and the item "start time." , "end time", and "text" are associated with each other.

The value of the item "utterance ID" is identification information for specifying the audio data acquired in the speech section specified by the start time and end time.

The values of the items "start time" and "end time" indicate the start time and end time of the speech section, respectively. The value of the item "text" is a character string obtained by the speech recognition model 253 performing speech recognition processing on the speech data specified by the utterance ID. In other words, the value of the item "text" indicates a character string converted from audio data.

Note that in the recognition result data storage section 280, information indicating the determination result by the determination section 261 may be added to each recognition result data.

Next, with reference to FIG. 16, processing of the information processing apparatus 200 of this embodiment will be described. FIG. 16 is a flowchart illustrating the processing of the information processing device.

The information processing device 200 of this embodiment acquires voice data using the acquisition unit 251 of the voice recognition unit 250 (step S1601). Subsequently, the speech recognition section 250 detects the utterance section using the section detection and determination section 252 (step S1602). Next, the speech recognition unit 250 performs speech recognition processing using the speech recognition model 253 to obtain a character string from the speech data corresponding to the detected utterance section (step S1603). At this point, recognition result data in which the utterance ID, the start time and end time of the utterance section, and the character string converted from the audio data are associated may be stored in the recognition result data storage unit 280.

Note that the processing from step S1601 to step S1603 in FIG. 16 may be executed at an independent timing, apart from the processing from step S1604 onward in FIG. In other words, in the present embodiment, the processing by the speech recognition unit 250 only needs to be executed before the processing by the generation support unit 260 shown after step S1604 is executed.

Next, the information processing device 200 uses the determination unit 261 of the generation support unit 260 to extract the utterance in the utterance section (step S1604). In other words, the determination unit 261 extracts audio data in the utterance section specified by the utterance ID included in the recognition result data.

Next, the determining unit 261 determines whether the extracted utterance is the utterance of the main speaker (step S1605). Specifically, the determination unit 261 may determine that the utterance is the utterance of the main speaker when the volume of the extracted audio data is lower than the volume of other utterance sections. Note that although the volume of the audio data may be relative, it may be absolute.

In step S1605, if the extracted utterance is not the utterance of the main speaker, the information processing device 200 proceeds to step S1610, which will be described later.

If it is determined in step S1605 that the extracted utterance is the utterance of the main speaker, the determination unit 261 determines whether the utterance is a solo utterance (step S1606). That is, in step S1606, the determining unit 261 determines whether the utterance in the extracted utterance section satisfies Condition 1 of the specific conditions.

Specifically, the determination unit 261 may determine whether or not the utterance is a single utterance based on the confidence level when performing voice recognition processing on the voice data. For example, if the utterances are repeated utterances, the voice becomes unclear and the confidence level decreases. Therefore, in this embodiment, when the confidence level is higher than a predetermined threshold, it is determined that the utterance indicated by this audio data is a solo utterance, and when the confidence level is less than the threshold value, the utterance indicated by this audio data is determined to be a solo utterance. may be considered to be a duplicate utterance. Note that the confidence level may be a value indicating a statistical measure of how certain the prediction or output is.

In step S1606, if the utterance is a solo utterance, the determination unit 261 assumes that this utterance satisfies condition 1, and selects the recognition result data including the utterance ID that specifies this utterance section as a candidate for teacher data ( Step S1607), the process advances to step S1613, which will be described later. In other words, the determination unit 261 selects the utterance that satisfies Condition 1 as the first utterance, and selects the utterance content of the first utterance as a candidate for teacher data.

Specifically, the determination unit 261 sets a flag on recognition result data that includes the utterance ID that specifies this utterance section. The flag may be stored in the recognition result data storage unit 280 in association with the recognition result data.

Further, in step S1606, if the utterance is not a single utterance, the determination unit 261 determines whether the utterance includes a portion that is not a duplicate utterance (step S1608). In other words, the determining unit 261 determines whether or not this utterance includes a portion that is a single utterance. That is, the determining unit 261 determines whether the utterance extracted in step S1604 satisfies condition 3.

In step S1608, if a portion that is a single utterance is not included, that is, if the utterance extracted in step S1604 does not satisfy condition 3, the generation support unit 260 proceeds to step S1612, which will be described later.

In step S1608, if a portion that is a solo utterance is included, that is, if the utterance extracted in step S1604 satisfies condition 3, the determination unit 261 extracts a portion that is a solo utterance from the utterance extracted in step S1604. (step S1609), and the process advances to step S1607. In other words, the determination unit 261 selects the utterance that satisfies condition 3 as the first utterance, and selects the utterance content of the first utterance as a candidate for teacher data.
At this time, if the utterance that overlaps with the utterance of the main speaker is an isolated compliment or filler, the utterance of the main speaker is directly used as the first utterance without extracting the independent utterance.

Note that, similarly to step S1606, the determination unit 261 may extract portions that are single utterances based on the confidence level when performing the voice recognition process.

Further, here, the determination unit 261 selects only the utterance content corresponding to the single utterance extracted in step S1609 from among the utterances extracted in step S1604 as candidates for teacher data.

If it is determined in step S1606 that the extracted utterance is not the utterance of the main speaker, the determining unit 261 determines whether or not this utterance is a solo utterance (step S1610). If it is determined in step S1610 that the utterance is not a solo utterance, the generation support unit 260 proceeds to step S1612, which will be described later.

If it is determined in step S1610 that the utterance is an isolated utterance, the determination unit 261 determines whether the utterance extracted in step S1604 is an isolated comment or filler (step S1611). In other words, the determining unit 261 determines whether the utterance extracted in step S1604 satisfies condition 2. Further, the determining unit 261 may determine whether the utterance is an isolated compliment or a filler based on the feature amount of the audio data corresponding to the extracted utterance.

If it is determined in step S1611 that the utterance is not an isolated compliment or filler, the determination unit 261 determines that the utterance extracted in step S1604 satisfies condition 2, and proceeds to step S1607. In other words, the determination unit 261 considers the utterance that satisfies Condition 2 as the first utterance, and selects the utterance content of the first utterance as a candidate for teacher data.

If it is determined in step S1611 that the utterance is an isolated compliment or filler, the determination unit 261 determines that the utterance extracted in step S1604 does not satisfy the specific condition, and stores recognition result data corresponding to this utterance. , are excluded from the training data candidates (step S1612), and the process proceeds to step S1613, which will be described later. In other words, the determination unit 261 determines the extracted utterance as a second utterance that does not satisfy the specific condition.

The information processing device 200 determines whether the processes from step S1604 to step S1612 have been executed for all the speech sections detected in step S1602 (step S1613). In step S1613, if the process has not been executed for all speech sections, the information processing device 200 returns to step S1604.

In step S1613, if the process has been executed for all speech sections, the output unit 262 outputs the recognition result data and the determination result by the determination unit 261 to the terminal device 400. (step S1614), and the process ends.

Note that in the example of FIG. 16, after all speech sections included in the audio data are detected, it is determined whether the utterance indicated by the audio data corresponding to each speech section satisfies a specific condition. The order of processing is not limited to this. In this embodiment, for example, each time a speech section included in audio data is detected, it may be determined whether the speech indicated by the audio data corresponding to the detected speech section satisfies a specific condition.

In addition, in FIG. 16, it is determined whether the extracted utterance is an utterance by the main speaker, and then it is determined whether the utterance is a solo utterance, but the order of processing is limited to this. Not done. For example, in the present embodiment, after the utterance is extracted, it may be determined whether the utterance is a solo utterance, and then it may be determined whether the utterance is uttered by the main speaker.

Furthermore, in this embodiment, when an utterance satisfies a specific condition, the recognition result data corresponding to this utterance is selected as a candidate for training data. In this case, the recognition result data of the utterance section corresponding to this utterance may be used as a candidate for the teacher data. In other words, regardless of whether the utterances are duplicated or are isolated fillers, if the utterance is uttered by the main speaker, this utterance content may be used as a candidate for training data.

Next, a display example of the terminal device 400 will be described with reference to FIG. 17. FIG. 17 is a diagram illustrating a display example of teacher data candidates.

A screen 171 shown in FIG. 17 is an example of a screen (first screen) displayed on the terminal device 400 based on the data output to the terminal device 400 in step S1614 of FIG. Note that the screen 171 may be displayed on the display 206 of the information processing device 200.

Screen 171 includes display areas 172, 173, and 174. In the display area 172, recognition result data and flags are displayed in association with each other. The flag is information indicating the determination result by the determination unit 261.

In the display area 173, information indicating whether the recognition result data displayed in the display area 172 has been selected by the user of the terminal device 400 is displayed. In the display area 174, operation buttons for operating the page to be displayed on the screen 171 are displayed.

In the display area 172, a flag "1" is assigned to recognition result data including the utterance IDs "0010", "0012", "0014", and "0016". Therefore, in the example of FIG. 17, the utterance content of the utterance (first utterance) identified by each of the utterance IDs "0010", "0012", "0014", and "0016" is considered as a candidate for training data. I can see that.

Note that the utterance identified by the utterance ID "0014" is an utterance in which there is a portion that is a single utterance among the utterances of the main speaker that include overlapping utterances. Therefore, in the display area 172, in the utterance identified by the utterance ID "0014," the utterance contents of the portions that are not duplicate utterances (single utterance portions) are candidates for teacher data.

Furthermore, in the display area 172, the flag "1" is not assigned to the recognition result data including the utterance IDs "0011", "0013", "0015", and "0017". Therefore, in the example of FIG. 17, the utterance contents of the utterances (second utterances) identified by the utterance IDs "0011", "0013", "0015", and "0017" are excluded from the training data candidates. I know what happened.

In this embodiment, in this way, the user of the terminal device 400 can be made aware of the recognition result data that are candidates for teacher data and the recognition result data that are excluded from the candidates for teacher data.

Furthermore, in the display area 173, "○" is displayed in association with recognition result data including utterance IDs "0010", "0012", "0014", and "0016". Therefore, in the example of FIG. 17, the utterance contents of the utterances (first utterances) specified by the utterance IDs "0010", "0012", "0014", and "0016" by the user of the terminal device 400 are as follows. You can see that it has been selected as teacher data.

Note that in the example of FIG. 17, the utterance content of the first utterance that is a candidate for teacher data is selected as the teacher data, but in the screen 171, the utterance content of the second utterance that was excluded from the candidates for teacher data is selected as the teacher data. Contents can also be selected as training data.

In addition, on the screen 171, when the utterance ID included in the recognition result data displayed in the display area 172 is selected by the user of the terminal device 400, the audio data of the utterance section specified by the utterance ID is played back. good. By reproducing the audio data in this manner, the user of the terminal device 400 can confirm whether or not the content of the utterance in the utterance section that is a candidate for teacher data is correct with a simple operation. Further, the user of the terminal device 400 can select teacher data after confirming whether the character string acquired from the audio data is correct.

Furthermore, in the present embodiment, the screen 171 displays the utterance section to which the flag "1" has been added to the recognition result data as a preselected state in the display area 173 (a state in which "○" has been added). Good too. In this case, the selection in the display area 173 may be canceled by the user's operation. Furthermore, in the present embodiment, the screen 171 displays the utterance section to which the flag "1" has been added to the recognition result data as an unselected state in the display area 173 (a state in which "○" is not added). Good too.

Further, in the present embodiment, when the user of the terminal device 400 selects a character string converted from audio data on the screen 171, an editing screen may be displayed for modifying the selected character string. . This allows the user to correct the character string converted from voice data if there is an error in the character string.

Further, the screen 171 may be provided with an operation button for instructing the information processing device 200 to generate teacher data. When this operation button is selected on the screen 171 after the selection of recognition result data to be used as teacher data is completed, the information processing device 200 uses the utterance content included in the recognition result data selected by the user to teach the teacher data. May generate data.

Here, generation of teacher data by the generation support unit 260 of the information processing device 200 will be explained. When the generation unit 263 selects recognition result data to be used as teacher data, the generation support unit 260 of the information processing device 200 generates data from the voice data included in the selected recognition result data from the recognition result data storage unit 280. Get the converted string. The generation unit 263 also acquires audio data associated with the utterance ID included in the selected recognition result data from the audio data storage unit 270. Then, the generation unit 263 uses the acquired audio data as input data, generates teacher data using the character string as correct answer data, and stores it in the teacher data storage unit 290.

In this way, in this embodiment, only utterances that meet specific conditions are presented to the user of the terminal device 400 as training data candidates, and training data is generated using the training data candidates selected by the user. Highly accurate training data can be created. Further, according to the present embodiment, the user can generate teacher data simply by selecting the presented teacher data candidates, and can support the generation of teacher data.

Next, with reference to FIG. 18, another display example when outputting training data candidates will be described. FIG. 18 is another diagram showing a display example of training data candidates.

Screen 181 shown in FIG. 18 includes display areas 182, 183, 174, and operation buttons 184. The display area 182 displays the waveform of the audio data that has been subjected to the audio recognition process. In the display area 183, character strings converted from audio data are displayed in association with the waveform of the audio data for each utterance section.

In the display area 183 of this embodiment, the utterance content included in the recognition result data to which the flag "1" is attached may be highlighted. In other words, in the display area 183, the display mode of the utterance content may be made different depending on whether the recognition result data is a candidate for teacher data or when it is excluded from the candidates for teacher data.

Specifically, in the display area 183, the utterance contents corresponding to the utterance sections K10, K12, K14, and K17 are highlighted. Therefore, the user of the terminal device 400 can recognize that the utterance contents corresponding to the utterance sections K10, K12, K14, and K17 are the utterance contents of the recognition result data selected as teacher data candidates.

Note that some of the utterances in the utterance section K14 are overlapping utterances, so the single utterance "email notification" is considered a candidate for teacher data, and among the character strings converted from the audio data, "email notification" ' is highlighted.

In addition, in the display area 183, the display mode of the utterance content is changed depending on the type of utterance, such as whether the utterance in the utterance section is a single utterance, multiple utterances, isolated utterances, or fillers. It's okay. Furthermore, in the display area 183, the display mode of the content of the utterance may be changed depending on whether the utterance is from the main speaker or not.

In the example of FIG. 18, the utterance section K16 is an utterance by the main speaker and is an isolated compliment. Furthermore, the utterance section K17 is an utterance by a speaker other than the main speaker, and is an isolated compliment. Therefore, the utterance contents in the utterance section K16 and the utterance contents in the utterance section K17 are displayed differently. By changing the display mode in this way, the user of the terminal device 400 can understand the type of utterance.

Further, in the display area 183, the content of the utterance may be selectable for each utterance section. In this case, in the display area 183, when the content of the utterance is selected by the user, the utterance section is surrounded by a frame line, making it visible that it has been selected as the subject of teacher data.

Further, on the screen 181, an editing screen for modifying a character string may be displayed by selecting a character string converted from audio data. By doing this, if there is an error in the character string displayed in the display area 183, the erroneous character string can be corrected. Note that it is preferable that the operation of selecting a character string to display the editing screen is a different operation from the operation of selecting the content of the utterance to be used as teacher data.

Next, with reference to FIG. 19, a method for acquiring audio data obtained by recording the utterances of a plurality of people will be described.

FIG. 19 is a diagram illustrating a method of acquiring audio data. Note that the example shown in FIG. 19 is an example of a method for acquiring audio data, and audio data obtained by recording the utterances of multiple people may be acquired by other methods.

FIG. 19 shows a case where audio data during a conference is recorded. Specifically, in FIG. 19, the tabletop microphone 500 placed on the table 110 in the conference room R1 collects the utterances of each of the conference participants P1 to P6 as audio data.

The desk microphone 500 may be a general sound collection device, and may include a storage device that stores collected audio data and a communication device that transmits the audio data to the information processing device 200.

The voice data collected by the tabletop microphone 500 is transmitted to the information processing device 200 and subjected to voice recognition processing by the voice recognition unit 250.

Here, the desk microphone 500 is placed in the center of the table 110 installed in the conference room R1, and may be placed at a position a predetermined distance or more from the mouths of the participants P1 to P6.

Therefore, the audio data acquired by the desk microphone 500 is the audio data acquired in the Far Field.

In this embodiment, voice data including utterances by a plurality of people is acquired in this way, and voice recognition processing is performed on this voice data.

As explained above, in this embodiment, voice recognition is performed to convert voice data into a character string for each utterance section, and the result of determining whether or not the utterance in this utterance section satisfies a specific condition is recognized. The recognition result data of the recognition result is stored in the recognition result data storage section 280.

By doing so, the worker who creates the teacher data only needs to check the speech recognition results for each utterance section held in the recognition result data storage section 280, which can reduce the workload. Therefore, in this embodiment, the cost for generating teacher data can be reduced.

Furthermore, in this embodiment, since training data can be efficiently generated, sufficient training data can be provided for the learning of the speech recognition model 253, and the accuracy of speech recognition by the speech recognition model 253 can be improved. can be improved.

In the present embodiment, by supporting the generation of teacher data in this way, highly accurate teacher data can be easily generated, which can contribute to improving the accuracy of speech recognition in machine learning.

Here, the speech recognition model 253 of this embodiment will be explained. The speech recognition model 253 of this embodiment may be configured with a DNN (Deep Neural Network) or the like, and may also be an End-to-End model.

The End-to-End model is a model that directly converts input speech into characters via one neural network. The End-to-End model has a simpler structure, making it easier to implement and faster response times than traditional speech recognition models that combine multiple parts such as acoustic models, language models, and pronunciation dictionaries by individually optimizing them. It has the advantage of being fast.

Furthermore, compared to conventional speech recognition models that individually optimize multiple parts, the End-to-End model can learn more efficiently from ungrammatical and highly fluctuating speech data such as spoken words. can. Audio data that is ungrammatical and has large fluctuations, such as spoken words, is, for example, audio data acquired in the Far Field.

Therefore, the teacher data generated by the method of this embodiment becomes useful teacher data in learning when the voice recognition model 253 is an End-to-End model.

Furthermore, in conventional speech recognition models, a front end that performs acoustic processing (noise cancellation, etc.) is often implemented in the previous stage, but in the case of an end-to-end model, noise cancellation is not performed, and noise Learning can be easily performed using audio data containing .

Therefore, when the voice recognition model 253 is an end-to-end model, the accuracy of voice recognition can be improved by training the voice recognition model 253 using the training data generated by the method of this embodiment. be able to.

Each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. This includes devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules.

Additionally, the devices described in the embodiments are merely illustrative of one of a plurality of computing environments for implementing the embodiments disclosed herein.

In some embodiments, information processing apparatus 200 includes multiple computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., to perform the processes disclosed herein. Similarly, information processing apparatus 200 may include multiple computing devices configured to communicate with each other.

Further, information processing system 100 can be configured to share the disclosed processing steps in various combinations. For example, a process executed by information processing device 200 may be executed by another information processing device. Similarly, the functions of information processing device 200 can be performed by another information processing device. Further, each element of the information processing device and other information processing devices may be combined into one information processing device, or may be divided into a plurality of devices.

Although the present invention has been described above based on each embodiment, the present invention is not limited to the requirements shown in the above embodiments. These points can be changed without detracting from the gist of the present invention, and can be determined appropriately depending on the application thereof.

Aspects of the present invention are, for example, as follows.
<1>
an acquisition unit that acquires audio data;
a speech recognition unit that detects a speech section that is a speech section from the speech related to the speech data;
a determination unit that determines whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
an output unit that outputs the content of the first utterance in the utterance section that is determined to satisfy the condition by the determination unit as a candidate for the teacher data;
An information processing device having:
<2>
The output section is
outputting a first screen that displays the content of the first utterance as a candidate for the teacher data;
The information processing device according to <1>.
<3>
The content of the first utterance is selectably displayed on the first screen as the content of the utterance targeted for the teacher data to be generated;
The information processing device further includes:
The information processing device according to <2>, further comprising a generation unit that generates teacher data based on the content of the first utterance selected by the user on the first screen.
<4>
The first screen further includes the content of a second utterance in the utterance section that is determined by the determination unit to not satisfy the condition, and further includes the content of the second utterance that is used for the teacher data. can be selected as the content of the utterance,
The generation unit generates the teacher data based on the content of the second utterance when the content of the second utterance is selected by the user on the first screen.
The information processing device according to <2> or <3>.
<5>
The first screen is a screen that visibly displays that the content of the first utterance among the content of the first utterance and the content of the second utterance is a candidate for the teacher data. ,
The information processing device according to <4>.
<6>
The first screen is a screen that can reproduce audio data corresponding to the content of the first utterance and the content of the second utterance,
The information processing device according to <4> or <5>.
<7>
The content of the utterance in the utterance section includes at least audio data corresponding to the utterance section and a character string obtained by converting the audio data. Information processing device.
<8>
The audio data is audio data related to the audio of conversations of multiple speakers including the main speaker.
The information processing device according to any one of <1> to <7>.
<9>
The one or more conditions include that the utterance is by the main speaker;
The information processing device according to <8>.
<10>
The one or more conditions include that the utterance is an utterance of the main speaker and does not temporally overlap with other utterances;
The information processing device according to <8> or <9>.
<11>
The one or more conditions include that the utterance is not an utterance of the main speaker and is not an utterance consisting only of compliments or fillers;
The information processing device according to any one of <8> to <10>.
<12>
The condition of 1 or more is that the utterance of the main speaker includes a part of the utterance that overlaps in time with other utterances, and the part that does not overlap in time with other utterances. including
The information processing device according to any one of <8> to <11>.
<13>
The output section is
The judgment unit determines that the detected utterances in the utterance section are utterances of the main speaker that include some utterances that overlap in time with other utterances, and are different in time from other utterances. If it is determined that there is a portion that does not overlap temporally, the content of the utterance of the main speaker's utterance that does not overlap temporally with the other utterances is used as a candidate for the training data. The information processing device according to <12>, which outputs.
<14>
The information processing apparatus according to claim 1, wherein the one or more conditions are set by a user.
<15>
An information processing method using an information processing device, the information processing device comprising:
Get audio data,
detecting an utterance interval that is an utterance interval from the audio related to the audio data;
determining whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
An information processing method that outputs the content of a first utterance in the utterance section that is determined to satisfy the condition as a candidate for the teacher data.
<16>
Get audio data,
detecting an utterance interval that is an utterance interval from the audio related to the audio data;
determining whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
A program that causes an information processing device to execute a process of outputting content of a first utterance in the utterance section that is determined to satisfy the condition as a candidate for the teacher data.

100 Information processing system 200 Information processing device 250 Speech recognition unit 260 Generation support unit 261 Judgment unit 262 Output unit 263 Generation unit 280 Recognition result data storage unit 290 Teacher data storage unit 400 Terminal device

JP 2017-045027 Publication

Claims (16)

an acquisition unit that acquires audio data;
a speech recognition unit that detects a speech section that is a speech section from the speech related to the speech data;
a determination unit that determines whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
an output unit that outputs the content of the first utterance in the utterance section that is determined to satisfy the condition by the determination unit as a candidate for the teacher data;
An information processing device having: The output section is
outputting a first screen that displays the content of the first utterance as a candidate for the teacher data;
The information processing device according to claim 1. The content of the first utterance is selectably displayed on the first screen as the content of the utterance targeted for the teacher data to be generated;
The information processing device further includes:
The information processing apparatus according to claim 2, further comprising a generation unit that generates teacher data based on the content of the first utterance selected by the user on the first screen. The first screen further includes the content of a second utterance in the utterance section that is determined by the determination unit to not satisfy the condition, and further includes the content of the second utterance that is used for the teacher data. can be selected as the content of the utterance,
The generation unit generates the teacher data based on the content of the second utterance when the content of the second utterance is selected by the user on the first screen.
The information processing device according to claim 3. The first screen is a screen that visibly displays that the content of the first utterance among the content of the first utterance and the content of the second utterance is a candidate for the teacher data. ,
The information processing device according to claim 4. The first screen is a screen that can reproduce audio data corresponding to the content of the first utterance and the content of the second utterance,
The information processing device according to claim 4. 2. The information processing device according to claim 1, wherein the content of the utterance in the utterance section includes at least audio data corresponding to the utterance section and a character string obtained by converting the audio data. The audio data is audio data related to the audio of conversations of multiple speakers including the main speaker.
The information processing device according to claim 1. The one or more conditions include that the utterance is by the main speaker;
The information processing device according to claim 8. The one or more conditions include that the utterance is an utterance of the main speaker and does not temporally overlap with other utterances;
The information processing device according to claim 8. The one or more conditions include that the utterance is not an utterance of the main speaker and is not an utterance consisting only of compliments or fillers;
The information processing device according to claim 8. The condition of 1 or more is that the utterance of the main speaker includes a part of the utterance that overlaps in time with other utterances, and the part that does not overlap in time with other utterances. including
The information processing device according to claim 8. The output section is
The judgment unit determines that the detected utterances in the utterance section are utterances of the main speaker that include some utterances that overlap in time with other utterances, and are different in time from other utterances. If it is determined that there is a portion that does not overlap temporally, the content of the utterance of the main speaker's utterance that does not overlap temporally with the other utterances is used as a candidate for the training data. The information processing device according to claim 12, wherein the information processing device outputs. The information processing apparatus according to claim 1, wherein the one or more conditions are set by a user. An information processing method using an information processing device, the information processing device comprising:
Get audio data,
detecting an utterance interval that is an utterance interval from the audio related to the audio data;
determining whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
An information processing method that outputs the content of a first utterance in the utterance section that is determined to satisfy the condition as a candidate for the teacher data. Get audio data,
detecting an utterance interval that is an utterance interval from the audio related to the audio data;
determining whether the detected utterance in the utterance section satisfies one or more preset conditions for outputting training data candidates;
A program that causes an information processing device to execute a process of outputting content of a first utterance in the utterance section that is determined to satisfy the condition as a candidate for the teacher data.

Images