JP2016090774A - Information processing device, information processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve user convenience.SOLUTION: An information processing device comprises a sound source type score calculation unit that calculates a sound source type score showing a probability of a sound source type of voice included in voice information, and a digest section determination unit that determines a digest section constituting a digest of the voice information, out of the voice information, on the basis of the calculated sound source type score.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an information processing apparatus, an information processing method, and a program.

  There is a demand for information having a predetermined length of time, such as audio information and video information, to obtain an overview of the content without viewing the entire content. Therefore, for example, in Patent Document 1, a climax part that is a scene to be noticed in speech information is detected from the feature amount indicating the feature of speech information, and an index is assigned to the swell part in speech information. Techniques to do this are disclosed. According to the technology, it is possible to generate a digest of the audio information in which only the climax part is extracted by reproducing only the part with the index from the audio information.

JP 2004-191780 A

  Here, for example, considering generating a digest of audio information that records the state of the meeting, we want to include in the digest scenes that are exciting to understand the outline of the contents of the meeting, that is, scenes where discussions are in conflict On the other hand, there may be a desire to generate a digest so that as many voices as possible are included in order to grasp the participants of the conference. As described above, there are various requests that the user requests for the digest depending on the purpose. Since the technique described in Patent Document 1 is specialized in detecting a rising portion, it is considered that it is difficult to meet such various requests of the user with the technique described in Patent Document 1. .

  Therefore, the present disclosure proposes a new and improved information processing apparatus, information processing method, and program capable of further improving user convenience.

  According to the present disclosure, the sound source type score calculating unit that calculates the sound source type score indicating the probability of the sound source type of the sound included in the sound information, and based on the calculated sound source type score, from the sound information, There is provided an information processing apparatus comprising: a digest section determining unit that determines a digest section constituting the digest of the voice information.

  Further, according to the present disclosure, the processor calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information, and based on the calculated sound source type score, from the sound information And determining a digest section that constitutes a digest of the voice information.

  Further, according to the present disclosure, a function of calculating a sound source type score indicating the probability of a sound source type of sound included in the sound information, and a sound processor type score of the sound information based on the calculated sound source type score. A program that realizes a function of determining a digest section that constitutes the digest of the voice information is provided.

  According to the present disclosure, the sound source type score indicating the probability of the sound source type of the sound included in the sound information is calculated, and the digest section constituting the digest of the sound information from the sound information based on the sound source type score Is determined. Therefore, it is possible to generate a digest according to various requests of the user according to the sound source type. Therefore, user convenience can be further improved.

  As described above, according to the present disclosure, user convenience can be further improved. Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with the above effects or instead of the above effects. May be played.

It is a functional block diagram which shows an example of a function structure of the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the sound source classification score calculated by the sound source classification score calculation part. It is explanatory drawing for demonstrating the relationship between audio | voice information and a digest. It is a flowchart which shows an example of the process sequence of an offline process. It is a flowchart which shows an example of the process sequence of the digest area determination process in single sound source mode in offline processing. It is a flowchart which shows an example of the process sequence of the digest area determination process in single sound source mode in offline processing. It is explanatory drawing for demonstrating the high score area determination process in an offline process. It is a flowchart which shows an example of the process sequence of the high score area determination process in an offline process. It is a flowchart which shows an example of the process sequence of the high score area determination process in an offline process. It is a flowchart which shows an example of the process sequence of the digest area determination process in multiple sound source mode in offline processing. It is a flowchart which shows an example of the process sequence of the digest area determination process in multiple sound source mode in offline processing. It is a functional block diagram which shows an example of a function structure of the information processing apparatus which performs each process in diversity reflection mode. It is a flowchart which shows an example of the process sequence of the digest area determination process in the diversity reflection mode in offline processing. It is a flowchart which shows an example of the process sequence of the digest area determination process in the diversity reflection mode in offline processing. It is a flowchart which shows an example of the process sequence of the digest area deletion process based on diversity in an offline process. It is a flowchart which shows an example of the process sequence of an online process. It is a flowchart which shows an example of the process sequence of the digest area determination process in single sound source mode in offline processing. It is a flowchart which shows an example of the process sequence of the frame deletion process in single sound source mode in online processing. It is explanatory drawing for demonstrating the high score area determination process in an online process. It is a flowchart which shows an example of the process sequence of the high score area determination process in an online process. It is a flowchart which shows an example of the process sequence of the high score area determination process in an online process. It is a flowchart which shows an example of the process sequence of the high score area determination process in an online process. It is a flowchart which shows an example of the process sequence of the digest area determination process in multiple sound source mode in online processing. It is a flowchart which shows an example of the process sequence of the frame deletion process in multiple sound source mode in online processing. It is a flowchart which shows an example of the process sequence of the frame deletion process in a diversity reflection mode in online processing. It is a flowchart which shows an example of the process sequence of the deletion frame selection process based on diversity in an online process. It is a functional block diagram which shows an example of a function structure of the information processing apparatus which concerns on the modification provided with an audio | voice sound collection function. It is a functional block diagram which shows an example of a function structure of the information processing apparatus which concerns on the modification provided with a digest production | generation function. It is a functional block diagram which shows an example of a function structure of the information processing apparatus which concerns on the modification provided with an audio | voice information database. It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus which concerns on this embodiment.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. Study on existing technology 2. Device configuration 3. Details of offline processing 3-1. Overall processing procedure 3-2. Single sound source mode 3-2-1. Processing procedure of digest section determination processing 3-2-2. High score section determination processing 3-3. Multiple sound source modes 3-3-1. Processing procedure of digest section determination processing 3-4. Diversity reflection mode 3-4-1. Functional configuration 3-4-2. Processing procedure of digest section determination processing 3-4-4. 3. Digest section deletion processing based on diversity Details of online processing 4-1. Overall processing procedure 4-2. Single sound source mode 4-2-1. Digest section determination processing 4-2-2. Frame deletion process 4-2-3. High score section determination processing 4-3. Multiple sound source modes 4-3-1. Processing procedure of digest section determination processing 4-3-2. Frame deletion processing 4-4. Diversity reflection mode 4-4-1. Processing procedure of frame deletion processing 4-4-2. 4. Delete frame selection process based on diversity Modification 6 Hardware configuration Summary

(1. Examination of existing technology)
Prior to describing a preferred embodiment of the present disclosure, the results of the study of existing general techniques by the present inventors will be described, and the background that the present inventors have conceived of the present disclosure will be described.

  In general, in order to easily grasp the outline of audio information, video information, etc., a technique for generating the digest has been developed. In particular, many techniques relating to video information have been proposed, such as generating a digest of a recorded television program. However, many techniques for generating a digest from video information are premised on a multimodal framework that uses both feature quantities calculated from video and feature quantities calculated from audio. Compared to video information with a large amount of information, it is considered more difficult to appropriately generate a digest of the audio information based only on the audio information.

  For example, as a general method for generating a digest of audio information, a method for generating a digest by simply extracting the beginning, middle and end of audio information, or a method for generating a digest by extracting a section with a high volume Etc. are considered. Alternatively, some existing IC recorders are equipped with a function for reproducing the first 5 seconds of a selected audio file. However, in the method of extracting a predetermined section regardless of the content of audio information, there is a high possibility that significant information is not included in the digest. In addition, in the method based on the volume, there is a possibility that a section that is not necessarily useful, such as a section where the noise is large, is included in the digest.

  Further, as a technique for generating a digest of voice information, there is a technique described in Patent Document 1, for example. However, as described above, the technique is specialized in generating a digest by extracting a rising portion. The content that the user wants to grasp by the digest is not necessarily limited to the climax part, and it is difficult for the technology to meet the various demands of the user required for the digest.

  In the above, the result which the present inventors examined with respect to the existing general technique was demonstrated. As described above, in the technology for generating a digest of audio information, a more convenient technology capable of meeting various user needs has been desired. The inventors of the present invention have intensively studied a technique that can further improve user convenience based on the above-described examination results for the existing technique, and have arrived at an embodiment of the present disclosure described below. . Hereinafter, a preferred embodiment of the present disclosure that has been conceived by the present inventors will be described in detail.

(2. Device configuration)
With reference to FIG. 1, a functional configuration of an information processing apparatus according to an embodiment of the present disclosure will be described. FIG. 1 is a functional block diagram illustrating an example of a functional configuration of the information processing apparatus according to the present embodiment.

  Referring to FIG. 1, the information processing apparatus 110 according to the present embodiment includes a feature amount extraction unit 111, a sound source type score calculation unit 113, and a digest section determination unit 115 as its functions. The information processing apparatus 110 receives arbitrary voice information, determines a digest section that is a section constituting the digest of the voice information in the voice information, and outputs information about the digest section (digest section information). It is a device to do.

  Note that the input source of audio information to the information processing apparatus 110 may be arbitrary. For example, the audio information input to the information processing apparatus 110 may be stored in a storage unit (not shown) provided in the information processing apparatus 110 or different from the information processing apparatus 110. It may be input from an external device. Alternatively, when the information processing apparatus 110 includes a sound collection unit that collects external sound, the sound information may be input via the sound collection unit (for such a configuration, the following (5- This will be described in detail in “1. Modified example in which voice collecting function is provided”.

  The feature amount extraction unit 111 extracts feature amounts of audio information. As the feature quantity, various physical quantities indicating the characteristics of audio information can be calculated. For example, as the feature amount, power, spectrum envelope shape, number of zero crossings, pitch (fundamental frequency), MFCC (Mel-Frequency Cepstrum Coefficients), and the like may be calculated. Further, in the case of audio information collected by microphones arranged at different positions, a correlation between the sound collection positions may be calculated as a feature amount. Further, the sound source direction may be further calculated based on the correlation. The feature quantity extraction unit 111 can calculate at least one of these feature quantities.

  In addition, as the process performed by the feature amount extraction unit 111 to extract feature amounts from the speech information, various methods generally used in speech information analysis processing may be used. A detailed description of the processing is omitted. In addition, the feature amounts calculated by the feature amount extraction unit 111 are not limited to those listed above, and the feature amount extraction unit 111 calculates various feature amounts that can be generally calculated in the analysis processing of audio information. It's okay.

  The feature amount calculated by the feature amount extraction unit 111 can be expressed, for example, as a vector (feature amount vector) in a space (feature amount space) having the dimension of the number of types of calculated feature amounts. The feature quantity extraction unit 111 provides information about the calculated feature quantity (that is, information about the feature quantity vector) to the sound source type score calculation unit 113.

  The sound source type score calculation unit 113 calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information based on the feature amount of the sound information extracted by the feature amount extraction unit 111. Here, the sound source type is obtained by classifying audio sound sources into several types. For example, the sound source type score includes a music score indicating the likelihood of music, a voice score indicating the likelihood of human voice, and / or a noise score indicating the likelihood of noise. Further, when the voice score is calculated, in more detail, a male voice score indicating male voice quality, a female voice score indicating female voice quality, a child voice score indicating child voice quality, and / or A specific voice score or the like indicating the character of a specific person who is producing a voice may be calculated.

  The sound source type score calculation unit 113 calculates at least one of the above-described sound source type scores for each predetermined section in the audio information. Hereinafter, the time unit in which the sound source type score calculation unit 113 calculates the sound source type score is referred to as a score calculation section. The score calculation section may be a section corresponding to a frame, for example.

  For the calculation of the sound source type score, various classifiers generally used in the sound information analysis process may be used. The discriminator can calculate each sound source type score by machine learning, for example, according to a feature vector of speech information to be analyzed, that is, according to coordinates in a feature space. When it is difficult to perform machine learning in the classifier in advance, the sound source type score calculation unit 113 calculates the sound source type score according to the distance from the average speaker characteristics derived from past calculations. be able to. For example, the sound source type score calculation unit 113 outputs a higher value as the sound source type score as the distance from the past speaker characteristics increases.

  FIG. 2 shows an example of the sound source type score calculated by the sound source type score calculating unit 113. FIG. 2 is a diagram illustrating an example of a sound source type score calculated by the sound source type score calculating unit 113. In FIG. 2, the horizontal axis represents time in the audio information, the vertical axis represents the sound source type score calculated for each score calculation section, and the relationship between the two is plotted. In the example illustrated in FIG. 2, three types of sound source type scores are calculated by the sound source type score calculation unit 113.

  The sound source type score calculation unit 113 provides information about the sound source type score calculated for each score calculation section to the digest section determination unit 115.

  Based on the sound source type score calculated by the sound source type score calculation unit 113, the digest section determination unit 115 determines a digest section, which is a time section constituting the digest of the audio information, from the audio information. Here, with reference to FIG. 3, the relationship between audio | voice information and a digest is demonstrated. FIG. 3 is an explanatory diagram for explaining the relationship between audio information and a digest.

  As shown in FIG. 3, the digest is composed of at least one time section in the audio information. In the example shown in the figure, four time intervals (digest intervals 1 to 4) in the audio information are determined as time intervals (digest intervals) constituting the digest, and the digest is generated by connecting these digest intervals. It is configured.

  In the following description, the time length of each digest section is referred to as the digest section length. In addition, the time length of the digest is referred to as the digest length. The digest length is set in advance by the user or the designer of the information processing apparatus 110 as the digest length to be obtained, for example, for one minute. The digest section is determined such that the sum of the digest section lengths substantially matches the digest length.

  The digest section determination unit 115 basically determines a time section having a higher sound source type score in the music information as the digest section. However, as shown in FIG. 2, a plurality of sound source type scores can be calculated independently for audio information. Therefore, it is necessary to set in advance which sound source type score is used to determine the digest section.

  Here, which sound source type score is preferentially used to determine the digest section may vary depending on the user's request. For example, for a user who wants to extract only a male voice from voice information, it is desirable to pay attention to a male voice score and to determine a time section with a higher male voice score as a digest section. . Alternatively, for a user who wants to extract various voices included in the voice information evenly, it is desirable that a time section having a high sound source type score is determined as a digest section in a balanced manner for each sound source type. .

  Therefore, in the present embodiment, the mode of the digest to be generated is set, and the digest section determination unit 115 performs processing for determining the digest section according to the set mode. A predetermined mode may be set in advance, or may be arbitrarily switched according to an operation input by a user via an input unit of the information processing apparatus 110 (not shown). Mode information indicating the set mode is input to the digest section determination unit 115. The digest section determination unit 115 determines the sound source type of the voice to be included in the digest based on the set mode, and determines a section having a higher sound source type score related to the determined sound source type as the digest section in the sound information. can do.

  For example, as a mode, a single sound source mode for generating a digest so as to include only sound of a single sound source type, a multiple sound source mode for generating a digest so as to include sound of a plurality of sound source types at a predetermined ratio, and There is a diversity reflection mode for generating a digest so that various voices are included from voices classified into the same sound source type.

  When the mode is the single sound source mode, the mode information includes information for designating the sound source type to be preferentially included in the digest. When the mode is the single sound source mode, the digest section determination unit 115 determines a section having a higher sound source type score related to the designated one sound source type as the digest section.

  When the mode is a multiple sound source mode, the mode information includes information for designating the ratio of the sound source types to be included in the digest. When the mode is the multiple sound source mode, the digest section determining unit 115 sets the time length of the sound included in the digest for each sound source type based on the specified ratio, and the sound source type score is set for each sound source type. A section that is a higher section and whose total length is equal to or less than the set time length for each sound source type is determined as a digest section.

  The ratio can be appropriately designated by the user as mode information. Thereby, the user can select the sound source type to be preferentially included in the digest in accordance with his / her own request. Conversely, it is also possible to set a low value for the proportion of voice types that are not desired to be included in the digest, such as noise.

  Note that the ratio of the sound source types to be included in the digest may be automatically set by the information processing apparatus 110 instead of being input from the outside as the mode information. For example, for each sound source type, the sum of time lengths of sections having a relatively high sound source type score may be calculated, the ratio may be determined as the ratio between the sound source types of the total, and the type digest length may be determined. The ratio determined in this way may reflect the appearance probability of sound for each sound source type in the sound information.

  In addition, when the mode is the diversity reflection mode, the digest section determination unit 115 calculates the variation of the feature amount within the same sound source type and the variation of the time when the sound is emitted within the same sound source type. Then, the digest section is determined so that the variation in the feature amount and the variation in the time become larger.

  For example, from the viewpoint of the sound source type score, even if the sound sources are classified into the same sound source type, they may actually be voices of different persons. By determining the digest section so that the variation of the feature quantity within the same sound source type becomes larger, from the viewpoint of the sound source type score, although it is classified into the same sound source type, the feature quantity is relatively different. The voice is included in the digest, and more diverse sounds are included in the digest.

  In addition, for example, from the viewpoint of the sound source type score, even if there is a high possibility that the voices of the same person are classified as the same sound source type, Can be quite different. The time at which a sound source type is classified from the viewpoint of the sound source type score is determined by determining the digest section so that the variation in time at which the sound is generated within the same sound source type becomes larger. Voices that are separated from each other will be included in the digest, and voices with more diverse contents will be included in the digest.

  For more specific processing contents of the digest section determination process in each of the single sound source mode, the multiple sound source mode, and the diversity reflection mode, the following (3-2. Single sound source mode), (3-3) .. Multiple sound source mode), (3-4. Diversity reflection mode), (4-2. Single sound source mode), (4-3. Multiple sound source mode), and (4-4. Diversity reflection mode) explain.

  When the digest section determination unit 115 determines the digest section, it outputs information about the determined digest section (digest section information). The digest section information includes, for example, information on the start time and end time of the digest section, the digest section length, the index (digest section index) attached to the digest section, and the like. That is, the digest section information is information for specifying the position of the digest section in the voice information, and a digest can be generated based on the voice information and the digest section information.

  The output destination of the digest section information by the digest section determination unit 115 may be arbitrary. For example, the digest section determination unit 115 may output the digest section information to a storage unit (not shown) provided in the information processing apparatus 110, or the digest section information may be output to an external device different from the information processing apparatus 110. It may be output.

  When the digest section information is stored in the information processing apparatus 110, the information processing apparatus 110 may further have a function of generating a digest based on the digest section information and the voice information (such a configuration). Will be described in detail in the following (5-2. Modification in which digest generation function is provided). In addition, when the digest section information is output to an external device, the external device may have a function of generating a digest based on the digest section information and audio information. As described above, in the present embodiment, the information processing apparatus 110 is configured to have at least a function of generating digest section information, and the function of actually generating a digest after that is not necessarily provided in the information processing apparatus 110. May be.

  The functional configuration of the information processing apparatus according to the present embodiment has been described above with reference to FIG. As described above, according to the present embodiment, the sound source type score indicating the probability of the sound source type of the sound included in the sound information is calculated, and the sound information is extracted from the sound information based on the sound source type score. The digest sections constituting the digests are determined. Therefore, for example, you want to include only the music in the digest, want to include only the voice of the person in the digest, or want to include the music and the voice of the person in a well-balanced digest. Is possible. Note that a series of processing by the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 may be started in response to an instruction by the user via an input unit (not shown), or voice When the information is input to the information processing apparatus 110, the processing for the voice information may be automatically started.

  Here, the specific apparatus configuration of the information processing apparatus 110 may be arbitrary. For example, the information processing apparatus 110 may be a variety of processors such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and an ASIC (Application Specific Integrated Circuit). Alternatively, the information processing apparatus 110 may be an apparatus such as a PC, a server, a smartphone, or a tablet PC on which various processors are mounted. Alternatively, the information processing apparatus 110 may be an apparatus having a sound collecting and recording function such as an IC recorder. Functions of the information processing apparatus 110 shown in FIG. 1 can be executed by various processors operating according to a predetermined program.

  Further, for example, each function of the information processing apparatus 110 (the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115) does not necessarily have to be executed by one device. For example, each function corresponding to the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 is distributed and implemented in a plurality of information processing devices (for example, a plurality of processors). The functions as the information processing apparatus 110 described above may be realized by connecting to each other so as to communicate with each other and operating in cooperation. Further, the information processing apparatus 110 may be a local information processing apparatus that is directly operated by a user, or may be an information processing apparatus on a so-called cloud connected to a user terminal via a network. Good. For example, when a user terminal such as a smartphone or an IC recorder has a recording function, audio information recorded by the terminal is transmitted from the terminal to the information processing apparatus 110 on the cloud. The various processes described above may be performed on the voice information by 110, and the digest section information or the voice information related to the digest as a processing result may be transmitted from the information processing apparatus 110 to the terminal.

  Note that a computer program for realizing each function of the information processing apparatus 110 according to the present embodiment as described above can be produced and mounted on a PC or the like. In addition, a computer-readable recording medium storing such a computer program can be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

  Hereinafter, the process executed by the information processing apparatus 110 will be described in more detail. Here, in the present embodiment, the processing performed by the information processing apparatus 110 can be broadly divided into two according to the processing mode. In one process, the information processing apparatus 110 performs a feature amount extraction process, a sound source type score calculation process, and a digest section determination process on audio information that has been acquired in advance. Hereinafter, such processing is referred to as offline processing.

  On the other hand, in the other process, the information processing apparatus 110 performs a feature amount extraction process, a sound source type score calculation process, and a digest section determination process as needed for the audio information that is just being acquired. In this case, while the voice information is continuously acquired, the digest section information is updated as needed. Hereinafter, such processing is referred to as online processing.

  The detailed processing contents can be different between the offline processing and the online processing. Therefore, in the following, detailed processing contents of each of the offline processing and the online processing will be described. In addition, for each of the offline processing and the online processing, the detailed processing content of the digest section determination processing may be different depending on the mode described above. Therefore, in the following, detailed processing contents of the digest section determination processing according to the mode will be described for each of the offline processing and the online processing.

  In the following description, a case where the score calculation section is a frame section will be described as an example. That is, a case where the sound source type score is calculated for each frame will be described. However, the present embodiment is not limited to such an example, and a section including a plurality of frames may be set as the score calculation section. In the following description, for the sake of simplicity, the sound source type score may be simply referred to as a score.

(3. Details of offline processing)
(3-1. Overall processing procedure)
With reference to FIG. 4, the processing procedure of offline processing will be described. FIG. 4 is a flowchart illustrating an example of a processing procedure of offline processing. The processing procedure shown in FIG. 4 corresponds to the processing procedure of the entire information processing method executed by the information processing apparatus 110 shown in FIG. 1 during offline processing. In the offline processing, after the scores of all frames of the voice information are calculated, a digest section is determined from the voice information based on the score.

  Referring to FIG. 4, in the off-line processing, first, feature values of audio information are extracted (step S101). In the process shown in step S101, various physical quantities indicating the characteristics of the speech information such as power and spectrum envelope shape are calculated as the feature quantities of the speech information. The process shown in step S101 corresponds to, for example, the process performed by the feature amount extraction unit 111 shown in FIG.

  Next, the sound source type score of each frame is calculated based on the extracted feature amount (step S103). In the process shown in step S103, for example, a sound source type score indicating the probability of the sound source type of the sound is calculated for each frame by a discriminator that identifies the sound source type of the sound according to the feature amount of the sound information. At this time, a plurality of types of sound source type scores such as a voice score, a voice score, and a noise score may be calculated. The process shown in step S103 corresponds to the process performed by the sound source type score calculation unit 113 shown in FIG.

  When the score calculation section is not a frame section but includes a plurality of frame sections, in step S103, the sound source type score of each frame is smoothed to calculate the sound source type score as the score calculation section. May be.

  Next, a digest section is determined from the audio information based on the calculated sound source type score (step S105). For example, in the process shown in step S105, a time interval with a higher sound source type score in the audio information is determined as the digest interval. Since the specific processing contents of step S105 differ depending on the mode, the detailed processing contents are described in (3-2. Single sound source mode), (3-3. Multiple sound source modes) and (3-4). .. Diversity reflection mode) will be described in more detail for each mode. Digest section information on the determined digest section is output, and a series of processing ends. Note that the process shown in step S105 corresponds to, for example, the process performed by the digest section determination unit 115 shown in FIG.

  The processing procedure of the offline processing has been described above with reference to FIG.

(3-2. Single sound source mode)
(3-2-1. Digest Section Determination Process Procedure)
In the single sound source mode, one kind of sound source type is specified, and a section having a higher sound source type score related to the specified one sound source type is determined as a digest section.

  With reference to FIG.5 and FIG.6, the process sequence of the digest area determination process in single sound source mode in an offline process is demonstrated. FIG. 5 and FIG. 6 are flowcharts showing an example of a processing procedure of digest section determination processing in the single sound source mode in offline processing.

  Referring to FIGS. 5 and 6, in the digest section determination process in the single sound source mode in the offline process, first, the score threshold theoretical upper limit value is set as the score threshold upper limit value (step S201). Next, the score threshold is set as a value lower than the score threshold upper limit (step S203).

  Here, as will be described in detail later, in the digest section determination process, a process (a high score section determination process shown in step S205) of determining a section having a higher score (high score section) from the speech information as a digest section is performed. Thereafter, the length of the digest section and the number of digest sections are adjusted so that the sum of the time lengths of the digest sections (digest section length) matches the digest length.

  The score threshold is a threshold for determining whether or not each frame is included in the high score section (that is, whether or not to include in the digest section) in the high score section determination process. The score threshold is appropriately changed during a series of digest section determination processes in order to adjust the sum of the digest section lengths according to the digest length, as performed in step S213 and step S219 described later. If the score threshold is changed to a higher value, the number of frames included in the digest section increases and the digest section length becomes longer. Conversely, if the score threshold is changed to a lower value, the number of frames included in the digest section decreases, and the digest section length becomes shorter.

  The score threshold upper limit value is a value that defines the upper limit of the score threshold to be changed. If the score threshold is too high, the number of frames included in the digest section is reduced, and there is a possibility that the sum of the digest section lengths may not be significantly less than the digest length. The score threshold upper limit value is set to prevent such a situation from occurring (see the process shown in step S217 described later).

  The score threshold theoretical upper limit value is a theoretical upper limit value that can be taken by the score, which is set according to, for example, the performance of the discriminator used for calculating the score. As described above, in step S201, the score threshold theoretical upper limit is set as the initial value of the score threshold upper limit.

  If the process shown in step S201 and step S203 is performed, next, the process (high score area determination process) which determines the area (high score area) which has a higher score in speech information as a digest area will be performed ( Step S205). The high score section is a section having a high score continuously in the voice information. However, in the present embodiment, when the time length of a section with a low score is extremely short, processing for including the section in the high score section is performed. When the time length of a section with a low score is extremely short, the section is a series of sections and the preceding and following sections from the viewpoint of information content such as breathing during a series of statements of a person, for example. Because it is considered.

  In the offline processing, in the digest section determination process, the high score section determined in step S205 is regarded as the digest section, and in the subsequent processes, the digest section length is summed up to a length corresponding to the digest length. Processing for adjusting the time length and number of sections is performed. It can be said that the high score section determined in the high score section determination process is a digest section candidate finally determined.

  Details of the high score section determination process will be described later with reference to FIGS. 7-9.

  When high score sections are determined in step S205, these sections are regarded as digest sections, and an average score (section average score) within each digest section is calculated (step S207). The section average score may be included in the digest section information together with the start time, end time, and index of the high score section (that is, the digest section) determined in the high score section determination process.

  Next, it is determined whether or not the total digest section length is significantly shorter than the digest length (step S209). Specifically, in step S209, it is determined whether or not the total digest section length is below the allowable range of deviation from the digest length set for the digest length. Since it is difficult to determine the digest section so that the total digest section length completely matches the digest length, in this embodiment, such an allowable range is set, and the total digest section length is the permissible range. It is determined whether or not the sum of the digest section lengths is appropriate depending on whether it is included in. When the user listens to the digest, the allowable range is a range of a deviation amount that does not give a sense of incongruity because the actual digest length is longer or shorter than the set value of the digest length. May be set as appropriate.

  If it is determined in step S209 that the total digest section length is significantly shorter than the digest length, the process proceeds to steps S211 to S213. In step S211 to step S213, a process for making the total digest section length longer is performed.

  Specifically, in step S211, the current score threshold is set as the score threshold upper limit. This is because the sum of the digest interval lengths is significantly shorter than the digest length, so the current score threshold is considered to be too high compared to the appropriate value. This is to prevent the score threshold from becoming larger than the current score threshold.

  Next, a value lower than the current score threshold is set as a new score threshold (step S213). And it progresses to step S207 and a high score area determination process is performed again using a new score threshold value. Since the number of frames included in the high score section increases by performing the high score section determination process using the new score threshold set to a lower value, the total digest section length becomes longer, and the digest section length Can be made closer to the digest length.

  If it is determined in step S209 that the total digest section length is not significantly shorter than the digest length, the process proceeds to step S215. Conversely, in step S215, it is determined whether or not the total digest section length is significantly longer than the digest length.

  If it is determined in step S215 that the total digest section length is not significantly longer than the digest length, the series of digest section determination processing ends. That is, the current digest section determined by the high score section determination process is determined as the final digest section. If it is determined in step S209 that the total digest length is not significantly shorter than the digest length and it is determined in step S215 that the total digest length is not significantly longer than the digest length, This is because the total digest section length is included in the allowable range of digest length.

  On the other hand, if it is determined in step S215 that the total digest section length is significantly longer than the digest length, the process proceeds to step S217. In the processing after step S217, processing for shortening the total digest section length is performed.

  In step S217, it is determined whether the score threshold is smaller than the score threshold upper limit value. If it is determined in step S217 that the score threshold is smaller than the score threshold upper limit value, the process proceeds to step S219. In step S219, a value higher than the current score threshold is set as the new score threshold. And it progresses to step S207 and a high score area determination process is performed again using a new score threshold value. Since the number of frames included in the high score section is reduced by performing the high score section determination process using a new score threshold set to a higher value, the total digest section length is shortened and the digest section length is shortened. Can be made closer to the digest length.

  If it is determined in step S217 that the score threshold is not smaller than the score threshold upper limit value, the process proceeds to step S221. In this case, since the score threshold cannot be made higher than the current value, the total digest section length cannot be shortened by changing the score threshold. Therefore, in the processing after step S221, processing is performed to shorten the total digest section length by deleting a frame from the current digest section or reducing the number of current digest sections.

  Specifically, in step S221, it is determined whether or not the digest section length can be shortened for each digest section. Here, whether or not the digest section length can be shortened is performed by comparing the digest section length with the minimum continuous section length. The continuous section minimum length is set as the minimum section in which a person can recognize the meaning of the voice when output as voice. If the digest section length is equal to or shorter than the minimum continuous length, the meaning of the part corresponding to the digest section cannot be grasped when the digest is listened to, so the digest section is not significant. Therefore, by performing the determination process shown in step S221, the digest section is determined so that the digest section length becomes larger than the continuous minimum length.

  If it is determined in step S221 that the digest section length can be shortened in any of the digest sections, the process proceeds to steps S223 to S227. In steps S223 to S227, processing is performed to shorten the total digest section length by deleting the frame from the current digest section.

  Specifically, in step S223, a digest section having a lower section average score in a digest section determined to be able to be shortened (ie, a digest section having a digest section length longer than the continuous minimum length). Digest section length is shortened. When shortening the digest section length, for example, of the predetermined number of frames at the beginning of the digest section to be shortened and the predetermined number of frames at the end, the lower average score is excluded from the digest section. The

  Next, the section average score of the digest section in which the frame is deleted and the digest section length is shortened is updated (step S225). Then, it is determined whether or not the sum of the digest section lengths substantially matches the digest length (step S227). In step S227, specifically, it is determined whether the total digest section length is included in the allowable range set in the digest length.

  If it is determined in step S227 that the total digest section length is substantially equal to the digest length, the series of digest section determination processing ends. That is, the current digest section is determined as the final digest section.

  On the other hand, if it is determined in step S227 that the total digest section length does not substantially match the digest length, the process returns to step S221 to determine again whether or not the digest section length can be shortened for each digest section. Is done.

  If it is determined in step S221 that the digest section length cannot be shortened in any digest section, the process proceeds to steps S229 to S231. In steps S229 to S231, processing is performed to shorten the total digest section length by reducing the number of current digest sections.

  Specifically, in step S229, a digest section with a lower section average score is deleted from the current digest section. Then, it is determined whether or not the total digest section length substantially matches the digest length (step S231). In step S231, as in step S227, it is determined whether or not the total digest section length is included in the allowable range set in the digest length.

  If it is determined in step S231 that the total digest length is substantially equal to the digest length, a series of digest segment determination processing ends. That is, the current digest section is determined as the final digest section.

(3-2-2. High score section determination processing)
Here, with reference to FIG. 7 to FIG. 9, the high score section determination process in the offline process shown in step S <b> 205, which has not been described in detail, will be described in detail. FIG. 7 is an explanatory diagram for describing high score section determination processing in offline processing. 8 and 9 are flowcharts showing an example of the processing procedure of the high score section determination processing in the offline processing.

  In the following description of the high score section determination process, the terms “current frame”, “current digest section”, “continuous section”, and “discontinuous section” are used. Prior to describing the specific processing procedure of the high score section determination process, the concept represented by these terms will be described with reference to FIG.

  In FIG. 7, the horizontal axis represents the time of the audio information, the vertical axis represents the score calculated for each frame, and the relationship between the two is plotted. In the high score section determination process, for each frame, it is determined whether to include the frame in the digest section in time series. In the figure, the current frame indicates a frame that is currently subject to determination processing.

  The current digest section means a digest section for which it is determined whether or not to include the current frame. The continuous section means a section where the score continuously exceeds the score threshold in the current digest section. The discontinuous section means a section from the end time of the immediately preceding continuous section to the current frame in the current digest section. The time lengths of the current digest section, the continuous section, and the discontinuous section are also referred to as the current digest section length, the continuous section length, and the discontinuous section length, respectively.

  With reference to FIG.8 and FIG.9, the specific process sequence of the high score area determination process in an offline process is demonstrated. 8 and 9, in the high score section determination process in the offline process, first, the frame index is set to zero (step S301). In addition, the digest section index is set to zero (step S303). The frame index is assigned to each frame of the audio information in chronological order, and the frame with the frame index of zero indicates the head frame of the audio information. The processing shown in step S301 and step S303 corresponds to processing in which the current frame is frame # 0 and the current digest section is digest section # 0.

  Next, it is determined whether the score of the current frame is larger than the score threshold (step S305). If it is determined in step S305 that the score of the current frame is equal to or lower than the score threshold value, the process proceeds to step S319 without including the current frame in the digest section. In this case, the current frame is added to the discontinuous section. The process in step S319 will be described later.

  On the other hand, if it is determined in step S305 that the score of the current frame is greater than the score threshold, the process proceeds to step S307. In steps S307 to S317, a process for including the current frame in the digest section is performed.

  First, in step S307, it is determined whether the discontinuous section length is smaller than the discontinuous section maximum length. Here, the discontinuous section maximum length is a time length that serves as a reference for determining whether the discontinuous section is a significant section to be included in the digest section. As described above, since the discontinuous section is a section from the end time of the immediately preceding continuous section to the current frame, it can be said that the discontinuous section is a section having a continuously low score that is not included in the continuous section. Accordingly, the discontinuous section is considered to be a silent section in which sound of the sound source type to be included in the digest is hardly emitted. For example, when the discontinuous section is extremely short, the section is, for example, From the viewpoint of information content, such as breathing in the middle of a series of utterances of a person, there is a high possibility that the section is a series of sections before and after. From this point of view, the maximum length of a discontinuous section indicates whether the silence section corresponding to the discontinuous section is a very short silence in a series of speech or a long silence with a change of the speaker, for example. It can be set as a time length for judgment.

  If it is determined in step S307 that the discontinuous section length is smaller than the discontinuous section maximum length, the process proceeds to step S309. In this case, as described above, the discontinuous section should be considered as the immediately preceding continuous section and a series of sections. Therefore, in step S309, processing for connecting the discontinuous section and the current frame to the current digest section (that is, adding the discontinuous section and the current frame to the end of the current digest section) is performed. In this way, when the discontinuity period is extremely short, the digest section is determined so as to include the discontinuity period, so that a series of voices are included in the digest without interruption. It is possible to generate a useful digest. At this time, if the process shown in step S309 is also performed for a frame having a frame index that is one smaller (that is, the previous frame in time series), the discontinuous section has already been changed to the current digest section. Since it is included, only the current frame is connected to the current digest section. When the process shown in step S309 ends, the process proceeds to step S319.

  On the other hand, if it is determined in step S307 that the discontinuous section length is greater than or equal to the discontinuous section maximum length, the process proceeds to step S311. In step S311, it is determined whether the continuous section length before the discontinuous section is greater than or equal to the minimum continuous section length. As mentioned when explaining the process shown in step S221 of FIG. 6, the minimum continuous section length is the time length set as the minimum section in which a person can recognize the meaning of the sound when output as speech. It is. That is, it can be said that the process shown in step S311 is a process of determining whether or not a continuous section is a significant section from the viewpoint of time length.

  If it is determined in step S311 that the continuous section length before the discontinuous section is greater than or equal to the minimum continuous section length, the process proceeds to steps S313 to S315. In this case, when the discontinuous section is not less than the maximum length of the discontinuous section and the continuous section is not less than the minimum length of the continuous section (that is, the discontinuous section is not a significant section and before the discontinuous section). If the continuous section is a significant section), the discontinuous section is discarded (not included in the digest section) and the continuous section before the discontinuous section is adopted (included in the digest section). Is called.

  Specifically, in step S313, the continuous section before the discontinuous section is determined as one digest section. Next, in step S315, the digest section index is incremented by 1 (that is, the current digest section to be processed is newly set), and the current frame is set to the start time of the new current digest section. When the process shown in step S315 is completed, the process proceeds to step S319.

  On the other hand, if it is determined in step S311 that the continuous section length before the discontinuous section is smaller than the continuous section minimum length, the process proceeds to step S317. In this case, when the discontinuous section is longer than the maximum length of the discontinuous section and the continuous section is smaller than the minimum length of the continuous section (that is, the discontinuous section is not a significant section and before the discontinuous section). Therefore, the discontinuous section and the continuous section before the discontinuous section are both discarded (not included in the digest section). In this way, when the continuous period is so short that it cannot be recognized by a person, the digest section is determined so as not to include the continuous period, so that it is difficult for the user to understand the content when listening to the digest. A less meaningful section can be omitted from the digest, and a higher quality digest can be generated.

  Specifically, in step S317, the continuous section before the discontinuous section is discarded, and the current frame is set as the start time of the current digest section. When the process shown in step S317 is completed, the process proceeds to step S319.

  In step S319, it is determined whether the audio information is at the end. If it is determined in step S319 that the audio information is not the end, the frame index is incremented by 1 (that is, the frame to be processed is set as the next frame) (step S321), and the processing after step S305 is performed. Is repeatedly executed.

  On the other hand, if it is determined in step S319 that the audio information is the end, the process proceeds to step S323. In step S323, it is determined whether the current digest section length is larger than the continuous section minimum length. That is, in step S323, whether or not the digest section that was the last processing target is a significant section from the viewpoint of time length (that is, whether or not the digest section has a time length that allows voice identification). Is judged.

  If it is determined in step S323 that the current digest section length is larger than the minimum continuous section length, the current digest section is considered to be a significant section in terms of time length. Terminate the process. On the other hand, if it is determined in step S323 that the current digest section length is less than or equal to the minimum continuous section length, the current digest section is considered not a significant section in terms of time length, so the digest section is discarded, A series of processing ends.

  The processing procedure of the digest section determination process in the single sound source mode in the offline process has been described above.

(3-3. Multiple sound source modes)
(3-3-1. Procedure of digest section determination process)
In multiple sound source mode, the time length of the audio included in the digest is set for each sound source type based on the specified ratio, and the sound source type score is higher for each sound source type and the total length of that interval is set A section that is equal to or shorter than the time length for each sound source type is determined as a digest section.

  With reference to FIG.10 and FIG.11, the process sequence of the digest area determination process in multiple sound source mode in an offline process is demonstrated. 10 and 11 are flowcharts showing an example of a processing procedure of digest section determination processing in the multiple sound source mode in offline processing.

  The digest section determination process in the multiple sound source mode shown in FIG. 10 and FIG. 11 is performed for each sound source type in the digest section determination process in the single sound source mode described with reference to FIGS. The content of each process itself may be substantially the same as the digest section determination process in the single sound source mode. However, in the digest section determination process in the single sound source mode, only one sound source type is targeted. Therefore, in the above-described step S209 and step S215, the digest section length determined based on the score related to the sound source type is used. Although the total value was compared with the digest length, in the digest section determination process in the multiple sound source mode, the total value of the digest section length determined based on the score for each sound source type is the value of each sound source type included in the digest. It is compared with the time length of the voice (hereinafter also referred to as the type digest length).

  In the following description of the digest section determination process procedure in the multiple sound source mode, the differences from the digest section determination process procedure in the single sound source mode will be mainly described, and detailed explanations will be given for overlapping items. Description is omitted.

  Referring to FIGS. 10 and 11, in the digest section determination process in the multiple sound source mode in the offline process, first, the score threshold theoretical upper limit is set as the score threshold upper limit (step S401). Next, the score threshold is set as a value lower than the score threshold upper limit (step S403). These processes are the same as the processes in steps S201 and S203 shown in FIGS.

  Next, the type digest length is set (step S405). For example, the type digest length can be set based on the mode information. For example, the mode information includes information indicating that the ratio of sound source types to be included in the digest is specified. In the process shown in step S405, the digest length is calculated for each sound source type by multiplying the digest length by the ratio.

  However, the processing shown in step S405 is not limited to such an example, and the ratio of the sound source types included in the digest may be automatically set by the information processing apparatus 110 instead of being input from the outside as mode information. For example, the high score section determination process shown in FIGS. 8 and 9 has already been executed once for each sound source type at some occasion, and the high score section has been determined for each type of sound source. For example, the ratio may be determined using the information about the high score section, and the type digest length may be determined.

  Specifically, the sum of the time lengths of the determined high score section is calculated for each sound source type from the result of the high score section determination process, and the ratio is calculated. Then, by multiplying the digest length by the calculated ratio, the type digest length can be calculated for each sound source type. Thus, the ratio determined based on the time length of the high score section may reflect the appearance probability of the sound for each sound source type in the sound information.

  In addition, when based on the mode information and based on the high score section, when the calculated type digest length is less than the minimum length of the continuous section, processing for adjusting the length is appropriately performed. This is because when the type digest length is less than the minimum length of the continuous section, the type digest length is too short and the voice is not significantly recognized by a person. Specifically, the type digest length below the minimum continuous section length is increased to the minimum continuous section length, and the increase is subtracted from the type digest length exceeding the other continuous section minimum length.

  When the type digest length is determined, next, a process (high score section determination process) of determining a section (high score section) having a higher score in the speech information as a digest section is performed (step S407). Since the process shown in step S407 is the same as the process in step S205 shown in FIGS. 5 and 6, that is, the series of processes shown in FIGS. 8 and 9, detailed description thereof will be omitted.

  Hereinafter, the processes shown in steps S409 to S433 are the same as the processes in steps S207 to S231 shown in FIGS. 5 and 6 except that they are executed for each sound source type. Is omitted. The processes shown in steps S411 to S421 correspond to the processes in steps S209 to S219 shown in FIGS. In the processing shown in steps S411 to S421, for each sound source type, it is determined whether the total digest section length is not significantly different from the type digest length, and the score threshold is adjusted so that the total digest section length is the type. Each digest section length is adjusted so as to be included in the allowable range of the digest length.

  The processes shown in steps S423 to S433 correspond to the processes in steps S221 to S231 shown in FIGS. The processes shown in steps S423 to S433 are processes performed when the score threshold value can no longer be adjusted. In the processes after step S423, the frame is deleted from the current digest section, or the current By reducing the number of digest sections, a process for shortening the total digest section length is performed. However, in the processing in steps S221 to S231 shown in FIGS. 5 and 6, the digest section to be deleted of the number of frames or sections relates to a single sound source type, but is shown in steps S423 to S433. In the processing, the digest section to be deleted from the number of frames or sections is a mixture of digest sections related to a plurality of sound source types.

  The processing procedure of the digest section determination process in the multiple sound source mode in the offline process will be described above with reference to FIGS. 10 and 11.

(3-4. Diversity reflection mode)
In the diversity reflection mode, a digest is generated so that various sounds are included from the sounds classified into the same sound source type. Specifically, in the diversity reflection mode, the digest section is determined so that the variation in the audio feature amount within the same sound source type and the time variation in the sound within the same sound source type become larger. .

(3-4-1. Functional configuration)
Here, each process in the single sound source mode and the plurality of sound source modes described above can be executed by the functional configuration of the information processing apparatus 110 illustrated in FIG. 1. However, each process in the diversity reflection mode can be executed by a slightly different functional configuration from the information processing apparatus 110 shown in FIG.

  With reference to FIG. 12, a functional configuration of the information processing apparatus that executes each process in the diversity reflection mode will be described. FIG. 12 is a functional block diagram illustrating an example of a functional configuration of the information processing apparatus that executes each process in the diversity reflection mode.

  Referring to FIG. 12, the information processing apparatus 120 corresponding to the diversity reflection mode includes a feature amount extraction unit 111, a sound source type score calculation unit 113, and a digest section determination unit 115 as its functions. Here, the functions of the feature quantity extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are the same as the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  In the information processing device 120, unlike the information processing device 110, information about the feature amount of the speech information calculated by the feature amount extraction unit 111 is also provided to the digest section determination unit 115. The digest section determination unit 115 can determine a digest section in consideration of diversity using information about the feature amount (see the process shown in step S531 of FIG. 14 described later).

(3-4-2. Digest Section Determination Process Procedure)
With reference to FIG. 13 and FIG. 14, a processing procedure of digest section determination processing in the diversity reflection mode in offline processing that can be executed by the information processing apparatus 120 illustrated in FIG. 12 will be described. FIG. 13 and FIG. 14 are flowcharts showing an example of a processing procedure of digest section determination processing in the diversity reflection mode in offline processing.

  Since the diversity reflection mode is to determine the digest section in consideration of diversity within the same sound source type, the sound source type to be included in the digest may be a single sound source type. However, it may be a plurality of sound source types. In FIGS. 13 and 14, as an example, a processing procedure in the case where the digest includes sound composed of a plurality of sound source types is illustrated.

  Here, each process in the digest section determination process in the diversity reflecting mode is the same as that in the digest section determination process in the multiple sound source mode described with reference to FIGS. 10 and 11 except for the process shown in step S531 described later. It is the same as the processing. Therefore, in the description of each process in the digest section determination process in the diversity reflection mode described below, items that are different from each process in the digest section determination process in the multiple sound source mode are mainly described, and overlapping items are Detailed description is omitted. Note that the digest section determining process procedure in the diversity reflecting mode when the digest includes sounds of a plurality of sound source types is the digest section determining process procedure in the single sound source mode shown in FIGS. 5 and 6. 3 corresponds to a process in which a process shown in step S531 described later is performed instead of the process shown in step S229.

  Referring to FIGS. 13 and 14, in the digest section determination process in the diversity reflection mode, the processes in steps S501 to S521 are the same as the processes in steps S401 to S421 shown in FIGS. . Also, the processing after step S523 is processing performed when the score threshold value can no longer be adjusted, similarly to the digest section determination processing in the multiple sound source mode. In the processing after step S523, processing is performed to shorten the total digest section length by deleting a frame from the current digest section or reducing the number of current digest sections.

  Here, in the diversity reflecting mode, when it is determined in step S523 that the digest section length can be shortened for each digest section, the digest section length is deleted by deleting the frame from the digest section having a lower section average score. A series of processes for shortening the total (processes shown in steps S525 to S529) are the same as those in the multiple sound source mode (processes shown in steps S425 to S429).

  On the other hand, in the diversity reflection mode, when it is determined in step S523 that it is impossible to shorten the digest section length in any digest section, the details of the process for reducing the number of digest sections are as follows. Different. Specifically, in the multiple sound source mode, a digest section having a low section average score has been deleted (see the process shown in step S431 in FIG. 11). On the other hand, in the diversity reflection mode, processing for deleting a digest section based on diversity (digest section deletion processing based on diversity) is performed (step S531). After the digest section is deleted, it is determined whether or not the total digest section length substantially matches the digest length (step S533), and the diversity shown in step S531 until the total digest section length substantially matches the digest length. A digest section deletion process based on the is executed.

(3-4-3 Digest section deletion processing based on diversity)
With reference to FIG. 15, the digest section deletion process based on diversity shown in step S531 of FIG. 14 will be described in detail. FIG. 15 is a flowchart illustrating an example of a processing procedure of digest section deletion processing based on diversity in offline processing.

  Referring to FIG. 15, in the digest section deletion process based on diversity in the offline process, first, an average of feature quantity vectors (average feature quantity vector) in each digest section is calculated (step S601).

  Next, the variance of the average feature quantity vector in the n feature quantity spaces in the case of all digest sections and the case where any one digest section is excluded is calculated (step S603).

  Next, the average time of each digest section is calculated (step S605). The average time is calculated as, for example, an intermediate time between the start time and the end time of each digest section.

  Next, the variance of the average time of each of the n digest sections in the case of all digest sections and the case where any one digest section is excluded is calculated (step S607).

  Next, after weighting the variance of the average feature vector and the variance of the average time, the sum is calculated, and the digest section excluded when the amount of reduction from the value in the case of all digest sections is the smallest, It is determined as a digest section to be deleted (step S609). That is, in the process shown in step S609, the digest section having the average feature vector and the average time that have the least influence when not used for calculating the variance of the average feature vector and the average time is determined as the digest section to be deleted. Is done. Thereby, the digest section included in the digest is selected so that the variance of the average feature vector and the average time becomes larger. Finally, the determined digest section is deleted (step S611).

  As above, the processing procedure of the digest section determination process in the diversity reflection mode in the offline process has been described with reference to FIGS. 13 and 14. In addition, the digest section deletion process based on diversity shown in step S531 has been described with reference to FIG.

  As described above, in the diversity reflection mode, the digest section is determined so that the feature amount vector and the time diversity are ensured for the speech classified into the same sound source type. By ensuring the diversity of the feature vector, it is possible to include both of these voices in the digest when they are classified into the same sound source type but actually have another person's voice. In addition, by ensuring the diversity of time, when voices classified into the same sound source type are speaking in places that are separated in time, both voices can be included in the digest. It becomes.

(4. Details of online processing)
(4-1. Overall processing procedure)
With reference to FIG. 16, the procedure of online processing will be described. FIG. 16 is a flowchart illustrating an example of a processing procedure of online processing. The processing procedure shown in FIG. 16 corresponds to the processing procedure of the entire information processing method executed by the information processing apparatus 110 shown in FIG. 1 during online processing.

  In online processing, each time a frame of speech information is newly input, the score of the newly input frame (input frame) is calculated, and a digest section is determined from the speech information based on the score. . That is, in the online processing, a series of processing shown in FIG. 16 is performed while voice information is being input. This is executed each time a frame is newly input, and the digest section information is updated.

  When the score calculation section is not a frame section but includes a plurality of frame sections, the series of processes shown in FIG. 16 can be executed each time a plurality of frames corresponding to the score calculation section are input.

  Referring to FIG. 16, in the online processing, first, the feature amount of the voice information acquired so far is extracted (step S701). In the process shown in step S701, various physical quantities indicating the characteristics of the speech information such as power and spectrum envelope shape are calculated as the feature quantities of the speech information. The process shown in step S701 corresponds to, for example, the process performed by the feature amount extraction unit 111 shown in FIG.

  Next, the sound source type score of the input frame is calculated based on the extracted feature amount (step S703). In the processing shown in step S703, for example, a sound source type score indicating the probability of the sound source type of the sound in the input frame is calculated by a discriminator that identifies the sound source type of the sound according to the feature amount of the sound information. At this time, a plurality of types of sound source type scores such as a voice score, a voice score, and a noise score may be calculated. The process shown in step S703 corresponds to the process performed by, for example, the sound source type score calculation unit 113 shown in FIG.

  If the score calculation section is not a frame section but includes a plurality of frame sections, in step S703, the sound source type score of each frame is smoothed to calculate the sound source type score as the score calculation section. May be.

  Next, a digest section is determined from the audio information based on the calculated sound source type score (step S705). The process shown in step S705 corresponds to the process performed by, for example, the digest section determination unit 115 shown in FIG.

  In the processing shown in step S705, when the time length of the audio information acquired so far is shorter than the digest length (digest time length setting value), the input frame is unconditionally added to the digest. . On the other hand, if the time length of the audio information acquired so far is equal to or longer than the digest length, the input frame is added to the digest, and instead, for example, a frame with a lower score is deleted from the digest. Is done.

  In addition, since the specific processing content in step S705 changes with modes, about the detailed processing content, the following (4-2. Single sound source mode), (4-3. Multiple sound source modes) and (4 -4. Diversity reflection mode) will be described in detail.

  Next, it is determined whether or not the input of voice information has been completed (step S707). If it is determined in step S707 that the input of the voice information has been completed, the digest section information for the determined digest section is output, and the series of processes ends. On the other hand, if it is determined in step S707 that the input of audio information has not been completed, the input of the next frame is awaited (step S709), and the processing after step S701 is performed for the newly input frame. Is repeatedly executed.

  The processing procedure of online processing has been described above with reference to FIG.

(4-2. Single sound source mode)
(4-2-1. Digest Section Determination Process)
With reference to FIG. 17, a processing procedure of digest section determination processing in the single sound source mode in offline processing will be described. FIG. 17 is a flowchart illustrating an example of a processing procedure for digest section determination processing in the single sound source mode in offline processing.

  Referring to FIG. 17, in the digest section determination process in the single sound source mode in the offline process, first, it is determined whether or not the current digest length is shorter than the digest length (step S801). If it is determined in step S801 that the current digest length is shorter than the digest length, the input frame is added to the digest, and the average score (digest average score) as a whole digest is updated (step S803). ). Then, the digest section determination process is terminated, and the next input frame is awaited.

  The processes shown in steps S801 and S803 correspond to a process of adding an input frame to the digest unconditionally when the time length of the audio information input so far is less than the digest length.

  If it is determined in step S801 that the current digest length is greater than or equal to the digest length, the process proceeds to step S805. In step S805, it is determined whether the score of the input frame is greater than or equal to the digest average score. If it is determined in step S805 that the score of the input frame is smaller than the digest average score, the digest section determination process ends without adding the input frame to the digest. That is, a frame with a lower score is not included in the digest.

  On the other hand, if it is determined in step S805 that the score of the input frame is greater than or equal to the digest average score, the input frame is added to the digest and the digest average score is updated (step S807). However, in this case, since the input frame is added to the digest, the current digest length exceeds the digest length by the time length corresponding to one frame. Therefore, after the process shown in step S807, a process of deleting a frame from the digest (frame deletion process) is performed (step S809). In the frame deletion process, for example, a frame with a lower score is deleted from the digest. Details of the frame deletion process shown in step S809 will be described later with reference to FIG.

  When the frame is deleted, the digest average score is updated (step S811), and the digest section determination process is terminated.

(4-2-2. Frame deletion processing)
Here, the details of the frame deletion process shown in step S809 of FIG. 17 will be described with reference to FIG. FIG. 18 is a flowchart showing an example of a processing procedure of frame deletion processing in the single sound source mode in online processing.

  Referring to FIG. 18, in the frame deletion process in the single sound source mode in the online process, first, a digest average score is set as the score threshold (step S901). And the process (high score area determination process) which determines the area (high score area) which has a higher score in a digest as a digest area using the set score threshold value is performed (step S903).

  In the high score section determination process shown in step S903, substantially the same process as the high score section determination process in the offline process shown in step S205 of FIG. 5 is performed, but some processes are different from those in the offline process. Specifically, in the offline processing, high score section determination processing is performed on the entire voice information in order to determine a digest section in the voice information. On the other hand, as described with reference to FIG. 17, in the online processing, the input frame is unconditionally added to the digest until the time length of the voice information acquired so far reaches the digest length. Therefore, before performing the high score section determination process, a provisional digest has already been generated. In online processing, when an input frame is added and the current digest length is longer than the digest length setting value by one frame, a frame with a lower score is found in the digest and a frame to be deleted is determined. In order to do so, a high score section determination process is performed. That is, in the online processing, high score section determination processing is performed for the digest.

  In addition, due to the above circumstances, in the offline processing, the section that is not determined as the high score section in the voice information is naturally not adopted as the digest section. On the other hand, in online processing, even if there is a section that was not determined as a high score section in the digest, the section to be deleted from the digest is a section for one frame, so it is determined as the high score section. It is not possible to delete all sections that have not been deleted from the digest. That is, in the online processing, a section that has not been determined as a high score section as a result of the high score section determination process may remain in the digest. In the following description, a section that is not determined as such a high score section is referred to as a deletion target section. For example, the frame with the lowest score is selected as the frame to be deleted from the deletion target section. Thus, although the deletion target section currently exists in the digest, it can be said that it is a section to be deleted as soon as voice information is input and the digest is updated.

  In online processing, as described above, an input frame is added to the digest and any frame is deleted, so when the scores in each frame in the digest are arranged in chronological order. There may be points where the score becomes discontinuous. In the high score section determination process in the offline process described above, the entire music information is a processing target, and there is no need to consider such discontinuity points in the score, but in the high score section determination process in the online process, Additional processing is required to deal with the discontinuous points.

  Details of the high score section determination process in the online process shown in step S903 will be described later with reference to FIGS.

  When a high score section is determined in step S903, it is determined whether there is a deletion target period that has not been determined as a high score section as a result of the high score section determination process (step S905). If it is determined in step S905 that there is a deletion target section, one frame having a lower score is selected from the deletion target section (step S907). Then, the selected frame is deleted from the digest (step S911).

  On the other hand, if it is determined in step S905 that there is no deletion target section, one frame having a lower score is selected from the digest (ie, S909). Then, the selected frame is deleted from the digest (step S911).

(4-2-3. High Score Section Determination Process)
Here, with reference to FIG. 19 to FIG. 22, the high score section determination process in the online process shown in step S903 of FIG. FIG. 19 is an explanatory diagram for describing high score section determination processing in online processing. 20 to 22 are flowcharts showing an example of the processing procedure of the high score section determination processing in the online processing.

  In FIG. 19, the horizontal axis represents the time of the audio information, the vertical axis represents the score calculated for each frame, and the relationship between the two is plotted. In the high score section determination process, for each frame, it is determined whether to include the frame in the digest section in time series. The meanings of the current frame, the current digest section, the continuous section, and the discontinuous section are the same as those in the high score section determination process in the offline process shown in FIG.

  However, as described above, in the online processing, unlike the offline processing, the processing target is a digest. Therefore, as shown in the figure, by deleting a frame from the digest, there may be a point (discontinuous point) where the score becomes discontinuous when the scores in each frame in the digest are arranged in time series. Also, as described above, as a result of the high score section determination process, the deletion target section that is a section that is not determined as a high score section (that is, a digest section) but exists in the digest is included in the digest. Can exist.

  With reference to FIGS. 20-22, the specific process sequence of the high score area determination process in an online process is demonstrated. The processing procedure of the high score section determination processing in the online processing shown in FIGS. 20 to 22 is that the processing target is not the entire voice information but a digest, and the processing shown in steps S1119 to S1123 described later is added. Except for this, the processing procedure of the high score section determination processing in the offline processing described with reference to FIGS. 8 and 9 is substantially the same. Therefore, in the following description of the processing procedure of the high score section determination process in the online processing, the detailed description of items that overlap with the processing procedure of the high score section determination process in the offline processing is omitted, and the differences are mainly described. Explained.

  20 to 22, in the high score section determination process in the online process, first, the frame index is set to zero (step S1101), and the digest section index is set to zero (ie, S1103). These processes are the same as the processes shown in steps S301 and S303 shown in FIGS.

  The subsequent processes shown in steps S1105 to S1117 are the same as the processes shown in steps S305 to S317 shown in FIGS. Specifically, in step S1105, it is determined whether the score of the current frame is greater than a score threshold. If it is determined that the score of the current frame is equal to or lower than the score threshold value, the process proceeds to step S1119 without including the current frame in the digest section. On the other hand, if it is determined that the score of the current frame is equal to or lower than the score threshold value, the process proceeds to steps S1107 to S1117, and processing for including the current frame in the digest section is performed.

  In steps S1107 to S1117, if the discontinuous section length is smaller than the discontinuous section maximum length, the discontinuous section and the current frame are connected to the current digest section (step S1109). Also, if the discontinuous section length is longer than the discontinuous section maximum length and the continuous section before the discontinuous section is longer than the continuous section minimum length, the continuous section before the discontinuous section is determined as one digest section. At the same time, the digest section index is incremented by 1, and the current frame is set to the start time of the new current digest section (steps S1113 and S1115). If the discontinuous section length is greater than or equal to the discontinuous section maximum length and the continuous section before the discontinuous section is smaller than the minimum continuous section length, the continuous section before the discontinuous section is discarded (that is, the deletion target) The current frame is set as the start time of the current digest section (step S1117). When any one of steps S1109, S1115, and S1117 ends, the process proceeds to step S1119.

  In step S1119, it is determined whether the current frame is a discontinuity point. If it is determined in step S1119 that the current frame is not a discontinuous point, no special processing is performed, and the process proceeds to step S1125.

  On the other hand, if it is determined in step S1119 that the current frame is a discontinuous point, the process proceeds to step S1123. In step S1123, it is determined whether the current digest section length is larger than the continuous section minimum length. That is, in step S1123, it is determined whether or not the digest section immediately before the discontinuity is a significant section from the viewpoint of time length (that is, whether or not the digest section has a time length that allows voice identification). Is done.

  If it is determined in step S1123 that the current digest section length is larger than the minimum continuous section length, the current digest section is considered to be a significant section in terms of time length. The process proceeds to S1125. On the other hand, if it is determined in step S1123 that the current digest section length is less than or equal to the minimum continuous section length, it is considered that the current digest section is not a significant section in terms of time length, and therefore the digest section is discarded ( That is, it is set as a deletion target section), and the process proceeds to step S1125.

  The subsequent processing shown in steps S1125 to S1131 is the same as the processing shown in steps S319 to S325 shown in FIGS. Specifically, in step S1125, it is determined whether the audio information is at the end. If it is determined in step S1125 that the audio information is not the end, the frame index is incremented by 1 (that is, the frame to be processed is set as the next frame) (step S1127), and the processing after step S1105 Is repeatedly executed.

  On the other hand, if it is determined in step S1125 that the voice information is the end, the process proceeds to step S1121, and whether or not the current digest section length is longer than the minimum continuous section length, that is, the last digest section to be processed is determined. From the viewpoint of time length, it is determined whether the interval is significant.

  If it is determined in step S1121 that the current digest section length is larger than the minimum continuous section length, the current digest section is considered to be a significant section in terms of time length. Terminate the process. On the other hand, if it is determined in step S1121 that the current digest section length is less than or equal to the minimum continuous section length, the current digest section is considered to be a section that is not significant in terms of time length, so the digest section is discarded ( In other words, it is set as a deletion target section), and a series of processing ends.

  The processing procedure of the digest section determination process in the single sound source mode in the online process has been described above.

(4-3. Multiple sound source modes)
(4-3-1. Digest Section Determination Process Procedure)
With reference to FIG. 23, the process sequence of the digest area determination process in a multiple sound source mode in an online process is demonstrated. FIG. 23 is a flowchart showing an example of a processing procedure of digest section determination processing in the multiple sound source mode in online processing.

  The digest section determination process in the multiple sound source mode shown in FIG. 23 is a part of the digest section determination process in the single sound source mode described with reference to FIG. The processing shown in step S1205) is changed, and the other processing is substantially the same as the digest section determination processing in the single sound source mode. Therefore, in the description of the digest section determination processing procedure in the multiple sound source mode below, the detailed description of items that overlap with the digest section determination processing procedure in the single sound source mode is omitted and is different. The matter is mainly explained.

  Referring to FIG. 23, in the digest section determination process in the multiple sound source mode, first, it is determined whether or not the current digest length is shorter than the digest length (set value of the digest time length) (step S1201). If it is determined that the current digest length is shorter than the digest length, the input frame is added to the digest and the digest average score is updated (step S1203). The processes shown in steps S1201 and S1203 are the same as the processes in steps S801 and S803 shown in FIG.

  If it is determined in step S1201 that the current digest length is greater than or equal to the digest length, the process proceeds to step S1205. In step S1205, the score of the input frame is compared with the digest average score for each sound source type, and it is determined whether the score of the input frame is greater than or equal to the digest average score for any one of the sound source types. If it is determined in step S1205 that the score of the input frame is smaller than the digest average score for any sound source type, the digest section determination process is terminated without adding the input frame to the digest.

  On the other hand, if it is determined in step S1205 that the score of the input frame is greater than or equal to the digest average score for any of the sound source types, the process proceeds to step S1207. The subsequent processing shown in steps S1207 to S1211 is the same as the processing in steps S807 to S811 shown in FIG. That is, the input frame is added to the digest, and the digest average score is updated (step S1207). Next, a frame deletion process (step S1209) is performed. When a frame is deleted, the digest average score is updated (step S1211), and the digest section determination process is terminated.

(4-3-2. Frame deletion processing)
Here, with reference to FIG. 24, the details of the frame deletion processing shown in step S1209 of FIG. 23 will be described. FIG. 24 is a flowchart showing an example of a processing procedure of frame deletion processing in the multiple sound source mode in online processing.

  Referring to FIG. 24, in the frame deletion process in the multiple sound source mode in the online process, first, a digest average score is set as a score threshold for each sound source type (step S1301). Next, the type digest length is set (step S1303). In the process shown in step S1303, the type digest length may be set by the same method as the process shown in step S405 of the digest section determination process in the multiple sound source mode in the offline process shown in FIG.

  And the process (high score area determination process) which determines the area (high score area) which has a higher score in a digest as a digest area using the set score threshold value is performed (step S1305). The process shown in step S1305 is the same as the process in step S903 shown in FIG. 18, that is, the series of processes shown in FIGS. However, in the frame deletion process in the multiple sound source mode, the high score section determination process is performed for each sound source type.

  When the high score section is determined in step S1305, it is determined whether or not there is a deletion target period in any of the sound source types as a result of the high score section determination processing (step S1307). If it is determined in step S1307 that there is a deletion target section in any of the sound source types, one frame having a lower score is selected from the deletion target section of the sound source type (step S1309). Then, the selected frame is deleted from the digest (step S1315).

  On the other hand, if it is determined in step S1307 that there is no deletion target section for any sound source type, the sound source type whose total digest section length exceeds the type digest length most is selected (step S1311). ). Then, for the selected sound source type, one frame having a lower score is selected (step S1313). Then, the selected frame is deleted from the digest (step S1315).

  The processing procedure of the digest section determination process in the multiple sound source mode in the online process has been described above.

(4-4. Diversity reflection mode)
The processing procedure of the digest section determination process in the diversity reflection mode in the online processing is the same as the processing procedure of the digest section determination processing in the multiple sound source mode in the online processing described with reference to FIG. However, in the diversity reflection mode, the details of the frame deletion processing shown in step S1209 in FIG. 23 are different from those in the multiple sound source mode. Therefore, in the following description of the digest section determination process in the diversity reflection mode in the online process, details of the frame deletion process will be mainly described.

  In online processing, each processing in the diversity reflection mode can be executed by the information processing apparatus 120 shown in FIG. 12, as in offline processing.

(4-4-1. Procedure of frame deletion processing)
With reference to FIG. 25, the process procedure of the frame deletion process in the diversity reflection mode in the online process will be described. FIG. 25 is a flowchart showing an example of a processing procedure of frame deletion processing in the diversity reflection mode in online processing.

  Here, since the diversity reflection mode is to determine the digest section in consideration of diversity within the same sound source type, even if the sound source type to be included in the digest is a single sound source type It may be a plurality of sound source types. In FIG. 25, as an example, a processing procedure in the case of including audio composed of a plurality of sound source types in the digest is illustrated.

  Each process in the frame deletion process in the diversity reflection mode is the same as each process in the frame deletion process in the multiple sound source mode described with reference to FIG. 24 except for the process shown in step S1413 described later. Therefore, in the description of the processing procedure of the frame deletion process in the diversity reflection mode below, items that are different from the processing procedure of the frame deletion process in the multiple sound source mode will be mainly described, and detailed descriptions will be given for the overlapping items. Description is omitted.

  Referring to FIG. 25, in the frame deletion process in the diversity reflection mode in the online process, first, a digest average score is set as a score threshold for each sound source type (step S1401), and then a type digest length is set. (Step S1403). Then, a high score section determination process is performed for each sound source type using the set score threshold (step S1405). These processes are the same as the processes in steps S1301 to S1305 shown in FIG.

  Next, as a result of the high score section determination process, it is determined whether or not there is a deletion target period in any sound source type (step S1407). If it is determined that there is a deletion target section in any of the sound source types, one frame having a lower score is selected from the deletion target section of the sound source type (step S1409), and the selected frame is selected from the digest. It is deleted (step S1415). These processes are the same as the processes in steps S1307, S1309, and S1315 shown in FIG.

  On the other hand, if it is determined in step S1407 that there is no deletion target section for any sound source type, the sound source type whose total digest section length exceeds the type digest length most is selected (step S1411). ). Then, for the selected sound source type, a process of selecting a frame to be deleted in consideration of diversity in the sound source type (deleted frame selection process based on diversity) is performed (step S1413). Then, the selected frame is deleted from the digest (step S1415).

(4-4-2. Deleted frame selection processing based on diversity)
With reference to FIG. 26, the deletion frame selection processing based on diversity shown in step S1413 of FIG. 25 will be described in detail. FIG. 26 is a flowchart showing an example of the processing procedure of the deletion frame selection processing based on diversity in the online processing.

  Referring to FIG. 26, in the deletion frame selection process based on diversity in the online process, first, distribution of feature quantity vectors in n feature quantity spaces in the case of all frames and when one arbitrary frame is excluded. Is calculated (step S1501).

  Next, the variance of the time of the n frames in the case of all frames and the case where any one frame is excluded is calculated (step S1503).

  Next, after weighting the variance of the feature vector and the variance of the time, the sum is calculated, and the frame excluded when the amount of reduction from the value in the case of all frames is the smallest is the frame to be deleted It is determined (step S1505). In other words, in the processing shown in step S1505, a frame having a feature vector and time that have the least influence when not used for calculation of the feature vector and time variance is determined as a frame to be deleted. As a result, the frames to be included in the digest are selected so that the variance of the feature vector and the time becomes larger.

  The processing procedure of the digest section determination process in the diversity reflection mode in the online process has been described above with reference to FIG. In addition, with reference to FIG. 26, the deletion frame selection process based on diversity illustrated in step S1413 of FIG. 25 has been described.

(5. Modifications)
Several modifications of the embodiment described above will be described. In addition, the matters described in the above-described embodiment and each modification described below may be combined with each other as much as possible.

(5-1. Modification Example Provided with Sound Collecting Function)
With reference to FIG. 27, a modification in which the information processing apparatus is provided with a sound collecting function will be described. FIG. 27 is a functional block diagram illustrating an example of a functional configuration of an information processing apparatus according to a modified example in which a sound collecting function is provided.

  Referring to FIG. 27, the information processing apparatus 130 according to the present modification includes, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, a voice sound collection unit 131, Have Here, the functions of the feature quantity extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are the same as the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The sound collection unit 131 is configured by a sound collection device such as a microphone, for example, and has a function of collecting external sound and inputting the sound as audio information to the information processing device 110. The sound collection unit 131 provides sound information related to the collected external sound to the feature amount extraction unit 111. The feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 perform various processes (feature amount extraction processing) according to the above-described embodiment on the audio information provided from the sound collection unit 131. , Sound source type score calculation processing and digest section determination processing).

  Note that the sound collection unit 131 may be configured by a single microphone, or may be configured by a plurality of microphones arranged at different positions. When the sound collection unit 131 includes a plurality of microphones arranged at different positions, the feature amount extraction unit 111 has a plurality of microphones such as a correlation between sound collection positions and a sound source direction. Various feature quantities that can be calculated can be calculated.

  As described above, with reference to FIG. 27, the modification example in which the information processing apparatus is provided with the sound collecting function has been described. As described above, according to the present modification, the information processing apparatus 130 itself has a sound collection function for collecting external sound, and outputs digest section information of sound information related to the collected external sound. Can do. Such an information processing apparatus 130 can be, for example, an IC recorder or a smartphone equipped with application software for recording external sound.

(5-2. Modified example in which a digest generation function is provided)
With reference to FIG. 28, a modification in which a digest generation function is provided in the information processing apparatus will be described. FIG. 28 is a functional block diagram illustrating an example of a functional configuration of an information processing apparatus according to a modification in which a digest generation function is provided.

  Referring to FIG. 28, the information processing apparatus 140 according to the present modification includes, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, an output audio generation unit 141, Have Here, the functions of the feature quantity extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are the same as the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The output sound generation unit 141 includes various processors, and can output a digest of the sound information on the sound output device based on the sound information and the digest section information generated by the digest section determination unit 115. Generate in data format. When generating the digest, the output sound generation unit 141 may appropriately perform various known sound processes in consideration of the user's listening comfort, such as performing a cross-fade process on the joint between the digest sections. . The output audio generation unit 141 outputs audio information (output audio information) corresponding to the generated digest to an audio output device such as a speaker. The digest is output as voice by the voice output device.

  The modification example in which the digest generation function is provided in the information processing apparatus has been described above with reference to FIG. As described above, according to this modification, the information processing apparatus 140 itself has a function of generating a digest, and the generated digest is used for the audio output device or the information processing apparatus 140 provided in the information processing apparatus 140 itself. Output from an external audio output device.

  Note that when the information processing apparatus 140 itself has an audio output device and can reproduce the digest, the information processing apparatus 140 may automatically generate the digest after acquiring the audio information. In this case, the information processing apparatus 140 performs, for example, an operation using a GUI (Graphical User Interface) such as placing a pointer on a file name representing audio information on the display screen, or a simple operation such as a preview operation. The digest may be played. By configuring the information processing apparatus 140 in this way, the user does not have to perform the operation for generating the digest and can listen to the digest with a simple operation. The digest of the voice information can be confirmed with the feeling of confirming the user's convenience, and the convenience for the user is further improved.

(5-3. Modification in which a voice information database is provided)
With reference to FIG. 29, a modified example in which an audio information database is provided in the information processing apparatus will be described. FIG. 29 is a functional block diagram illustrating an example of a functional configuration of an information processing apparatus according to a modification in which a voice information database is provided.

  Referring to FIG. 29, the information processing apparatus 150 according to the present modification includes, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, and an audio information database 151. Have. Here, the functions of the feature quantity extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are the same as the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The voice information database 151 is configured by a storage device such as an HDD, and stores voice information in a database. The feature quantity extraction unit 111 can extract a feature quantity from arbitrary voice information in the voice information database 151 by accessing the voice information database 151. That is, according to the present modification, the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are performed on the voice information stored in the database in the storage unit provided in the information processing device 150. Various processes according to the embodiment described above (feature amount extraction process, sound source type score calculation process, and digest section determination process) are performed.

  The modification example in which the information processing apparatus is provided with the audio information database has been described above with reference to FIG. As described above, according to this modification, the information processing apparatus 150 itself has a database in which voice information is stored, and digest section information of the voice information in the database can be output.

(6. Hardware configuration)
Next, a hardware configuration of the information processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 30 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus according to the present embodiment. 30 can realize the functional configuration of the information processing apparatuses 110, 120, 130, 140, and 150 illustrated in FIGS. 1, 12, and 27 to 29, for example.

  The information processing apparatus 900 includes a CPU 901, a ROM (Read Only Memory) 903, and a RAM (Random Access Memory) 905. The information processing apparatus 900 may include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a communication device 921, a drive 923, and a connection port 925. The information processing apparatus 900 may include a processing circuit called a DSP or an ASIC instead of or together with the CPU 901.

  The CPU 901 functions as an arithmetic processing unit and a control unit, and controls all or a part of the operation in the information processing apparatus 900 according to various programs recorded in the ROM 903, the RAM 905, the storage apparatus 919, or the removable recording medium 929. The ROM 903 stores programs used by the CPU 901, calculation parameters, and the like. The RAM 905 temporarily stores programs used in the execution of the CPU 901, parameters at the time of execution, and the like. The CPU 901, the ROM 903, and the RAM 905 are connected to each other by a host bus 907 configured by an internal bus such as a CPU bus. Further, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909. The CPU 901 can configure, for example, the feature amount extraction unit 111, the sound source type score calculation unit 113, the digest section determination unit 115, and the output audio generation unit 141 in the above-described embodiment.

  The host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

  The input device 915 is configured by a device operated by a user, such as a mouse, a keyboard, a touch panel, a button, a switch, and a lever. The input device 915 may be, for example, a remote control device (so-called remote controller) that uses infrared rays or other radio waves, or an external connection device such as a mobile phone or a PDA that supports the operation of the information processing device 900. It may be 931. Furthermore, the input device 915 includes an input control circuit that generates an input signal based on information input by the user using the above-described operation means and outputs the input signal to the CPU 901, for example. A user of the information processing apparatus 900 can input various data and instruct a processing operation to the information processing apparatus 900 by operating the input device 915. In the present embodiment, for example, an instruction to start digest section determination processing, a mode switching instruction, or the like may be input to the information processing apparatuses 110, 120, 130, 140, and 150 via the input device 915.

  Further, the input device 915 may be a microphone that picks up surrounding sounds and inputs the surrounding sounds to the information processing apparatus 900 as sound information. When the input device 915 is a microphone, the input device 915 can constitute the sound collection unit 131 in the above-described embodiment.

  The output device 917 is a device that can notify the user of the acquired information visually or audibly. Examples of such devices include CRT display devices, liquid crystal display devices, plasma display devices, EL display devices, display devices such as lamps, audio output devices such as speakers and headphones, printer devices, and the like. For example, the output device 917 outputs results obtained by various processes performed by the information processing apparatus 900. Specifically, the display device visually displays results obtained by various processes performed by the information processing device 900 in various formats such as text, images, tables, and graphs. On the other hand, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, and the like into an analog signal and outputs it aurally. In the present embodiment, for example, a digest of voice information generated by the information processing apparatus 140 may be output via the voice output apparatus. Further, a display related to a GUI for inputting various instructions via the input device 915 may be displayed on the display device.

  The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 includes, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like. The storage device 919 can constitute, for example, the audio information database 151 in the above-described embodiment.

  The communication device 921 is a communication interface configured with, for example, a communication device for connecting to a communication network (network) 927. The communication device 921 is, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). Further, the communication device 921 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communication, or the like. The communication device 921 can transmit and receive signals and the like according to a predetermined protocol such as TCP / IP, for example, with the Internet or other communication devices. The network 927 connected to the communication device 921 is configured by a wired or wireless network, and may be, for example, the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, or the like. In the present embodiment, for example, the information processing apparatuses 110, 120, 130, 140, and 150 receive information, such as voice information, digest section information, and output voice information, via the communication apparatus 921. Various information that is input and output of 140 and 150 may be exchanged with an external device.

  The drive 923 is a recording medium reader / writer, and is built in or externally attached to the information processing apparatus 900. The drive 923 reads information recorded on a removable recording medium 929 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 905. The drive 923 can also write information to a removable recording medium 929 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. The removable recording medium 929 is, for example, a DVD medium, an HD-DVD medium, a Blu-ray (registered trademark) medium, or the like. Further, the removable recording medium 929 may be a compact flash (registered trademark) (CompactFlash: CF), a flash memory, an SD memory card (Secure Digital memory card), or the like. The removable recording medium 929 may be, for example, an IC card (Integrated Circuit card) on which a non-contact IC chip is mounted, an electronic device, or the like. In the present embodiment, for example, various types of information processed by the information processing apparatuses 110, 120, 130, 140, and 150 are read from the removable recording medium 929 by the drive 923 or written to the removable recording medium 929. Also good.

  The connection port 925 is a port for directly connecting a device to the information processing apparatus 900. Examples of the connection port 925 include a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, and the like. As another example of the connection port 925, there are an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, and the like. By connecting the external connection device 931 to the connection port 925, the information processing apparatus 900 acquires various data directly from the external connection device 931 or provides various data to the external connection device 931. In the present embodiment, for example, various types of information processed by the information processing apparatuses 110, 120, 130, 140, and 150 are acquired from the external connection device 931 via the connection port 925 or output to the external connection device 931. May be.

  Heretofore, an example of the hardware configuration capable of realizing the functions of the information processing apparatus 900 according to the present embodiment has been shown. Each component described above may be configured using a general-purpose member, or may be configured by hardware specialized for the function of each component. Therefore, it is possible to change the hardware configuration to be used as appropriate according to the technical level at the time of carrying out this embodiment.

  Note that a computer program for realizing each function of the information processing apparatus 900 according to the present embodiment as described above can be produced and mounted on a PC or the like. In addition, a computer-readable recording medium storing such a computer program can be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

(7. Summary)
As described above, according to the present embodiment, the sound source type score indicating the probability of the sound source type of the sound included in the sound information is calculated, and the sound information is extracted from the sound information based on the sound source type score. The digest sections constituting the digests are determined. Therefore, for example, you want to include only the music in the digest, want to include only the voice of the person in the digest, or want to include the music and the voice of the person in a well-balanced digest. Is possible. Therefore, user convenience can be further improved.

  In addition, by setting the mode and appropriately adjusting the sound source type of the sound included in the digest, it is possible to generate a digest more in line with the user's intention. For example, by appropriately setting the mode, such as setting the ratio of the voice related to the noise score in the digest to a low value in the multiple sound source mode, it is possible to generate a digest with reduced noise and easier for the user to hear. Is possible.

  Here, generally, with respect to video information, for example, by displaying a thumbnail, an outline of the video information can be visually notified to the user. However, audio recording devices that mainly acquire audio information instead of video information (for example, IC recorders, smartphones equipped with recording application software, wearable devices in situations where camera functions are not installed or camera functions cannot be used, etc. ), The file name of the audio information, the sound collection date and time, etc. can be displayed visually, but it is difficult for the user to visually grasp the outline of the audio information from these information. It is. Even when audio information and video information are included, the display screen cannot be viewed depending on the situation, for example, when the backlight of the display screen is turned on during an event in a dark room. Visual confirmation may not be possible.

  In such a case, in order to grasp the contents of audio information (or audio information and video information), the user needs to actually listen to the audio information. However, when the time length of the voice information is long, listening to the voice information in order to confirm the contents has a large time load and is a heavy burden on the user.

  On the other hand, according to the present embodiment, as described above, it is possible to create a digest of voice information in accordance with a user's request. Therefore, for example, it is possible to grasp the contents of the audio information only by listening to a digest of several seconds, and it is possible to greatly reduce the time required to confirm the contents, which has taken a long time until now.

  In addition, according to the present embodiment, for example, the digest information is automatically digested with respect to the acquired sound information by the apparatus main body that records the sound, or another apparatus that manages the sound information after moving to the storage. May be generated. Further, when a digest is automatically generated for the acquired audio information, for example, an operation using a GUI such as placing a pointer on a file name representing the audio information on the display screen, a preview operation, etc. The digest may be reproduced by a simple operation. Thereby, the user can confirm the digest more easily without performing troublesome operations.

  Further, the technology according to the present embodiment can be suitably applied to a use for analyzing so-called big data. For example, by applying the technology according to the present embodiment to call records collected at call centers, investigation agencies, etc., and generating a digest of call records, the contents of a huge amount of call records can be confirmed in a shorter time It becomes possible to do. Therefore, analysis of the call record becomes easier.

  Further, even when the video information is included together with the audio information, there may be a situation where it is difficult to grasp the contents by a visual method using a thumbnail or the like based on the video information. For example, if there are multiple files that differ greatly only in the audio part for similar images, or if video information is not available due to implementation restrictions such as the processing speed of the device, it is a video from a fixed point camera etc. Such a situation may correspond to a case where a sound source is not shown in the video (that is, a speaker cannot be identified). The technique according to the present embodiment can be suitably applied even when video information cannot be used effectively for grasping the content.

  Furthermore, the technique according to the present embodiment is effective in easily comprehending the content of audio information that is a material before editing, even in the operation of editing audio information, such as when editing a moving image. For example, in recent years, there are services that generate and provide photographs with audio information that combine still images and audio. The technique according to the present embodiment can be effectively utilized when editing a sound part when generating a file having a format in which still images and sound are combined.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

  In addition, the effects described in the present specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

  Here, in this specification, expressions such as “below” or “greater than” are used when comparing the score with a threshold value in the determination process in the processing procedure of each process. The expression is merely an example, and does not limit the boundary condition in the determination process. In this embodiment, when a value such as a score is equal to a threshold value, how to determine the magnitude relationship may be arbitrarily set. In this specification, the expression “below” can be read as appropriate with the expression “less than”, and the expression “greater than” can be read as appropriate with the expression “above”. Is possible.

The following configurations also belong to the technical scope of the present disclosure.
(1) A sound source type score calculation unit that calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information, and the sound information is selected from the sound information based on the calculated sound source type score. And a digest section determination unit that determines a digest section that constitutes the digest of the information processing apparatus.
(2) The information processing apparatus according to (1), wherein the sound source type score includes at least one of a music score indicating music, a voice score indicating human voice, and a noise score indicating noise.
(3) The voice score includes a male voice score indicating a male voice, a female voice score indicating a female voice, a child voice score indicating a voice of a child, and a specific person who is producing the voice. The information processing apparatus according to (2), further including at least one of specific voice scores to be shown.
(4) The information processing according to any one of (1) to (3), wherein the sound source type score calculation unit calculates the sound source type score based on a feature amount indicating a feature of the audio information. apparatus.
(5) The feature quantity is at least one of power, spectrum envelope shape, number of zero crossings, pitch, MFCC, correlation between sound collection positions, and a physical quantity indicating characteristics of sound source direction for the voice information. The information processing apparatus according to (4), including:
(6) The digest section determination unit determines a sound source type of the voice to be included in the digest based on a mode of the digest to be generated, and the sound source type score related to the determined sound source type is included in the sound information. The information processing apparatus according to any one of (1) to (5), wherein a higher section is determined as the digest section.
(7) The mode is a single sound source mode for generating the digest so as to include only the sound of a single sound source type, and the digest is generated so as to include the sound of a plurality of sound source types at a predetermined ratio. The sound source mode is selected from at least one of a plurality of sound source modes and a diversity reflection mode for generating the digest so that various sounds are included from the sounds classified into the same sound source type, The information processing apparatus according to 6).
(8) When the mode is the single sound source mode, the digest section determination unit determines a section having a higher sound source type score related to one designated sound source type as the digest section. The information processing apparatus according to (7).
(9) When the mode is the multiple sound source mode, the digest section determination unit sets the time length of the voice included in the digest for each sound source type, and the sound source type score is set for each sound source type. The information processing apparatus according to (7), wherein the section that is a higher section and is substantially equal to the time length for each sound source type for which the total length of the section is set is determined as the digest section.
(10) When the mode is the diversity reflecting mode, the digest section determination unit determines the variation in the feature amount indicating the feature of the audio information within the same sound source type and the same sound source type. The information processing apparatus according to (7), wherein a variation in time at which the sound is generated is calculated, and the digest section is determined so that the variation in the feature amount and the variation in the time become larger.
(11) The digest section determination unit includes a first section in which the sound source type score is higher than a predetermined threshold and a second section in which the sound source type score is lower than a predetermined threshold. And when the time length of the second section is shorter than a predetermined time, the digest section is determined to include both the first and second sections, The information processing apparatus according to any one of (6) to (10).
(12) If the time length of the first section in which the sound source type score is higher than a predetermined threshold is a length that cannot be recognized as speech by a person, the digest section determination unit The information processing apparatus according to any one of (6) to (11), wherein the digest section is determined so as not to include the section.
(13) The sound source type score calculation unit calculates the sound source type score for the voice information for which all has been acquired in advance, and the digest section determination unit has the sound information for which all has been acquired in advance. The information processing apparatus according to any one of (1) to (12), wherein a digest is generated.
(14) The sound source type score calculation unit newly acquires voice information having a duration corresponding to a score calculation section composed of a length of time equal to or shorter than the digest section for the voice information that has just been acquired. Each time the score is calculated, the sound source type score is calculated for each of the score calculation sections, and the digest section determination unit generates the voice information while updating the digest as needed. The information processing apparatus according to any one of (1) to (12).
(15) If the time length of the audio information acquired so far is shorter than the set value of the time length of the digest, the digest section determination unit determines the newly acquired audio information. When the time length of the audio information acquired so far in addition to the digest is equal to or greater than the set time length of the digest, the newly acquired audio for the score calculation section The information processing apparatus according to (14), wherein information is added to the digest and a section having a time length corresponding to the score calculation section and having a lower sound source type score is deleted from the digest. .
(16) A sound collecting unit that picks up external sound, and the sound information is sound information related to the external sound collected by the sound collecting unit. The information processing apparatus according to any one of the above.
(17) A storage unit for storing the voice information in a database is further provided, and the sound source type score calculation unit calculates a sound source type score for the voice information in the database, and determines the digest section The information processing apparatus according to any one of (1) to (15), wherein the unit determines the digest section for the voice information stored in a database.
(18) Output audio generation for generating a digest of the audio information in a data format that can be output by an audio output device based on the audio information and information on the digest interval determined by the digest interval determination unit The information processing apparatus according to any one of (1) to (17), further including a unit.
(19) The processor calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information, and based on the calculated sound source type score, the processor calculates the sound information from the sound information. Determining a digest section that constitutes the digest.
(20) A function of calculating a sound source type score indicating a probability of a sound source type of sound included in the sound information in a processor of the computer, and the sound from the sound information based on the calculated sound source type score. And a function for determining a digest section constituting a digest of information.

110, 120, 130, 140, 150 Information processing device 111 Feature amount extraction unit 113 Sound source type score calculation unit 115 Digest section determination unit 131 Audio sound collection unit 141 Output audio generation unit 151 Audio information database (DB)

Claims (20)

A sound source type score calculating unit that calculates a sound source type score indicating the probability of the sound source type of the audio included in the audio information;
Based on the calculated sound source type score, a digest section determination unit that determines a digest section that constitutes a digest of the voice information from the voice information;
An information processing apparatus comprising: The sound source type score includes at least one of a music score indicating the likelihood of music, a voice score indicating the likelihood of human voice, and a noise score indicating the likelihood of noise.
The information processing apparatus according to claim 1. The voice score is a male voice score indicating the voice like a man, a female voice score indicating the voice like a woman, a child voice score indicating the voice like a child, and a specific voice indicating the particular character who is producing the voice Further comprising at least one of the scores,
The information processing apparatus according to claim 2. The sound source type score calculating unit calculates the sound source type score based on a feature amount indicating a feature of the audio information;
The information processing apparatus according to claim 1. The feature amount includes at least one of power, spectrum envelope shape, number of zero crossings, pitch, MFCC, correlation between sound collection positions, and physical quantity indicating sound source azimuth characteristics for the audio information.
The information processing apparatus according to claim 4. The digest section determination unit determines a sound source type of the voice to be included in the digest based on a mode of the digest to be generated, and a section having a higher sound source type score related to the determined sound source type in the voice information Is determined as the digest interval,
The information processing apparatus according to claim 1. The mode includes a single sound source mode for generating the digest so as to include only the sound of a single sound source type, and a multiple sound source mode for generating the digest so as to include the sound of a plurality of sound source types at a predetermined ratio. And a diversity reflection mode for generating the digest so that various voices are included from the voices classified into the same sound source type.
The information processing apparatus according to claim 6. When the mode is the single sound source mode, the digest section determination unit determines a section having a higher sound source type score related to one designated sound source type as the digest section.
The information processing apparatus according to claim 7. When the mode is the multiple sound source mode, the digest section determination unit sets a time length of the voice included in the digest for each sound source type, and a section in which the sound source type score is higher for each sound source type And determining, as the digest section, the section such that the total length of the section is approximately equal to the time length for each sound source type set.
The information processing apparatus according to claim 7. When the mode is the diversity reflection mode, the digest section determination unit determines variations in feature quantities indicating features of the audio information within the same sound source type and the audio within the same sound source type. Calculating the variation of the time at which is issued, and determining the digest section so that the variation of the feature amount and the variation of the time become larger.
The information processing apparatus according to claim 7. The digest section determination unit continuously includes a first section in which the sound source type score is higher than a predetermined threshold and a second section in which the sound source type score is lower than a predetermined threshold. And when the time length of the second section is shorter than a predetermined time, the digest section is determined so as to include both the first and second sections.
The information processing apparatus according to claim 6. The digest section determination unit determines the first section when the time length of the first section in which the sound source type score is higher than a predetermined threshold is a length that cannot be recognized as speech for a person. Determine the digest interval so that it does not include,
The information processing apparatus according to claim 6. The sound source type score calculation unit calculates the sound source type score for the audio information that has been acquired in advance,
The digest section determination unit generates the digest of the audio information that has been acquired in advance.
The information processing apparatus according to claim 1. The sound source type score calculation unit, for the sound information that has just been acquired, newly acquires sound information having a time length corresponding to a score calculation section consisting of a length of time equal to or shorter than the digest section. And calculating the sound source type score for each of the score calculation sections,
The digest section determination unit generates the voice information while updating the digest as needed while the voice information is acquired.
The information processing apparatus according to claim 1. When the time length of the audio information acquired so far is shorter than the set value of the time length of the digest, the digest section determination unit adds the newly acquired audio information to the digest. Add
When the time length of the voice information acquired so far is equal to or longer than the set time length of the digest, the voice information for the newly obtained score calculation section is added to the digest And deleting a section of the time length corresponding to the score calculation section from the digest and having a lower sound source type score,
The information processing apparatus according to claim 14. A sound collecting unit for collecting external sound;
The audio information is audio information related to external audio collected by the audio collection unit,
The information processing apparatus according to claim 1. A storage unit for storing the voice information in a database;
The sound source type score calculation unit calculates a sound source type score for the voice information stored in a database,
The digest section determination unit determines the digest section for the voice information stored in a database.
The information processing apparatus according to claim 1. Based on the voice information and information on the digest section determined by the digest section determination section, an output voice generation section that generates a digest of the voice information in a data format that can be output by a voice output device, In addition,
The information processing apparatus according to claim 1. The processor calculates a sound source type score indicating the probability of the sound source type of the audio included in the audio information;
Determining a digest section that constitutes a digest of the voice information from the voice information based on the calculated sound source type score;
Including an information processing method. Computer processor,
A function of calculating a sound source type score indicating the probability of the sound source type of the audio included in the audio information;
A function for determining a digest section constituting a digest of the voice information from the voice information based on the calculated sound source type score;
A program that realizes