JP6683231B2 - Information processing apparatus and information processing method - Google Patents

Description

The present disclosure relates to an information processing device and an information processing method .

  With respect to information having a predetermined time length such as audio information and video information, there is a demand for grasping the outline of the content without watching all the content. Therefore, for example, in Patent Document 1, a climax part that is a noteworthy scene in the audio information is detected from the feature amount indicating the characteristics of the audio information, and an index is given to the climax part in the audio information. Techniques for doing so are disclosed. According to this technique, it is possible to generate a digest of the audio information in which only the climax is extracted by reproducing only the part to which the index is attached from the audio information.

JP 2004-191780 A

  Here, for example, considering the generation of a digest of audio information that records the state of a meeting, we want to include scenes that are exciting to grasp the outline of the meeting, that is, scenes in which the discussion is confusing. On the other hand, there may be a desire to generate a digest so that the voices of as many people as possible are included in order to grasp the participants of the conference. As described above, there are various requests made by the user for the digest depending on the purpose. Since the technique described in Patent Document 1 is specialized in detecting a swelling portion, it is considered difficult for the technique described in Patent Document 1 to meet such various user requests. .

  Therefore, the present disclosure proposes a new and improved information processing device capable of further improving the convenience of the user.

  According to the present disclosure, based on the feature amount extraction unit that extracts the feature amount of the voice included in the voice information and the sound source type calculated according to the feature amount, the voice information of the voice information is selected from among the voice information. A digest section determining unit for determining a digest section constituting a digest, the digest section determining unit determines a sound source type of the voice to be included in the digest based on a preset mode, the preset The information processing apparatus having a single sound source mode for generating the digest so as to include at least the sound source type specified to be preferentially included in the digest is provided.

  Further, according to the present disclosure, based on the feature amount extraction unit that extracts the feature amount of the voice included in the voice information and the sound source type calculated according to the feature amount, the voice A digest section determining unit for determining a digest section constituting a digest of information, the digest section determining unit determines a sound source type of the voice included in the digest based on a preset mode, the pre- An information processing apparatus is provided that has a plural sound source mode in which the digest is generated so that the mode to be set includes at least the sound of plural sound source types at a predetermined ratio.

  Further, according to the present disclosure, based on the feature amount extraction unit that extracts the feature amount of the voice included in the voice information and the sound source type calculated according to the feature amount, the voice A digest section determining unit for determining a digest section constituting a digest of information, the digest section determining unit determines a sound source type of the voice included in the digest based on a preset mode, the An information processing apparatus having a diversity reflection mode in which the digest is generated so that various voices are included in the voices classified into at least the same sound source type is provided as the preset mode. It

  According to the present disclosure, a feature amount of a voice included in voice information is extracted, and a digest forming a digest of the voice information is extracted from the voice information based on a sound source type calculated according to the feature amount. The section is determined. Therefore, it is possible to generate a digest that meets various user requests according to the sound source type. Therefore, the convenience of the user can be further improved.

  According to the present disclosure as described above, it is possible to further improve the convenience of the user. Note that the above effects are not necessarily limited, and in addition to or in place of the above effects, any of the effects shown in this specification, or other effects that can be grasped from this specification. May be played.

It is a functional block diagram which shows an example of a functional structure of the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the sound source classification score calculated by the sound source classification score calculation part. It is explanatory drawing for demonstrating the relationship between audio | voice information and a digest. It is a flow figure showing an example of the processing procedure of offline processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in single sound source mode in offline processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in single sound source mode in offline processing. It is explanatory drawing for demonstrating the high score area determination process in an offline process. It is a flow figure showing an example of a processing procedure of high score section determination processing in off-line processing. It is a flow figure showing an example of a processing procedure of high score section determination processing in off-line processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in a multi-source mode in offline processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in a multi-source mode in offline processing. It is a functional block diagram which shows an example of a functional structure of the information processing apparatus which performs each process in diversity reflection mode. It is a flow figure showing an example of a processing procedure of digest section determination processing in diversity reflection mode in off-line processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in diversity reflection mode in off-line processing. It is a flow figure showing an example of a processing procedure of digest section deletion processing based on diversity in offline processing. It is a flow figure showing an example of a processing procedure of online processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in single sound source mode in offline processing. It is a flow figure showing an example of a processing procedure of frame deletion processing in single sound source mode in online processing. It is explanatory drawing for demonstrating the high score area determination process in an online process. It is a flow figure showing an example of a processing procedure of high score section determination processing in online processing. It is a flow figure showing an example of a processing procedure of high score section determination processing in online processing. It is a flow figure showing an example of a processing procedure of high score section determination processing in online processing. It is a flow figure showing an example of a processing procedure of digest section determination processing in a multiple sound source mode in online processing. It is a flow figure showing an example of a processing procedure of frame deletion processing in a multiple sound source mode in online processing. It is a flow figure showing an example of a processing procedure of frame deletion processing in diversity reflection mode in online processing. It is a flow figure showing an example of the processing procedure of deletion frame selection processing based on diversity in online processing. It is a functional block diagram which shows an example of a functional structure of the information processing apparatus which concerns on the modification provided with a voice sound collection function. It is a functional block diagram which shows an example of a functional structure of the information processing apparatus which concerns on the modification provided with the digest generation function. It is a functional block diagram which shows an example of a functional structure of the information processing apparatus which concerns on the modification provided with a voice information database. It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus which concerns on this embodiment.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, constituent elements having substantially the same functional configuration are designated by the same reference numerals, and duplicate description will be omitted.

The description will be given in the following order.
1. Examination of existing technologies 2. Device configuration 3. Details of offline processing 3-1. Overall processing procedure 3-2. Single sound source mode 3-2-1. Processing procedure of digest section determination processing 3-2-2. High score section determination process 3-3. Multiple sound source mode 3-3-1. Processing procedure of digest section determination processing 3-4. Diversity reflection mode 3-4-1. Functional configuration 3-4-2. Processing procedure of digest section determination processing 3-4-4. 3. Digest section deletion processing based on diversity Details of online processing 4-1. Overall processing procedure 4-2. Single sound source mode 4-2-1. Digest section determination process 4-2-2. Frame deletion process 4-2-3. High score section determination process 4-3. Multiple sound source mode 4-3-1. Processing procedure of digest section determination processing 4-3-2. Frame deletion process 4-4. Diversity reflection mode 4-4-1. Processing procedure of frame deletion processing 4-4-2. 4. Deleted frame selection processing based on diversity Modified example 6. Hardware configuration 7. Conclusion

(1. Study on existing technology)
Prior to describing a preferred embodiment of the present disclosure, the results of the present inventors 'consideration of existing general techniques will be described, and the background of the present inventors' idea of the present disclosure will be described.

  Generally, in order to easily grasp the outline of audio information, video information, etc., a technique for generating the digest has been developed. In particular, many techniques relating to video information have been proposed, such as generating a digest of a recorded television program. However, many techniques for generating a digest from video information assume a multimodal framework that uses both the feature amount calculated from video and the feature amount calculated from audio. It can be considered that it is more difficult to appropriately generate the digest of the audio information based on only the audio information, as compared with the video information having a large amount of information.

  For example, as a general method of generating a digest of audio information, a method of simply extracting the beginning portion, an intermediate portion, and an end portion of the audio information to generate a digest, or a method of extracting a high volume section to generate a digest Etc. are possible. Alternatively, some existing IC recorders have a function of reproducing the first 5 seconds of the selected audio file. However, in the method of extracting a predetermined section regardless of the content of voice information, it is highly likely that significant information is not included in the digest. Further, in the method based on the volume, there is a possibility that the digest may include a segment that is not necessarily useful, such as a segment having a large amount of noise.

  Further, as a technique for generating a digest of voice information, there is, for example, the technique described in Patent Document 1 above. However, as described above, the technique is specialized in extracting a swelling portion and generating a digest. Since the content that the user wants to grasp in the digest is not necessarily limited to the excitement part, it is difficult for the technique to meet various user's demands for the digest.

  Heretofore, the results of the present inventors' examination of existing general techniques have been described. As described above, in the technique of generating a digest of voice information, a more convenient technique that can meet various needs of users has been desired. The present inventors have earnestly studied a technique capable of further improving the convenience of the user based on the above-described examination result of the existing technique, and have arrived at one embodiment of the present disclosure described below. . Hereinafter, a preferred embodiment of the present disclosure conceived by the present inventors will be described in detail.

(2. Device configuration)
A functional configuration of an information processing apparatus according to an embodiment of the present disclosure will be described with reference to FIG. 1. FIG. 1 is a functional block diagram showing an example of the functional configuration of the information processing apparatus according to this embodiment.

  Referring to FIG. 1, the information processing apparatus 110 according to the present embodiment has, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, and a digest section determination unit 115. The information processing apparatus 110 receives arbitrary audio information as input, determines a digest section that is a section constituting a digest of the audio information in the audio information, and outputs information about the digest section (digest section information). It is a device that does.

  The input source of the voice information to the information processing device 110 may be arbitrary. For example, the voice information input to the information processing device 110 may be stored in a storage unit (not shown) provided in the information processing device 110, or may be different from the information processing device 110. It may be input from an external device. Alternatively, when the information processing device 110 has a sound pickup unit that picks up an external voice, the voice information may be input via the sound pickup unit (for such a configuration, see (5- This will be described in detail in 1. Modified example in which a voice pickup function is provided).

  The feature amount extraction unit 111 extracts a feature amount of voice information. As the feature quantity, various physical quantities indicating the characteristics of audio information can be calculated. For example, as the feature amount, power, spectrum envelope shape, number of zero crossings, pitch (fundamental frequency), MFCC (Mel-Frequency Cepstrum Coefficients), or the like may be calculated. Further, in the case of voice information picked up by microphones arranged at mutually different positions, the correlation between the picked-up positions may be calculated as the feature amount. The sound source azimuth may be further calculated based on the correlation. The feature amount extraction unit 111 can calculate at least one of these feature amounts.

  As the process of extracting the feature amount from the voice information performed by the feature amount extraction unit 111, various methods generally used in the voice information analysis process may be used. Detailed description of the processing is omitted. Further, the feature amount calculated by the feature amount extraction unit 111 is not limited to those listed above, and the feature amount extraction unit 111 calculates various feature amounts that can be generally calculated in the analysis process of voice information. You may

  The feature amount calculated by the feature amount extraction unit 111 can be expressed as a vector (feature amount vector) in a space (feature amount space) having a dimension of the number of types of the calculated feature amount, for example. The feature amount extraction unit 111 provides the sound source type score calculation unit 113 with information about the calculated feature amount (that is, information about the feature amount vector).

  The sound source type score calculation unit 113 calculates a sound source type score indicating the probability of the sound source type of the voice included in the voice information, based on the feature amount of the voice information extracted by the feature amount extraction unit 111. Here, the sound source classification is a classification of sound sources of voice into several types. For example, the sound source type score includes a music score indicating the likelihood of music, a voice score indicating the likelihood of human voice, and / or a noise score indicating the likelihood of noise. Further, when the voice score is calculated, in more detail, a male voice score indicating a male voice-likeness, a female voice score indicating a female voice-likeness, a child voice score indicating a childlike voice-likeness, and / or the above A specific voice score indicating the specificity of a specific person making a sound may be calculated.

  The sound source type score calculation unit 113 calculates at least one of the sound source type scores described above for each predetermined section in the audio information. Hereinafter, the time unit in which the sound source type score calculation unit 113 calculates the sound source type score is referred to as a score calculation section. The score calculation section may be a section corresponding to a frame, for example.

  For the calculation of the sound source type score, various discriminators generally used in the analysis process of the voice information may be used. For example, the classifier can calculate each sound source type score by machine learning according to the feature amount vector of the voice information to be analyzed, that is, according to the coordinates in the feature amount space. When it is difficult to perform machine learning in the classifier in advance, the sound source type score calculation unit 113 calculates the sound source type score according to the distance from the average speaker characteristic derived from past calculations. be able to. For example, the sound source type score calculation unit 113 outputs a higher value as the sound source type score as the distance to the past speaker characteristic is larger.

  FIG. 2 shows an example of the sound source type score calculated by the sound source type score calculation unit 113. FIG. 2 is a diagram showing an example of a sound source type score calculated by the sound source type score calculation unit 113. In FIG. 2, the horizontal axis represents time in the audio information, the vertical axis represents the sound source type score calculated for each score calculation section, and the relationship between the two is plotted. In the example illustrated in FIG. 2, the sound source type score calculation unit 113 calculates three types of sound source type scores.

  The sound source type score calculation unit 113 provides the digest section determination unit 115 with information about the sound source type score calculated for each score calculation section.

  Based on the sound source type score calculated by the sound source type score calculation unit 113, the digest section determination unit 115 determines a digest section, which is a time section that constitutes a digest of the voice information, from the voice information. Here, the relationship between the voice information and the digest will be described with reference to FIG. FIG. 3 is an explanatory diagram for explaining the relationship between voice information and a digest.

  As shown in FIG. 3, the digest is composed of at least one time section in the audio information. In the example shown in the figure, four time intervals (digest intervals 1 to 4) are determined as time intervals (digest intervals) that make up the digest in the audio information, and the digest intervals are combined to form a digest. It is configured.

  In the following description, the time length of each digest section is referred to as the digest section length. Also, the time length of the digest is called the digest length. The digest length is set in advance by the user, the designer of the information processing apparatus 110, or the like as the length of the desired digest, such as one minute. The digest section is determined so that the total digest section length substantially matches the digest length.

  The digest section determination unit 115 basically determines a time section having a higher sound source type score in the music information as the digest section. However, as shown in FIG. 2, a plurality of sound source type scores can be calculated independently for audio information. Therefore, it is necessary to set in advance which sound source type score is used to determine the digest section.

  Here, which sound source type score is preferentially used to determine the digest section may be various according to the user's request. For example, for a user who wants to extract only a male voice from the voice information, it is desirable to pay attention to the male voice score and determine a time period having a higher male voice score as the digest period. . Alternatively, for a user who wants to uniformly extract various voices included in voice information, it is desirable that a time section having a high sound source type score is determined as a digest section in good balance for each sound source type. .

  Therefore, in the present embodiment, the mode of the generated digest is set, and the digest section determination unit 115 performs the process of determining the digest section according to the set mode. A predetermined mode may be set in advance, or the mode may be arbitrarily switched according to an operation input by the user via the input unit of the information processing apparatus 110 (not shown). The mode information indicating the set mode is input to the digest section determination unit 115. The digest section determination unit 115 determines the sound source type of the sound to be included in the digest based on the set mode, and determines, as the digest section, a section having a higher sound source type score related to the determined sound source type in the audio information. can do.

  For example, as the mode, a single sound source mode in which a digest is generated so as to include only sound of a single sound source type, a multiple sound source mode in which a digest is generated so as to include sound of a plurality of sound source types at a predetermined ratio, and Or, there is a diversity reflection mode for generating a digest so that various voices are included from among the voices classified into the same sound source type.

  When the mode is the single sound source mode, the mode information includes information indicating that the sound source type to be preferentially included in the digest is designated. When the mode is the single sound source mode, the digest section determination unit 115 determines a section having a higher sound source type score associated with one designated sound source type as a digest section.

  When the mode is the multiple sound source mode, the mode information includes information indicating that the ratio of the sound source types included in the digest is designated. When the mode is the multiple sound source mode, the digest section determination unit 115 sets the time length of the voice included in the digest for each sound source type based on the designated ratio, and the sound source type score is set for each sound source type. A section that is a higher section and whose total length is less than or equal to the set time length for each sound source type is determined as the digest section.

  The ratio can be appropriately designated by the user as the mode information. With this, the user can select the sound source type to be preferentially included in the digest, in accordance with his / her request. On the contrary, it is also possible to set a low value for the ratio of voice types that should not be included in the digest, such as noise.

  The ratio of the sound source types to be included in the digest may be automatically set by the information processing apparatus 110, instead of being externally input as the mode information. For example, the sum of the time lengths of the sections having a relatively high sound source type score may be calculated for each sound source type, and the above ratio may be determined as the ratio between the sound source types of the total, and the type digest length may be determined. The ratio thus determined may reflect the appearance probability of the sound for each sound source type in the sound information.

  In addition, when the mode is the diversity reflection mode, the digest section determination unit 115 calculates the variation of the feature amount within the same sound source type and the variation of the time when the sound is emitted within the same sound source type. Then, the digest section is determined so that the variation in the feature amount and the variation in the time become larger.

  For example, from the viewpoint of the sound source type score, even if they are classified into the same sound source type, they may actually be voices of different persons. Since the digest section is determined so that the variation of the feature amount within the same sound source type becomes larger, the feature amount is relatively different from the viewpoint of the sound source type score, though the feature amount is relatively classified. Audio will be included in the digest, and more diverse audio will be included in the digest.

  In addition, for example, from the viewpoint of sound source type score, even if there is a high possibility that the voices of the same person are classified into the same sound source type, statements made at intervals in time are Can be quite different. From the viewpoint of the sound source type score, the time at which the sound is emitted is determined by determining the digest section so that the variation in the time at which the sound is emitted within the same sound source type becomes larger. The voices separated by are included in the digest, and the voices of more diverse contents are included in the digest.

  In addition, for more specific processing contents of the digest section determination processing in each of the single sound source mode, the multiple sound source mode, and the diversity reflection mode, the following (3-2. Single sound source mode), (3-3 (Multiple sound source mode), (3-4. Diversity reflection mode), (4-2. Single sound source mode), (4-3. Multiple sound source mode), (4-4. Diversity reflection mode) explain.

  When the digest section determination unit 115 determines the digest section, it outputs information about the determined digest section (digest section information). The digest section information includes, for example, information about a start time, an end time of the digest section, a digest section length, an index attached to the digest section (digest section index), and the like. That is, the digest section information is information for specifying the position of the digest section in the voice information, and the digest can be generated based on the voice information and the digest section information.

  The output destination of the digest section information by the digest section determining unit 115 may be arbitrary. For example, the digest section determination unit 115 may output the digest section information to a storage unit (not shown) provided in the information processing apparatus 110, or may output the digest section information to an external device different from the information processing apparatus 110. You may output.

  When the digest section information is stored in the information processing apparatus 110, the information processing apparatus 110 may further have a function of generating a digest based on the digest section information and the voice information (such a configuration). Will be described in detail in the following (5-2. Modification example in which digest generating function is provided). In addition, when the digest section information is output to the external device, the external apparatus may have a function of generating a digest based on the digest section information and the voice information. As described above, in the present embodiment, the information processing apparatus 110 is configured to have at least the function of generating digest section information, and the function of actually generating the digest thereafter is not necessarily provided in the information processing apparatus 110. May be.

  The functional configuration of the information processing apparatus according to the present embodiment has been described above with reference to FIG. As described above, according to the present embodiment, the sound source type score indicating the probability of the sound source type of the sound included in the sound information is calculated, and the sound information is selected from the sound information based on the sound source type score. The digest section which comprises the digest of is determined. Therefore, for example, to generate only the music in the digest, to include only the human voice in the digest, to include the music and the human voice in the digest in a well-balanced manner, etc., to generate a digest that meets various user needs. Will be possible. Note that a series of processes performed by the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 may be started in response to an instruction from a user via an input unit (not shown). When the information is input to the information processing device 110, the process for the voice information may be automatically started.

  Here, the specific device configuration of the information processing device 110 may be arbitrary. For example, the information processing apparatus 110 may be various types of processors such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and an ASIC (Application Specific Integrated Circuit). Alternatively, the information processing device 110 may be a device such as a PC, a server, a smartphone, or a tablet PC in which various processors are mounted. Alternatively, the information processing device 110 may be a device such as an IC recorder having sound collecting and recording functions. The functions of the information processing apparatus 110 shown in FIG. 1 can be executed by the various processors operating according to a predetermined program.

  Further, for example, each function of the information processing device 110 (the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115) does not necessarily have to be executed by one device. For example, the respective functions corresponding to the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are distributed and implemented in a plurality of information processing devices (for example, a plurality of processors), and the plurality of devices are The functions of the information processing apparatus 110 described above may be realized by being communicably connected to each other and operating in cooperation with each other. The information processing device 110 may be a local information processing device that is directly operated by a user, or may be a so-called cloud information processing device that is connected to a user terminal via a network. Good. For example, when a user's terminal such as a smartphone or an IC recorder has a recording function, the voice information recorded by the terminal is transmitted from the terminal to the information processing apparatus 110 on the cloud, and the information processing apparatus The various types of processing described above may be performed on the voice information by 110, and the digest section information or the voice information related to the digest, which is the processing result, may be transmitted from the information processing apparatus 110 to the terminal.

  Note that it is possible to create a computer program for realizing each function of the information processing apparatus 110 according to the present embodiment as described above, and mount the computer program on a PC or the like. It is also possible to provide a computer-readable recording medium in which such a computer program is stored. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed, for example, via a network without using a recording medium.

  Hereinafter, the process executed by the information processing device 110 will be described in more detail. Here, in the present embodiment, the processing performed by the information processing device 110 can be roughly divided into two according to the processing mode. In one process, the information processing apparatus 110 performs a feature amount extraction process, a sound source type score calculation process, and a digest section determination process on the audio information of which all have been acquired in advance. Hereinafter, such processing is referred to as off-line processing.

  On the other hand, in the other process, the information processing apparatus 110 performs the feature amount extraction process, the sound source type score calculation process, and the digest section determination process on the audio information that is just being acquired at any time. In this case, the digest section information is updated at any time while the voice information is continuously acquired. Hereinafter, such processing will be referred to as online processing.

  The detailed processing contents may differ between the offline processing and the online processing. Therefore, in the following, detailed processing contents of each of the offline processing and the online processing will be described. Further, the details of the digest section determination process may be different depending on the mode described above for each of the offline process and the online process. Therefore, in the following, for each of the offline processing and the online processing, the detailed processing contents of the digest section determination processing according to the mode will be described.

  In the following description, a case where the score calculation section is a frame section will be described as an example. That is, a case where the sound source type score is calculated for each frame will be described. However, the present embodiment is not limited to this example, and a section including a plurality of frames may be set as the score calculation section. In the following description, the sound source type score may be simply referred to as a score for simplicity.

(3. Details of offline processing)
(3-1. Overall processing procedure)
The processing procedure of the offline processing will be described with reference to FIG. FIG. 4 is a flowchart showing an example of the processing procedure of offline processing. The processing procedure shown in FIG. 4 corresponds to the processing procedure of the entire information processing method executed by the information processing apparatus 110 shown in FIG. 1 during offline processing. In the off-line processing, after the scores of all frames of the audio information are calculated, the digest section is determined from the audio information based on the score.

  Referring to FIG. 4, in the offline processing, first, the feature amount of the voice information is extracted (step S101). In the process shown in step S101, various physical quantities indicating the characteristics of the audio information such as power and spectrum envelope shape are calculated as the characteristic quantities of the audio information. The process shown in step S101 corresponds to the process performed by the feature amount extraction unit 111 shown in FIG. 1, for example.

  Next, the sound source type score of each frame is calculated based on the extracted feature amount (step S103). In the process shown in step S103, for example, a sound source type score indicating the probability of the sound source type of the sound is calculated for each frame by a discriminator that identifies the sound source type of the sound according to the feature amount of the sound information. At this time, a plurality of sound source type scores such as a voice score, a voice score, and a noise score may be calculated. The process shown in step S103 corresponds to the process performed by the sound source type score calculation unit 113 shown in FIG. 1, for example.

  In addition, when the score calculation section is not a frame section but includes a plurality of frame sections, a process of smoothing the sound source type score of each frame to calculate a sound source type score as a score calculation section is performed in step S103. May be.

  Next, the digest section is determined from the audio information based on the calculated sound source type score (step S105). For example, in the process shown in step S105, a time section having a higher sound source type score in the audio information is determined as the digest section. Since the specific processing content of step S105 differs depending on the mode, the detailed processing content thereof will be described in the following (3-2. Single sound source mode), (3-3. Multiple sound source mode) and (3-4). (Diversity reflection mode) will be described in more detail for each mode. The digest section information about the determined digest section is output, and a series of processing is ended. The process shown in step S105 corresponds to the process performed by the digest section determining unit 115 shown in FIG. 1, for example.

  The processing procedure of the offline processing has been described above with reference to FIG.

(3-2. Single sound source mode)
(3-2-1. Processing procedure of digest section determination processing)
In the single sound source mode, a certain one sound source type is designated, and a section having a higher sound source type score related to the one designated sound source type is determined as the digest section.

  The procedure of the digest section determination process in the single sound source mode in the offline process will be described with reference to FIGS. 5 and 6. 5 and 6 are flowcharts showing an example of the processing procedure of the digest section determination processing in the single sound source mode in the offline processing.

  Referring to FIGS. 5 and 6, in the digest section determination process in the single sound source mode in the offline process, first, the score threshold theoretical upper limit value is set as the score threshold upper limit value (step S201). Next, the score threshold is set as a value lower than the score threshold upper limit value (step S203).

  Here, as will be described later in detail, in the digest segment determination process, a process of determining a segment with a higher score (high score segment) from the audio information as a digest segment (high score segment determination process shown in step S205) is performed. After that, the length of the digest section and the number of digest sections are adjusted so that the sum of the time lengths of those digest sections (digest section length) matches the digest length.

  The score threshold is a threshold for determining whether or not each frame is included in the high score section (that is, whether or not it is included in the digest section) in the high score section determination processing. The score threshold value is appropriately changed during a series of digest section determination processing in order to adjust the total digest section length according to the digest length, as will be described later in steps S213 and S219. If the score threshold is changed to a higher value, the number of frames included in the digest section increases and the digest section length becomes longer. Conversely, if the score threshold is changed to a lower value, the number of frames included in the digest section decreases, and the digest section length becomes shorter.

  The score threshold upper limit value is a value that defines the upper limit of the score threshold to be changed. If the score threshold becomes too high, the number of frames included in the digest section may decrease, and the total digest section length may be less than the digest length. The score threshold upper limit value is set to prevent such a situation from occurring (see the processing shown in step S217 described later).

  The score threshold theoretical upper limit value is a theoretical upper limit value that the score can take, which is set according to, for example, the performance of the discriminator used for the score calculation. As described above, in step S201, the score threshold theoretical upper limit value is set as the initial value of the score threshold upper limit value.

  When the processes shown in steps S201 and S203 are performed, next, a process of determining a section having a higher score (high score section) in the audio information as a digest section (high score section determination process) is performed ( Step S205). The high score section is a section in which the score is continuously high in the audio information. However, in the present embodiment, when the time length of a section with a low score is extremely short, a process of including the section in the high score section is also performed. When the time length of the section with a low score is extremely short, the section is a series of sections before and after, from the viewpoint of information content, such as breathing during a series of statements by a person. Because it is considered.

  In the offline processing, in the digest section determination processing, the high-score section determined in step S205 is regarded as the digest section, and in the subsequent processing, the digest length is adjusted so that the total digest section length becomes the length according to the digest length. Processing for adjusting the time length and number of sections is performed. It can be said that the high score section determined in the high score section determination processing is, so to speak, a candidate of the finally determined digest section.

  Note that the more detailed processing contents of the high score section determination processing will be described later later with reference to FIGS.

  When the high score sections are determined in step S205, these sections are regarded as digest sections, and an average score (section average score) within each digest section is calculated (step S207). The section average score may be included in the digest section information together with the start time and end time of the high score section (that is, the digest section) and the index determined in the high score section determination processing.

  Next, it is determined whether the total digest section length is significantly shorter than the digest length (step S209). Specifically, in step S209, it is determined whether or not the total digest section length is below the allowable range of the deviation amount from the digest length set for the digest length. Since it is difficult to determine the digest section so that the total digest section length completely matches the digest length, in the present embodiment, such an allowable range is set, and the total digest section length is the allowable range. Whether or not the sum of the digest section lengths is appropriate is determined by whether or not it is included in. The permissible range is a range of a shift amount that does not give a feeling of strangeness when the user listens to the digest and the actual digest length is longer or shorter than the set value of the digest length. May be appropriately set by.

  When it is determined in step S209 that the total digest section length is significantly shorter than the digest length, the process proceeds to steps S211 to S213. In steps S211 to S213, processing for increasing the total digest section length is performed.

  Specifically, in step S211, the current score threshold value is set as the score threshold upper limit value. This is because the sum of the digest section length is significantly shorter than the digest length, it is considered that the current score threshold is too high compared to an appropriate value, so when the score threshold is changed in future processing. This is to prevent the score threshold value from becoming larger than the current score threshold value.

  Next, a value lower than the current score threshold is set as a new score threshold (step S213). Then, the process proceeds to step S207, and the high score section determination process is performed again using the new score threshold. By performing the high score section determination process using the new score threshold set to a lower value, the number of frames included in the high score section increases, so the total digest section length becomes longer, and the digest section length becomes longer. Can be brought closer to the digest length.

  If it is determined in step S209 that the total digest section length is not significantly shorter than the digest length, the process proceeds to step S215. In step S215, conversely, it is determined whether the total digest section length is significantly longer than the digest length.

  When it is determined in step S215 that the total digest section length is not significantly longer than the digest length, the series of digest section determination processing ends. That is, the current digest section determined by the high score section determination processing is confirmed as the final digest section. When it is determined in step S209 that the total digest section length is not significantly shorter than the digest length, and when it is determined in step S215 that the total digest section length is not significantly longer than the digest length, This is because the total digest section length is included in the digest length allowable range.

  On the other hand, when it is determined in step S215 that the total digest section length is significantly longer than the digest length, the process proceeds to step S217. In the processing after step S217, processing for further shortening the total digest section length is performed.

  In step S217, it is determined whether the score threshold value is smaller than the score threshold upper limit value. When it is determined in step S217 that the score threshold value is smaller than the score threshold upper limit value, the process proceeds to step S219. In step S219, a value higher than the current score threshold is set as the new score threshold. Then, the process proceeds to step S207, and the high score section determination process is performed again using the new score threshold. By performing the high score section determination process using the new score threshold set to a higher value, the number of frames included in the high score section is reduced, so the total digest section length becomes shorter, and the digest section length becomes shorter. Can be brought closer to the digest length.

  When it is determined in step S217 that the score threshold value is not smaller than the score threshold upper limit value, the process proceeds to step S221. In this case, since the score threshold cannot be set higher than the current value, the total digest section length cannot be shortened by changing the score threshold. Therefore, in the processing from step S221, processing is performed to shorten the total digest section length by deleting a frame from the current digest section or reducing the number of current digest sections.

  Specifically, in step S221, it is determined whether or not the digest section length can be shortened for each digest section. Here, whether or not the digest section length can be shortened is determined by comparing the digest section length and the minimum continuous section length. The minimum continuous section length is set as a minimum section in which a person can recognize the meaning of the voice when outputting the voice. If the digest section length is equal to or less than the continuous minimum length, the meaning of the portion corresponding to the digest section cannot be grasped when listening to the digest, so that the digest is not significant. Therefore, by performing the determination process shown in step S221, the digest section is determined so that the digest section length becomes larger than the continuous minimum length.

  When it is determined in step S221 that the digest section length can be shortened in any of the digest sections, the process proceeds to steps S223 to S227. In steps S223 to S227, processing is performed to shorten the total digest section length by deleting the frame from the current digest section.

  Specifically, in step S223, among digest sections determined to be capable of shortening the digest section length (that is, digest sections whose digest section length is longer than the continuous minimum length), digest sections with a lower section average score are selected. The digest section length is shortened. When shortening the digest section length, for example, of the predetermined number of frames at the beginning and the predetermined number of frames at the end of the digest section to be shortened, the one with the lower average score is excluded from the digest section. It

  Next, the section average score of the digest section in which the frame is deleted and the digest section length is shortened is updated (step S225). Then, it is determined whether or not the total digest section length substantially matches the digest length (step S227). In step S227, specifically, it is determined whether or not the total digest section length is included in the allowable range set for the digest length.

  If it is determined in step S227 that the total digest section length is substantially equal to the digest length, the series of digest section determination processing ends. That is, the current digest section is confirmed as the final digest section.

  On the other hand, if it is determined in step S227 that the total digest section length does not substantially match the digest length, the procedure returns to step S221, and again it is determined whether or not the digest section length can be shortened for each digest section. To be done.

  When it is determined in step S221 that the digest section length cannot be shortened in any of the digest sections, the process proceeds to steps S229 to S231. In steps S229 to S231, a process of shortening the total digest section length by reducing the number of current digest sections is performed.

  Specifically, in step S229, a digest section having a lower section average score is deleted from the current digest sections. Then, it is determined whether or not the total digest section length is substantially equal to the digest length (step S231). In step S231, similarly to step S227, it is determined whether or not the total digest section length is included in the allowable range set in the digest length.

  If it is determined in step S231 that the total digest section length is substantially equal to the digest length, the series of digest section determination processing ends. That is, the current digest section is confirmed as the final digest section.

(3-2-2. High score section determination process)
Here, the high score section determination processing in the offline processing, which is shown in step S205, whose detailed description is omitted, will be described in detail with reference to FIGS. 7 to 9. FIG. 7 is an explanatory diagram for explaining the high score section determination processing in the offline processing. FIG. 8 and FIG. 9 are flowcharts showing an example of the processing procedure of the high score section determination processing in the offline processing.

  In the following description of the high score section determination process, the terms current frame, current digest section, continuous section and discontinuous section are used. Before describing the specific processing procedure of the high score section determination processing, the concept represented by these terms will be described with reference to FIG. 7.

  In FIG. 7, the horizontal axis represents the time of audio information, the vertical axis represents the score calculated for each frame, and the relationship between the two is plotted. In the high score section determination processing, it is determined for each frame in time series whether or not the frame is included in the digest section. In the figure, the current frame indicates the frame currently targeted for the determination process.

  The current digest section means a digest section which is a target for determining whether to include the current frame. The continuous section means a section in which the score continuously exceeds the score threshold in the current digest section. The discontinuous section means a section from the end time of the immediately preceding continuous section in the current digest section to the current frame. The time lengths of the current digest section, the continuous section, and the discontinuous section are also referred to as the current digest section length, the continuous section length, and the discontinuous section length, respectively.

  A specific processing procedure of the high score section determination processing in the offline processing will be described with reference to FIGS. 8 and 9. Referring to FIGS. 8 and 9, in the high score section determination processing in the offline processing, first, the frame index is set to zero (step S301). Further, the digest section index is set to zero (step S303). The frame index is attached to each frame of the audio information in chronological order, and the frame having a frame index of zero indicates the first frame of the audio information. The processing shown in steps S301 and S303 corresponds to the processing in which the current frame is set to frame # 0 and the current digest section is set to digest section # 0.

  Next, it is determined whether the score of the current frame is larger than the score threshold (step S305). When the score of the current frame is determined to be equal to or lower than the score threshold value in step S305, the current frame is not included in the digest section, and the process proceeds to step S319. In this case, the current frame will be added to the discontinuous section. The process in step S319 will be described later.

  On the other hand, if it is determined in step S305 that the score of the current frame is higher than the score threshold value, the process proceeds to step S307. In steps S307 to S317, processing for including the current frame in the digest section is performed.

  First, in step S307, it is determined whether the discontinuous section length is smaller than the maximum discontinuous section length. Here, the maximum length of the discontinuous section is a time length serving as a reference for determining whether or not the discontinuous section is a significant section that should be included in the digest section. As described above, since the discontinuous section is a section from the end time of the immediately preceding continuous section to the current frame, it can be said that the section is not included in the continuous section and has a continuously low score. Therefore, it is considered that the discontinuous section is a silent section in which almost no sound of the sound source type targeted for inclusion in the digest is produced. For example, when the discontinuous section is extremely short, the section is, for example, From the viewpoint of information content, such as breathing during a series of statements by a person, there is a high possibility that the section is a front and back section and a series of sections. From this point of view, the maximum length of the discontinuity section is whether the silence section corresponding to the discontinuity section is a very short silence in a series of voices or a long silence, for example, with a change in speaker. It can be set as the length of time for making a decision.

  When it is determined in step S307 that the length of the discontinuous section is smaller than the maximum length of the discontinuous section, the process proceeds to step S309. In this case, as described above, the discontinuous section should be considered as a series of sections including the immediately preceding continuous section. Therefore, in step S309, a process of connecting the discontinuous section and the current frame to the current digest section (that is, adding the discontinuous section and the current frame to the end of the current digest section) is performed. In this way, when the discontinuity period is extremely short, by determining the digest section to include the discontinuity period, a series of audio will be included in the digest without interruption, and from the viewpoint of understanding the content, It becomes possible to generate a useful digest. At this time, if the process shown in step S309 is performed on a frame with a frame index that is one smaller (that is, the frame that is one frame before in time series), the discontinuous section has already become the current digest section. Since it is included, only the current frame is connected to the current digest section. When the process shown in step S309 ends, the process proceeds to step S319.

  On the other hand, if it is determined in step S307 that the discontinuous section length is greater than or equal to the discontinuous section maximum length, the process proceeds to step S311. In step S311, it is determined whether or not the continuous section length before the discontinuous section is greater than or equal to the minimum continuous section length. As mentioned in the description of the process shown in step S221 of FIG. 6, the minimum continuous section length is the time length set as the minimum section in which a person can recognize the meaning of the sound when output as a sound. Is. That is, it can be said that the process shown in step S311 is a process of determining whether or not the continuous section is a significant section from the viewpoint of time length.

  When it is determined in step S311 that the continuous section length before the discontinuous section is equal to or longer than the continuous section minimum length, the process proceeds to steps S313 to S315. In this case, if the discontinuous section is greater than or equal to the maximum length of the discontinuous section, and the continuous section is greater than or equal to the minimum length of the continuous section (that is, the discontinuous section is not a significant section, and before the discontinuous section Since the continuous section of is a significant section), the discontinuous section is discarded (not included in the digest section) and the continuous section before the discontinuous section is adopted (included in the digest section). Be seen.

  Specifically, in step S313, the continuous section before the discontinuous section is determined as one digest section. Next, in step S315, the digest interval index is incremented by 1 (that is, the current digest interval to be processed is newly set), and the current frame is set to the start time of the new current digest interval. When the process of step S315 ends, the process proceeds to step S319.

  On the other hand, if it is determined in step S311 that the continuous section length before the discontinuous section is smaller than the minimum continuous section length, the process proceeds to step S317. In this case, if the discontinuous section is greater than or equal to the maximum length of the discontinuous section, and the continuous section is smaller than the minimum length of the continuous section (that is, the discontinuous section is not a significant section and the Since the continuous section of is also insignificant), the discontinuous section and the continuous section before the discontinuous section are both discarded (not included in the digest section). In this way, when the continuous period is so short that it cannot be recognized by a person, the digest section is determined so as not to include the continuous period. The meaningless section can be omitted from the digest, and a higher quality digest can be generated.

  Specifically, in step S317, the continuous section before the discontinuous section is discarded, and the current frame is set to the start time of the current digest section. When the process of step S317 ends, the process proceeds to step S319.

  In step S319, it is determined whether the voice information is the end. When it is determined in step S319 that the voice information is not the end, the frame index is incremented by 1 (that is, the frame to be processed is set to the frame one ahead) (step S321), and the processes in step S305 and subsequent steps are performed. Is repeatedly executed.

  On the other hand, if it is determined in step S319 that the voice information is at the end, the process proceeds to step S323. In step S323, it is determined whether the current digest section length is larger than the continuous section minimum length. That is, in step S323, whether or not the digest section that is the last processing target is a meaningful section from the viewpoint of time length (that is, whether or not it has a time length such that speech can be identified). Is judged.

  If it is determined in step S323 that the current digest section length is larger than the continuous section minimum length, the current digest section is considered to be a section that is significant in terms of time length. Ends the process. On the other hand, when it is determined in step S323 that the current digest section length is less than or equal to the continuous section minimum length, the current digest section is considered not to be a section that is significant in terms of time length, so the digest section is discarded, A series of processing ends.

  The processing procedure of the digest section determination processing in the single sound source mode in the offline processing has been described above.

(3-3. Multiple sound source mode)
(3-3-1. Processing procedure of digest section determination processing)
In the multiple sound source mode, the time length of the sound included in the digest is set for each sound source type based on the specified ratio, and the sound source type score is higher for each sound source type and the total length of the relevant section is set. A section that is equal to or less than the time length for each sound source type is determined as the digest section.

  With reference to FIG. 10 and FIG. 11, the processing procedure of the digest section determination processing in the multiple sound source mode in the offline processing will be described. 10 and 11 are flowcharts showing an example of the processing procedure of the digest section determination processing in the multiple sound source mode in the offline processing.

  In the digest section determination process in the multiple sound source mode shown in FIGS. 10 and 11, each process in the digest section determination process in the single sound source mode described with reference to FIGS. 5 to 9 is performed for each sound source type. The content itself of each processing may be substantially the same as the digest section determination processing in the single sound source mode. However, in the digest section determination process in the single sound source mode, since only one sound source type is targeted, the digest section length determined based on the score related to the sound source type in step S209 and step S215 described above. Although the total value was compared with the digest length, in the digest section determination process in the multiple sound source mode, the total value of the digest section length determined based on the score related to each sound source type, of each sound source type to be included in the digest It is compared with the time length of the voice (hereinafter, also referred to as the type digest length).

  In the following description of the processing procedure of the digest section determination processing in the multiple sound source mode, matters that are different from the processing procedure of the digest section determination processing in the single sound source mode will be mainly described, and detailed descriptions of overlapping matters will be given. The description is omitted.

  Referring to FIGS. 10 and 11, in the digest section determination process in the multiple sound source mode in the offline process, first, the score threshold theoretical upper limit value is set as the score threshold upper limit value (step S401). Next, the score threshold is set as a value lower than the score threshold upper limit value (step S403). These processes are similar to the processes in steps S201 and S203 shown in FIGS. 5 and 6.

  Next, the type digest length is set (step S405). For example, the type digest length may be set based on the mode information. For example, the mode information includes information for designating the ratio of the sound source types included in the digest. In the process shown in step S405, the type digest length is calculated for each sound source type by multiplying the digest length by the ratio.

  However, the process shown in step S405 is not limited to this example, and the ratio of the sound source types to be included in the digest may be automatically set by the information processing apparatus 110 instead of being input from the outside as mode information. For example, when the high score section determination process shown in FIGS. 8 and 9 has already been executed once for each sound source type and the high score section has been determined for each type of sound source, for example. For example, the ratio may be determined and the type digest length may be determined using information about the high score section.

  Specifically, from the result of the high score section determination processing, the sum of the time lengths of the determined high score sections is calculated for each sound source type, and the ratio thereof is calculated. Then, the type digest length can be calculated for each sound source type by multiplying the calculated ratio by the calculated digest length. In this way, the proportion determined based on the time length of the high score section may reflect the appearance probability of the voice for each sound source type in the voice information.

  If the calculated type digest length is less than the minimum length of the continuous section both based on the mode information and based on the high score section, a process for adjusting the length is appropriately performed. This is because when the type digest length is shorter than the minimum length of the continuous section, the type digest length is too short, and the voice is not significantly recognized by a person. Specifically, the type digest length that is less than the minimum continuous section length is increased to the minimum continuous section length, and the increase is subtracted from the type digest lengths that are longer than the other minimum continuous section lengths.

  When the type digest length is determined, a process of determining a section having a higher score (high score section) in the voice information as a digest section (high score section determination processing) is performed (step S407). The process shown in step S407 is the same as the process in step S205 shown in FIGS. 5 and 6, that is, the series of processes shown in FIGS. 8 and 9, and thus detailed description thereof will be omitted.

  After that, the processing shown in steps S409 to S433 is the same as the processing in steps S207 to S231 shown in FIGS. 5 and 6 except that it is executed for each sound source type, and therefore detailed description thereof will be given. Is omitted. The processing shown in steps S411 to S421 corresponds to the processing in steps S209 to S219 shown in FIGS. 5 and 6. In the processing shown in steps S411 to S421, it is determined for each sound source type whether the total digest section length is significantly different from the type digest length, and the score threshold is adjusted, so that the total digest section length is the type. Each digest section length is adjusted so that it is included in the allowable range of the digest length.

  The processing shown in steps S423 to S433 corresponds to the processing in steps S221 to S231 shown in FIGS. 5 and 6. The processing shown in steps S423 to S433 is processing that is performed when the score threshold cannot be adjusted any more. In the processing after step S423, the frame is deleted from the current digest section, or the current digest section is deleted. By reducing the number of digest sections, processing for shortening the total digest section length is performed. However, in the processing in steps S221 to S231 shown in FIG. 5 and FIG. 6, the digest section to be deleted of the frame or the section number relates to a single sound source type, but is shown in steps S423 to S433. In the processing, the digest section whose frame or the number of sections is to be deleted is a mixture of digest sections relating to a plurality of sound source types.

  The processing procedure of the digest section determination processing in the multiple sound source mode in the offline processing will be described above with reference to FIGS. 10 and 11.

(3-4. Diversity reflection mode)
In the diversity reflection mode, a digest is generated so that various voices are included among the voices classified into the same sound source type. Specifically, in the diversity reflection mode, the digest section is determined so that the variation in the feature amount of the voice within the same sound source type and the temporal variation of the voice within the same sound source type become larger. .

(3-4-1. Functional configuration)
Here, each processing in the single sound source mode and the multiple sound source mode described above can be executed by the functional configuration of the information processing apparatus 110 illustrated in FIG. 1. However, each processing in the diversity reflection mode may be executed by a functional configuration slightly different from that of the information processing apparatus 110 shown in FIG.

  The functional configuration of the information processing apparatus that executes each process in the diversity reflection mode will be described with reference to FIG. FIG. 12 is a functional block diagram illustrating an example of the functional configuration of the information processing device that executes each process in the diversity reflection mode.

  Referring to FIG. 12, the information processing apparatus 120 corresponding to the diversity reflection mode has a feature amount extraction unit 111, a sound source type score calculation unit 113, and a digest section determination unit 115 as its functions. Here, the functions of the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are similar to the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  Unlike the information processing apparatus 110, the information processing apparatus 120 also provides the digest section determination unit 115 with information about the characteristic amount of the audio information calculated by the characteristic amount extraction unit 111. The digest section determination unit 115 can determine the digest section in consideration of the diversity by using the information about the feature amount (see the processing shown in step S531 of FIG. 14 described later).

(3-4-2. Processing procedure of digest section determination processing)
With reference to FIGS. 13 and 14, a processing procedure of the digest section determination processing in the diversity reflection mode in the offline processing, which can be executed by the information processing apparatus 120 illustrated in FIG. 12, will be described. 13 and 14 are flowcharts showing an example of the processing procedure of the digest section determination processing in the diversity reflection mode in the offline processing.

  Since the diversity reflection mode determines the digest section in consideration of the diversity within the same sound source type, the sound source type to be included in the digest may be a single sound source type. However, there may be a plurality of sound source types. 13 and 14 show, as an example, a processing procedure in the case where the digest includes sounds of a plurality of sound source types.

  Here, each process in the digest segment determination process in the diversity reflection mode is different from each process in the digest segment determination process in the multiple sound source mode described with reference to FIGS. 10 and 11 except for the process shown in step S531 described later. It is similar to the processing. Therefore, in the description of each process in the digest section determination process in the diversity reflection mode below, mainly the matters different from each process in the digest section determination process in the multiple sound source mode will be described, and the overlapping matters will be described. Detailed description is omitted. In addition, the processing procedure of the digest section determination process in the diversity reflection mode in the case where the digest includes voices of a plurality of sound source types is the same as the processing procedure of the digest section determination processing in the single sound source mode shown in FIGS. 5 and 6. In this case, instead of the process shown in step S229, the process shown in step S531 described later is performed.

  Referring to FIGS. 13 and 14, in the digest section determination process in the diversity reflection mode, the processes in steps S501 to S521 are the same as the processes in steps S401 to S421 shown in FIGS. 10 and 11. . In addition, the process from step S523 is also a process performed when the score threshold cannot be adjusted any more, like the digest section determination process in the multiple sound source mode. In the processing from step S523, processing is performed to shorten the total digest section length by deleting a frame from the current digest section or reducing the number of current digest sections.

  Here, in the diversity reflection mode, when it is determined in step S523 that the digest section length can be shortened for each digest section, the digest section length is deleted by deleting the frame from the digest section having a lower section average score. A series of processes (processes shown in steps S525 to S529) for shortening the sum of the above are similar to these processes (processes shown in steps S425 to S429) in the multiple sound source mode.

  On the other hand, in the diversity reflection mode, the details of the process of reducing the number of digest sections when it is determined that the digest section length cannot be shortened in any digest section in step S523 different. Specifically, in the multiple sound source mode, the digest section having a low section average score has been deleted (see the processing shown in step S431 in FIG. 11). On the other hand, in the diversity reflection mode, processing for deleting the digest section based on the diversity (digest section deletion processing based on the diversity) is performed (step S531). After the digest section is deleted, it is determined whether the total digest section length substantially matches the digest length (step S533), and the diversity shown in step S531 until the total digest section length substantially matches the digest length. The digest section deletion process based on is executed.

(3-4-3. Digest section deletion processing based on diversity)
With reference to FIG. 15, the digest section deletion processing based on diversity shown in step S531 of FIG. 14 will be described in detail. FIG. 15 is a flowchart showing an example of a processing procedure of digest section deletion processing based on diversity in the offline processing.

  Referring to FIG. 15, in the diversity-based digest section deletion processing in the offline processing, first, the average of the feature quantity vectors of each digest section (average feature quantity vector) is calculated (step S601).

  Next, the variances of the average feature quantity vectors in the n feature quantity spaces in the case of all digest sections and the case of excluding one arbitrary digest section are calculated (step S603).

  Next, the average time of each digest section is calculated (step S605). The average time is calculated, for example, as an intermediate time between the start time and the end time of each digest section.

  Next, the variance of the average time of each of the n digest sections in the case of the entire digest section and the case of excluding one arbitrary digest section is calculated (step S607).

  Next, the sum is calculated after weighting the variance of the average feature amount vector and the variance of the average time, and the digest section excluded when the reduction amount from the value in the case of the entire digest section is the smallest, It is determined as the digest section to be deleted (step S609). That is, in the processing shown in step S609, the digest section having the average feature quantity vector and the average time having the least influence when not used in the calculation of the variance of the average feature quantity vector and the average time is determined as the digest section to be deleted. To be done. As a result, the digest section included in the digest is selected so that the variance of the average feature quantity vector and the average time becomes larger. Finally, the determined digest section is deleted (step S611).

  The processing procedure of the digest section determination processing in the diversity reflection mode in the offline processing has been described above with reference to FIGS. 13 and 14. Also, with reference to FIG. 15, the digest section deletion processing based on diversity shown in step S531 has been described.

  As described above, in the diversity reflection mode, the digest section is determined so that the diversity of the feature amount vector and the time is secured for the sounds classified into the same sound source type. By ensuring the diversity of the feature quantity vectors, it is possible to include both voices in the digest when they are actually classified by the same sound source type but different voices actually exist. Also, by ensuring the diversity of time, it is possible to include both voices classified into the same sound source type in the digest when they are speaking at distant locations in time. Becomes

(4. Details of online processing)
(4-1. Overall processing procedure)
The procedure of the online process will be described with reference to FIG. FIG. 16 is a flowchart showing an example of the processing procedure of online processing. The processing procedure shown in FIG. 16 corresponds to the processing procedure of the entire information processing method executed by the information processing apparatus 110 shown in FIG. 1 during online processing.

  In the online processing, each time a frame of voice information is newly input, the score of the newly input frame (input frame) is calculated, and the digest section is determined from the voice information based on the score. . That is, in the online processing, the series of processing shown in FIG. 16 is performed while the voice information is being input. It is executed every time a frame is newly input, and the digest section information is updated.

  If the score calculation section is not a frame section but a plurality of frame sections, the series of processes shown in FIG. 16 may be executed every time a plurality of frames corresponding to the score calculation section are input.

  Referring to FIG. 16, in the online processing, first, the feature amount of the voice information acquired so far is extracted (step S701). In the process shown in step S701, various physical quantities indicating characteristics of the audio information, such as power and spectrum envelope shape, are calculated as the characteristic quantities of the audio information. The process shown in step S701 corresponds to, for example, the process performed by the feature amount extraction unit 111 shown in FIG.

  Next, the sound source type score of the input frame is calculated based on the extracted feature amount (step S703). In the process shown in step S703, the sound source type score indicating the probability of the sound source type of the sound in the input frame is calculated by, for example, the classifier that identifies the sound source type of the sound according to the feature amount of the sound information. At this time, a plurality of sound source type scores such as a voice score, a voice score, and a noise score may be calculated. The process shown in step S703 corresponds to the process performed by the sound source type score calculation unit 113 shown in FIG. 1, for example.

  If the score calculation section is not a frame section but a plurality of frame sections, a process of smoothing the sound source type score of each frame to calculate a sound source type score as a score calculation section is performed in step S703. May be.

  Next, the digest section is determined from the audio information based on the calculated sound source type score (step S705). The process shown in step S705 corresponds to, for example, the process performed by the digest section determining unit 115 shown in FIG.

  In the processing shown in step S705, if the time length of the audio information acquired so far is shorter than the digest length (the set value of the digest time length), the input frame is unconditionally added to the digest. . On the other hand, if the time length of the audio information acquired so far is equal to or longer than the digest length, the input frame is added to the digest, and instead, for example, a frame with a lower score is deleted from the digest. To be done.

  Since the specific processing contents in step S705 differ depending on the mode, the detailed processing contents will be described in the following (4-2. Single tone generator mode), (4-3. Multiple tone generator mode) and (4). -4. Diversity reflection mode) will be described in more detail for each mode.

  Next, it is determined whether the input of voice information is completed (step S707). When it is determined in step S707 that the input of the voice information is completed, the digest section information about the determined digest section is output, and the series of processes is completed. On the other hand, if it is determined in step S707 that the input of voice information is not completed, the input of the next frame is awaited (step S709), and the processing of step S701 and subsequent steps is performed on the newly input frame. Is repeatedly executed.

  The processing procedure of the online processing has been described above with reference to FIG.

(4-2. Single sound source mode)
(4-2-1. Digest section determination process)
With reference to FIG. 17, a processing procedure of the digest section determination processing in the single sound source mode in the offline processing will be described. FIG. 17 is a flowchart showing an example of a processing procedure of digest section determination processing in the single sound source mode in the offline processing.

  Referring to FIG. 17, in the digest section determination process in the single sound source mode in the offline process, first, it is determined whether the current digest length is shorter than the digest length (step S801). If it is determined in step S801 that the current digest length is shorter than the digest length, the input frame is added to the digest and the average score (digest average score) of the entire digest is updated (step S803). ). Then, the digest section determination process is ended, and the next input frame is waited for.

  The processes shown in steps S801 and S803 correspond to the process of unconditionally adding the input frame to the digest when the time length of the voice information input so far is less than the digest length.

  If it is determined in step S801 that the current digest length is greater than or equal to the digest length, the process proceeds to step S805. In step S805, it is determined whether the score of the input frame is equal to or higher than the digest average score. When it is determined in step S805 that the score of the input frame is smaller than the digest average score, the digest section determination process ends without adding the input frame to the digest. That is, frames with lower scores should not be included in the digest.

  On the other hand, if it is determined in step S805 that the score of the input frame is equal to or higher than the digest average score, the input frame is added to the digest and the digest average score is updated (step S807). However, in this case, since the input frame is added to the digest, the current digest length exceeds the digest length by the time length corresponding to one frame. Therefore, after the processing shown in step S807, processing for deleting a frame from the digest (frame deletion processing) is performed (step S809). In the frame deletion process, for example, a frame with a lower score is deleted from the digest. Note that details of the frame deletion processing shown in step S809 will be described later with reference to FIG.

  When the frame is deleted, the digest average score is updated (step S811), and the digest section determination process ends.

(4-2-2. Frame deletion processing)
Here, with reference to FIG. 18, details of the frame deletion processing shown in step S809 of FIG. 17 will be described. FIG. 18 is a flowchart showing an example of a processing procedure of frame deletion processing in the single sound source mode in the online processing.

  Referring to FIG. 18, in the frame deletion process in the single sound source mode in the online process, first, the digest average score is set as the score threshold (step S901). Then, using the set score threshold value, a process of determining a section having a higher score (high score section) in the digest as a digest section (high score section determination processing) is performed (step S903).

  In the high score section determination processing shown in step S903, substantially the same processing as the high score section determination processing in the offline processing shown in step S205 of FIG. 5 is performed, but some processing is different from that of the offline processing. Specifically, in the offline processing, a high score section determination process is performed on the entire voice information to determine a digest section in the voice information. On the other hand, as described with reference to FIG. 17, in the online processing, the input frame is unconditionally added to the digest until the time length of the voice information acquired so far reaches the digest length. Therefore, a so-called temporary digest has already been generated before performing the high score section determination process. In online processing, if an input frame is added and the current digest length is one frame longer than the digest length setting value, a section with a lower score is found from the digests and the frame to be deleted is determined. In order to do so, the high score section determination process is performed. That is, in the online processing, the high score section determination processing is performed for the digest.

  Further, from the above circumstances, in the offline processing, the section that is not determined as the high score section in the audio information is not naturally adopted as the digest section. On the other hand, in the online processing, even if there is a section that is not determined as a high score section in the digest, the section that is deleted from the digest is a section for one frame, so it is determined as the high score section. It is not possible to delete all the sections that have not been deleted from the digest. That is, in the online processing, a section that is not determined as a high score section as a result of the high score section determination processing may remain in the digest. In the following description, a section that is not determined as such a high score section is referred to as a deletion target section. From the deletion target section, for example, the frame with the lowest score is selected as the frame to be deleted. As described above, the deletion target section is present in the digest at present, but it can be said that the section should be deleted as the voice information is input and the digest is updated.

  In addition, in the online processing, as described above, the input frame is added to the digest and one of the frames is deleted, so when arranging the scores in each frame in the digest in chronological order , There may be points where the scores become discontinuous. In the high score section determination processing in the above-mentioned offline processing, the entire music information is the processing target, and it was not necessary to consider such a score discontinuity, but in the high score section determination processing in the online processing, Additional processing is required to deal with the discontinuity.

  Note that the more detailed processing contents of the high score section determination processing in the online processing shown in step S903 will be described later again with reference to FIGS. 19 to 22.

  When the high score section is determined in step S903, as a result of the high score section determination processing, it is determined whether there is a deletion target period that has not been determined as the high score section (step S905). If it is determined in step S905 that there is a deletion target section, one frame with a lower score is selected from the deletion target section (step S907). Then, the selected frame is deleted from the digest (step S911).

  On the other hand, when it is determined in step S905 that there is no deletion target section, one frame with a lower score is selected from the digest (that is, S909). Then, the selected frame is deleted from the digest (step S911).

(4-2-3. High score section determination processing)
Here, with reference to FIGS. 19 to 22, the high score section determination processing in the online processing, which is shown in step S903 in FIG. 18 and whose detailed description is omitted, will be described in detail. FIG. 19 is an explanatory diagram for explaining the high score section determination processing in the online processing. 20 to 22 are flowcharts showing an example of the processing procedure of the high score section determination processing in the online processing.

  In FIG. 19, the horizontal axis represents the time of audio information, the vertical axis represents the score calculated for each frame, and the relationship between the two is plotted. In the high score section determination processing, it is determined for each frame in time series whether or not the frame is included in the digest section. The meanings of the current frame, the current digest section, the continuous section, and the discontinuous section are the same as those in the high score section determination processing in the offline processing shown in FIG. 7.

  However, as described above, in online processing, unlike offline processing, the processing target is a digest. Therefore, as shown in the figure, when frames are deleted from the digest, points may become discontinuous (discontinuous points) when the scores in each frame in the digest are arranged in chronological order. Also, as described above, as a result of the high score section determination processing being performed, the deletion target section, which is a section that is not determined as the high score section (that is, the digest section) but exists in the digest, is included in the digest. Can exist

  A specific processing procedure of the high score section determination processing in the online processing will be described with reference to FIGS. The processing procedure of the high score section determination processing in the online processing shown in FIGS. 20 to 22 is that the processing target is not the entire voice information but a digest, and the processing shown in steps S1119 to S1123 described later is added. Except for this, the processing procedure of the high score section determination processing in the offline processing described with reference to FIGS. 8 and 9 is substantially the same. Therefore, in the following description of the processing procedure of the high score section determination processing in the online processing, detailed description will be omitted for the matters overlapping with the processing procedure of the high score section determination processing in the offline processing, and the different matters will be mainly described. Explained.

  20 to 22, in the high score section determination processing in the online processing, first, the frame index is set to zero (step S1101) and the digest section index is set to zero (that is, S1103). These processes are similar to the processes shown in steps S301 and S303 shown in FIGS. 8 and 9.

  The subsequent steps S1105 to S1117 are the same as the steps S305 to S317 shown in FIGS. 8 and 9. Specifically, in step S1105, it is determined whether the score of the current frame is larger than the score threshold. If it is determined that the score of the current frame is less than or equal to the score threshold, the current frame is not included in the digest section, and the process proceeds to step S1119. On the other hand, if it is determined that the score of the current frame is equal to or lower than the score threshold value, the process proceeds to steps S1107 to S1117, and processing for including the current frame in the digest section is performed.

  In steps S1107 to S1117, when the discontinuous section length is smaller than the maximum discontinuous section length, the discontinuous section and the current frame are connected to the current digest section (step S1109). When the length of the discontinuous section is greater than or equal to the maximum length of the discontinuous section and the continuous section before the discontinuous section is greater than or equal to the minimum length of the continuous section, the continuous section before the discontinuous section is determined as one digest section. At the same time, the digest section index is incremented by 1, and the current frame is set to the start time of the new current digest section (steps S1113 and S1115). If the length of the discontinuous section is greater than or equal to the maximum length of the discontinuous section and the continuous section before the discontinuous section is smaller than the minimum length of the continuous section, the continuous section before the discontinuous section is discarded (that is, to be deleted). The current frame is set to the start time of the current digest section (step S1117). When any one of steps S1109, S1115, and S1117 ends, the process proceeds to step S1119.

  In step S1119, it is determined whether the current frame is a discontinuous point. If it is determined in step S1119 that the current frame is not a discontinuous point, no special process is performed and the process proceeds to step S1125.

  On the other hand, if it is determined in step S1119 that the current frame is a discontinuous point, the process advances to step S1123. In step S1123, it is determined whether the current digest section length is larger than the continuous section minimum length. That is, in step S1123, it is determined whether or not the digest section immediately before the discontinuity point is a meaningful section from the viewpoint of time length (that is, whether or not it has a time length that allows speech to be identified). To be done.

  If it is determined in step S1123 that the current digest section length is greater than the minimum continuous section length, the current digest section is considered to be a section that is significant in terms of time length. Proceeds to S1125. On the other hand, if it is determined in step S1123 that the current digest section length is less than or equal to the continuous section minimum length, the current digest section is considered not to be a section that is significant in terms of time length, and the digest section is discarded ( That is, the deletion target section), and the process proceeds to step S1125.

  The subsequent steps S1125 to S1131 are the same as the steps S319 to S325 shown in FIGS. 8 and 9. Specifically, in step S1125, it is determined whether the voice information is the end. When it is determined in step S1125 that the audio information is not the end, the frame index is incremented by 1 (that is, the frame to be processed is set to the frame one ahead) (step S1127), and the processing in step S1105 and subsequent steps is performed. Is repeatedly executed.

  On the other hand, if it is determined in step S1125 that the voice information is the end, the process proceeds to step S1121, and it is determined whether the current digest section length is larger than the continuous section minimum length, that is, the digest section that is the last processing target is , It is judged from the viewpoint of time length whether it is a significant section.

  If it is determined in step S1121 that the current digest section length is greater than the minimum continuous section length, the current digest section is considered to be a section that is significant in terms of time length. Ends the process. On the other hand, if it is determined in step S1121 that the current digest section length is less than or equal to the continuous section minimum length, the current digest section is considered not to be a section that is significant in terms of time length, and the digest section is discarded ( That is, the deletion target section is set), and the series of processes is ended.

  The processing procedure of the digest section determination processing in the single sound source mode in the online processing has been described above.

(4-3. Multiple sound source mode)
(Processing procedure of 4-3-1. Digest section determination processing)
With reference to FIG. 23, the processing procedure of the digest section determination processing in the multiple sound source mode in the online processing will be described. FIG. 23 is a flowchart showing an example of the processing procedure of the digest section determination processing in the multiple sound source mode in the online processing.

  Note that the digest section determination process in the multiple sound source mode shown in FIG. 23 is part of the processing (specifically described later) with respect to the digest section determination process in the single sound source mode described with reference to FIG. The process shown in step S1205) is changed, and the other processes are substantially the same as the digest section determination process in the single sound source mode. Therefore, in the following description of the processing procedure of the digest section determination processing in the multiple sound source mode, the detailed description of the matters overlapping with the processing procedure of the digest section determination processing in the single sound source mode will be omitted and different. Items will be mainly described.

  Referring to FIG. 23, in the digest section determination process in the multiple sound source mode, first, it is determined whether or not the current digest length is shorter than the digest length (the set value of the digest time length) (step S1201), When it is determined that the current digest length is shorter than the digest length, the input frame is added to the digest and the digest average score is updated (step S1203). The processing shown in steps S1201 and S1203 is the same as the processing in steps S801 and S803 shown in FIG.

  When it is determined in step S1201 that the current digest length is equal to or longer than the digest length, the process proceeds to step S1205. In step S1205, the score of the input frame is compared with the digest average score for each sound source type, and it is determined whether the score of the input frame is equal to or higher than the digest average score for any sound source type. If it is determined in step S1205 that the score of the input frame is smaller than the digest average score in any sound source type, the digest section determination process is terminated without adding the input frame to the digest.

  On the other hand, if it is determined in step S1205 that the score of the input frame is equal to or higher than the digest average score in any sound source type, the process proceeds to step S1207. The subsequent steps S1207 to S1211 are the same as the steps S807 to S811 shown in FIG. That is, the input frame is added to the digest and the digest average score is updated (step S1207). Next, frame deletion processing (step S1209) is performed, and when the frame is deleted, the digest average score is updated (step S1211), and the digest section determination processing ends.

(4-3-2. Frame deletion processing)
Here, with reference to FIG. 24, details of the frame deletion processing shown in step S1209 of FIG. 23 will be described. FIG. 24 is a flowchart showing an example of a processing procedure of frame deletion processing in the multiple sound source mode in the online processing.

  Referring to FIG. 24, in the frame deletion processing in the multiple sound source mode in the online processing, first, a digest average score is set as a score threshold for each sound source type (step S1301). Next, the type digest length is set (step S1303). In the process shown in step S1303, the type digest length may be set by the same method as the process shown in step S405 of the digest section determination process in the multiple sound source mode in the offline process shown in FIG.

  Then, using the set score threshold value, a process of determining a section having a higher score (high score section) in the digest as a digest section (high score section determination processing) is performed (step S1305). Since the processing shown in step S1305 is similar to the processing in step S903 shown in FIG. 18, that is, the series of processing shown in FIGS. 20 to 22, detailed description thereof will be omitted. However, in the frame deletion process in the multiple sound source mode, the high score section determination process is performed for each sound source type.

  When the high score section is determined in step S1305, as a result of the high score section determination processing, it is determined whether or not there is a deletion target period in any sound source type (step S1307). When it is determined in step S1307 that there is a deletion target section in any sound source type, one frame with a lower score is selected from the deletion target section of that sound source type (step S1309). Then, the selected frame is deleted from the digest (step S1315).

  On the other hand, if it is determined in step S1307 that there is no deletion target section in any sound source type, a sound source type whose total digest section length most exceeds the type digest length is selected (step S1311). ). Then, for the selected sound source type, one frame having a lower score is selected (step S1313). Then, the selected frame is deleted from the digest (step S1315).

  The processing procedure of the digest section determination processing in the multiple sound source mode in the online processing has been described above.

(4-4. Diversity reflection mode)
The processing procedure of the digest section determination processing in the diversity reflection mode in the online processing is the same as the processing procedure of the digest section determination processing in the multiple sound source mode in the online processing described with reference to FIG. However, in the diversity reflection mode, the details of the frame deletion processing shown in step S1209 in FIG. 23 are different from those in the multiple sound source mode. Therefore, in the following description of the digest section determination process in the diversity reflection mode in the online process, the details of the frame deletion process will be mainly described.

  In the online processing as well, similar to the offline processing, each processing in the diversity reflection mode can be executed by the information processing apparatus 120 shown in FIG.

(4-4-1. Processing procedure of frame deletion processing)
With reference to FIG. 25, a processing procedure of frame deletion processing in the diversity reflection mode in online processing will be described. FIG. 25 is a flowchart showing an example of a processing procedure of frame deletion processing in the diversity reflection mode in online processing.

  Here, since the diversity reflection mode determines the digest section in consideration of the diversity within the same sound source type, even if the sound source type to be included in the digest is a single sound source type, Alternatively, a plurality of sound source types may be used. FIG. 25 illustrates, as an example, a processing procedure in the case where the digest includes sounds of a plurality of sound source types.

  Note that each process in the frame deletion process in the diversity reflection mode is the same as each process in the frame deletion process in the multiple sound source mode described with reference to FIG. 24, except for the process shown in step S1413 described later. Therefore, in the following description of the processing procedure of the frame deletion processing in the diversity reflection mode, matters that are different from the processing procedure of the frame deletion processing in the multiple sound source mode will be mainly described, and the overlapping matters will be described in detail. The description is omitted.

  Referring to FIG. 25, in the frame deletion process in the diversity reflection mode in the online process, first, a digest average score is set as a score threshold for each sound source type (step S1401), and then a type digest length is set. (Step S1403). Then, the high score section determination process is performed for each sound source type using the set score threshold (step S1405). These processes are similar to the processes in steps S1301 to S1305 shown in FIG.

  Next, as a result of the high score section determination processing, it is determined whether or not there is a deletion target period in any sound source type (step S1407). When it is determined that the deletion target section exists in any sound source type, one frame having a lower score is selected from the deletion target section of the sound source type (step S1409), and the selected frame is selected from the digest. It is deleted (step S1415). These processes are similar to the processes in steps S1307, S1309, and S1315 shown in FIG.

  On the other hand, if it is determined in step S1407 that the deletion target section does not exist in any sound source type, the sound source type whose sum of digest section lengths most exceeds the type digest length is selected (step S1411). ). Then, with respect to the selected sound source type, a process of selecting a frame to be deleted in consideration of diversity within the sound source type (deleted frame selection process based on diversity) is performed (step S1413). Then, the selected frame is deleted from the digest (step S1415).

(4-4-2. Deleted frame selection processing based on diversity)
With reference to FIG. 26, the diversity-based deletion frame selection processing shown in step S1413 of FIG. 25 will be described in detail. FIG. 26 is a flowchart showing an example of the processing procedure of diversity-based deletion frame selection processing in online processing.

  Referring to FIG. 26, in the deletion frame selection processing based on diversity in the online processing, first, the variance of the feature amount vectors in the n feature amount spaces in the case of all frames and the case of excluding one arbitrary frame Is calculated (step S1501).

  Next, the variances of the times of the n frames are calculated for all the frames and the case where any one frame is excluded (step S1503).

  Next, the sum of the variances of the feature amount vector and the variance of the time is weighted, and the sum is calculated. The frame excluded when the reduction amount from the value of all frames is the smallest is the frame to be deleted. It is determined (step S1505). That is, in the process shown in step S1505, the frame having the feature amount vector and the time having the least influence when not used for the calculation of the variance of the feature amount vector and the time is determined as the frame to be deleted. Thereby, the frames to be included in the digest are selected so that the variance of the feature amount vector and the time becomes larger.

  The procedure of the digest section determination process in the diversity reflection mode in the online process has been described above with reference to FIG. In addition, with reference to FIG. 26, the deletion frame selection processing based on diversity shown in step S1413 of FIG. 25 has been described.

(5. Modified example)
Some modifications of the embodiment described above will be described. The matters described in the above-described embodiment and each modification described below may be combined with each other within a possible range.

(5-1. Modified example in which voice collecting function is provided)
With reference to FIG. 27, a modification in which the voice pickup function is provided in the information processing device will be described. FIG. 27 is a functional block diagram showing an example of the functional configuration of the information processing apparatus according to the modified example provided with the voice pickup function.

  Referring to FIG. 27, the information processing apparatus 130 according to the present modification has, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, a voice sound collection unit 131, Have. Here, the functions of the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are similar to the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The sound collecting unit 131 is configured by a sound collecting device such as a microphone, and has a function of collecting an external sound and inputting it as sound information to the information processing device 110. The voice pickup unit 131 provides the feature amount extraction unit 111 with voice information related to the collected external voice. The feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 perform various processes (feature amount extraction process) according to the above-described embodiments on the voice information provided from the voice sound pickup unit 131. , Sound source type score calculation processing and digest section determination processing).

  The voice pickup unit 131 may be configured by one microphone, or may be configured by a plurality of microphones arranged at different positions. When the sound pickup unit 131 is composed of a plurality of microphones arranged at mutually different positions, the feature amount extraction unit 111 causes the plurality of microphones such as the correlation between the sound pickup positions and the sound source direction to exist. It is possible to calculate various types of feature amounts that can be calculated.

  In the above, with reference to FIG. 27, the modification in which the voice pickup function is provided in the information processing apparatus has been described. As described above, according to the present modification, the information processing apparatus 130 itself has a sound collecting function of collecting an external sound, and outputs digest section information of audio information related to the collected external sound. You can Such an information processing apparatus 130 may be, for example, an IC recorder, a smartphone equipped with application software for recording an external voice, or the like.

(5-2. Modified example in which digest generation function is provided)
With reference to FIG. 28, a modification in which the digest generating function is provided in the information processing device will be described. FIG. 28 is a functional block diagram showing an example of the functional configuration of the information processing apparatus according to the modified example provided with the digest generation function.

  With reference to FIG. 28, the information processing apparatus 140 according to the present modification has, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, an output voice generation unit 141, Have. Here, the functions of the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are similar to the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The output voice generation unit 141 includes various processors, and can output a digest of the voice information by a voice output device based on the voice information and the digest section information generated by the digest section determination unit 115. Generate in data format. When generating the digest, the output voice generation unit 141 may appropriately perform various known voice processes in consideration of the listening comfort of the user, such as performing a crossfade process on the joint between the digest sections. . The output voice generation unit 141 outputs the voice information (output voice information) corresponding to the generated digest to a voice output device such as a speaker. The digest is output as voice by the voice output device.

  The modification example in which the information processing device is provided with the digest generation function has been described above with reference to FIG. 28. As described above, according to this modification, the information processing apparatus 140 itself has a function of generating a digest, and the generated digest is stored in the audio output device or the information processing apparatus 140 provided in the information processing apparatus 140 itself. It can be output from an external audio output device.

  When the information processing device 140 itself has a voice output device and can reproduce the digest, the information processing device 140 may automatically generate the digest when the voice information is acquired. Further, in that case, in the information processing apparatus 140, for example, by an operation using a GUI (Graphical User Interface) such as placing a pointer on a file name indicating audio information on the display screen, or a simple operation such as a preview operation, The digest may be played. With the information processing device 140 configured in this manner, the user does not have to bother to perform the operation for generating the digest, and since the user can listen to the digest with a simple operation, it is as if the thumbnail in the video information. It is possible to confirm the digest of the voice information as if confirming, and the convenience for the user is further improved.

(5-3. Modified example in which voice information database is provided)
A modification in which the voice information database is provided in the information processing device will be described with reference to FIG. FIG. 29 is a functional block diagram showing an example of the functional configuration of the information processing apparatus according to the modified example in which the voice information database is provided.

  Referring to FIG. 29, the information processing apparatus 150 according to the present modification includes, as its functions, a feature amount extraction unit 111, a sound source type score calculation unit 113, a digest section determination unit 115, and a voice information database 151. Have. Here, the functions of the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 are similar to the functions of these functional blocks in the information processing apparatus 110 shown in FIG. Is omitted.

  The voice information database 151 is configured by a storage device such as an HDD, and stores the voice information in a database. By accessing the voice information database 151, the feature amount extraction unit 111 can extract the feature amount from arbitrary voice information in the voice information database 151. That is, according to the present modification, the feature amount extraction unit 111, the sound source type score calculation unit 113, and the digest section determination unit 115 for the database-based voice information in the storage unit provided in the information processing device 150, Various processes (feature amount extraction process, sound source type score calculation process, and digest section determination process) according to the embodiment described above are performed.

  The modification example in which the information processing apparatus is provided with the voice information database has been described above with reference to FIG. As described above, according to this modification, the information processing apparatus 150 itself has a database in which voice information is stored, and the digest section information of voice information in the database can be output.

(6. Hardware configuration)
Next, the hardware configuration of the information processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 30 is a block diagram showing an example of the hardware configuration of the information processing apparatus according to this embodiment. The information processing apparatus 900 illustrated in FIG. 30 can realize the functional configurations of the information processing apparatuses 110, 120, 130, 140, and 150 illustrated in FIGS. 1, 12, and 27 to 29, for example.

  The information processing apparatus 900 includes a CPU 901, a ROM (Read Only Memory) 903, and a RAM (Random Access Memory) 905. Further, the information processing device 900 may include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a communication device 921, a drive 923, and a connection port 925. The information processing apparatus 900 may have a processing circuit called DSP or ASIC instead of or in addition to the CPU 901.

  The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the information processing apparatus 900 or a part thereof according to various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 929. The ROM 903 stores programs used by the CPU 901, calculation parameters, and the like. The RAM 905 temporarily stores programs used in the execution of the CPU 901, parameters at the time of execution, and the like. The CPU 901, the ROM 903, and the RAM 905 are mutually connected by a host bus 907 configured by an internal bus such as a CPU bus. Further, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909. The CPU 901 can configure, for example, the feature amount extraction unit 111, the sound source type score calculation unit 113, the digest section determination unit 115, and the output sound generation unit 141 in the above-described embodiment.

  The host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

  The input device 915 is configured by a device operated by a user, such as a mouse, a keyboard, a touch panel, a button, a switch and a lever. Further, the input device 915 may be, for example, a remote control device (so-called remote control) using infrared rays or other radio waves, or an external connection device such as a mobile phone or a PDA corresponding to the operation of the information processing device 900. 931 may be sufficient. Further, the input device 915 is composed of, for example, an input control circuit that generates an input signal based on the information input by the user using the above-described operation unit and outputs the input signal to the CPU 901. By operating the input device 915, the user of the information processing device 900 can input various data to the information processing device 900 and instruct a processing operation. In the present embodiment, for example, an instruction to start the digest section determination process, a mode switching instruction, or the like may be input to the information processing apparatuses 110, 120, 130, 140, and 150 via the input device 915.

  In addition, the input device 915 may be a microphone that picks up surrounding sounds and inputs the surrounding sounds into the information processing device 900 as sound information. When the input device 915 is a microphone, the input device 915 can configure the sound pickup unit 131 in the above-described embodiment.

  The output device 917 is configured by a device capable of visually or auditorily notifying the user of the acquired information. Such devices include CRT display devices, liquid crystal display devices, plasma display devices, EL display devices and display devices such as lamps, audio output devices such as speakers and headphones, and printer devices. The output device 917 outputs results obtained by various processes performed by the information processing device 900, for example. Specifically, the display device visually displays the results obtained by various processes performed by the information processing device 900 in various formats such as text, images, tables, and graphs. On the other hand, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, and the like into an analog signal and outputs it audibly. In the present embodiment, for example, a digest of audio information generated by the information processing device 140 may be output via the audio output device. Further, a display related to a GUI for inputting various instructions via the input device 915 may be displayed on the display device.

  The storage device 919 is a device for storing data configured as an example of a storage unit of the information processing device 900. The storage device 919 includes, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like. The storage device 919 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like. The storage device 919 can configure, for example, the voice information database 151 in the above-described embodiment.

  The communication device 921 is, for example, a communication interface including a communication device or the like for connecting to a communication network (network) 927. The communication device 921 is, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 921 may be a router for optical communication, a router for ADSL (Asymmetrical Digital Subscriber Line), a modem for various kinds of communication, or the like. The communication device 921 can transmit and receive signals and the like to and from the Internet and other communication devices, for example, according to a predetermined protocol such as TCP / IP. The network 927 connected to the communication device 921 is configured by a network connected by wire or wirelessly, and may be, for example, the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, or the like. In the present embodiment, for example, the information processing devices 110, 120, 130, 140, 150, via the communication device 921, information processing devices 110, 120, 130, such as voice information, digest section information, output voice information, etc. Various types of information, which are inputs and outputs of 140 and 150, may be exchanged with an external device.

  The drive 923 is a reader / writer for recording medium, and is built in or externally attached to the information processing apparatus 900. The drive 923 reads the information recorded in the removable recording medium 929 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs it to the RAM 905. The drive 923 can also write information on a removable recording medium 929 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. The removable recording medium 929 is, for example, a DVD medium, an HD-DVD medium, a Blu-ray (registered trademark) medium, or the like. In addition, the removable recording medium 929 may be a compact flash (registered trademark) (CompactFlash: CF), a flash memory, an SD memory card (Secure Digital memory card), or the like. Further, the removable recording medium 929 may be, for example, an IC card (Integrated Circuit card) equipped with a non-contact type IC chip, an electronic device, or the like. In the present embodiment, for example, various types of information processed by the information processing devices 110, 120, 130, 140, 150 are read from the removable recording medium 929 by the drive 923 or written in the removable recording medium 929. Good.

  The connection port 925 is a port for directly connecting a device to the information processing device 900. Examples of the connection port 925 include a USB (Universal Serial Bus) port, an IEEE 1394 port, and a SCSI (Small Computer System Interface) port. Other examples of the connection port 925 include an RS-232C port, an optical audio terminal, and an HDMI (registered trademark) (High-Definition Multimedia Interface) port. By connecting the external connection device 931 to the connection port 925, the information processing apparatus 900 directly acquires various data from the external connection device 931 and provides the external connection device 931 with various data. In the present embodiment, for example, various types of information processed by the information processing devices 110, 120, 130, 140, 150 are acquired from the external connection device 931 via the connection port 925 or output to the external connection device 931. You may.

  The example of the hardware configuration capable of realizing the function of the information processing device 900 according to the present embodiment has been described above. Each component described above may be configured by using a general-purpose member, or may be configured by hardware specialized for the function of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at the time of implementing the present embodiment.

  A computer program for realizing each function of the information processing apparatus 900 according to the present embodiment as described above can be created and mounted on a PC or the like. It is also possible to provide a computer-readable recording medium in which such a computer program is stored. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed, for example, via a network without using a recording medium.

(7. Summary)
As described above, according to the present embodiment, the sound source type score indicating the probability of the sound source type of the sound included in the sound information is calculated, and the sound information is selected from the sound information based on the sound source type score. The digest section which comprises the digest of is determined. Therefore, for example, to generate only the music in the digest, to include only the human voice in the digest, to include the music and the human voice in the digest in a well-balanced manner, etc., to generate a digest that meets various user needs. Will be possible. Therefore, the convenience of the user can be further improved.

  Also, by setting the mode and appropriately adjusting the sound source type of the voice included in the digest, it becomes possible to generate a digest more in line with the user's intention. For example, in the multiple sound source mode, it is possible to generate a digest in which noise is reduced and which is more audible to the user by appropriately setting the mode, such as setting a low value for the ratio of the sound related to the noise score included in the digest. It is possible.

  Here, in general, for video information, by displaying a thumbnail, for example, it is possible to visually notify the user of the outline of the video information. However, mainly audio recording devices that acquire audio information instead of video information (for example, IC recorders, smartphones equipped with recording application software, wearable devices in situations where the camera function is not installed or the camera function cannot be used, etc. ), The file name of the audio information, the date and time of sound collection, etc. can be displayed visually, but it is difficult for the user to visually grasp the outline of the audio information from these information. Is. In addition, even if the video information is included with the audio information, the display screen cannot be seen in some situations, for example, when the backlight of the display screen is lighted up during an event in a dark room. In some cases, visual confirmation is not possible.

  In such a case, in order to understand the content of the audio information (or the audio information and the video information), the user needs to actually listen to the audio information. However, when the time length of the voice information is long, listening to the voice information in order to confirm the content causes a large time load and a heavy burden on the user.

  On the other hand, according to the present embodiment, as described above, it becomes possible to create a digest of audio information that meets the user's request. Therefore, for example, the content of the voice information can be grasped only by listening to the digest for several seconds, and the time required for the content confirmation, which has required a great amount of time until now, can be greatly shortened.

  Further, according to the present embodiment, for example, a digest of the acquired voice information is automatically performed by the device body that has recorded the voice or another device that manages the voice information after being moved to the storage. May be generated. When a digest is automatically generated for the acquired audio information, for example, an operation using a GUI such as placing a pointer on a file name representing the audio information on the display screen, a preview operation, or the like. The digest may be reproduced by a simple operation of. This allows the user to more easily check the digest without performing a troublesome operation.

  Further, the technology according to the present embodiment can be suitably applied to the purpose of analyzing so-called big data. For example, by applying the technology according to the present embodiment to call records collected by a call center or an investigative institution and generating a digest of the call records, it is possible to check the contents of a huge amount of call records in a shorter time. It becomes possible to do. Therefore, the analysis of the call record becomes easier.

  Further, even when the video information is included together with the audio information, it may be difficult to grasp the content by a visual method using a thumbnail based on the video information. For example, if there are multiple files that differ greatly only in the audio part for similar images, or if the image information cannot be used due to implementation restrictions such as the processing speed of the device, it is an image from a fixed-point camera, etc. Such a situation may be the case where no sound source is shown in the video (that is, the speaker cannot be identified). The technique according to the present embodiment can be suitably applied even when such video information cannot be effectively used for grasping the content.

  Further, the technique according to the present embodiment is effective in easily grasping the content of the audio information which is the material before the editing even in the operation of editing the audio information such as when editing a moving image. For example, in recent years, there is a service for generating and providing a photograph with voice information, which is a combination of still images and voice. The technique according to the present embodiment can be effectively used also when editing a voice portion when generating a file of a format in which a still image and voice are combined.

  Although the preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can conceive various changes or modifications within the scope of the technical idea described in the claims. It is understood that the above also naturally belongs to the technical scope of the present disclosure.

  Further, the effects described in the present specification are merely illustrative or exemplary, and are not limitative. That is, the technology according to the present disclosure may have other effects that are apparent to those skilled in the art from the description of the present specification, in addition to or instead of the above effects.

  Here, in this specification, expressions such as “less than or equal to” and “greater than” are used in the judgment processing in each processing procedure when comparing a score with a threshold value, etc. The above expression is merely an example, and does not limit the boundary condition in the determination process. In the present embodiment, when the value of the score or the like is equal to the threshold value, how to determine the magnitude relation may be set arbitrarily. In the present specification, the expression "less than or equal to" can be read as appropriate as the expression "less than", and the expression "greater than" can be read as appropriate as the expression "greater than or equal to". Is possible.

Note that the following configurations also belong to the technical scope of the present disclosure.
(1) Based on the calculated sound source type score, a sound source type score calculation unit that calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information; An information processing apparatus, comprising: a digest section determining unit that determines a digest section that configures the digest of.
(2) The information processing device according to (1), wherein the sound source type score includes at least one of a music score indicating a music likeness, a voice score indicating a human voice likeness, and a noise score indicating a noise likeness.
(3) The voice score includes a male voice score indicating a male voice likelihood, a female voice score indicating a female voice likelihood, a child voice score indicating a child voice likelihood, and a specific person likelihood making the voice. The information processing apparatus according to (2), further including at least one of the specific voice scores shown.
(4) The information processing according to any one of (1) to (3), wherein the sound source type score calculation unit calculates the sound source type score based on a feature amount indicating a feature of the voice information. apparatus.
(5) The feature amount is at least one of a power, a spectrum envelope shape, a number of zero crossings, a pitch, an MFCC, a correlation between sound pickup positions, and a physical amount indicating a characteristic of a sound source direction for the voice information. The information processing apparatus according to (4), including:
(6) The digest section determination unit determines a sound source type of the sound to be included in the digest based on a mode of the generated digest, and in the sound information, the sound source type score related to the determined sound source type is The information processing apparatus according to any one of (1) to (5), wherein a higher section is determined as the digest section.
(7) The mode is a single sound source mode in which the digest is generated so as to include only the sound of a single sound source type, and the digest is generated so as to include the sound in a plurality of sound source types at a predetermined ratio. A plurality of sound source modes, and a diversity reflection mode in which the digest is generated so that various sounds are included from among the sounds classified into the same sound source type; The information processing device according to 6).
(8) When the mode is the single sound source mode, the digest section determination unit determines, as the digest section, a section in which the sound source type score related to one designated sound source type is higher. The information processing device according to (7).
(9) When the mode is the plural sound source mode, the digest section determination unit sets the time length of the voice included in the digest for each sound source type, and the sound source type score is set for each sound source type. The information processing apparatus according to (7), wherein a higher section, in which the total length of the section is substantially equal to the time length for each set sound source type, is determined as the digest section.
(10) In the case where the mode is the diversity reflection mode, the digest section determination unit determines, within the same sound source type, variation in feature amounts indicating the characteristics of the audio information within the same sound source type. The information processing apparatus according to (7), wherein the variation of the time when the voice is uttered is calculated, and the digest section is determined so that the variation of the feature amount and the variation of the time become larger.
(11) The digest section determination unit continuously has a first section in which the sound source type score is higher than a predetermined threshold value and a second section in which the sound source type score is lower than a predetermined threshold value. And the time length of the second section is shorter than a predetermined time, the digest section is determined so as to include both the first and second sections. The information processing apparatus according to any one of (6) to (10).
(12) If the time length of the first section in which the sound source type score is higher than a predetermined threshold is a length that cannot be recognized as speech by a person, the digest section determination unit determines the first section. The information processing apparatus according to any one of (6) to (11), wherein the digest section is determined so as not to include the section.
(13) The sound source type score calculation unit calculates the sound source type score for the sound information that has been acquired in advance, and the digest section determination unit may calculate the sound information of the sound information that has been acquired in advance. The information processing apparatus according to any one of (1) to (12), which generates a digest.
(14) The sound source type score calculation unit newly obtains voice information of a time length corresponding to a score calculation section including a time of a length equal to or less than the digest section, for the voice information that is currently being just acquired. Each time is calculated the sound source type score for each score calculation section, the digest section determination unit, while the voice information is being acquired, while generating the digest of the voice information while updating at any time, The information processing apparatus according to any one of (1) to (12) above.
(15) When the time length of the voice information acquired so far is shorter than the set value of the time length of the digest, the digest section determining unit determines the newly acquired voice information. When added to the digest, the time length of the voice information acquired so far is equal to or greater than the set value of the time length of the digest, the voice for the newly acquired score calculation section The information processing device according to (14), wherein information is added to the digest and a section having a time length corresponding to the score calculation section and having a lower sound source type score is deleted from the digest. .
(16) A voice pickup unit that picks up an external voice is further provided, and the voice information is voice information relating to an external voice picked up by the voice pickup unit. ) The information processing device according to any one of 1).
(17) A storage unit for storing the database-converted voice information is further provided, and the sound source type score calculation unit calculates a sound source type score for the database-based voice information to determine the digest section. The section is the information processing device according to any one of (1) to (15), wherein the digest section is determined for the voice information stored in a database.
(18) Output voice generation that generates a digest of the voice information in a data format that can be output by a voice output device, based on the voice information and information about the digest section determined by the digest section determination unit. The information processing apparatus according to any one of (1) to (17), further including a unit.
(19) The processor calculates a sound source type score indicating the probability of the sound source type of the sound included in the sound information, and based on the calculated sound source type score, the processor extracts the sound information of the sound information from the sound information. Determining a digest section forming a digest, the information processing method.
(20) The processor of the computer has a function of calculating a sound source type score indicating the probability of the sound source type of the voice included in the voice information, and the voice from the voice information based on the calculated voice source type score. A program that realizes the function of determining a digest section that constitutes a digest of information.
(21) A digest of the voice information is configured from the voice information based on a feature amount extraction unit that extracts a feature amount of the voice included in the voice information and a sound source type calculated according to the feature amount. A digest section determining unit for determining a digest section to be, the digest section determining unit determines a sound source type of the voice included in the digest based on a preset mode, in the preset mode. Has a diversity reflection mode for generating the digest so that various sounds are included from the sounds classified into at least the same sound source type, and the mode is the diversity reflection mode In the above, the digest section determination unit is a feature quantity indicating the feature of the audio information in the same sound source type and / or the same sound source type. A program for functioning as an information processing device, which determines the digest section on the basis of the time when the voice is emitted within another.

110, 120, 130, 140, 150 Information processing device 111 Feature amount extraction unit 113 Sound source type score calculation unit 115 Digest section determination unit 131 Voice pickup unit 141 Output voice generation unit 151 Voice information database (DB)

Claims (20)

A feature amount extraction unit that extracts a feature amount of voice included in voice information,
Based on the sound source type calculated according to the feature amount, from the audio information, a digest interval determination unit that determines a digest interval configuring a digest of the audio information,
Equipped with
The digest section determination unit determines a sound source type of the sound to be included in the digest based on a preset mode, and the preset mode includes at least the sound sources classified into the same sound source type. the have a diversity reflecting mode for generating the digest to include diverse the sound from within,
When the mode is the diversity reflection mode, the digest section determination unit,
An information processing device for determining the digest section based on a feature amount indicating a feature of the audio information in the same sound source type and / or a time at which the sound is emitted in the same sound source type . The information processing apparatus according to claim 1, wherein the digest section determination unit determines the digest section such that the characteristic amount variation and / or the time variation is large. The digest section determination unit determines a sound source type of the voice to be included in the digest based on a preset mode, and the preset mode is specified to include at least the digest in priority. The information processing apparatus according to claim 1, having a single sound source mode that generates the digest so as to include the sound source type. When the mode is the single sound source mode, the digest section determination unit determines a section having a higher sound source type score related to the one designated sound source type as the digest section,
The information processing apparatus according to claim 3 . Before SL digest section determination unit, determines a sound source type of the audio to be included in the digest based on the mode set in advance, wherein the mode set in advance, said predetermined ratio of the audio of at least a plurality of sound source type The information processing apparatus according to claim 1 , further comprising a plurality of sound source modes for generating the digest so as to include the above. When the mode is the plural sound source mode, the digest section determination unit sets the time length of the voice included in the digest for each sound source type, and a section with a higher sound source type score for each sound source type And, the section in which the total length of the section is substantially equal to the time length for each of the set sound source types is determined as the digest section,
The information processing device according to claim 5 . The digest section determination unit determines a sound source type of the voice included in the digest based on the mode of the generated digest, in the audio information, a section with a higher sound source type score related to the determined sound source type. , Determined as the digest section,
The information processing device according to claim 1. The digest section determination unit continuously has a first section in which the sound source type score is higher than a predetermined threshold value and a second section in which the sound source type score is lower than a predetermined threshold value. And the time length of the second section is shorter than a predetermined time, the digest section is determined so as to include both the first and second sections.
The information processing device according to claim 7. The digest section determining unit determines the first section when the time length of the first section in which the sound source type score is higher than a predetermined threshold is a length that cannot be recognized as voice by a person. Determine the digest section so as not to include,
The information processing device according to claim 7. Further comprising a sound source type score calculator that calculates a sound source type score indicating the probability of the sound source type of the voice included in the voice information,
The sound source type score calculation unit calculates the sound source type score based on the feature amount indicating the feature of the audio information,
The information processing device according to claim 1. The sound source type score calculation unit calculates the sound source type score for the audio information that is acquired in advance,
The digest section determination unit generates the digest of the audio information, all of which are acquired in advance,
The information processing device according to claim 10. The sound source type score calculation unit, with respect to the voice information that is just being acquired at present, when the voice information of the time length corresponding to the score calculation section including the time of the digest section or less is newly acquired. In, calculating the sound source type score for each score calculation section,
The digest section determination unit generates while updating the digest of the voice information at any time while the voice information is being acquired,
The information processing device according to claim 10. The digest section determination unit, if the time length of the voice information acquired so far is shorter than a set value of the time length of the digest, the newly acquired voice information to the digest. Add
When the time length of the voice information acquired so far is equal to or greater than the set value of the time length of the digest, the newly acquired voice information for the score calculation section is added to the digest. Along with the deletion, a section having a lower sound source type score and having a time length corresponding to the score calculation section from the digest is deleted.
The information processing apparatus according to claim 12. A storage unit for storing the voice information in a database,
The sound source type score calculation unit calculates the sound source type score for the voice information stored in a database,
The digest section determination unit determines the digest section for the voice information stored in a database,
The information processing apparatus according to claim 10. The sound source type score includes at least one of a music score indicating music likeness, a voice score indicating human voice likeness, and a noise score indicating noise likeness,
The information processing device according to any one of claims 7 to 13. The voice score is a male voice score indicating a male voice likelihood, a female voice score indicating a female voice likelihood, a child voice score indicating a child voice likelihood, and a specific voice indicating a specific person likelihood making the voice. Further including at least one of the scores,
The information processing device according to claim 15. The feature amount includes at least one of a power, a spectrum envelope shape, a number of zero crossings, a pitch, an MFCC, a correlation between sound collection positions, and a physical amount indicating a characteristic of a sound source direction, for the voice information.
The information processing apparatus according to any one of claims 1 to 16. A voice pickup unit for picking up external voice is further provided,
The voice information is voice information about an external voice collected by the voice collecting unit.
The information processing apparatus according to any one of claims 1 to 17. An output voice generation unit that generates a digest of the voice information in a data format that can be output by a voice output device, based on the voice information and information about the digest period determined by the digest period determination unit. Further prepare,
The information processing apparatus according to claim 1.   Extract the voice feature amount included in the voice information,
  Based on the sound source type calculated according to the feature amount, from the audio information, to determine the digest section constituting the digest of the audio information,
  A sound source type of the voice to be included in the digest is determined based on a preset mode, and in the preset mode, various voices are selected from the voices classified into at least the same sound source type. Has a diversity reflection mode that produces the digest to be included,
  In the case where the mode is the diversity reflection mode, at the time when the feature amount indicating the feature of the voice information in the same sound source type and / or the voice in the same sound source type is emitted. An information processing method for determining the digest section based on the above.