JP2003255981A - Method, device and program for providing summary information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a summary distributing method which can be used by a job offerer to decide talent of many job seekers without taking a long time. <P>SOLUTION: Speech waveform information obtained from a speaker's speech, video information generated by photographing the speaking state of the speaker, and personal information that the speaker inputs are stored in a database and personal information stored in the database is retrieved according to retrieval conditions that a user requests to extract a speaker meeting the retrieval conditions; and a stressed speech section of the speaker is extracted as summary information from speech information of the speaker and video information corresponding to the summary speech information is extracted, so that the video information and summary speech information are distributed to the user. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a summary information providing method, a summary information providing apparatus, and a summary information providing program for determining a main part of the contents of video-added audio. For example, a job seeker's self-PR video is automatically generated. , It was applied to a human resource finding system that reduces the burden of recruiting activities.

[0002]

2. Description of the Related Art In the prior art, a job placement office, such as a public employment security office, has been established in the prior art. Job seekers and job seekers independently communicate and exchange various information, interview appointments, and simple interviews using videophones. There was a job placement system to help
For example, it is disclosed in Japanese Patent Laid-Open No. 11-143957. In addition, there was a system for connecting applicants and companies via a network and interactively managing information about applicants and companies. For example, it is shown in Japanese Patent Laid-Open No. 2001-202407.

There has also been a temporary worker registration system for diagnosing the suitability and performance of temporary workers from holding qualifications, years of work experience, desired job types, history information and the like. For example, it is shown in Japanese Patent Laid-Open No. 2001-229278. A computer that uses character data such as age, height, weight, sibling relationships, educational background, occupation, income, hobbies, and special skills for marriage introduction, etc., and, if necessary, self-introduction voices and images, and does not require an intermediary. There was a network.
For example, it is shown in Japanese Patent Laid-Open No. 6-19926.

In addition, according to the input items, skills, contents of work in charge, etc. are automatically extracted, and self PR points are also automatically extracted, and when the job seeker inputs the personal work history, the company side asks. There was a job hunting and job information system such as creating a summary. For example, Japanese Patent Laid-Open No. 20
No. 01-142939, etc. Also,
There was a system that allows the excavator side to easily apply for appeal information, including talents such as the subject's abstract, hobbies, thoughts, expressiveness, and singing ability, through auditions.
For example, it is shown in Japanese Patent Laid-Open No. 2000-305980.

[0005]

For example, in order for an employer to carry out recruitment activities, the employer examines the documents of the job seeker and interviews the job seeker one or more times to decide whether or not to hire. After the interview, only the documents are saved, and the impression of the job seeker can only rely on the memory of the employer. Further, even if it is recorded in a video or the like, it is not realistic to watch all the videos of the employer because it wastes time. Although there is a method such as the first selection based on documents, it is for reducing time waste, and the job seeker does not have to meet the job seeker himself or see the video of the job seeker It is impossible to judge whether it does not meet your wishes.

In Japanese Unexamined Patent Publication No. 2001-142939, skills, tasks in charge, etc. are automatically extracted according to input items, self-PR points are automatically extracted, and a job seeker inputs personal job history. Although a process such as creating a summary requested by the side is performed, these are derived from the text information, and it is impossible to judge the self-PR only by the text information, for example. Further, in Japanese Unexamined Patent Publication No. 2001-229278, the suitability and ability of the dispatched worker are judged, but it is not necessary to employ only the suitability and ability, and if it is possible to employ only these, an interview etc. is necessary. There is no. The adoption system that relies on text information is merely a simplification of the above-mentioned primary selection, and cannot be said to be a useful method.

Japanese Unexamined Patent Publication (Kokai) No. Hei 6-19926 (Japanese Unexamined Patent Publication No. 6-199326) registers photographs and images in addition to text-based self-data.
In the official gazette, we are developing a system that allows us to perform our own appearance by using images and videos for auditions, etc., but these eventually require time to reproduce the movie, so to understand the gist There is a limit in using functions such as fast forward. In Japanese Patent Laid-Open No. 2001-202407, job seekers and employers are connected via a network and information is interactively managed, but the configuration does not reduce the burden of recruitment. In addition, JP-A-11-1
In Japanese Patent No. 43957, a simple interview is realized by a TV phone or the like, but it is only through a network, and the burden of recruiting activity on the employer is not reduced other than moving the place for the interview. .

The present invention has been made in view of the above-mentioned drawbacks of the prior art, and efficiently summarizes the image provided by an information provider who provides personal information such as a job seeker. , A summary information providing method, a summary information providing apparatus, and a summary information providing program capable of drastically reducing the amount of work for browsing a large amount of information in a short time and searching for information that matches a purpose from a large number of information. It is the one we are trying to provide.

[0009]

In order to solve the above-mentioned problems, the most important feature of the present invention is to provide a summary information providing method for summarizing a video with audio provided by an information provider. is there. According to the present invention, a data storage means for storing an audio signal recorded at the same time as a video signal for each item and attribute information of the audio signal in association with each other, and at least a fundamental frequency or pitch period, power, and a time change characteristic of a dynamic feature amount. , Or using a codebook in which feature quantities including these inter-frame differences and appearance probabilities in the emphasized state are stored in association with each other,
Desired attribute information is input, attribute information satisfying the conditions indicated by the desired attribute information, and video signals and audio signals for each item corresponding to the attribute information are read out from the data storage means, and the audio signal is read for each frame. Obtaining the appearance probability in the emphasized state corresponding to the analyzed characteristic amount, calculating the probability of becoming the emphasized state based on the appearance probability in the emphasized state, the probability of becoming the emphasized state is larger than a predetermined probability A method for providing summary information, which determines a signal section as a summary section and outputs at least a part of the video signal of the summary section and the read attribute information, is proposed.

According to the present invention, further, in the summary section, the codebook has at least a fundamental frequency or a pitch period, a power,
The time-varying characteristic of the dynamic feature amount, or the feature amount including these inter-frame differences and the appearance probability in the quiet state corresponding to the appearance probability in the emphasized state are stored, and the speech signal is analyzed for each frame. Obtaining the appearance probability in the calm state corresponding to the feature amount, calculating the probability of becoming a calm state based on the appearance probability in the calm state, the probability ratio of the probability of becoming the emphasized state to the probability of becoming the calm state A voice that is calculated for each voice signal section, accumulates the times of the voice signal sections corresponding to the descending order of the probability ratios, and calculates the sum of the times of the summary sections, and the sum of the times of the summary sections becomes a predetermined summarization time. We propose a method for providing summary information that determines a signal section as a summary section.

According to the present invention, it is further determined for each frame whether the voice signal is in a silent section or in a voiced section, and a portion surrounded by a voiceless section of a predetermined number of frames or more and a section including a voiced section is determined as a voice sub-paragraph. However, the average power of voiced sections included in the audio sub-paragraph is determined to be a voice sub-paragraph group ending with a voice sub-paragraph that is smaller than a predetermined constant multiple of the average power in the voice sub-paragraph, and the audio signal The section is defined for each audio paragraph, and the summary time is obtained by accumulating the summary time for each audio paragraph, and the video signal and the audio of the summary section are calculated for each audio paragraph in descending order of the probability of the emphasized state or the probability ratio. We propose a method for providing summary information that outputs a signal.

According to the present invention, further, the data storage means for storing the audio signal recorded simultaneously with the video signal for each item and the attribute information of the audio signal in association with each other, at least the fundamental frequency or the pitch period, the power and the dynamic characteristics. Using a codebook in which the time change characteristics of the amount, or the feature amount including these inter-frame differences and the appearance probability in the emphasized state are stored in association with each other,
Desired attribute information is input, attribute information satisfying the conditions indicated by the desired attribute information, and video signals and audio signals for each item corresponding to the attribute information are read out from the data storage means, and the audio signal is read for each frame. An emphasis state probability calculation unit that calculates the appearance probability in the emphasized state corresponding to the analyzed feature amount and calculates the probability of becoming the emphasized state based on the appearance probability in the emphasized state, and the probability of becoming the emphasized state is predetermined. Providing summary information including a summary section determination unit that determines a voice signal section that is greater than the probability as a summary section, and an output unit that outputs at least a part of the video signal of the summary section and the read attribute information. Suggest a device.

The present invention further proposes a summary information providing program which is described by a computer readable code and executes any of the above summary information providing methods on a computer. [Operation] According to the present invention, the voice summarizing means analyzes the voice of the video provided by the information provider and extracts the important part (emphasized section) of the voice. Therefore, by connecting and reproducing the video of the important part of the audio, it becomes possible to strongly convey the gist of the video and the impression of the information provider, and it is possible to perform the summary of the provided information, which is the object of the present invention. become.

The data center summarizes the video of the speaker who is the information provider at an arbitrary time and the number of scenes. Therefore, it becomes possible to view the summary video at the time and the number of scenes desired by the information user (for example, a user who uses this summary information providing method), which is an object of the present invention. It will be possible to reduce the operation of hiring activities for employees. The information user watches the video of the desired information provider in a shorter time than the original video, focusing on the main part. Therefore, the image of the information provider can be strongly impressed on the information user, and the efficiency of the search activity, which is the object of the present invention, can be improved.

The information provider views the self-PR video by the information user. Therefore, it becomes possible for information providers to make self-appeal to information users in addition to text information.
The search activity independent of the text information, which is the object of the present invention, can be performed.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Here, a method for extracting a voice sub-paragraph, a method for extracting a voice paragraph, and a method for obtaining a probability of being in an emphasized state and a probability of being in a quiet state for each voice sub-paragraph used in the present invention will be described. . FIG. 17 shows a basic procedure of an embodiment of the previously proposed voice summarization method. In step S1, the input voice signal is analyzed to obtain a voice feature amount. Step S
In step 2, a voice sub-paragraph of the input voice signal and a voice paragraph composed of a plurality of voice sub-paragraphs are extracted. In step S3, it is determined whether the frame forming each audio sub-paragraph is in a calm state, emphasized state, or uttered state. Based on this determination, step S4
Create summary voice with and get summary voice.

An embodiment in which natural spoken language or conversational voice is applied to the summary will be described below. As the voice feature amount, one that is more stable than the spectral information even in a noisy environment and is less likely to depend on the speaker is used. The fundamental frequency (f0), the power (p), the time variation characteristic (d) of the dynamic feature amount of the voice, and the pause time length (silent section) (ps) are extracted from the input voice signal as the voice feature amount. The method of extracting these speech feature amounts is, for example, “acoustic / acoustic engineering” (Sadahiro Furui, Modern Science Co., 1998), “speech coding” (Takehiro Moriya, Institute of Electronics, Information and Communication Engineers, 1998), “Digital”. Speech processing "(Sadahiro Furui, Tokai University Press, 1985),
"Sound analysis algorithm based on complex sine wave model" (Shigeki Sagayama, Ph.D. thesis, 1998). The change over time in the dynamic feature amount of voice is a parameter that is a measure of the speech rate, and is disclosed in Japanese Patent No. 2976998.
You may use the thing of the No. That is, the time variation characteristic of the LPC spectrum coefficient that reflects the spectrum envelope as the dynamic variation is obtained, and the speech rate coefficient is obtained based on the time variation. More specifically, the LPC spectrum coefficient C1 (t), ... d (t) = Σi = 1k [Σf = t-f0t + f0 [f
× Ci (t)] / (Σf = t-f0t + f0f2) 2 where f0 is the number of preceding and following speech section frames (not necessarily an integral number of frames but may be a fixed time section), and k is the LPC spectrum. The order is i = 1, 2, ... K. As the coefficient of the speech rate, the number of maximum points of the change in the dynamic feature amount per unit time or the rate of change per unit time is used.

In the embodiment, for example, 100 ms is set as one frame and the shift is set as 50 ms. An average fundamental frequency is calculated for each frame (f0 '). Regarding the power, similarly, the average power (p ') for each frame is obtained. Further, the difference between f0 ′ of the current frame and f0 ′ before and after ± i frames is taken as ± Δf0′i (Δ component). Similarly for power, p ′ of the current frame and p ′ before and after ± i frames
And the difference ± Δp′i (Δ component) is calculated. f0 ', ± Δ
Normalize f0'i, p ', ± Δp'i. In this standard, for example, f0 ′ and ± Δf0′i are divided by the average fundamental frequency of the entire voice waveform and standardized. These standardized values are represented as f0 ″ and ± f0 ″ i. Similarly, p ′ and ± Δp′i are also normalized by dividing by the average power of the entire speech waveform that is the target of speech state determination. In normalizing, it may be divided by an average power for each audio sub-paragraph and audio paragraph described later. These normalized values are p ″, ± Δp ″
Denote by i. The value of i is, for example, i = 4. The number of peaks of the dynamic measure, that is, the number dp of the maximum points of the change of the dynamic feature amount, within ± T1 ms before and after the current frame, is calculated. Δ component (−Δdp) between this and the dp of the frame including the time T2ms before the start time of the current frame in the section
Ask for. Dp of the above-mentioned ± T1ms and the dp of the frame including the time T3ms after the end time of the current frame
And the Δ component (+ Δdp) of These T1, T
The values of 2 and T3 are, for example, T1 = T2 = T3 = 450 ms. The time length of the silent section before and after the frame is ± ps. In step S1, each value of these audio characteristic parameters is extracted for each frame.

FIG. 18 shows an example of the voice sub-paragraph of the input voice and the method of extracting the voice paragraph in step S2. Here, the audio sub-paragraph is used as a unit for determining the utterance state. In step S201, a silent section and a voiced section of the input voice signal are extracted. For example, the silent section is determined to be a silent section when the power of each frame is equal to or lower than a predetermined power value, and the voiced section is determined to be a voiced section when the correlation function of each frame is equal to or higher than a predetermined correlation function value. Voiced / unvoiced decision is
In many cases, the peak value of the autocorrelation function or the modified correlation function is used by equating it with the characteristic of periodicity / aperiodicity.
The autocorrelation function of the prediction residual obtained by removing the spectrum envelope from the short-time spectrum of the input signal is the modified correlation function, and the voiced / unvoiced determination is made depending on whether the peak of the modified correlation function is larger than a predetermined threshold value. Pitch cycle 1 / f0 (fundamental frequency f0) depending on the delay time that gives the peak
Is extracted. Details of the method for extracting these sections are described in, for example, “Digital Speech Processing” (Sadahiro Furui, Tokai University Press, 1985). Here, it has been described that each voice feature amount is analyzed for each frame from the voice signal. Good.

In step S202, when the time of each silent section surrounding the voiced section is t seconds or more, the portion including the voiced section surrounded by the silent section is set as a speech sub-paragraph.
This t is, for example, t = 400 ms. Step S20
In 3, the average power of the voiced section, preferably in the latter half of this audio sub-paragraph, is compared with the constant β times the average power value BA of the audio sub-paragraph. If the former is smaller, then The audio sub-paragraph is determined as the final audio sub-paragraph, and the audio sub-paragraph after the immediately preceding final audio sub-paragraph to the currently detected final audio sub-paragraph is determined as the audio paragraph.

FIG. 19 schematically shows a voiced section, a voice sub-paragraph, and a voice paragraph. The voice sub-paragraph is extracted under the condition that the time of the silent section surrounding the voiced section is t seconds. FIG. 19 shows audio subparagraphs j-1, j, and j + 1. Here, the speech subsection j is composed of n voiced sections, and the average power is Pj. As a typical example of the voiced section, the average power of the voiced section v included in the speech subsection j is pv. The voice paragraph k is extracted from the power of the voiced section in the latter half of the voice sub-paragraph j and the voice sub-paragraph. The average power pi of the voiced section from i = n-α to n
Is smaller than the average power Pj of the audio sub-paragraph j, that is, when Σpi / (α + 1) <βPj Expression (1) is satisfied, the audio sub-paragraph j is assumed to be the last audio sub-paragraph of the audio paragraph k. However, Σ is from i = n−α to n. Α and β in the equation (1) are constants, and these are manipulated to extract a voice paragraph. In the example, α was 3 and β was 0.8. In this way, a group of audio sub-paragraphs between the adjacent final audio sub-paragraphs can be determined as an audio paragraph with the final audio sub-paragraph as a delimiter.

FIG. 20 shows an example of the speech subparagraph utterance state determination method in step S3 in FIG. Step S3
At 01, the voice feature amount of the input voice sub-paragraph is vector-quantized. For this purpose, a codebook in which at least two quantized speech feature quantities (codes) are stored is created in advance. Here, the voice feature stored in the codebook is compared with the voice feature of the input voice or the voice already obtained by analysis, and the distortion (distance) between the voice features in the codebook is minimized. It is conventional to specify the quantized speech feature amount to be used.

FIG. 21 shows an example of the method of creating this codebook. The test subject listens to a large number of learning voices and labels the one in the quiescent state and the one in the emphasized state (S501). For example, as the reason why the test subject puts emphasis in the utterance, (a) utter a loud voice and utter to extend nouns and conjunctions (b) extend the beginning of the utterance to insist on topic change and collect opinions Make the voice louder (c) Make the voice louder to emphasize important nouns, etc. (d) High tones but not too loud (e) When laughing and cheating the real intention (f) ) Speaking or asking for consent from others, when the ending is high-pitched (g) Slowly and powerfully, when the ending is loud, when the ending is loud (h) Voice is loud and high, and speaks , I.e., louder than the other party (i) when making a real or secret statement that can be overwhelmed by a loud voice, or when uttering something important to a loud speaker (for example, a soft voice , He said that he was liable. In this example, the calm state is
It was assumed that the subject felt that the utterance was calm, not any of the above (a) to (i).

In the above description, the object to be determined as the emphasized state is the utterance, but the emphasized state can be specified by music. Here, in music with audio,
When trying to specify the emphasis state from the voice, the reasons for feeling emphasis are as follows: (a) loud voice and high voice (b) strong voice (c) high voice and strong accent (d) voice High voice, voice quality changes (e) voice is extended, and voice is loud (f) voice is high, voice is high and accent is strong (g) voice is voice, voice is high, and yell A high (h) voice with a high accent and a changing voice (i) a long voice, and a large voice with a high ending (j) a high voice and a long voice (k) a long voice, Also, screaming and high voice (l) Word rising, powerful (m) Slowly strengthening (n) Irregular tone (o) Irregular tone and high voice The emphasis state can be specified. The reasons for feeling the emphasis are as follows: (a) increase in power of the entire emphasized part (b) large difference in pitch between sounds (c) increase in power (d) change in number of musical instruments (e) change in tone and tempo Etc.

By creating a codebook based on these, not only utterance but also music can be summarized. For each label section in the quiet state and the emphasized state, the voice feature amount is extracted (S502) and parameters are selected (S50), as in step S1 in FIG.
3). A codebook is created by the LBG algorithm using the parameters in the label sections in the quiet state and the emphasized state (S504). For the LBG algorithm,
For example, (Y. Linde, A. Buzo and
R. M. Gray, "Analgorithm for
vector quantizer design
n, "IEEE Trans. Commun., vo
l. Com-28, pp. 84-95, 1980). The codebook size can be changed to 2 to the n-th power. For this codebook creation, it is preferable to use a voice feature amount standardized in voice sub-paragraphs or for each appropriate section longer than this, or the voice feature amount of the entire learning voice.

In step S301 in FIG. 20, the code feature is used to standardize the voice feature amount of the input voice sub-paragraph for each voice feature amount, and the standardized voice feature amount is collated for each frame or Vector quantization is performed to obtain a code (quantized speech feature amount) for each frame.
The voice feature quantity extracted from the input voice signal at this time is the same as the parameter used for the codebook creation. The likelihood of the utterance state is determined by using the code in the voice sub-paragraph to identify the voice sub-paragraph containing the emphasis state.
For the calm and stressed states. For this,
In advance, the appearance probability of an arbitrary code (quantized speech feature amount) is calculated for the case of the quiet state and the case of the emphasized state, and the appearance probability and the code are stored as a set in a codebook. An example of how to obtain this appearance probability will be described below. Codes (obtained for each frame) of the speech feature amount of one section (label section) given a label in the learning speech used for creating the codebook described above are Ci, Cj, Ck, ... Cn in time series. , The probability that the label section α is in the emphasized state is Pα (e), and the probability that the label section is in the stationary state is Pα (n). Pα (e) = Pemp (Ci) Pemp (Cj | Ci)
... Pemp (Cn | Ci ... Cn-1) = Pemp (C
i) ΠPemp (Cx | Ci ... Cx-1) Pα (n) = Pnrm (Ci) Pnrm (Cj | Ci)
... Pnrm (Cn | Ci ... Cn-1) = Pemp (C
i) ΠPnrm (Cx | Ci ... Cx-1). However, Pemp (Cx | Ci ... Cx-1) is a conditional probability that Cx is in an emphasized state next to the code sequence Ci ... Cx-1, and Pnrm (Cx | Ci ... Cx-1) is similarly Ci ... Cx-. It is the probability that Cx will be in a calm state with respect to 1. However, Π is a product of x = i + 1 to n. In addition, Pemp (Ci) quantizes the learning speech in frames, counts the number of Ci in these codes existing in the portion labeled as the emphasized state, and counts the count value for the total number of codes (the number of frames) of all the learning speeches. Pnrm (Ci) is a value obtained by dividing the number of Cis present in the portion labeled as a quiescent state by the total number of codes.

In order to simplify each state probability of this label section α, in this example, using the N-gram model (N <n), Pα (e) = Pemp (Cn | Cn-N + 1 ...
1) Pα (n) = Pnrm (Cn | Cn−N + 1 ... Cn−
1) That is, N-1 past code strings Cn from Cn
The probability that Cn will be obtained as an emphasized state next to -N + 1 ... Cn-1 is Pα (e), and similarly, the probability value of N-gram is linear with the probability value of lower-order M-gram (N ≧ M). It is preferable to apply a linear interpolation method that interpolates to For example, N-1 past code strings Cn-N + 1 ... Cn from Cn
−1, the probability that Cn is obtained in a calm state is Pα
(N). All of the conditional probabilities of Pα (e) and Pα (n) are obtained from the quantized code strings of the labeled learning speech, which correspond to the quantized code strings of the speech feature amount of the input speech signal. May not be obtained from the learning voice. Therefore, a high-order (that is, a long code string) conditional probability is obtained by interpolating a single occurrence probability and a lower-order conditional occurrence probability. For example, N = 3 trigr
am, bigram of N = 2, unigram of N = 1
Is used to perform the linear interpolation method. Regarding N-gram, linear interpolation method, and trigram, for example, “Spoken language processing” (Kenji Kita, Satoshi Nakamura, Masaaki Nagata, Morikita Publishing, 19
96, 29). That is, N = 3 (trigram): Pemp (Cn | Cn-2
Cn-1), Pnrm (Cn | Cn-2Cn-1) N = 2 (bigram): Pemp (Cn | Cn-
1), Pnrm (Cn | Cn-1) N = 1 (unigram): Pemp (Cn), Pnr
m (Cn), the probability of occurrence of Cn in these three emphasized states, and the probability of occurrence of Cn in three calm states are respectively calculated by the following equations, Pemp (Cn | Cn-2Cn-1), Pm
We will calculate nrm (Cn | Cn-2Cn-1). Pemp (Cn | Cn-2Cn-1) = [lambda] emp1Pemp (Cn | Cn-2Cn-1) + [lambda] emp2Pemp (Cn | Cn-1) + [lambda] emp3Pemp (Cn) Formula (2) Pnrm (Cn | Cn-2Cn-1) = [lambda] nrmlPnrm (). Cn | Cn-2Cn-1) + [lambda] nrm2Pnrm (Cn | Cn-1) + [lambda] nrm3Pnrm (Cn) Formula (3) When the learning data of Trigram is N, that is, the codes are C1, C2 ,. ． ． When the CN is obtained, the re-estimation formulas for λemp1, λemp2, and λemp3 are as follows from the above-mentioned reference “Spoken Language Processing”. λemp1 = 1 / NΣ (λemp1Pemp (Cn | C
n-2C-1) / (λemp1Pemp (Cn | Cn-
2C-1) + λemp2Pemp (Cn | C-1) + λ
emp3Pemp (Cn))) λemp2 = 1 / NΣ (λemp2Pemp (Cn | C
-1) / (λemp1Pemp (Cn | Cn-2C-
1) + λemp2Pemp (Cn | C-1) + λemp
3Pemp (Cn))) λemp3 = 1 / NΣ (λemp3Pemp (Cn) /
(Λemp1Pemp (Cn | Cn-2C-1) + λe
mp2Pemp (Cn | C-1) + λemp3Pemp
(Cn))) where Σ is the sum of n = 1 to N. Similarly, λnrm1, λnrm2, and λnrm3 are obtained in the same manner.

In this example, the label section α is the number of frames N
When the code obtained in α is Ci1, Ci2, ..., CiNα, the probability Pα that this label section α is in the emphasized state
(E), the probability Pα (n) of being in a calm state is as follows: Pα (e) = Pemp (Ci3 | Ci1Ci2) ... Pemp (CiNα | Ci (Nα-1) Ci (Nα-2)) Formula (4) Pα ( n) = Pnrm (Ci3 | Ci1Ci2) ... Pnrm (CiNα | Ci (Nα-1) Ci (Nα-2)) Formula (5) is obtained. To enable this calculation,
m, unigram, and bigram are obtained for arbitrary codes and stored in the codebook. That is, the codebook stores a set of the voice feature amount of each code, the appearance probability in the emphasized state, and the appearance probability in the quiet state in this example. The appearance probability in the emphasized state is the voice feature amount. Only the probability of appearing in an emphasized state irrespective of the voice feature amount in the past frame (unigram: described as a single appearance probability) or this and the voice feature amount in the past frame to the voice feature amount in the current frame For each frame-based audio feature quantity sequence, the audio feature quantity is either a combination with the conditional probability of appearing in the emphasized state, and the appearance probability in the quiet state is also the same as that of the previous frame Probability of appearing in a quiet state irrespective of the voice feature amount (unigr
am: written as a single appearance probability) or this and the voice feature quantity appears in a quiet state for each voice feature quantity sequence in frame units from the voice feature quantity in the past frame to the voice feature quantity in the current frame. It is either a conditional probability to perform or a combination.

For example, as shown in FIG. 10, for each code C1, C2, ... Each state is stored as a set. In step S302 in FIG. 20, the likelihood of the utterance state is calculated for the calm state and the emphasized state from the probabilities stored in the codebook for the codes of all the frames of the input speech sub-paragraph. FIG. 23 shows a schematic diagram of the example. Among the audio sub-paragraphs starting from time t, up to the fourth frame are indicated by ˜. As described above, here, the frame length is 100 ms and the frame shift is 50 ms, which is longer. Code Ci at frame number f and times t to t + 100, code C at frame number f + 1 and times t + 50 to t + 150
j is the frame number f + 2, time t + 100 to t + 2
At 00, code Ck is frame number f + 3, time t +
When the code Cl is obtained at 150 to t + 250, that is, when the code is Ci, Cj, Ck, Cl in the frame order,
The trigram can be calculated for frames with frame numbers f + 2 and above. Ps is the probability that the voice sub-paragraph s is in the emphasized state.
(E), the probability of being in a calm state is Ps (n)
The probabilities up to the frame are: Ps (e) = Pemp (Ck | CiCj) Pemp (Cl | CjCk) Formula (6) Ps (n) = Pnrm (Ck | CiCj) Pnrm (Cl | CjCk) Formula (7) Become. However, in this example, Ck,
Obtaining the individual appearance probabilities of the Cl emphasized state and the calm state,
The conditional probability that Ck appears next to Cj in the emphasized state and the stationary state, Ck is Ci, and Cj is C next to Cj.
The conditional probabilities that appear in the emphasized state and the calm state next to j and Ck, respectively, are obtained from the codebook as follows. Pemp (Ck | CiCj) = λemp1Pemp (Ck | CiCj) + λem p2Pemp (Ck | Cj) + λemp3Pemp (Ck) Formula (8) Pemp (Cl | CjCmpλ = λemp1Pemp (Cl | CjCk) + λemp (Pe) (λ | emp)) Cl) Formula (9) Pnrm (Ck | CiCj) = λnrm1Pnrm (Ck | CiCj) + λnr m2Pnrm (Ck | Cj) + λnrm3Pnrm (Ck) Formula (10) Pnrm (ClrCrCr) Prrm (Cr) Crn (CrCm) CnCrm) Cl | Ck) + λnrm3Pnrm (Cl) Expression (11) Probability Ps of being in the emphasized state up to the fourth frame shown in Expressions (6) and (7) using Expressions (8) to (11) above.
From (e), the probability Ps (n) of being in a calm state is obtained. Here, Pemp (Ck | CiCj) and Pnrm (Ck |
CiCj) can be calculated at frame number f + 2.

In this example, the audio sub-paragraph s has N frames.
When the code obtained in s is Ci1, Ci2, ..., CiNs, the probability Ps (e) that this speech subsection s is in the emphasized state
And the probability Ps (n) of being in a calm state is calculated by the following equation. Ps (e) = Pemp (Ci3 | Ci1Ci2) ... Pe
mp (CiNs | Ci (Ns-1) Ci (Ns-2)) Ps (n) = Pnrm (Ci3 | Ci1Ci2) ... Pn
rm (CiNs | Ci (Ns-1) Ci (Ns-2)) In this example, if these probabilities are Ps (e)> Ps (n), the audio subsection S is in the emphasized state, Ps (n). )> P
If s (e), it is in a calm state.

FIG. 24 shows a speech enhancement state determination device and speech using the speech sub-paragraph extraction method, the speech paragraph extraction method, and the method for obtaining the probability of being in the emphasized state and the probability of being in the quiet state for each speech sub-paragraph described above. 1 illustrates an embodiment of a summarizing device. Input voice (input voice signal) for which the voice enhancement state should be determined in the input unit 11 or the voice summary should be detected
Is entered. The input unit 11 also includes a function of converting an input voice signal into a digital signal as needed. The digitized voice signal is stored in the storage unit 12 as needed. The voice feature amount extraction unit 13 extracts the voice feature amount described above for each frame. The extracted voice feature amount is normalized by the average value of the voice feature amount as necessary, and the quantization unit 1
In 4, the voice feature amount of each frame is quantized with reference to the codebook 15, and the quantized voice feature amount is sent to the emphasis probability calculation unit 16 and the quietness probability calculation unit 17. The code book 15 is, for example, as shown in FIG.

The appearance probability in the emphasized state of the quantized speech feature quantity by the emphasis probability calculation unit 16 is calculated by using the corresponding probability stored in the codebook 15, for example, equation (8) or (9). Calculated by Similarly, the quietness probability calculation unit 17 uses the corresponding probability that the appearance probability of the quantized speech feature amount in a quiet state is stored in the codebook 15, for example, according to Expression (10) or (11). Calculated. The appearance rate in the emphasized state, the appearance probability in the quiet state, and the voice feature amount of each frame calculated for each frame by the emphasis probability calculation unit 16 and the quietness probability calculation unit 17 are stored in the storage unit together with the frame number assigned to each frame. Store in 12.

The control of each of these units is sequentially performed under the control of the control unit 19. An embodiment of the voice summarization device is shown in FIG.
A broken line block is added to the solid line block.
That is, the audio feature amount of each frame stored in the storage unit 12 is sent to the silent section determination unit 21 and the sound section determination unit 22, and the silent section determination unit 21 determines whether each frame is a silent section or not. Also, the voiced section determination unit 22 determines for each frame whether or not it is a voiced section. The silent segment determination result and the voiced segment determination result are input to the audio sub-paragraph determining unit 23. Based on the silent section determination and the voiced section determination, the speech subsection determining unit 23 determines that a portion including a voiced section surrounded by a continuous silent section of a predetermined number of frames is a speech subsection based on the determination of the voiced section. To determine. The determination result of the audio sub-paragraph determining unit 23 is written in the storage unit 12, added to the audio data sequence stored in the storage unit 12, and the audio sub-paragraph number sequence is given to the frame group surrounded by the silent section. At the same time, the determination result of the audio sub-paragraph determination unit 23 is input to the final audio sub-paragraph determination unit 24.

The trailing voice sub-paragraph determining section 24 detects the trailing voice sub-paragraph by the method described with reference to FIG.
The voice paragraph determination unit 25 determines that a portion including a plurality of voice sub-paragraphs between two end voice sub-paragraphs is a voice paragraph. This audio paragraph determination result is also written in the storage unit 12, and an audio paragraph sequence number is given to the audio small paragraph number sequence stored in the storage unit 12. When operating as a voice summarizing device, the emphasis probability calculation unit 16 and the quietness probability calculation unit 17 read the emphasis probability and the quietness probability of each frame forming each audio subparagraph from the storage unit 12, and the probability of each audio subparagraph is calculated. For example, it is calculated by equation (8) and equation (10). The emphasis state determination unit 18 compares the probability calculation values for each audio sub-paragraph to determine whether or not the audio sub-paragraph is in the emphasized state, and the summary segment extraction unit 26 determines even one audio sub-paragraph in the audio paragraphs. If there is one that is determined to be in the emphasized state, the audio paragraph including the audio sub-paragraph is extracted. The control of each unit is performed by the control unit 19.

From the above, it can be understood that the method of separating the speech waveform composed of speech into the speech sub-paragraphs and the speech sub-paragraphs, and the probability of being in the emphasized state and the probability of being in the quiet state can be calculated for each of the speech sub-paragraphs. Embodiments relating to a voice processing method, a voice processing device, and a voice processing program according to the present invention using the above-described methods will be described below.
FIG. 25 shows the basic procedure of the embodiment of the voice processing method of the present invention. In this embodiment, the voice emphasis probability calculation process is executed in step S11 to obtain the emphasis probability and the quietness probability of the voice sub-paragraph.

In step S12, a summary condition input step S12 is executed. In this summarization condition input step S12, for example, information for prompting the user to input the summarization time or summarization rate or compression rate is provided, and the summarization time or summarization rate or summarization rate or compression rate is input. It is also possible to adopt an input method of selecting one from a plurality of preset summarization times or summarization rates and compression rates. In step S13, the operation of repeating the change of the extraction condition is executed, and the extraction condition satisfying the summarization time or the summarization ratio and the compression ratio input in the summarization condition input step S12 of step S12 is determined.

In step S14, the abstract extraction step is executed. In this abstract extraction step S14, the voice paragraph to be adopted is determined using the extraction condition determined in the extraction condition change step S13, and the total extension time of this voice paragraph to be adopted is calculated. In step 15, a summary reproduction process is executed,
The voice paragraph string extracted in the abstract extraction step S14 is reproduced. FIG. 26 shows details of the voice enhancement probability calculating step shown in FIG. In step S101, the speech waveform string to be summarized is separated into speech sub-paragraphs.

In step S102, a voice paragraph is extracted from the voice sub-paragraph string separated in step S101. As described with reference to FIG. 19, the audio paragraph is a unit which is composed of one or more audio sub-paragraphs and whose meaning can be understood. Step S
103 and the probability that each audio subparagraph will be in an emphasized state by using the codebook described in FIG. This is called the emphasis probability) Ps (e)
And the probability of being in a calm state (hereinafter referred to as calm probability) Ps
(N) is obtained.

In step S105, the emphasis probability P obtained for each audio sub-paragraph in steps S103 and S104.
The s (e), the quietness probability Ps (n), and the like are sorted for each voice subparagraph and stored in the storage unit as a voice emphasis probability table. FIG. 27 shows an example of the voice enhancement probability table stored in the storage means. 27, F1, F2, F3 ... Show the speech sub-paragraph emphasis probability Ps (e) obtained for each speech sub-paragraph,
The small paragraph probability memory | storage part which recorded the audio small paragraph calm probability Ps (n) is shown. These subparagraph probability storage units F1, F2, F
3 ... is the audio sub-paragraph number i attached to each audio sub-paragraph S.
The start time (time counted from the beginning of the language string), the end time, the audio subparagraph emphasis probability, the audio subparagraph calm probability, the number of frames fn forming each audio subparagraph, and the like are stored.

As a condition input in the summarization condition input step S12, a summarization rate X (summarization rate according to claim 1) indicating that the total length of contents to be summarized is summarized in a time of 1 / X (X is a positive integer). Input) or the summary time t. In response to the setting of the summary condition, the weighting factor W is set to W = 1 as an initial value in the extraction condition changing step S13, and this weighting factor is input to the summary extracting step S14. The summary extraction step S14 compares the emphasis probability Ps (e) and the quietness probability Ps (e) stored for each voice sub-paragraph from the voice emphasis probability table with the weighting factor W = 1, and W · Ps (e) A voice sub-paragraph having a relation of> Ps (n) is extracted, and a voice paragraph including even one of the extracted voice sub-paragraphs is extracted to obtain a total extension time MT (minutes) of the extracted voice paragraph string. .

Total extension time MT of the extracted voice paragraph sequence
(Minutes) is compared with a predetermined summary time YT (minutes) determined by the summary condition. If MT≈YT (the error of MT with respect to YT is within a range of ± several percent, for example), the adopted audio paragraph string is reproduced as a summary audio. The error value between the summarization time YT set in the summarization condition and the summed total extension time MT of the content is larger than the stipulation, and the relationship is M.
If T> YT, the total extension time MT of the extracted audio paragraph sequence
(Minutes) is determined to be longer than the summary time YT (minutes) defined by the summary conditions, and the extraction condition changing step S1 shown in FIG.
Re-execute 3. In the extraction condition changing step S13, the total extension time MT of the voice paragraph sequence extracted when the weighting factor is W = 1.
In response to the determination result that (minutes) is “longer” than the summarization time YT (minutes) defined in the summarization conditions, the weighting coefficient W (the predetermined coefficient according to claim 1) smaller than the current value is added to the emphasis probability Ps (e). If it is larger than the current value, multiply by W · Ps
(E) Then, weighting is performed. The weighting factor W is obtained by, for example, W = 1-0.001 × K (K is the number of loops).

That is, in the arrangement of the emphasis probabilities Ps (e) found in all the voice sub-paragraphs of the voice paragraph sequence read from the voice emphasis probability table, W = 1−1 in the first loop.
Weighting is performed by multiplying the weighting coefficient W = 0.999 determined by 0.001 × 1. The emphasis probabilities W · Ps (e) of all the weighted voice sub-paragraphs are compared with the quietness probabilities Ps (n) of the voice sub-paragraphs, and W · Ps (e)> Ps.
An audio sub-paragraph having a relationship of (n) is extracted. In accordance with this extraction result, in the abstract extraction step S14, the voice paragraph including the extracted voice sub-paragraph is extracted, and the summary voice paragraph string is obtained again. At the same time, the total extension time MT (minutes) of this summary voice paragraph sequence is calculated, and this total extension time MT (minutes) is compared with the summary time YT (minutes) defined by the summary condition. If the comparison result is MT.apprxeq.YT, the audio paragraph string is determined as the summary audio and reproduced.

If the result of the first weighting process is still MT> YT, the extraction condition changing step is executed as a second loop. At this time, the weight coefficient W is W = 1
It is calculated by −0.001 × 2. All emphasis probabilities Ps (e)
Is weighted with W = 0.998. As described above, in this example, the extraction condition is changed such that the value of the weighting coefficient W is gradually decreased every time the loop is repeatedly executed, so that the voice amount satisfying the condition of WPs (e)> Ps (n) is reduced. You can gradually reduce the number of paragraphs. As a result, it is possible to detect the state of MT≈YT that satisfies the summary condition.

In the above description, MT≈YT is set as the convergence condition of the summarization time MT, but it is also possible to strictly set MT = YT. In this case, if the summary condition is insufficient for 5 seconds, for example, if another voice paragraph is added, it will exceed 10 seconds, but by reproducing only 5 seconds from the voice paragraph, the summary condition of the user is met. be able to. Further, the 5 seconds may be 5 seconds near the audio sub-paragraph determined to be emphasized, or 5 seconds from the beginning of the audio paragraph. When it is determined that MT <YT in the above-mentioned initial state, the weighting factor W is smaller than the current value, for example, W = 1-0.001 × K.
Then, the weighting coefficient W is multiplied by the array of the calm probability Ps (n) to weight the calm probability Ps (n). As another method, when it is determined that MT> YT in the initial state, the weighting coefficient is made larger than the current value W = 1 +
0.001 × K, and the weighting factor W is set to the calm probability Ps.
The array of (n) may be multiplied.

In addition, in the summary reproducing step S15, the audio paragraph string extracted in the abstract extracting step S14 is described as being reproduced. However, in the case of image information with voice, the image information corresponding to the audio paragraph extracted as the summary voice is reproduced. It is possible to summarize a television broadcast, a movie, or the like by cutting out, connecting, and playing back together with the sound. Further, in the above description, it has been described that either the emphasis probability or the quietness probability obtained for each audio sub-paragraph stored in the audio emphasis probability table is directly multiplied by the weighting coefficient W to perform weighting. For good detection, it is desirable to perform weighting as WF by multiplying the weighting coefficient W by the number F of frames forming each audio sub-paragraph.

That is, the conditional emphasizing probability Ps (e) calculated by the equations (8) and (10) is a product of the probabilities of the emphasizing state obtained for each frame. Further, the probability Ps (n) of being in a calm state is also calculated by multiplying the probability of being in a calm state calculated for each frame. Therefore, for example, in order to weight the emphasis probability Ps (e), correct weighting is performed by multiplying the weighting coefficient W for each probability of the emphasis state obtained for each frame. In this case, if the number of frames forming the audio sub-paragraph is F, the weighting factor W is W
It becomes F.

As a result, the influence of the weighting is increased or decreased according to the number F of frames, and the audio sub-paragraph having a larger number of frames, that is, the audio sub-paragraph having a longer extension time is given a larger weight. However, if it suffices to simply change the extraction condition for determining the emphasized state, the extraction condition may be obtained by multiplying the product of the probability of the emphasized state or the product of the calm state obtained for each frame by the weighting factor W. Can be changed. Therefore, the weighting coefficient W does not necessarily have to be WF.

Further, in the above, as the extraction condition changing means, the emphasis probability Ps (e) or the quietness probability Ps (n) obtained for each speech sub-paragraph is weighted and Ps (e)> Ps (n).
I adopted the method of changing the number of audio sub-paragraphs that satisfy
As another method, the emphasis probability Ps (e) of all audio sub-paragraphs
And the calm probability Ps (n), the probability ratio Ps (e) /
Ps (n) is calculated, and voice signal sections (voice sub-paragraphs) corresponding to the descending order of the probability ratios are accumulated to calculate the sum of the summary sections. If they match, a method of deciding the voice signal section at that time as a summary section and organizing the summary voice may be considered.

In this case, when the total extension time of the organized summary voices is more or less than the summarization time set in the summarization condition, the probability ratio Ps for determining the emphasized state is set.
The extraction condition can be changed by changing the threshold value for selecting the value of (e) / Ps (n). When this extraction condition changing method is adopted, it is possible to obtain an advantage that the process up to organizing the summary voice satisfying the summary condition can be simplified. In the above description, the emphasis probability Ps (e) and the quietness probability Ps (n) obtained for each audio sub-paragraph are calculated as the product of the probability of the emphasized state and the product of the probability of the calm state calculated for each frame. However, as another method, the average value of the probabilities of the emphasized state obtained for each frame is obtained, and the average value is used as the emphasis probability Ps (e) and the quietness probability Ps of the audio sub-paragraph.
It can also be used as (n).

Therefore, when the calculation method of the emphasis probability Ps (e) and the calm probability Ps (n) is adopted, the weighting coefficient W used for weighting is directly multiplied by the emphasis probability Ps (e) or the calm probability Ps (n). do it. An embodiment of a voice processing device capable of freely setting the summarization rate will be described with reference to FIG. In this embodiment, the summary condition input unit 31, the voice emphasis probability table 32, the emphasized sub-paragraph extraction unit 33, the extraction condition change unit 34, and the summary section provisional determination are added to the configuration of the voice emphasis state summarizing device shown in FIG. The unit 35, the total extension time calculation unit 35A for obtaining the total extension time of the summary voice inside the summary section provisional determination unit 35, and the total extension time calculation unit 35A.
The total extension time of the summary voice calculated by the summary condition input unit 31
The summary section determination unit 35B that determines whether or not the summary time that has been input in the setting range is included, and the summary voice storage / playback unit 35 that stores and reproduces the summary voice that matches the summary condition.
It is characterized in that it is provided with C.

As described with reference to FIG. 23, the speech feature amount of the input speech is obtained for each frame, and the emphasis probability calculation unit 16 and the quietness probability calculation unit 17 calculate the enhancement probability and the quietness probability for each frame according to the speech feature amount. And are stored in the storage unit 12 together with the frame number assigned to each frame. Further, to this frame string number, the audio small paragraph string number assigned to the audio small paragraph string determined by the audio small paragraph determination unit is added, and an address is assigned to each frame and audio small paragraph. In the voice processing device according to the present invention, the emphasis probability calculation unit 16 and the quietness probability calculation unit 17 are the storage unit 1.
The emphasis probability and the quietness probability of each frame stored in 2 are read out, and the emphasis probability Ps (e) and the quietness probability Ps (n) are obtained for each speech sub-paragraph from the emphasis probability and the quietness probability,
The emphasis probability Ps (e) and the quietness probability Ps (n) are stored in the voice emphasis table 32.

The voice enhancement table 32 stores the enhancement probability and the quietness probability obtained for each voice sub-paragraph of the voice waveform of various contents, and is arranged so that the summary can be executed at any time according to the user's request. There is. The user inputs the summary condition into the summary condition input unit 31. The summarization condition mentioned here indicates the name of the content to be summarized and the summarization rate for the total length of the content. As the summarization rate, an input method such as summarizing the entire length of content to 1/10 or summarizing to 10 minutes in time can be considered. Here, for example, 1
When / 10 is input, the digest time calculation unit 31A calculates a time that is 1/10 of the total time of the content, and sends the calculated digest time to the digest segment determination unit 35B of the digest segment temporary determination unit 35.

In response to the summary condition being input to the summary condition input unit 31, the control unit 19 starts the operation of generating the summary voice. As the starting work, the emphasis probability and the quietness probability of the content desired by the user are read from the voice emphasis table 32. The read emphasis probabilities and quietness probabilities are sent to the emphasis subparagraph extraction unit 33, and the voice subparagraph numbers determined to be in the emphasis state are extracted. As a method of changing the condition for extracting the voice section in the emphasized state, the above-mentioned emphasis probability Ps (e) or the quietness probability Ps (n) is multiplied by a weighting coefficient W which is the reciprocal of the probability ratio, and W · Ps ( e)> Ps
A method of extracting a voice sub-paragraph having a relationship of (n) and obtaining a summary voice by a voice paragraph including a voice sub-paragraph, and a probability ratio Ps
(E) / Ps (n) is calculated, and the probability ratio is accumulated in descending order to obtain the digest time.

As an initial value of the extraction condition, when changing the extraction condition by weighting, it is conceivable that the weighting coefficient W is set to W = 1 to be the initial value. Further, the emphasis probability Ps (e) and the quietness probability Ps (n) obtained for each audio sub-paragraph
When the emphasis state is determined according to the value of the probability ratio Ps (e) / Ps (n) of Ps (e) / Ps (n), the value of the ratio is, for example, Ps as an initial value.
It may be considered that the case where (e) / Ps (n) ≧ 1 is determined as the emphasized state. Data indicating the audio sub-paragraph number, start time, and end time determined to be emphasized in this initial setting state are output from the emphasized sub-paragraph extraction unit 33 to the summary section provisional determination unit 3
Send to 5. The summary section provisional determination unit 35 searches the voice paragraph string stored in the storage unit 12 for a voice paragraph including the emphasized small paragraph number determined to be in the emphasized state, and extracts the voice paragraph. The total extension time calculation unit 35A calculates the total extension time of the extracted speech paragraph sequence, and the summary section determination unit 35B compares the total extension time with the summary time input under the summary condition. If the comparison result satisfies the summarization condition, the sound paragraph sequence is stored and reproduced by the summarization sound storage / reproduction unit 35C. In this reproducing operation, an audio paragraph is extracted from the number of the audio sub-paragraph determined to be in the emphasized state by the emphasized sub-paragraph extracting unit 33, and the audio data or the video data of each content is specified by specifying the start time and the end time of the audio paragraph. Is read out and transmitted as summarized audio and summarized video data.

When the summary section determination unit 35B determines that the summarization conditions are not satisfied, the summary section determination unit 35B outputs an extraction condition change command to the extraction condition change unit 34, and the extraction condition change unit 34 is extracted. To make changes. The extraction condition changing unit 34 changes the extraction condition and inputs the extraction condition to the emphasized small paragraph extracting unit 33. Emphasized subparagraph extraction unit 3
In accordance with the extraction condition input from the extraction condition changing unit 34, 3 again makes a comparison determination between the emphasis probability and the quietness probability of each audio subparagraph stored in the audio emphasis probability table 32.

The extraction result of the emphasized small paragraph extracting unit 33 is sent to the summary section temporary judging unit 35 again, and the speech paragraph including the audio small paragraph judged to be emphasized is extracted. The total extension time of the extracted voice paragraph is calculated, and the summary section determination unit 35B determines whether or not the calculation result satisfies the summary condition. This operation is repeated until the summarization condition is satisfied, and the audio paragraph string satisfying the summarization condition is read from the storage unit 12 as the summarized audio and summarized video data and distributed to the user terminal. As described above, the method of separating the audio waveform into audio sub-paragraphs and audio sub-paragraphs, the probability of being emphasized and the probability of being calm for each audio sub-paragraph can be calculated, and the voice summarization rate can be freely changed to set an arbitrary length. It can be understood that the summary voice of Sa can be obtained.

In the above description, the start time and the end time of the summary section are extracted as the start time and the end time of the audio paragraph string that is determined to be the summary section, but in the case of the content with video, it is determined to be the summary section. The cut point of the video signal approaching the start time and end time of the audio paragraph sequence is detected by, for example, the means described in Japanese Patent Laid-Open No. 8-32924, and the time of this cut point (using a signal generated when switching screens) A method of defining the start time and end time of the summary section can be considered. In this way, when the cut points of the video signal are used for the start time and end time of the summary section, the switching of the summary section is synchronized with the switching of the image, so that the visibility is improved visually and the comprehension of the summary is improved. The advantage that can be obtained is obtained.

Embodiments relating to the summary information providing method, the summary information providing apparatus, and the program thereof according to the present invention using the above-described methods will be described below. [First Embodiment] As a first embodiment, for example, hiring students who are going to graduate, hiring temporary workers,
An embodiment in which the summary information providing method according to the present invention is applied to a human resource finding system such as part-time work recruitment will be described. Figure 1
Example 1 of the present invention is shown in FIG. The summary information providing apparatus according to the present invention is arranged on the output side of the data center 300, the network 200, the job seeker registration unit 100 for inputting personal information such as applicants and the audio signal with video as attribute information. The output unit 309, the employer excavation unit 400, and the billing unit 500 are provided. The job seeker registration unit 100 registers that the job seeker is a job seeker. The registration data is sent to the data center 300 via the network 200 and stored in the database together with other attribute information.

In the recruiter excavation unit 400, recruitment condition information is input as desired attribute information, and a summary portion of the applicant's video-added audio signal having attribute information satisfying the desired attribute information is received from the data center 300, and this is received. The employer views the reproduced video or audio as a clue for selecting a job seeker. The billing unit 500 carries out a billing process in the data center 300 along with processes such as providing data to an employer and inputting data from a job seeker. For example, receiving a billing request signal corresponding to each process from the data center, deducting the consideration equivalent to each process from the financial balance in each user financial account, or the amount equivalent to the usage fee to the financial balance in the financial account of the data administrator. Is added.

FIG. 2 shows an example of the configuration of the job seeker registration unit 100. The job seeker registration unit 100 includes a personal information registration unit 101,
The image capturing unit 102, the storage recording unit 103, and the data center transmitting unit 104 are included. The personal information registration unit 101 inputs the individual attribute information of the job seeker. The terminal used for input may be a personal computer, a home electric appliance capable of inputting / outputting information, or a mobile phone. FIG. 3 is a typical example of a personal information registration screen. For example, step SI10
Enter the name of the job seeker in 1-1, and step SI101-
Enter the age in step 2, enter the address in step SI101-3, enter the telephone number in step SI101-4, select the desired occupation in step SI101-5, and select the desired number of working days / week in step SI101-6. Is selected, and the work form is selected in step SI101-7, and step SI1 is selected.
Select the desired annual income in 01-8, step SI101-9
Enter the educational background with, and enter the license etc. in step SI101-10. All of the steps SI101-1 to SI101-10 may be selection expressions or description input expressions.

The registration contents of the personal information registration screen shown in FIG. 3 are arbitrary, and the contents may be set by the operator of the data center 300, which will be described later, or by the employer excavation unit 400. Good. In addition, it is possible to use free notation for all job seekers. 4 is a block diagram of the image capturing unit 1 shown in FIG.
In 02, the video signal and the audio signal are simultaneously photographed and acquired as a PR image of the job seeker himself. FIG. 4A shows a camera 102-
1 shows how to record a self PR. Camera 102-
1 may be a commercially available video camera, a personal computer, or a camera attached to a mobile phone capable of shooting a moving image. Also, it may be digitally recorded or analog recorded, and if it is digitized, it may or may not be compressed. If it is compressed, which compression format is used? But it's okay.

FIG. 4B shows a job seeker performing self-promotion toward the camera 102-1. The items used in the self-PR, for example, educational background and work history may be set by the operator of the data center 300, which will be described later, or may be set by the employer. In addition, it is possible to use free notation for all job seekers. FIG. 5 shows items for self-promotion when set by the operator of the data center 300 or the employer. For example, when a job seeker advertises each item, he / she may decide a rule to say the item name such as "My name is ...", or set a rule to photograph each item. The PR start time of each item may be indicated by, for example, video switching, or the rule may not be determined at all.

The storage recording unit 103 digitizes the imaged data in a personal computer or the like and stores it as a file. At this time, it does not matter whether or not the digitized file is compressed, and in the case of compression, any compression format may be used. The data center transmission unit 104 (FIG. 2) includes the personal information registration unit 1
The personal information registered in 01 and the self-PR video stored in the storage recording unit 103 are transmitted to the data center 300 described later. As a transmission method, for example, the digitized self-PR video file may be transmitted to the data center 300 via the network 200. However, when digitizing the self-PR video in the data center 300,
The procedure of digitizing and storing in the storage recording unit 103 is unnecessary.

The network 200 is the Internet, L
It may be any of AN, telephone line, BS, CS and CATV. For example, it may be operated by a network intermediary such as an internet provider. FIG. 6 shows an example of the configuration of the data center 300. The data center 300 includes a job seeker data input unit 301, a job seeker personal information database 302, a self PR audio / video database 303,
Employment condition input unit 304, search unit 305, self PR audio / video summarization unit 306, self PR audio / video delivery unit 307
And an employer evaluation unit 308 and a contact unit 309.

The job seeker data input unit 301 inputs the job seeker attribute information and the self PR audio / video file transmitted from the data center transmission unit 104 (FIG. 2), and the job seeker personal information database 302 and the self PR voice Save in the video database 303. The employment condition input unit 304 inputs a employment condition item which is desired attribute information from the employment person.
FIG. 7 shows an example of employment condition items which are desired attribute information.
For example, an employer designates educational background, work experience, etc. as a condition for recruitment. The adoption condition items may be created according to some format, for example, on a personal computer, or may be selected using a device such as a mouse. The input may be input from a home page on the Internet, for example. The search unit 305 searches the job applicant personal information database 302 for attribute information that matches the desired attribute information input by the employer in the recruitment condition input unit 304. If there is no attribute information that matches the desired attribute information, the attribute information closest to the desired attribute information is searched. The self PR audio / video summarizing unit 306 summarizes the self PR video corresponding to the attribute information searched by the searching unit 305.

FIG. 8 shows an example of the self PR audio / video summarizing section 306. The self PR audio / video summarization unit 306 includes a summarization condition input unit 306-1 and a self PR audio / video input unit 306-2.
A video / audio separation unit 306-3, a self PR item detection unit 306-4, a cut point extraction unit 306-5, a voice recognition unit 306-6, an audio / video summarization unit 306-8, and a video editing. And a unit 306-9. The summary condition input unit 306-1 inputs, for example, the desired attribute information input by the employer from a terminal corresponding to the employer excavation unit 400 via the network 200 or the data center 300.
Can be set by the operator. Input can be made on a screen of a personal computer, for example.

FIG. 9 shows a typical example of a screen for inputting summary conditions. As a summary condition, it is selected whether the viewing of the self-PR video per person is set by the viewing time or the number of scenes of the video. In the example shown in FIG. 9, step SI306-1-1 is selected and the circle on the left is checked. If you select a viewing time,
At step SI306-1-2, the viewing time desired by the employer is input as the summary time, and the self-PR video is summarized at approximately this viewing time for each job seeker. In FIG. 9, it is set to 30 seconds. When the number of viewing scenes is selected, the number of scenes desired by the employer is input in step SI306-1-4.

In the self PR video input unit 306-2, the self P
The self-PR video of the human resources searched by the search unit 305 is input from the R audio / video database 303, and the video / audio separation unit 306-3 separates the audio from the video. However, the original self-PR video is saved as it is with audio, and the separated audio is saved in, for example, a hard disk or a memory of a computer. When performing the audio processing described below, the audio separated by the video / audio separating unit 306-3 is used. When performing video processing, a self-PR video with audio is used.

The self PR item detection unit 306-4 uses the self P
Detect R item. When the PR item of the self-PR video is set in advance by the operator or the employer of the data center 300, as described above, the job seeker takes a picture for each PR item and temporarily stops the recording by the camera before and after that. The difference in the video information between the frames becomes remarkable, and the cut point appears in the video. The cut point extraction unit 306-5 can obtain the start time and end time of each PR using the cut points.
Also, when the job seeker registers with the data center 300, P
A self-PR video file may be created for each R item. When starting each PR item, for example,
In the 7th PR item of FIG. 5, the job seeker speaks “What I want to do most about future work ...”, and the voice recognition unit 306-7 can obtain the start time and end time of each PR. .

The voice recognition method is described, for example, in Japanese Patent Laid-Open No. 8-6588. The audio / video summarizing unit 306-8 detects the emphasized state of the utterance by the emphasized state determination unit described above, extracts the audio paragraph as a unit whose meaning can be understood by listening, and connects the audio paragraphs including the emphasis to each other. A summary voice is generated by the method described in 1. At the same time, the video corresponding to the summary voice section is cut out to obtain the summary video information. FIG. 10 is a schematic diagram of means for self-PR video summarization. In step SI306-8-1, the above-described emphasis probabilities are obtained in time series, and step SI306-
In 8-2, a voice paragraph is extracted. Step SI306-8
If the audio paragraph extracted in -3 is an audio paragraph including emphasis, the corresponding self-PR video of step SI306-8-4 is set as a candidate to be used for the self-PR summary video. Regarding the summary condition set by the summary condition input unit 306-1 (FIG. 8), for example, regarding the self-PR video summarization method in the case where the self-PR viewing time shown in FIG. It shows in FIG. Examples will be described below.

At step SI306-8-5, the voice paragraph section including the emphasized state section is extracted. In step SI306-8-6, the emphasis probabilities of the respective voice paragraphs are determined in descending order of the emphasis probabilities obtained for the respective voice paragraphs or the probability ratios of the emphasis probabilities to the probabilities of being in a quiet state. In step SI306-8-7, in order to create a self-PR viewing time of, for example, 30 seconds per person, the playback time for each audio paragraph is accumulated for each defined emphasis order, and the given viewing time (30 in this case) is accumulated. Seconds) determines the number of audio paragraphs that most closely matches In the example shown in FIG. 11, step S
This is an example of a self-PR summary video of 30 seconds when three audio paragraphs (audio paragraphs determined to be acceptable) are joined in accordance with the order of emphasis given in I306-8-6. In Step SI306-8-8, the above Step SI306-
The playback order is determined for the three audio paragraphs determined to be acceptable in 8-7. For example, the reproduction may be performed in time series as shown in step 306-8-8.
The reproduction may be performed according to the order of emphasis given in I306-8-6. Self-promotion in step SI306-8-9
Create a summary video. Step S for self-promotion summary video
It is generated by connecting audio paragraphs according to the reproduction order determined in I306-8-8.

The self-PR audio / video delivery unit 307 (see FIG. 6) uses the self-PR summary audio / video created by the self-PR video summarization unit 306 and attribute information including personal information to find the employer excavation unit 4.
Send to 00. After the recruiter excavator 400 views the job seeker's self PR summary audio / video received from the self PR audio / video distributor 307, the employer evaluation information is input (by watching the job seeker's video), and the data center is entered. It is transmitted to the employer evaluation receiving unit 308 of 300. The evaluation of the hired employees obtained here is transmitted to the job applicant terminal, which is the job applicant registration unit 100, as necessary.

FIG. 13 is a flow chart for explaining the operation of the employer excavation unit 400. In step SI402, the employer inputs the hiring conditions. For example, conditions are input for the items shown in FIG. Self PR in step SI403
Enter the video summary conditions. For example, input as shown in FIG. In step SI404, the self-PR summary video is viewed, and in step SI405, information indicating whether to view the self-PR summary video again is input. When viewing the self-PR summary video again, in step SI406, information indicating whether it is the same as the adoption condition of step SI402 is input. If not, the adoption condition of step SI402 is input again. If the adoption conditions in step SI402 are the same, in step SI407, information indicating whether the self-PR summarization conditions are the same is input.
The self-PR video summarization condition of 403 is input again.

For example, when viewing a 30-second self-PR summary video and considering positively for hiring a job seeker, 2
After the first time, there is a usage method such as viewing a 60-second self-promotion summary video. If the user does not want to view the self-PR video again in step SI405, information indicating whether to view the self-PR video is input in step SI408. If the original self-PR video is viewed instead of the summary, step SI4
09, the self-PR video is viewed, and in step SI410, the information indicating whether or not to view the self-PR video again is input. If the self-PR video is viewed again, the process goes to step SI409. Enter information indicating whether or not.

Similarly, if the user does not wish to view the self-PR video in step SI408, the user inputs information indicating whether or not to interview the job seeker in step SI411. If an interview is desired, in step SI412, information indicating the interview request is transmitted to the terminal of the job seeker. Here, the interview may include, for example, contacting the job seeker from the job seeker contact section 309 (see FIG. 6), setting a location, and interviewing the employer with the job seeker.
The employer may contact the job seeker directly to determine the interview location. Instead of the direct interview, the interview may be performed using an Internet home appliance such as an Internet telephone.

If the interview with the job seeker is not desired in step SI411, information indicating whether or not the job seeker is hired is input in step SI413. Also when deciding to hire, the decision information is transmitted to the terminal of the job seeker in step SI412. When the adoption is not decided, the information indicating that the adoption is suspended is input in step SI414. Select whether to consider later. If a signal indicating selection later is input, information indicating that the job seeker is employed is retained in step SI415, and if information indicating that the job seeker is not held is input, information indicating not employed is transmitted to the job seeker terminal. To do. In step SI416, information indicating whether to view the self-PR summary video of job seekers who have not yet viewed the self-PR summary video, other than the hiring pending person, is input. In step SI412, the information indicating the interview request is transmitted to the job seeker's terminal, and similarly in step SI416, the information indicating whether to view the job seeker's self-PR summary video is input. Also, step SI415
Similarly, after putting a job seeker on hold in step SI41
In step 6, information indicating whether to watch the self-PR summary video is input.

If another job seeker wishes to watch, step SI406 is executed and the subsequent steps are repeated. When inputting information that another job seeker does not wish to view in step SI416, information indicating whether or not to view the held job seeker's self-PR summary video is input in step SI417, and step SI406 is selected. By repeating the above, the job seekers desired by the employer are determined, and step SI
It ends at 418. FIG. 14 shows a procedure for charging, which is executed by a computer constituting the charging unit 500. The charging procedure includes a job seeker registration fee charging step SI501, a user registration fee charging step SI502, a self PR summary video viewing fee charging step SI503, a self PR video viewing fee charging step SI504, and an intermediary fee charging step SI50.
5 and is satisfied if any one of the above applies.

In the job seeker registration fee department charging step SI501, the registration fee is charged when the job seeker registers in the data center for job seeking. That is, this charging process is executed, for example, by a procedure of deducting the amount equivalent to the registration fee from the financial balance in the job seeker's financial account and adding the amount equivalent to the registration processing fee to the financial balance in the financial account of the data manager. As the registration fee, for example, set an annual registration fee of 3000 yen,
During that period, the job seeker can register personal information, self-PR video, etc. in the data center 300. In the employer registration fee charging step SI502, the fee for the employer to register in the data center for recruitment is charged. This billing process is executed, for example, by a procedure of deducting the amount equivalent to the registration fee from the financial balance in the employer financial account, and adding the amount equivalent to the registration processing fee to the financial balance in the financial account of the data administrator.

In this case, the registration fee is set to, for example, an annual registration fee of 10,000 yen, and during that period, the employer views the job seeker information registered in the data center 300 and the self-promotion video of the job seeker. be able to. In the self-PR summary video viewing fee billing step SI503, billing processing is performed according to the number of job seekers viewed by the employer and the viewing time of the self-PR summary video. This billing process is also executed, for example, by deducting from the financial balance in the employer financial account the amount equivalent to the usage charge corresponding to the number of viewers or viewing time, and adding the amount equivalent to the usage fee to the financial balance in the financial account of the data administrator. To be done. As the usage fee, for example, a fee of (100 yen / job seeker) may be set. Alternatively, (100
0 yen / 1 hour) may be set. Also, each time an employer views the self-promotion summary video of the job seeker, the viewing fee may be set to, for example, (100 yen / employer), and the viewing fee equivalent amount may be deducted from the job seeker's financial account. .

Self-PR video viewing fee charging step SI50
In No. 4, when the employer views the self-PR video, the employer is charged an amount corresponding to the usage fee according to the number of job seekers the viewer has watched and the viewing time of the self-PR video. This charging process is executed by a procedure of subtracting the usage charge equivalent data from the employer financial account balance data and adding the usage charge equivalent data to the data manager financial account balance. The usage fee for the self-PR video can be set to, for example, (1000 yen / job seeker). Or (1000 yen / 1
Time). Also, when the employer views the self-PR summary video, set it as (1000 yen / employer),
In this case, a job seeker who has viewed the self-PR summary video may be charged a usage fee corresponding to the number of the hired employees.

In the intermediary fee charging step SI505, FIG.
A fee is charged when contacting the job seeker in step SI412 shown in (4), for example, the employer is charged 1000 yen per interview. Or 1 for each hiring decision
The form may be such that 0000 yen is charged to the employer, or 10000 yen is charged to the job seeker. As is clear from this result, there is an improvement in enabling self-PR video summarization by using the audio summarization technique as compared with the conventional technique. Further, it becomes possible to watch the self-PR summary video at the time and the number of scenes that the employer desires, and there is an improvement that reduces the operation of the recruiting activity of the employer. In addition, the recruiter can freely watch the video of the job seeker who wants, and the image of the job seeker can be strongly impressed to the employer, which improves the efficiency of the recruiting activity.
In addition, job seekers will be able to view self-PR videos by employers and make self-appeal in addition to text information.
There was an improvement that job hunting activities that did not depend on text information were possible. [Second Embodiment] As an application of the first embodiment, a second embodiment will now be described with reference to FIGS. 15 and 16.

The summary condition input unit 306-1 shown in FIG.
A self PR video input unit 306-2, a video / audio separation unit 306-3, a self PR item detection unit 306-4, a cut point extraction unit 306-5, a voice recognition unit 306-6,
The voice summarizing unit 306-8 is the same as that shown in FIG. In the second embodiment, after the processing of the voice summarizing unit 306-8, the voice keyword extracting unit 306-9 and the facial expression extracting unit 3 are executed.
06-10, the point that the process of the video editing unit 308-11 is performed, and the point that the procedure of performing the process of the text summarizing unit 306-12 after the voice recognition process in the voice recognizing unit 306-6 is adopted. is there.

In the voice keyword extracting step executed by the voice keyword extracting unit 306-9, the probability (keyword likelihood) indicating the likelihood of a word being repeatedly emphasized from the spoken words is obtained. For keyword extraction, see, for example, "Reference Interval-free continuous DP for spotting with arbitrary intervals of standard patterns.
(RIFCDP) "(Yoshiaki Ito, Jiro Kinoshita, Hiroshi Kojima, Seki
Susumu, Ryuichi Oka, IEICE Technical Report, SP95-34, 1995-
06) and the like. The facial expression extraction method executed by the facial expression extraction unit 306-10 is described in, for example, Japanese Patent Laid-Open No.
The method shown in, for example, Japanese Patent Laid-Open No. 1-2232456 can be used. A time series of probabilities (basic facial expression likelihood) showing the likelihood of basic facial expressions (anger, disgust, fear, sadness, happiness, surprise) with reference to no facial expression.

In the video editing unit 306-11, the audio summarizing unit 3
In addition to the audio paragraph section information determined to be the emphasized state obtained in 06-8, the video section or the basic facial expression likelihood in which the keyword likelihood obtained in the keyword extraction unit 306-9 is equal to or more than a predetermined first threshold is predetermined. It is also possible to extract a video section equal to or more than the second threshold value as a summary section and use this summary section as a self-PR summary video. FIG. 16 shows a schematic diagram of the self-PR summary video creating method of the second embodiment. Step SII306-11-1 is the speech paragraph section including emphasis, step SII306-11-2 is the emphasis probability, step SII306-1-3 is the keyword likelihood, and step SII306-11-4 is the basic facial expression likelihood. I want each.

Step SII306-11-2, Step SII306-11-3, Step SII306-11-4
The probability value obtained in step SII306-11-1 is further extracted from the voice paragraph section determined to be the emphasized state obtained in step SII306-11-1 based on this probability value. For example, a section having a probability value larger than a predetermined threshold may be determined as a summary section and sequentially combined to generate a self-PR summary video. In the multiplication of probability values, the three probabilities may be weighted with different contribution rates. For example, weighting may be performed by increasing the effect of the emphasis probability and decreasing the effect of the basic facial expression likelihood.

As described above, according to the second embodiment, the keyword likelihood is used, as compared with the conventional technique, so that the self-PR summary video includes the wording habit of the job seeker, and Self PR including key points of job seekers
There was an improvement that made it possible to create a summary video. In addition, by using the basic facial expression likelihood, it becomes possible to create a self-PR summary video that includes the job seeker's facial expression included only in the video information, and the employer adds the job seeker's unique facial expression to the recruitment evaluation. There were improvements that would be possible. The summary information providing method according to the present invention described above is used in the data center 3
This can be realized by executing the summary information providing program of the present invention by the computer constituting 00. Here, the program is downloaded via a communication line, or installed from a storage medium such as a CD-ROM or a magnetic disk to a processing unit such as a CPU and executed.

[0087]

As is apparent from this result, the use of the voice summarization technique has the effect that the summarization of the self-PR video can be automated as compared with the conventional technique. Further, it becomes possible to view the self-PR summary video at the time and the number of scenes desired by the employer, which has an effect of reducing the operation of the recruiting activity of the employer. In addition, the recruiter can freely watch the video of the job seeker who desires, and the image of the job seeker can be strongly impressed to the employer, which has the effect of increasing the efficiency of the recruiting activity. In addition, the job seeker can view the self-PR video by the employer and can make a self-appeal in addition to the text information, which has the effect of being able to perform job seeking activities that do not depend on the text information.

Further, by using the keyword likelihood described in the second embodiment, the self-PR summary video contains the wording and habits of the job seeker, and the self PR summary including the key points of the job seeker for the job seek. There was an effect that it was possible to create a video. In addition, by using the basic facial expression likelihood, it becomes possible to create a self-PR summary video that includes the job seeker's facial expression included only in the video information, and the employer adds the job seeker's unique facial expression to the recruitment evaluation. The effect that becomes possible is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram for explaining the basic configuration of a summary information distribution system according to the present invention.

FIG. 2 is a block diagram for explaining the configuration of a job seeker registration unit shown in FIG.

3 is a flowchart for explaining a procedure for registering personal information performed by a job seeker registration unit shown in FIG.

FIG. 4 is a diagram for explaining a data acquisition situation of the image capturing unit shown in FIG.

FIG. 5 is a diagram showing an example of items to be PR when the job seeker shown in FIG. 2 shoots a self-PR video.

FIG. 6 is a block diagram for explaining an internal configuration of the data center shown in FIG.

FIG. 7 is a diagram showing an example of inputting an adoption condition into an adoption condition input unit shown in FIG.

8 is a block diagram for explaining an example of a configuration of a self-PR audio / video summarizing unit shown in FIG.

9 is a diagram showing an example of inputting a summary condition to a summary condition input section shown in FIG.

10 is a flowchart for explaining the operation of the audio / video summarizing unit shown in FIG.

FIG. 11 is a flowchart for explaining the operation of the video editing unit shown in FIG.

12 is a flowchart showing an example of inputting a telop of a self-PR item to the self-PR summary audio / video information shown in FIG.

FIG. 13 is a flowchart showing an example of a procedure until an employer decides to adopt in the employer excavation section shown in FIG.

FIG. 14 is a flowchart showing an example of a charging procedure in the charging unit shown in FIG.

FIG. 15 is a flowchart for explaining the second embodiment of the present invention.

16 is a flow chart of the self-P in the video editing unit shown in FIG.
6 is a flowchart for explaining an example of generating an R summary audio-video.

FIG. 17 is a flowchart for explaining the previously proposed voice summarizing method.

FIG. 18 is a flowchart for explaining a previously proposed method of extracting a voice paragraph.

FIG. 19 is a diagram for explaining the relationship between audio paragraphs and audio subparagraphs.

20 is a flowchart showing an example of a method of determining the utterance state of an input voice sub-paragraph in step S2 shown in FIG.

FIG. 21 is a flowchart showing an example of a procedure for creating a codebook used in the previously proposed speech summarization method.

FIG. 22 is an example showing a storage example of a codebook used in the present invention.

FIG. 23 is a waveform chart for explaining a speech state likelihood calculation.

FIG. 24 is a block diagram for explaining an embodiment of the previously proposed voice emphasis state determination device and voice summarization device.

FIG. 25 is a flowchart for explaining a summarization method in which the summarization rate can be changed freely.

FIG. 26 is a flowchart for explaining an operation of extracting a voice sub-paragraph used for summarizing a voice, an operation of calculating an emphasis probability of each voice sub-paragraph, and an operation of extracting a voice sub-paragraph calm probability.

FIG. 27 is a view for explaining the structure of a speech emphasis probability table used in the speech summarization device.

FIG. 28 is a block diagram for explaining an example of a voice summarizing device capable of freely changing the summarization rate.

[Explanation of symbols]

100 Job Seeker Registration Department 200 networks 300 data centers 301 Job Seeker Data Input Section 302 Job Seeker Personal Information Database 303 Self PR audiovisual database 304 Employment condition input section 305 Search section 306 Self PR audio / video summary section 307 Self PR audio / video distribution unit 308 Recruiter evaluation receiver 309 Contact 400 Recruiter excavation department 500 Billing department

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) G10L 11/02 G10L 3/00 521U 15/00 H04N 5/91 N 15/04 G10L 3/00 551G 15/06 513B 15 / 10 531N H04N 5/91 551A 513A (72) Inventor Osamu Mizuno 2-3-1, Otemachi, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation (72) Inventor Hidego Kuwano Two Otemachi, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation (72) Inventor Haruhiko Kojima 2-3-1 Otemachi, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation F-term (reference) 5B075 ND12 ND14 NK06 NS01 UU08 5C053 FA14 JA01 JA30 LA01 5D015 CC18 FF00

Claims (5)

[Claims] 1. A data accumulating means for accumulating an audio signal recorded simultaneously with a video signal for each item and attribute information of the audio signal, and at least a basic frequency or pitch period, power, and time of a dynamic feature amount. The desired attribute information is input by using a codebook that stores the change characteristics or the feature amount including the difference between frames and the appearance probability in the emphasized state in association with each other, and satisfies the condition indicated by the desired attribute information. The attribute information and the video signal and the audio signal for each item corresponding to the attribute information are read from the data accumulating unit, and the appearance probability in the emphasized state corresponding to the feature amount obtained by analyzing the audio signal for each frame is obtained, The probability of becoming the emphasized state is calculated based on the appearance probability in the emphasized state, and the speech signal section in which the probability of becoming the emphasized state is larger than a predetermined probability is determined as the summary section, A method for providing summary information, comprising outputting a video signal of about a section and at least a part of the read attribute information. 2. The summarization interval is a feature quantity including at least a fundamental frequency or a pitch period, power, a temporal change characteristic of a dynamic feature quantity, or a feature quantity including a difference between frames of these codebooks and an appearance probability in an emphasized state. Correspondingly stored appearance probability in a quiet state, obtain the appearance probability in a quiet state corresponding to the feature amount analyzed the audio signal for each frame, and a quiet state based on the appearance probability in the quiet state The probability ratio is calculated, the probability ratio of the probability of being in the emphasized state to the probability of being in the quiet state is calculated for each voice signal section, and the time of the voice signal section corresponding to the descending order of the probability ratio is accumulated to summarize the section. 2. The method for providing summary information according to claim 1, further comprising: calculating a sum of the times of the above, and determining a voice signal section in which the sum of the times of the summary sections is a predetermined summary time as the summary section. 3. It is determined whether or not the voice signal is a soundless section for each frame, and whether or not it is a voiced section, and a portion surrounded by a soundless section of a predetermined number of frames or more and including a voiced section is determined as a voice sub-paragraph, The average power of the voiced section included in the audio sub-paragraph is determined as a voice paragraph a voice sub-paragraph group ending with a voice sub-paragraph less than a predetermined constant multiple of the average power in the voice sub-paragraph, the voice signal section is It is defined for each audio paragraph, the summarization time is obtained by accumulating for each audio paragraph, the video signal and the audio signal of the summary section for each audio paragraph in descending order of the probability of the emphasis state or the probability ratio. The summary information providing method according to claim 1, wherein the summary information is output. 4. A data accumulating means for accumulating an audio signal recorded simultaneously with a video signal for each item and attribute information of the audio signal, and at least a fundamental frequency or pitch period, power, and a dynamic feature quantity. By using a codebook in which the time-change characteristics or the feature amount including the difference between frames and the appearance probability in the emphasized state are stored in association with each other, desired attribute information is input, and the condition indicated by the desired attribute information is set. Satisfied attribute information and item-specific video signals and audio signals corresponding to the attribute information are read from the data storage means, and the probability of appearance in the emphasized state corresponding to the feature amount obtained by analyzing the audio signal for each frame is obtained. An emphasis state probability calculation unit that calculates a probability of becoming an emphasis state based on an appearance probability in the emphasis state, and summarizes a voice signal section in which the probability of becoming the emphasis state is greater than a predetermined probability. A summary information providing device, comprising: a summary section determination unit that determines a section; and an output unit that outputs at least a part of the video signal of the summary section and the read attribute information. 5. A summary information providing program, characterized by being written in a computer-readable code, for executing any of the summary information providing methods according to any one of claims 1 to 4 on a computer.