JP2753255B2 - Voice-based interactive information retrieval device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval system using speech recognition, and is particularly suitable for a user to search for necessary information from ambiguous memory while freely interacting with the system. The present invention relates to an information retrieval apparatus suitable for improving the performance of speech recognition.

[Conventional technology]

There have been many attempts to use speech as an input / output interface for information retrieval systems and the like. However, the reality is that it has not yet reached a practical level, mainly due to the lack of performance of the speech recognition unit. Under these circumstances, IEICE Transactions D, Vol.J70-D, No.11, pp2121-2127 (1
In order to realize a smooth man-machine interface in voice-based interactive services as in the technology described in October 987), the ambiguity of input caused by the recognition system is used by using the relevance between input information. And IEICE Technical Report Vol.87, No.209,
As in the technology described in NL87-11 (1987), a speech comprehension system extracts the subjects of the input content from all input candidates and efficiently narrows the vocabulary associated therewith to improve the recognition performance. There is something to measure.

[Problems to be solved by the invention]

Any of the above prior arts requires input information (for example, word voice)
Although it is excellent as a technique for narrowing down recognition candidates by using the relationship between them, the former requires that input attributes (eg, location, date and time, name, etc.) be known, and as a result, system-driven input The problem is that the interface becomes an interface, uncertain contents cannot be entered in the input of each attribute, and the latter conventional technology is a language processing technology such as sentence comprehension and cannot be directly used for information retrieval systems. There was a problem.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems of the prior art, and enable a user to search for information from an ambiguous memory while freely interacting with the system, and to improve the recognition performance, and to improve the recognition performance. Is to provide.

[Means for solving the problem]

The above-described object is to recognize recognition candidates (a plurality of words for each input word) obtained by recognizing freely input voices (for example, a plurality of word voices) without limiting the attributes of the input. By using only the information that describes the relationship between the words, only the combinations of the word candidates to be related are extracted to narrow down the recognition candidates with high accuracy, and the information is searched using the word sequence of the finally obtained recognition candidates. Achieved. Since a plurality of recognition candidates obtained in the above process can be narrowed down with high accuracy, the recognition performance is improved as a result.

Information describing the relationship between words corresponding to the content to be searched can be easily obtained using the concept network technology described in detail in, for example, Hitachi Review, vo1.69, No. 3 (1987-3). Is achieved.

[Action]

The concept network describing the relationship between words according to the present invention is used as a judgment material for narrowing down recognition candidates of an input word. For example, when three words “voice”, “recognition”, and “technology” are input, the first and second candidates of the recognition result for each input are (“hot spring”, “voice”), (“recognition”, respectively). , "Minshuku"), ("fact", "technology"), the concept network has "speech"-"recognition"-
Since there is a high possibility that the relationship of "technology" is described and the relationship of other words is unlikely to be described, finally, as a recognition result, "speech"-"recognition"-"technology"
Will be selected. The concept network described above describes relationships between important words (keywords, for example, presentation organizations, words representing contents) included in information to be searched (for example, newspaper articles).

〔Example〕

Hereinafter, an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an embodiment of a voice interactive information retrieval apparatus according to the present invention. Voice 1 input interactively
Is obtained after the voice feature pattern is obtained by the voice recognition unit 2.
It is recognized by comparison and collation with the standard voice read from the standard voice pattern memory 6. As for the recognition result, a plurality of candidates are output in the most likely order for each input voice. The word series selection unit 3 reads from the conceptual network table 8 which describes the relationship between important words (keywords, for example, a publication organization, date and time, a word group representing the contents, etc. in the case of a newspaper article) relating to the information to be searched. Using the information, only a series of word contents related to each other is selected from a plurality of recognition candidates obtained each time a voice is input by the voice recognition unit 2. That is, narrowing down of recognition word candidates is performed using the concept network. Next, in the search processing unit 4, the selected word series (search word series)
, A search condition sentence necessary for execution of the search is generated, and a test (concept matching) using knowledge describing a relationship between keywords regarding information to be retrieved read from the concept network table 8 is executed. Information (such as an article) of the search result is selected. A specific example of the above search processing 4 is Hitachi Review, vol. 69, No. 3 (March 1987).
In more detail. On the display unit 5, information selected by the search is fetched from the information storage unit 9 and displayed.

The input control unit 10 generates an appropriate input sequence based on the hierarchical structure of the knowledge necessary for the search read from the concept network table 8 and based on the generated input sequence,
Is performed. Further, it has a function of generating a voice pattern corresponding to a concept (word) described in the concept network table 8 by the standard voice word group generation unit 7 and storing the voice pattern in the standard voice pattern memory 6. As a result, when the information to be searched is changed, a standard voice required for the search is arbitrarily generated.

All of the above processing is executed while being controlled by the control unit 12.

FIG. 2 shows an embodiment of the speech recognition section 2 in detail. The input voice passes through a voice input unit 21 and a voice analysis unit 22
, A recognition pattern is executed by the word recognition unit 23, and a word candidate string is obtained. When the input speech is an interactive sentence or word string, a keyword portion (main word portion) is searched for in the keyword location search unit 24 using the prosodic information of the input speech, and the keyword is searched for in the word recognition unit. The word content will be recognized.

FIG. 3 shows an embodiment of the voice input unit 21 and the voice analysis unit 22 in detail. The analog input voice _xn is converted to a digital value by an LPF (low-pass filter) 211 and an ADC (analog-to-digital converter) 212 while removing aliasing noise in sampling. Next, the feature pattern extraction unit 221 calculates feature parameters of the speech at predetermined time intervals (frames), and extracts feature patterns of the input speech. As the voice characteristic parameters, various parameters of a BPF (band-pass filter) output value, an LPC analysis (linear prediction analysis) result, and the like are used. The prosody information extraction unit 222 extracts feature parameters (for example, power and pitch cycle) representing prosody information such as stress or intonation of the voice.

FIG. 4 shows an embodiment of the characteristic pattern extracting unit 221 in detail. In the examples, BPF analysis is described. The registered voice _xn converted to a digital value is input to K BPF groups 2211 having different center frequencies and band values. The BPF is a second-order Putterworth-type filter, and includes two adders, four multipliers, and two delay units. The waveform of the BPF output is rectified by the absolute value (ABS) 2212, and the registered voice pattern X _i (i; frame) is obtained while the high frequency component is cut by the LPF 2213. LPF is a Putterworth type with the same processing scale as BPF.

In the present invention, the configuration of the feature pattern extraction unit 221 is a BPF analysis, but may be an LPC analysis. A detailed example in this case is described in the document "Speech Analysis and Synthesis by Linea
r Prediction of the Speech Wave) "by BSAtal et
al, Journal of Acoustic Society of America, Vol.50,
p, p, 637-655 (1971).

FIG. 5 shows an embodiment of the prosody information extracting unit 222 in detail. Power and pitch cycle are given as parameters representing prosody information.

The power calculator 2221 calculates the power P (short-time energy) of the input voice _xn from the time point M based on the following equation.

Here, N indicates the number of sample points in one frame. Fifth
In the illustrated embodiment, a multiplier 22211 receives an input speech _xn as an input.
_Xn ² = _xn × _xn is calculated by the adder 22212.
Intermediate value of power up to the point (Where M = 0 and the simplified ratio in equation (1)) and x _n ² are added to obtain a new power intermediate value P _n at the time point _n .
Thereafter, the same processing is repeated via the delay buffer 22213 to obtain the final power P (corresponding to n = N).

The pitch cycle calculation unit 2222 calculates the pitch cycle of the input speech waveform from the autocorrelation function of the center-clipped waveform. The pitch cycle (the reciprocal is called pitch frequency or fundamental frequency) is an important parameter that governs the pitch of sound, and is basically based on the physical characteristics of the mouth shape of the speaker (technically, the length of the vocal tract). Is determined, and some value fluctuation is caused by emphasis or intonation. There are many methods for calculating the pitch period, but no complete method has been found yet. As an embodiment of the present invention, a method of deriving from a waveform auto-correlation function is used. This technique is described in the document “Digital Processing of Audio Signals”.
Speech Signals) "by LRRabiner et al, PRENTICE-HA
LL, pp. 150-157. Hereinafter, the method will be briefly described. Center-clipped waveform
y _n is obtained from the following equation.

y _n = C [x _n ] (2) where C [X] is a center clipping function.
The pitch period is the i-th autocorrelation function R (i) of the center-clipped waveform. Is obtained by comparing the magnitudes of That is, if the pitch period and _{t p, R (i) R} (O) i = t p, 2t p, 3t p ... (4) R (i) O i is related other than the above, the pitch period t _p is obtained from the value of Ri. In the embodiment of FIG. 5, the clipping function memory
Multiplication of the input speech waveform x _n Kuritsupingu functions and C [x] read from 22222 is executed at the multiplier 22221, a waveform y _n which is the center chestnut tree ping sought. next,
Using the i-th order delay buffer 22223 connexion, product of y _n and y _{n_1} is obtained at the multiplier 22224. In the adder 22225, n-1
Intermediate value of i-th autocorrelation function up to time And y _n y _{n_1} are added, and a new intermediate value R _n at time n is added.
(I) is required. Hereinafter, the final value R (i) obtained by repeating the same processing is obtained via the delay buffer 22226. Then enter the values of R (i), the magnitude of the value in the comparator 22227 are compared, pitch period t _p is determined from the relationship of formula (4).

FIG. 6 shows an embodiment of the word recognition section 23 in detail. As input an input speech pattern X _i, the phoneme recognition section 23
In step 1, the phoneme constituting the word is recognized using the feature pattern of the standard phoneme read from the phoneme standard pattern memory 61. In the word matching unit 232, the word dictionary memory 62
The comparison and collation between the phoneme symbol string of the vocabulary read from the word and the phoneme sequence of the word obtained by the phoneme recognition unit 231 are executed on the symbol sequence. The judgment unit 233 uses the result of symbol matching to
The recognition result of the word content is output.

FIG. 7 shows an embodiment of the phoneme recognition unit 231 in detail. The distance-frame distance d _ij between calculator in 2311 the input speech pattern X _i phoneme standard pattern Y _j is calculated, the matching section 2
At 312, the input voice is compared with the phoneme standard. The collating unit generally performs a DP (Dynamic Programming) matching process. Then the candidate determination unit 2313, the matching value for the reference pattern ^m ▲ D m _{i Jm} ▼ (where J _m is the standard pattern m
, A standard pattern m that gives the minimum value is obtained, and a phoneme symbol string IPHCD (k) (where k is a phoneme string number) is output.

FIG. 8 shows an embodiment of the distance calculator 2311 in detail. In this embodiment, a case where an absolute value distance is used is shown.
The feature of the two voices is that the absolute distance d _ij between the patterns X _i and Y _j is Is obtained as Here, i and j are frames, and K is the number of BPF channels. Therefore, in the embodiment, two feature patterns are used.
X _i and Y _j are input through the frame pattern registers 23111 and 23112, respectively. The subtractor 23113 calculates x _ki −y _kj , and the absolute value converter 23114 calculates | x _ki −y _kj | Adder 2
At 3115, the accumulation from k = 1 to K is calculated. The result _dij is stored in the distance register 23116. Although the absolute value distance is used in the embodiment of the present invention, a correlation scale of a feature pattern obtained by the LPC analysis may be used. A specific embodiment in this case is described in the document "Minimum Prediction Residual Principle Applied
to Speech Recognition) "by F. Itakura et al. IEEE T
rans on Acoustics, Speech and Signal Processing, vo
l. ASSP-23, p, p, 57-72 (Feb. 1975).

FIG. 9 shows an embodiment of the collation unit 2312 in detail. The principle is an improvement of JP-A-55-2205 "continuous DP method". Based on the inter-frame distance d _ij between the i-frame of the input voice and the j-frame of the standard pattern, the cumulative distance D _ij is calculated using the following recurrence formula.

From the above recurrence formula, for each frame i of the input voice,
Optimal matching value for the reference pattern ^m ▲ D m _{i Jm} ▼ is determined (J _m is the frame length of the reference pattern m).

A specific example of the matching unit 2312 is that the inter-frame distance _dij between the input voice and the standard pattern is set in the frame distance register 23121.
And using the delay memory 23122 and the intermediate cumulative distance storage memory 23127, the di _{-1, j-1} and D
_{i-1, j-2} , _{Di-1, j-1} , _{Di-2, j-1} are stored. Based on each distance value, the adder 23123 passes the path Di _{-1, j-2i} + d
_{i-1, j-1} and the adder 23124 passes the path D _{i-2, j-1} + d _{i-1, j-1}
Is calculated, and the minimum value is searched for by the comparator 23125 together with Di _{-1, j-1 of} the path. Further, 2d _ij is added to the minimum value by the adder 23126, and the intermediate cumulative distance D _ij is newly obtained.
This result is stored in the intermediate cumulative distance storage memory 23127, and D
_This is information for calculating _{i + 1 and j + 1} . In the matching unit, the i
Optimal matching value between the standard pattern m for each frame ▲ D _{m i Jm} ▼
Outputs (i frames within the vowel interval information i _sk ~i _ek), as an input candidate determination unit 2313. From the verification value ▲ D ^m _{i Jm} ▼ magnitude relation of the determination unit, a determination whether the input speech is most similar to any standard speech made. The judgment unit is composed of a simple magnitude comparator.

FIG. 10 shows an embodiment of the word matching unit 232 in detail. Phoneme symbol string IPHCD of phoneme recognition result of word
(K) and the phoneme symbol strings of the standard words read from the word dictionary and input via the phoneme symbol string registers 2321 and 2322, respectively. Next, in the comparator 232, the comparison between the respective phoneme symbol strings is performed for each code, and the difference (total distance) in all the series is obtained in the adder 2324. The comparison of the total distance of all standard words is performed via the distance register 2325 by the comparator 23.
Executed in 26, the recognition result (the standard word having the minimum total distance) is output.

 The judgment unit 233 is composed of a simple magnitude comparator.

FIG. 11 shows an embodiment of the keyword location searching section 24 in detail. Before describing the embodiment, the principle of keyword search will be described. The information that the speaker intends to convey is generally loosely uttered or its content (mainly words)
It is known to utter emphasis. in this way,
The prosody information is rational and natural information for the utterance content, and is important information for dividing a spoken word into a meaningful unit. The above detailed description is in Japanese Patent Application No. 61-755
No. 28, "Speech Conversation Sentence Structure Estimation Method". The keyword search unit 24 according to the present invention uses an embodiment of the above patent. Specifically, the input voice dividing unit 24
In step 1, the input voice is divided into semantic units based on the characteristics of prosodic information (pitch cycle and power) such as emphasis or inflection of the voice. After estimating the wording, the sentence structure estimating unit 243 estimates the sentence structure of the input speech using the information read from the sentence structure dictionary 244. Next, using the information on the sentence structure, a keyword (keyword) that is semantically significant is obtained by the keyword extraction unit 245, and a keyword search is executed.

Before showing an embodiment of the word sequence selection unit 3 which is the main point of the present invention, the principle of the present invention will be described in detail. FIG. 12 shows an operation example of the word sequence selection unit 3. For example, when three words “speech”, “recognition”, and “technique” are input, a word candidate string T _ij (i is a number indicating the input order of the input voice, j is each input) As shown in the figure, the first and second candidates are "hot spring", "voice", and "recognition" for input "recognition". ",""Minshuku", and "input""technology" are "fact", "...", "technology".
At this time, the word sequence selection unit, which is the main point of the present invention, narrows down candidate words using information on the semantic relationship between words described on the conceptual network. In the example shown in the figure, "speech"-"recognition"-
Since the relation of "technology" is described, the word series that may be the final system is "voice"-"recognition"-"technology"
Is obtained. Here, the relationship between "hot spring" and "minshuku" may possibly be described on the concept network, but the recognition candidates obtained from the third input result "fact""...""technology" Cannot be related to the final word sequence.

FIG. 13 simply shows the structure of the concept network used in the word sequence selection unit. The concept network expresses semantic connections between concepts (words) in a hierarchical subsumption relationship (eg, "company"-"Hitachi"-("Hitachi Chuken") This is a knowledge expression system expressed by a network of two relationships with the expressed concrete relationship (for example, "Hitachi Chuken"-(developed)-"S820") (For details, see, for example, "Hitachi Review Vol. 69, No. .3 (1987-
3) "). The conceptual network in the example of FIG. 12 is also shown.

FIG. 14 shows in detail one embodiment of the word sequence selection unit 3 which is the main point of the present invention. For each of the plurality of input word sounds, a candidate word string having a plurality of recognition candidates is input, and each word candidate is assigned a word number, and sent to a comparison unit 32 via a word number buffer 31. Using a concept network 78 that represents knowledge about the relationship between concepts and the hierarchical subsumption of recognizable vocabulary (concepts),
Only relevant recognition candidates are selected. The comparison of the association at this time is not a plurality of recognition candidates obtained for one input voice, but a relationship between recognition candidates obtained for each of a plurality of input voices. Therefore, the comparison section
The process at 32 is performed by inputting the result selected for the already input speech from the word series intermediate memory 33, and the process is repeated until all the input speech is completed. The plurality of word sequences obtained in the final system are output via the word sequence buffer 34. The output word sequence is used as a search word sequence.

As described above, the main point of the present invention is to select a likely result from a plurality of recognition candidates obtained for each input word voice by utilizing the knowledge about the relationship between the concepts expressed in the concept network. .

FIG. 15 shows an embodiment of the search processing section 4 in detail. The search condition sentence generation unit 41 receives a word sequence for search and generates a search condition sentence required for information search by using a relationship between the concepts expressed in the concept network. The search condition sentence generated here is an abstract concept created from ambiguous input (for example, when the input is “article”, “society”, “computer”, “development”, the search condition sentence is “ Article about a computer developed by a company "). Next, the concept matching unit 42 tests the association with a more specific concept (lower concept of the inclusive relation) from the search condition sentence while referring to the knowledge expressed in the concept network. As a result, it is possible to search for information such as articles expressed by a specific concept that satisfies the search condition sentence. The above processing is called concept matching. The search result is output via the search number buffer 43.

The display unit 5 has a function of reading the contents of the searched information from the information storage unit 9 based on the search number and displaying the contents on a display device or the like. Details of the generation of the search condition sentence are described in “Hitachi Review, vol. 69, No. 3 (1987-3)”.

FIG. 16 shows an embodiment of the standard speech word dictionary generator 7 in detail. A standard speech pattern (specifically, a word dictionary represented by tone code etc.) used for recognition of input speech from concepts (words) described in a concept network table obtained corresponding to information to be searched. Generate
The concept of the concept network is represented by word syllable codes. (For example, "Hitachi" is Hi ta chi /). Deformation comparison part 7
In 1, the input word syllable codes are compared based on a deformation rule 73 that describes how the voice fluctuates and deforms due to articulation of the voice, and the word dictionary generation unit 72 generates a word dictionary. . For example, in the case of "Hitachi", the i of / chi / is often silenced, resulting in (here Produces an indication that / i / may be silenced). The result is stored in the word dictionary of the standard voice pattern 6.

FIG. 17 is a block diagram showing another embodiment of the voice interactive information retrieval apparatus according to the present invention. This embodiment uses knowledge about the relationship between the input and recognized word content and the words expressed in the concept network to narrow down the category of standard speech used to recognize the next input speech, Restrict. More specifically, the vocabulary selection unit 13 receives the word string recognized by the voice recognition unit 2 as an input, and converts the word content that may be input next from the combination of words input up to that point into the concept network. It narrows down using the knowledge about the relationship between words read from the table 8. The narrowed-down result is input to the standard voice pattern input control unit 14, and limits the category of the standard voice used when recognizing the next input voice. Voice recognition unit
2, a search processing unit 4, a display unit 5, a standard voice pattern memory 6, a conceptual network table 8, an information storage unit 9, and a control unit 12 are the same as those in the voice interactive information search device shown in FIG. It is realized by the configuration.

FIG. 18 shows an embodiment of the vocabulary selection unit 13 in detail. The word recognition result obtained by the voice recognition unit 2 is stored in the word sequence intermediate memory 132 via the word number buffer 131 in a form added to the word sequence recognized up to that point. Next, the vocabulary narrowing unit 133 uses the combination of the word contents read from the word sequence intermediate memory 132 and the knowledge about the relationship between the words read from the concept network table 8 to determine the next word content vocabulary to be input. Perform a refinement. Result is vocabulary category buffer 134
Is input to the standard voice pattern input control unit 14 via the.

The standard voice pattern input control unit 14 performs a process of taking in standard voices belonging only to the category from the standard voice pattern memory 6 based on the narrowed vocabulary category.

〔The invention's effect〕

According to the present invention, the most probable candidate can be selected from a plurality of output recognition candidates, so that the current speech recognition performance can be used sufficiently, and interactive information retrieval using speech even from ambiguous storage. There is an effect that becomes possible.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice interactive information retrieval apparatus showing one embodiment of the present invention, and FIGS. 2 to 16 are block diagrams showing in detail one embodiment of each processing unit and the operation principle of FIG. Figure,
FIG. 17 is a block diagram showing an embodiment of a voice interactive information retrieval apparatus according to one embodiment of the present invention, and FIG. 18 is a block diagram showing a part of FIG. 17 in detail. 3... Word series selection unit, 4... Search processing unit, 7... Standard speech word dictionary generation unit, 8... Concept relation table memory,
10 ... Input control unit, 13 ... Vocabulary selection unit.

Continuation of the front page (56) References JP-A-62-226374 (JP, A) JP-A-59-53985 (JP, A) Fujisawa, Hatakeyama, Fujina "Intelligent filing system using concept network". Hitachi Review Vol. 69, No3, 1987, p231-238

Claims (6)

(57) [Claims] 1. A voice input means, a voice analysis means for extracting characteristics of an input voice, and characteristics of the input voice obtained from the voice analysis means are preliminarily analyzed and stored as a standard voice pattern. Using a recognition network that compares and collates with the features of the standard speech to mainly perform word recognition, and a concept network that is expressed in advance by a relationship between keywords related to information to be searched, the recognition processing unit obtains a plurality of words. A word sequence selection means for selecting one or a plurality of word sequences to be used for information retrieval from the word candidate sequence of the input speech to be obtained, and the information retrieval word sequence obtained by the word sequence selection device and the concept network. Means for generating a search condition sentence necessary for a search process using a search condition, and a concept matcher for selecting a content to be searched by comparing the search condition sentence with the concept relation network Means and, interactive information retrieval apparatus by voice, characterized in that the search results obtained in 該概 precaution matching means and a means for displaying on a display. 2. The interactive information retrieval apparatus according to claim 1, wherein said input voice is a voice naturally uttered interactively and naturally, and said input voice obtained from said voice analysis means. A voice-based interactive information retrieval apparatus characterized in that the input speech is divided into units of words or phrases using the prosodic information of the above, and keyword location searching means for searching for an important word part is provided. 3. A voice interactive information retrieval apparatus according to claim 1, further comprising input control means for controlling the content of voice to be input. Interactive information retrieval device. 4. A speech interactive information retrieval apparatus according to claim 1, further comprising means for outputting said retrieval result by speech. Interactive information retrieval device. 5. A speech-based interactive information retrieval apparatus according to claim 1, wherein an acoustic feature pattern of the word is obtained from a word name representing the concept of the concept network. Characterized in that a means for arbitrarily generating the information as the standard voice pattern is provided. 6. A means for inputting voice, a voice analyzing means for extracting characteristics of the input voice, and a standard which is preliminarily analyzed with the characteristics of the input voice obtained from the voice analyzing means and stored as a standard voice pattern. Vocabulary selection that narrows down the category of the standard speech by using a recognition processing unit that compares and collates with the features of speech to mainly perform word recognition, and a concept network that is expressed in advance by a relationship between keywords related to information to be searched. Means for generating a search condition sentence necessary for search processing using the word sequence of the word recognition result of the input speech obtained in the recognition processing means and the concept network, and the search condition sentence and the concept network And a concept matching means for selecting a content to be searched by collating with the information.