FR2581469A1 - Vocal entry/exit device and speech recognition or synthesis installation making use of it - Google Patents

Abstract

The device includes a signal processor 10 equipped with an internal working memory and an address bus for external memories. The external memories are distributed into data memories and program memories 26, 28, organised into addressable pages of such a size that the address bus can simultaneously address in a program page and in a data page which will be used simultaneously.

Description

Voice input-output device and installation for speech recognition or synthesis with application
The subject of the invention is a voice input / output device using a signal processor and applicable whenever it is necessary to recognize words spoken by a speaker (or by several speakers), possibly after a learning phase, and / or synthesize speech using relatively sophisticated processing algorithms.

such as linear prediction coding algorithms and the algorithm of Sakoe and S. Chiba "IEEE TRANSACTIONS on Acoustic speech and signal processing", Vol. ASPP 26,
No. 1, February 1978. pp. 43-49. Many other algorithms, which can be used in the case of multi-parties also exist.

 We have already implemented high-performance algorithms of this type on powerful computers, which also provide computing and interface functions, but at the cost of a high investment and limited flexibility of adaptation.

 Simple speech recognition systems have also been proposed, making it possible to search, among words belonging to a previously memorized vocabulary, which word is close to a word to be identified. These systems often use a signal processing processor (or a battery of filters), an analog-to-digital converter and a microprocessor which performs the calculations for determining the word to be retained. These systems cannot ensure speech synthesis at the same time. They also do not allow adaptation of speech references to a new speaker without obliging him to a complete process of learning the system.

 There are also known speech synthesis systems comprising a source, a voice channel modeling filter, a digital / analog converter and a processor which controls the rest of the installation and sends the coefficients representing the coded speech to the synthesizer: such a system has only limited possibilities, in particular in that it does not allow recognition.

 The invention aims to provide a voice input-output device usable in a speech recognition and / or synthesis facility, which meets the requirements of practice better than those previously known, in particular in that it is simple and presents however a great flexibility of adaptation to particular needs. The invention more particularly aims at providing a device using a signal processor making it possible, better than a conventional synthesizer, to carry out speech synthesis by multi-pulse coding of the excitation source of a synthesis filter, while eliminating the problems associated with the limited addressing capacity of such a processor.

 To this end, the invention proposes in particular a voice input-output device for speech recognition and / or synthesis installation, comprising a signal processor provided with an internal working RAM and a bus. addressing of external program memories, characterized in that the external memories are divided into data memories and program memories, organized in addressable volume pages such that the address bus can address simultaneously in a program page and in a data page that will be used simultaneously.

 The invention also proposes a device of the type defined above, characterized in that the external program memories are assigned some to recognition and some to synthesis and organized in pages, and in that are placed at the same address in two program pages with different assignments: the transition instructions in summary and the return instructions in recognition.

When, for example, the signal processor is of a type allowing direct addressing only of four K words of external memory, these four will be divided
K words in two parts and we will perform paginations.

The first half of the address area, from O to '7FF will be reserved for the program. The rest, from 800 to FFF, will be used to store the coded speech references used during recognition and during synthesis,
Thanks to this pagination of program memories, it will be possible to have parallel recognition software and synthesis software. We can thus switch from one function to another without loading.

The same speech references can be used for recognition and synthesis, which is particularly interesting when you want to create a learning installation that synthesizes the recognized word, allowing the speaker to recognize his vocal characteristics during synthesis operation.

 On the other hand, the paging of the data memories makes it possible to have a significant capacity for storing the coded speech references.

 The architecture of the device lends itself well to a simple and very flexible constitution of a speech recognition or synthesis installation. The components of the device can indeed be distributed on three cards, replaceable independently according to the needs to be fulfilled.

 In particular, it is possible to constitute the device of a motherboard, of an extension card and of an analoginumerial and digital / analog converter card. The motherboard carries the signal processor, the program memories (read only memories or programmable read only memories) and a fraction of the data memories (random access memories Of the programmable read only memories1.

 The expansion card is placed on the first. It will carry additional data memories. These memories allow the installation to be adapted to various uses. We could, for example, switch from a single-speaker recognition installation with learning, using only random-access data memories, to a device completely independent of the speaker, with fixed vocabulary, comprising only programmable read-only memories (without adaptation to the speaker) or programmable read only memories and random access memories, (with adaptation to the speaker).

The pagination allows the surplus to have
direct access to each page of all the memories by simple sending of a word of one or two bytes via an access or "port" of the signal processing processor on a specific locking circuit (flip-flop network),
- a possible hierarchy of speech references for vocabulary recognition, for example twelve word references for each word, which makes it possible to adopt a recognition strategy of the following type: pre-processing of the word, based on vector quantification , eliminates word classes. The unknown word is then compared only with part of the stored references, that is, with part of the dictionary.

 The third card of the installation will include the analog / digital and digital / analog converter, as well as the filtering means required to form an interface with the input device (microphone in general) and the output device (speaker in general), The filtering means will ensure anti-aliasing filtering and smoothing. This card can be of relatively conventional constitution: it will frequently perform sampling at a frequency between 8 and 10 kHz, the filtering limiting the bandwidth to 3.4 kHz and generally removing low frequencies from the spectrum. The cutoff frequencies of the filtering means can be manually programmable.

 The pagination of the program memories has an additional advantage when the installation uses an external clock for analog-digital and digital-analog converters and when it is desired to work in interruption on the signal processor. We know that interrupted work has the disadvantage that it is necessary to save the results in progress and then restore them. In the case of a device of the type concerned by the invention, the interruption in principle presents a serious drawback. During an interruption, the processor comes to execute an instruction always at the same address; it is therefore necessary to determine later whether the interruption program which is to take place is a program relating to recognition or synthesis.

 Thanks to pagination, you can store two separate programs in the same addresses in two different pages. Thus, at each interruption, the signal processor executes the instruction which is at a given address, fixed once and for all in the specifications of the processor, but in 1 or the other of two different pages. There is therefore no loss of computation time in determining whether it is the interruption to the synthesis program or the interruption to the recognition program which must be executed, that is to say whether a data must be entered. or, on the contrary, issue it.

 It is advantageous to provide, on the motherboard which already carries the processor, the program memories and the data memories used in all cases or in the majority of cases, a random access memory with sequential access with which the processor communicates directly. by an access or input-output port further requiring a port for initializing its address counter. Thanks to this sequential access random access memory addressed by a counter which increments by itself, we can, in speech recognition. memorize the speech signal or the coefficients given by the analysis of the word to be recognized which must be stored in random access memory and reduce the access time to the memorized data, due to self-incrementing.

The invention will be better understood on reading the following description of a particular embodiment, given by way of non-limiting example. The description refers to the accompanying drawings, in which
FIG. 1 is a block diagram of an installation according to the invention,
FIG. 2 is a block diagram showing the components carried by the motherboard of the device incorporated in the installation of FIG. 2,
- Figure 3 is a block diagram showing the components of the device carried by the expansion card,
FIG. 4 is a principle flow diagram intended to show the operation of the device,
- Figure 4A is a flow diagram of a subprogram.

 - Figure 5 is a diagram showing a possible embodiment of a voice input member with microphone, intended to be associated with the analog / digital and analog to digital conversion card.

 The voice input-output device which will now be described by way of example can be used in connection with any system accepting a link of the type provided for by the CCITT V 24 recommendation (RS 232 link for example). The device can in particular constitute an input-output member of a computer or be associated with a teletext terminal. It can also be used to carry out voice commands, for example controlling a robot, interrogating the databases of data using a teletext terminal. Conversely, it can be incorporated into a voice dialer.

 The central element of the device in FIG. 1 is constituted by a signal processing microprocessor 10. We will assume that this microprocessor is of a type which only allows direct addressing of four K words of sixteen bits of external memories and that it has eight input-output ports, identified by numbers from 0 to 7 on the Figure 1. The command input-outputs are managed by a microcomputer 12 connected to the access 0 of the processing microcomputer via a network of flip-flops 14 forming a lock. The microcomputer 12 can in particular be of the one-piece type. single box, such as a 68 701 circuit, and constitute an asynchronous transmission unit (UART). The signal processing microprocessor 10 can, for its part, be a TMS 320 circuit. The access 1 of the processing microprocessor 10 is connected to a converter card 16 which will not be described in detail since it can be of conventional constitution, provided with a voice input device such as a microphone 18 and a voice output device such as a speaker 20.

 Access 2 of the processing microprocessor 10 is connected to a sequential read random access memory 22 allowing storage of speech #, the role of which is especially important when the data memories include programmable read-only memories intended to adapt to the speaker. The sequential reading of this random access memory 22, comprising for example two boxes of 2 K words of eight bits, is assumed by self-incrementing counters 23, as will be seen below. Access 3 of the microprocessor makes it possible to initialize these counters.

 As indicated above, the microprocessor 10 makes it possible to directly address only four K words of external memories. The TMS circuit 320, which will often be used to constitute the microprocessor 10, is generally used by using its 4K words to store the program to be executed, the data being stored on an external memory addressed using counters. In the present case, on the contrary, half of the address area is used, comprising 212 addresses, half to store the programs, half to store the coded speech references used during recognition and during synthesis.

In FIG. 1, a memory control circuit 24 has been indicated, allowing the selection of the memory which works as a resident, therefore which is directly accessible. The choice of the page will be made by a word of sixteen bits, used to select that of the memories which is connected to the bus of the microprocessor 10. Thanks to this arrangement, it will be possible to access in a single instruction, each memory position, it that is to say within a time frame that is only three cycles. It will thus be possible in particular to choose either a page of a memory 26 storing a recognition program, or a page of memory 28 storing a synthesis program.

 The data memories may include memories 30 carried by a motherboard on which the microprocessors 10 and 12 are mounted (for example programmable read only memories in the case of a multi-speaker synthesis and recognition device: or memory # s read only memory and read-only memories in the case of a recognition device with adaptation to the speaker or read-only memories in the case of a recognition device. Additional memories 32 can be carried by an extension card, accessed by accesses 4, 5, 6 for example.

 This arrangement makes it possible to use any desired composition of data memories and of program memories, according to the functions to be fulfilled. It should also be noted that the device will operate in all cases with only having to access 4 K words of memories residents, directly connected to the data bus of microprocessor 10, which ensures high processing speed.

 One might think a priori that the association of programmable read-only memories of the kind available at the present time with a processing microprocessor of TMS 320 type is impossible, because the guaranteed access time of the programmable random access memories available is of the order 150 ns, while the TMS 320 is designed to operate with memories having an access time of 70 ns. But in practice, by checking and sorting the programmable read-only memories available, it can be seen that such memories can be available.

 Figure 2 shows a possible constitution of the components carried by the motherboard of a device of the kind shown in Figure 1. In Figure 2, only the elements concerned by the invention have been shown, in a configuration usable for recognition with adaptation to the speaker.

 We find, in Figure 2, the signal processing microprocessor 10, comprising a data bus 36 with sixteen bits DO-D15 and an address bus 38 with twelve bits AO-A11, making it possible to address up to 4K words. The data bus is connected to the line interface microcomputer 12 by the flip-flop network 14, consisting for example of two 74 LS 373 circuits.

 The first eight bits DO-D7 (most significant bits) of the data bus are applied to a network of flip-flops 40 intended to ensure the selection of the memory pages which will be processed in resident memories.

The output of the flip-flop network 40 consists of an eight-bit selection bus 42 S1-S8, the bits Si to S7 ensuring the selection of the data memories, the bit S8 the selection of the program memory. The bits Si and S2 are used for the selection of the data memories which are on the motherboard; bits S3 to ST are used for the selection of the memories which are on the extension card, as will be seen further on.

In the embodiment shown, the bits
D8 to D15 (least significant bits) of the data bus 36 are applied to self-incrementation counters 44 for addressing the sequential access data random access memory 22. This memory 22 can in particular consist of two boxes 46 and 48 of random access memories, each having a capacity of 2 K bytes, used together to have words of sixteen bits.

The counters 44 must be initialized to the address value from which we want to access the memories 22, then are automatically incremented by one at each input operation (read validation / DEN) or output (validation writing / WE),
These functions are fulfilled by a circuit comprising two boxes 50 and 52 (LS 138 for example) respectively assigned to the decoding of the write and read accesses, controlled by the outputs / WE and / DEN of the microprocessor 10 and by the bits of address. In the case illustrated in FIG. 2, the bit A11 is used to address the expansion memory card and the network of control doors 54, 55 is provided accordingly.

 So that the incrementation takes place both at the input and at the output, the incrementation of the counters 44 is carried out from the boxes 50 and 52 by means of an OR gate 56. A network of flip-flops (LS 74 by example) with several offset outputs 58 makes it possible to obtain the delays required between reading from the memory ts 22 and incrementing the counters 44.

 The circuit also includes initialization means. To simplify these means, it is planned, in the embodiment of Figure 2. to carry out the control using only eight bits, which allows to have only one rocker box, without as much to limit the possibilities of use. The purpose of the sequential access memory is only to temporarily store a signal which will subsequently be compared to the reference words stored in the other memories: it is not necessary to be able to start from any of the addresses at start of counting.

 The memories shown in FIG. 2 also include data memories constituted by programmable read only memory (EPROM) boxes organized in page between which the selection is made by the bits Si. S2. SB. The page 1 is constituted for example by two boxes 60 of each 2 K bytes (box 2732 for example). A NAND AND 62 door allows selection from Si and / A11. The constitution of page 2 can be similar, the selection being made by the NAND gate 64.

 The program memory used comprises, in FIG. 2, two pages of 2 K words of sixteen bits, addressable at the same time as a page of data memory of 2 K words by the microprocessor 10. It is for example made up of two boxes 66, the selection being made by bit S8.

 The extension card 32 shown by way of example in FIG. 3 comprises six random access memory boxes 68 of each 4 K bytes, grouped in pairs to constitute pages of 2 K words of sixteen bits.

 The selection of each data page on the expansion card is ensured by a NAND gate 70.

72 or 74 receiving, on one input, the lAil bit, on the other input the selection bit S3, S4 or S5.

Addressing in a page is carried out using bits AD-A10. The read and write commands are taken from the outputs / MEN and #WE of the microprocessor 10.

 Depending on the application envisaged for the device, various constitutions can be given to the memories.

 In the case for example where the device is intended to carry out speech synthesis and multi-speaker recognition, with fixed vocabulary, all the memories, on the motherboard as on the expansion card, may be memories programmable dead. In this case, the voice is frozen and no attempt is made to reproduce the voice of a new speaker.

 The vocabulary is also frozen and inextensible.

The random access memory of the processing microprocessor is in this case necessary to store the characteristics of each incoming word to be recognized,
In the case. recognition with adaptation to the speaker, it will be possible to use a mother card comprising only programmable read-only memories and place non-volatile memories on the expansion card,
It is the importance of these random access memories which fixes the possibilities of adaptation of the system. Adaptation can be done in several ways
- by adapting the voice, that is to say the incoming signal, to make it correspond to the speech.

memorized,
- or by adapting the spectral characteristics of the references already memorized to create new references better suited to the speaker.

 The recognition process is only done on the new vocabulary references.

 The two examples above are obviously given only without limitation and many other applications can be realized, such as for example the synthesis and the recognition of a single speaker, by constituting the set of memories of data in RAM.

 The implementation of the device can be provided according to the flow diagram of Figures 4 and 4A, the latter giving the speaker recognition sub-program.

 On power-up, the device executes an initialization program which intervenes on all the program memories and automatically ensures the selection of the program memory n 0 (if there are several program memories). The initialization of the variables common to all the programs is carried out, then a test of availability of the microcomputer 12. In the event of availability, the microprocessor 10 gives control to the microcomputer 12 and the latter checks the availability of the system to be controlled and puts on standby eventually. Once the system to be ordered available, the hand is returned to the microprocessor 10. The latter selects the memory containing the recognition program and the input operation is carried out word by word with a storage in the random access memory 22. recognition is carried out and causes the selection of the memory containing the synthesis program, then of the data memory containing the coefficients in linear coding of the word to be synthesized and the timbre of this word. The synthesis is carried out and the recognized word is sent to the microcalcula tfsur i 2. Finally, If the recognized word corresponds to the order of end of use of the system, the device stops.

 The converter card 16 can have a conventional constitution. It is controlled by access 1.

consisting of the exit from a network of doors 76 (Figure 2). The input and output members 18 and 20 can belong to the same assembly and it is advantageous, from the point of view of cost, to use a telephone handset type T 83. The two components of a such a handset The fact that the microphone capsule of such a handset has a frequency response which increases up to around 3 kHz (which is a defect in the case of telephone sets, due to the risk of Larsen effect), represents a advantage in the case of a device according to the invention which can be used for installations not linked to the telephone, since the pre-emphasis is obtained directly which one is generally obliged to do at the input of the recognition systems,
Microphones intended for telephone handsets, however, have the disadvantage of lacking directivity, which makes them sensitive to ambient noise and degrades the recognition rate when the microphone is far from the speaker's mouth.

 Figure 5 shows an arrangement to overcome this drawback. The microphone 18 is placed close to the focal point of a parabolic reflector 80, its capsule facing the speaker 82. Because the capsule is facing the speaker. the components are kept in the low frequencies of the speech signal which arrives directly at the microphone. Provided that the reflector is given an appropriate diameter, a pre-accentuation is obtained which compensates for the fall in the frequency response of the capsule beyond 3 kHz. In addition, and above all, the reflector focuses on the capsule the signals coming from a given direction and therefore provides the voice input member with the required directivity at the same time as it screens the noise likely to come from the back.

 The arrangement of Figure 5 is particularly interesting in the case of a voice input of a device in an environment affected by noise.

Claims (9)

 1. Speech input-output device for speech recognition and / or synthesis installation, comprising a signal processor provided with an internal working memory and an address bus for external program memories, characterized in what the external memories are divided into data memories and program memories, organized in addressable volume pages such that the address bus can address so simultaneously in a program page and in a data page which will be used simultaneously.  2. Speech input-output device for speech recognition and / or synthesis installation, comprising a signal processor provided with an internal working memory and an address bus for external program memories, characterized in that that the external program memories are assigned some to recognition and some to synthesis and organized in pages, and that the summary instructions and the instructions are placed at the same address in two program pages having different assignments back in reconnaissance.  3. Device according to claim 1 or 2, characterized in that the external memories consist of word boxes of eight or sixteen bits, associated in pairs, each page grouping words from each of the two associated boxes and each page being addressable by sending a single sixteen-bit word through an I / O port on the processor.  4. Device according to claim 1. 2 or 3, characterized in that each program memory box comprises the same instruction for passing from one page to another at the head or at the end of the page.  5. Device according to any one of the preceding claims, characterized in that the data memory boxes comprise programmable read-only memories for synthesis and multi-speaker recognition, and / or adaptive random access memories.  6. Device according to any one of the preceding claims, characterized in that it comprises a mother board carrying the processor, the program memories and the frequently used data memories and at least one memory expansion card.  7. Device according to claim 6, characterized in that it comprises at least one interface card (16) having analog / digital and digital / analog converters, the analog / digital input converter comprising an analog filter -replication, a sampler at a frequency between 8 and 10 kHz and means of cutting above about 3.4 kHz and elimination of low frequencies.  8. Device according to claim 7, characterized in that the analog / digital converter is associated with an input telephone microphone.  9. Device according to claim 8, characterized in that the microphone is placed at the focus of a paraboloid reflector.