JPH10124291A - Speech recognition communication system for mobile terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a speech recognition function as a user interface of a mobile terminal in communication environment using the mobile terminal at practical accuracy and cost. SOLUTION: In the mobile terminal 101, a speech signal inputted from an input part 109 in a call or an off-line state is transmitted from a control part 110 and a communicating part 111 to a PHS network 103, from which the speech signal is transmitted to a speech control host device 108 with a mobile terminal control host device 104 and an internet 105. The speech signal is received by a mobile terminal communication control part 116 with a packet transmitting/receiving part 115 in the speech control host device 108 and recognized by a sentense/voice recognizing part 117. Recognized speech/sentense data obtained as the result is returned in block to the mobile terminal 101 in real time or a desired timing, received by the control part 110 with the communication part 111 and displayed in an output part 112.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for recognizing voice such as a voice input by a mobile (portable) terminal device.

[0002]

2. Description of the Related Art There are various voice recognition technologies for recognizing voice signals, converting them into character data and storing the data, and providing recognition results to use of various services. It has been demanded in various industrial fields.

In recent years, with the development of speech recognition algorithms, speech recognition systems using a mainframe computer or a workstation computer have been developed.

[0004] These systems include various systems such as a bank balance inquiry system and a seat reservation system using telephone voice as input, and a delivery package sorting system that recognizes workers' voices and automatically delivers packages. It is being adopted by other industrial fields.

However, such a speech recognition system has
Only under the environment of a large-scale computer system as described above has reached a level having practical recognition accuracy, and under the environment of a small-sized computer system such as a so-called personal computer, the practical recognition accuracy has been reached. At present, an inexpensive speech recognition system having recognition accuracy has not been realized yet.

[0006] On the other hand, in parallel with the above information processing technology, mobile terminals such as car phones, mobile phones, and PHS (Personal Handy Phone System) have been rapidly spreading in recent years.

[0007] In particular, the PHS is small in size, and at the same time, the call charge is lower than that of a car phone or a mobile phone.
It has the feature of being able to talk with high quality "anytime, anywhere, with anyone" and is exploding. In addition, PHS is an ISDN (Integrated Services)
Since it is a public network using a Digital Network (Integrated Services Digital Network) as a backbone, high-speed digital communication at a transmission rate of 32 kilobits / second is possible, and expectations for its application to the multimedia communication field are increasing.

Further, in order to take advantage of the convenience of the mobile terminal,
It is also expected that it can be used as a multimedia information management / communication terminal device that can be used not only as a mobile phone device but also as a mobile information management device. Specifically, such a mobile terminal is expected to have not only a call function / fax function, but also a homepage access function and an e-mail communication function as an access function to the Internet and a company network. It is expected to have information management functions such as address management, schedule management, and database search.

[0009] Such a mobile terminal is required to have a natural user interface that is as gentle as possible to a person so that the user can use it easily. As user interfaces currently realized, finger operation input using a keyboard or a mouse, handwriting input using an electronic pen, and the like have been put to practical use, but if they can respond to voice input and the like, they are ideal as user interfaces. . That is, for example, if it becomes possible to process a voice signal indicating the contents of a call as data while using a call function as a basic function, it is possible to dramatically increase the convenience of the mobile terminal. Here, the value of applying a voice recognition function as a user interface to a mobile terminal can be found.

[0010] However, while the mobile terminal is small and has limited information processing capability, as described above, in the current speech recognition processing, the mobile terminal must be installed in a mainframe computer or workstation computer class environment. It is difficult to achieve practical recognition accuracy. Therefore, at present, there is a problem that it is very difficult to realize a voice recognition function as a user interface of a mobile terminal.

An object of the present invention is to realize a speech recognition function as a user interface at a practical accuracy and cost in a communication environment using a mobile terminal.

[0012]

The present invention first has a mobile terminal having the following configuration. In other words, the host connection means (the control unit 110 and the communication unit 111) is connected to a relay network (PHS network 10) composed of one or both of a wireless network and a wired network.
3 and the Internet 105) indirectly or directly to a voice control host device (voice control host device 108) as a host device without going through a relay network. This means includes, for example, a function of selectively transmitting a real-time reply request command or a non-real-time reply request command for causing the recognition voice data to return in real time or non-real time to the voice control host device, and And a function of transmitting a batch transfer request command for batch transfer.

The voice input means (input section 109) inputs voice. After the connection operation by the host connection unit, the audio data transmission unit (the control unit 110 and the communication unit 111) transmits the audio data input from the audio input unit to the audio control host device.

[0014] Recognition voice data receiving means (control unit 110,
The communication unit 111) receives the recognized voice data returned from the voice control host device. The recognition voice data display / edit means (control unit 110, output unit 112) displays or edits the received recognition voice data.

Next, the present invention has a voice control host device 108 having the following configuration. That is, mobile terminal connection means (packet transmitting / receiving section 115, mobile terminal communication control section 11
6) responds to the connection operation by the host connection means in the mobile terminal, and identifies and connects the mobile terminal.

Voice data receiving means (packet transmitting / receiving unit 1)
15. The mobile terminal communication control unit 116) receives voice data for each currently connected mobile terminal. Voice recognition means (mobile terminal communication control unit 116, sentence voice recognition unit 117)
Executes voice recognition processing on voice data received by the voice data receiving means for each mobile terminal currently connected.

Recognition voice data return means (mobile terminal communication control section 116, packet transmission / reception section 115) converts recognition voice data obtained by voice recognition processing by voice recognition means for each currently connected mobile terminal. Reply to the mobile terminal. This means is provided, for example, for each currently connected mobile terminal, when the mobile terminal connection means receives a real-time reply request command from the mobile terminal, the voice recognition processing by the voice recognition means correspondingly. The recognized voice data is immediately returned to the corresponding mobile terminal, and if the mobile terminal connection means has received a non-real-time reply request command from the mobile terminal, corresponding voice recognition processing by the voice recognition means After holding the obtained recognition voice data, when the mobile terminal connection means receives the batch transfer request command from the mobile terminal, the recognition voice data held correspondingly is collectively returned to the corresponding mobile terminal. I do.

With the above-described mobile terminal voice recognition communication system according to the present invention including the mobile terminal and the voice control host device, the mobile terminal does not need to be equipped with an advanced voice recognition environment and has a practically accurate voice recognition function. Can be provided at low cost.

The configuration of the invention described above can include the following limitations. That is, first, the mobile terminal has a personal handyphone system communication function (communication unit 111).

Next, the relay network includes a personal handy phone system communication network (PHS network 103) and the Internet (Internet 105). The voice control host device connects to the Internet.

Then, the host connection means in the mobile terminal includes:
A mobile terminal control host device (mobile terminal control host device 104) having a gateway function between a public network including the personal handyphone system communication network and the Internet via a personal handyphone system communication network.
The mobile terminal control host device connects to the voice control host device via the Internet using the communication protocol on the Internet by calling and connecting to the voice control host device.

With this limited configuration, it is possible to provide a practically accurate voice recognition function at a lower cost and through a personal handyphone system communication network and the Internet, which are now spreading nationwide and worldwide. At the same time, the functions provided by the present invention can be seamlessly combined with the personal handyphone system call function and the Internet access function.

Further, the above communication protocol may include the following limitations. That is, the communication protocol is
It is a hierarchical protocol that includes an Internet Protocol (IP) layer and a Transmission Control Protocol (TCP) layer.

Next, an Internet Protocol datagram (IP datagram) which is packet data of an Internet protocol layer transmitted on the Internet.
The header (IP header) field stores a source Internet protocol address and a destination Internet protocol address designating addresses of a mobile terminal and a voice control host device on the Internet, and the data field of the Internet protocol datagram contains And a transmission control protocol segment which is packet data of the transmission control protocol layer.

The transmission control protocol segment (TCP segment) header (T
The CP header field stores a source port number and a destination port number that specify a communication protocol for voice recognition processing. The data field of the transmission control protocol segment includes a terminal identification for identifying a mobile terminal. A code, a real-time reply request command, a non-real-time reply request command, a batch transfer request command, voice data, or recognized voice data are stored.

With this limited configuration, the mobile terminal and the voice control host device can be easily specified worldwide, and
The coexistence of the voice recognition processing service and other information processing services can be easily realized.

In the configuration of the present invention described above, the voice control host devices are mutually connected by a network, and the functions corresponding to the mobile terminal connecting means, the voice data receiving means, the voice recognition means, and the recognized voice data return means are distributed. And a plurality of host computers.

With this limited configuration, load distribution on the host device side can be easily realized. Note that the above-described mobile terminal and voice control host device alone are also within the scope of the present invention.

[0029]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the present embodiment, in a mobile terminal having a built-in PHS function, a voice signal input from a microphone during a call or in an off-line state is connected to a LAN in a specific voice service provider from the PHS network via the Internet. According to the present invention, the voice signal is sent to the voice control host device, and after the voice signal is recognized, the voice signal is returned to the mobile terminal in real time or at a desired timing in a lump and used for information management processing in the mobile terminal. This is a major feature related to. According to such a system, the mobile terminal can receive the provision of a practically accurate speech recognition function at a low cost without having to provide a sophisticated speech recognition environment. <System Configuration> FIG. 1 is an overall system configuration diagram of an embodiment of the present invention.

The mobile terminal 101 has a PHS terminal function.
It is connected to the HS network 103. The wireless base 102 is a public wireless base provided on a public telephone booth, a telephone pole, a building rooftop, an underground passage, or the like on a street, or a parent-child telephone device in a subscriber's house. When connected to the parent-child telephone device, the PH
It is directly connected to the public telephone network without going through the S network. In addition,
Instead of the wireless base 102, a configuration may be adopted in which the wireless base 102 is connected to the PHS network 103 or the public telephone network by wired communication via a wired connection device.

The PHS network 103 is a public telephone network or an ISD
It is interconnected with N networks, and a mobile terminal control host device 104 connected to the Internet 105 by a high-speed digital leased line or the like is connected to these networks.

The mobile terminal 101 is connected to the radio base 102 and the P
By automatically dialing up the mobile terminal control host device 104 connected to the public telephone network or the ISDN network via the HS network 103, it is possible to connect to the Internet 105.

A predetermined voice service provider LA is connected to the Internet 105 via a high-speed digital leased line or the like.
The router device 106 connected to N107 is connected. LAN 107 is an Ethernet system, ATM (As
Synchronous Transfer Mode) or FDDI. LAN 107
Is connected to a voice control host device 108.

After automatically dialing up the mobile terminal 101 to the mobile terminal control host device 104, the mobile terminal 101, the Internet 105, the router device 106, and the LAN 10
7 can communicate with the voice control host device 108.

Now, when the user instructs communication with the voice control host device 108 from the touch panel on the input unit 109 in the mobile terminal 101, the control unit 110 sends the voice control host device 108 to the communication unit 111. Request to start communication.

When the communication unit 111 is requested by the control unit 110 to start communication, the current mobile terminal control host device 104
If not connected to a wireless base (or wired connection device)
After making a wireless (or wired) call to 102 and connecting to the PHS network 103, a dial-up call is made by specifying the access telephone number of the mobile terminal control host device 104.

When the mobile terminal control host device 104 receives an incoming call, the communication unit 111 in the mobile terminal 101 first communicates with the connection establishment unit 113 in the mobile terminal control host device 104, thereby establishing a standard communication protocol on the Internet 105. Negotiation for establishing a connection by the TCP / IP and PPP methods. As a result, an IP address, which is an identification address on the Internet 105, is assigned from the mobile terminal control host device 104 to the communication unit 111 in the mobile terminal 101, and the mobile terminal 101
Can access the Internet 105.

If the communication unit 111 in the mobile terminal 101 is already connected to the mobile terminal control host device 104, the above dial-up transmission is omitted. Then, the mobile terminal 10
1 includes a “destination IP address” that is a preset IP address of the voice control host device 108 and a “source IP address” that is an IP address assigned by the mobile terminal control host device 104. And a "terminal identification code" (for example, a PHS telephone number) for identifying the mobile terminal 101, and a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing based on a user's designation. Stored T
The CP / IP packet is transmitted to the Internet 105.

The TCP / IP packet is transmitted by the routing unit 114 in the mobile terminal control host device 104 and the relay host device (not shown) in the Internet 105 based on the “destination IP address” stored in the TCP / IP packet. Router device 106 in the provider
After that, the packet is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving unit 115 receives the received TC
From the P / IP packet, the “source IP address”
The “terminal identification code” and the real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing are taken out and passed to the mobile terminal communication control unit 116 in the speech control host device 108.

The mobile terminal communication control unit 116 transmits the delivered “source IP address”, “terminal identification code”,
After registering information relating to a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing in a processing terminal registration table (FIG. 13) described later, transmission permission data is transmitted to the packet transmitting / receiving unit 115. Mobile terminal 1 of stored TCP / IP packet
Request a reply to 01.

The packet transmission / reception section 115
The P / IP packet is transmitted to the IP address corresponding to the mobile terminal 101. In this way, the voice control host device 108 can execute sentence voice recognition processing on voice data transferred from the mobile terminal 101.

When the communication unit 111 in the mobile terminal 101 receives the TCP / IP packet storing the transmission permission data from the voice control host device 108, the communication unit 111 transfers the transmission permission data stored therein to the control unit 110.

After the transmission permission data is delivered, the control unit 110 in the mobile terminal 101 sends the communication unit 111 the voice of the voice data input from the microphone by the voice call operation or the voice input operation in the offline state. Request transmission to the control host device 108.

The communication unit 111 transmits the TCP / IP packet storing the voice data to the voice control host device 10.
8 is transmitted to the IP address corresponding to No. 8. This TC
The P / IP packet has the “destination IP” stored therein.
Based on the address, via the routing unit 114 in the mobile terminal control host device 104, the relay host device (not shown) in the Internet 105, the router device 106 in the voice service provider, and the LAN 107,
Packet transmission / reception unit 115 in voice control host device 108
Transferred to

Packet transmitting / receiving section 115 receives the received TC
The voice data stored in the P / IP packet is extracted and delivered to the mobile terminal communication control unit 116 in the voice control host device 108.

The mobile terminal communication control section 116 delivers the delivered speech data to the sentence speech recognition section 117. The sentence speech recognition unit 117 executes sentence speech recognition processing on the delivered speech data, and delivers the recognized speech sentence data as the recognition result to the mobile terminal communication control unit 116.

The mobile terminal communication control unit 116
When the real-time start request command of the sentence speech recognition process is designated from 01, the packet transmission / reception unit 115 immediately transmits the T
A request is made to return a CP / IP packet to the mobile terminal 101.

The packet transmitting / receiving unit 115
The P / IP packet is transmitted to the IP address corresponding to the mobile terminal 101. Communication unit 1 in mobile terminal 101
When receiving the TCP / IP packet storing the recognized voice sentence data from the voice control host device 108, the control unit 110 transfers the recognized voice sentence data stored therein to the control unit 110.

The control unit 110 in the mobile terminal 101 outputs the above-described recognized voice sentence data to the output unit 112. The output unit 112 outputs a sentence corresponding to the recognized speech sentence data as L
Display on the CD display. The user writes this sentence data
It can be arbitrarily stored or processed.

The mobile terminal communication control unit 116 controls the mobile terminal 1
When a non-real-time start request command for sentence speech recognition processing is designated from 01, the recognized speech sentence data is not immediately returned to the mobile terminal 101 but is sequentially stored. Then, the mobile terminal communication control unit 116
Later, at the timing when the batch transfer request command of the sentence speech recognition result is received from the mobile terminal 101, the packet transmission / reception unit 115 sends the stored TCP / IP packet of the stored speech / speech data to the mobile terminal 101. Request a bulk reply to.

The packet transmitting / receiving unit 115
The P / IP packet is transmitted to the IP address corresponding to the mobile terminal 101. Thereby, the mobile terminal 101
Then, the batch-transferred recognition voice text data is displayed on the LCD display unit, and the user can arbitrarily store or process the text data.

On the other hand, in addition to the communication with the voice control host device 108, the mobile terminal 101 makes a dial-up call to the mobile terminal control host device 104 by using a homepage browsing tool or an e-mail provided with the voice control host device 108. , It is possible to freely access desired resources on the Internet 105. <External Configuration of Mobile Terminal 101> FIG. 2 is an external view of the mobile terminal 101 of FIG.

The mobile terminal 101 has the appearance of a compact portable information management device, and has a microphone 201 also serving as a transmitter for inputting voice, and a microphone 201 for inputting an image which is not particularly related to the present invention. 1. The camera 202, an LCD display unit 203 having a touch panel function of displaying various information and receiving a touch input or a pen input, and a speaker 204 also serving as a receiver for outputting voice. It has a wireless antenna 205 for transmitting to the base 102 and a socket 206 for connecting to a wired connection device instead of the wireless base 102.

Further, an I for inserting various IC cards is provided.
An optical transceiver 208 for performing infrared optical communication between the C card slot 207 and another mobile terminal 101 or a personal computer or the like is provided.

The switch 209 is a power switch. <Functional Block Configuration of Mobile Terminal 101> FIG. 3 is a functional block diagram of the mobile terminal 101.

As shown in FIG. 1, the mobile terminal 101 includes an input unit 109, a control unit 110, a communication unit 111, and an output unit 112, and is mutually connected by a bus 326.

First, the input unit 109 includes a part for inputting voice, a part for inputting an image which is not particularly related to the present invention, and a part of a touch panel mechanism described later in the operation of the output unit 112. .

The part for inputting sound is the microphone 301, A
It comprises a / D conversion unit 302 and a microphone control unit 303. The microphone 301 (corresponding to 201 in FIG. 2) is a PHS
Also serves as a telephone transmitter, and inputs a voice uttered by the user.

The A / D converter 302 converts an analog audio signal input from the microphone 301 into digital audio data, and further converts the digital audio data into an ADPCM (Adaptive Differential) which is a PHS standard audio encoding system.
l Pulse Code Modulation: Adaptive differential linear pulse coding
Encode according to the method. This part has already been put to practical use as an LSI integrated circuit constituting a PHS terminal.

The microphone control unit 303 transfers the coded voice data to the communication control unit 321 in the communication unit 111 during a call and places it on the communication channel, and further performs the control unit during the sentence voice recognition processing. RAM 31 in 110
Transfer to 7.

On the other hand, a part for inputting an image is a CCD (Ch
arge Coupled Device) Camera 304, A / D converter 3
05, a memory 306, and a camera control unit 307.

The CCD camera 304 captures an arbitrary image based on a user operation. The A / D conversion unit 305
An analog video signal captured by the CCD camera 304 is converted into digital image data.

The memory 306 stores digital image data in frame units. The camera control unit 307 includes a CCD
Camera 304, A / D converter 305, and memory 306
Control the operation of.

Next, the output unit 112 is composed of a part for outputting a sound and a part for outputting an image. The part that outputs audio includes a speaker 308, a D / A converter 309,
And a speaker control unit 310.

The speaker control unit 310 sends the PHS call voice data received from the communication control unit 321 in the communication unit 111 or the synthesized voice data received from the RAM 317 in the control unit 110 to the D / A conversion unit 309. Forward.

The D / A converter 309 decodes the received audio data, converts it into an analog audio signal, and emits it as sound from the speaker 308 (corresponding to 204 in FIG. 2).

The part for outputting the image is the LCD display unit 20
3, LCD driver 312, memory 313, and LCD
It comprises a control unit 314. The LCD control unit 314
Various image data such as character data, image data, and command button data received from the RAM 317 in the control unit 110 are stored in the memory 313 in frame units.
The CD driver 312 is started.

The LCD driver 312 displays the image data read from the memory 313 in frame units on the LCD display unit 311 (corresponding to 203 in FIG. 2). In addition,
A transparent touch panel is provided on the surface of the LCD display unit 311 (203 in FIG. 2). The user touches the touch panel with his / her finger or pen in accordance with command button data displayed on the LCD display unit 311. Command input. This input signal is transferred by the touch panel control unit 315 to the RAM 317 in the control unit 110.

Subsequently, the control unit 110 controls the CPU 316,
It comprises a RAM 317 and a ROM 318, an IC card interface unit 319, and an IC card 320 inserted into an IC card slot 207 (FIG. 2) as required.

The CPU 316 controls the operation of the entire mobile terminal 101 according to the control program stored in the ROM 318 while using the RAM 317 as a work area.

The IC card interface unit 319
The input / output of data to / from the C card 320 is controlled. Lastly, the communication unit 111 includes a communication control unit 321, a wireless driver 322, a wireless antenna 323, a wired driver 324,
And a socket 325.

The communication control unit 321 executes a PHS call process and a TCP / IP communication process (to be described later) with the Internet 105, and controls the wireless driver 322 or the wired driver 324.

The wireless driver 322 mutually converts communication data with a PHS wireless signal transmitted / received via the wireless antenna 323 (corresponding to 205 in FIG. 2) during wireless communication. The PHS radio signal has a radio frequency of 1.9 GHz, a carrier frequency interval of 300 kHz, a TDMA-TDD radio access scheme of 4 channels / carrier,
π / 4 shift QPSK modulation method and 384 kbits /
This is a wireless signal based on the wireless transmission speed of sec.

On the other hand, at the time of wired communication, the wired driver 324 mutually converts communication data with a wired signal transmitted / received via the socket 325 (corresponding to 206 in FIG. 2). This is a typical telephone band modem modulated signal.
The operation of the embodiment of the present invention having the above configuration will be described in detail below. <Process of Mobile Terminal 101> First, the process of the mobile terminal 101 will be described.

FIG. 4 shows the CPU 3 in the control unit 110 shown in FIG.
16 stores the ROM 318 in the control unit 110 after the power is turned on.
4 is an overall operation flowchart showing a control operation realized as an operation of executing a control program stored in the control program.

The control programs for realizing the functions shown in the operation flowcharts of FIGS. 4, 5 and 8 and the data necessary for them are stored in, for example, the IC shown in FIG.
IC card 320 detachable from card slot 207
Is stored in the form of a program code readable by the CPU 316, and the program code is stored in the CPU 316.
Or the program code can be directly executed by the RAM 317 or the writable ROM 31 as needed.
8 to be executed by the CPU 316. Alternatively, the above-described control program and data necessary for the control program are received from another device via a communication unit 111 from a wireless or wired communication line or an optical transceiver 208 (FIG. 2), and stored in a RAM 317 or a writable ROM 318. It may be configured to be loaded and executed by the CPU 316.

First, steps 401 → 402 → 403 →
In the repetition loop of 404 → 401, it is determined whether touch panel input has been detected from the touch panel control unit 315 in FIG. 3 (401), and the recognized voice text data is received from the voice control host device 108 (FIG. 1). (402), other reception / display processing (403), and necessary data transmission processing (404)
Is executed.

Touch panel control section 315 notifies of touch panel input detection, and determination in step 401 is YES.
Then, in step 405 or 406, the touch panel input is performed by the CCD camera 304 in FIG. 3 (202 in FIG. 2).
It is determined whether or not the input instruction is the input instruction of the microphone 301 of FIG. 3 (201 of FIG. 2).

The touch panel input is the CCD camera 3 shown in FIG.
04 (202 in FIG. 2) and the
If the determination at 5 is YES, at step 407, the camera control unit 307 in the input unit 109 in FIG. 3 is instructed to start input processing of, for example, a handwritten character image. Thereafter, the process proceeds to the transmission process of step 404. Since the image input processing is not particularly related to the present invention, a detailed description thereof will be omitted.

The touch panel input is the microphone 301 shown in FIG.
If the input instruction is (201 in FIG. 2) and the determination in step 406 is YES, in step 408, the microphone control unit 303 in the input unit 109 in FIG. 3 is instructed to start the voice input process. . The start instruction of the voice input process is, for example, a start instruction of the PHS call process or a start instruction of the voice input process in an offline state for executing the sentence voice recognition process.

The microphone control unit 303 is compatible with the CPU 31 described above.
6, the microphone 301 (201 in FIG. 2)
And instruct the A / D converter 302 to start voice input. As a result, the audio data input from the microphone 301 (201 in FIG. 2) is output from the A / D converter 302.

After that, when the above-mentioned voice input processing start instruction is a PHS telephone call start instruction, the above-mentioned voice data is loaded on a predetermined telephone channel by the transmission processing (not shown) of the communication control unit 321. And sent to the other party.

In the case where the above-mentioned voice input processing start instruction includes a voice input processing start instruction for sentence voice recognition processing, the voice input processing is thereafter input from microphone 301 (201 in FIG. 2) and microphone control unit 303 The audio data output from is transmitted to the audio control host device there in the transmission processing of step 404 described later.

The touch panel input is the CCD camera 3 shown in FIG.
04 (202 in FIG. 2), the microphone 30 in FIG.
If the input instruction is not the input instruction 1 (201 in FIG. 2), the determinations in steps 405 and 406 are NO, and in step 409, another key input processing is executed. Thereafter, the process proceeds to the transmission process of step 404.

On the other hand, the voice control host device 108 (FIG. 1)
From the RAM 31 in the control unit 110 via the communication unit 111
7, the recognition voice sentence data is received, and step 401 →
If the determination in step 402 in the iterative loop of 402 → 403 → 404 → 401 is YES, step 4
In step 10, the recognized voice sentence data is stored in the RAM 317.
To the memory 313 in the output unit 112,
The control unit 314 is instructed to display the recognized voice sentence data.

As a result, under the control of the LCD control unit 314, the received recognized voice sentence data is displayed on the LCD display unit 311 (203 in FIG. 2) from the memory 313 via the LCD driver 312.

Next, the transmission processing in step 404 will be described. FIG. 5 is an operation flowchart showing details of the transmission processing. First, in step 501, it is determined whether or not a key input from the touch panel processed by the other key input processing in step 409 in FIG. 4 is accompanied by a transmission instruction.

If this determination is NO, step 50
Proceed to step 5. If the determination in step 501 is YES, in step 502, it is determined whether the mobile terminal 101 is currently connected to the mobile terminal control host device 104 in FIG.

If mobile terminal 101 is currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 502 is YES, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 in FIG. 3 to transmit a “terminal identification code” of the mobile terminal 101 and a command corresponding to the key input process in step 504. As a result, the communication control unit 321 generates a TCP / IP packet storing the “terminal identification code” and the command, and transmits the TCP / IP packet to a predetermined host connected to the Internet 105 (for example, the voice control host device in FIG. 1). 1
08).

If mobile terminal 101 is not currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 502 is NO, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 503, and then executes step 504.

As will be described later in detail, a transmission instruction of a real-time start request command for sentence speech recognition processing or a non-real-time start request command of sentence speech recognition processing based on a user's designation, and a transmission instruction of a termination request command for sentence speech recognition processing The transmission instruction of the batch transfer request command of the sentence speech recognition processing is issued in step 504 described above.

As described above, the determination at step 501 is N
In the case of O or after the processing of step 504, step 50
In step 5, in step 408 of FIG. 4, an instruction to start a speech input process for a sentence speech recognition process is executed.
It is determined whether an instruction to transmit audio data to the audio control host device 108 (FIG. 1) has been issued.

If this determination is NO, step 51
Proceed to process 0. If the determination in step 505 is YES, in step 506, transmission permission data, which is a response to the real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing, is already sent from the speech control host device 108. It is determined whether a reply has been sent.

If the determination is NO, the voice control host device 108 has not yet completed the preparation for the real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing from the mobile terminal 101. Since there is not, the process proceeds to step 510.

The voice control host device 108 has already returned the transmission permission data which is a response to the real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing, and step 506 is performed.
If the determination is YES, then at step 507,
The mobile terminal 101 is currently the mobile terminal control host device 1 of FIG.
04 is determined.

If mobile terminal 101 is currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 507 is YES, CPU 316 in control unit 110 in FIG.
In step 509, the microphone control unit 303 in the input unit 109 shown in FIG.
The communication control section 321 in the communication section 111 is requested to transmit the voice data transferred to the communication section 7. As a result, the communication control unit 321 generates a TCP / IP packet in which the above-mentioned voice data is stored, and transmits the TCP / IP packet to the voice control host device 10 of FIG.
Send to 8

If mobile terminal 101 is not currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 507 is NO, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 508, and then executes step 509.

As will be described in detail later, an instruction to transmit voice data for sentence voice recognition processing is issued in step 509 described above. As described above, when the determination in step 505 or 506 is NO or after the processing in step 509, in step 510, a start instruction of the image input processing is executed by step 407 in FIG. It is determined whether a transmission instruction to an image control host device (not shown) connected to the Internet 105 is issued.

If the determination is NO, the transmission process of step 404 in FIG. 4 ends. If the determination in step 510 is YES, in step 511, the mobile terminal 10
1 is currently connected to the mobile terminal control host device 104 in FIG.

If the mobile terminal 101 is currently connected to the mobile terminal control host device 104 in FIG. 1 and the determination in step 511 is YES, the CPU 316 in the control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 to transmit the image data obtained in the memory 306 in the input unit 109 shown in FIG. As a result, the communication control unit 321 generates a TCP / IP packet storing the image data, and transmits the TCP / IP packet to the image control host device 108 (not shown) connected to the Internet 105.

If the mobile terminal 101 is not currently connected to the mobile terminal control host device 104 in FIG. 1 and the determination in step 511 is NO, the CPU 316 in the control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 512, and then executes step 513.

Since the image data transmission instruction in step 513 is not particularly related to the present invention, a detailed description thereof will be omitted. As described above, when the determination in step 510 is NO or after the processing in step 513, the transmission processing in step 404 in FIG. 4 ends. <Format of Communication Data> FIG.
And mobile terminal control host device 104 and Internet 1
FIG. 5 is a format diagram of communication data communicated with the MFP 05 (voice control host device 108).

Communication data between the mobile terminal 101 and the mobile terminal control host device 104 is PPP (Point-to-P
oint Protocol).
Using the PPP frame shown in FIG. 6A (transferred from left to right in the figure), 32 kbits of the PHS standard
It is transmitted over a digital communication channel having a transmission rate of s / sec.

"Flag", which constitutes a PPP frame,
In each of the "address" and "control" fields, each fixed bit string shown in FIG. 6A is set. The FCS having a data length of 2 octets is called a frame check sequence, and is data for error detection / correction of PPP frame data. “Information” of a PPP frame transferred after a PPP link is established between the mobile terminal 101 and the mobile terminal control host device 104
The field (having a variable data length) includes I, which is a basic transmission unit of data on the Internet 105 (FIG. 1).
P datagram is stored, in which case a "protocol" field having a data length of 2 octets contains:
A hexadecimal value “0021” indicating that the IP datagram is stored is stored in the “information” field.

"Information" of PPP frame
The IP datagram is stored in the field as described above. This IP datagram is a basic transmission unit of data on the Internet 105 as described above. IP
The datagram is defined in accordance with the Internet Protocol (IP), provides a function for uniquely transferring data stored in its “data” field to a destination host device on the Internet 105, and has an address on the Internet 105. , The function of transferring the IP datagram itself to the host specified by the "destination IP address" through a fixed route on the Internet 105, and the fragmentation (division) and reassembly of the IP datagram itself. It has functions and the like.

As shown in FIG. 6B, an IP datagram is composed of an IP header field and a data field. The IP header field contains all the information necessary to deliver the IP datagram containing it. FIG. 7A is a format diagram of the IP header.

The IP header has a data length of 5 to 6 words, with 32 bits as one word. This data length is stored in the “header length” field of the first word. The length is stored in the first word of the "total length of IP datagram" field.

In the “version” field of the first word, the version of the Internet Protocol (IP) that defines the transfer method of the IP datagram is set, and the current version is 4.

[0110] The "service type" field of the first word stores information indicating the priority of delivery.
This is not particularly relevant to the present invention. Each field of the second word indicates that the IP datagram is
The control information in the case of fragmentation (division) due to the above-described transfer restriction is defined. First, the “identification number”
The field contains this fragmented I
A unique integer for identifying the undivided IP datagram to which the P datagram belongs is set. Next, in the “fragment offset” field, offset information indicating which part of the IP datagram before the division corresponds to the IP datagram that is the divided fragment is set. In the "flag string" field, it is set whether or not another fragment constituting the undivided IP datagram to which this divided IP datagram belongs is followed by the divided fragment. With the above information, even if the IP datagram is fragmented at the relay host on the Internet 105,
The IP datagram before division can be accurately restored on the receiving side.

The “lifetime” of the third word (TTL: Time
In the “To Live” field, time information in seconds indicating how long the IP datagram is allowed to exist on the Internet 105 is set. The relay host on the Internet 105 subtracts the above field value each time the IP datagram is processed, and discards the IP datagram whose value becomes 0 or less from the Internet 105. This suppresses the occurrence of excessive traffic on the Internet 105. Note that retransmission control for a discarded IP datagram is realized in control processing for a TCP segment stored in the IP datagram.

In the “protocol” field of the third word, an integer value for defining the format of data stored in the “data” field of the IP datagram is set. In the case of the present embodiment, as shown in FIG. 6C, since the TCP segment data is stored in the "data" field of the IP datagram, an integer value 6 defining the format is set. You.

In the “header checksum” field of the third word, checksum data for detecting an error in the data of the IP header is set. A 32-bit “source IP address” is set in the fourth word. For example, when an IP datagram is transferred from the mobile terminal 101 to the voice control host device 108, the “source IP address” is assigned from the mobile terminal control host device 104 to the mobile terminal 101 by a transmission process described later. The set IP address is set. By storing this “source IP address”, the voice control host device 108 in FIG. 1 can return format text data and the like to the mobile terminal 101 via the Internet 105.

The fifth word contains a 32-bit “destination IP”.
For example, when an IP datagram is transferred from the mobile terminal 101 to the voice control host device 108, the “destination IP address” is fixedly assigned to the voice control host device 108. The IP address is set.The routing unit 114 in the mobile terminal control host device 104, each relay host device on the Internet 105, and the router device 106 in the voice service provider are stored in the received IP datagram. By identifying the "destination IP address", the delivery route of the IP datagram is determined in advance according to the routing control table information of each device, and finally the IP datagram is transferred to the voice control host device in the voice service provider. 108.

The "IP option" field of the sixth word is optional, and is used for information for testing or debugging each network constituting the Internet 105 and control for controlling or monitoring a delivery route on the Internet 105. Information and the like are set, but this is not particularly relevant to the present invention.

In the “padding” field of the sixth word, padding data for adjusting the data length is set. Next, TCP segment data is stored in the "data" field of the IP datagram. This T
The CP segment is defined according to the Transmission Control Protocol (TCP), and its "data"
The data stored in the field is transferred to the Internet 105
A function is provided for accurately delivering the packet to the above destination host device in an appropriate order. IP datagrams provide only the unique transfer function of data on the Internet 105, and the function of ensuring data reliability (retransmission control function, etc.)
Is provided, whereas the TCP segment provides a function for ensuring data reliability.

As described above, the communication data has the hierarchical structure of the (PPP frame), the IP datagram and the TCP segment. Therefore, it is necessary to efficiently deliver the data on the Internet 105 under a processing load as small as possible. Yes, in order to efficiently address the different demands of achieving the most reliable data delivery end-to-end. As a result, the relay host device on the Internet 105 refers to only the IP header of the IP datagram, so that the information (TCP segment) stored in the “data” field of the IP datagram is as fast and efficiently as possible. To the destination host device, and between end-to-end (source host device and destination host device), TCP
By referring to the TCP header of the segment, highly reliable data communication such as retransmission control can be realized.

The TCP segment is composed of a TCP header field and a data field as shown in FIG. FIG. 7B is a format diagram of the TCP header.

As in the case of the IP header, the TCP header has a data length of 5 to 6 words with 32 bits as one word, and this data length is stored in the “header length” field of the fourth word. Also, the data length of the entire IP datagram is stored in the “total length of IP datagram” field of the first word.

In the "source port number" field and the "destination port number" field of the first word, a 16-bit integer value specifying a communication protocol for the sentence speech recognition process is set.

The packet transmitting / receiving unit 115 (FIG. 1) in the voice control host device 108 stores various data such as e-mail data in addition to the TCP segment in which voice data for sentence voice recognition processing is stored. Is transmitted and received to transmit and receive various TCP segments.
By recognizing the value of the "destination port number" field set in the TCP header of the P segment, the data stored in the "data" field of the TCP segment can be transmitted to any application executed by the voice control host device 108. You can decide to hand it over.

The packet transmitting / receiving unit 115 determines that the value of the “destination port number” field set in the TCP header of the received TCP segment indicates a value corresponding to the communication protocol for the sentence speech recognition process. , The voice data stored in the “data” field of the TCP segment can be delivered to the mobile terminal communication control unit 116.

Similarly, the communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 also includes homepage data and e-mail data in addition to the TCP segment in which the recognized voice text data is stored. To send and receive various TCP segments storing various data,
By recognizing the value of the “destination port number” field set in the TCP header of the received TCP segment, the data stored in the “data” field of the TCP segment can be transmitted to any application executed by the mobile terminal 101. You can decide to hand it over.

Then, the communication control unit 321 transmits the received T
If the value of the “destination port number” field set in the TCP header of the CP segment indicates a value corresponding to a communication protocol for sentence speech recognition processing, the control unit 110 (FIGS. 1 and 3) , The reception of data for sentence speech recognition processing is notified, and the recognized speech sentence data and the like stored in the “data” field of the TCP segment can be delivered.

Furthermore, the packet transmission / reception unit 115 in the voice control host device 108 and the communication unit 111 of the mobile terminal 101
The communication control unit 321 in the above can confirm the source application by confirming the “source port number” set in the TCP header of the received TCP segment.

Next, the second part of the TCP header shown in FIG.
The word "sequence number" field contains the current TC
From the entire byte stream transmitted from the transmission side to the reception side in the P connection, the transmission side determines from which byte of the byte stream the data stored in the "data" field of this TCP segment corresponds. This is a field for notifying the receiving side. Conversely, the “acknowledgement number” field of the third word indicates the number of bytes that the receiving side has received without error in the entire byte stream transmitted from the transmitting side to the receiving side in the current TCP connection. This is a field for notification from the receiving side to the transmitting side. As a result, for example, the voice data can be transferred from the mobile terminal 101 to the voice control host device 108 in the correct order and with high reliability.

In the "flag string" field of the fourth word, a value indicating the type of the TCP segment is set. T
In the CP communication, for example, various control data for acknowledgment is transmitted at the start or end of the connection, for example, and the type of the control data is set in the “flag string” field.

The “window” field of the fourth word is a field for notifying the receiving side to the transmitting side window data indicating how many bytes of data the receiving side can currently receive continuously. is there. This enables data flow control from the receiving side to the transmitting side. For example, when the load on the voice control host device 108 is high, the transmission of voice data to the mobile terminal 101 is suppressed. Fine control is possible.

The "reserved" field of the fourth word is a reserved field. The “checksum” field of the fifth word stores checksum data for detecting an error in the data stored in the TCP header and the “data” field. Thus, for example, the voice control host device 108 can correctly receive voice data from the mobile terminal 101. The "urgent pointer" of the fifth word is control data for communicating urgent data (interrupt data, abort data, etc.), but this is not particularly relevant to the present invention.

The "option" field of the sixth word is used, for example, to specify the maximum segment length that can be communicated between the transmitting and receiving apparatuses, but this is not particularly relevant to the present invention.

In the "padding" field of the sixth word, padding data for adjusting the data length is set. The communication (termination) processing function of the TCP segment having the above-described configuration is realized by the communication control unit 321 (FIG. 3) in the communication unit 111 in the mobile terminal 101, and the packet transmission / reception unit 115 ( 1). Note that the control program executed by the CPU 316 in the mobile terminal 101 may be configured to realize the above processing functions. <Outgoing Call Processing> As described above, the control unit 1 of the mobile terminal 101
The CPU 316 in FIG. 10 (FIG. 3)
5, when the mobile terminal 101 is not currently connected to the mobile terminal control host device 104 in FIG. 1 and the determination in step 502, 507, or 511 is NO, In step 503, 508, or 512, the communication control unit 3 in the communication unit 111 in FIG.
Request the transmission processing to 21. By this request,
The transmission process executed by the communication control unit 321 is shown by the operation flowchart in FIG.

First, in step 801, a link establishment phase is executed. In this phase, the access telephone number of the mobile terminal control host device 104 is automatically dialed up and the mobile terminal control host device 104
Is received, the link control protocol (LC
P), a maximum data length of a PPP frame (FIG. 6A) used for communication, a non-transparent character to be escaped, a "protocol" field of the PPP frame (FIG. 6). (a)) Determine whether to compress the data length from 2 octets to 1 octet, and omit (compress) the "address" field (FIG. 6 (a)) having the fixed value "11111111" of the PPP frame. A negotiation regarding the determination of the presence / absence is performed with the connection establishment unit 113 (FIG. 1) in the mobile terminal control host device 104. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. By setting a hexadecimal value “c021” specifying the LCP in the “protocol” field and setting necessary control data in the “information field”,
Be executed.

Next, in step 802, an authentication phase is executed. In this phase, PAP (Pass
word Authentication Protocol) or CHAP (Challe
The authentication of the user using the mobile terminal 101 is performed from the connection establishment unit 113 (FIG. 1) in the mobile terminal control host device 104 to the mobile terminal 101 using an authentication protocol called ngeHandshake Authentication Protocol). As a result, the Internet provider operating the mobile terminal control host device 104 becomes the mobile terminal 101
Can be determined whether or not the user who uses is a contracted user. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. A hexadecimal value “c023” specifying PAP or a hexadecimal value “c223” specifying CHAP is set in the “protocol” field, and necessary authentication data is set in the “information field”. Will be executed.

Finally, at step 803, a network layer protocol phase is executed. In the case of the present embodiment, in this phase, a protocol called an IP control protocol (IPCP) is used, and T
Whether to compress the CP header (see FIG. 7B) is determined, and one of the free (unused) IP addresses that can be allocated by the mobile terminal control host device 104 is allocated to the mobile terminal 101. In addition, necessary route information is set in the communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 and the routing unit 114 (FIG. 1) in the mobile terminal control host device 104. After this,
By using the IP address, the mobile terminal 101 can access the voice control host device 108 connected to the Internet 105 and any resource desired by the user on the Internet 105. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. The "protocol" field is used to set a hexadecimal value "8021" for specifying the IPCP, and the "information field" is set with data for negotiating a necessary IP address, and executed.

By the above series of operations, the mobile terminal 101
Can exchange a PPP frame storing a TCP / IP packet for communication with the routing unit 114 in the mobile terminal control host device 104, and the mobile terminal 101 can freely use resources on the Internet 105. Can be accessed.

Note that the mobile terminal 101 can be configured to have, for example, a two-channel simultaneous communication function in order to allow access to the voice control host device 108 and the like even during a PHS call.

The communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 communicates with the mobile terminal control host device 104 when the transmission / reception data is not detected for a predetermined time (for example, 10 minutes). May be configured to automatically disconnect the PPP link. <Detailed operation of transmission / reception processing of mobile terminal 101 related to sentence voice recognition processing> When the user operates the touch panel of mobile terminal 101 to instruct to start real-time or non-real-time sentence voice recognition processing, and thereafter, mobile terminal 101 The detailed operation of the transmission / reception processing to be executed will be described.

After the operation of the touch panel described above is detected by the touch panel control unit 315 in FIG.
In the control operation corresponding to the above-described operation flowchart of FIG. 4 executed by the CPU 316 (FIG. 3) in the CPU 10, the determination in step 401 is YES, and the determinations in steps 405 and 406 are NO.
09 is detected by executing another key input process. Further, in the transmission processing in step 404, the determination in step 501 in FIG. 5 described above is YES, and the transmission processing is executed in step 503 as necessary. Then, in step 504, the “terminal identification code” of the mobile terminal 101 is transmitted. Then, the communication control unit 321 in the communication unit 111 of FIG. 3 is requested to send a command corresponding to the key input process indicating the start instruction of the above sentence voice recognition process.

As a result, the communication control section 321 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 6 (c) and 7 (b),
In the “source port number” field and the “destination port number” field, a 16-bit integer value that specifies a communication protocol for sentence speech recognition processing is set. Then, in the “data” field of the TCP segment,
A “terminal identification code” (for example, the PHS telephone number) specifying the mobile terminal 101 and a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing based on a user's designation are stored. .

Next, the communication control unit 321 executes the above-described TCP
Figure 6 with segments stored in the "data" field
An IP datagram having the format shown in (b) is generated. In this case, in the IP header having the format shown in FIGS. 6 (b) and 7 (a), the "protocol" format includes a format that defines the format of the TCP segment data stored in the "data" field. Numerical value 6 is set. In the “source IP address” field, the connection establishment unit 113 in the mobile terminal control host device 104 transmits the communication unit of the mobile terminal 101 by the transmission processing already executed (see the description of step 803 in FIG. 8). The assigned IP address is set for the communication control unit 321 in 111. Further, the "destination IP address" field contains the voice control host device 1
08 is set.

Then, the communication control unit 321 transmits the IP
The datagram is stored in the "Information" field, and the IP
Hexadecimal value "00" indicating that the datagram is stored
FIG. 6A in which “21” is stored in the “protocol” field
A PPP frame having the format shown in FIG. 8 is generated, and according to the path information (see the description of step 803 in FIG. 8) set in the communication control unit 321, the PPP frame is generated.
The P frame is transmitted to the mobile terminal control host device 104. Hereinafter, when a data unit including the above-described TCP segment, IP datagram, and PPP frame is transferred in the Internet 105, the data unit is simply referred to as a TCP / IP packet.

This TCP / IP packet is based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. In particular, by a relay host device (not shown),
After being transferred to the router device 106 in the voice service provider, the data is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving section 115 has its own IP address set in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. That the TCP / I
Receive a P packet.

Then, the packet transmitting / receiving unit 115 specifies a communication protocol for text / speech recognition processing in the “destination port number” field and the “source port number” field of the TCP segment constituting the received TCP / IP packet. By confirming that a 16-bit integer value is set, the mobile terminal communication control unit 116
(FIG. 1) is notified of the reception notification.

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram constituting the received TCP / IP packet, and reads “terminal identification code” from the “data” field of the TCP segment constituting the TCP / IP packet. A real-time start request command for voice recognition processing or a non-real-time start request command for sentence voice recognition processing is extracted, and the data is transferred to the mobile terminal communication control unit 116.

As a result, a TCP / IP packet storing transmission permission data is returned from the voice control host device 108 to the mobile terminal 101 as described later. This TCP / IP packet is composed of the IP
“Destination IP” stored in the IP header of the datagram
After being transferred to the routing unit 114 in the mobile terminal control host device 104 by the router device 106 in the voice service provider and the relay host device (not shown) in the Internet 105 based on the "address", the PHS network Mobile terminal 1 via 103 (FIG. 1)
01 to the communication control unit 321 (FIG. 3) in the communication unit 111.

The communication control unit 321 in the communication unit 111 of the mobile terminal 101 stores its own mobile terminal 101 (in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. By identifying that the IP address assigned to it (temporarily or dynamically) is set.
/ IP packet is received.

Then, the communication control unit 321 transmits the received T
Confirm that a 16-bit integer value that specifies a communication protocol for text-to-speech recognition processing is set in the “destination port number” field and the “source port number” field of the TCP segment constituting the CP / IP packet By doing so, C in the control unit 110 of the mobile terminal 101
The reception notification is notified to the PU 316.

Along with this notification, the communication control unit 321 extracts transmission permission data from the “data” field of the TCP segment constituting the received TCP / IP packet, and delivers it to the CPU 316.

The CPU 316 processes the above-described reception notification and transmission permission data in step 403 of FIG.
The transmission permission data is stored in the RAM 317. In the mobile terminal 101, when the user operates the touch panel to instruct the start of the real-time or non-real-time sentence speech recognition processing, the CPU 316 determines the microphone in the input unit 109 in FIG. The control unit 303 is instructed to start a PHS call process or to start a voice input process in an offline state for executing a sentence voice recognition process. As a result, the user has started inputting sound from the microphone 301 (201 in FIG. 2) through a call operation or a sound input operation in an offline state.

Thereafter, the CPU 316 executes the steps 401 → 402 → 403 → 404 → 401 of FIG.
Step 40 executed as one processing of a repetition loop of
4 in steps 505 and 506 in FIG.
Is determined to be YES, and if necessary, the transmission processing is executed again in step 508, and then, in step 509, the data is transferred from the microphone control unit 303 in the input unit 109 to the RAM 317 in the control unit 110 shown in FIG. The transmitted voice data transmission instruction is transmitted to the communication control unit 3 in the communication unit 111.
21 is requested.

As a result, the communication control unit 321 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 6 (c) and 7 (b),
In the “source port number” field and the “destination port number” field, a 16-bit integer value that specifies a communication protocol for sentence speech recognition processing is set. Then, in the “data” field of the TCP segment,
Audio data transferred from the microphone control unit 303 in the input unit 109 shown in FIG. 3 to the RAM 317 in the control unit 110 is stored.

Next, the communication control unit 321 executes the above-described TCP
Figure 6 with segments stored in the "data" field
An IP datagram having the format shown in (b) is generated. In this case, in the IP header having the format shown in FIGS. 6 (b) and 7 (a), the "protocol" format includes a format that defines the format of the TCP segment data stored in the "data" field. Numerical value 6 is set. In the “source IP address” field, the connection establishment unit 113 in the mobile terminal control host device 104 transmits the communication unit of the mobile terminal 101 by the transmission processing already executed (see the description of step 803 in FIG. 8). The assigned IP address is set for the communication control unit 321 in 111. Further, the "destination IP address" field contains the voice control host device 1
08 is set.

Then, the communication control unit 321 transmits the IP
The datagram is stored in the "Information" field, and the IP
Hexadecimal value "00" indicating that the datagram is stored
FIG. 6A in which “21” is stored in the “protocol” field
A PPP frame having the format shown in FIG. 8 is generated, and according to the path information (see the description of step 803 in FIG. 8) set in the communication control unit 321, the PPP frame is generated.
The P frame is transmitted to the mobile terminal control host device 104.

This TCP / IP packet is routed to the routing unit 114 in the mobile terminal control host device 104 and to the Internet 105 in the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. In particular, by a relay host device (not shown),
After being transferred to the router device 106 in the voice service provider, the data is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving section 115 has its own IP address set in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. That the TCP / I
Receive a P packet.

Then, the packet transmitting / receiving unit 115 specifies a communication protocol for the sentence speech recognition processing in the "destination port number" field and the "source port number" field of the TCP segment constituting the received TCP / IP packet 16 By confirming that the integer value of the bit is set, the reception notification is notified to the mobile terminal communication control unit 116 (FIG. 1).

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram constituting the received TCP / IP packet, extracts the audio data from the “data” field of the TCP segment constituting the TCP / IP packet, and Is transferred to the mobile terminal communication control unit 116.

As a result, the mobile terminal communication control unit 116
As will be described later, the control of the sentence speech recognition process is executed, and the sentence speech recognition unit 117 executes the recognition process of the received speech data. When the real-time start request command of the sentence speech recognition process is specified by the mobile terminal 101 for the above-described speech data, the mobile terminal communication control unit 116 transmits the sentence speech recognition unit 11 as described later.
7 returns a TCP / IP packet storing the resulting recognized voice sentence data to the mobile terminal 101.

This TCP / IP packet is based on the “destination IP address” stored in the IP header of the IP datagram that composes the TCP / IP packet. After being transferred by the relay host device (not shown) to the routing unit 114 in the mobile terminal control host device 104, the communication control unit 321 (in the communication unit 111 of the mobile terminal 101) via the PHS network 103 (FIG. 1). It is transferred to Fig. 3).

The communication control section 321 in the communication section 111 of the mobile terminal 101 stores its own mobile terminal 101 (in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. By identifying that the IP address assigned to it (temporarily or dynamically) is set.
/ IP packet is received.

Then, the communication control unit 321
Confirm that a 16-bit integer value that specifies a communication protocol for text-to-speech recognition processing is set in the “destination port number” field and the “source port number” field of the TCP segment constituting the CP / IP packet By doing so, C in the control unit 110 of the mobile terminal 101
The reception notification is notified to the PU 316.

At the same time as this notification, the communication control unit 321 extracts the recognized voice text data from the “data” field of the TCP segment constituting the received TCP / IP packet, and delivers it to the CPU 316.

The CPU 316 processes the above-described reception notification and the recognized voice text data in the above-described step 402 in FIG. 4, and displays the recognized voice text data on the LCD display unit 311 (203 in FIG. 2).

By operating the touch panel of the mobile terminal 101, the user can instruct the voice control host device 108 to issue an end request command for the sentence speech recognition process to indicate the end of the sentence speech recognition process. .

In this case, the operation of the touch panel described above is detected by the touch panel control unit 315 of FIG. 3, and then executed by the CPU 316 (FIG. 3) of the control unit 110. In the control operation corresponding to the flowchart, the determination is made by YES in Step 401 and NO in Steps 405 and 406, and the other key input processing of Step 409 is executed. Further, in the transmission processing of step 404, the determination of step 501 in FIG.
In step 503, the mobile terminal 10 becomes an ES, and if necessary, a calling process is executed in step 503.
The transmission instruction of the “terminal identification code” of No. 1 and the above-mentioned sentence speech recognition processing end request command is requested to the communication control unit 321 in the communication unit 111 of FIG.

As a result, the communication control unit 321 first
A TCP segment having a format shown in FIG. 6 (c) in which a "terminal identification code" for identifying the mobile terminal 101 and a command for ending the sentence / speech recognition process in the "data" field are generated. Generating an IP datagram having the format shown in FIG. 6 (b) with the TCP segment stored in the "data"field;
Furthermore, the IP datagram is "information"
A PPP frame having the format shown in FIG. 6A stored in the field is generated, and the T
Send a CP / IP packet. In this case, the TCP header (FIGS. 6 (c) and 7 (b)) and the IP header (FIG. 6 (b),
FIG. 7 (a)) and "protocol" field (FIG. 6 (a)
The information set in ()) is the same as the information set when the above-described real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing is transmitted.

As a result, the TCP / IP packet is transmitted via the Internet 105 in the same manner as the TCP / IP packet storing the real-time start request command for the sentence speech recognition processing. The packet is transferred to the packet transmitting / receiving unit 115 in the device 108.

The packet transmitting / receiving unit 115 transmits the transferred TCP / IP packet in exactly the same manner as when the TCP / IP packet storing the above-described real-time start request command for the sentence / voice recognition processing is transferred. It receives and notifies the mobile terminal communication control unit 116 (FIG. 1) of a reception notification.

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “terminal identification code” and the command to end the sentence / speech recognition process from the “data” field of the TCP segment constituting the received TCP / IP packet, and delivers the data to the mobile terminal communication control unit 116 .

As a result, the mobile terminal communication control unit 116
The sentence speech recognition process for the mobile terminal 101 is terminated as described later. The user specifies the non-real-time start request command of the sentence speech recognition process, so that the speech control host device 108 does not immediately return the recognized speech sentence data to the mobile terminal 101, but sequentially sends it. Can be accumulated. In this case, the user later operates the touch panel of the mobile terminal 101 to collectively return the recognized voice sentence data stored in the voice control host device 108 to the mobile terminal 101. Of the batch transfer request command.

In this case, the operation of the touch panel described above is detected by the touch panel control section 315 of FIG. 3, and then executed by the CPU 316 (FIG. 3) of the control section 110. In the control operation corresponding to the flowchart, the determination is made by YES in Step 401 and NO in Steps 405 and 406, and the other key input processing of Step 409 is executed. Further, in the transmission processing of step 404, the determination of step 501 in FIG.
In step 503, the mobile terminal 10 becomes an ES, and if necessary, a calling process is executed in step 503.
The transmission instruction of the “terminal identification code” of No. 1 and the above-mentioned batch transfer request command of the sentence / voice recognition result is requested to the communication control unit 321 in the communication unit 111 of FIG.

As a result, the communication control unit 321 first stores the “terminal identification code” for specifying the mobile terminal 101 in the “data” field and the batch transfer request command of the sentence / voice recognition result in FIG. 6 (c). Generating a TCP segment having the format shown in FIG. 6B with the TCP segment stored in the "data" field, and generating an IP datagram having the format shown in FIG.
Furthermore, the IP datagram is "information"
A PPP frame having the format shown in FIG. 6A stored in the field is generated, and the T
Send a CP / IP packet. In this case, the TCP header (FIGS. 6 (c) and 7 (b)) and the IP header (FIG. 6 (b),
FIG. 7 (a)) and "protocol" field (FIG. 6 (a)
The information set in ()) is the same as the information set when the above-described real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing is transmitted.

As a result, the TCP / IP packet is transmitted via the Internet 105 in the same manner as the TCP / IP packet in which the real-time start request command for the sentence speech recognition processing is stored. The packet is transferred to the packet transmitting / receiving unit 115 in the device 108.

The packet transmitting / receiving unit 115 transmits the transferred TCP / IP packet in exactly the same manner as when the TCP / IP packet storing the above-described real-time start request command for the sentence / voice recognition processing is transferred. It receives and notifies the mobile terminal communication control unit 116 (FIG. 1) of a reception notification.

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram constituting the received TCP / IP packet, and reads “terminal identification code” from the “data” field of the TCP segment constituting the TCP / IP packet. The command and the batch transfer request command of the voice recognition result are extracted, and the data is transferred to the mobile terminal communication control unit 116.

As a result, the mobile terminal communication control unit 116
As will be described later, the TCP / IP packet storing the recognized voice sentence data is transferred to the mobile terminal 101 in a lump.

The TCP / IP packet is transferred to the communication control unit 321 (FIG. 3) as described above,
The reception notification and the recognized voice sentence data are delivered from the communication control unit 321 to the CPU 316.

The CPU 316 processes the above-described reception notification and the recognized voice text data in the above-described step 402 in FIG. 4, and displays the recognized voice text data on the LCD display unit 311 (203 in FIG. 2). <Schematic Operation of Mobile Terminal Communication Control Unit 116 and Sentence Voice Recognition Unit 117> Next, the schematic operation of the mobile terminal communication control unit 116 and the sentence voice recognition unit 117 in the voice control host device 108 will be described.

The mobile terminal communication control unit 116 transmits the “terminal identification code” (the above-mentioned command) assigned to the mobile terminal 101 that has transmitted the real-time start request command for sentence speech recognition processing or the non-real-time start request command for sentence speech recognition processing. Is stored in the processing terminal registration table having the data structure shown in FIG. 13 and is recognized as a buffer file for receiving audio data (audio buffer file). A buffer file (sentence buffer file) for transmitting voice text data is transmitted to the voice control host device 10.
8 is created on the file system managed by. When the registration of the entry and the file is successful, the mobile terminal communication control unit 116 permits transmission to the mobile terminal 101 of the “source IP address” stored in the IP datagram that transferred the command. Reply the data.

The mobile terminal communication control unit 116 converts the voice data received from the mobile terminal 101 thereafter into the processing terminal corresponding to the “source IP address” (stored in the IP datagram that transferred the data). Write additionally to the audio buffer file specified from the entry in the registration table.

The sentence / speech recognition unit 117 performs a sentence / speech recognition process on each entry of the processing terminal registration table shown in FIG. 13 if the speech data is received in the speech buffer file specified from each entry. Then, the recognition voice text data obtained as a result is additionally written to the text buffer file corresponding to each of the above entries.

For each entry in the processing terminal registration table for which the real-time request is set, the mobile terminal communication control unit 116, if the recognized voice text data is obtained in the text buffer file specified from each entry, deletes it. A reply is sent to the mobile terminal 101 of the “source IP address” registered in each entry.

The mobile terminal communication control unit 116 receives the command for ending the sentence / voice recognition process and sets an entry in the processing terminal registration table in which the real-time request is set, or sets the real-time request and sets the last access time. For an entry in the processing terminal registration table that is a time earlier than the time before the current time by a fixed time, the content of the entry is deleted, and the voice / audio specified from the entry is deleted.
Delete the sentence buffer file.

For the entry in the processing terminal registration table for which no real-time request is set (that is, for which a non-real-time request is set), the mobile terminal communication control unit 116 specifies the contents of the entry and the voice / text specified from the entry. The buffer file is stored even after the communication with the corresponding mobile terminal 101 is completed, and later, when a batch transfer request command of the sentence speech recognition result is received from the mobile terminal 101, the “terminal identification code” of the mobile terminal 101 is stored. The recognized voice text data stored in the text buffer file specified from the entry of the processing terminal registration table corresponding to the above-mentioned mobile terminal of the "source IP address" stored in the IP datagram that transmitted the command. 101
Send to. <Detailed Operation of Mobile Terminal Communication Control Unit 116> FIGS. 9 to 12
Is a mobile terminal communication control unit 1 for realizing the above function.
16 is an operation flowchart illustrating a control operation performed by the control unit 16. This operation flowchart is based on the mobile terminal communication control unit 1.
A processor (not shown) for controlling the CPU 16 is realized as an operation for executing a control program (not shown).

First, in step 901, it is determined whether or not a reception notification has been received from the packet transmission / reception unit 115 (FIG. 1) in the voice control host device. As described above, the packet transmitting / receiving unit 115 communicates with the Internet 105
I that constitutes the TCP / IP packet transferred from
In the "destination IP address" field of the IP header of the P datagram, the IP address of the voice control host
By identifying that the address is set,
A 16-bit integer value for receiving the TCP / IP packet and specifying a communication protocol for text-to-speech recognition processing in a “destination port number” field and a “source port number” field of a TCP segment constituting the TCP / IP packet Is confirmed, the mobile terminal communication control unit 116 is notified of the reception notification. This reception notification includes a real-time start request command for sentence speech recognition processing, a non-real-time start request command for sentence speech recognition processing, speech data to be subjected to sentence speech recognition processing, a termination request command for sentence speech recognition processing, or a sentence speech recognition. This is a reception notification regarding any of the resultant batch transfer request commands.
If a reception notification is notified from the packet transmitting / receiving unit 115 and the determination in step 901 is YES, in step 902,
The data delivered together with the reception notification from the packet transmission / reception unit 115 is captured. In this case, if the reception notification is any one of a real-time start request command for sentence speech recognition processing, a non-real-time start request command for sentence speech recognition processing, or a batch transfer request command for sentence speech recognition results, , The “source IP address”, the “terminal identification code”, and the above command are fetched. When the reception notification is a reception notification of audio data, the “source IP address” and the audio data are captured. Further, when the reception notification is a reception notification of a termination request command for the sentence speech recognition process, the “terminal identification code” and the command are fetched.

After the processing of step 902, the judgments of step 903 in FIG. 9, step 909 in FIG. 10, step 911 in FIG. 10, or step 915 in FIG. 11 are sequentially examined, and any judgment result becomes YES. . That is, if the data transferred from the packet transmitting / receiving unit 115 in step 902 is related to a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing, the determination in step 903 becomes YES. Steps 904 to 908 are executed, and if it is related to audio data, Step 9 in FIG.
If the determination in step 09 is YES and step 910 is executed, and if it is related to the end request command of the sentence speech recognition process, the determination in step 911 in FIG. 10 is YES and steps 912 to 914 are executed and the sentence If the voice recognition result is related to the batch transfer request command, the determination in step 915 in FIG. 11 becomes YES, and steps 916 to 918 are executed.

If the reception notification has not been received from the packet transmission / reception unit 115 and the determination in step 901 is NO, or after the processing corresponding to the reception of each command or voice data described above, steps 919 and 920 in FIG. In step 921 and step 922, a process for terminating communication with the mobile terminal 101 whose last access time is a predetermined time or more is performed, and then the process shown in FIG. The process returns to the determination processing of step 901.

When the determination in step 901 is YES, and the data transferred from the packet transmitting / receiving unit 115 in step 902 relates to a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing. In step 903
Steps 904 to 9 to be executed when the judgment of the step is YES
The process 08 is described.

First, in step 904, step 90
In step 2, it is determined whether an entry corresponding to the “terminal identification code” passed from the packet transmitting / receiving unit 115 is registered in the processing terminal registration table.

When a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing is specified for the first time from the mobile terminal 101, this determination is NO.

As a result, in step 905, the voice control host device 108 manages the voice buffer file, which is a buffer file for receiving voice data, and the text buffer file, which is a buffer file for transmitting recognized voice text data. Is created on the file system.

Next, in step 906, one entry (one horizontal row) is stored in the processing terminal registration table having the data structure shown in FIG. 13 and stored in the memory (not shown) in the mobile terminal communication control unit 116. Data set) is secured. Then, in the entry, "terminal identification code"
, “Source IP address”, presence / absence of a real-time request, last access time, audio buffer file name, and text buffer file name are registered. The “terminal identification code” is data passed from the packet transmitting / receiving unit 115 in step 902,
This is stored in the "data" field of the TCP segment constituting the TCP / IP packet transferred from No. 01 (see FIG. 6 (c)). The “source IP address” is also data passed from the packet transmitting / receiving unit 115 in step 902,
This is stored in the IP header of the IP datagram constituting the TCP / IP packet transferred from No. 01 (see FIGS. 6 (b) and 7 (a)). The presence or absence of a real-time request is determined in step 902 by the packet
“Yes” if the command passed from No. 15 is a real-time start request command for sentence speech recognition processing
(Value "1"), and if the command is a non-real-time start request command for sentence speech recognition processing, it is set to "none" (value "0"). The current time is set as the last access time. The voice buffer file name and the sentence buffer file name are file names indicating the files created in step 905.

Next, as described above, when a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing is specified from the mobile terminal 101 for the first time, the processing terminal registration table is used. Since there is no entry to perform,
The determination of 4 is YES. Further, as described above, after the real-time start request command of the sentence speech recognition process is designated from the mobile terminal 101 and the execution of the sentence speech recognition process is started, the end request command of the sentence speech recognition process is sent from the mobile terminal 101. If received, or if the last access time is earlier than the time before the current time by a certain time (that is, there is no access for a certain time), the corresponding entry on the processing terminal registration table is deleted.

However, after the mobile terminal 101 specifies the real-time start request command of the sentence speech recognition process and starts the execution of the sentence speech recognition process, before the corresponding entry on the processing terminal registration table is deleted, it is re-entered. If a real-time start request command for sentence speech recognition processing or a non-real-time start request command for sentence speech recognition processing is specified from the same mobile terminal 101, the determination in step 904 is YES. In addition, after the sentence speech recognition process in which the non-real-time start request command of the sentence speech recognition process is specified from the mobile terminal 101 last time, the batch transfer request command of the sentence speech recognition result is still specified from the mobile terminal 101. Before the same mobile terminal 101
When the real-time start request command of the sentence speech recognition process or the non-real-time start request command of the sentence speech recognition process is specified, the determination in step 904 becomes YES (see step 917 in FIG. 11 described later).

In such a case, the same entry in the processing terminal registration table that has not been deleted is used. Then, in step 907, the “source IP address” stored in the above entry on the processing terminal registration table
Only the address “” and the presence / absence of a real-time request are updated to the new data passed from the packet transmitting / receiving unit 115 in step 902 of FIG. 9, and the last access time is updated to the current time.

After step 906 or step 907 described above, step 908 proceeds to step 90
In step 2, the transmission permission data is returned to the "source IP address" passed from the packet transmission / reception unit 115 and registered in the entry of the processing terminal registration table.

Specifically, mobile terminal communication control section 116
Requests the packet transmission / reception unit 115 (FIG. 1) to return transmission permission data to the “source IP address”. As a result, the packet transmitting / receiving unit 115 first
Generate a TCP segment having the format shown in (c). In this case, in the TCP header having the format shown in FIGS. 6C and 7B, the “source port number” field and the “destination port number” field include a communication protocol for the text / speech recognition process. Is set as a 16-bit integer. And T
Transmission permission data is stored in the “data” field of the CP segment.

Next, the packet transmitting / receiving unit 115 generates an IP datagram having the format shown in FIG. 6B in which the above-mentioned TCP segment is stored in the “data” field. In this case, in the IP header having the format shown in FIGS. 6 (b) and 7 (a),
In the “protocol” format, an integer value 6 defining the format of the TCP segment data stored in the “data” field is set. The “source IP address” field contains the voice control host device 1
08 is set. Further, in the “destination IP address” field, the “source IP address” passed from the packet transmission / reception unit 115 in step 902 of FIG. 9 is set.

Then, the packet transmitting / receiving unit 115 generates a frame in accordance with the protocol on the LAN 107 in which the above-described IP datagram is stored, and transmits the frame to the LAN 107
To send to. For example, if the LAN 107 is a local area network based on the Ethernet system, the frame is an Ethernet frame.

The above frame, IP datagram and TCP
A TCP / IP packet composed of a segment and
After being transferred to the mobile terminal control host device 104 via the router device 106 and the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the PHS network,
03 and the communication control unit 32 in the communication unit 111 of the mobile terminal 101 via the wireless base (or wired connection device) 102
1 (FIG. 3).

Thereafter, voice data is transferred from the mobile terminal 101 to the voice control host device 108 as described above. After the processing of step 908, FIG.
In steps 919 and 920 of step 2, transmission processing of the recognized speech text data is executed, and steps 921 and 92
After processing for ending communication with the mobile terminal 101 whose last access time is a fixed time or more in 2 is performed,
The process returns to the determination processing of step 901 in FIG. 9 again.

Next, the determination in step 901 in FIG.
In the case where S is S and the data delivered from the packet transmission / reception unit 115 in step 902 is audio data, the process of step 910 executed when the determination of step 909 in FIG.

That is, in step 910, an entry in the processing terminal registration table (FIG. 13) in which the same “source IP address” passed from the packet transmitting / receiving section 115 in step 902 in FIG. 9 is stored is searched. The audio buffer file corresponding to the audio buffer file name stored in the corresponding entry (step 90 in FIG. 9)
6), the audio data transferred from the packet transmitting / receiving unit 115 in step 902 of FIG. 9 is additionally written. The size of the audio buffer file at the time of additional writing is automatically adjusted by the file system managed by the audio control host device 108.

At step 910, the last access time stored in the relevant entry is updated to the current time. In this way, the mobile terminal communication control unit 116 sends the sentence voice recognition unit 11 via the voice buffer file for each mobile terminal 101 (for each “terminal identification code”).
7 (FIG. 1) is delivered. That is, as described later, the sentence speech recognition unit 117 performs, for each entry in the processing terminal registration table shown in FIG. The voice recognition processing is executed, and the resulting recognized voice text data is additionally written to the text buffer file corresponding to each of the entries.

After the processing of step 910, the transmission processing of the recognized speech text data is executed in steps 919 and 920 of FIG. 12, and in subsequent steps 921 and 922, the mobile terminal 101 whose last access time is a predetermined time or more ago.
After the processing for terminating the communication with is performed, the flow returns to the determination processing of step 901 in FIG. 9 again.

Next, the determination in step 901 in FIG.
If it is S and the data transferred from the packet transmitting / receiving unit 115 in step 902 is related to a command for requesting termination of the sentence speech recognition process, the determination in step 911 in FIG.
The processing of 12 to 914 will be described.

First, in step 912, the real-time request is entered in the entry of the processing terminal registration table (FIG. 13) storing the same “terminal identification code” passed from the packet transmitting / receiving section 115 in step 902 of FIG. It is determined whether or not “present” (value “1”) is stored.

[0209] In the above entry, "present" (value "1") is not stored as a real-time request, that is, "absent" (value "0") is stored as a real-time request. If the non-real-time start request command of the recognition process is instructed,
After receiving the command for requesting termination of the sentence speech recognition process, the contents of the entry in the processing terminal registration table, the sentence buffer file, and the speech buffer file (the contents of which become empty when the sentence speech recognition process ends) are retained. Therefore, the determination in step 912 is NO and step 91
Steps 3 and 914 are not executed, and step 9 in FIG.
At 19 and 920, transmission processing of the recognized voice sentence data is performed, and at subsequent steps 921 and 922, processing for terminating communication with the mobile terminal 101 whose last access time is a predetermined time or more is performed. The process returns to the determination processing of step 901 in FIG. 9 again.

On the other hand, if “present” (value “1”) is stored as a real-time request in the entry, that is, if a real-time start request command of the sentence speech recognition processing is first instructed from the mobile terminal 101, Step 91
The determination of 2 is YES.

As a result, first, in step 913, FIG.
In step 902, all the contents of the entry of the processing terminal registration table (FIG. 13) storing the same "terminal identification code" passed from the packet transmitting / receiving unit 115 are deleted.

At step 914, the voice buffer file corresponding to the voice buffer file name and the text buffer file corresponding to the text buffer file name stored in the entry are stored in the voice control host device 108.
Is deleted from the file system managed by.

After the processing of step 914, the transmission processing of the recognized speech text data is executed in steps 919 and 920 of FIG. 12, and in subsequent steps 921 and 922, the mobile terminal 101 whose final access time is a predetermined time or more ago.
After the processing for terminating the communication with is performed, the flow returns to the determination processing of step 901 in FIG. 9 again.

Subsequently, the determination in step 901 in FIG.
ES, and the packet transmitting / receiving unit 115
Steps 916 to 918 executed when the determination result of step 915 in FIG. 11 is YES when the data transferred from is related to the batch transfer request command of the sentence speech recognition result will be described.

First, in step 916, a sentence obtained from an entry in the processing terminal registration table (FIG. 13) in which the same “terminal identification code” passed from the packet transmitting / receiving unit 115 in step 902 in FIG. 9 is stored. From the sentence buffer file corresponding to the buffer file name,
Recognized voice sentence data is read out in a lump, stored in one or more TCP / IP packets, and transmitted to the “source IP address” passed from the packet transmitting / receiving unit 115 in step 902 of FIG. Is done.

More specifically, mobile terminal communication control section 116
Requests the packet transmitting / receiving unit 115 to transmit the recognized voice text data to the “source IP address”. As a result, the packet transmitting / receiving unit 115 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 6C and 7B, the “source port number” field and the “destination port number” field include a communication protocol for the text / speech recognition process. Is set as a 16-bit integer. The "data" field of the TCP segment stores the recognized voice sentence data.

Next, the packet transmission / reception unit 115 generates an IP datagram having the format shown in FIG. 6B in which the above-mentioned TCP segment is stored in the “data” field. In this case, in the IP header having the format shown in FIGS. 6 (b) and 7 (a),
In the “protocol” format, an integer value 6 defining the format of the TCP segment data stored in the “data” field is set. The “source IP address” field contains the voice control host device 1
08 is set. Further, in the “destination IP address” field, the “source IP address” passed from the packet transmission / reception unit 115 in step 902 of FIG. 9 is set.

Then, the packet transmitting / receiving unit 115 generates a frame according to the protocol on the LAN 107 in which the above-described IP datagram is stored, and transmits the frame to the LAN 107
To send to.

The above frame, IP datagram and TCP
A TCP / IP packet composed of a segment and
After being transferred to the mobile terminal control host device 104 via the router device 106 and the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the PHS network,
03 and the communication control unit 32 in the communication unit 111 of the mobile terminal 101 via the wireless base (or wired connection device) 102
1 (FIG. 3), and as described above, the mobile terminal 1
01 is displayed on the LCD display unit 311 (203 in FIG. 2).

Next, in step 917, the contents of the entry of the processing terminal registration table (FIG. 13) storing the same “terminal identification code” passed from the packet transmitting / receiving section 115 in step 902 in FIG. All are deleted.

At step 918, the voice buffer file corresponding to the voice buffer file name and the text buffer file corresponding to the text buffer file name stored in the entry are stored in the voice control host device 108.
Is deleted from the file system managed by.

After the processing of step 918, the transmission processing of the recognized voice text data is executed in steps 919 and 920 of FIG. 12, and in subsequent steps 921 and 922, the mobile terminal 101 whose last access time is a predetermined time or more ago.
After the processing for terminating the communication with is performed, the flow returns to the determination processing of step 901 in FIG. 9 again.

When the reception notification is not notified from the packet transmitting / receiving unit 115 and the determination in step 901 in FIG. 9 is NO, or after the processing corresponding to the reception of each command or voice data described above, FIG. Processing of steps 919 and 920, and subsequent steps 921 and 922
Will be described.

In these processes, the sentence speech recognition section 11
The transmission processing of the recognized voice sentence data obtained from the step 7 is executed. First, in step 919, in the processing terminal registration table (FIG. 13), "present" (value "1") is stored as a real-time request, and the recognized voice text data is stored in the text buffer file corresponding to the text buffer file name. It is determined whether or not there is an entry in which.

There is no such entry and step 919
Is NO, the transmission process of the recognized voice text data in step 920 is not executed, and the
And 922.

If there is one or more entries as described above and the determination in step 919 is YES, step 92
0, for each applicable entry, the recognized speech text data in the text buffer file corresponding to the text buffer file name stored in that entry is directed toward the “source IP address” stored in that entry. The sent recognized speech sentence data is deleted from the sentence buffer file. The size of the sentence buffer file at the time of deletion is automatically adjusted by the file system managed by the voice control host device 108.

After the processing in step 920 or when the determination in step 919 is NO, step 921
Is executed. Here, the processing terminal registration table (FIG. 1)
Among the entries of 3), an entry in which “present” (value “1”) is set as the real-time request and the last access time is a time earlier than the current time by a fixed time is detected. , The contents of the entry are all deleted.

At step 922, the voice buffer file corresponding to the voice buffer file name and the text buffer file corresponding to the text buffer file name stored in the entry are stored in the voice control host device 108.
Is deleted from the file system managed by.

After the process of step 922, the process returns to the determination process of step 901 in FIG. <Detailed Operation of Sentence Speech Recognition Unit 117> FIG. 14 is a functional block diagram of the sentence speech recognition unit 117.

As described above, this sentence speech recognition unit 117 performs, for each entry in the processing terminal registration table shown in FIG. 13, if speech data is received in a speech buffer file specified from each entry, Then, the sentence speech recognition processing is executed, and the resulting recognized speech sentence data is additionally written into the sentence buffer file corresponding to each entry.

The reading of the voice data from the voice buffer file for each entry and the writing of the recognized voice text data to the text buffer file are controlled by the input / output control unit 1409 in FIG. First, the input / output control unit 1409
Will be described. FIG. 15 shows the input / output control unit 1.
409 is an operation flowchart illustrating a control operation performed by the control unit. This operation flowchart is based on the input / output control unit 140
9 is realized as an operation of executing a control program (not shown).

First, in step 1501, it is determined whether or not an entry in which audio data is stored in the audio buffer file corresponding to the audio buffer file name exists in the processing terminal registration table (FIG. 13).

If such an entry exists and step 15
If the determination of 01 is YES, in step 1502, for each entry, the “terminal identification code” stored in the entry and the audio buffer file name corresponding to the audio buffer file name stored in the entry Of the input buffer queue 140 shown in FIG.
1 and the audio data is deleted from the audio buffer file.

The input buffer queue 1401 stores the audio data queued in the input buffer queue 1401,
02. Voice section detector 14
02, the speech analysis unit 1403, the phoneme recognition unit 1404, the word recognition unit 1406, and the sentence recognition unit 1
Reference numeral 407 forms a data processing pipeline, and has a function of processing input data independently of each other. Each of the parts 1402 to 1407 can recognize the “terminal identification code” (input from the input buffer queue 1401) corresponding to the audio data currently being processed. Therefore, finally, a set of the “terminal identification code” and the recognized voice text data is output from the text recognition unit 1407 to the output buffer queue 1408.

After the processing in step 1502 or when the determination in step 1501 is NO, step 1503
Then, it is determined whether or not a set of “terminal identification code” and recognized speech text data has been obtained in the output buffer queue 1408 in FIG.

When such a set is obtained, step 15
If the determination in step 03 is YES, in step 1504, for each set in the output buffer queue 1408, for the entry in the processing terminal registration table corresponding to the "terminal identification code" of that set, the text buffer stored in that entry The set of recognized speech text data in the output buffer queue 1408 is additionally written to the text buffer file corresponding to the file name.

After the processing in step 1504 or when the determination in step 1503 is NO, step 150 is executed again.
1 is performed. As described above, the sentence / speech recognition unit 117 can efficiently execute the sentence / speech recognition process on the speech data requested by the plurality of mobile terminals 101 in a streamlined manner.

Next, 1 for realizing the sentence speech recognition processing is described.
The function of each part of 402 to 1407 will be described below. Each of the methods described below can be realized by referring to, for example, the document “Electronic / Information Engineering Introduction Series 2, Sound and Speech Engineering” (Furui, Modern Science Co., Chapter 14).

The voice section detection section 1402 detects a section in which voice exists in the sample time series of the voice data input from the input buffer queue 1401. More specifically, the voice section detection unit 1402 calculates an average power (power) of predetermined samples (for example, 32 to 256 samples for 8 kHz sampling data),
A section in which the state in which the average power exceeds a predetermined threshold continues for a predetermined number of times or more is detected as a voice section. Thereby, it is possible to prevent a sentence voice from being erroneously recognized in a section where no voice exists.

[0240] The voice analysis section 1403 includes the voice section detection section 1
A feature amount parameter vector is detected by performing a feature analysis on the audio data output from 402. Any of the following well-known analysis methods can be adopted as the voice analysis method. (1) A method of smoothing each output of a band filter bank to which a time series of audio data is input, and using each smoothed output as an element of a feature parameter vector. (2) Smoothing each short-time spectrum component calculated by Fast Fourier Transform (FFT) which receives an audio data time series of a predetermined number of continuous samples as input, and converts each smoothed component value into a feature amount parameter vector. Element method. (3) A method in which a cepstrum coefficient group is calculated by cepstrum analysis using a time series of audio data for each successive predetermined sample as an input, and these are used as elements of a feature parameter vector. (4) A method of calculating a Δ (delta) cepstrum (differential coefficient of a cepstrum) group for the cepstrum coefficient group in addition to the cepstrum coefficient group of the above (3), and adding them to the element of the feature amount parameter vector. (5) LPC analysis is performed by linear prediction analysis (LPC analysis, more specifically, line spectrum pair analysis: LSP analysis) that receives a time series of audio data for each successive predetermined sample as input.
(LSP) A method of calculating coefficient groups and using them as elements of a feature parameter vector. (6) An autocorrelation function is calculated by an autocorrelation analysis using a time series of voice data of each successive predetermined sample as an input, and a pitch fundamental frequency pattern of voice detected based on the autocorrelation function is calculated as one element of a feature parameter vector. Method to add to Next, the phoneme recognition unit 1404 stores the feature parameter vector output from the speech analysis unit 1403 at every predetermined frame period (predetermined sample) and the phoneme standard pattern dictionary 140
5. The similarity (distance) of the feature parameter vector of each phoneme stored in No. 5 with the standard pattern is calculated, and as a result, a set of phonemes with high similarity obtained at every predetermined frame period is determined along with the phonetic lattice along with the similarity. Output as data.
The phoneme recognizing unit 1404 does not determine the final phoneme at every predetermined frame period in order to avoid occurrence of a phoneme recognition error, and outputs the result data in the form of phoneme lattice data in which phoneme candidates are listed. Is output.

The word recognizing unit 1406 receives the phoneme lattice data output from the phoneme recognizing unit 1404 at every predetermined frame period as input, and outputs word lattice data in a table of word candidates at every predetermined frame period. As the word recognition method, any of the following well-known analysis methods can be adopted. (1) The word recognizing unit 1406 performs time normalization on the time series of phoneme lattice data output from the phoneme recognizing unit 1404 over a plurality of frame periods and the all-phoneme standard pattern sequence stored in the word dictionary ( DP matching or
DTW (Dynamic Time Warping) is executed to output word lattice data. Also in this case, the word recognition unit 1406
Outputs the result data in the form of word lattice data in which word candidates are tabulated without determining a final word at every predetermined frame period in order to avoid utterance of a word recognition error. (2) The word recognition unit 1406 uses HMM (Hidden Markov)
Model), all the words are modeled, and the phoneme recognition unit 1
The time series of phoneme lattice data spanning a plurality of frame periods output from 404 is input to the HMM analysis unit, and each word corresponding to a plurality of models from the one with the highest probability of occurrence is regarded as word lattice data as a word candidate. Output. Finally, the sentence recognizing unit 1407 sequentially inputs the word lattice data output from the word recognizing unit 1406 as a first-stage processing, and generates a grammar (phrase order rule) in a phrase related to a phrase structure of Japanese (or English). , The possibility of various phrases is calculated as phrase lattice data. Then, the sentence recognizing unit 1407 analyzes the semantic dependency between the phrases according to the inter-phrase grammar as the second stage processing,
The recognition voice sentence data is determined, and it is written in the output buffer queue 1408 in combination with the “terminal identification code” sequentially transmitted from the input buffer queue 1401. <Other Embodiments> In the embodiment described above, mobile terminal 101 is a PHS terminal, and mobile terminal 101 and voice control host device 108 are connected via PHS network 103 and Internet 105. I have. However, the present invention is not limited to this, and the present invention may be applied to any form connected to the voice control host device 108 indirectly or directly by wireless or wired. Can be applied.

[0242]

According to the present invention, it is possible for a mobile terminal to provide a speech recognition function having practical accuracy at a low cost without having to provide a sophisticated speech recognition environment.

In addition, according to the present invention, by using a personal handyphone system communication network and the Internet, which are currently spreading nationwide and worldwide,
It is possible to provide a voice recognition function having practical accuracy at lower cost and easily, and to seamlessly combine the function provided by the present invention with the personal handyphone system call function and the Internet access function. Becomes possible.

Further, according to the present invention, the mobile terminal and the voice control host device can be easily specified worldwide, and
It is possible to easily realize coexistence of the voice recognition processing service and another information processing service.

In addition, according to the present invention, it is possible to easily realize load distribution on the host device side.

[Brief description of the drawings]

FIG. 1 is an overall system configuration diagram.

FIG. 2 is an external view of a mobile terminal.

FIG. 3 is a functional block diagram of a mobile terminal.

FIG. 4 is an overall operation flowchart of processing of a mobile terminal.

FIG. 5 is an operation flowchart of a transmission process.

FIG. 6 is a format diagram of communication data.

FIG. 7 is a format diagram of an IP header and a TCP header.

FIG. 8 is an operation flowchart of a calling process using PPP.

FIG. 9 is an operation flowchart (part 1) of a mobile terminal communication control unit.

FIG. 10 is an operation flowchart (part 2) of the mobile terminal communication control unit.

FIG. 11 is an operation flowchart (part 3) of the mobile terminal communication control unit.

FIG. 12 is an operation flowchart (part 4) of the mobile terminal communication control unit.

FIG. 13 is a data configuration diagram of a processing terminal registration table.

FIG. 14 is a configuration diagram of a sentence speech recognition unit.

FIG. 15 is an operation flowchart of the input / output control unit.

[Explanation of symbols]

 Reference Signs List 101 mobile terminal 102 wireless base (wired connection device) 103 PHS network (public telephone network, ISDN network) 104 mobile terminal control host device 105 internet 106 router device 107 LAN (local area network) 108 voice control host device 109 input unit 110 control Unit 111 communication unit 112 output unit 113 connection establishment unit 114 routing unit 115 packet transmission / reception unit 116 mobile terminal communication control unit 117 sentence speech recognition unit 201, 301 microphone 202, 304 camera (CCD camera) 203, 311 LCD display unit 204, 308 Speakers 205, 323 Wireless antennas 206, 325 Socket (for communication) 207 IC card slot 208 Optical transceiver (for optical communication) 302, 305 A / D converter 303 Microphone controller 306, 3 3 Memory 307 Camera control unit 309 D / A conversion unit 310 Speaker control unit 312 LCD driver 314 LCD control unit 315 Touch panel control unit 316 CPU 317 RAM 318 ROM 319 IC card interface unit 320 IC card 321 Communication control unit 322 Wireless driver 324 Wired Driver 1401 Input buffer queue 1402 Voice section detector 1403 Voice analyzer 1404 Phoneme recognizer 1405 Phoneme standard pattern dictionary 1406 Word recognizer 1407 Text recognizer 1408 Output buffer queue 1409 Input / output controller

Claims (7)

[Claims] 1. A communication system in which a mobile terminal communicates with a host device, wherein the mobile terminal indirectly or via a relay network including one or both of a wireless network and a wired network in the mobile terminal. A host connecting means for directly connecting to the voice control host device, which is the host device, without going through a relay network; a voice input means for inputting voice; and a connection operation by the host connecting means, after the voice input means, Voice data transmitting means for transmitting input voice data to the voice control host device; recognition voice data receiving means for receiving recognition voice data returned from the voice control host device; And a recognition voice data display / edit means for displaying or editing, wherein the voice control host device is provided with a connection operation by a host connection means in the mobile terminal. Mobile terminal connecting means for identifying and connecting the mobile terminal, voice data receiving means for receiving the voice data for each currently connected mobile terminal, and for each currently connected mobile terminal, Voice recognition means for performing voice recognition processing on voice data received by the voice data receiving means; and for each currently connected mobile terminal, recognition voice data obtained by voice recognition processing by the voice recognition means. ,
A mobile terminal voice recognition communication system, comprising: recognition voice data return means for returning a response to a mobile terminal corresponding thereto. 2. The mobile terminal used in a communication system in which the mobile terminal communicates with a host device, wherein the mobile terminal is indirectly or via a relay network including one or both of a wireless network and a wired network. Host connection means for directly connecting to the audio control host device which is the host device without passing through the relay network; voice input means for inputting voice; and after the connection operation by the host connection means, the voice input means Voice data transmitting means for transmitting voice data input from the voice control host device to the voice control host device; recognition voice data receiving means for receiving recognition voice data returned from the voice control host device; and the received recognized voice data. And a recognition voice data displaying / editing means for displaying or editing the data. 3. The host device used in a communication system in which a mobile terminal communicates with a host device, wherein the host device is indirectly or via a relay network including one or both of a wireless network and a wired network. Mobile terminal connection means for identifying and connecting to the mobile terminal in response to a connection operation performed by the mobile terminal directly without passing through the relay network; and voice data for each currently connected mobile terminal. Voice data receiving means for receiving voice data, voice recognition means for performing voice recognition processing on voice data received by the voice data receiving means, for each currently connected mobile terminal, For each terminal, recognition voice data obtained by voice recognition processing by the voice recognition unit,
A voice control host device, comprising: recognition voice data return means for returning a response to a mobile terminal corresponding thereto. 4. The host connection means in the mobile terminal selectively selects a real-time reply request command or a non-real-time reply request command for causing the voice control host device to return recognized voice data in real time or non-real time. And a function of transmitting a batch transfer request command for collectively transferring the recognition voice data, wherein the recognition voice data returning means is configured to perform, for each mobile terminal currently connected, the mobile terminal When the connection means has received the real-time reply request command from the mobile terminal, the corresponding voice data obtained by the voice recognition processing by the voice recognition means is immediately returned to the corresponding mobile terminal in response thereto. Receiving the non-real-time reply request command from the mobile terminal by the mobile terminal connection means. If the mobile terminal connection unit receives the batch transfer request command from the mobile terminal after holding the recognized voice data obtained by the voice recognition process by the voice recognition unit, The mobile terminal voice recognition communication system according to any one of claims 1 to 3, wherein the stored recognition voice data corresponding thereto is collectively returned to the corresponding mobile terminal. , Mobile terminal, or voice control host device. 5. The mobile terminal has a personal handyphone system communication function; the relay network includes a personal handyphone system communication network and the Internet; the voice control host device connects to the Internet; The host connection means in the terminal transmits and connects to the mobile terminal control host device having a gateway function between the Internet and a public network including the personal handyphone system communication network via the personal handyphone system communication network. 5. The mobile terminal control host device connects to the voice control host device via the Internet by using the communication protocol on the Internet. The mobile terminal speech recognition communication system according to the paragraph, mobile End, or voice control host device. 6. The communication protocol is a hierarchical protocol including an Internet protocol layer and a transmission control protocol layer, and includes a header field of an Internet Protocol datagram, which is packet data of the Internet protocol layer transmitted on the Internet. Stores a source Internet protocol address and a destination Internet protocol address designating addresses of the mobile terminal and the voice control host device on the Internet, and the transmission control protocol is stored in a data field of the Internet protocol datagram. A transmission control protocol segment, which is packet data of a layer, is stored. In the header field of the control protocol segment, a source port number and a destination port number for specifying a communication protocol for the voice recognition processing are stored, and in the data field of the transmission control protocol segment, the mobile terminal is identified. The terminal identification code, the real-time reply request command, the non-real-time reply request command, the batch transfer request command, the voice data, or the recognized voice data are stored. Mobile terminal voice recognition communication system, mobile terminal, or voice control host device. 7. The voice control host device is mutually connected by a network, and disperses functions corresponding to the mobile terminal connection means, the voice data receiving means, the voice recognition means, and the recognized voice data return means. The mobile terminal voice recognition communication system, the mobile terminal, or the voice control host device according to any one of claims 1 to 6, comprising a plurality of host computers realized by: