JP3864197B2 - Voice client terminal - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a voice client terminal , and more particularly to a client / server configuration system including a computer and a network, particularly a server of a World Wide Web (hereinafter simply referred to as WWW) system on the Internet, from a microphone of a client terminal. The present invention relates to a voice client terminal that inputs by voice and outputs information stored in a server by voice.
[0002]
[Prior art]
As is well known, when the hardware and software of the server and client are properly configured on the network as a WWW system, it is stored in the server by using a browser such as Netscape Navigator installed on the client terminal. It is possible to display the information on the text and image displayed on the client screen for browsing.
[0003]
In this system, when specific information on the screen is selected with a mouse or the like, it is associated with this information (hereinafter, this is called a link, and the previous specific information is called a link item). It can be displayed on the screen and viewed.
These services are premised on enjoying information visually, and cannot be enjoyed without looking at the screen, or cannot be enjoyed at all by visually impaired persons. As a method for solving this, by using recent speech recognition technology and speech synthesis technology, it is possible to input speech from a microphone and output by speech synthesis. For example, if [Prime official residence] is input by voice, the information of [Prime official residence] can be accessed, and the text portion can be output as synthesized sound from the speaker of the client terminal.
[0004]
[Problems to be solved by the invention]
However, in the above-described conventional method, there are cases where there are 10 to 20 long sentences and link parts everywhere in the well-known WWW information. Of course, color image information is mixed in the text. The reality is that links to videos and information that appeals to the eye are used abundantly. There is a problem of how to output such information to the visually impaired.
[0005]
The present invention has been made in view of the above points, and an object of the present invention is to provide a voice client terminal capable of acquiring WWW information even for a visually impaired person.
[0006]
[Means for Solving the Problems]
FIG. 1 is a principle configuration diagram of the present invention.
The present invention (claim 1), the HTML file stored in the servers acquired via the Internet, an audio client terminal 200 for outputting a voice,
Voice input means 201 for inputting a user's voice request;
The input user's voice is transmitted to the voice recognition server, the voice recognition result is received from the voice recognition server, the URL is extracted from the received voice recognition result, and the HTML file is requested to the proxy server based on the URL. Request issuing means 202;
Means for analyzing the HTML file obtained from the proxy server and classifying it into a title, a link item, and a body;
Means for transmitting the classified link items to the speech recognition server;
Means for transmitting displayed text information including a title, a link item, and a body to a speech synthesis server, and receiving speech data synthesized from the speech synthesis server;
Using the received audio data, audio output means 203 that reads out the title, the link item, and the text in order and outputs the audio.
[0009]
As above SL, in the present invention, that is converted into voice information from the visual information HTML (Hyper Text Markup Langu a ge ) format files that are published on the Internet through a commercial Web browser, provides a user It is a system that made it possible. In addition, when information is acquired on the client side, it is possible to operate a visually impaired person by using sound.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 2 shows the configuration of a system to which the present invention is applied.
In the system shown in FIG. 2, each processing engine is arranged on a high-speed network, and load distribution is performed so that a high-speed response can be realized by the client terminal 10. The system in the figure is roughly divided into two systems.
[0011]
First, a processing unit as a front end composed of the workstations 20, 30, and 40 in FIG. The workstations 20 and 30 are systems for providing a translation service. The workstation 40 is a function that is commonly used on the Internet in general, and is mainly used for data caching and kanji code conversion in this system.
[0012]
Second, workstations 50 and 60 used as back ends. The workstation 50 compares the voice data input from the user to the client terminal 10 with a candidate list (link item list), and selects an appropriate item. In addition, the morphological analysis process is performed on the link item in order to eliminate the necessity that the voice data input by the user and the link item are exactly the same. By doing so, an appropriate link item is inferred and selected from the user simply by inputting a fragment of the link item. The workstation 60 is called a speech synthesis engine that receives the text information extracted by the client terminal 10 (with a language type parameter) and generates speech data.
[0013]
Next, the HTML file request transmitted from the client terminal 10 is transferred to the external Internet 70 using the proxy server of the workstation 40 via the proxy server while operating on the workstation 20. Response data (HTML file) returned from the Internet 70 is sent to each translation engine by the proxy server of the workstation 20 via the proxy server of the workstation 40 (only request, The processing result is transmitted to the client terminal 10 at any time as requested by the user (transferred to the client terminal 10). The HTML file that has arrived at the client terminal 10 is analyzed, text information to be displayed is transmitted to the speech synthesis engine of the workstation 60, converted into speech data, and output at the client terminal 10. The list of link items necessary for speech recognition is subjected to morpheme analysis processing by the morpheme analysis engine of the workstation 50, passed to the speech recognition engine, and waits for transmission of speech data from the client terminal 10.
[0014]
Next, the voice data input (instructed) at the client terminal 10 is transmitted to the voice recognition engine 50, and the voice recognition engine 50 is compared and collated with the link item (text information) input before, and an appropriate result is obtained. . The obtained result is transmitted to the client terminal 10. As a result, the client terminal 10 extracts the URL from the link item, and acquires the following information to the Internet 70 via the proxy servers 20 and 40.
[0015]
【Example】
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 3 shows the system configuration of an embodiment of the present invention. In the system shown in the figure, the same components as those in FIG.
The system shown in FIG. 1 includes a client terminal 10 and workstations 20, 30, 40, and 50.
[0016]
The client terminal 10 includes a web browser 11, a voice input button monitoring program 12, a voice browser client 13, a voice input start button 15 connected to the voice input button monitoring program 12, and a speaker 16 connected to the voice browser client 13. And a microphone 17.
The workstation 20 has a function of the proxy server 21 and an English-Japanese translation engine 22 that translates text from English to Japanese.
[0017]
The workstation 30 has a Japanese-English translation engine 31 that translates text from Japanese to English.
The workstation 40 has a function of a proxy server 41.
The workstation 50 includes a voice recognition I / F program 51, a voice recognition engine 52, and a morphological analysis engine 53.
[0018]
The workstation 60 has a speech synthesis engine 61.
The Web browser 11 is commonly used and includes, for example, Netscape Navigator. In this embodiment, description will be made using the browser. The Web browser 11 acquires necessary information as a window to the Internet 70 and passes it to the voice browser client 13. It also displays on a web browser. The voice browser client 13 analyzes the information obtained from the Web browser 11, converts the text information to be read aloud to the speech synthesis engine 61 of the workstation 60, and the link item information to be recognized by the voice recognition I of the workstation 50. / F program 51 respectively.
[0019]
The client terminal 10 reproduces the voice data received from the voice synthesis engine 61 through the speaker 16 while recording the voice data on the local disk. Further, an input from the user is notified from the voice input button monitoring program 12 to the voice browser client 13 by pressing the voice input start button 15. Upon receiving the notification, the voice browser client 13 starts recording voice from the microphone 17. The voice browser client 13 stops the recording by releasing the user's voice input start button 15 and transmits the voice data recorded in the voice recognition I / F program 51 to the voice synthesis engine 61 of the workstation 60.
[0020]
The proxy server 21 of the workstation 20 connected to the client terminal 10 transmits information (HTML file) to be transferred from the Internet 70 to each translation engine (English-Japanese translation engine 22, Japanese-English translation engine 31), and translation processing is performed. To do. The translation result is recorded in the storage device of each translation engine 22, 31, and the translation result is transmitted to the client terminal 10 when a translation request is received from the user.
[0021]
The proxy server 41 of the workstation 40 has functions such as partial conversion of information from the Internet 70 (kanji code and the like) and temporary caching of information.
The speech recognition I / F program 51 of the workstation 50 inputs the link item transmitted from the speech browser client 13 to the morphological analysis engine 53 and outputs an appropriate result based on the part of speech of the decomposed character string as the output result. Perform reconfiguration processing. The result is registered in the voice recognition engine 52, the recorded voice data transmitted from the voice browser client 13 is passed to the voice recognition engine 52, and the result collated in the voice recognition engine 52 is returned to the voice browser client 13. .
[0022]
The speech synthesis engine 61 of the workstation 60 receives the text information to be read aloud extracted by the speech browser client 13, generates speech data, and returns it to the speech browser client 13.
FIG. 4 is a sequence chart of the operation of one embodiment of the present invention.
First, when the user presses the voice input start button 15 (step 101), the notification is transferred from the voice input button monitoring program 12 to the voice browser client 13. The voice browser client 13 starts voice recording from the microphone 17 and waits for the end. When the user releases the button (step 102), the voice input button monitoring program 12 reports the end notification to the voice browser client 13 again. Upon receiving the report, the voice browser client 13 stops the recording and transmits the recorded voice data to the voice recognition I / F program 51 of the workstation 50 (step 103).
[0023]
The voice recognition I / F program 51 of the workstation 50 receives the data, transfers the data to the voice recognition engine 52 (step 104), and requests a recognition process. When the speech popularity I / F program 51 acquires the result of recognition processing by the speech recognition engine 52 (step 105), the recognition result is transferred to the speech browser client 13 (step 106). The voice browser client 13 obtains the URL from the resulting link item, and issues a request transmission instruction to the Web browser 11 (step 107).
[0024]
The Web browser 11 transmits a data request message to the designated URL via the proxy servers 21 and 41 to the Internet 70 (step 108).
When the proxy server 21 acquires a response corresponding to the request from the Internet 70 via the proxy server 41 (step 109), the response is transmitted to the English-Japanese translation engine 22 or the Japanese-English translation engine 31 (step 110). ).
[0025]
Either the English-Japanese translation engine 22 or the Japanese-English translation engine 31 performs processing instructed by the proxy server 21 and returns the result to the proxy server 21. Thereby, the proxy server 21 transmits the response data to the Web browser 11 of the client terminal 10 (step 111).
When the web browser 11 of the client terminal 10 receives the response data, it is passed to the voice browser client 13 (step 112). At this time, if the page is multi-frame, the above processing is repeated from the Web browser 11 for the configuration view. The voice browser client 13 analyzes the obtained response data, obtains link items and text information to be displayed, and the like, and respectively obtains the speech synthesis engine 61 of the workstation 60 and the voice recognition I / F program 51 of the workstation 50. (Step 113). At this time, if there is text information such as an explanatory text added to the image (picture) information by performing HTML analysis, the information is also processed appropriately and transmitted to the speech synthesis engine 61, so that the user can understand the details of the image. Is read aloud from the speaker 16. Further, when transmitting to the speech synthesis engine 61, it is possible to provide the user with an appropriate service (such as rewinding / fast-forwarding in units of one sentence) by transmitting in units of sentences for each language. The speech recognition I / F program 51 transmits dynamically changing recognition candidate list information such as link items (step 114). Note that fixed commands are not transmitted every time.
[0026]
In the voice recognition I / F program 51, the received link item is subjected to a morphological analysis process once by the morphological analysis engine 53 (steps 115 and 116) and registered with the fixed command in the voice recognition engine 52 (step 117). Wait for voice data. Note that the registration is necessary for use in comparison with the voice data transmitted from the user.
[0027]
The speech synthesis engine 60 synthesizes speech from the user and transmits the synthesized speech data to the speech browser client 13 (step 118).
Next, a specific example will be described.
FIG. 5 is a diagram illustrating a user interface according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating an example of reading aloud by a voice browser according to an embodiment of the present invention.
[0028]
FIG. 5 shows a user interface of the Web browser 11. On the page, a title 110, a link item 130, and a body 120 are displayed as text information. These pieces of information are obtained from a file written in a grammar according to HTML, and the file is analyzed on the voice browser client 13 side and classified into items such as a title, a link item, and a text. Then, the information to be output is read aloud as shown in FIG. Provide users with detailed information by adding appropriate guidance.
[0029]
Further, when the link item is instructed at the time of input, it is possible to access by inputting only the words that remain in the impression without inputting the full text of the link item to the voice browser client 13. Regarding the mechanism, first, the link item full text is input to the morphological analysis engine 53, and the combination is performed again from the result of the part of speech decomposition. By reconstructing a compound word or the like from a word that is the smallest element, it is possible to cope with an input (from a word to a compound word) from a user.
[0030]
FIG. 7 shows a homepage type displayed by the multi-frame configuration of the Web browser according to the embodiment of the present invention. FIG. 8 shows an example of reading aloud by the voice browser according to the embodiment of the present invention. In this case, the voice browser client 13 obtains by analyzing the HTML file that it has a plurality of views and conveys it to the user by voice. Further, it is assumed that reading aloud is performed in view units.
[0031]
FIG. 9 is an example in which an image is posted on a Web browser according to an embodiment of the present invention, and FIG. 10 is an example in which reading is performed by the voice browser according to an embodiment of the present invention. Since the image shown in FIG. 9 is visual information, it is difficult to convey the information by voice. However, by analyzing the tag information of HTML, the explanation corresponding to the image can be extracted on the voice browser client 13 side by adding the explanation to the image, and can be read out by voice. In this case, it is a condition that the HTML creator adds an explanatory note regarding the image diagram as text information.
[0032]
Next, the playback control function will be described. This function is a function that can change the reading speed of the reading aloud, the volume, the gender of the speaker, and the like in real time by voice instructions. With regard to reading speed and speaker gender, voice data can be recreated by changing parameters of the speech synthesis engine 61, and can be changed in real time by pre-creating from the playback point. . The volume is dealt with by changing system parameters.
[0033]
As an additional function of the re-control function, an interrupt of a specific service (notification of current time, etc.) can be added. Specifically, when the current time is asked while reading aloud, the system is inquired about the time, and once the voice synthesis engine 61 is requested to create voice data, the reading is temporarily suspended when the creation is completed, and the current time is announced. Further, it can be realized by restarting the reading aloud after that.
[0034]
It is also possible to control playback of audio data embedded in the HTML file. FIG. 11 is a configuration diagram for reproducing music / reading content according to an embodiment of the present invention. When the audio data is reproduced with contents such as music or reading, the Web browser 11 automatically activates the reproducible reproduction software 14. By controlling the playback software 14 by the voice browser client 13, it is possible to control conventional functions of the software such as pause and playback by voice.
[0035]
FIG. 12 is a sequence chart of audio data reproduction control according to an embodiment of the present invention. First, when a URL instruction is issued from the voice browser client 13 to the Web browser 11 (Step 201), the Web browser 11 transmits the instruction to the Internet 70. As a result, the Web browser 11 acquires HTML from the Internet 70, transfers it to the voice browser client 13, and the voice browser client 13 analyzes the HTML. Further, the Web browser 11 requests the audio data embedded in HTML from the Internet 70 (Step 203), acquires a response corresponding to the request acquired from the Internet 70, activates the reproduction software 14, and transfers it. (Step 204). Further, the voice browser client 13 issues a voice recognition request to the voice recognition engine 53 based on the analyzed result (step 205), and when the voice recognition result is acquired, the result is transferred to the voice browser client 13. (Step 206). Thereby, the voice browser client 13 controls the playback software 14 to play back the voice.
[0036]
The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.
[0037]
【The invention's effect】
As described above, according to the present invention, it is possible to access and output the WWW information on the Internet by voice input.
In addition, when the accessed information is in English, it can be output in Japanese by the translation function, so even a user with little English knowledge can grasp the contents of the information.
[0038]
Furthermore, since it is possible to access not only text but also music information and reading information, it can be used for entertainment.
In addition, when playing music information and reading information, playback control such as pause, playback, stop, etc. is possible, just like a normal radio cassette player (radio / cassette with output / playback / recording function). Less user resistance.
[0039]
Furthermore, since it is possible to listen to the time signal while interrupting text reading, the time signal can be obtained by voice even for visually handicapped persons or when not holding a clock.
With the above functions, it is possible to support visually impaired people using the Internet.
[Brief description of the drawings]
FIG. 1 is a principle configuration diagram of the present invention.
FIG. 2 is a system configuration diagram to which the present invention is applied.
FIG. 3 is a system configuration diagram of an embodiment of the present invention.
FIG. 4 is a sequence chart of an operation according to an embodiment of the present invention.
FIG. 5 is a diagram showing a normal user interface of a Web browser according to an embodiment of the present invention.
FIG. 6 is an example of reading aloud by a voice browser according to an embodiment of the present invention.
FIG. 7 is an example of a homepage type displayed by a multi-frame configuration of a Web browser according to an embodiment of the present invention.
FIG. 8 is an example of reading aloud by a voice browser according to an embodiment of the present invention.
FIG. 9 is an example in which an image is posted on a Web browser according to an embodiment of the present invention.
FIG. 10 is an example of reading aloud by a voice browser according to an embodiment of the present invention.
FIG. 11 is a configuration diagram in the case of reproducing music / reading content according to an embodiment of the present invention.
FIG. 12 is a sequence chart of audio data reproduction control according to an embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Client terminal 11 Web browser 12 Voice input button monitoring program 13 Voice browser client 14 Reproduction software 20, 30, 40, 50, 60 Workstation 21 Proxy server 22 English-Japanese translation engine 31 Japanese-English translation engine 41 Proxy server 51 Speech recognition I / F program 52 Speech recognition engine 53 Morphological analysis engine 61 Speech synthesis engine 70 Internet 100 Server 110 Title 120 Text 130 Link item 200 Client terminal 201 Voice input means 202 Request issue means 203 Voice output means 210 First frame 220 Two-frame 310 image

Claims (1)

An HTML file stored on servers acquired via the Internet, an audio client terminal for outputting a voice,
Voice input means for inputting a user's voice request;
The input user's voice is transmitted to the voice recognition server, a voice recognition result is received from the voice recognition server, a URL is extracted from the received voice recognition result, and an HTML file is stored in the proxy server based on the URL. Request issuing means to request;
Means for analyzing an HTML file acquired from the proxy server and classifying the file into a title, a link item, and a body;
Means for transmitting the classified link items to a speech recognition server;
Means for transmitting displayed text information including the title, link item, and body to a speech synthesis server, and receiving speech data synthesized from the speech synthesis server;
Using the received voice data, voice output means for reading out the voice in the order of title, link item, and text;
Audio client terminal characterized in that it comprises a.

Images