JP2005321706A - Method for reproducing digital book and apparatus for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a method for reproducing a digital book with which a plurality of voices varying in sound quality with ample humanity can be inexpensively provided and an apparatus for the same. <P>SOLUTION: The reproducing apparatus including a data base 11 for synthesizing a plurality of the voices corresponding to respective speakers generated by separately taping the voices of a plurality of the speakers is used and the data base 11 for the plurality of the speakers is changed over according to the control information in an execution file 1 of the digital book or the assignment information inputted from the user by a reproduction controller 16 such that the voices corresponding to the text information in the execution file 1 of the digital book is subjected to synthesizing processing in a voice synthesizing section 12. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method and apparatus for reproducing an electronic book expressed by auditory information such as voice together with visual information such as characters and images.

  Conventionally, contents such as novels, picture books, and manga have been distributed in a state where characters, pictures, and photographs (still images) constituting them are printed on paper, that is, provided on paper as a medium. However, with the advancement of computer technology today, CD-ROMs (Compact Disk Read Only Memory) and flexible disks are distributed on various storage media with excellent portability and networks such as the Web. Books) (see Non-Patent Document 1).

In addition, in the case of electronic books, the playback device used for use often includes a display device capable of displaying a moving image, such as a personal computer, a PDA, or a mobile phone, and a speaker capable of reproducing audible sound. It often includes information such as video and audio as well as still images, and is characterized by being able to provide a variety of information compared to paper.
“What is an e-book?”, E-book consortium, [Search on April 26, 2004], Internet <URL: http://www.ebj.gr.jp/concept/a0101.html>

  Here, there are two methods for providing sound, a method using recorded sound and a method using synthesized sound. In the method using recorded sound, the sound itself is full of humanity, but there is a problem that the cost increases because it is necessary to record the sound for each content. On the other hand, in the method using synthesized speech, if a speech synthesis database and a speech synthesis engine are prepared, there is no need to record speech for each content, but multiple voices with different voice qualities can be handled by voice quality conversion by signal processing. When it was done, there was a problem that it became the sound which is not human.

  In addition, during playback of an e-book, there is a request that the user wants to stop playback, return to play a little before to hear the missed part again, or ask a question to solve the question that occurred during playback. It is possible that For example, when a child has a question while watching a picture book, he often raises that question. Answering this question is considered very important for education. However, conventional electronic books have not been considered to cope with these requirements.

  An object of the present invention is to provide an electronic book reproducing method and apparatus capable of providing a plurality of human-sounding voices having different voice qualities at low cost and capable of coping with user requests such as repeated reproduction and questions during reproduction. It is in.

  The electronic book reproducing method and apparatus according to the present invention is a method for easily expressing content that has been conventionally expressed by a medium such as paper, such as a novel, a picture book, and a manga, using electronic means.

  The electronic book in the present invention is described in a markup language or the like, and is distributed via various storage media such as a CD-ROM and a flexible disk and a network such as the Web.

  With this markup language, text information can be included as text information and data files can be specified for image information of still images and moving images. The specified data files can also be stored in various storage media. Or via a network, and can be stored in a memory included in the playback device.

  For speech, the content of utterance is registered as text information using a program language related to text utterance that is compatible with the markup language. Further, it is assumed that the voice language of the voice to be reproduced can be identified using the program language related to the text utterance, and the voice of the desired voice quality can be designated.

  Desired speech can be realized by generating synthesized speech by speech synthesis technology. In order to perform speech synthesis, a speech synthesis database including speech units of units constituting a language and a speech synthesis engine are required. In order to generate a large number of voice qualities, the present invention is not a method of changing the voice quality by using a speech synthesis engine that changes the fundamental frequency, but by recording the voices of a plurality of speakers separately to each speaker. A method is used in which a plurality of corresponding speech synthesis databases are created, and the plurality of speech synthesis databases are selectively used according to the situation. That is, the voice quality of speech depends on the speech synthesis database to be used. It is assumed that these many speech synthesis databases can also be stored in a memory provided in the playback device.

  In this way, by using the speech synthesis database corresponding to each voice quality, regardless of the signal processing in the speech synthesis engine, a plurality of voices with different voice quality can be uttered more naturally, It can be assigned for each character. Moreover, by preparing a speech synthesis database corresponding to a speaker having a desired voice quality, synthesized voices of any voice quality can be generated. Note that signal processing may be performed on the synthesized speech created from the speech synthesis database corresponding to the speaker of the desired voice quality in order to further improve speech quality.

  By executing the markup language described for playback control, it is possible to measure the timing of using the visual information source and audio information source stored in the memory in the device, and smoothly play back e-books. can do.

  Next, means for coping with a user's request during playback of an electronic book will be described.

  When a user has a question or request, the user can notify the playback device by operating a predetermined button or the like. At this time, the playback device pauses the playback process and stores the position in the memory. Above, the microphone is set to the voice input standby state. The voice input by the microphone is recognized by the voice recognition unit, the question response processing unit performs semantic analysis of the recognized sentence, and if it is a request for operation control, its intention is indicated. The answer derived from the correspondence dictionary can be transmitted to the dialogue control unit. In response to the instruction obtained from the question handling processing unit, the operation control is performed accurately programmed by the dialogue control unit, or a response to the question is shown using a speaker and a display. After finishing the question about the contents of the book, information can be obtained from the memory storing the paused position, and the electronic book reproduction process can be resumed.

  According to the present invention, in synthesized speech that can greatly reduce the cost of generating reproduced speech of an electronic book, the expression of the difference in voice quality is not based on signal processing, but a plurality corresponding to each of a plurality of speakers. By using the speech synthesis database, it is possible to generate speech with more humanity.

  Further, according to the present invention, it is possible to use the voice of the voice quality desired by the user for any character.

  Further, according to the present invention, it is possible to appropriately cope with a user's question or request while playing an electronic book.

  Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
FIG. 1 shows a first embodiment of an electronic book reproducing apparatus of the present invention together with an electronic book.

  In the present invention, an electronic book includes text information of characters and synthesized speech, and control information for reproduction, that is, execution procedure of reproduction, specification information of a file and database used for reproduction, etc. described in a markup language It is assumed that the file 1 is composed of a data file 2 including image information of one or both of a still image and a moving image to be presented as visual information.

  The executable file 1 for e-books such as picture books, novels, and manga is uniquely used to define markup languages such as HTML, XML, VoiceXML, Speech Application Language Tags (SALT), languages that can handle their own defined audio, and audio objects. It can be described using created tags. In addition, synchronization between visual information such as characters and images and auditory information such as voice can be realized using a markup language described in the execution file 1 or a language or tag defined uniquely.

  The e-book execution file 1 and data file 2 can be obtained through various storage media such as CD-ROM, flexible disk, semiconductor memory, and magneto-optical disk, and information providing medium 3 such as Web. be able to.

  In FIG. 1, 11 is a speech synthesis database, 12 is a speech synthesis unit (speech synthesis engine), 13 is a display memory, 14 is a display control unit, 15 is a markup language memory, and 16 is a reproduction control unit. These components constitute a main part (reproduction-only system part) 10 of the electronic book reproduction apparatus of the present invention. Reference numeral 21 denotes a speaker, and 22 denotes a display.

  The electronic book reproduction process is basically performed by the reproduction control unit 16 executing the markup language described in the execution file 1.

  The voice to be uttered described in the text information in the execution file 1 is output from the speaker 21 as voice by using the synthesis database 11 and the voice synthesis unit 12.

  At this time, by selecting the speech synthesis database 11 to be used in accordance with the control information in the execution file 1 of the electronic book or the designation information input by the user, the voice can be reproduced with different voice quality for each character, It can be played back with the voice of your favorite voice quality. This playback apparatus includes a plurality of speech synthesis databases 11 corresponding to each speaker created by separately recording a plurality of speaker speeches in advance, that is, a sufficient number of speech synthesis databases. And If a speech synthesis database corresponding to another speaker is required, it can be obtained via a network such as the Web or from a storage medium having excellent portability.

  It is necessary to display an image using the display 22 in synchronization with the sound generated in this way. As a part of the electronic book, a data file 2 including image information obtained in advance from the Web or a storage medium is stored in the display memory 13. The data file 2 including the image information to be displayed can be designated by a command issued from the reproduction control unit 16 to the display control unit 7 in synchronization with the voice utterance. The display control unit 7 can take out the data file 2 designated here from the display memory 13 and display it on the display 22.

  In the case of an electronic book consisting mostly of characters, such as a novel, it may be desirable to display, as visual information, an image that looks like a book using paper as a medium.

  If there is an illustration, it can be displayed in the same layout as a paper book, but the illustration can also be used as the background.

  Also, in order to clearly indicate which part is being reproduced, it is considered effective to change the color of the reading part. The range for changing the color may be a sentence unit, a line unit or a paragraph unit.

  Since these controls can be realized by a markup language, the electronic book publisher can realize the electronic book having the specifications as described above by creating the executable file 1 by a predetermined description method. it can.

  On the other hand, in the case where the electronic book is a book including many images such as a picture book, it is considered that expression by a picture is important. Therefore, it is not preferable that characters (text sentences) are displayed over the picture.

  In this case, it can be avoided by separating the picture display space and the text sentence display space. In this case, it is possible to display the current reproduction portion in color, as in an electronic book such as a novel.

  In the case of a picture book using an electronic book, a moving picture can be displayed unlike a picture book using paper as a medium. In addition, although it is a format similar to a moving image, it can be considered that a realistic feeling unique to a picture book can be obtained by displaying still images one after another at short intervals.

<Second Embodiment>
In the case of picture books for young children using electronic books, it is considered that an appropriate approach is required in consideration of viewing by young children. Picture books play a very important role in improving humanity, and it is especially important to answer questions that arise in children.

  Also, when you want to stop playback once in the middle of playback, or go back a little while to hear the missed part (rewind playback), or skip the less interesting part and play the part ahead You also need to deal with things like when you want (fast forward playback).

  For these reasons, it is considered that an electronic book having a response function for a request, such as giving an answer to a question, is required.

  FIG. 2 shows a second embodiment of an electronic book reproducing apparatus according to the present invention, which is a form provided with a simple dialog response function. In FIG. 2, the same components as those in FIG. That is, 10 is a reproduction-only system unit, 21 is a speaker, 22 is a display, 31 is a microphone, 32 is a voice recognition unit, 33 is a question response processing unit, 34 is a question response dictionary, 35 is a dialog control unit, and 36 is a dialog button. , 37 are reproduction ID tag memories.

  When the user has any doubts or requests, the user pushes the dialogue button 36 to notify the intention to the reproduction-only system unit 10 side through the dialogue control unit 35, and temporarily stops reproduction. At that time, it is necessary to know which part of the markup language is currently being executed. For example, an ID tag is attached in advance to a certain unit such as a paragraph or a page, and the ID tag is stored in the memory 37 while being reproduced. Since the memory 37 contains the ID tag of the stopped location when the pause is made, after the request is processed, the reproduction of the electronic book is resumed from the head of the ID tag stored in the memory 37.

  Regarding the response to the user's question, the operation of the voice recognition unit 32 is started by the dialogue control unit 35 when the dialogue button 36 is pressed. More specifically, a state in which voice input from the microphone 31 is received is set.

  A question is inputted into the microphone 31 by voice, the voice is recognized by the voice recognition unit 32, and a response to the question is derived by the question correspondence processing unit 33 while referring to the question correspondence dictionary 34 based on the recognition result.

  As an example, when the question item input from the microphone 31 is a question for which an answer is prepared in advance, and the question is specified from the semantic analysis result of the question handling processing unit 33, the dialogue control unit 35 displays the question response processing unit. The response content and the presentation material can be received from the question correspondence dictionary 34 via 33, and these are shown using the speaker 21 and the display 22 via the reproduction-only system unit 10.

  Thereafter, the user confirms whether or not the question has been solved by using the dialogue button 36, and resumes the reproduction of the electronic book by the method described above.

  As for the response to the user's request such as rewind playback, when the dialogue button 36 is pressed, the operation of the voice recognition unit 32 is started by the dialogue control unit 35, and the user inputs the request from the microphone 31 to recognize the voice. The voice is recognized by the unit 32, and the content of the request is analyzed by the question handling processing unit 33 and the question handling dictionary 34 based on the recognition result. A command for returning and reproducing is performed by causing the dialogue control unit 35 to send the command to the reproduction-only system unit 10 including the paragraph designation by the ID tag or the like.

  The microphone 31 and the voice recognition unit 32 described above constitute a voice recognition unit as defined in the claims, the question response processing unit 33 and the question correspondence dictionary 34 constitute a request recognition unit as defined in the claims, a dialogue control unit 35, The dialogue button 36 and the memory 37 constitute dialogue control means in the claims.

  The electronic book reproducing apparatus described above can be reproduced as long as there is an executable file 1 written in a markup language or the like and a data file 2 including image information, which are components of the electronic book.

  In other words, an executable file can be created with some knowledge of the markup language, and a data file can be easily prepared by using a simple picture, an image taken with a digital camera, or a recorded video. In other words, there is an advantage that a self-made electronic book can be easily created.

  One of the factors that make it easy to create an electronic book is the generation of speech using synthesized speech. As for the sound, a desired sound can be reproduced simply by writing it as text information in the execution file.

  With such a usage method, the electronic book reproducing apparatus can be considered as a system excellent in entertainment properties.

It is a block diagram which shows 1st Embodiment of the reproducing | regenerating apparatus of the electronic book of this invention. It is a block diagram which shows 2nd Embodiment of the reproducing | regenerating apparatus of the electronic book of this invention.

Explanation of symbols

  1: execution file, 2: data file, 3: information providing medium, 10: reproduction-only system unit, 11: database for speech synthesis, 12: speech synthesis unit (speech synthesis engine), 13: display memory, 14: display Control part, 15: Memory for markup language, 16: Playback control part, 21: Speaker, 22: Display, 31: Microphone, 32: Voice recognition part, 33: Question correspondence processing part, 34: Question correspondence dictionary, 35: Dialog control unit 36: Dialog button 37: Memory for reproduction ID tag

Claims (5)

A method of reproducing an electronic book expressed by auditory information such as sound together with visual information such as characters and images and distributed via a storage medium or a network,
An electronic book comprising text files of text and synthesized speech, and an executable file in which control information for reproduction is described in a markup language, and a data file containing image information;
Using a speech synthesis database, a speech synthesis engine, a display control unit, and a playback device including a playback control unit,
The playback controller
According to the control information in the execution file of the electronic book, the display control unit displays the character corresponding to the text information in the execution file of the electronic book and the image corresponding to the data file, and the control in the execution file of the electronic book A method for reproducing an electronic book, comprising: causing a speech synthesis engine to synthesize speech corresponding to text information in an execution file of the electronic book according to information or designation information input from a user. The electronic book reproducing method according to claim 1,
As a speech synthesis database, using a playback device equipped with a plurality of speech synthesis databases corresponding to each speaker created by separately recording the speech of a plurality of speakers,
The playback controller
According to the control information in the execution file of the electronic book or the designation information input from the user, the plurality of databases for speech synthesis are switched, and the speech corresponding to the text information in the execution file of the electronic book is synthesized in the speech synthesis engine. A method for reproducing an electronic book, characterized in that: The electronic book reproducing method according to claim 2,
In addition to the above, using a playback device including voice recognition means, request response means, and dialogue control means,
The dialogue control means
Receiving a request from the user during playback of the electronic book, the playback control unit temporarily stops playback,
Let the voice recognition means and request response means analyze the request from the user,
A method for reproducing an electronic book, comprising: causing a reproduction control unit to execute processing according to an analysis result, and then restarting reproduction. An apparatus for reproducing an electronic book expressed by auditory information such as sound together with visual information such as characters and images and distributed via a storage medium or a network,
Multiple speech synthesis databases corresponding to each speaker created by separately recording the speech of multiple speakers,
A speech synthesis engine that synthesizes speech corresponding to text information using a speech synthesis database;
According to the control information in the execution file of the electronic book or the designation information input from the user, the plurality of databases for speech synthesis are switched, and the speech corresponding to the text information in the execution file of the electronic book is synthesized in the speech synthesis engine. An electronic book reproducing apparatus comprising: a reproduction control unit that controls the electronic book. The electronic book reproducing apparatus according to claim 4, wherein
Voice recognition means for recognizing voice including a request from a user;
Request response means for analyzing the content of the request from the user based on the recognition result and deriving an answer or command to the request;
Accepts a request from the user during the reproduction of the electronic book, causes the reproduction control unit to suspend reproduction, causes the voice recognition unit and request response unit to analyze the request from the user, and causes the reproduction control unit to perform processing according to the analysis result An electronic book reproducing apparatus comprising: an interactive control unit configured to execute and then resume reproduction.

Images