JP2011233047A - Electronic book device, reproducing method for electronic book and reproducing program for electronic book - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic book device capable of reproducing sound data expressively as intended by a writer.SOLUTION: An electronic book device 100 for reading and reproducing electronic book data including text data having information on character strings that construct sentences and sound data corresponding to the text data comprises an attention position estimating section 40 for estimating an attention position in the text data and a sound data reproducing control section 50 for determining a reproducing form of sound data on the basis of a sound data type. The sound data reproducing control section 50 controls the sound data corresponding to the text data of the attention position estimated by the attention position estimating section 40 so as to reproduce the data in the determined reproducing form.

Description

  The present invention relates to an electronic book device, an electronic book reproduction method, and an electronic book reproduction program. In particular, the present invention relates to an electronic book device that reads and reproduces information of an electronic book configured by combining text data including information on character strings constituting a sentence and sound data that can be associated with the text data.

  For example, techniques disclosed in Patent Document 1, Patent Document 2, and Patent Document 3 are known as conventional techniques related to electronic book devices. For example, as disclosed in Patent Document 3, an electronic book is often information content in which text data, audio / sound effect data, image data, and the like are integrated.

  When reproducing such an electronic book, generally, it is managed in units of pages as in the case of paper books, and the text and images of the contents are visible information for each page so that the user can read the book. Is displayed on a predetermined screen, and a sound is also output if necessary.

  Patent Document 1 discloses a technique for automating page turning of an electronic book. Specifically, the length of time required for the user to finish reading the corresponding page is calculated based on the number of characters and figures in the text on each page of the electronic book, and the user reads the corresponding page. The page to be displayed is automatically moved to the next page (automatic page turning) at the timing estimated to be finished.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses that an associated text or image is associated with an image or music data, and when the associated text or image is selected, the associated content is reproduced.

  In Patent Document 3, a position (line or the like) expected to be read by the user is displayed by making an underline or a character bold, and audio / sound effect data or an image is designated at the corresponding position. It is disclosed that it is played back and displayed in the case of a failure.

JP-A-6-231187 JP-A-8-106472 JP 2006-331257 A

  If the technique of Patent Document 2 is adopted, the content of sound data can be associated with texts and images in the electronic book. Further, if the techniques of Patent Literature 1 and Patent Literature 3 are employed, a position expected to be read by the user can be automatically calculated, and this position can be automatically displayed to the user in an easily understandable manner. Also, with the technology of Patent Document 3, it is possible to reproduce sound / sound effects based on the reached position.

  By using sound data such as sound and sound effects in combination, richer expressions are possible compared to paper books. In other words, it is possible to directly convey to the reader a sound or the like intended by the book author. However, in the prior art, since the position where the reproduction of the sound data is started can only be controlled, the sound reproduction function is not sufficient for various expressions that the author intends and various situations of the reader.

  For example, when a background sound such as music (BGM) is to be output in accordance with the text of an electronic book, if the length of the corresponding text does not coincide with the length of the background sound, the text Because the situation that does not match the background sound and the background sound may occur, the intention of the author cannot be correctly reflected.

  Also, for example, when outputting background sounds, it is considered that the speed at which the reader reads the text changes, so if the position is estimated assuming that the reading speed is constant, the position that the reader is actually paying attention to There is a high possibility that a large deviation occurs between the estimated position and the estimated position.

  In addition, for sound data associated with e-books, there are actually various types of sounds such as speech (speech, reading of text), sound effects, and background sounds (BGM), so playback control is optimal for background sounds. In other words, there is a high possibility that inappropriate playback control is performed for other sounds and sound effects.

  The present invention has been made in view of the above circumstances, and provides an electronic book device, an electronic book reproduction method, and an electronic book reproduction program capable of reproducing sound data rich in expressiveness as intended by the author The purpose is to do.

  An electronic book device according to the present invention is an electronic book device that reads and reproduces data of an electronic book including text data including character string information constituting a sentence and sound data corresponding to the text data, Estimated by an attention position estimation unit that estimates an attention position in the text data, a sound data reproduction form determination unit that determines a reproduction form of the sound data based on the type of the sound data, and the attention position estimation part. A sound data reproduction control unit that controls to reproduce the sound data corresponding to the body data at the target position in the reproduction form determined by the sound data reproduction form determination unit.

  According to this electronic book apparatus, even when an electronic book in which various types of sound data are mixed is reproduced, the respective sound data can be reproduced under optimum conditions. Therefore, sound data can be reproduced with rich expressiveness as intended by the author.

  In the electronic book device of the present invention, the sound data reproduction form determination unit recognizes at least one of background sound, sound effect, speech sound, and text reading sound as the sound data type, As the sound data reproduction mode, at least one of a reproduction stop condition, the number of reproduction repetitions, a reproduction volume, and the influence of sound data reproduction on the estimation of the target position is determined.

  According to this electronic book device, a plurality of different types of sound data are distinguished from each other in the electronic book data, and reflected in the influence of the sound data reproduction on the reproduction stop condition, the number of reproduction repetitions, the reproduction volume, and the estimated position of interest. Can be suitably reproduced.

  In the electronic book device according to the present invention, when the sound data reproduction form determination unit includes a character indicating a quotation mark in the body data corresponding to the sound data, the sound data is speech of speech. Is determined.

  According to this electronic book apparatus, the speech of the dialogue can be distinguished from the data of the electronic book, and the reproduction can be performed in a reproduction form suitable for the speech of the speech.

  In the electronic book device of the present invention, the sound data reproduction form determination unit determines that the sound data is a sound effect when the text data corresponding to the sound data includes an onomatopoeia.

  According to this electronic book apparatus, the sound effect can be distinguished from the data of the electronic book, and reproduction can be performed in a reproduction form suitable for the sound effect.

  Further, in the electronic book device of the present invention, the sound data reproduction form determination unit is configured to store the sound data based on a reading time based on the number of characters of the body data corresponding to the sound data and a reproduction time of the sound data. It is determined whether the sound is a background sound or a sound effect.

  According to this electronic book apparatus, the background sound or the sound effect can be distinguished from the data of the electronic book, and the reproduction can be performed in a reproduction form suitable for the background sound or the sound effect.

  Further, in the electronic book device of the present invention, the sound data reproduction control unit is configured such that the sound data corresponding to the body data of the target position estimated by the target position estimation unit is a background sound, and the target table is recorded in the target table. When it is stored that the position estimation is to be stopped, the attention position estimation by the attention position estimation unit is interrupted.

  According to this electronic book apparatus, when the sound data is background sound and further needs, the idea of the text data at the estimated position of interest and the idea of the background sound are interrupted by interrupting the estimation of the position of interest. Thus, it is possible to avoid the occurrence of deviation.

  Further, in the electronic book device of the present invention, the sound data reproduction control unit, when the sound data corresponding to the body data of the target position estimated by the target position estimation unit is speech of speech, The estimation of the attention position by the estimation unit is interrupted.

  According to this electronic book apparatus, when the sound data is speech of speech, by interrupting estimation of the attention position, there is a difference between the idea based on the text data at the estimated attention position and the idea based on speech of the speech. Can be avoided.

  In the electronic book device of the present invention, when the sound data reproduction control unit is assigned a plurality of the sound data corresponding to the same target position estimated by the target position estimation unit, The reproduction of the low-priority type sound data among the sound data is restricted.

  According to this electronic book apparatus, when there is a plurality of sound data to be reproduced at the same time, the sound data of the type with low priority is reduced to a low volume or the reproduction is stopped, thereby reproducing the suitable sound data. It can be performed.

  In the electronic book device of the present invention, when the sound data reproduction control unit accepts a page turning instruction, reproduction of the sound data is stopped under different conditions based on the type of the sound data being reproduced.

  According to this electronic book apparatus, when page turning occurs, the reproduction of the type of sound data that should not be reproduced can be stopped, and the reproduction of the type of sound data that should be reproduced can be continued.

  Moreover, the electronic book apparatus of this invention is provided with the data communication part which receives the said sound data from an external device.

  According to this electronic book apparatus, even when the electronic book includes only link information indicating the location of the entity as sound data, the main body of the necessary sound data is acquired from the external apparatus and reproduced. Can do.

  The electronic book reproducing method of the present invention is an electronic book reproducing method for reading and reproducing electronic book data including text data including character string information constituting a sentence and sound data corresponding to the text data. A step of estimating a position of interest in the text data; a step of determining a reproduction form of the sound data based on a type of the sound data; and corresponding to the text data of the estimated position of interest Controlling the sound data to be played back in the determined playback mode.

  According to this electronic book reproduction method, even when an electronic book in which various types of sound data are mixed is reproduced, the respective sound data can be reproduced under optimum conditions. Therefore, sound data can be reproduced with rich expressiveness as intended by the author.

  Moreover, the electronic book reproduction | regeneration program of this invention is a program for performing each step of said electronic book reproduction | regeneration method.

  With this electronic book reproduction program, even when an electronic book in which various types of sound data are mixed is reproduced, the respective sound data can be reproduced under optimum conditions. Therefore, sound data can be reproduced with rich expressiveness as intended by the author.

  According to the present invention, sound data can be reproduced with rich expressiveness as intended by the author.

1 is a block diagram showing a configuration example regarding main functions of an electronic book device according to an embodiment of the present invention; The schematic diagram which shows the specific example regarding the structure of the electronic book which the electronic book apparatus of embodiment of this invention reproduces | regenerates The schematic diagram which shows the specific example regarding the structure of the electronic book which the electronic book apparatus of embodiment of this invention reproduces | regenerates The schematic diagram which shows the structural example of the sound data management table and resume data management table which the electronic book apparatus of embodiment of this invention uses. The flowchart which shows the outline | summary regarding operation | movement of the electronic book apparatus of embodiment of this invention. The flowchart which shows a part of process for automatically classifying the classification of the sound data of embodiment of this invention. The flowchart which shows a part of process for automatically classifying the classification of the sound data of embodiment of this invention. The schematic diagram which shows the specific example of the electronic book data used as the object of the process for automatic classification | category of the classification of the sound data of embodiment of this invention It is a schematic diagram which shows the specific example of the electronic book data used as the object of the process for automatically classifying the classification of the sound data of embodiment of this invention. The flowchart which shows the process for estimating the reproduction | regeneration stop position of the sound data of embodiment of this invention. The flowchart which shows the process for carrying out reproduction | regeneration control of the sound data of embodiment of this invention The flowchart which shows a process when the instruction | indication of the page turning of embodiment of this invention is received The flowchart which shows a process when the instruction | indication of the page return of embodiment of this invention is received

  An electronic book apparatus, an electronic book reproduction method, and an electronic book reproduction program according to embodiments of the present invention will be described below with reference to the drawings.

  A configuration example relating to main functions of the electronic book device according to the embodiment of the present invention is shown in FIG. As shown in FIG. 1, the electronic book device 100 includes a text data storage unit 10, a sound data storage unit 20, a data communication unit 25, a text / sound data analysis unit 30, an attention position estimation unit 40, a clock unit 45, a sound A data reproduction control unit 50, a page switching processing unit 60, an output unit 70, an input unit 80, and a position storage unit 90 are provided.

  As for the data of the electronic book (content) to be reproduced by the electronic book apparatus 100, sound that can be associated with at least text data (or text data) including character string information constituting the text and the text data. It consists of a combination with data. Of course, in addition to sound data, an electronic book including attached content such as an image can be reproduced. The sound data is associated with a desired position of the text of the text data by predetermined link information. The main body of the sound data, that is, the digital sound data necessary for outputting the sound signal may be embedded and integrated in the text of the text data, or outside the text data, that is, separately from the text data Sometimes it is arranged.

  In the present embodiment, “reproducing an electronic book” means here that at least the content of one page of body data included in the electronic book is visually displayed as a character string on a predetermined screen. This means that you can read the contents by looking at the screen. Of course, when there is a function that automatically reads the entire sentence using the utterance function of the device, reading automatically without displaying the screen is equivalent to “reproducing an electronic book”. . Further, when there is sound data associated with the link information in the reproduced body data, the sound data is output as sound so that the reader (user) can hear it, as will be described later. Such an operation is defined as “sound data reproduction” in the present embodiment.

  The text data storage unit 10 shown in FIG. 1 stores and holds text data (or text data, including link information) of an electronic book to be reproduced. The sound data storage unit 20 stores and holds the contents of the main body of the sound data associated with the body data of the electronic book and the link information. As a specific example, a recording medium such as a CD-ROM or a DVD disk or a storage device such as a hard disk is used as the text data storage unit 10 or the sound data storage unit 20.

  About the content of the main body of the sound data linked | related with the electronic book, it can also be arrange | positioned on the predetermined | prescribed server which can connect via the arbitrary places outside the electronic book apparatus 100, for example, the internet. The data communication unit 25 shown in FIG. 1 has a data communication function for connecting to a communication network such as the Internet. Based on the link information, the data communication unit 25 acquires the required sound data body from a specified location. It is stored on the sound data storage unit 20.

  Therefore, by reading the data held in the text data storage unit 10 and the data held in the sound data storage unit 20, it is possible to reproduce the electronic book and the associated sound data. .

  The sentence / sound data analysis unit 30 analyzes the entire contents of the electronic book, checks the link status between the body data such as sentences and the attached contents such as sound data, and determines the type of sound data associated with the link information. Performs processing for classification.

  The attention position estimation unit 40 estimates the attention position (position of characters or the like in the sentence) that is estimated to be read by the reader about the contents of one page of the electronic book being reproduced. For example, assuming that the time Tc1 required for the reader to read one character is constant, the number of characters on the corresponding page is the position of interest based on the elapsed time since the corresponding page was opened and the time Tc1. Can be determined. Of course, since the time Tc1 varies depending on individual differences and circumstances, various processes are performed in estimating the actual attention position.

  The clock unit 45 outputs information on the current time. This time information is used when the attention position estimation unit 40 estimates the attention position.

  The sound data reproduction control unit 50 performs control related to reproduction of sound data. In other words, information on the main body of the sound data to be reproduced is acquired and decoded, converted into an electrical electrical signal, and the timing of starting and ending the acoustic output is controlled, and the volume control is performed to output the acoustic signal Or output to the unit 70. Also, it has a function as a sound data reproduction form determination unit that decides the reproduction form of sound data based on the type of sound data.

  The page switching processing unit 60 performs processing related to page switching. That is, since the content of the electronic book is managed in units of pages, one specific page displayed on the current screen is switched as necessary. In the state where the attention position estimation unit 40 estimates the attention position of the reader, the reader can predict the timing when the reader finishes reading the content of the currently displayed one page to the end. The switching processing unit 60 switches to the next page. Further, when receiving an instruction to turn pages or return pages by an input operation by the reader, the page switching processing unit 60 switches pages according to the instructions.

  Note that the functions of the sentence / sound data analysis unit 30, the attention position estimation unit 40, the sound data reproduction control unit 50, and the page switching processing unit 60 are realized by microcomputer hardware and a program executed by the microcomputer. The Of course, part or all of these functions can be replaced with hardware that executes dedicated processing.

  The output unit 70 includes a display device for visually displaying a reproduced electronic book text or the like on a screen, and a converter (speaker or the like) for outputting an acoustic electric signal as sound.

  The input unit 80 is an input device for receiving a necessary input operation from a reader who is a user. As a specific example of the input unit 80, a touch panel disposed on the display screen of the output unit 70 is assumed. The input unit 80 receives, for example, a reproduction start instruction, a page turning instruction, a page return instruction, a reproduction end instruction, or the like from the user.

  The position storage unit 90 stores information such as a position as necessary when the page switching processing unit 60 performs page switching.

  Regarding the sound data included in the electronic book handled by the electronic book apparatus 100 shown in FIG. 1, there are four types: “background sound”, “sound effect”, “serif”, and “text reading speech”. The definition of these four types of sound data is as follows.

  Background sound: Represents sound that appears almost continuously over a long period of time, such as background music (BGM), sound that represents the environment (sounds of insects and sound of waves), music and movie works, etc. .

  Sound effects: For example, sounds expressed by things or creatures, such as car horns or cat calls, or sounds that represent conditions or emotions. It is shorter than the background sound and appears suddenly for a short time.

  Dialogue (speech speech): Sounds used to express words uttered by characters in plays or creative plays with human speech or pseudo-speech.

  Text-to-speech reading: Represents a sound that reads a sentence (other than a word uttered by a person) in an electronic book using a human voice or a pseudo voice similar to it.

  These four types of sound data can be used properly for the author of the electronic book to express the contents of the content with richer expressive power. In order to accurately convey such an expression of the author to the reader, the electronic book device 100 shown in FIG. 1 performs various controls according to the type regarding the reproduction of the sound data. A summary of this control is shown in the following Tables 1 and 2 as a list.

  As shown in Tables 1 and 2, regarding the reproduction of sound data, “stop control”, “influence on position estimation”, “sound superposition”, “volume adjustment”, “page return”, “page” For “turning”, special control is performed according to the type of sound.

  “Stop control” in Table 1 is control related to under what conditions (reproduction stop condition) the reproduction of the corresponding sound data is stopped. “Influence on position estimation” is control for taking into account the influence of reproduction of corresponding sound data on the position estimation of the target position estimation unit 40. “Sound superposition” is control for determining how to process a plurality of types of sound data when they are output at the timing when they are overlapped. “Volume adjustment” is control for determining the sound volume at which the corresponding sound data is reproduced, that is, the reproduction sound volume. “Page return” in Table 2 is control for determining how to process the sound data being played back when page return occurs. “Page turning” is control for determining how to process the sound data being played back when page turning (operation to advance to the next page) occurs.

  Next, a specific example regarding the configuration of an electronic book to be played back by the electronic book device 100 will be described. Specific examples are shown in FIGS. 2 and 3, respectively. FIGS. 2 and 3 show the relationship between the text of a sentence and sound data associated with each part of the content of one page in an electronic book.

  The electronic book shown in FIG. 2 includes “background sound 1”, “background sound 2”, “sound effect”, “line 1”, and “line 2” in addition to the text of the sentence. It is. In “background sound 1”, information specifying the start position and end position in the sentence for reproducing this sound, that is, the range surrounded by the dotted line in FIG. 2, is embedded in the body data of the sentence. There is a background sound. “Background sound 2” is a background sound in which the start position in the sentence for reproducing this sound and the number of reproductions (number of reproduction repetitions) are embedded in the text data of the sentence. “Serif 1” and “Serif 2” are words associated with one line in the sentence. In a line with “Serif 1” and “Serif 2”, there is a character string enclosed in square brackets (“”). Although only parentheses are illustrated here, quotation marks such as “” (English) and << >> (French) may be used. “Sound effect” is a sound effect associated with an onomatopoeia in a sentence in FIG.

  In the electronic book shown in FIG. 3, in addition to the text of the sentence, “background sound” associated therewith is included. The link information related to the “background sound” includes a sound reproduction start position and designation for temporarily stopping (for example, 10 seconds) the position estimation after the sound reproduction is started. When this designation is made, the electronic book device 100 starts reproduction of the linked background sound at a normal volume, pauses position estimation, passes 10 seconds, and then estimates the position. Restart and change the volume to a lower level.

  The sentence / sound data analysis unit 30 of the electronic book apparatus 100 uses, for example, a sound data management table as shown in FIG. 4 in order to manage sound data in the electronic book to be reproduced. Referring to FIG. 4, this sound data management table includes “sound file storage location”, “sound file name”, “reproduction time”, “sound classification”, “number of times of reproduction”, “position of interest after starting reproduction”. An area is provided for managing information of each item of “time to pause estimation” and “reproduction stop position”. The sound data management table is stored in the sentence / sound data analysis unit 30, for example.

  Among these items, “sound file storage location (folder name, address, etc.)” and “sound file name” are always included in the link information in the text in order to identify the corresponding sound data. For information on each item of “Playback time”, “Sound classification”, “Number of playbacks”, “Time to pause the estimation of the target position after starting playback”, and “Playback stop position” items, please refer to the link information in the text. The electronic book device 100 may determine the contents as described later.

  The sentence / sound data analyzing unit 30 can use a resume data management table in addition to the sound data management table. The resume data management table is used to store information before and after page turning when a page turning and page returning operation described later is performed. Referring to FIG. 4, the resume data management table includes “reading position”, “reproducing background sound”, “reproducing background sound stop position”, and “reproducing sound effect”. , An area is provided for managing information of each item of “the stop position of the sound effect being played”, “the line being played”, and “the stop position of the line being played”. The resume data management table is stored in the position storage unit 90, for example.

  FIG. 5 shows an outline (main routine) regarding the operation of the electronic book apparatus 100 shown in FIG. The operation shown in FIG. 5 will be described below.

  After starting the electronic book reproduction operation, when there is a page switching input, the operation of the electronic book device 100 proceeds from step S11 to step S12. That is, the first page is displayed or the page is turned by the input operation of the reader.

  In step S12, the page switching processing unit 60 performs page switching. That is, the position of the next page to be reproduced on the content of the electronic book is specified.

  In step S <b> 13, the sentence / sound data analysis unit 30 reads the entire sentence data of the specific page selected by the page switching processing unit 60 from the sentence data storage unit 10. The text data for one page read here is displayed (reproduced) on the display screen of the output unit 70 as text string information (visible information) as shown in FIG.

  In step S14, the sentence / sound data analysis unit 30 sequentially analyzes the contents of the read sentence data, and checks whether or not there is sound data associated with the link information. When linked sound data is found, information necessary for managing the sound data management table (sound file storage location, file name, number of playbacks, and estimation of the position of interest after playback is paused) Write time, etc.).

  In step S15, when the sound data linked in step S14 is found, the sentence / sound data analysis unit 30 determines the type of sound data (“background sound”, “sound effect”, “serif”, “text”). Categorization of “speech for reading”). Details of the processing will be described later. In addition, general link information does not include information indicating the type of sound data, but if the link information itself includes information indicating the type of sound data, a special classification process is unnecessary. The process of step S15 is omitted. The result of this classification process is reflected in the contents of the sound data management table.

  In step S16, the sentence / sound data analysis unit 30 estimates a position where the sound data reproduction is stopped. Details of the processing will be described later. Also, if the link information itself includes information indicating the reproduction stop position of the sound data, the process of step S16 is unnecessary, so the execution of this process is omitted. When the reproduction of the corresponding sound data is designated to be reproduced once or reproduced a plurality of times, the position where the sound data reproduction is stopped is estimated in step S16. The result of this estimation process is reflected in the contents of the sound data management table.

  In step S <b> 17, the sentence / sound data analysis unit 30 identifies whether or not the above processing has been completed for all sound data linked to one page of sentence data. If unprocessed sound data remains, the process returns to step S14, and if the process is completed, the process proceeds to step S18.

  In step S <b> 18, the attention position estimation unit 40 estimates the current attention position of the reader in the page being reproduced using the time information output from the clock unit 45. That is, the position of interest (position information indicating the line number in the page and the number of the character in the line) is estimated based on the elapsed time since the page was switched.

  In step S19, the sound data reproduction control unit 50 identifies whether or not the sound data associated with the link information exists in the sentence of the latest target position estimated by the target position estimation unit 40 in step S18. When it exists, it progresses to step S20, and when it does not exist, it progresses to step S21.

  In step S20, the sound data reproduction control unit 50 performs a process for reproducing (sound output) the sound data associated with the sentence at the current position of interest. Details of this processing will be described later.

  In step S21, the sound data reproduction control unit 50 identifies whether or not the latest attention position estimated by the attention position estimation unit 40 satisfies a specific condition. When the specific condition is satisfied, the process proceeds to step S22, and when not satisfied, the process proceeds to step S23. The specific condition is, for example, a condition indicating whether “the position of interest straddles a chapter of a sentence” or “a scene has been switched (for example, straddles a portion separated by one blank line)”.

  In step S22, the sound data reproduction control unit 50 identifies whether or not the sound data of the “background sound” is currently being reproduced, and stops reproducing the sound data (sound output) if the sound data is being reproduced.

  In step S23, the sound data reproduction control unit 50 identifies whether the target position estimated by the target position estimation unit 40 has reached the end of the page. If the end has not been reached, the process returns to step S18 to repeat the sound data processing within the same page. If the end has been reached, the process returns to step S12 to switch the page to be reproduced.

  Next, details of the processing related to the classification of sound data shown in step S15 of FIG. 5 will be described. The contents of this processing are shown in FIGS. A specific example of the electronic book data to be processed in FIG. 6 is shown in FIG. 8, and a specific example of the electronic book data to be processed in FIG. 7 is shown in FIG.

  In step S31 shown in FIG. 6, the sentence / sound data analysis unit 30 sets the paragraph start position and the paragraph end representing the range of the paragraph in the sentence including the detected link information (which associates the sentence with the sound data). Detect position. In the example shown in FIG. 8, a range surrounded by a frame indicated by a dotted line is detected as the range of the corresponding paragraph.

  In step S32, the sentence / sound data analysis unit 30 determines whether or not the sentence at the position where the link information exists is surrounded by a pair of square brackets (“”) within the range of the paragraph detected in step S31. Identify. If it is surrounded, the process proceeds from step S33 to S34. If it is not surrounded, the process proceeds to step S35 in FIG.

  In the example shown in FIG. 8, it is assumed that the link information exists within the range of the underlined text, and the text in this range is surrounded by square brackets (“”). move on.

  In step S34, the sentence / sound data analysis unit 30 determines that the type of the sound data of the corresponding link information is “serif”. Then, the sentence / sound data analysis unit 30 writes the information of the determination result in the corresponding item of the sound data management table. As a result, when the lines are reproduced, the reproduction can be performed in a desired reproduction form suitable for the sound data.

  In step S35, the sentence / sound data analysis unit 30 extracts a passage in the sentence that includes the detected link information (what associates the sentence with the sound data). Specifically, the range from “.” To “.”, Which is a punctuation mark, is extracted as one passage.

  In the example shown in FIG. 9, since it is assumed that link information corresponding to the position indicated by “sound effect 1” (the range of the character string “Gee”) exists, this is included when step S35 is executed. The range from “.” To “.”, That is, “Ghee, a squeaking sound of the door suddenly” was extracted as one passage.

  In step S36, it is checked whether or not a predetermined explanation is included in the passage of text extracted in step S35. Specifically, it is checked whether or not “sound of xx” or “voice of xx” (xx represents an arbitrary character string) is included. If it is included, the process proceeds to step S37, and if it is not included, the process proceeds to step S38.

  In step S <b> 37, the sentence / sound data analysis unit 30 determines that the type of sound data of the corresponding link information is “sound effect”. Then, the sentence / sound data analysis unit 30 writes the information of the determination result in the corresponding item of the sound data management table. Thus, when the text data includes information indicating a specific sound or voice, it is determined that the sound is a sound effect. When the sound effect is reproduced, the reproduction is performed in a desired reproduction form suitable for the sound data. It can be carried out. In addition, in order to further improve the determination accuracy in this case, the final determination may be performed after performing the processing after step S38 after step S37.

  In step S38, the sentence / sound data analysis unit 30 calculates the number of characters of the sentence associated with the sound data of the corresponding link information. When the link information includes a start position and an end position, the corresponding character string can be specified thereby.

  For the link information of “sound effect 2” shown in FIG. 9, since the sound data is associated with the range of the character string “batan” shown with an underline, the number of characters in this range (in this case, 3) is the step. Calculated in S38.

  In step S39, the sentence / sound data analysis unit 30 indicates how long the sound is output when the sound data of the corresponding link information is actually reproduced as sound. Is calculated. In the case of “sound effect 2” shown in FIG. 9, for example, it is calculated as 2 seconds.

  In step S40, the sentence / sound data analysis unit 30 indicates a required time (reading time) indicating how much time is required when the reader reads a character string in a range associated with the corresponding link information. ) Is calculated based on the number of characters. In the case of “sound effect 2” shown in FIG. 9, for example, it is calculated as 1 second.

  In step S41, the sentence / sound data analysis unit 30 compares the reproduction time calculated in step S39 with the required time calculated in step S40. Specifically, the absolute value of the difference between the reproduction time and the required time is compared with a predetermined threshold value. If the difference is within about several seconds (the reproduction time is equal to the required time), the process proceeds to step S42, and the difference is larger than that. If larger, the process proceeds to step S43.

  In step S42, the sentence / sound data analysis unit 30 determines that the type of the sound data of the corresponding link information is “sound effect”. Then, the sentence / sound data analysis unit 30 writes the information of the determination result in the corresponding item of the sound data management table.

  In step S43, the sentence / sound data analysis unit 30 determines that the type of the sound data of the corresponding link information is “background sound”. Then, the sentence / sound data analysis unit 30 writes the information of the determination result in the corresponding item of the sound data management table.

  In this way, since it is possible to determine whether the sound effect or the background sound in consideration of the reading time and the reproduction time of the sound data, it is possible to reproduce the sound data in a suitable reproduction form.

  Next, details of the processing relating to “estimation of the sound data reproduction stop position” shown in step S16 of FIG. 5 will be described. The contents of this process are shown in FIG.

  In step S51 of FIG. 10, the sentence / sound data analysis unit 30 refers to the sound data management table, and specifies the length of the reproduction time when the sound data of the corresponding link information is reproduced.

  In step S52, when the sound / sound data analysis unit 30 reproduces the corresponding sound data over the length of the reproduction time specified in step S51, how far the position in the sentence being read by the reader advances ( (Playback stop position) is calculated. In other words, from the state where the reader's attention position in the text is at the beginning of the linked portion of the sound data, the estimated position where the reader's attention position arrives when the playback time has passed is the reader's reading speed (estimated value) Etc.) and the playback time.

  In step S53, the sentence / sound data analysis unit 30 writes the reproduction stop position information calculated in step S52 into the sound data management table. In the example of the sound data management table shown in FIG. 4, “20th character of the 10th paragraph from the beginning” is written as the playback stop position.

  Next, details of the processing relating to “reproduction of content (sound data)” shown in step S20 of FIG. 5 will be described. The contents of this processing are shown in FIG. That is, the sound associated with the current reader's attention position estimated by the attention position estimation unit 40 in the state in which the electronic book is reproduced and a certain page of text or the like is displayed on the display screen of the output unit 70. When data exists, the process of FIG. 11 is executed.

  First, in step S61, the sound data reproduction control unit 50 identifies whether the type of the corresponding sound data is “background sound” based on the contents of the sound data management table. If it is “background sound”, the process proceeds to step S62; otherwise, the process proceeds to step S69.

  In step S62, the sound data reproduction control unit 50 checks the reproduction state of the current sound and identifies whether the “background sound” is already sounding (reproducing). If it is already ringing, it will progress to step S63, and if it is not ringing, it will progress to step S64.

  In step S63, the sound data reproduction control unit 50 stops the reproduction of the other “background sound” that is currently ringing (reproduction is started first). Therefore, when a plurality of “background sounds” are assigned to the same area in the sentence, the “background sound” that was started first is stopped, and then the next “background sound” is reproduced. “Background sounds” do not overlap at the same time. Here, the priority of the background sound to be reproduced from now on is higher than that of the background sound that has been reproduced first.

  In step S <b> 64, the sound data reproduction control unit 50 acquires from the sound data storage unit 20 the body of the corresponding sound data associated with the current estimated position of interest by link information.

  In step S65, the sound data reproduction control unit 50 adds “designation to pause the estimation of the target position” to the corresponding sound data based on the contents of the sound data management table (the sixth from the top of the sound data management table in FIG. 4). Whether or not there is an item content). If not specified, the process proceeds to step S66, and if specified, the process proceeds to step S67.

  In step S66, the sound data reproduction control unit 50 starts reproduction of the sound data body acquired in step S64 as a “background sound” at a relatively low volume (volume lower than normal).

  In step S67, the sound data reproduction control unit 50 starts reproducing the main body of the sound data acquired in step S64 at a normal volume. Furthermore, here, the estimation of the target position in the target position estimation unit 40 is temporarily stopped. The attention position does not change during the pause. The length of the pause time is specified by the contents of the sound data management table. Thereby, it is possible to prevent a deviation between the concept based on the text data and the concept based on the background sound at the estimated position of interest.

  In step S68, the sound data reproduction control unit 50 sets the volume of the “background sound” currently being reproduced to a relatively small volume (normally) after a specified time has elapsed since the estimation of the target position was temporarily stopped in step S67. The target position estimation unit 40 resumes the target position estimation from the position where it was temporarily stopped.

  In step S69, the sound data reproduction control unit 50 identifies whether the type of the corresponding sound data is “sound effect” or “line” based on the contents of the sound data management table. If it is “sound effect”, the process proceeds to step S70, and if it is “line”, the process proceeds to step S74.

  In step S <b> 70, the sound data reproduction control unit 50 acquires the body of the corresponding sound data associated with the current estimated attention position by the link information from the sound data storage unit 20.

  In step S71, the sound data reproduction control unit 50 checks the reproduction state of the current sound and identifies whether or not “line” is already ringing (reproducing). If it is already ringing, it will progress to step S72, and if it is not ringing, it will progress to step S73.

  In step S72, the sound data playback control unit 50 sets the volume of the “sound effect” to be played back to a relatively low volume (volume lower than normal). If step S72 is not executed, a normal volume is selected as the playback volume. Here, serif has higher priority than sound effect.

  In step S73, the sound data reproduction control unit 50 reproduces the main body of the sound data acquired in step S70 as “sound effect”.

  In step S74, the sound data reproduction control unit 50 checks the reproduction state of the current sound, and identifies whether the “sound effect” is already sounding (reproducing). If it is already ringing, it will progress to step S75, and if it is not ringing, it will progress to step S76.

  In step S75, the sound data reproduction control unit 50 sets the volume of the sound effect being reproduced to a relatively low volume (volume smaller than normal). Here, serif has higher priority than sound effect.

  In step S76, the sound data reproduction control unit 50 acquires from the sound data storage unit 20 the main body of the corresponding sound data associated with the current estimated position of interest by link information.

  In step S77, the sound data reproduction control unit 50 reproduces the main body of the sound data acquired in step S76 as “line”.

  In step S <b> 78, the sound data reproduction control unit 50 controls suspension / resumption regarding the estimation of the attention position estimation unit 40. Specifically, even if the estimated position of interest reaches the end of the sentence position associated with the sound data, if the output of “Serif” started to be reproduced in step S77 is continued, Pause the estimation of the position of interest. Then, when the output of “Serif” is completed, the estimation of the position of interest is resumed from the position where it was temporarily stopped. By temporarily stopping the estimation of the attention position, it is possible to prevent a deviation between the concept based on the text data and the concept based on the serif at the estimated attention position.

  As described above, when it is necessary to reproduce a plurality of sound data at the same time, it is possible to reproduce suitable sound data by limiting the reproduction of the low-priority type sound data.

  Next, operations in the case of “page turning” and “page returning” will be described. The operation in the case of “page turning” is shown in FIG. 12, and the operation in the case of “page return” is shown in FIG.

  When an instruction for moving the page to be reproduced (displayed) to the current next page or the previous page, that is, “page turning” is generated by the input operation of the reader on the input unit 80, the processing after step S81 is performed. .

  In step S81, the sound data reproduction control unit 50 compares the manually designated “page turning” cycle with the automatic page turning cycle to identify whether or not the page turning is quick.

  When the “page turning” is not performed manually, the page turning is automatically executed when the end of the page is reached according to the progress of the target position estimated by the target position estimation unit 40. Therefore, the automatic page turning cycle (the length of time) is determined according to the estimated value of the speed at which the reader reads the characters and the number of characters in the page, and does not change greatly. On the other hand, when the reader manually instructs “turning pages”, various situations can be considered. For example, there may be a case where the reader has read all the way to the end of the page before automatic page turning is performed, or a case where the reader repeatedly flips through the pages without reading the contents of each page. To distinguish these differences, we identify whether manual page turning is quick. Specifically, the elapsed time from the previous page turning timing to the current page turning timing is detected as the manual page turning cycle. If the manual page turning cycle is within a predetermined period (for example, within 1/5) of the automatic page turning cycle, the process proceeds to step S89, and if longer, the process proceeds to step S82.

  In step S82, the sound data reproduction control unit 50 identifies whether or not automatic reproduction is stopped. Stopping automatic playback is a state in which the estimation of the target position by the target position estimation unit 40 is stopped. If stopped, the process proceeds to step S83, and if not stopped, the process proceeds to step S84.

  In step S83, the sound data reproduction control unit 50 resumes the estimation of the target position by the target position estimation unit 40. That is, the reproduction control is resumed assuming that the reader reads the content from the top position of a new page opened by manual page turning.

  In step S84, the sound data reproduction control unit 50 reads the position (including the page number) of the position that the reader was reading at the timing immediately before the manual page turning, that is, the position of interest estimated last by the position of interest estimation unit 40. The data is recorded in the resume data management table of the position storage unit 90. In addition, when the sound data reproduction control unit 50 is simultaneously reproducing the sound data, the position data is also stored with respect to information specifying the corresponding sound data and information indicating the reproduction state (position, time, etc.). The data is recorded in the resume data management table of the unit 90.

  In step S85, the sound data reproduction control unit 50 checks the type of sound data corresponding to the reproduction of sound data. When the type of the sound data is “background sound”, it is identified whether or not the estimated position of interest exceeds the reproduction stop position of the sound data. If the reproduction stop position is exceeded, the process proceeds to step S87, and if not, the process proceeds to step S86.

  In step S86, the sound data reproduction control unit 50 checks the type of sound data corresponding to the reproduction of sound data. If this is “sound effect” or “serif”, it is identified whether or not the range of the link position of the sound data straddles the page. When straddling the page, the process proceeds to step S88, and when not straddling, the process proceeds to step S87.

  In step S87, the sound data reproduction control unit 50 stops reproduction of the sound data being reproduced. In other words, if the “background sound” playback stop position is exceeded due to page movement, or if the page moves during playback of “sound effects” or “lines” that do not straddle the page, sound playback stops. To do.

  In step S88, the sound data reproduction control unit 50 continues the reproduction of the sound data being reproduced as it is.

  In step S89, the sound data reproduction control unit 50 identifies whether or not automatic reproduction is stopped. If it is stopped, the process proceeds to step S92, and if not stopped, the process proceeds to step S90.

  In step S90, the sound data reproduction control unit 50 determines that the page skipping state (a state in which the reader is flipping pages continuously). If there is sound data being reproduced, the sound data reproduction control unit 50 stops the reproduction. In this case, recording of the position read by the reader (estimated attention position) is unnecessary and is omitted.

  In step S91, the sound data reproduction control unit 50 stops the target position estimation operation by the target position estimation unit 40. As a result, “automatic playback is stopped”.

  In step S92, the sound data reproduction control unit 50 identifies whether or not the page opened by the current page turning matches the page recorded in step S84 (the page read before). If they match, the process proceeds to step S93, and if they do not match, the process proceeds to step S94.

  In step S93, the sound data reproduction control unit 50 estimates the position of the attention position estimation unit 40 from the previously read position in the page based on the information recorded in the position storage unit 90 in step S84. Resume. If the sound data is being reproduced at that position, the corresponding sound data is also resumed.

  In step S94, the sound data reproduction control unit 50 waits for the next page turning operation.

  When an instruction to move the page to be reproduced (displayed) to the current previous page, that is, “page return” is generated by an input operation of the reader by the input unit 80, the processing after step S101 in FIG. 13 is performed.

  First, in step S101, the sound data reproduction control unit 50 checks the type of sound data corresponding to the case where sound data is being reproduced. If either “background sound” or “sound effect” is being reproduced, the process proceeds to step S102. If neither “background sound” nor “sound effect” is being reproduced, the process proceeds to step S104.

  In step S102, the sound data reproduction control unit 50 identifies whether or not the current “page return” has reached the page where the reproduction start position of the “background sound” or “sound effect” being reproduced exists. If reached, the process proceeds to step S103, and if not reached, the process proceeds to step S104.

  In step S103, the sound data reproduction control unit 50 immediately stops the reproduction (sounding) operation regarding the “background sound” or “sound effect” being reproduced.

  In step S104, the sound data reproduction control unit 50 checks the type of sound data when the sound data is being reproduced, and stops the reproduction of the sound data if it is “Serif”. To do.

  In this way, when page turning or page reversion occurs, sound data playback is stopped under different conditions for each type of sound data being played back, so that sound data suitable for the scene after the page change is played back. It can be performed.

  In other words, in the electronic book device 100 shown in FIG. 1, the sound data reproduction control unit 50 including the sound data reproduction form determining unit determines the sound data reproduction form according to the type of sound data, that is, “background sound”, “effect”. Sound data corresponding to the sentence data at the target position estimated by the target position estimation unit 40 is determined according to the classification of “sound”, “serif”, and “speech for reading the text”, and is reproduced in the determined reproduction form. Control to do.

  Therefore, the electronic book device 100 can reproduce the sound under the optimum conditions for each type of sound data, and the author of the electronic book uses the sound by properly using a plurality of types of sound data in various situations. Can express the intended situation in a rich manner.

  That is, as shown in Table 1, by performing stop control according to the type of sound data, it is possible to prevent a sound that does not match the situation of the scene from being output. In addition, during playback of sound, by pausing or resuming the estimation operation of the target position according to the type of sound data, the error between the position actually observed by the reader and the estimated position can be reduced. Get smaller. Further, by determining whether or not to superimpose a plurality of sounds according to the type of sound data, it is possible to output a sound suitable for the situation of the scene even when a plurality of sound data is reproduced simultaneously. Furthermore, by adjusting the volume according to the type of the sound data, it becomes possible to make it easy to hear the contents of the speech or to output the sound at a volume suitable for the situation of the scene. In addition, when sound data is being played in response to a user's page turning or page back operation, a sound that does not match the situation of the scene is output by controlling the stop position according to the type of sound data. Can be prevented.

  The present invention is useful for an electronic book device, an electronic book method, and an electronic book program capable of reproducing sound data with rich expressiveness as intended by the author.

DESCRIPTION OF SYMBOLS 10 Text data memory | storage part 20 Sound data memory | storage part 25 Data communication part 30 Text / sound data analysis part 40 Attention position estimation part 45 Clock part 50 Sound data reproduction | regeneration control part 60 Page switching process part 70 Output part 80 Input part 90 Position memory part 100 Electronic book device

Claims (12)

An electronic book device that reads and reproduces data of an electronic book including text data including character string information constituting a sentence and sound data corresponding to the text data,
An attention position estimation unit for estimating an attention position in the text data;
A sound data reproduction form determination unit for determining a reproduction form of the sound data based on the type of the sound data;
A sound data reproduction control unit that controls to reproduce the sound data corresponding to the text data of the attention position estimated by the attention position estimation unit in a reproduction form determined by the sound data reproduction form determination unit;
An electronic book device comprising: The electronic book device according to claim 1,
The sound data reproduction form determination unit
As the type of the sound data, at least one of background sound, sound effect, speech speech, and text reading speech is recognized,
An electronic book device that determines at least one of a reproduction stop condition, a reproduction repetition count, a reproduction volume, and an influence of reproduction of sound data on the estimation of the target position as a reproduction form of the sound data. The electronic book device according to claim 2,
The sound data reproduction form determination unit is an electronic book device that determines that the sound data is speech of speech when the text data corresponding to the sound data includes a character indicating a quotation mark. The electronic book device according to claim 2,
The sound data reproduction form determination unit is an electronic book device that determines that the sound data is a sound effect when the text data corresponding to the sound data includes an onomatopoeia. The electronic book device according to claim 2,
The sound data reproduction form determination unit determines whether the sound data is a background sound or a sound effect based on a reading time based on the number of characters of the body data corresponding to the sound data and a reproduction time of the sound data An electronic book device for determining The electronic book device according to claim 2,
The sound data reproduction control unit stores that the sound data corresponding to the text data of the target position estimated by the target position estimation unit is a background sound, and stops estimating the target position in the management table. An electronic book device that interrupts the estimation of the target position by the target position estimation unit. The electronic book device according to claim 2,
When the sound data corresponding to the body data of the target position estimated by the target position estimation unit is a dialogue, the sound data reproduction control unit interrupts the estimation of the target position by the target position estimation unit. Book device. The electronic book device according to claim 1,
When the sound data reproduction control unit is assigned a plurality of the sound data corresponding to the same target position estimated by the target position estimation unit, the sound data reproduction control unit An electronic book device that restricts playback of sound data. The electronic book device according to claim 1,
When the sound data reproduction control unit receives a page turning instruction, the sound data reproduction control unit stops reproduction of the sound data under different conditions based on the type of the sound data being reproduced. The electronic book device according to claim 1, further comprising a data communication unit that receives the sound data from an external device. An electronic book reproducing method for reading and reproducing electronic book data including text data including character string information constituting a sentence and sound data corresponding to the text data,
Estimating a position of interest in the text data;
Determining a reproduction form of the sound data based on a type of the sound data;
Controlling to reproduce the sound data corresponding to the body data of the estimated position of interest in the determined reproduction form;
An electronic book reproducing method comprising: The electronic book program for performing each step of the electronic book reproduction | regenerating method of Claim 11.

Images