JP2010015088A - Data generating device, data generating program, and reproduction system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data generating device for generating linked display data for displaying a text including a character string not provided with reading by a predetermined timing coinciding with sound reproduction. <P>SOLUTION: The data generating device for generating linked display data for displaying a text by a predetermined timing coinciding with sound reproduction generates a language string by extracting a feature parameter from a spectrum component of a sound bandpass included in sound source data, classifies the text into a plurality of character strings for applying a reading to each character string, generates linked display data including the character strings provided with reading and time stamp information indicating a timing of reproduction of the character string in a language string with the same reading, and allots a time zone of a predetermined length to the character string not provided with reading based on the time stamp information of the character string immediately before the character string not provided with reading so that the character string not provided with reading and the time stamp information indicating the time zone allotted to the character string are included in the linked display data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a data generation device, a data generation program, and a playback device that generate linked display data for displaying text at a predetermined timing in accordance with the playback of audio.

  Karaoke devices for singing songs while watching the lyrics displayed on the screen according to the music are widely used. Many karaoke devices display lyrics on the screen as music is played, and smoothly change the display color of the lyrics to be sung at an appropriate timing. For example, the display color of the lyrics is changed from gray to white. Such karaoke equipment uses linked display data for changing the display color of lyrics in accordance with music reproduction, in addition to music data and lyrics data. Patent Documents 1 and 2 describe a technique for creating linked display data.

  Patent Document 1 discloses a file creation apparatus that creates a link table for setting the synchronization timing between audio content and text data such as lyrics. The file creation apparatus includes a link creation unit, which, as shown in FIG. 18, includes a frequency discrimination unit 201, a feature extraction unit 202, a language creation unit 203, a language model database 204, and a language model memory. A section 205, a frame count section 206, an elapsed time calculation section 207, a text division section 208, a text memory section 209, and a table creation section 211.

  The frequency discriminating unit 201 analyzes the frequency spectrum of the music data supplied as the frame data, and extracts the spectrum component of the human voice band. The feature extraction unit 202 performs acoustic analysis on the extracted spectral components to extract feature parameters. The language creation unit 203 compares the feature parameters extracted by the feature extraction unit 202 with the feature parameters of each reference language stored in the speech model database 204 to generate a language string. The speech model database 204 stores feature parameters of each language model such as Japanese syllabary and muddy sound for each language model. The language model memory unit 205 converts the language string generated by the language creation unit 203 into text data and stores it.

  The frame count unit 206 accumulates the supplied number of frames from the start of supply, and supplies the accumulated number of frames to the elapsed time calculation unit 207. The elapsed time calculation unit 207 calculates the elapsed time from the start of playback based on the cumulative number of frames. The text dividing unit 208 divides the text data into a plurality of blocks. The text memory unit 209 stores text data divided into blocks.

  The matching unit 210 compares the text data for each block stored in the text memory unit 209 with the text data of the language string stored in the language model memory unit 205, and creates a table of identification signals at the timing when both match. To the unit 211. The link creation unit 211 acquires elapsed time data from the elapsed time calculation unit 207 at the timing when the identification signal is supplied from the matching unit 210, and sets the elapsed time data in the link table as the reproduction timing of the text block.

  In Patent Document 2, as shown in FIG. 19, lyric information composed of character strings mixed with kanji characters is converted into readings using the conversion dictionary unit 16, and the converted readings are score information such as notes. A lyrics allocating device for allocating to is disclosed. The conversion dictionary unit 16 stores predetermined words and their reading information in association with each other.

JP 2003-280670 A JP 2002-82665 A

  As described above, the file creation apparatus of Patent Document 1 acoustically analyzes a spectral component of a human voice band to extract a feature parameter, and extracts the feature parameter from the feature parameter of each reference language stored in the speech model database 204. Compared with, the language string is generated. For this reason, the language string does not include silent symbols such as punctuation marks and question marks. As a result, even if silence data is included in the text data stored in the text memory unit 209, the silence symbol does not match the language string generated by the language creation unit 203, and therefore, the silence data including the silence symbol is classified. There is a possibility that a suitable synchronization timing is not set for the text block. Further, in the file creation device of Patent Document 1, the time when the language information corresponding to the first character of the text block matches the language information of the extracted language string is set as the comparison start time, so that the mismatch occurs when the silence symbol is included. May be suppressed or avoided in units of text blocks, but if the text block consists only of silence symbols, or if it consists of a significant number of silence symbols, the appropriate synchronization timing may not be set. There is.

  Further, the lyrics allocating device of Patent Document 2 converts the lyrics information into readings using the conversion dictionary unit 16 as described above. However, the lyrics allocating device cannot convert silence symbols such as punctuation marks and question marks that do not have readings because they are not registered in the conversion dictionary unit 16. For this reason, a silence symbol cannot be assigned to the score information.

  In addition, there is a possibility that the lyrics allocating device cannot convert a character string such as a proper noun recorded in the conversion dictionary unit 16 into an accurate reading. It is not preferable to assign a reading different from the original to the musical score information because the lyrics may not be displayed at an appropriate timing. The user can add and delete the contents of the conversion dictionary. For this reason, character strings such as the above-mentioned proper nouns and their readings can be newly added, but this work has to be done by the user who knows the operation method for changing the contents of the conversion dictionary, taking time and effort. .

  An object of the present invention is to generate data for generating interlocking display data for displaying text including a character string to which no reading is given or a character string to which an accurate reading is not given at a predetermined timing in accordance with audio reproduction. An apparatus, a data generation program, and a playback apparatus are provided.

  The present invention is a data generation device for generating linked display data for displaying text at a predetermined timing in accordance with voice reproduction, and extracts a feature parameter from a spectral component of a voice band included in sound source data. A language string generation unit that generates a language string by comparing the extracted feature parameter with a feature parameter of a predetermined language; and text processing that divides the text into a plurality of character strings and gives a reading to each character string Each character string to which the reading is given is compared with the language string, and the target character string and the character string in the language string whose reading matches the target character string are A first data processing unit for generating interlocking display data including time stamp information indicating the playback timing by the elapsed playback time of the sound source data, and no reading was given Based on the time stamp information of the character string immediately before the character string, a predetermined length of time zone is assigned to the character string to which the reading is not given, and the character string to which the reading is not given and the character string assigned to the character string. There is provided a data generation device comprising a second data processing unit including time stamp information indicating a specific time zone in the interlocking display data.

  The data generation apparatus stores a character string that has not been read by the text processing unit or a character string that does not have a matching character string in the language string, and information that indicates the position of the character string in the text. A position management unit, wherein the second data processing unit is configured such that the first data processing unit includes all of the character strings to which the reading included in the text is given and the time stamp information of each character string to which the reading is given. Are generated, and a predetermined time period is assigned to each character string recorded in the position management unit.

  In the data generation device, the second data processing unit may receive the time stamp information of the character string immediately after the end time indicated by the time stamp information of the character string immediately before the character string to be stored by the position management unit. A time length up to the start time shown is calculated, and if the time length is equal to or longer than a predetermined time, a time zone of a predetermined length within the time from the end time to the start time is assigned to the target character string If the time length is less than a predetermined time, at least one of the time zone assigned to the immediately preceding character string and the time zone assigned to the immediately following character string is shortened, and the shortened time zone is , Assign to the target character string.

  In the data generation device, the second data processing unit shortens one of a time zone assigned to the immediately preceding character string and a time zone assigned to the immediately following character string by the predetermined length.

  In the data generation device, the second data processing unit assigns the time zone from the end time to the start time and the shortened time zone to the character string to which the reading is not given.

  In the data generation device, the information indicating the position in the text of the character string recorded in the position management unit indicates the number of characters from the beginning of the text of the character string.

  In the data generation device, the information indicating the position in the text of the character string recorded in the position management unit indicates a row number and a column number in a display area when the text is displayed in a predetermined display form.

  In the data generation device, the information indicating the position in the text of the character string recorded in the position management unit indicates a character string number from the first character string of the text of the character string.

  In the data generation apparatus, the information indicating the position in the text of the character string recorded in the position management unit indicates an order relationship between the character strings classified by the text processing unit included in the text.

  The present invention is a data generation device for generating linked display data for displaying text at a predetermined timing in accordance with voice reproduction, and extracts a feature parameter from a spectral component of a voice band included in sound source data. A language string generation unit that generates a language string by comparing the extracted feature parameter with a feature parameter of a predetermined language; and text processing that divides the text into a plurality of character strings and gives a reading to each character string Each character string to which the reading is given is compared with the language string, and the target character string and the character string in the language string whose reading matches the target character string are A data processing unit for generating linked display data including time stamp information indicating the playback timing by the elapsed playback time of the sound source data, and characters that are not read A special reading assigning unit that gives special readings corresponding to all the language strings, and the data processing unit, based on time stamp information of a character string immediately before the character string to which the special reading is given, Compare the character string with the special reading and the language string, and assign the timing at which the character string in the language string that matches the reading is reproduced to the character string with the special reading; There is provided a data generation device that includes a character string to which the special reading is given and time stamp information indicating a time zone assigned to the character string in the linked display data.

  The data generation device includes a reading estimation unit that estimates reading for any character included in a character string to which reading is not given, and the special reading giving unit can guess reading by the reading estimation unit. The special reading is given to the missing character string.

  In the data generation device, the data processing unit is provided with the special reading for a predetermined time period from the end time indicated by the time stamp information of the character string immediately before the character string to which the special reading is given. Assign to the specified string.

  The present invention provides a data generation program for causing a computer to function as each unit included in the data generation apparatus.

  The present invention provides a playback device that displays a character string that cannot be read in a text at a predetermined timing in accordance with voice playback based on the interlocking display data created by the data generation device or the data generation program. .

  The present invention displays a character string that cannot be read in the text at a predetermined timing in accordance with the reproduction of the voice based on the interlocking display data created by the data generation device or the data generation program, and displays the display portion. Provided is a playback device that starts playback from a character string that cannot be read by using a user-selected location.

  According to the data generation device, the data generation program, and the playback device according to the present invention, the predetermined timing in which the text including the character string to which the reading is not given or the character string to which the accurate reading is not given is adapted to the voice reproduction. It is possible to generate interlocking display data for display with.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. A data generation apparatus according to an embodiment described below generates linked display data for displaying text at a predetermined timing in accordance with reproduction of sound source data, based on sound source data and text data. The text is Japanese or foreign language sentences, lyrics, etc. including punctuation marks and special characters. In addition, when displaying text, for example, the transparency of the display color of the text is changed from 50% to 0%, the display color of the text is changed to another easy-to-see color, or the text is not displayed at all. It is to display from the state.

  When the data generating device generates the interlocking display data, the text is divided into character strings composed of one or more characters. This character string may be a morpheme (minimum unit having meaning in the language), a single character, or a phoneme (minimum unit of sound possessed by the language). The generated interlocking display data includes text data and time stamp information which is unique time information assigned to each character string obtained by subdividing the text. The time stamp information includes a start time for displaying each character string and a time length or an end time until the character string is displayed. These start time and end time are indicated by a predetermined playback elapsed time of the sound.

(First embodiment)
FIG. 1 is a block diagram illustrating a data generation apparatus according to the first embodiment. As illustrated in FIG. 1, the data generation device 100 according to the first embodiment includes an input unit 101, a data generation unit 103, a location management unit 105, a data processing unit 107, and a data storage unit 109. The data generation unit 103, the position management unit 105, and the data processing unit 107 operate by executing a program.

  The input unit 101 is an interface for inputting sound source data and text data to the data generation apparatus 100. The data generation unit 103 performs matching between the sound source data input via the input unit 101 and the text data, and generates linked display data. Details of the matching between the sound source data and the text data by the data generation unit 103 will be described later. The position management unit 105 stores a character string that has not been read during the process of the data generation unit 103 or a character string that does not match text based on sound source data and information indicating the position in the text. It should be noted that a character string that does not match the text based on the sound source data may not match the text based on the sound source data as a result of an accurate reading being not given to the character string in the process of the data generation unit 103. .

  The data processing unit 107 assigns time stamp information to a character string that does not match the text based on the character string and the sound source data to which reading is not applied, and updates the interlocking display data generated by the data generation unit 103. The data storage unit 109 stores the interlocking display data generated by the data generation unit 103 and updated by the data processing unit 107.

  Note that the information indicating the position in the text recorded in the position management unit 105 is shown in any one of a plurality of forms. The plurality of forms are (1) the number of characters from the beginning of the text indicated by the text data, (2) the row number and column number in the display area when the text indicated by the text data is displayed in a predetermined display form, and (3) Character string number from the first character string when the text indicated by the text data is divided into a plurality of character strings by performing morphological analysis processing, etc., (4) By performing morphological analysis processing on the text indicated by the text data Descriptive information based on the order relationship between the divided character strings when divided into a plurality of character strings (for example, a data structure such as VoiceXML which is a spoken dialogue description language is preferably described in an easy-to-understand format). It is.

  FIG. 2 is a flowchart illustrating the operation of the data generation device 100 according to the first embodiment. As shown in FIG. 2, in step S <b> 101, sound source data and text data input via the input unit 101 are sent to the data generation unit 103. Next, in step S103, the data generation unit 103 performs matching between the sound source data and the text data to generate linked display data. Next, in step S105, the data processing unit 107 assigns time stamp information to the character string that cannot be read, and updates the interlocking display data generated in step S103. Next, in step S107, the data processing unit 107 stores the interlocking display data updated in step S105 in the data storage unit 109.

  3 and 4 are flowcharts showing details of step S103 performed by the data generation apparatus 100 according to the first embodiment. As shown in FIG. 3, the data generation unit 103 analyzes the frequency spectrum of the sound source data input in step S101 of FIG. 2, and extracts a spectrum component of the human voice band (step S201). Next, the data generation unit 103 acoustically analyzes the spectrum component extracted in step S201 and extracts a feature parameter (step S203). Next, the data generation unit 103 generates a language string by comparing the feature parameter extracted in step S203 with a feature parameter of a predetermined language (step S205). Next, the data generation unit 103 converts the language string generated in step S205 into text data (step S207). If the feature parameter of the predetermined language used in step S205 is a Japanese feature parameter, the text data of the language string obtained in step S207 is hiragana or katakana.

  Next, the data generation unit 103 divides the text indicated by the text data input in step S101 of FIG. 2 into a plurality of character strings by performing structural analysis processing such as morphological analysis (step S209). If the text data input in step S101 in FIG. 2 is Japanese, there is a high possibility that the text contains kanji or punctuation marks. On the other hand, the text data of the language string obtained in step S207 is hiragana or katakana. For this reason, as shown in FIG. 4, the data generation unit 103 gives a reading to each character string divided in step S209 using a reading conversion dictionary (not shown) (step S211).

  Next, the data generation unit 103 determines, for each character string, whether or not reading is given in step S211 (step S213). The data generation unit 103 registers, in the position management unit 105, the character string that is determined not to be read in step 213 and information indicating the position in the text in the position management unit 105 (step S215). On the other hand, the data generation unit 103 compares the character string determined to have been read in step 213 with the text data of the language string obtained in step S207 (step S217), and the text indicated by the text data of the language string It is determined whether or not there is a character string whose reading matches the target character string (step S219).

  If it is determined in step S219 that the target character string is within the text of the language string, the process proceeds to step S221. If the character string is not within the text of the language string, the process proceeds to step S215. Information indicating the position is recorded in the position management unit 105. In step S221, the data generation unit 103 writes the target character string and time stamp information corresponding to the character string in the interlocking display data. The time stamp information of each character string is determined according to the timing at which the target character string is uttered when the sound source data is reproduced.

  After performing step S221, the data generation unit 103 updates the analysis target position of the text data of the language string to be compared with the character string in step S217 based on the time information (step S223). The data generation unit 103 determines whether all of the character strings classified in step S209 have been recorded in the position management unit 105 in step S215 or written in the interlocking display data in step S221 (step S215). S225) If the process has been performed for all character strings, this sub-function is terminated. Otherwise, the process returns to step S211.

  FIG. 5 is a flowchart illustrating details of step S105 performed by the data generation device 100 according to the first embodiment. As shown in FIG. 5, the data processing unit 107 reads a character string and information indicating the position in the text from the position management unit 105 (step S301). Next, the data processing unit 107 specifies character strings before and after the character string read in step S301 based on the interlocking display data generated in step S103, and the character string after the end time of the previous character string. The time length up to the start time is calculated (step S303).

  Next, the data processing unit 107 determines whether the time length calculated in step S303 is equal to or longer than a predetermined time such as 10 milliseconds (step S305). If the time length is equal to or longer than the predetermined time, the data processing unit 107 proceeds to step S307. If it is less than the time, the process proceeds to step S309. In step S307, the data processing unit 107 assigns a time period of a predetermined length within the time from the end time of the previous character string to the start time of the subsequent character string to the character string read in step S301, and displays the interlocked display. Update the data. That is, in step S307, the character string read in step S301 and the time stamp information indicating the time zone assigned to the character string are written in the interlocking display data.

On the other hand, in step S309, the data processing unit 107 assigns at least one of the two time zones assigned to the character strings before and after the character string so as to assign a predetermined time zone to the character string read in step S301. And the linked display data is updated after assigning the shortened time zone to the character string. When the data processing unit 107 shortens the time zone assigned to the character string before the character string, the data processing unit 107 shortens the end time of the time zone of the character string before the character string by a predetermined time, and is freed by the setting change. Assign the specified time zone to the string. Further, when shortening the time zone assigned to the character string after the character string, the data processing unit 107 delays the start time of the time zone of the character string after the character string by a predetermined time length, and changes the setting. Assign a free time zone to the string.
In addition, when shortening the time zone assigned to the character string before and after the character string, the time zone assigned to either the character string before or after the character string (start character, end character, etc.) You may decide whether to shorten.

  The data processing unit 107 advances the end time of the time zone assigned to the character string before the character string, and delays the start time of the time zone assigned to the character string after the character string. Thus, a time zone vacated by the setting change may be assigned to the character string. Furthermore, the time length shortened from the character strings before and after the character string may be adjusted according to the time length calculated in step S303. That is, the time zone of the character string before and after the character string is set so that the sum of the time length calculated in step S303 and the time length obtained by shortening the character string before and after the character string becomes a predetermined length. It may be shortened.

  After step S307 or step S309 has been performed, the data processing unit 107 determines whether or not the above processing has been performed on all character strings registered in the position management unit 105 (step S311), and all character strings. If the process is performed, the subfunction is terminated. Otherwise, the process returns to step S301.

  As shown in FIG. 6, the data generating apparatus 100 of the present embodiment may be connected with a reproducing apparatus 50 that reproduces sound source data or displays text based on linked display data in accordance with an instruction from the user. good. The playback device 50 includes an operation receiving unit 51, a playback control unit 53, a playback unit 55, and a display unit 57. The operation receiving unit 51 receives an operation of the playback device 50 by the user, and the playback control unit 53 performs a process according to the operation content. By this processing, the playback unit 55 plays back the sound source data, and the display unit 57 displays text. At this time, based on the interlocking display data stored in the data storage unit 109 of the data generation device 100, the playback control unit 53 selects a character string that cannot be read in accordance with the playback of the sound source data or that is not accurately read. Processing is performed so that the contained text is displayed at a predetermined timing. FIG. 7 is a diagram illustrating an example of text displayed on the display unit 57.

  Further, when a touch panel is provided on the display unit 57 as one form of the operation receiving unit 51, the user cannot give a reading displayed on the display unit 57 or a desired text in a text including a character string that is not correctly read. Can be selected. In response to the operation, the reproduction control unit 53 reproduces the sound source data from the reproduction time corresponding to the selected character string and performs processing to display the text. The unit of the character string to be selected may be either a character string delimited by morphemes or a display line on the display unit 57. In addition, if there is a character string consisting only of character strings that are not given readings to each character string to be selected, by including it in the character strings to which the previous and next readings are given, May include a character string with reading.

  As described above, according to the data generation device 100 of the present embodiment, a predetermined time period is assigned to a character string to which no reading is given or a character string to which no accurate reading is given. Therefore, in the display of text based on the interlocking display data generated and updated by the data generation apparatus 100 of the present embodiment, character strings represented by silence symbols such as punctuation marks and question marks, or proper nouns and abbreviations are accurately displayed. A text containing a character string that is difficult to read is displayed at a timing close to the actual playback of the sound source. Furthermore, the user can designate such a character string as a reproduction start point.

(Second Embodiment)
FIG. 8 is a block diagram illustrating a data generation apparatus according to the second embodiment. As illustrated in FIG. 8, the data generation device 110 according to the second embodiment includes an input unit 111, a data generation unit 113, a special reading provision unit 115, and a data storage unit 117. Note that the data generation unit 113 and the special reading provision unit 115 operate by executing a program.

  The input unit 111 is an interface for inputting sound source data and text data to the data generation device 110. The data generation unit 113 performs matching between the sound source data input via the input unit 111 and the text data in the same manner as in the first embodiment, and generates linked display data. The special reading assigning unit 115 assigns a special reading to the character string to which no reading is given in the process of the data generation unit 113. The “special reading” is “* (corresponding to a wild card in an operating system such as Linux (registered trademark))” corresponding to all the language strings. The data storage unit 117 stores the interlocking display data generated by the data generation unit 113.

  FIG. 9 is a flowchart illustrating the operation of the data generation device 110 according to the second embodiment. As shown in FIG. 9, in step S <b> 111, sound source data and text data input via the input unit 111 are sent to the data generation unit 113. Next, in step S113, the data generation unit 113 performs matching between the sound source data and the text data to generate linked display data. Next, in step S115, the data generation unit 113 stores the interlocking display data generated in step S113 in the data storage unit 117.

  10 and 11 are flowcharts illustrating details of step S113 performed by the data generation device 110 according to the second embodiment. As shown in FIGS. 10 and 11, the data generation unit 113 performs steps S201 to S213 and steps S217 to S225 described with reference to FIGS. 3 and 4 in the same manner as in the first embodiment. In this embodiment, after step S213 shown in FIG. 11 is performed, the special reading assigning unit 115 assigns a special reading to the character string to which no reading is given (step S251). Next, the data generation unit 113 compares the special reading with the corresponding language string after the end time of the character string immediately before the character string to the character string to which the special reading is given (step S253). It is determined whether or not there is a language string that matches the reading (step S255). Thereafter, if the corresponding language string matches the special reading, the process proceeds to step S257, and if not matched, the process proceeds to step S259. In step S257, the time information of the corresponding language string is assigned to the character string to which the special reading is given, and the process proceeds to step S221. On the other hand, in step S259, after assigning a predetermined time period starting from the end time of the character string immediately before the character string to which the special reading is given, the process proceeds to step S221. In this embodiment, when it is determined in step S219 that the target character string is not included in the text of the language string, the process proceeds to step S253, and the data generation unit 113 performs the same process.

  As described above, according to the data generation device 110 of the present embodiment, a special reading is given to a character string to which no reading is given, and therefore punctuation marks and question marks are not performed without updating the interlocking display data. Data for interlocking display of text including a character string represented by a silent symbol such as can be generated. Note that the playback device 50 may be connected to the data generation device 110 of the present embodiment as in the first embodiment.

(Third embodiment)
FIG. 12 is a block diagram illustrating a data generation apparatus according to the third embodiment. As illustrated in FIG. 12, the data generation device 120 according to the third embodiment includes an input unit 121, a data generation unit 123, a reading estimation unit 125, a special reading assigning unit 127, and a data storage unit 129. . Note that the data generation unit 123 and the reading estimation unit 125 operate by executing a program.

  The input unit 121 is an interface for inputting sound source data and text data to the data generation device 120. The data generation unit 123 performs matching between the sound source data input via the input unit 121 and the text data in the same manner as in the first embodiment, and generates linked display data. The reading estimation unit 125 estimates reading of a character string to which readings are not given in the course of processing in the data generation unit 123. The special reading giving unit 127 gives a special reading to the character string that the reading guessing unit 125 could not guess. Note that the “special reading” is “* (corresponding to a wild card in an operating system such as Linux (registered trademark)”) corresponding to all the language strings as in the second embodiment. The data storage unit 129 stores the interlocking display data generated by the data generation unit 123.

  FIG. 13 is a flowchart illustrating the operation of the data generation device 120 according to the third embodiment. As shown in FIG. 13, in step S 121, the sound source data and text data input via the input unit 121 are sent to the data generation unit 123. Next, in step S123, the data generation unit 123 performs matching between the sound source data and the text data, and generates interlocking display data. Next, in step S125, the data generation unit 123 stores the interlocking display data generated in step S123 in the data storage unit 129.

  14 and 15 are flowcharts illustrating details of step S123 performed by the data generation device 120 of the third embodiment. As illustrated in FIGS. 14 and 15, the data generation unit 123 performs steps S201 to S213 and steps S217 to S225 described with reference to FIGS. 3 and 4 in the same manner as in the first embodiment. In this embodiment, after step S213 shown in FIG. 15 is performed, the reading estimation unit 125 estimates reading of a character string to which reading is not given (step S271). In step S273, it is determined whether or not the reading estimation unit 125 has estimated reading in step S271. If reading can be estimated, the process proceeds to step S217. If not, the process proceeds to step S275.

  In step S275, the special reading giving unit 127 gives a special reading to the character string that the reading guessing unit 125 could not guess. Next, the data generation unit 123 assigns a predetermined length of time zone starting from the end time of the character string immediately before the character string to the character string to which the special reading is given (step S253), and then to step S221. move on. In this embodiment, as in the second embodiment, when it is determined in step S219 that the target character string is not in the text of the language string, the process proceeds to step S253, and the data generation unit 123 is executed. Performs the same process.

  FIG. 16 is a flowchart illustrating details of step S271 performed by the reading estimation unit 125 in the third embodiment. As illustrated in FIG. 16, the reading estimation unit 125 analyzes whether a phoneme registered as a reading is included for each character included in a character string to which reading is not given (step S <b> 401). The reading estimation unit 125 performs the analysis using a phoneme list, an example of which is shown in FIG. Next, the reading estimation unit 125 determines, for each character included in the character string, whether the phoneme corresponding to each character is included in the phoneme list (step S403). If included, the process proceeds to step S405. If not, the process proceeds to step S407.

  In step S405, the reading estimation unit 125 sets a phoneme corresponding to the targeted character as a reading. On the other hand, in step S407, the reading estimation unit 125 does not set the reading for the target character. After step S405 or step S407 is performed, the reading estimation unit 125 determines whether or not the above processing has been performed on all characters included in the target character string (step S409). On the other hand, if the process is performed, the subfunction is terminated. Otherwise, the process returns to step S401.

  As described above, according to the data generation device 120 of the present embodiment, for a character string to which reading is not given, a reading is inferred and compared with text data in a language string. Data for interlocking display of text including character strings can be generated more appropriately. In addition, since special readings are given to character strings that cannot be guessed, similar to the second embodiment, data for interlocking display of text including character strings represented by silent symbols such as punctuation marks and question marks is displayed. Can be generated. Note that the playback device 50 may also be connected to the data generation device 120 of this embodiment, as in the first embodiment.

  The data generation device according to the present invention generates linked display data for displaying text including a character string to which no reading is given or a character string to which an accurate reading is not given at a predetermined timing in accordance with the reproduction of the voice. It is useful as a device to perform.

1 is a block diagram illustrating a data generation device according to a first embodiment; The flowchart which shows operation | movement of the data generation apparatus of 1st Embodiment. The flowchart which shows the detail of step S103 which the data generation apparatus of 1st Embodiment performs The flowchart which shows the detail of step S103 which the data generation apparatus of 1st Embodiment performs The flowchart which shows the detail of step S105 which the data generation apparatus of 1st Embodiment performs The block diagram which shows the structure by which the reproducing | regenerating apparatus was connected to the data generation apparatus of 1st Embodiment The figure which shows an example of the text displayed on the display part of a reproducing | regenerating apparatus The block diagram which shows the data generation apparatus of 2nd Embodiment The flowchart which shows operation | movement of the data generation apparatus of 2nd Embodiment. The flowchart which shows the detail of step S113 which the data generation apparatus of 2nd Embodiment performs. The flowchart which shows the detail of step S113 which the data generation apparatus of 2nd Embodiment performs. The block diagram which shows the data generation apparatus of 3rd Embodiment The flowchart which shows operation | movement of the data generation apparatus of 3rd Embodiment. The flowchart which shows the detail of step S123 which the data generation apparatus of 3rd Embodiment performs. The flowchart which shows the detail of step S123 which the data generation apparatus of 3rd Embodiment performs. The flowchart which shows the detail of step S271 which a reading estimation part performs in 3rd Embodiment. Figure showing an example of a phoneme list The block diagram which shows the internal structure of the link preparation part with which the file preparation apparatus disclosed by patent document 1 is provided. The block diagram which shows the internal structure of the lyrics allocation apparatus disclosed by patent document 2

Explanation of symbols

100, 110, 120 Data generation device 101, 111, 121 Input unit 103, 113, 123 Data generation unit 105 Position management unit 107 Data processing unit 109, 117, 129 Data storage unit 115, 127 Special reading assigning unit 125 Reading estimation unit 50 playback device 51 operation receiving unit 53 playback control unit 55 playback unit 57 display unit

Claims (15)

A data generation device that generates interlocking display data for displaying text at a predetermined timing in accordance with audio reproduction,
A language string generating unit that extracts a feature parameter from a spectral component of a voice band included in the sound source data and compares the extracted feature parameter with a feature parameter of a predetermined language;
A text processing unit that divides the text into a plurality of character strings and gives a reading to each character string;
Each character string to which reading is given is compared with the language string, and the target character string and the character string in the language string whose reading matches the target character string are reproduced. A first data processing unit for generating data for interlocking display including time stamp information indicating timing by reproduction elapsed time of the sound source data;
Based on the time stamp information of the character string immediately before the character string to which no reading is given, assigning a predetermined time zone to the character string to which the reading is not given, and the character string to which the reading is not given A second data processing unit including time stamp information indicating a time zone assigned to the character string in the linked display data;
A data generation device comprising: The data generation device according to claim 1,
A character string that is not given a reading by the text processing unit or a character string that does not have a matching character string in the language string, and a position management unit that stores information indicating a position in the text of the character string,
The second data processing unit is for interlocking display in which the first data processing unit includes all of character strings to which readings included in the text are given and time stamp information of each character string to which readings are given. A data generation device characterized by assigning a predetermined time period to each character string recorded in the position management unit after generating data. The data generation device according to claim 2,
The second data processing unit
Calculating the time length from the end time indicated by the time stamp information of the character string immediately before the character string to be stored by the position management unit to the start time indicated by the time stamp information of the character string immediately after,
If the time length is equal to or longer than a predetermined time, a predetermined length of time zone from the end time to the start time is assigned to the target character string,
If the time length is less than a predetermined time, shorten at least one of the time zone assigned to the immediately preceding character string and the time zone assigned to the immediately following character string, the shortened time zone, A data generation device, wherein the data generation device is assigned to the target character string. The data generation device according to claim 3,
The second data processing unit shortens the predetermined time by either one of a time zone assigned to the immediately preceding character string and a time zone assigned to the immediately following character string. apparatus. The data generation device according to claim 3,
The second data processing unit assigns a time zone from the end time to the start time and the shortened time zone to a character string to which the reading is not given. The data generation device according to claim 2,
The data generation apparatus characterized in that the information indicating the position of the character string in the text recorded in the position management unit indicates the number of characters from the beginning of the text of the character string. The data generation device according to claim 2,
The data generation apparatus characterized in that the information indicating the position in text of the character string recorded in the position management unit indicates a row number and a column number in a display area when the text is displayed in a predetermined display form. The data generation device according to claim 2,
The data generation apparatus characterized in that the information indicating the position of the character string in the text recorded in the position management unit indicates a character string number from the first character string of the text of the character string. The data generation device according to claim 2,
The data generation apparatus, wherein the information indicating the position of the character string in the text recorded in the position management unit indicates an order relationship between the character strings classified by the text processing unit included in the text. A data generation device that generates interlocking display data for displaying text at a predetermined timing in accordance with audio reproduction,
A language string generating unit that extracts a feature parameter from a spectral component of a voice band included in the sound source data and compares the extracted feature parameter with a feature parameter of a predetermined language;
A text processing unit that divides the text into a plurality of character strings and gives a reading to each character string;
Each character string to which reading is given is compared with the language string, and the target character string and the character string in the language string whose reading matches the target character string are reproduced. A data processing unit for generating interlocking display data including time stamp information indicating timing by the elapsed reproduction time of the sound source data;
A special reading assigning unit that gives special readings corresponding to all the language strings to character strings to which no reading is given,
The data processing unit compares the character string with the special reading with the language string based on the time stamp information of the character string immediately before the character string with the special reading. The timing at which the matching character string in the language string is reproduced is assigned to the character string to which the special reading is given, and the character string to which the special reading is given and the time zone assigned to the character string And a time stamp information to be included in the linked display data. The data generation device according to claim 10,
It has a reading guesser that guesses the reading for any character included in the character string that has not been given a reading,
The data generation device according to claim 1, wherein the special reading assigning unit assigns the special reading to a character string that the reading estimation unit cannot guess. The data generation device according to claim 10 or 11,
The data processing unit assigns a time zone having a predetermined length from an end time indicated by time stamp information of a character string immediately before the character string to which the special reading is given to the character string to which the special reading is given. A data generator characterized by the above.   The data generation program for functioning a computer as each part with which the data generation apparatus of Claims 1-12 is provided. A playback device for playing back sound while displaying text,
A character string that cannot be read in the text in accordance with the reproduction of the voice based on the data for linked display created by the data generation device according to claim 1 or the data generation program according to claim 13 at a predetermined timing. A playback device characterized by being displayed. A playback device for playing back sound while displaying text,
A character string that cannot be read in the text in accordance with the reproduction of the voice based on the data for linked display created by the data generation device according to claim 1 or the data generation program according to claim 13 at a predetermined timing. The playback apparatus is characterized in that playback is started from a character string that cannot be read by acquiring the user's selection location using the display portion.

Images