TW408284B - Method and apparatus for displaying characters with voice therefore - Google Patents

Abstract

The invention is a method for displaying characters with voice and apparatus therefore, which discloses a technique that determines the operation content for character display (such as color change or agitation) according to the character voice length of each character section to adjust the speed and rhyme of display operation automatically; according to the technique disclosed in the invention, it includes: a memory unit which stores the respective data of the characters and one table for the voice of characters (such as the vowel of English characters), a logical process unit which is used to look up the voice length corresponding to the presently displayed character from the voice length table corresponding to each character before scanning, and calculate the display operation speed of each word in the display with respect to the related data of other displays, display each word on the display with respect to the respective voice speed via a display unit (such as the vowel of English characters).

Description

408234 V. Description of the invention (1) [Scope of the invention] The present invention relates to a method and a device for displaying subtitles with pronunciation, especially a technical field suitable for displaying subtitles in a technical field, based on the pronunciation of characters in each word. The method of calculating the display speed of each word, and then performing the display operation on each word at its own display speed. [Background of the Invention] At present, in some entertainment or learning optical discs or software products, a function is often required for users to read and record subtitles while playing media such as video and audio. When reading and recording, it is necessary to scan the subtitles to change the color to guide the user to read the correct speech rate. In some multimedia teaching CD-ROM products (such as BBC English), the method used is to scan each subtitle at the same speed, that is, to average the scan time according to the display length (or number of letters) of each single word in the subtitle. For example, scanning subtitles 11 T hisisab ο 〇k ": Suppose the pronunciation of the entire sentence, that is, the scanning time is 1 second, so the sentence has a total of 11 letters. If you use the above scanning method, the scanning time assigned to each letter is 1/1 1 Seconds, so the scan time for the word "M Th is" is 1/1 1 X 4 = 4/1 1 second. Similarly, " is 11 、 11 a11 、 " b ο 〇k M's scan speed is 2 / 1 1 second, 1/1 1 second, 4/1 1 second, it can be seen that the scan time ratio of "T hi S" to "a" is 4: 1. Obviously, the two single-word There is no such big difference in the actual pronunciation time length. Reading at this speed will feel that "Th is" too long and "a" is too short. Therefore, the disadvantage of this method is that the scanning speed and natural speech speed (actually Pronunciation speed) are inconsistent, causing users to fail to follow the correct speaking rate Subtitle

Page 4 408284 V. Description of the invention (2) In order to improve the above-mentioned shortcomings, it is necessary to use manual editing methods, that is, to listen to the pronunciation of the subtitles and control the scanning speed of color changes one by one. After repeated adjustments, the scanning progress can be made at the same speed as natural speech, but it is clear that this kind of work is very complicated and heavy. [Objective and Summary of the Invention] Therefore, the main object of the present invention is to provide a method for displaying subtitles with pronunciation. The method disclosed in the present invention can automatically adjust the scanning speed and rhythm (that is, the aforementioned display operation speed), and at the same time Users can also be guided to follow the scanning speed of subtitles to follow or sing at the natural language speed, which can improve the conventional color-changing scanning method using manual editing, which requires considerable time to adjust the inconvenience of scanning speed. According to the technology of the present invention, it is disclosed that a display operation content (such as color change or pop-up) can be determined according to the pronunciation length of characters in each subtitle (which can be English, Chinese or other language characters), so as to A technology for automatically adjusting the speed and tempo of the display operation according to the present invention. The technology includes the following: providing a memory unit in which related data of subtitles are stored (including sentences, words, and The corresponding _ pronunciation symbol, the start time and end time value of each subtitle paragraph) and a one-character pronunciation length table (such as the English alphabet sound length table) provide a logical processing unit for Before scanning, find out the corresponding number of pronunciation length units from the pronunciation length table of a character according to each single character (consisting of more than one character) in the subtitle paragraph, and calculate it in accordance with the relevant data of other subtitles. Display the display of each word in the subtitles

408284 V. Description of the invention (3) Display operation speed. By controlling a display unit, each word in the subtitle is displayed according to its own sounding speed (such as color changing or popping processing). In order to make the above and other purposes, features, and advantages of this creation more comprehensible, a preferred embodiment is given below in conjunction with the accompanying drawings for detailed description as follows: [Schematic description of the drawings] Figure 1, It is a block diagram of the first embodiment of the apparatus for displaying subtitles with pronunciation according to the present invention. Fig. 2 is a block diagram of a first embodiment of a device for displaying subtitles with pronunciation according to the present invention. Fig. 3 is an operation flowchart of the method for displaying subtitles with pronunciation in accordance with the present invention. Figure 4 shows that the "Thi s i s s b ο ○ k" color scan has not been scanned. Figure 5 shows that the word "book" is being scanned. Figure 6 shows that "This is s book" has been scanned for color changes. [Explanation of the embodiment] The meanings of the nouns in the subsequent description are explained as follows: Pixels: the smallest unit constituting an image on the screen; the display length of a word: the length of a word displayed on the screen, which is expressed in pixels (pi X e 1) is the unit; scanning speed: its unit is milliseconds / pixels (milliseconds / pi X e 1), which is

408234 V. Description of the invention (4) Refers to the time taken to scan each pixel. Please refer to "Fig. 1", which discloses a device for implementing the method of the present invention, which includes: a memory unit 10 for storing related data of subtitles, wherein the subtitles can be English, Chinese, or other language subtitles, In the embodiment disclosed in the present invention, English subtitles are used as an example. As for the implementation methods of subtitles in other languages, they are similar to English. I will not go into details here. In this embodiment, the English subtitles are related. The data is stored into three databases, each of which is a subtitle database, as shown in "Table 1 J, which contains the English sentences in the subtitle paragraphs, and the start and end playback time of each subtitle paragraph; a dictionary library, "As shown in Table 2", which contains the English words and corresponding phonetic symbols in the subtitle paragraphs; and an English vowel sound length table, as shown in "Table 3", which includes the number of pronunciation length units for all English vowels, and Regarding the number of vowel length units of English vowels, the rule is: take the vowel length of vowel [I] as the comparison standard, and set it as one vowel length unit (also Set to other values), and then measure the pronunciation length of other vowels and compare with the pronunciation length of [I], you can get the pronunciation length of other vowels. Of course, it is not limited to using the vowel [I] as The comparison standard can also be other vowels as the comparison standard. As for which vowel, it depends on the user.

_408284 V. Description of the invention (5) Table 1 ”The start time (seconds) and end time (seconds) of each subtitle paragraph This is a book. 15:00 16:30 That is a pencil. 18:00 19:35 Table 2 "Single word phonetic symbol a [9] book [buk] is [iz] pencil [p £ ns!] That [red t] this [sen] […]

Page 8 408234 V. Description of the invention (6) "Table 3 Number of vowel vowel sound length units i: 1.5 a: 1.4 Ό: 1.5 u: 1.5 孓 1.4 I 1 e 1 ae 1 Λ 1 0 1 u 1 a 1 el 1 al 1.3 Ol 1.3 su 1.3 au 1.3 Id 1.3 QB 1.3 ua 1.3 1111 408234 V. Description of the invention (7) A logical processing unit 3 0 is used to extract data from the memory unit 10 and calculate according to the method disclosed in the present invention. Display speed of each single character (composed of more than one English letter);-Display unit 40, which can be a screen, LCD screen, touch screen, or one of other functions similar to each other Single words are displayed at their respective pronunciation speeds (can be changed color or pop-up processing). Of course, some units can be added to the above device to enhance the advantages of the present invention. For example, an output unit 50 can be provided for While the display unit 40 performs a display operation on the subtitles, a corresponding pronunciation is output to teach the user to read the correct pronunciation: or a memory storage unit 60 is provided to store the display unit 40 which has been displayed. Display the relevant data of the subtitle section of the operation to increase the speed of data extraction by the logical processing unit 30, as shown in "Figure 2". In addition, if it is to increase the visual or auditory effect, an MPEG (Movie Picture Expert Group, an image compression format standard) image medium can be stored in the memory unit 10, and displayed or output by the display unit 40 or the output unit 50. The corresponding image or sound is used as a guide to the day or background music. The operation of the entire device will be described with the device of "Figure 1" and the flow chart of "Figure 3": Step 1: Start the program Step 2: Put a subtitle library, a dictionary library, and a character The pronunciation length table (such as the pronunciation length table of English vowels) is stored in a memory unit.

Page 10 V. Description of the invention (8) Set the current subtitle paragraph number N to 1, where if the PEG image media is stored in the memory unit, the media can be played here; Step 3: The logic processing unit will start Stop comparing the current time with the start playback time of each subtitle section in the memory unit. If the current time is greater than or equal to (> =) the start playback time of a subtitle section, proceed to the next step, and vice versa , Then proceed to step 3; if there is an MPEG video media playing, the logical processing unit will compare the current playing time with the start time of each subtitle paragraph in the memory unit to determine whether the media playing time is greater than Equal to (> =) the start time of one of the subtitle paragraphs, if yes, proceed to the next step, if no, continue to this step 3; step 4: the display unit will display all the characters in the subtitle paragraph ; Step 5: The logical processing unit will find the corresponding pronunciation from the pronunciation length table of one character according to each single word in the subtitle paragraph (which is composed of more than one character). Length, together with the relevant data of other subtitle paragraphs (such as the data of the subtitle library and dictionary database), so as to calculate the display operation speed of each word in the subtitle paragraph; Step 6: the display unit will Display operations are performed at respective pronunciation speeds. At this time, if there is an output unit, the output unit will play the corresponding pronunciation or music while the display unit performs the display operation. Step 7: The logic processing unit will determine whether the subtitle paragraph is The last subtitle paragraph, if yes, go to step 9, if no, continue to the next step;

Page 11 408284 V. Description of the invention (9) Step 8: Set the subtitle paragraph number N +1 = 2 and return to Step 3; Step 9 · End. The detailed steps for calculating the display operation speed of each word from the pronunciation length table of one character in step 5 are as follows: In order to facilitate the understanding of the following formulas, we will first explain the relevant symbols: C Subtitle paragraph playing time B: Subtitle paragraph starting playback time S: Subtitle paragraph playing length E: Subtitle paragraph playing end time A: This subtitle paragraph makes the sum of the number of pronunciation length units of all words A1: No. 1 Number of pronunciation length units per word A2: Number of pronunciation length units per second word A3: Number of pronunciation length units per third word

An: the number of sound length units of the ηth character t: the time length of the sound length unit of a character in the subtitle paragraph t1: the display operation time of the first character t2: the display operation time of the second character t3 : Display operation time of the third character t η: Display operation time of the nth character 1 1: Display length of the first character 1 2: Display length of the second character i 3: Display length of the third character 1 η: display length of the ηth word

Page 12 408284 V. Description of the invention (ίΟ) V 1 Display operation speed of the first single character V 2 Display operation speed of the second single character V 3 Display operation speed of the third single character V η Display operation of the n single character Speed Step 1 Calculate the playback length (S) of the subtitle paragraph, which is calculated as: S = Ε--B Step 2 Calculate the sum of the pronunciation length units of all the words in the subtitle paragraph according to the pronunciation length table of the Zhao character (A) This system finds the pronunciation symbols of each word in the dictionary library and counts the number of types of pronunciation symbols it contains. Then, it searches the pronunciation length table of the character to calculate the number of units of pronunciation length contained in it. Its calculation formula is A = A 1 + A 2 + A 3 + + A η Step 3 The length of time (t) is calculated as t = S / A. Step 4 Calculate the display operation time of each word in the subtitle segment Α-Α. The calculation formula is t 1 = A 1 X tt 2 = A 2 X tt 3 = A 3 X tt η = A η X t Step 5 Calculate the display operation speed of each single word in the subtitle paragraph. The calculation formula is: V 1 = 1 1 / t 1 V 2 1 2 / t 2 V 3 = 1 3 / t 3 V η = 1 η / t η 〇

Page 13 408284} Neilaiiss} pre-play as the screen change screen (such as η example si ^) the word calculation step of the broadcast room fire 1 yuan Shi hllsM falling segment is the time to continue the step-Shan Shi " TTh color segment curtain energy can be shown first and then counted. Show one > = ^ '' Changing the screen word can make the show need to be performed. • If the word is in this second, the yuan will come, and its 3 will be represented by the current table. 5} Go to the front of the color, 1 pick up, and go to show 1 on the other side as Yan Zhifa. Before going straight, IJMC finds the place 2 in the day before the display, which can be taken too seriously. 1 This calculation indicates that the time of the 爿 喿 η table is compared with the time of the first count, the number of seconds such as C, such as C, and the beginning of the I. The second chart of 5 is only 1 and the second list is from 亍 Γ The 5th tone 1 has this, the upper C is reasonable, and it will be placed in the light from the location where the 1st scene is located. The long-distance analysis series Mr-Wubo is the time. J, -Understanding the clear logic of Chuluo early in the morning, or this clear-cut character in Luoming ffl, the paragraph is more 4-speed characters. Let ’s take a closer look at the falling second scene. 11) Just know what you want. This 5 episodes of this series can be seen from the beginning. It ’s clear when the logic is broadcast. In order to say okok, the first word 6 is shown, and the word M and the first broadcast with bobo will open a 1-sweep order: Μ It is a department plus aa, let ’s set another color segment.

Page 14 408284 V. Description of the invention (12) = 1 + 1 + 1 + 1.5 = 4.5; 3. The time length of a vowel length unit in this subtitle paragraph (t) = S / A = 1.5 / 4.5 = 0.33 seconds; 4. The scanning time of each word in the subtitle paragraph:

This: scan time (t 1) = 1 X 0. 3 3 = 0 _ 3 3 seconds; is: scan time (t 2) = 1 X 〇. 3 3 = 0 3 3 seconds; a: scan time (t 3 ) 2 1 X 0. 3 3 = 0.3 3 seconds; book: scan time (t 4) = 1.5x0. 33 = 0.5 seconds; 5. The display length of each single word is: T his · _ 1 1 = 20 pixels; is: 1 2 = 10 pixels; a: 1 3 = 5 pixels; book: 1 4 = 20 pixels; therefore, the scanning speed of each single word is:

This: V 1 = 2 0/0, 3 3 = pixels / 16. 5 ms; is: V 2 = 10 / 0.33 = pixels / 33 ms; a: V 3 = 5/0. 33 = pixels / 66 milliseconds; and book: V 4 = 20/0. 5 two pixels / 25 milliseconds. According to the calculated scanning speed, the display unit scans each single word point by point at the speed of scanning one pixel point every 16.5, 3, 3, 6, 6, and 25 milliseconds. The scanning process is as described in "4. As shown in Figures 5 and 6, when the color scan has not yet been performed, the display unit will display "This is a book" in a dark color (as shown in Figure 4). Bright colors are scanned once for each word,

Page 15 408284 V. Description of the invention (13) As shown in Fig. 5, the display unit has scanned the pixels at a speed of "Th is π", "isn", "" a " These three characters have been scanned for color change, and they are now scanning nb ο 〇k at a speed of 25 milliseconds to scan one pixel. Figure 6 shows that the word "&b; b ο 〇k" has changed color. Scanning is complete, that is, the color scanning process of this subtitle paragraph is completely completed

#, at 会 方 ，， Shows its examples can be issued in 1'aTh system description, only C real world L Where in the scanning process, if there is an output unit, the output unit in the display unit according to Sequential scan "Th is π", "isp,", " b ο ok " and output the corresponding pronunciation at the same time. Among them, if " T hisisab ο 〇k "is still a subtitle paragraph" is a pencil "(as shown in Table 1), based on the data of this subtitle paragraph, repeat the above calculation Mode and sweep until the last subtitle paragraph. Of course, the above-mentioned color-changing scanning method is only one of the display operators. According to their needs, the user terminal can change to other display operation methods such as jumpers or other operation methods. In addition, although the embodiments disclosed in the present invention operate on English subtitles, this does not mean that the technology of the present invention can only be implemented for English subtitles. For example, Chinese, Japanese, or other languages can also be implemented. According to the present invention The disclosed technology can be modified according to the type of subtitle used (such as Chinese or Japanese). Therefore, the scope of protection of the present invention shall be determined by the scope of the attached patent application. Ming effect

Page 16 408284

Page 17

Claims (1)

408284 Six 'Patent Application Scope 1. A method for displaying subtitles with pronunciation, used to display the subtitles currently displayed on the display unit, and control the display operation content according to the pronunciation speed of the characters contained therein, including: (1) A subtitle database, a dictionary database, and a character's pronunciation length table into a memory unit; (2) judging whether the current time is greater than or equal to the starting playback time of a certain subtitle paragraph, and if yes, proceed to the next step, If not, continue with this step; (3) display the subtitle paragraph in a display unit; (4) find out the corresponding length from the character pronunciation length table according to the characters contained in each single word in the subtitle paragraph The length of the pronunciation, together with the relevant data of other subtitle paragraphs, calculate the display operation speed of each word in the subtitle paragraph; (5) Display each word in the subtitle paragraph according to its own pronunciation speed through a display unit ; And (6) determine whether the subtitle paragraph is the last subtitle paragraph, if yes, end the job, if no, return to step (2) ° 2, as applied Please refer to the method of displaying subtitles accompanied by pronunciation as described in item 1 of the patent scope, wherein the display operation of the display unit is to change the color or pop-up of the subtitles. '3. The method of displaying subtitles with pronunciation as described in item 1 of the scope of patent application, wherein the character can be an English character. 4. The method of displaying subtitles with pronunciation as described in item 1 of the scope of patent application Page 18 408284 6. The scope of patent application method, in which the pronunciation length table of the character can be an English vowel pronunciation length table. 5. The method for displaying subtitles accompanied by pronunciation as described in item 1 of the scope of patent application, wherein the calculation method for obtaining the display speed includes: a. Obtaining the play length of the subtitle paragraph; b. Obtaining all the characters of the subtitle paragraph The sum of the number of pronunciation length units; c, obtaining the time length of the pronunciation length unit of a character in the subtitle paragraph; d, obtaining the display time of each word in the subtitle paragraph; and e, obtaining each of the subtitle paragraphs This display operation speed for single words. 6. The method for displaying subtitles with pronunciation as described in item 1 of the scope of the patent application, wherein the subtitle library contains sentences in each subtitle paragraph and the start and end playback time values of each subtitle paragraph. 7. The method of displaying subtitles with pronunciation as described in item 1 of the scope of patent application, wherein the dictionary library contains all the words in each subtitle paragraph and the corresponding pronunciation symbols. 8. The method for displaying subtitles with accompanying pronunciation as described in item 1 of the scope of patent application, wherein the pronunciation length table of the character includes the number of pronunciation length units of all the characters. 9. The method for displaying subtitles accompanied by pronunciation as described in item 8 of the scope of the patent application, wherein the pronunciation length table of the character is completed by the following methods, including: A. Select the pronunciation length of one character to A standard value; Page 19, 408284, the scope of patent application, and B. The pronunciation length of other characters is measured and compared with the pronunciation length of the standard value, and the pronunciation length values of other characters are recorded. 1 0. A method for displaying subtitles with pronunciation, used to display the subtitles currently displayed on the display unit, and control the operation content of the subtitles according to the pronunciation speed of the characters contained therein. The method includes: (1) According to the subtitle paragraph The characters contained in each single character in the character are used to find the corresponding number of pronunciation length units from the one-character pronunciation length table, and then cooperate with other subtitle-related data to calculate the display operation speed of each single character in the subtitle; and ( 2) The display operation is performed on each word in the subtitle paragraph according to its own pronunciation speed through a display unit. 11. The method for displaying subtitles with accompanying pronunciation as described in item Γ0 of the scope of patent application, wherein the character can be an English character. 12. The method for displaying subtitles with accompanying pronunciation as described in item 10 of the scope of patent application, wherein the pronunciation length table of the character can be an English vowel pronunciation length table. 13. The method for displaying subtitles with pronunciation as described in item 10 of the scope of patent application, wherein the display operation of the display unit may be to change the color of the subtitles or pop-up processing. 14. The method for displaying subtitles with accompanying pronunciation as described in Item 10 of the scope of patent application, wherein the pronunciation length table of the character includes the number of pronunciation length units of all the characters. Page 20, 408284 VI. Patent application scope 15, the method for displaying subtitles with accompanying pronunciation as described in item 14 of the patent application scope, wherein the pronunciation length table of the character is completed by the following methods, including: A 1. Select the pronunciation length of one of the characters and set it to a standard value: and-B. Measure the pronunciation length of other characters and compare it with the standard value · and record the pronunciation length of the other characters. 16. The method for displaying subtitles accompanied by pronunciation as described in Item 10 of the scope of patent application, wherein the calculation method for obtaining the display speed includes: a. Obtaining the playback length of the subtitle segment; b. Obtaining all of the subtitle segment The sum of the number of pronunciation length units of the character; c. Obtaining the time length of the pronunciation length unit of a character in the subtitle paragraph; d, obtaining the display operation time of each single word in the subtitle paragraph; and e, obtaining the The speed of this display operation for each word in a subtitle paragraph—degrees. 17. A device for displaying subtitles with pronunciation, used to display the subtitles currently displayed on the display unit, and control the operation content of the subtitles according to the pronunciation speed of the characters contained therein. The device includes: a memory unit for storing The subtitle-related information includes at least a subtitle library, a dictionary library, and a character's pronunciation length Page 21 408284 VI. Patent application scope table, a logical processing unit for calculating the display operation speed of each word in the subtitle paragraph, and a display unit for each word in the subtitle paragraph according to it Display operations are performed at respective pronunciation speeds. 18. The device for displaying subtitles with pronunciation as described in item 17 of the scope of patent application, wherein the character can be an English character. 19. The device for displaying subtitles accompanied by pronunciation as described in item 17 of the scope of patent application, wherein the pronunciation length table of the character can be an English vowel pronunciation length table. 2 〇 The device for displaying subtitles with pronunciation as described in item 17 of the scope of the patent application, wherein the subtitle library contains sentences in each subtitle paragraph and the start and end playback time of each subtitle paragraph. 0 " 2 1. The device for displaying subtitles with pronunciation as described in item 17 of the scope of the patent application, wherein the dictionary library contains all words in each subtitle paragraph and their corresponding pronunciation symbols. 22. The device for displaying subtitles with pronunciation as described in item 17 of the scope of patent application, wherein the pronunciation length table of the character includes the number of pronunciation length units of all the characters. 2 3. The device for displaying subtitles with pronunciation as described in item 22 of the scope of patent application, wherein the pronunciation length table of the character is completed by the following steps, including: A. Select the pronunciation length of one character Set as a standard Page 22 408284 VI. Value of the scope of patent application; and B. Measure the pronunciation length of other characters and compare the pronunciation length with the standard value, and record the pronunciation length values of other characters. 24. The device for displaying subtitles with pronunciation as described in item 17 of the scope of patent application, wherein the device may further include an output unit for outputting corresponding pronunciations while the display unit performs a display operation on the subtitles. . 25. The device for displaying subtitles with pronunciation as described in item 17 of the scope of patent application, wherein the device may further include a memory memory unit for storing the related data of the subtitle and subtitle paragraphs that have been displayed. 26. The device for displaying subtitles with pronunciation as described in item 17 of the scope of patent application, wherein the display unit may be a screen, an LCD screen, a touch screen, or other similar functions. Page 23