TW201120834A - Audio-visual synthesis interaction system, its method, and its computer program product. - Google Patents

Abstract

The present invention discloses an audio-visual synthesis interaction system, its method, and its computer program product, which is suitable for use in the audio-visual synthesis interaction system of electronic paper, story machine, and network, and whose synthesis method includes: at first, receiving and transforming the printed content such as the book or magazine through the receiving end into the figure content recognizable by the disclosed system; subsequently, loading from the database the audio-visual data relative to the above-mentioned figure content, in which the audio-visual data can comprise the audio data, the visual data, and the data for controlling the physical character; finally, combining the visual data and the audio data loaded from the database into the multimedia audio-visual module, and then outputting to the output end to carry out the action of multimedia broadcasting or controlling of the physical teaching material.

Description

201120834 VI. Description of the invention: [Technical field of invention] The right-wing moon is used in a technology of interactive audio-visual interaction. In particular, it is a sound effect, a sound effect, a sound effect, a time-sensitive video, and a scene video. The role of video or - special effects = video and other audio and video and audio and video synthesis system, its methods and computer program products. [Prior Art] Speech synthesis refers to the artificial generation of human speech, and speech synthesis can be realized by software or hardware. The technology of converting text into speech is one of the important tools in the new generation of human-machine interface. It can be widely used in visually impaired people who do not want to trouble others, read non-word-making books, record broadcast information at stations or airports, and train staff in company training. Different situations, such as data or audio dictionary pronunciation data, and the most commonly used technology should be text-to-speech (TTS) technology. For humans, language is a very important communication tool for talking, sharing, and discussing. Research indicates (Jensen, 1998) that even infants who are in strong can listen to the language even if they still can't speak, all these languages Whether it is understood or not, it will help to develop grammar, vocabulary and meaning in the future. The Department of Applied Arts has a "structure of electronic audio books" as disclosed in the Republic of China New Patent No. 590288. This structure changes the traditional book to provide a static reading method, which combines hardware and electronic circuits with a point selection 201120834. The standard produces a function of video and audio playback, and the structure comprises a casing, a circuit module and an antenna embedded in the casing, and an index device connected with the circuit module, and the book is selected by the indexer. The object of the inner page, the object corresponds to the coordinate value of the antenna in the housing, and the circuit module obtains the data of the corresponding coordinate value of the memory, and generates an RF signal through the antenna, and the signal is captured by the indicator. The signal is sent back to the circuit module inside the casing, and the function of remotely playing the image or sound playing is generated, so that the user can select the content of the inner page of the book to listen to the audio and sound effect of the content at the same time. However, most of the conventional text-to-speech technology only converts text data into voice ticks, which are usually output by non-real human mechanical sounds. Even if the semantics are adjusted according to semantic analysis, it still sounds dull and not natural. In order to solve the problem of dull and unnatural output, and then develop a real human voice technology, this technology mostly connects many recorded live human voices in the database, and the output effect is compared with that of ordinary people. Close, so more acceptable to the average person. However, even in the ordinary life, there are many products that can be used for human beings, but human beings are essentially group animals. Talking with others, communicating with each other and conducting various learning activities together is a human nature. Many studies at home and abroad point out Parent-child reading can increase children's language and learn cognitive ability. By telling children about storybooks, you can improve parent-child relationships, language, and cognitive learning. Static reading, graphic recognition and other methods, so that children are more focused, more popular than the popular animation, film, can cultivate children's cognitive and attention development, through the sound of reading, can adapt to 201120834 when resonating, can be benign to stimulate the body and mind, brain Thought development. Relevant research also pointed out that children must be able to learn well and learn well. It takes a lot of time and energy to learn to read. It takes a little time to read a day and it helps a lot. However, many parents hold their children to read together when they are still babies. The baby's reaction is often to open books, swearing, and even crying. The enthusiasm of parents to read together is greatly reduced. Children's attention is not enough. There is no way to keep their attention on the same thing for too long. It is easy to be distracted and active. Parents are monotonous and have no change in their studies. They cannot let their children put their thoughts on books. The story usually has some strings with sound characteristics. In the conventional technology, no matter whether the text-to-speech technology or the human voice is used, the sound characteristics are not taken into consideration, so that the converted voice is not vivid and cannot effectively attract the attention of the child. In the process of books, it is actually a kind of message transmission. The transmission of messages can usually be done in both linguistic and non-verbal ways. In general, we may pay more attention to the message of speech because they are direct, easy to express, think and analyze. In addition, if parents can use body language, they can concentrate on children's attention. Non-verbal behaviors such as facial expressions, gestures, eye contact, body proximity and posture can not be ignored. Therefore, parents can use a voice-conversion technology to turn a story into a voice, so that parents can reduce the burden of language and non-language, and share the language transfer for parents. Therefore, in view of the above problems, it is necessary to propose an audio-visual synthesis 201120834 interactive system, its method and its computer program product, which can read the text content of the printed content or the electronic file of the file from a book, a magazine or a network, and take out the corresponding content. The sound effects and video are converted into immersive multimedia audio and video modules to increase the vividness of reading books. SUMMARY OF THE INVENTION In view of the above problems of the prior art, one of the objects of the present invention is to provide an audio-visual synthesis interactive system, a method thereof, and a computer program φ product, which are digital content generated through the synthesis of sound effects and video. With the appropriate textbooks in a timely manner, the composition of the synthesized multimedia modules and physical characters can be expressed in a more vivid way. According to the object of the present invention, an interactive method for video and audio synthesis is proposed. The method comprises the following steps: first, converting a printed content or an electronic file of a book, a magazine or a web page into a recognizable text content via receiving at the receiving end. Next, the multi-audio data relative to the above text content is loaded from the database, and the audio-visual material may include sound data and video materials. Finally, the video data and sound data loaded from the database are output to the output end of the multimedia audio and video module for instant playback or storage. When receiving the text content from the receiving end, an interactive control signal may be received from the receiving end to generate an entity role control data output to the output terminal according to the interactive control signal, so as to control at least one entity role. Among them, the sound data in the audio-visual data is based on the parameters of the text of the 201120834 received at the current stage t 'extracting the keyword parameters, etc., to synthesize the q character parameters or adjust the color sound effects and adjust the sound effect and the context sound effect. The angle module can be (4) synthesized and the money is slow, the tone is high or low, or the swaying is stunned, 曰: the speed of the i major 2 is quickly adjusted to adjust the sound effect, and the sound effect is the same. Synthetic number, interjection, or help is the punctuation character that appears in the text content.

The video data received is based on the parameters of the current stage, such as 4 time parameters, scene parameters, role 4 2 = effect parameters, etc. to synthesize and correspond to these parameters; "It! Do not video, role Do not video and special effects window, and 13. And the multimedia audio and video module can be adjusted according to the synthesized temple special effect video, and the video effect such as the front view configuration, fade out, or speed can be adjusted when the output is received. When books or miscellaneous printed content from the entity is used, the step of receiving the text content from the receiving end can be added to the key parameters, time parameters, and scene parameters included in the content. The identifiable text content, such as a black color parameter or a special effect parameter, etc. In addition, when the received text content does not include parameters such as a time parameter, a scene parameter, a character parameter, or a special effect parameter, the present invention can use a preset. The video data is combined with the synthesized sound data to be output to the output end of the multimedia audio and video module. The preset video data can be a still landscape. , photos, dynamics, or physical visual effects 8 201120834. In order to make full use of the multimedia audio and video modules generated by combining sound data and video data, it can be output to the output end of the multimedia audio and video module. Storing for later viewing, editing, etc. According to the purpose of the present invention, an interactive audio-visual computer program product is provided, and when the audio-visual composite interactive computer program of the product is loaded into a computer and executed, the audio-visual synthesis interaction of the present invention can be performed. According to the purpose of the present invention, an audio-visual synthesis interactive system is further provided, which comprises: a receiving end, a data base and an audio-video synthesis interactive device. The receiving end is used for receiving input of text content or interactive control signals or a combination thereof. Electrically connected to the receiving end, storing audio-visual data relative to the text content and entity character control data relative to the interactive control signal. The audio-visual material may include a plurality of video data and a plurality of sound data. Connected to the database, after loading video and audio data from the database After combining the video data and the sound data into the multimedia audio and video module, the multimedia audio and video module is output to the output end, and the at least one physical character can be controlled according to the entity role control data. Among them, the printing of the reading entity The content receiving end may further include an optical reading unit and a text content identifying unit. The optical reading unit is configured to receive the printing content input, and the text content identifying unit is configured to identify the received printing content into a text content recognizable by the system. And take out parameters such as keyword parameters, adjustment parameters, time parameters, scene parameters, role parameters or special effect parameters. Among them, the sound data and video data are stored in the database, and the sound 201120834 is at least corresponding to the keyword parameters. Or adjust the parameter's situational sound effect, character sound effect or adjust the sound effect; the video data includes the time parameter, the scene parameter, the character parameter or the special effect video, the scene video, the character video or the special effect video. The device includes a video editor between the receiving end and the database, and is mainly used for receiving the text content, and outputting a basic audio and video data to the database for temporary storage according to the received text content. In the video editor, there is a controller, which is mainly used for receiving the interactive control signal, and generating the entity character control data and storing it in the database. In f, the video and audio synthesis interactive device includes a memory unit, a sound effect < synthesis unit, a video synthesis unit, a sound effect generation unit, a video generation unit, and an output unit. The memory unit temporarily stores the information, the sound data and the entity role control data loaded from the database. The sound synthesis unit and the video synthesis unit are each electrically connected to the memory unit for synthesizing sound data and video data. The sound effect generating unit and the video generating unit 77 are electrically connected to the 0-effect combining unit and the video synthesizing unit, and use the = to generate the combined sound data and the video data. The output unit is electrically connected to the sound generation, the 7L and the video generating unit, and is configured to output the multimedia audio and video module of the sound data and the video material generated by the combined video and video generating unit, and can also output the entity character control data/ , in the middle, for the instant playback of multimedia audio and video modules, transmission devices. In addition, the output terminal may also include a storage device for storing the media media module for later use, playing, editing or copying 201120834. The invention relates to the audio-visual composite interactive line, the method thereof and the computer program product thereof, which may have one or more of the following advantages: (1) key numbers and adjustment parameters extracted by recognizable text content , time parameter, scene parameter = class, etc. corresponding to the multi-media = chess group containing sound effects and video. (2) Through the combination of sound effects and video, it can be more vivid, the books of the original entity or the inconspicuous effects of the original and the media audio and video module that persuaded L °+ can make the user easier to integrate into the text. In the environment, it can solve the problem of reading pure text, easy to be dull, and can be used for learning of children or children to improve the willingness to learn. > (3) Synthesize text content into sound effects by quick completion: media shadow, group, for commercial applications, for theater recording, recording of broadcast messages such as airports or airports, company duties or schools When you study the materials for training, or on your personal life, you can read the dir story time, and the busy housewife wants to read the coughing page while doing family affairs, and can make full use of time. (4) By synthesizing the text content into a containing/multi-audio module, and receiving the interactive control signal to generate real: color control data to control the entity role, in the industrial utilization, can be used to assist the stage supervision to simulate The rehearsal effect, in order to save a wide range of costs, and to reduce the time required for the production of the work. [2011] [Embodiment] Please refer to Fig. 1, which is a flow chart of the interactive method for video and audio synthesis of the present invention. In the figure, the audio-visual synthesis interaction method includes the following steps: First, in step S110, the text content f is received from the receiving end 210: the control signal, the text content may include a book or a magazine, a mobile phone or a webpage substantially having a printed text. Short articles, downloads to e-book novels, or electronic broadcasts stored on electronic devices such as computers. The interactive control signal can be based on the control function of the entity role, and can be a radio frequency identification signal, an infrared control signal or a Bluetooth, and the like. The text content of the receiving end 210 may include two digits of characters and punctuation, and the text content may be composed of a single-word or a plurality of characters, or may be a fixed-unit-segment article, for example, the unit is 20 KB. The electronic file in the form of text, but not limited to this. Depending on the implementation type, after receiving the whole novel, or after receiving the whole novel in a non-fixed unit size segmentation mode, the next step is performed. And, in the step S12G, the video material relative to the text content is loaded from the database 23G. In response to the text, the database 230 stores a plurality of voice data 232, a plurality of video materials 242, and a plurality of entity role control materials 238. In the specific implementation, the following steps may be added to the step S110 to the step S120: Step S112: determining whether the text content received by the receiving end 21〇 has a keyword parameter, a role parameter or an adjustment parameter; Step S114. Extracting the keyword parameter, The role parameter or adjustment parameter 12201120834 corresponding situational sound effect 2321, character-specific sound effect 2323 or adjustment sound effect 2325, etc.; Step S116: further determining whether the text content received by the receiving end 210 has a time parameter, a scene parameter, a character parameter or a special The effect parameter; Step S118: Take out the time-series video 2421, the scene-specific video 2423, the character-specific video 2425, and the special effect video corresponding to the time parameter, the scene parameter, the character parameter or the special effect parameter. In the above embodiment, although it is first performed to determine whether there is a parameter related to the sound effect, and then whether the determination has a video-related parameter, but not limited thereto, it may be performed to determine whether the video-related parameter is performed before executing the The judgment of the sound effect related parameters can also determine the parameters related to the video and sound effects at the same time. After the step S120 is performed, in step S130, the video data 242 and the sound data 232 loaded from the data library 230 are combined into a multimedia audio and video module. In order to combine the plurality of voice resources 232 and a plurality of video materials 242 loaded from the database 230. In the specific implementation, the following steps may be added to the step S120 to the step S130: Step S122: storing various sound effects extracted from the data library 230, such as a situational sound effect 2321, a character-specific sound effect 2323 or an adjustment sound effect 2325, and various video messages, such as Time video 2421, scene video 2423, character video 2425 and special effect video 2427; step S124: synthesize various stored sound effects, such as situational sound effect 2321, character sound effect 2323 or adjustment sound effect 2325; 201120834 Step S126: synthesizing various stored video messages, such as time-in-time video 2421, scene-by-video 2423, character video 2425, and special effect video 2427; step S128: combining various synthesized audio effects with various video signals Output. After the step S130 is performed, the multimedia video module and the entity character control data 238 are output to the output terminal 270 in step S140. In a specific implementation, after the step S140, the generated multimedia audio and video module is added to the output terminal 270. The above-mentioned illustrated video and audio synthesis interaction method is implemented by receiving text content, but may also be appropriately The replacement receiving end 210 is implemented by receiving the sound file or the sound image file recorded by the user itself in advance. This embodiment can modify the sound quality, quality and effect in the previously recorded sound file or sound image file. And other shortcomings, in the commercial application, can effectively use the video and audio synthesis interactive method in the small cost of the dynamic or the post-production of the short film, so as to save production time, cost and improve the quality of the multimedia audio and video produced, due to This video and audio synthesis interactive method can interact with the multimedia audio quality and the physical character, and can simulate the stage effect to reduce the rehearsal time required for the stage supervision, which can make the stage supervision work more efficient. In the above embodiment, although the control of the entity role can be controlled by the output entity role control data 238, in other embodiments, the user can also use other control methods to control the entity role according to his own needs, for example, When the entity role is a physical teaching material with a radio frequency tag attached, the user can use the radio frequency controller to additionally control the real role of the 201120834 body. The video and audio synthesis interactive method can be embodied as an audio-visual composite interactive computer program product. After the computer is loaded into the computer program and executed, the above-mentioned computer system, a separate computer system or other appropriate electronic device can be executed. The steps described in the video synthesis interaction method. For example, when executed on a network system, the electronic file file shared by the online community can be downloaded and downloaded into a format that can be played on a personal playback device. In addition, when executed on a separate computer system, you can refer to the book at hand and make it into a portable computer-readable medium. Specifically, it can be made into a disc or stored in MP3 format on the memory. It is convenient to use whenever you want to listen. β, wherein the technique for converting text data into sound data 232 suitable for the interactive method of video and audio synthesis of the present invention can be performed by a so-called text-t〇-speech (TTS) technique, a human voice technique or the like. Converting the word into a voice technology, converting it, and extracting various sound effects in the sound θ > 232 according to the actual different needs, the character • the sound effect 2323 as an example, may include Male voices, female voices, old voices, earth sounds and cartoon characters. Please refer to FIG. 2, which is a block diagram of the video and audio synthesis interaction system of the present invention. In the figure, the video and audio synthesis interactive system 2 includes, 210, video editor 22, database 23, video = and output 270. ^250, the receiving end 210 is mainly used to receive the input of the text content and the interactive signal. In this embodiment, the optical reading unit ^ 201120834 and the text content identifying unit 214 are used to implement the receiving end 210, via optical reading. The capturing unit 212 converts the printed characters into electrical signals, and then the text content identifying unit 214 can recognize the printed content on the book 202 to include keyword parameters, adjustment parameters, time parameters, scene parameters, role parameters, or The purpose of the text content of one of the special effect parameters or a combination thereof is not limited thereto. For example, the optical reading unit 212 may be a radio frequency identification tag reader for reading the content on the e-book. The text content identification unit 214 may further include a wireless radio frequency receiver having a decoding function to demodulate the data stored in the radio frequency identification tag into text content having various parameters that the database 230 can understand. Because the size of the text content that can be read by the optical reading unit 212 is different, the text content received by the receiving end 210 can be in accordance with the fixed unit size or the text content according to the user's desired size. . The video editor 220 is disposed between the receiving end 210 and the database 230, and is adapted to receive text content, thereby outputting a basic video material to the data library 230. In this video editor 220, there is further provided a controller 22, wherein the controller 221 can receive an interactive control signal to generate an entity role control data for controlling the entity role 280, which can be radio frequency, infrared or Blue-controlled puppets, plastic dolls or paper dolls. ^ The database 230 stores a plurality of voice data 232 and a plurality of video materials 242. The sound data 232 includes the situational sound effect 2321, the character sound 21720834 effect 2323, and the adjustment sound effect 2325, etc., but not limited thereto, just as the movie can be moved 'because it is provided by the second space In addition to the sound, the sound, including the vocals and soundtracks to create a three-dimensional sense of space three 'so that the sound data 232 has a stronger spatial expression' can include contextual sound effects 2321, character sound effects 2323, adjustment 曰Effect 2325 and user-set music as a soundtrack to make the synthesized multimedia module more influential. ^ For example, when the text content input by the receiving end 210 extracts keyword parameters such as "rain", "thunder", "knocking", "tick answer" and "footstep", the data can be self-made. In the 23rd section of the library, the situational sound effect 2321 corresponding to the situation of "raining", "knocking", "drip answer", and "footstep" is found. ¥The text content input by the receiving end 210, and the characters such as "cats and dogs", "bees" and "grandfathers" are taken from the temples, including "丨.人r. · - J Θ Bei Shumiao", "Dog", "Bees" and "Grandfather's Library 230" query the role of the character that is commensurate with the role of the character. 2323 Adjusting the sound effect 2325 is mainly used to adjust the speed of the vertical, the twist of the ancient you „ 门金°口“The speed of the effect With the same low, swaying or volume, etc., the content of the text that she entered is used to obtain the knowledge of Baotian. 2: For example, the punctuation marks appearing in "二" 2t Λ: "Yes", "喔", "Shouting """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" When adjusting the parameters, the two, ",", the sound effect plus the speed of the conversion, the note will be played slowly. Low adjustment, swaying or loud volume 17 201120834 minor adjustment, for example, when the word content appears, "Ah ^ is down!" 'Because the - paragraph phrase in the 'two consecutive punctuation (9) punctuation marks, - The interjection at the beginning of the sentence, so this paragraph can be presented in a larger volume, higher tone and faster. In addition, the video data 242 includes time video 2421, scene video 2423, character video 2425, and special effects video, etc., but not limited thereto. Time video 2421 is mainly used to present different time. Video, if, when a word appears, time parameters such as "morning", "dusk", "3 am", "two midnight", "spring", "summer", "autumn", "winter", etc. Then, the light time video corresponding to the time parameters can be taken out from the database 230. Specifically, when the text appears "spring", the time video of the brighter and brighter effect can be used to make the multimedia video module. It shows the breath of spring. On the contrary, when there is "three nights", the time video with darker and unclear effect is used to show the dark atmosphere of night. Scene Scene 2423 is mainly used to display video of different scenes. For example, when the scene parameters such as "Imperial Palace", "Outer Space", "Forest", "Snow Country" and "Desert" are taken out from the content, then The video that appears next adds the lighting effects that are appropriate for these time parameters. Specifically, when the text appears in the "Imperial Palace", you can use the scene of the magnificent palace 2423. The character video 2425 can be matched with the character sound effect 2323. When the character content is taken out, the character parameters such as "cat", "dog", "bee" and "grandfather" can be queried from the database 230. Cats, "dogs", 201120834 "bees" and "grandfathers" and other roles commensurate with the role of video 2425. The special effect video 2427 can perform different processing on the video data 242 according to the special effect parameters set by the user in the text content, including the foreground vision configuration, fade or fade speed adjustment, for example, when the text content is "long and long time" In the past, there was a log cabin in the depths of a forest, ...", according to the special effect parameters set by the user in the forest scenes of the video 2423, the cabin scenes video 2423 into a distant view. When a cartoon character is used to create a health promotion video, the cartoon character can be faded out of the mouth except for the mouth, so as to highlight the original dirty teeth in the mouth and become clean through the brushing action. Video of the teeth. In addition, when the situation of the train flying fast is to be expressed, the feeling of the train moving quickly can be expressed by playing the surrounding scenes video 2423 at a faster speed. The video-synthesis interactive device 250 is electrically connected to the data library 230, and is mainly configured to load the video data 242 and the sound data 232 into a multimedia audio-visual module after loading the audio-visual data (the video data 242 and the sound data 232), and output the audio-visual data device 232. To the output 270. In this embodiment, the video and audio synthesis interaction device 250 may include a memory unit 252, a sound effect synthesis unit 254, a sound effect generation unit 256, a video synthesis unit 264, a video generation unit 266, and an output unit 258, etc., to achieve the above functions, but not limit. The memory unit 252 is configured to temporarily store at least one video material 242 loaded from the database 230, at least one voice material 232, and at least one entity role control element 238. Then, when the memory unit 252 loads the at least one sound material 232, it can be sent to the sound effect synthesizing unit 254 electrically connected thereto to synthesize the situational sound effect 2321, the character sound effect 2323 or the adjustment sound effect of the sound data 232. A synthetic step of one of 2325 or a combination thereof. On the other hand, when the memory unit 252 loads the at least one video material 242, it can be sent to the video synthesizing unit 264 electrically connected thereto to perform the time-series video 2421, the scene video 2423, and the role of the video data 242. The synthesizing step of one of the video 2425 or the special effect video 2427 or a combination thereof combines the sound data 232 and the video data 242 by the above steps to achieve the effect expected by the user. In this embodiment, although the sound synthesis unit 254 and the video synthesis unit 264 respectively synthesize the sound data 232 and the video material 242 to implement the present invention, an audio-video synthesis interaction unit capable of simultaneously realizing sound synthesis and video synthesis may be used. To implement the invention. Next, in the present embodiment, the sound data 232 subjected to the synthesizing step is sent to the sound effect generating unit 256 to convert the sound material 232 subjected to the synthesizing step into a form of analog data from the digital form, and wait for output. Similarly, the video data 242 subjected to the synthesizing step is sent to the video generating unit 266 to convert the video data 242 subjected to the synthesizing step into a form of analog data from the digital form to prepare for output. Finally, the sound data 232 and the video data 242 converted to the analog data form are sent to the output unit 258, and the output unit 258 determines the combined pulse signal according to the sound signal 232 and the clock signal carried on the video data 242. The sound data 232 and the video data 242 are outputted as a multimedia audio and video module, and can also be obtained from the memory unit 252 to the output terminal 27 when the multimedia image 20 201120834 sound module is output. In other embodiments, the material V is z phase _ two: and then output to the output terminal 27 〇, the phase ο 4 ' $ only has the sound data 232 when the video data 242 is set as the background music is synthesized as = 2 The destination 270, as such, the purpose of the video-synthesis interactive device 250 is achieved.

In addition, 'While contextual needs should be used, multiple entities must be used: when used as physical textbooks', users can also use other control methods to control entity roles other than physical characters (not shown). Controls such as infrared, Bluetooth, or radio to control other entity roles. The user's needs are more satisfied by using the physical characters of different control modes and using the present invention. In addition, the output terminal 270 can include a display device 274 and a sound effect device 272 for playing the sound data 232 and the video data 242 in the media audio and video module output by the video synthesis interaction device 25, respectively. The display device 274 suitable for the present embodiment may be a liquid crystal display, a plasma display, electricity, paper, etc., but not limited thereto. The applicable sound device 272 can be a speaker or a headphone, etc., but is not limited thereto. In addition, in this embodiment, the output terminal 270 may further include a storage device 276, which may be one of a memory, a flash drive, a hard disk, or a compact disk, or a combination thereof, but not limited thereto, mainly The multimedia audio and video module output by the video and audio synthesis interaction device 250 is used.

21 201120834 Stored for later editing, playback, archiving or production of finished products. Hereinafter, please refer to FIG. 2 and FIG. 3 for convenience of understanding the following description, wherein FIG. 3 is a view showing an aspect of the audio-visual synthesis interactive system and the method thereof for actually implementing the present invention. A mode of sound synthesis in the video-synthesis interactive system 200 will be actually implemented to further explain the interactive method of the audio-video synthesis interactive system 200 and the audio-video synthesis of the present invention. Suppose there is a paragraph-story A's text: "There is heavy rain outside, and the roof is ringing and ticking. The sudden (4), thunder and loudness are scared by the sound of the sound - big flea alone Shrinking in the wall, the following,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Pre-stored a variety of texts and texts

The parameter or adjustment parameter corresponding to the = word J color sound effect 2323 or the adjustment sound effect 2325, as described in the text 2321, the angle will be condensed, will represent the sound effect of X with <X>, for example: < 〇 not (for square sound, That is, the sound of the wind) ^ Wind > represents the wind 1 parameter and the corresponding sound effect parameters - 1 a __ sounds --------- wind &wind; wind &th; thunder > rain <;rain>footsteps<steps> knocking on the door<knocking> 22 201120834

Open <Open the door> Drip &Answer;Trip &Drop> Bite Biting &<Bite Cough &Cough> Description <Cat> Dog &Dog; Dog &Bee; Bee > Grandpa & Grandfather > cattle <cow> sheep <sheep> | <!> ? <?> Ah <ah> Yeah &Yeah 喔<喔> First, the receiving end 210 receives The text of story A. The text content recognition unit 214 compares the various parameters included in the text data of the story A with the various sound effects in the greedy library 230, and then extracts the valid parameters: "Rain", "Drip Answer", "Ray" "," "丨", "猫", and "丨" c Next, according to the result of the parameter extraction, the effective sound effect in the sound data buckle of the database 23 is as follows: Ray II <!〉, <Book Miao> and ◊, and these sound effects into the video synthesis early 7G 250 synthesized into sound data 232. The audio-synthesis interactive unit is (the private synthesis unit 254 can synthesize the sound data 331 generated by the real hairpin technology and the sound data 332 of the user demand 23 201120834 into a sound data 333 according to the text data of the story A, The sound effect generating unit 256 converts the sound data 333 into the audio data 333 of the analog form, and then sends the sound data 333 to the output unit 258. Finally, the synthesized sound data 333 can be output through the output unit 258, and can also be output according to the user's needs. The preset video data 242 is combined into a multimedia audio and video module for output, and only the multimedia audio and video module including only the sound data 232 can be output, and the multimedia video module can be combined with the self-memory unit 252 when the output includes the multimedia audio and video module. The acquired entity role control data 238 is output to the output terminal 270. Hereinafter, please refer to FIG. 2 and FIG. 4 for convenience of understanding the following description, wherein FIG. 4 is an actual implementation of the video-synthesis interactive system 200 of the present invention and Another aspect of the method will be described in the actual implementation of the video-synthesis interactive system 200, the aspect of video synthesis, The method for interacting with the audio-visual synthesis system of the present invention will be described in detail in one step. The method of video synthesis in the video-synthesis interactive system will be implemented in detail to further explain the video-synthesis interactive system 200 and video-to-speech synthesis of the present invention. Interactive method. Suppose there is a story B's text: "What did you see?", "It's a hat", "Or a big python that swallows an elephant" (for convenience, the following story) B), if the audio-visual interactive system 200 of the present invention is used, the text data of the story B of the monotonous tablet can be converted into video data 242 containing various video, and of course, the data library 230 is pre-stored in various forms. All kinds of time parameters corresponding to time parameters, scene parameters, role parameters or a special effect parameter in the text data 2421, scene video 24 201120834 character video 2425 and special effect video 2427 one and two = 4 As shown in the figure, when the sound data appears? What do you see?", then the video material 242 can appear the first - bribery

= Voice data is combined into a multimedia audio and video module. When U 2 St is carried out to "is a hat", then the video data: the screen of the hat of the second column is switched to the sound of the sound image, or it is an elephant swallowing. In the case of the big python, the video data 242 = the inside of the child contains the head of the elephant and the voice =! ^ more ^ body shadow f module. The above embodiment is only capable of: dynamic animation. In the case of inter-page, you can use it. It should be noted that the above embodiment is for convenience of understanding. Only Chinese is used as an example. However, the finder of the present invention, ^ and its computer program dog mouth* system 2〇 〇 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 其 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电 电:: Multimedia audio and video modules for audio and video effects. The above description is intended to be illustrative only and not limiting, and the spirit and scope of the invention may be included in the scope of the appended claims. Change or good [schematic description] Fig. 1 is a flow chart of the interactive method of audio and video synthesis of the present invention; 25 201120834 Fig. 2 is a block diagram of the interactive system for video and audio synthesis of the present invention; and Fig. 3 is actual An aspect map of an audio-visual synthesis interactive system and method thereof embodying the present invention. Fig. 4 is a view showing another aspect of the interactive system for video and audio synthesis and the method for actually implementing the present invention. [Main component symbol description] S110-S140: each process step; 200: video and audio synthesis interactive system; 202: book; 210: receiving end; 212: optical reading unit; 214: text content recognition unit; 220: video editing device 221: Controller; 230: Database; 232, 33 Bu 332, 333: Sound data; 2321: Situational sound effects; 2323: Character sound effects; 2325: Adjust sound effects; 238: Physical character control data; 242: Video data 2421: Time Video; 26 201120834 2423: Scene Video; 2425: Character Video; 2427: Special Effects Video; 250: Video and Audio Interactive Device; 252: Memory Unit; 254: Sound Synthesis Unit; 256: Sound Generation Unit 258: output unit; • 264: video synthesizing unit; 266: video generating unit; 270: output; 272: audio device; 274: display device; 276: storage device; and 280: physical character.

27

Claims (1)

201120834 VII. Patent application scope: 1. An interactive method for video and audio synthesis, which comprises the following steps: receiving a text content from a receiving end; loading a video material relative to one of the text contents from a database and the video material is at least One or a combination of the audio data and the video data; the video data and the sound data are combined into a multimedia audio and video module; and the multimedia audio and video module is output to an output. 2. The video and audio synthesis interaction method as described in item i of the patent application scope = the receiving end receives the text material, and further comprises receiving an interactive control signal from the terminal. 3. The video synthesis according to item 2 of the patent application scope further includes according to the interactive control signal, and the data is rotated to the output end to control to the real:: color role control 4 = please 2 range The audio-synthesis interaction described in item 1 or item 2 = the sound data of the audio-visual material is included in the content of the text - the keyword parameter, the parameter is adjusted, and the corresponding situational sound is synthesized. B adjusts and adjusts the sound effect by one of ten or a combination thereof. Xiao color, "Efficacy 5. As stated in the fourth paragraph of the patent application scope, the application for the application of the 申 效 效 效 r ^ ^ ^ ^ ^ ^ ^ 成 成 成 成 成 ' ' ' I I I I I 依据 依据 依据 依据 依据 依据 依据', the time the sound data is speed, '古: 曰,,, and the output is adjusted back low, swaying 28 201120834 or volume adjustment. 6. The method for interactive video and audio as described in claim 4, wherein the adjustment parameter comprises punctuation, an interjection or a helper. A. The method for interactive video and audio according to claim 1 or 2, wherein the video data further includes the time parameter, a scene parameter, and a role included in the text content. A parameter or a special effect parameter, which is one of or a combination of one of the scene video, a character video, and a special effect video. 8. The method for interactive video and audio according to item 7 of the patent application scope, which further comprises: configuring, visualizing, fading, or slowing the foreground video data of the video data according to the special effect video. Adjustment. 9. In the video synthesis method as described in item i or item 2 of the patent application, wherein the step of receiving the text content from the receiving end is in the step of inputting A printed content is identified as containing the keyword content, a = number one time parameter, a scene parameter, a character parameter, or a special effect parameter. The application of the patent range Scope or item 2 of the video = material: the c-body audio and video material is the multimedia audio-visual module β α video Beko and the sound please (4) range i or 2 The audio and video method 'outputs the multimedia audio and video module to the output step 29 201120834 includes storing the multimedia audio and video module to the output terminal. 12 people into an interactive computer program product, when the video and audio of the product; = computer program is loaded into the computer and executed, can complete the application for patents to the first to the πth item - [13]; - an audio-visual synthesis interactive system, comprising: a text content, an interactive control signal-receiving end, is received by receiving or a combination thereof; 'the system is electrically connected to the receiving end, and stores relative to the audio-visual data, The audio-visual material comprises a plurality of video factories, a plurality of sound data and a plurality of physical materials; and (4) a Bayi video and audio synthesis interactive device, which is electrically connected to the database, and the θ > material is combined to combine the video The data and the sound data are a multimedia audio and video module that outputs the multimedia video module to an output, and controls at least one physical character according to the entity role control data. 14. The audio-visual composite interactive system of claim 13 (4), wherein the receiving end further comprises: an optical reading unit that receives a printed content input; and a text content identifying unit that identifies the printed content as including a key The text content of one or a combination of a word parameter, an adjustment parameter, a time parameter, a scene parameter, a character parameter, or a special effect parameter. 30 201120834 15. If the patent application scope item 13 is in the receiving end and the data _ ^ ^ interactive system, receive the text content, according to the wheel 2 3 - video editor, library. The basic audio and video data is sent to the data. 16·If the application is 鄕 _ 15 , the audio and video editor further includes a synthetic interactive system and a signal to generate the interactive role to receive the interactive control. 17 . + The text content is at least a voice-synthesis interactive system, number, time parameter, a key key, and effect parameters. Role parameter or a special 18. As in the patent application, the 15th of which is stored in the database, the interactive interaction system, the keyword parameter, the angle == sound effect, a role sound effect or one of the parameters of the situation 19. If the patent application scope is the μth item, Lu Qi, the database, is stored in the database. At this time, 4 == the information data contains at least the phase parameter - time video, for the news Or a special effect video. The character does not regard the video and audio interactive system as described in the article 13 of the Chinese version of the syllabus, and the singer-synthesis interactive system includes: = unit 'sto temporarily store the sound from the database loading to the evening Poor material and at least one entity role 31 201120834 control data; unit 'electrically connected to the memory unit, synthesizing the unit' electrically connected with the memory unit 'synthesizing: the sound effect generating unit' is electrically connected with the sound effect synthesizing unit , the cover sound synthesis of the sound data; a video generating unit, the video data synthesizing unit Newson synthesized the video data; and an output unit, and the sound effect generating unit and the video generating unit, the sexual connection 'combines the sounds and the video media audio and video After the module is output to the output, and output to the entity role control data to the output. Xi item, the dynamic system, the Zhuang, Zhongzhong, "the system includes a device for playing the multimedia audio-visual module and a sound effect device." = Application Specialization (4) 19 items of f-sound synthesis. The output terminal further includes the storage of the multi-video mode, and the device. - 32