TWI276961B - System, method and machine-readable storage medium for synchronization of still image and audio - Google Patents

Abstract

A method is employed for synchronization of still image and audio. The method first displays a still image, plays the audio, and partially analyzes the audio in the sampling intervals. The method determines a transition point in the sampling interval using a musical content analysis method and displays the next still image when the transition point is achieved.

Description

1276961 V. INSTRUCTION DESCRIPTION (l) - TECHNICAL FIELD OF THE INVENTION This invention is an image display technology, and more particularly, an image display system and method for synchronizing sound. Prior Art Conventional image display systems, in addition to playing a series of still images (a sequence 〇f stiU images), can also be used to play digital audio. Still images include files in a variety of image formats, such as GIF, JPEG, SVG, PNG, JPEG 2 0 0 or °/Hi shirt format. Digital music also includes a variety of sound formats, such as MP3, MP4, AAC, VGF, 0GG, WAV or other sound formats. However, in general image display systems, most of the images are played independently of a series of still images and digital music. In order to synchronize digital music and still images, a Karode device is disclosed in U.S. Patent No. 5,508,470, the disclosure of which is incorporated herein by reference. In addition, a music and video coordination device is disclosed in the U.S. Patent No. 6,6, 9, 094, for analyzing a whole piece of music and generating a video switching signal according to a significant recurring feature. Let the image display switch the signal according to the image to play the next static digital image. Although the prior art technology can automatically generate image switching signals, there are also several disadvantages. First of all, before playing an image, you must analyze a whole piece of music, which takes a lot of time and computing resources. Therefore, an image and music playback system and method are needed to save analysis time and computing resources. Summary of the invention

V. Description of the Invention (2) First of all, the present embodiment discloses a method for playing a music and a method for playing music. Part 8 is for a string of sound = 9 streams, using a piece of music in Shangcheng:, when the music interval is changed at the sampling interval of the sampling interval t汍, cut and set an image switching time point, point, this The method further includes determining that the still image has not been t-down. The production is based on the segment point, and it is decided that the packager's voice stream is ordered by a segment/melon, and the corpse day is 1 . & 一 contains a part of the sound string before and after the segmentation point. This embodiment further discloses a computer program, the computer program == accessing the library media, for storing a computer system to perform the above 2 In the system and in addition, the embodiment discloses a two-two: a storage device, a display device, a _:: cum music playing system, including a meta. The storage device is used for storing the sound, and the processing of the sound is used to play the sound stream. And multiple still images. The sound image is displayed to the display device, and the sound string is determined; tr is used for outputting - Zhang Jing use - a music content analysis method to sample the interval of one of the two pairs of I, so that the time point is changed at the time of switching, and the second sampling is performed. Interval decision - a cutting device. When determining the sampling interval, the second to the display of a segmentation point, based on the segmentation point; the interval in the stream. , 疋匕έ && point sample audio stream (audio stream) can be MP3, MP4, A 〇 GG or WAV format digital sound; while still image can be ff, toilet, SVG, PNG or JPEG 2 0 0 0 Digital image of the format. Sound & 0599-A20372TWF (Nl); 2004-03; snowball.ptd Page 5 ----, 111, 1276961

V. Description of invention (3) _ Analytical method is used to obtain an attack time f〇r an instrument, a mel〇dy discontinuity, a coma discontinuity (harm) 〇nic discontinui ty), beat discontinuity, pitch discontinuity, a maximum surface peak (seven) "long r〇ugh peak value" or a maximum surface valley (minimum r The above-mentioned switching time point of the 〇ugh valley value. Embodiment 1 is a system architecture diagram of an image and music playing system according to an embodiment of the present invention. The video and music playing system 10 according to an embodiment of the present invention includes a processing unit. 11. A memory device 2, a storage device 13, a display device 14, and an audio device 15 are connected together by a bus bar 16. In addition, those skilled in the art also The present invention can be implemented on other computer system configurations, such as hand-held de vices, multi-processor systems, microprocessor-based or Processing unit 11 may include a single central processing unit (CPU) or a microprocessor-based or programmable consumer electronics Is a parallel parallel processing unit associated with a parallel processing environment. The memory 12 includes a read only memory (ROM), a flash memory, and/or a dynamic access memory. body

0599-A20372TWF(Nl);2004-03;snowball.ptd Page 61276961 V. Invention Description (4) (random access memory; RAM), Tian, ^, used to store the private unit available for processing unit 11 Nuclear group. In general, not r + · Λ , the private module contains routines, programs, objects / , λ _ ~ Cob ject, components, etc. Lisuke π丨,, and # from \丁~1豕dan music playback function. The present invention can also be applied to the decentralized operation to raise the p ^ 1+ j & 1 brother, whose operation is connected to the end of the communication network by the processing equipment of the communication network. The execution of the cum music playback function may be made up of n ^ _ . ^ f from local and multiple remote computer systems. Storage Description 1 3 contains the 爿 爿 爿 + + π & ^ $ hard disk description, floppy device, CD device, to read hard disk, floppy disk, CD, flash drive, storage 2 =, and , music files (that is, sound streaming), configuration broadcasts (,, and sad, and, recorded) and / or static image files. The first embodiment, the embodiment discloses that an instant processing image and music player/f is executed by the processing unit 11. Figure 2 is a flow chart showing the method of video and music playback according to the consistent embodiment of the present invention. If the second step S 2 11 'receives a stream of sound and a string of static shadows, the audio stream can be Mp3, Mp4, AAC, VGF, 0GG, WAV or send a fairy. The digital sound of the format. In step S212, it passes the sound device 15 to play the sound stream. If the step s22i, the display device 14 displays the first still image in the still image of the sequence. Step S 2 3 1. After starting the w/I starting point or the last segment point (partiti ο η po lnt), a segmentation point is determined in the sound power, * at string, and the segment point is from the starting point or the previous point The length of the knife slave point can be a fixed length or one

0599-A20372TWF(Nl);2004-03;snowbal1.ptd Page 7 1276961 V. Description of invention (5) Length of change. For example, this length can be calculated from equation (1).

Lseg = Ltotal/(Nimg-1)........................Formula (1), where L t 〇ta 1 represents the total length of the music stream; N i mg represents the number of still images. In addition to this, the length of the segment point from the starting point or the last segment point can also be preset and stored in the configuration slot or configuration record. In step S23 2, a sampling interval is determined based on the segmentation point. In the preferred case, the length of the sampling interval can be preset

Set and save in the configuration file or configuration record. In step S233, a variety of image content points are used to determine an image transi t ion point from the sampling interval. The music content analysis method is known to those skilled in the art to determine the "attacktime for an instrument", the mel〇dy discontinuity, the harmonic discontinuity, Beat discontinuity, pitch discontinuity, maximum rough peak value, maximum 矣 欠 , , · · · · · · · · · · · · · · · · · · · · · · · · · · And one of the other sound features. Cow

In step S234, the image is idle for a period of time until the switch 1 is determined by the step S233. In step S 2 3 5, at the switching time point, through the display device 丨 4, N: factory, the next still image. In a preferred case, a transition effect is used, such as zooming in (zoom in, u 111 1 n), fade in ff, only H p in), flying in (fly in) Or other, to display the next - static image. In step S241, it is determined whether there is a known image that does not display ^~1豕 ,,

1276961 V. Inventive Note (6) Then proceed to step S 2 3 1; otherwise, the entire image and music playing method is ended. Figure 3 is a diagram showing an exemplary video and music playback in accordance with a first embodiment of the present invention. Referring to step S21 1, a sound stream AS and a sequence of still images 11 to I4 are first received. Referring to step S 2 1 2, the sound stream A S is played through the sound device j 5 . Referring to step S 2 2 1, a still image 11 is displayed. After that, referring to steps S231 to S235, the segmentation point a1 is determined; based on the segmentation point, a sampling interval, al' to al', is determined; using a music content analysis method, in this sampling interval, the switching time is determined. Point s 1, and finally when the music is played to the switching time point s 1, the static energy is displayed on the display device 14. Referring to step S241, since the still images I 3 and I 4 are still stored, the entire flow proceeds to step $2 3 1 to continue the processing of the next stage. , reference basis, decision analysis method When the music broadcast device 1 4 like 14, because of the reference, decide the analysis method to play to 14. Reference, therefore steps S231 to S235 'determine segmentation point a2; segmentation segment based on a sampling interval 'a2' to a2''; use - music content 'in this sampling interval, determine the switching time point to Finally, when the switching time point s2 is placed, the still image 13 is displayed on the display. Referring to step S241, since there is still a static shadow that has not been displayed, the entire process proceeds to step S2 3 1 to continue to the next stage. Steps S231 to S235, determining the segmentation point ^ · at/receiving the wide day and the sun, and then a3, taking a segmentation point as a sampling interval, a3 to a3',; using a halftone, in this sampling interval When 'deciding the switching time (4), and turning the switching time point S3, the static image is:: test step S24 because there is no such thing; the entire process ends. The static shadow of the county

0599-A20372TWF(N1);2004-03;s nowba11.ptdpage 91276961 -15, invention description (7) second embodiment second embodiment discloses a batch processing image and music playing method, the method It is executed by the processing unit 11. Figure 4 is a flow chart showing a method of video and music playback method in accordance with an embodiment of the present invention. In step S 4 1 1, a stream of sounds is received, which may be digital sounds of MP3, MP4, AAC, VGF, OGG, WAV or other sound formats. In step S 4 2 1, at least one partition point is determined in the sound stream, and the length between the segment points may be a fixed length or a variable length. In the preferred case, the length between each segment point is calculated by the above formula (1). In addition to this, each length between the two segment points can also be pre-set and stored in the configuration slot or configuration record. For example, in step S422, a sampling interval is determined based on each segment point. In the preferred case, the length of each sampling interval is preset and stored in the configuration file or configuration record. In step S423, an image transition point is determined from each sampling interval using a variety of music content analysis methods (musiCal content analysis methods). The music content analysis method is known to those skilled in the art and can be used to determine an instrument starting point, a melody discontinuity point, a chorus discontinuity point, a beat discontinuity point, an A word discontinuity point, a maximum surface peak value, a maximum surface. One of the valleys and other sounds. In step S431, the sequence of the static bear shadow-=still image is received, which can be TIFF, GIF, ϊρπ~ Zhang Li 2_ and other image formats: J: EG, SVG, PNG, over-the-sound device 15, playing the sound string If the flow is like a cow, such as a step, the ττ machine is as in step S433, based on the previous production.

1276961 V. Description of the invention (8) ^ The switching time point is transmitted through the display device 丨4, image. In the preferred case, a new ;:::: static shadow transition effect, such as a magnified, $human, φ sorrowful shirt image, can be used to zoom in or out. The figure 0 shows a second embodiment of the present invention. Referring to step S4U, the first "cross-image 1 cum sound test step S42l ' divides the sound stream into four $ soils. Go to a3. Referring to step S422, each of the points: "and ai-like &, respectively, ", to al, ', a2, 疋 a reference step S423, using a music content analysis 2; and: 3 to: 3 , '. In the interval, the sampling time S1, S2 and the sampling S43 are determined, and the still image of the sequence is received, I2; [3 and . = S4 32 'Playing the sound stream through the sound device 15, step S433 'When the music is played to the switching time point s static: on the display device 14; the player* plays until the static image 12 is displayed and pushed τ q You Tianyu When the playback to the switching time point S2, the static image ~ image 13 is not displayed on the display device 14; and when the vertical position = the heart point S3, the still image 丨 4 is displayed on the display device ^. The knives, the day, and the sixth image show that the soft body 41 of the soft image and music playing system according to the first and second embodiments of the present invention is processed by the processing unit 11 to include a sound stream analysis module. The group 411 and the control module, the deaf string analysis module 4 1 1 input the audio stream from the storage device 15, and determine at least one segment point in the audio stream. The sound stream analysis module 4 11 determines a sampling interval based on the parent-segment point, and uses a music content analysis method to determine a switching time point from each sampling interval. The voice stream analysis module 4 1 1 further transmits the determined switching time point to

0599-A20372TWF(N1);2004-03;snowball.ptd

1276961

Control module 412. The control module 412 drives the sonar stream and the sequence of still images to drive the sonar to receive the sound stream, and outputs the first still image to the display device 15 to play the sound string and detects the sound stream to be played to When switching, the gate 1 4 ° control module 4 1 2 turns out to the display device 14 . "Shou Shou" will be the next static image. The present invention proposes a computer program for storing a computer program. The computer program is used to display the image and music playing method of the storage case. _ I and the second embodiment. Figure 7 shows a computer readable storage medium according to the first step of the present invention as described above. The image format of the embodiment includes seven logics. , respectively, receive sound power, ώ θ ” computer program year 曰 stream logic 5 2 1 , , 叔 + \ segment point logic 5 2 2, determine the sampling interval logic R 9 q ^ ', 疋 for chest, receive The static image logic 52:: broadcast =; = point logic 526 and display static image logic 527.曰string w logic method, only a large number, therefore, the video playback system provided by the present invention and the music stream segmentation for the sampling segment can save the analysis time and computing resources' Sound and still images. Although the embodiments of the present invention are disclosed above, they are not intended to be invented, and any one skilled in the art can make some changes and retouchings without departing from the scope of the present invention. And as defined in the scope of the patent application attached to Fanshi. Behave

1276961 BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a system architecture diagram of an image and music playback system in accordance with an embodiment of the present invention. Figure 2 is a flow chart showing a method of video and music playback in accordance with an embodiment of the present invention. Figure 3 is a diagram showing exemplary image and music playback in accordance with an embodiment of the present invention. Figure 4 is a block diagram showing the software architecture in accordance with an embodiment of the present invention. Figure 5 is a diagram showing a computer readable storage medium in accordance with an image and music playing method according to an embodiment of the present invention. Fig. 6 is a view showing a software architecture in accordance with the first and second embodiments of the present invention. Figure 7 is a diagram showing a computer readable storage medium in accordance with the image and music playing method of the first and second embodiments of the present invention. Symbol Description 1 0 ~ video and music playback system; 11 ~ processing unit; 1 2 ~ memory, 1 3 ~ storage device; 1 4 ~ display device; 1 5 ~ sound device; 1 6 ~ bus bar; S211, S221. ....S 233~ method operation steps; AS~sound stream;

0599-A20372TWF(Nl);2004-03;snowball.ptd Page 131276961 Schematic description a 1, a 2, a 3 ~ segmentation point; al,, al,,, a2, a2,,, a3 , a3,, ~ sampling reference point; s 1, s 2, s 3 ~ switching time point; 11, I 2, I 3, I 4 ~ static image; 4 1 1 ~ sound stream analysis module; 4 1 2~ control module; 5 2 0~ video playback computer program; 5 2 1~ receive sound stream logic; 5 2 2~ decide segmentation point logic; 5 2 3~ decide sampling interval logic; 5 2 4~ decide to switch Time point logic; 52 5~ Receive sequence static image logic; 5 2 6~ play sound stream logic; 5 2 7~ display still image logic.

0599-A20372TWF(N1);2004-03;snowba11.p t d第14页

Claims (1)

1276961 __ Case No. 93118490 VI. Patent Application No. 1 · An image and music playing method is loaded and executed by a processing unit, and the method comprises the following steps: displaying a still image; playing a sound stream; using a music content The analysis method 'partial analysis of the music content for a sampling interval of a segment of the sound stream; determining a switching time point in the sampling interval; and switching to display the next still image at the switching time point. 2. The video and music playing method according to claim 1, wherein the audio stream (audio stream) is a digital sound in an MP3, MP4, AAC, VGF, OGG or WAV format. 3. The image and music playing method as described in claim 1, wherein the still image is a digital image in a TIFF, GIF, JPEG, SVG, PNG or JPEG 2000 format. 4. The image and music playing method according to claim 1, wherein the music content analyzing method is used to obtain an attack time f 〇r an instrument and a melody discontinuity point ( Melody discontinuity), a harmonic discontinuity, a beat discontinuity, a pi tch discontinuity, a maximum rough peak value, or a The above switching time point of the maximum valley value. 5. The video and music player as described in item 1 of the patent application scope 1276961 Ρ . ν _ case number 93ii^irg : — L1 正 __ VI. Patent application 1-1—: The method further includes the following steps: determining a segmentation point in the above-mentioned sound stream that has not been played; Based on the segmentation point, the sampling interval including the segmentation point is determined, and the image and music playing method described in claim 5, wherein the segmentation point is used to cut the sound stream Divided into the same aliquot. 7. The image and music playing method according to claim 6, wherein the music content analysis is used to obtain an attack time for an instrument and a melody discontinuity. , a harmonic discontinuity, a beat discontinuity, a pitch discontinuity, a maximum peak value, or a maximum surface valley (minimum) r〇Ugh vaney vaiue) The above switching time point. 8. A computer readable storage medium for storing a computer program for loading into a computer system and causing the computer system to perform any of claims 1 to 7 The method described. 9. An image and music playback system comprising: a storage device </ RTI> for storing 'a stream of sounds and a plurality of still images; a display device; a sound device for playing the stream of sounds; 0599-A20372TWF1CN1);2004-03;snowbal1.ptc Page 161276961 r Month.,曰修正i号93]!|你为〇6 VI. Patent scope _ 一处理单疋' used to rotate a static The image is sent to the display device X, so that a music content analysis method performs partial analysis of the music content for a sampling interval of one piece of the audio stream, and in the sampling interval, the switching time interval is determined in the above switching time. When the point is output, the next still image is output to the above display device. I 0 · The video and music playing system described in claim 9 of the patent application, and the audio stream of the above-mentioned audio stream is a digital sound of the MP3, MP4, AAC, VGF, 0GG or WAV format. II. The video and music playback system of claim 9, wherein the still image is a digital image of the gTlFF, GI F, jpEG, SVG, pNG or JPEG 200 format. 1 2. The image and music playing system according to claim 9, wherein the music content analysis method is used to obtain an attack time for an instrument and a melody discontinuity (melody) Discontinuity), harmonic discontinuity, beat discontinuity, pitch discontinuity, orlaximum rough peak value or Yuan Da surface The above switching time point of the value (iniiiiniuni rough valley value). The image and music playing system of claim 9, wherein the processing unit is configured to determine a segment point of the sound stream that has not been played, based on the segmentation point, and determine to include The above sampling interval of the above segmentation point. 0599-A20372TWF1(N1);2004-03;snowba11.pt c第17页