JP2005202425A - System for synchronously outputting accompaniment tone of musical piece and words title video - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a rational method for editing words title data without require remaking of color change related information of music and words titles in spite of a change in the type faces and screen layout of the titles. <P>SOLUTION: The words data includes a series of character information constituting the words titles, title coordinate information regulating the screen layout of the character information and singing time information regulated with respect to each one character of the character information. A controls means conducts first to second processing while managing the time according to a time base signal outputted by a speech reproduction means at the time the speech reproduction means plays a musical piece. A first processing converts words data character information to information of character fonts and simultaneously generates words title image data by arranging the respective character fonts in compliance with the words data title coordinate information. A second processing changes the color information partially and gradually by each of one character of the words title image data according to the words data singing time information. A video output means reads out the words title image data written into the frame memory, repetitively at a high speed, converts the same to a video signal and outputs the signal. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an apparatus for synchronously outputting an accompaniment sound of a song and a lyrics subtitle video by applying a rational production technique of a lyrics subtitle video that is indispensable to a video of a song.

  One of the indispensable conditions for lyric subtitles in singing music is that it is lyric subtitles that are easy to sing and easy to capture poetry, and is laid out on the screen with appropriate phrase units. While displaying lyrics subtitles at a time when it is easy to sing, the display color of the subtitles is smoothly changed to trace the part being sung. Conventionally, there has been an apparatus that can generate a series of subtitle images as a material by a personal computer equipped with a keyboard and a display device. In this conventional apparatus, a screen layout of subtitle characters is set by a screen interactive text editing technique. In the process of compiling the video material produced in this editing work as a video with subtitle color change linked to the progress of the music, manually operated video editing equipment is used, and recorded in order to synchronize with the progress of the music While actually listening to the playback sound of the song, the display timing of the lyrics subtitles was adjusted, and the subtle movements in the production of the subtitle color-change video that encourages the progression of lyrics were also expressed manually.

  However, in the color-change video editing work that is performed manually by tracing the subtitle video as the music progresses while listening to the music, the movement of the hand is reflected in the video as it is, so a considerable amount of practice for each music It was necessary to start over and over again to obtain a highly complete color change video. For this reason, a considerable amount of skill is required for video editing work of lyric subtitles that change colors, and it has been troublesome and time-consuming for such skilled people.

  In addition, when the subtitle typeface or screen layout is changed, all of the same music, including video editing work, must be redone.

  The present invention has been made in view of the above-described conventional problems, and the purpose of the invention is to appropriately perform data editing work of lyrics subtitles that change color even if not a highly skilled person. Is to provide rational technology that does not require re-creating information related to color change of music and lyric subtitles even if the typeface or screen layout is changed.

The present invention is an apparatus for synchronously outputting a musical accompaniment sound and lyrics subtitle video, and is specified by the following items (1) to (8).
(1) An apparatus that includes audio reproduction means, storage means, control means, and video output means, and that synchronously outputs the accompaniment sound of the music and the lyrics subtitle video. (2) The audio reproduction means is the music performance information. (3) The storage means stores the lyrics data of the music (4) The lyrics data is the lyrics Including a series of character information constituting the subtitle, subtitle coordinate information defining the screen layout of the character information, and singing time information defined for each character of the character information. When playing music, the first and second processes are performed while managing the time according to the time base signal output by the sound reproduction means. (6) The first process converts the lyric data character information into character font information. At the same time, lyric subtitle image data is created by arranging each character font according to the lyric data subtitle coordinate information. (7) The second process is for each character of the lyric subtitle image data according to the lyric data singing time information. (8) The video output means repeatedly reads the lyrics subtitle image data written in the frame memory at high speed, converts it into a video signal, and outputs the video signal.

(A) By storing musical score data and lyric data with high synchronism with performance sound in a database, a medium that is generally familiar and easy to accept as a manual work target in the process of visualizing lyrics subtitles In addition to the work of editing lyrics subtitles to set the screen layout of lyrics subtitles, the work of combining lyrics and score to set the link to the song progression of the lyrics subtitles The video signal of lyric subtitles is automatically generated in units of music in two working modes.

(B) Conventionally, using a video editing device, while actually listening to music, performing manual operations such as tracing lyrics subtitles as the music progresses, and changing the color of the lyrics subtitles as the music progresses I was editing. It was very difficult and troublesome to properly perform this editing work, and it was necessary to rely on a long time work by an expert.
On the other hand, according to the present invention, the work of adding lyric characters in accordance with the note position of the musical score is performed on the computer screen, so that the musical time relationship between the lyrics subtitles and the music progression is related to each character of the lyrics. Since the prescribed singing time information is added to the text information of the lyrics subtitles, anyone who has basic knowledge about how to read the score can easily edit the lyrics subtitle data appropriately. Based on the lyrics subtitle data that is easily and reasonably produced in this way, the lyrics subtitles can be displayed on the screen and can be easily sung according to the music.

(C) Lyric subtitle data according to the present invention includes a series of character information constituting the lyrics subtitle, subtitle coordinate information defining what layout the lyrics subtitle is displayed on the screen, lyrics subtitles and music progression Is composed of singing time information that defines the musical time relationship for each character of the lyrics. Therefore, even if the subtitle font or screen layout of the existing lyrics subtitle data is changed for some reason, the singing time information is defined for each character of the lyrics, so There is no need to recreate the singing time information. Therefore, the lyrics subtitle data becomes general-purpose data suitable for various display output systems.

  A first embodiment of the present invention will be described below with reference to FIG. 1. 1 is a display device using a CRT display, 2 is an input device using a keyboard, 3 is a storage device using a hard disk, 4 is a video output device, and 5 is a video output device. A CPU processing device 6 represents a timer device.

  To describe the subtitle editing of lyrics data, which is a work process for satisfying the first condition of the lyrics subtitle video, in order to generate lyrics subtitles that are easy to sing and to lay out the screen in appropriate paragraph units that are easy to capture poetry, First, the text information of the lyrics is input, the lyrics data for each song is created, the lyrics data is displayed on the display device 1 using the subtitle character font, and the screen layout of the lyrics subtitle is displayed while checking the displayed image 10. By the setting operation, the lyrics data is subjected to caption editing by a processing function for adding information (caption coordinates) relating to the character layout as the caption to the lyrics data. In this case, the subtitle character font is stored in the storage device so that the font of the subtitle character can be changed as necessary.

Specifically, the lyric data is input using the document creation function in an interactive format with the screen of the CRT display 1 using the keyboard 2, and the lyric document created in units of music is stored in the hard disk 3 as a lyric data file. Stored.
Further, the subtitle image 10 of the lyrics data drawn using the subtitle character font pattern of the hard disk 3 is copied to the CRT display 1 and edited using the subtitle layout function for each page.
By performing appropriate subtitle editing utilizing these functions, it is possible to obtain lyric data with fixed subtitle coordinate information including page allocation and subtitle layout.

  On the other hand, the musical score data is input in a form in which the pitch, pitch, etc. of the note unit is input numerically from the keyboard 2, and the content of the musical score is not sufficient with only the melody part for recognizing the singing progress. However, it is stored in the hard disk 3 as a musical score data file in units of music in which the time passage of the entire music including the prelude and interlude is expressed.

  Next, the work process for satisfying the second condition of the lyrics subtitle video will be described. While displaying the lyrics subtitles at a timing that is easy to sing in conjunction with the progress of the music, the subtitles are displayed so that the part being sung can be traced. In order to smoothly change the color, it is effective to use a score as a means of imagining the progress of music in the work of setting the linkage with the music progress of the lyrics subtitles.

When describing the composition of lyrics data, we will input the data necessary to express the score, such as notes, rests, time signature, tempo, etc., and create music data for each song. The synthesized image 11 is displayed on the display device 1.
The horizontal axis of this screen is treated as time coordinates that depend on the time signature, tempo, etc., and the song data progresses to the lyrics data by the work of arranging the lyrics data exactly as the original score in correspondence with the notes on the score in character units It has a processing function to add information (singing time) related to linking.

  More specifically, the composite edited image 11 obtained by combining and editing the score data and the lyrics data is displayed on the CRT display 1, and the lyrics are moved left and right in character units so that the lyrics can be arranged corresponding to the score. And various edits are performed using various functions provided for efficient placement at an arbitrary position.

In this case, since the whole score cannot be displayed on one screen of the CRT display 1, the previous and next scores are displayed in order by the page feed function to cover the whole score.
In addition, if measures that are repeatedly sung are handled as common measures, the first lyrics 12, the second lyrics 13, and the third lyrics 14 can be displayed in parallel, thereby improving the efficiency of editing work.
Utilizing these functions, the lyric data in which the lyric data is synthesized and edited in accordance with the original musical score and the lyric time information in character units is fixed is created and stored in the hard disk 3 again.

  In this way, based on the subtitle coordinate information in units of characters and the lyrics data for which the lyric time information is determined, the first character of the page-assigned character group assuming the passage of time of the music progression starting from the prelude and ticking Subtitle display at the optimal display timing of the page estimated from the singing time is performed with a screen layout according to the subtitle coordinate information, and the singing characters that are executed based on the singing time information of each displayed character are smoothed Batch generation of lyrics subtitle video, such as repainting the character color to follow, and continuously sending drawing signals to the video output device 4 to image a series of subtitle developments appropriately linked to the progression of music A processing function is provided.

  In this case, if music score data and lyrics data that are highly synchronized with the performance sound are stored in the database, in addition to the work of editing the lyrics subtitles to set the screen layout of the lyrics subtitles, the lyrics subtitles It is possible to provide a production environment that can automatically generate video signals of lyrics subtitles in units of songs with two work modes, which combines the work of composition editing of lyrics and music to set the linkage to the music progression of The manual work in the process of converting lyric subtitles to video can be integrated into desk editing through a generally familiar and easy-to-accept medium.

  Based on the subtitle coordinate information, singing time information, and subtitle character font information added to the lyric data by the work process using the above processing functions, the lyric subtitle display at the optimal timing according to the passage of time from the start of the song By continuously executing a processing function of sending a drawing signal for the video and a drawing signal for smooth color change expressing the singing progress of the subtitle to the video output device 4 until the end of the song, The batch automatic generation process is completed.

  A time base signal for time management corresponding to the tempo value representing the speed of music is generated by the timer device 6. The time base signal for managing the passage of time reproduces the speed of the music and is output to the processing device 5 as a timing clock having a frequency depending on the tempo value on the score.

In this case, the tempo value is the number of quarter notes played in one minute. When the number of timing clocks corresponding to quarter notes is 48, the number of timing clocks is 48 times the tempo value per minute. Given in.
By using a timing clock that is proportional to the tempo value as a time base signal for time management, lyrics subtitles that are accurately linked to the speed of the music given at an arbitrary tempo value are automatically generated in a batch. Processing is achieved.

  Further, as shown in FIG. 4, the video output device 4 has a drawing signal input from the processing device 5, that is, on the frame memory that changes every moment according to the writing address and color information to the frame memory that images the video screen. Subtitle images are read from the address counter with the timing for the video signal and the read address generated in the scanning order, and converted into a video signal by the video signal encoder, so that the lyrics subtitle video is obtained as a video signal. You may do it.

  In this case, as shown in FIG. 5, if the subtitle image signal obtained by reading the frame memory is preferentially mixed on the external video signal at the read address synchronized with the external video, the lyrics subtitles are displayed on the external video. Can be obtained as a superimposed video signal.

In the processing operation described above, if there is a time lag between a series of lyrics subtitle images generated in real time according to the score data and music played according to the same score, this is due to a difference in tempo. .
FIG. 2 shows a second embodiment of the present invention which solves this problem. That is, a metronome sound generator 7 linked to the timer device 6 is provided to eliminate the above-described tempo difference, and the time base signal serving as a reference for the length of the musical score data has a frequency 48 times that of the metronome sound. The timing tempo is output, and the progress tempo of the lyrics subtitle video is managed with the quarter note on the score as 48 clocks.

This is provided to further improve the synchronism between the lyrics subtitle video generated in real time according to the input musical score data and the performance sound of the music, and the time signature determined by the tempo value set arbitrarily. A metronome sound is generated, and the metronome sound and a time base signal that is a reference for engraving the musical note length are unified and generated, and for music played at a tempo that matches the metronome sound. And fully synchronized lyrics subtitle images can be generated.
The lyrics subtitle image generated in the above-described embodiment 2 is one in which the performance speed is controlled to enhance the synchronization, and is more synchronized with the music played at a tempo that matches the metronome sound. It becomes a high quality thing.

FIG. 3 shows a third embodiment of the present invention in which the synchronism is enhanced by inputting musical score data in accordance with the performance.
This is to replace the above-mentioned timer device with a sound reproducing device 9 with time code so as to obtain score data in accordance with the tempo of the played music, and a recording medium (tape) on which the time code is recorded together with the performance sound. A time code reproduced by the sound reproducing device 9 and used as a time base signal is used. In this case, it is possible to generate an effective lyric subtitle video for music that has already been recorded or music that has an indefinite tempo.

In other words, musical score data is automatically generated based on performance information obtained by playing the melody in real time on the MIDI keyboard 8 in accordance with the performance sound and the time code of the accompaniment playback sound of the recorded recording tape. Music score data that is completely synchronized with the recorded accompaniment sound by automatically generating score data based on the melody performance information input through the MIDI signal and the playback time code from the tape. Will be obtained.
Note that, even at the stage of generating the lyrics subtitle image, the time code reproduced from the audio reproduction device 9 is time-managed as the time base signal, so that more complete synchronization is ensured.

The video output device 4 converts an image of the drawing signal output from the processing device 5 into a video signal and outputs the video signal, which is recorded on a VTR (video tape recorder), LD (laser disk) or the like. In addition to outputting video signals, digitally encoded video signals having an encoding function corresponding to code systems for various computer graphics video media such as CD (compact disc) graphics and CD-I. May be output.
Alternatively, an external video may be input and a video signal superimposed with lyrics subtitles may be output on the video.

It is a block diagram of a lyrics subtitle video batch automatic generation system showing a 1st example of the present invention, and an example figure of an edit screen. It is a block diagram of a lyrics subtitle video batch automatic generation system showing a second embodiment of the present invention. It is a block diagram of the lyrics subtitle video batch automatic generation system showing a third embodiment of the present invention. It is a block diagram which shows an example of the video output apparatus used in this invention. It is a block diagram which shows an example of the video output apparatus with a superimpose function used in this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Input apparatus 3 Storage apparatus 4 Video output apparatus 5 Processing apparatus 6 Timer apparatus 7 Metronome sound generation apparatus 8 MIDI keyboard 9 Audio | voice reproduction apparatus 10 Subtitle image 11 of lyrics data 11 Composition edit image

Claims (1)

Inventions specified by the following items (1) to (8).
(1) An apparatus that includes audio reproduction means, storage means, control means, and video output means, and that synchronously outputs the accompaniment sound of the music and the lyrics subtitle video. (2) The audio reproduction means is the music performance information. (3) The storage means stores the lyrics data of the music (4) The lyrics data is the lyrics Including a series of character information constituting the subtitle, subtitle coordinate information defining the screen layout of the character information, and singing time information defined for each character of the character information. When playing music, the first and second processes are performed while managing the time according to the time base signal output by the sound reproduction means. (6) The first process converts the lyric data character information into character font information. At the same time, lyric subtitle image data is created by arranging each character font according to the lyric data subtitle coordinate information. (7) The second process is for each character of the lyric subtitle image data according to the lyric data singing time information. (8) The video output means repeatedly reads the lyrics subtitle image data written in the frame memory at high speed, converts it into a video signal, and outputs the video signal.

Images