JP2008180794A - Data reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technology capable of displaying a lyrics telop in accordance with flow of a music piece, even when tempo of the music piece is changed like live music. <P>SOLUTION: In a storage section 12 of an image display device 10, lyrics data are stored by dividing it into a plurality of blocks, and collating data for indicating a string of phonemes of the lyrics corresponding to the lyrics data are stored for each block. A control section 11 detects motion of lips of a singer from image data, and detects a string (a pattern) of vowels included in singing voice of the singer from a detection result. The control section 11 collates the detected pattern with a collating pattern stored in the storage section 12, and specifies the block which is being sung, based on a matching degree. The control section 11 starts generation of a lyrics image data for expressing an image of the lyrics of the block in timing of the block being specified, and combines the generated lyrics image data with an image data for expressing the imaged image, and displays it on a display section 13. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for reproducing data.

  The karaoke apparatus has a function of displaying lyrics telop on the screen and changing the color of the telop in order according to the accompaniment. With such a function, the karaoke apparatus can guide the singer to pronounce the correct lyrics at the correct timing. Moreover, the display of the lyrics telop is used not only in a karaoke apparatus but also in a song program in a television broadcast or the like.

By the way, as a technique for correcting a shift between a moving image and a sound, Patent Document 1 detects an occurrence timing of the first sound by performing image analysis on an image indicating the sound generation in the moving image, A technique for detecting the generation timing of the second sound from the sound, measuring the difference between the sound and the moving image based on the generation timing of the first sound and the generation timing of the second sound, and delaying the sound by the measurement result. Proposed.
JP 2000-196917 A

By the way, while the karaoke accompaniment by a karaoke apparatus is always reproduced | regenerated with fixed tempo, when a live performance (live performance) is performed in song programs, such as a television broadcast, the tempo of a performance often changes with the occasion. For this reason, when the lyrics telop data used in the karaoke apparatus is used in television broadcasting, a time lag occurs between the lyrics telop and the actual singing video, which is unnatural. In general, in television broadcasting, the operator manually switches the display of the lyrics telop as the music progresses, and the work is complicated. With the technique described in Patent Document 1, it is possible to correct the shift between the moving image and the sound, but it is not possible to correct the display timing of the lyrics telop according to the live performance.
Also, in live performances, there are cases in which arrangements are added to the music composition, such as increasing the number of chorus performances from time to time.
The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique capable of adjusting the display timing of the lyrics telop to the video.

  A data reproducing apparatus according to a preferred aspect of the present invention includes a lyric data storage means for storing lyric data representing lyrics of a music and divided into a plurality of blocks, and video data including a video of a singer. From the video data acquisition means to be acquired, the detection means for detecting the lip shape of the singer from the video data acquired by the video data acquisition means, and the lip shape detected by the detection means, the singer A phoneme pattern detecting means for detecting the arrangement of phonemes included in the uttered voice, and the arrangement of the phonemes detected by the phoneme pattern detecting means is checked against the lyrics data for each block, and the singer Singing block identifying means for identifying the block being sung, and the lyrics data is read from the lyrics data storage means, and the song indicated by the read lyrics data Lyric video data generating means for generating lyric video data representing the video of the lyric, and at the timing when the block being sung is specified by the singing block specifying means is identified among the lyric data generated by the lyric video data generating means Output means for outputting lyric video data representing the lyric video of the block to the display means.

  In the above aspect, the lyric video data generating means starts generating lyric video data representing the lyric video of the specified block at the timing when the block being sung is specified by the singing block specifying means. Also good.

  In the above aspect, the lyrics data is divided into a plurality of blocks and includes synchronization information indicating the reproduction start timing of each of the divided blocks, and the output means is configured to synchronize each of the blocks. When the difference between the reproduction start timing of each block indicated by the information and the timing at which the block is specified by the block specifying means is within a predetermined range, lyrics video data representing the lyrics video of the block is output May be.

The output means may output the video data acquired by the video data acquisition means and the lyric video data generated by the lyric video data generation means.
In this aspect, the image processing apparatus includes a delay unit that delays the video data acquired by the video data acquisition unit by a predetermined time, and the output unit generates the video data delayed by the delay unit and the lyrics video data generation The lyrics video data generated by the means may be output.

  Further, in the above aspect, the first sound data storage unit that stores the first sound data representing the sound, the sound data obtaining unit that obtains the second sound data representing the sound, and the sound data obtaining unit A time alignment unit that associates the acquired second audio data with the first audio data stored in the first audio data storage unit in units of frames of a predetermined time length, and the output unit includes: The lyric video data representing the lyric video of the identified block among the lyric data generated by the lyric video data generating unit may be output to the display unit at a timing according to the association result by the time alignment unit.

  According to the present invention, the display timing of the lyrics telop can be matched to the video.

Next, the best mode for carrying out the present invention will be described.
FIG. 1 is a block diagram showing a hardware configuration of a video display apparatus 10 according to an embodiment of the present invention. In the figure, the control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and reads and executes a computer program stored in the ROM or the storage unit 12. The respective units of the video display device 10 are controlled through the bus BUS. The storage unit 12 is a storage unit for storing a computer program executed by the control unit 11 and data used at the time of execution, and is, for example, a hard disk device. The display unit 13 includes a liquid crystal panel and the like, and displays a menu screen for operating the video display device 10 and a moving image such as a live video under the control of the control unit 11. The operation unit 14 outputs an operation signal corresponding to the operation by the user to the control unit 11. The microphone 15 is a sound collecting device that picks up a singer's singing voice (hereinafter referred to as “input voice”). The microphone 15 outputs an analog electric signal representing a waveform of the input sound on the time axis. The audio processing unit 16 converts an electrical signal input from the microphone 15 into a digital signal (hereinafter referred to as “input audio signal”). In addition, the audio processing unit 16 converts the digital data into an analog signal and outputs the analog signal to the speaker 17 under the control of the control unit 11. The speaker 17 is a sound emitting unit that emits sound with an intensity corresponding to the sound signal that is converted from the digital data into an analog signal and output by the sound processing unit 16. The photographing unit 18 photographs the singer and outputs video data representing the photographed video to the control unit 11.

  In this embodiment, the case where the microphone 15 and the speaker 17 are included in the video display device 10 will be described. However, the audio processing unit 16 is provided with an input terminal and an output terminal, and the input terminal is connected via an audio cable. An external microphone may be connected, and similarly, an external speaker may be connected to the output terminal via an audio cable. In this embodiment, the case where the audio signal input from the microphone 15 to the audio processing unit 16 and the audio signal output from the audio processing unit 16 to the speaker 17 are analog audio signals will be described. You may make it input / output. In such a case, the audio processing unit 16 does not need to perform A / D conversion or D / A conversion. The same applies to the display unit 13, the operation unit 14, and the imaging unit 18, and may be a format built in the video display device 10 or an externally attached format.

As illustrated, the storage unit 12 includes a lyrics data storage area 121 and a collation data storage area 122. The lyrics data storage area 121 stores lyrics data representing the lyrics of the music displayed as the lyrics telop. This lyric data is data created in advance for karaoke. FIG. 2 is a diagram illustrating an example of the contents of the lyrics data. As shown in the figure, the lyrics data includes text data indicating the lyrics of the music, line feed data indicating the line break of the lyrics, and wipe start timing data indicating the wipe start timing for each character of the lyrics. Then, when it is played back by the video display device 10 and the lyrics telop is displayed on the screen, one character of the lyrics starts to change color from the left side of the character when the corresponding wipe start timing starts, and the next character wipe starts When the timing is reached, the color change is controlled so that the color change is completed for the entire character. In this case, the wipe start timing data is also provided in the line feed data, and for the character displayed at the end of one line, the time interval between the wipe start timing data of the character and the wipe start timing data of the line feed data is It becomes the color change time of characters. The speed at which the color of each character changes is determined from the number of dots in the horizontal direction of the character and the color change time (the difference between the wipe start timing of the character and the wipe start timing of the next character).
Each character changes color within the color change time. In this figure, the wipe start timing of the first character is wt1, the next character is wt2, and so on.

  In this embodiment, the lyrics data is divided into a plurality of blocks B1, B2,... (A melody, B melody, rust, etc.), and the head portion of each divided block B1, B2,. Have block start data b1, b2,. The wipe start timings wt1, wt11,... At the head of each block are information indicating the playback start timing of each block. In the following description, for the sake of convenience, the information indicating the playback start timing of each block is referred to as “synchronization”. Referred to as “information”.

The collation data storage area 122 of the storage unit 12 stores collation data indicating the arrangement of vowels of phonemes constituting the lyrics of the lyrics data for each block constituting the lyrics data. FIG. 3 is a diagram illustrating an example of the content of the collation data. As shown in the figure, this collation data is composed of data indicating the vowels of phonemes constituting the lyrics. This data is data that is referred to when performing the singing block specifying process (details will be described later) performed by the control unit 11. In this embodiment, the collation data indicates a sequence of vowels of 6 to 8 characters at the head of each block.
The collation data is not limited to the arrangement of the vowels of the first 6 to 8 characters in each block, and any data may be used as long as it indicates the arrangement of phonemes included in each block.

  Next, the functional configuration of the video display apparatus 10 will be described with reference to FIG. FIG. 4 is a block diagram showing a functional configuration of the video display device 10. In the figure, a video analysis unit 111, a karaoke lyrics generation unit 112, a video playback unit 113, and a delay unit 114 are read and executed by the control unit 11 of the video display device 10 by reading a computer program stored in the ROM or the storage unit 12. It is realized by. The arrows in the figure schematically show the flow of data. In this embodiment, the video analysis unit 111 and the karaoke lyrics generation unit 112 are realized as software, but the configuration is not limited to this, and may be realized by hardware.

  In FIG. 4, the video analysis unit 111 has a function of acquiring video data from the photographing unit 18 and detecting the shape of the singer's lips from the acquired video data. The video analysis unit 111 analyzes the video data output from the photographing unit 18 and performs face detection processing, and detects the movement of the singer's lips (change in lip shape) based on the detection result. In the face detection process, for example, skin color detection may be performed, and a captured image may be collated with an image of a predetermined pattern, and a face portion may be detected based on the degree of coincidence. In addition, the aspect of a face detection process is not restricted to this, What kind of thing may be sufficient if it is an aspect which can detect a singer's face part.

  When the detection of the movement of the lips of the singer is started, the video analysis unit 111 determines that the singing of any of the blocks B1, B2,. The video analysis unit 111 detects the arrangement of phonemes at the beginning of each block by detecting the arrangement of phonemes included in the voice uttered by the singer when it is determined that the singing of the block has started. This phoneme arrangement detection process will be described below.

The video analysis unit 111 detects the movement of the lips of the singer (changes in the shape of the lips) from the detected image of the face portion, and based on the detection result, arranges the phoneme sequence included in the voice uttered by the singer. To detect. In this embodiment, the video analysis unit 111 detects the arrangement of vowels included in the voice uttered by the singer. In the phoneme arrangement detection process, for example, pattern data indicating the shape of the lips corresponding to the vowel (a, i, u, e, o) is stored in advance, and the detected lip shape is stored. You may detect by collating with pattern data. In addition, the phoneme alignment is detected by detecting a change in the shape of the lip by the video analysis unit 111 for a predetermined time after the detection of the lip movement of the singer is started. The phoneme sequence may be detected based on the result.
As a method for converting the shape of the lips into phonemes, the following two methods are mainly conceivable. The first method is a method in which a person's lip shape is photographed character by character and the relationship between the lip shape and phonemes is compiled into a database. This method can be converted into phonemes relatively accurately if the image is the user, but has a problem that it is not versatile. The second method is a method of converting this into a phoneme from the shape of the lips or the movement of the lips. This method can be said to be a highly versatile method although accuracy is difficult compared to the first method. The second method is to associate phonemes with general lip shapes. For example, a shape with a wide mouth is associated with “A”, a shape elongated in a horizontal direction with “I”, a shape with a small jar is associated with “U”, and so on. Extracting such a lip shape change from image data can be realized by obtaining a difference value between image data of frames that are temporally forward and backward. That is, in the case of an image of a person who is scolding, the difference value between the previous and next frames is almost only the shape of the lips.

The video analysis unit 111 supplies data indicating the detected phoneme arrangement to the karaoke lyrics generation unit 112.
The karaoke lyrics generation unit 112 includes a sequence of vowels (phonemes) detected by the video analysis unit 111 (hereinafter referred to as “phoneme pattern”) and collation data for each block stored in the collation data storage area 122 (arrangement of vowels of lyrics). And a block of the lyrics that the singer is singing based on the degree of matching. Specifically, this specifying process may specify, for example, a block having the highest degree of matching, for example, and may specify a block having a matching degree equal to or higher than a predetermined value.
As described above, the video analysis unit 111 uses the collation data shown in FIG. 3 to set the 6-8 characters at the beginning of each block as a set and a pattern with lyrics (the singing portion of each phrase (block)). Perform matching. Thus, in this embodiment, since the phoneme string of a certain length is used for collation, even if there is some error in the phoneme pattern (phoneme arrangement) analyzed from the singing video, the collation Accuracy can be increased.

  Further, the karaoke lyrics generating unit 112 has a function of reading out lyrics data from the lyrics data storage area 121 and generating lyrics video data representing a video of lyrics (hereinafter referred to as “lyric video”) indicated by the read lyrics data. At this time, the karaoke lyrics generation unit 112 starts generating lyric video data representing the video of the lyrics of the specified block at the timing when the block being sung is specified.

  The karaoke lyrics generation unit 112 sequentially supplies the generated lyrics video data to the video playback unit 113. The delay unit 114 supplies the video data acquired from the imaging unit 18 to the video playback unit 113 with a delay of about the processing time of the video analysis unit 111 and the karaoke lyrics generation unit 112. The video playback unit 113 generates composite video data obtained by superimposing the lyrics video represented by the lyrics video data supplied from the karaoke lyrics generation unit 112 on the video represented by the video data supplied from the delay unit 114. Output to the display unit 13.

  In this way, the composite video data is output to the display unit 13, and the singing sound collected by the microphone 15 is output to the speaker 17. Thereby, the video display apparatus 10 according to this embodiment can reproduce a video and a music in which lyrics video is synthesized in time synchronization, that is, in accordance with the progress of the music, with respect to the shot video.

  Here, the specific example of the process which the karaoke lyrics production | generation part 112 performs is demonstrated below, referring FIG. (A) of FIG. 5 is a figure which shows an example of the content of lyrics data, (b) is a figure which shows an example of the content of the singing voice actually sung by the singer, (c), It is a figure which shows an example of the content of the lyrics telop which the video display apparatus 10 displays. As shown in FIG. 5A, the lyric data is divided into a plurality of blocks B1, B2, B3, B4, B5. On the other hand, the performer (singer) who performed the live performance did not sing according to the lyrics data and performed block B4 twice in the order of blocks B1, B2, B3, B4, B4, B5. Suppose that Further, as shown in the drawing, the start time of the block B1 is shifted by the time Δt between the singing in the live performance shown in (b) and the lyrics data shown in (a).

In the example shown in FIG. 5, when the singer performs the singing of the block B1, the control unit 11 analyzes the video data output from the photographing unit 18 to detect the shape of the singer's lips, and detects the detected lips. The phoneme pattern of the voice uttered by the singer is specified from the shape of Next, the control unit 11 collates the identified phoneme pattern with the collation data stored in the collation data storage area 122, and identifies the block B1. The control unit 11 starts generating lyric video data representing the lyric video of the identified block B1, and starts wiping processing of the lyrics telop corresponding to the block B1. That is, the control unit 11 starts displaying the lyrics telop corresponding to the block B1, and based on the wipe start timings wt1, wt2, wt3,... (See FIG. 2) of each character included in the lyrics data, the time (wt1 + Δt ), (Wt2 + Δt), (wt3 + Δt),...
Accordingly, even when the start timing of the singer is shifted from the display start timing of the lyrics telop (in the example shown in FIG. 5, they are shifted by time Δt), they can be synchronized, and the display is It can prevent becoming unnatural.

In the example shown in FIG. 5, the singer sings block B4 again after singing block B4. At this time, similarly to the block B1 described above, the control unit 11 detects the movement of the lips of the singer from the video data, specifies the phoneme pattern from the detected lip shape, and compares the specified phoneme pattern with the matching pattern. Then, the block B4 is specified. And the control part 11 performs the production | generation and wipe process of the lyric image data of the specified block B4.
Thus, in this embodiment, since the phoneme pattern of the head part of each block is collated, as shown to (b) of FIG. 5, a singer sings block B4 continuously, etc. lyrics data Even if the songs are sung in a different order, the lyrics telop corresponding to the lyrics actually sung can be displayed as shown in FIG. Can be prevented from being displayed.

  As described above, in this embodiment, the shape of the lip of the singer is detected to identify which part of the lyrics is being sung, and the lyrics telop of the identified part is displayed in synchronization with the video. A highly accurate lyrics telop can be added to the performance video. Also, since it is only necessary to detect the movement pattern of the singer's lips, the lyrics telop can be easily synchronized. Further, since the timing is detected and matched by analyzing the video, the lyrics display portion of the content created for karaoke can be applied to the live / video video.

By the way, when a singer does not sing according to a reference such as a CD, for example, the singing is performed by exchanging the numbers 2 and 3, or the singing is performed by changing the order of the chorus and other phrases, for example. Or, for example, when singing a large number of refrains, the actual lyrics are often not according to the reference. In such a case, the conventional apparatus often has a difference between the contents of the lyrics screen and the song (song itself).
On the other hand, in this embodiment, when the singer does not sing as the original song CD, for example, when singing No. 3 before No. 2 or singing more or less times However, it is possible to prevent the difference between the lyrics card (lyric data) and the actual performance.

<Modification>
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. An example is shown below. In addition, you may combine each following aspect suitably.
(1) In the above-described embodiment, the control unit 11 is configured to acquire the video data output from the photographing unit 18, but the configuration is not limited thereto, and the video data may be read from a storage unit such as a hard disk. In addition, a communication unit of the video display device 10 may be provided to receive video data via a communication network. In short, any mode may be used as long as the control unit 11 acquires video data.

(2) In the embodiment described above, the control unit 11 is configured to output video data obtained by superimposing the lyrics telop to the display unit 13. The output destination of the video data is not limited to the display unit 13. For example, the video display device 10 is provided with a communication unit, and the video data is transmitted to the other device connected via the communication network via the communication unit. It is good also as a structure to output. For example, the video data may be stored in a storage unit such as a hard disk. In short, any mode may be used as long as the control unit 11 outputs video data.

(3) In the embodiment described above, the delay unit 114 delays the video data acquired from the photographing unit 18 by the processing time of the video analysis unit 111 and the karaoke lyrics generation unit 112, but the delay unit 114 delays the video data. The time to be performed is not limited to this, and for example, a time for setting the wipe start timing of the lyrics a little earlier than the actual sing timing may be included. In this way, since the display and wipe of the lyrics telop is started a predetermined time before the singing start timing of the singer, the viewer can more appropriately grasp the timing of the lyrics.

(4) In the above-described embodiment, the singer identified the lyrics being sung by detecting the movement of the lips of the singer who analyzed the video. In addition to this, the voice data representing the singing voice of the singer may be voice-analyzed, and the lyrics that the singer is singing may be specified according to the analysis result. Specifically, for example, audio data in which a performance including a voice as an example of music or accompaniment is recorded in the storage unit 12 and a time code indicating the reproduction time is attached (hereinafter referred to as “first data”). (Sound data) ”is stored in advance. Then, the control unit 11 performs time alignment processing for associating the first audio data and the audio data input from the microphone 15 (hereinafter referred to as “second audio data”) in units of frames having a predetermined time length. A time shift between the first sound data and the second sound data is detected according to the result, and the generation timing of the lyrics video data is corrected so that the detected shift is eliminated. In this way, the display timing of the lyrics telop is corrected using the audio analysis result in addition to the video analysis result, so that the accuracy of the correction of the lyrics telop display timing can be further increased.

(5) In the above-described embodiment, the video display device is applied as the data playback device according to the present invention. However, the device applied as the data playback device is not limited to the video display device, for example, installed in a broadcasting station for television broadcasting. Various devices such as a dedicated computer device, a personal computer, and a mobile communication terminal can be applied as the data reproducing device according to the present invention.

(6) In the above-described embodiment, the case where the lip movement of the singer and the lyrics telop are synchronized has been described as an example. However, the present invention is not limited to singing, and a musical performance operation may be detected. In this case, for example, data representing the musical score of the music is stored instead of the lyrics data, and the control unit 11 performs the performance operation of the performer (for example, the hand on the keyboard when playing the keyboard instrument) from the video data. Position) may be detected, a phoneme combination pattern may be detected based on the detection result, the detected phoneme pattern may be compared with the verification pattern, and the part being played may be identified based on the verification result. In this case, the image of the score represented by the score data and the video of the performer can be synchronized. As described above, the “voice” in the present invention includes various sounds such as a voice generated by a person and a performance sound of a musical instrument.

(7) In the above-described embodiment, the data reproduction device is configured to output video data obtained by superimposing lyrics video data representing lyrics video on acquired video data (representing captured video). However, the present invention is not limited to this, and the data reproduction device may be configured to output only the lyric video data. Specifically, for example, it is configured to output lyric video data to a connected external display device at a timing in accordance with the video, and the external display device displays video data and lyric video data representing the captured video. May be combined and output.

(8) In the above-described embodiment, the control unit 11 detects the movement of the lips of the singer and detects the arrangement (combination) of vowels in the voice uttered by the singer from the detection result. For example, the arrangement (combination) of phonemes including consonants may be detected. In short, a phoneme combination pattern may be detected.

(9) In the embodiment described above, the control unit 11 starts generating lyric video data representing the video of the lyrics of the specified block at the timing when the block being sung is specified, thereby The display and the video display indicated by the video data were synchronized. The mode of synchronizing the lyrics telop and the video is not limited to this. For example, the lyric video data is generated and stored in advance, and the lyrics of the specified block at the timing when the block being sung is specified. The video data representing the video may be read out and output to the display means. In short, the singing block is specified, and the lyric video data representing the lyric video of the specified block is output to the display means at the timing when the singing block is specified.

(10) In the above-described embodiment, the control unit 11 specifies, for each block, the reproduction start timing of each block indicated by the synchronization information (information indicating the reproduction start timing of each block) and the block being sung. When the deviation from the timing is within a predetermined range, the lyric video data representing the lyric video of the block may be output to the display unit 13.

(11) In the embodiment described above, the video display device 10 has realized all the functions according to the embodiment. On the other hand, a plurality of devices connected via a communication network may share the above functions, and a system including the plurality of devices may realize the video display device 10 of the embodiment. For example, a dedicated computer device having a function of detecting lip movement and a function of generating a lyrics telop image and a terminal device having a display unit and a speaker may be configured as a system connected by a network. .

(12) The program realized by the control unit 11 of the video display device 10 described above is stored in a recording medium such as a magnetic tape, a magnetic disk, a flexible disk, an optical recording medium, a magneto-optical recording medium, a RAM, or a ROM. Can be provided. It is also possible to download the video display device 10 via a network such as the Internet.

It is a block diagram which shows an example of a structure of a video display apparatus. It is a figure which shows an example of the content of lyrics data. It is a figure which shows an example of the content of collation data. It is a block diagram which shows an example of a functional structure of a video display apparatus. It is a figure for demonstrating the display timing of a lyrics telop.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video display apparatus, 11 ... Control part, 12 ... Memory | storage part, 13 ... Display part, 14 ... Operation part, 15 ... Microphone, 16 ... Audio | voice processing part, 17 ... Speaker, 18 ... Shooting part, 111 ... Video analysis part 112 ... karaoke lyrics generation unit, 113 ... video playback unit, 121 ... lyric data storage area, 122 ... collation data storage area.

Claims (6)

Lyrics data storage means for storing lyrics data representing the lyrics of a song and divided into a plurality of blocks;
Video data acquisition means for acquiring video data including the video of the singer;
Detection means for detecting the shape of the lips of the singer from the video data acquired by the video data acquisition means;
Phoneme pattern detection means for detecting the arrangement of phonemes included in the voice uttered by the singer from the shape of the lips detected by the detection means;
The in-singing block identifying means for collating the phoneme pattern detected by the phoneme pattern detecting means with the lyrics data for each block, and identifying the block that the singer is singing based on the matching result;
Lyrics video data generating means for reading out the lyrics data from the lyrics data storage means and generating lyrics video data representing a video of lyrics indicated by the read lyrics data;
At the timing when the block being sung is specified by the singing block specifying unit, the lyric video data representing the video of the lyrics of the specified block among the lyric data generated by the lyric video data generating unit is output to the display unit And a data reproducing apparatus. The lyric video data generation means starts generating lyric video data representing a lyric video of the specified block at a timing when the block being sung is specified by the singing block specifying means. Item 4. The data reproducing device according to Item 1. In the data reproducing device according to claim 1 or 2,
The lyrics data is divided into a plurality of blocks, and includes synchronization information indicating a reproduction start timing of each divided block,
The output means, for each of the blocks, when the deviation between the reproduction start timing of each block indicated by the synchronization information and the timing at which the block is specified by the block specifying means is within a predetermined range, A data reproduction apparatus characterized by outputting lyric video data representing a lyric video of the block. The data reproduction apparatus according to any one of claims 1 to 3,
The data output apparatus characterized in that the output means outputs the video data acquired by the video data acquisition means and the lyrics video data generated by the lyrics video data generation means. The data reproducing apparatus according to claim 4, wherein
Delay means for delaying the video data acquired by the video data acquisition means by a predetermined time;
The data output device characterized in that the output means outputs the video data delayed by the delay means and the lyrics video data generated by the lyrics video data generation means. The data reproducing apparatus according to any one of claims 1 to 5,
First voice data storage means for storing first voice data representing voice;
Voice data acquisition means for acquiring second voice data representing voice;
Time alignment means for associating the second sound data acquired by the sound data acquisition means with the first sound data stored in the first sound data storage means in units of frames of a predetermined time length;
The output means displays, on the display means, lyric video data representing a lyric video of a specified block among the lyric data generated by the lyric video data generating means at a timing according to a result of association by the time alignment means. A data reproducing device characterized by outputting.

Images