JP2007079413A - Audio reproduction device, audio distribution system, audio reproduction program and authoring program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an audio reproduction device by which a KARAOKE function is attained using audio data with song sounds. <P>SOLUTION: A player 22 reads a multimedia file 21 having the audio data with the song sounds, text data, graphic data and sequence information for drawing texts and graphics from a disk 5. In the audio data in the multimedia file 21, the song sounds are canceled by a song sound cancellation module 24 after decoding and reproduced from a loudspeaker 8 via a D/A part 7. On the other hand, both of the text data and the graphic data in the multimedia file 21 are output to a drawing module 9 after decoding. The drawing module 9 displays texts and graphics on a display according to reproduction of audio according to the sequence information in the multimedia file 21. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an audio playback device capable of playing back audio data, an audio distribution system, an audio playback program, and an authoring program for editing audio data.

  2. Description of the Related Art Conventionally, music playback devices such as mobile phones that can play back music data have been developed that have a karaoke function using music data in MIDI (Musical Instruments Digital Interface) format without singing sound. Such a karaoke function in a mobile phone is realized by displaying graphic data and text data in synchronization with music reproduced using a MIDI sound source.

On the other hand, with an increase in the capacity of communication lines, a service for distributing music with original singing sound to mobile phones as audio data comparable to the quality of a CD (Compact Disc) has been started.
Note that Patent Document 1 is known as a reference of the prior art related to the present application.
JP 2001-197168 A

  However, in the conventional mobile phone, since the music data in the MIDI format is used for the karaoke function, the performance to be played is less realistic than the CD quality. Further, when using the karaoke function, it is necessary to newly acquire MIDI format music data even if the original music with singing sound is held in the mobile phone.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an audio reproducing apparatus capable of realizing a karaoke function using audio data with singing sound.

  The present invention has been made to solve the above problems, and the invention according to claim 1 stores audio data, one or more text data, and sequence data indicating sequence information for displaying the text data. Data storage means, singing sound canceling means for canceling singing sound from the audio data, audio reproducing means for reproducing audio data after canceling the singing sound, and the sequence at the time of audio data reproduction by the audio reproducing means An audio reproducing apparatus comprising: a drawing control unit that controls display of the text data based on data.

  According to a second aspect of the present invention, in the first aspect of the invention, the data storage means includes one or a plurality of image data and second sequence data indicating the sequence information for displaying the image data. Further, the drawing control means further controls the display of the image data based on the second sequence data when the audio reproduction means reproduces the audio data.

  According to a third aspect of the present invention, in the first or second aspect of the invention, the data storage means further stores MIDI format music data, and the audio reproduction means receives the singing sound. The music data is reproduced in synchronization with the reproduction of the audio data after cancellation.

  The invention according to claim 4 is the invention according to any one of claims 1 to 3, wherein the data storage means further stores cancellation period information for specifying a period for canceling the singing sound. The singing sound canceling means cancels the singing sound of the audio data based on the cancellation period information.

  According to a fifth aspect of the present invention, in the invention according to any one of the first to fourth aspects, the singing sound canceling means is in accordance with a user instruction when the audio data is reproduced by the audio data reproducing means. It is possible to execute or stop the singing sound canceling process.

  According to a sixth aspect of the present invention, there is provided storage means for storing audio data and text data, sequence data indicating one or a plurality of the text data and sequence information for displaying the text data for the audio data, A multimedia file production means for producing a multimedia file to which is added, a delivery server comprising a delivery means for delivering the produced multimedia file, and a data storage means for receiving and storing the delivered multimedia file Singing sound canceling means for canceling the singing sound from the audio data included in the multimedia file, audio playback means for playing back the audio data after canceling the singing sound, and at the time of audio data playback by the audio playback means Maru An audio distribution system characterized in that it is composed of an audio reproducing apparatus for and a drawing control unit for controlling the display of the text data on the basis of the sequence data contained in the media file.

  Further, the invention according to claim 7 is directed to storage means for storing audio data and text data, singing sound cancellation means for canceling singing sound from the audio data, and audio data after canceling the singing sound. Multimedia file producing means for producing a multimedia file to which one or a plurality of text data and sequence data indicating the sequence information for displaying the text data are added, and delivery means for delivering the produced multimedia file. A delivery server; data storage means for receiving and storing the delivered multimedia file; audio playback means for playing back audio data in the multimedia file; and the multimedia playback at the time of audio data playback by the audio playback means An audio distribution system characterized in that it is composed of an audio reproducing apparatus for and a drawing control unit for controlling the display of the text data on the basis of the sequence data contained in the file.

  According to an eighth aspect of the present invention, there is provided a data storage procedure for causing a computer to store audio data, one or more text data, and sequence data indicating the sequence information for displaying the text data, and singing from the audio data. A singing sound canceling procedure for canceling the sound, an audio playing procedure for playing back the audio data after canceling the singing sound, and controlling the display of the text data based on the sequence data during the audio data playing in the audio playing procedure And an audio playback program for executing a drawing control procedure to be executed.

  The invention according to claim 9 is an authoring for causing a computer to execute a procedure for adding one or more text data and sequence data indicating sequence information for displaying the text data to audio data. It is a program.

  The invention according to claim 10 is directed to a computer for canceling the singing sound from the audio data, and to the audio data after canceling the singing sound. An authoring program for executing a procedure of adding sequence data indicating sequence information for data display.

  According to the present invention, the user can enjoy karaoke according to the accompaniment of the original song because the audio data of the original song can be reproduced after performing the process of canceling the song sound. . Also, audio data with one singing sound can be reproduced as it is and used for music appreciation, or the singing sound can be canceled and used for karaoke. Furthermore, by designating a period during which the singing sound is canceled from the audio data, it is possible to prevent deterioration of the sound quality of the accompaniment portion due to the singing sound canceling process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram showing the configuration of a mobile phone (audio playback device) according to an embodiment of the present invention. In FIG. 1, a CPU 1 is a CPU (Central Processing Unit) that controls each part in the mobile phone.

  The ROM 2 is a ROM (Read Only Memory) that holds programs executed by the CPU 1. The RAM 3 is a RAM (Random Access Memory) that provides a memory area used by the CPU 1.

  The interface 4 is an interface for connecting to the data disk 5 by serial communication or parallel communication. The data disk 5 (data storage means) is a disk for storing multimedia files, and the CPU 1 reads the multimedia files in the data disk 5 via the interface 4. Here, the multimedia file refers to a file in which graphic data (image data), text data, MIDI data, etc. are combined with audio data.

  FIG. 2 is a diagram showing an example of the format structure of the multimedia file. In FIG. 2, data in the format structure is managed in chunk units. Contents Info Chunk stores data for file management such as content type and content duplication permission. Optional Data Chunk includes unique information of music data.

  The MIDI Track Chunk stores sequence data that is reproduced by the sound source 11. Setup Data Chunk stores setup parameters, tone parameters, effect data, etc. for the entire MIDI Track Chunk. In the Sequence Data Chunk, sequence data reproduced by the sound source 11 is stored.

  Here, the sequence data is a collection of MIDI events having time information. The MIDI event is, for example, a sound generation event (note on / off), a tone color event (program change), a volume change or pan pot change event (control change), a pitch change event (pitch change), or the like.

  Audio Track Chunk stores sampled audio data and sequence data for actual sound generation. Setup Data Chunk stores setup parameters for the entire Audio Track Chunk. The Sequence Data Chunk stores sequence data for generating audio data.

  Here, the sequence data includes an event associated with audio data for sounding and muting in synchronization with time, and a duration for designating timing. In addition, events for changing the volume or panpot are included in the sequence data.

  In the Wave Data Chunk, actual audio data is compressed and stored in a specific compressed audio format. The Wave Data Chunk header stores information such as the type of audio data, sampling frequency, and bits. One Wave Data Chunk corresponds to one audio data, and a plurality of Wave Data Chunks (that is, a plurality of audio data) can be stored in the Audio Track Chunk.

  In the Audio Track Chunk, management is performed by assigning an ID to each Wave Data Chunk, and this ID is associated with the sounding event of the Sequence Data Chunk. In the present embodiment, one or a plurality of graphics (images) and text are displayed in synchronization with one audio data. Therefore, one Wave Data Chunk is stored in the Audio Track Chunk.

  The Graphics Track Chunk stores drawing sequence data, graphic data (image data), text data, and the like for display on the display 10. Setup Data Chunk stores setting parameters for the entire Graphics Track Chunk. The Sequence Data Chunk stores sequence data for drawing on the display 10.

Here, the sequence data is represented by a numerical expression (Coordinates) for specifying coordinates and a numerical expression (Duration) for specifying display timing, display time length, etc., and text and images are displayed in synchronization with time. A sequence for erasing is described.

  The Graphics Track Chunk assumes a display device having a plurality of virtual planes, and the Graphics Track Chunk can store a plurality of Sequence Data Chunks corresponding to each virtual plane. In this case, the result of combining the virtual planes is displayed on the actual display device (display 10).

  Returning to FIG. 1, the DSP 6 is a DSP (Digital Signal Processor) that performs real-time decoding processing on the file format of the audio data stored in the Audio Track Chunk of FIG. The D / A unit 7 is a DAC (Digital-to-Analog Converter) that converts audio data decoded by the DSP 6 or digital musical tone data generated by the sound source 11 into an analog audio signal. The speaker 8 (audio reproduction means) is a speaker that produces an analog audio signal converted by the D / A unit 7.

  The drawing module 9 (drawing control means) controls display on the display 10 based on graphic data and text data in the multimedia file. The display 10 is a display that displays the output of the drawing module 9. The sound source 11 is a sound source that interprets MIDI melody data stored in the MIDI Track Chunk of FIG. 2 and generates digital musical tone data by the FM (Frequency Modulation) method or the WT (Wave Table) method. The bus line 12 is a bus line that connects each part in the mobile phone. The input unit 13 is an input unit having operation keys and the like for inputting user instructions.

  In addition to the functions shown in FIG. 1, the cellular phone of the present invention has a telephone function, a function for transmitting / receiving files by e-mail, and a function for acquiring files via the Internet. Further, it has a light emitting function such as an LED (Light Emitting Diode) and a vibration function such as a vibrator.

  Next, the operation of the above-described embodiment will be described with reference to FIGS. FIG. 3 is a functional configuration diagram showing functions executed by the mobile phone of FIG. 1, and FIG. 4 is a diagram showing a procedure for playing a song by the mobile phone of FIG.

  When the user gives an instruction to select a song using the input unit 13 (step S101), the player 22 realized as a function of the CPU 1 reads the corresponding multimedia file 21 from the data disc 5 (step S102). . Here, the multimedia file 21 has the format shown in FIG. 2, and the original audio data with singing (stored in the Audio Track Chunk), one or a plurality of text data, and one in one file. It is possible to include one or more graphic data (stored in the Graphics Track Chunk) and synchronize each data on the time axis.

  Subsequently, when the user gives an instruction to reproduce the song selected using the input unit 13 (step S103), the player 22 starts the reproduction process of the multimedia file 21 read in step S102 (step S104). Thereafter, the player 22 determines processing to be performed at each time according to the sequence data stored in each chunk of the multimedia file 21, and distributes the output destination of the data according to the type of data to be processed. (Step S105).

  When processing the audio data in the Audio Track Chunk, the player 22 outputs the audio data to the audio decoder 23 which is one function of the DSP 6. The audio decoder 23 decodes the input audio data in a linear audio data format in real time (step S106). Here, the audio data stored in the multimedia file 21 is compressed in a format such as MP3 format or AAC format.

  The audio data decoded by the audio decoder 23 is then output to the singing sound canceling module 24 (singing sound canceling means) which is one function of the DSP 6. The singing sound cancellation module 24 detects a singing sound from the input audio data, performs a process of canceling the detected singing sound, and generates singing-free audio data 25 (step S107).

  Here, the data for designating the period for canceling the singing sound (for example, the time from the start of the singing song) is added to the multimedia file of FIG. It is also possible to perform the singing sound cancellation process only for the singing sound.

  In addition, a button for selecting on / off of the singing sound cancellation is displayed on the screen of the display 10, and the user can operate the input unit 13 during the playback of the singing song to turn on / off the singing sound cancellation. . The singing sound cancellation module 24 executes or stops the singing sound cancellation process in accordance with a user instruction.

  With regard to the method of canceling the singing sound, for example, there are known methods such as removing the component localized at the center from the left and right stereo audio signals, or removing the middle range component. The processing such as band limitation that affects not only the singing sound but also the sound quality of the performance sound, but by specifying the section where the singing sound is canceled as described above, the sound quality is maintained for only the performance sound. be able to.

  The non-singing audio data 25 generated by the singing sound cancellation module 24 is converted into an analog audio signal by the D / A unit 7, amplified by an amplifier (not shown), and then generated from the speaker 8 (step S108). .

  On the other hand, when processing the graphic data in the Graphic Track Chunk in step S105, the player 22 outputs the graphic data to the graphic decoder 26 which is one function of the CPU 1. The graphic decoder 26 decodes the graphic data (for example, compressed in JPEG format or PNG format) input from the player 22 into actual data to be displayed on the display 10 (step S109).

  When processing the text data in the Graphic Track Chunk in step S105, the player 22 outputs the text data to the text decoder 27 which is a function of the CPU 1. The text decoder 27 replaces the text data input from the player 22 with a font and decodes it into actual data to be displayed on the display 10 (step S110).

  The graphic data and text data decoded in steps S109 and S110 are output to the drawing module 9. The drawing module 9 performs processing such as lyric (text) wipe, graphic movement, display switching, and the like according to the drawing time sequence based on the input graphic data and text data (step S111), and displays them on the display 10. (Step S112).

  When processing the MIDI data in the MIDI Track Chunk in step S105, the player 22 outputs the MIDI data to the sound source 11. The sound source 11 generates digital musical tone data based on the input MIDID data. The generated digital musical tone data is converted into an analog musical tone signal by the D / A section 7, amplified by an amplifier (not shown), and then reproduced as a melody from the speaker 8 (step S113).

  In addition, although not described in the format format of the multimedia file in FIG. 2, it is also possible to drive the LED in synchronization with the playback of the song (step S114) or drive the vibrator (step S115). is there.

  When the process related to one piece of data is finished, if the reproduction of the song is not finished (step S116: No), the process returns to step S105, the player 22 determines the process to be performed at the next time, and the data to be processed. Depending on the type of data, the output destination of the data is distributed. Thereafter, the process of steps S105 to S116 is repeatedly executed during the reproduction of the song.

  In this way, by using the mobile phone according to the present embodiment, the user listens to a song in which the singing sound has been canceled from the audio data with singing, and karaoke while watching the text and graphics displayed on the display 11 of the mobile phone. Can enjoy.

  Moreover, since the audio data used in karaoke is not the MIDI data conventionally used but the data of the original song, the user can sing with the original accompaniment. Furthermore, it is not necessary to prepare a plurality of audio data for music appreciation and for karaoke, and one audio data can be used for both music appreciation and karaoke.

  Next, a method for producing a multimedia file in which lyrics (text) data and graphic data are combined with audio data with a song will be described. Multimedia file production methods can be broadly divided into two methods: a user-created method and a distribution server that distributes song songs.

  When a user creates a multimedia file, existing audio data with singing and text data and graphic data as lyrics are prepared. A user creates sequence data indicating the display timing of text data and graphic data using a dedicated authoring tool on a personal computer, and adds it to audio data to produce a multimedia file. After production of the multimedia file, the multimedia file is transferred from the personal computer to the mobile phone by some communication means.

  Next, a case where a multimedia file is created by the distribution server will be described with reference to FIG. FIG. 5 is a configuration diagram showing the configuration of an audio distribution system that distributes audio from a distribution server to a mobile phone in the form of a multimedia file.

  In FIG. 5, a mobile phone 51 (audio reproduction device) is a mobile phone having the structure shown in FIG. The public line network 52 is a network that connects the mobile phone 51 and the distribution server 53. The distribution server 53 is a server that creates a multimedia file and distributes it to the mobile phone 51. The database 54 (storage means) is a database connected to the distribution server 53, and holds audio data, lyrics data, graphic data, etc. of the song.

  Subsequently, a procedure until the multimedia file is distributed to the mobile phone 51 through the audio distribution network of FIG. 5 will be described. First, the user designates a song to be distributed to the distribution server 53 on the WEB screen of the mobile phone 51. The distribution server 53 reads the audio data of the designated song and corresponding lyric data and graphic data from the database 54.

  Subsequently, the distribution server 53 produces a multimedia file in the format shown in FIG. 2 from the read audio data, lyrics data, and graphic data (multimedia file production means). Finally, the distribution server 53 distributes the created multimedia file to the mobile phone 51 (distribution means).

  A plurality of lyrics data and graphic data corresponding to one audio data are prepared in the database 54, and the user selects one of the plurality of lyrics data and graphic data on the WEB screen of the mobile phone 51, and the distribution server 53 can also create a multimedia file from the selected lyric data / graphic data and audio data.

  In addition, when the user already possesses audio data with a song, a method of transmitting the audio data from the mobile phone 51 to the distribution server 53 and receiving distribution of the multimedia file from the distribution server 53 is also conceivable. . At this time, the distribution server 53 reads the lyrics data / graphic data corresponding to the received audio data with singing from the database 54, creates a multimedia file, and then sends the multimedia data to the mobile phone 51 that is the transmission source of the audio data. Deliver the file.

  As mentioned above, although embodiment of this invention was explained in full detail, the concrete structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included. For example, in the present embodiment, the mobile phone performs the process of canceling the singing sound in real time during the reproduction of the audio, but it may be performed on the personal computer using an authoring tool or the like, and the distribution server 53 in FIG. It may be performed before delivery of the multimedia file. The audio data need not be compressed in a specific format, and the present invention can be applied to uncompressed audio data.

  The present invention is suitable for use in an audio reproducing apparatus having a karaoke function and capable of reproducing audio data.

It is a block diagram which shows the structure of the mobile telephone which concerns on embodiment of this invention. It is the figure which showed an example of the format structure of a multimedia file. It is a functional block diagram which showed the function performed with the mobile telephone of FIG. It is a figure which shows the procedure which reproduces | regenerates song song with the mobile telephone of FIG. 1 is a configuration diagram showing a configuration of an audio distribution system that distributes audio from a distribution server to a mobile phone in a multimedia file format.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 3 ... RAM, 4 ... Interface, 5 ... Data disk (data storage means), 6 ... DSP, 7 ... D / A part, 8 ... Speaker (audio reproduction means), 9 ... Drawing module (Drawing control means) 10 ... Display, 11 ... Sound source, 12 ... Bus line, 13 ... Input section, 24 ... Singing sound cancellation module (singing sound canceling means), 51 ... Mobile phone (audio playback device), 52 ... Public Network, 53 ... distribution server, 54 ... database (storage means)

Claims (10)

Data storage means for storing audio data, one or more text data, and sequence data indicating sequence information for displaying the text data;
Singing sound canceling means for canceling the singing sound from the audio data;
Audio playback means for playing back the audio data after canceling the singing sound;
An audio reproducing apparatus comprising: a drawing control unit that controls display of the text data based on the sequence data when the audio reproducing unit reproduces audio data. The data storage means further stores one or a plurality of image data and second sequence data indicating the sequence information for displaying the image data,
2. The audio reproducing apparatus according to claim 1, wherein the drawing control unit further controls display of the image data based on the second sequence data when the audio data is reproduced by the audio reproducing unit. The data storage means further stores music data in MIDI format,
The audio reproducing apparatus according to claim 1 or 2, wherein the audio reproducing means reproduces the music data in synchronization with reproduction of the audio data after canceling the singing sound. The data storage means further stores cancellation period information for specifying a period for canceling the singing sound,
The audio playback device according to any one of claims 1 to 3, wherein the singing sound canceling unit cancels the singing sound of the audio data based on the cancellation period information.   5. The singing sound canceling process of the singing sound canceling means can be executed or stopped according to a user instruction at the time of audio data reproduction by the audio data reproducing means. Audio playback device. Multimedia for producing a multimedia file in which storage means for storing audio data and text data, and one or more text data and sequence data indicating sequence information for displaying the text data are added to the audio data A file production means; a distribution server comprising distribution means for distributing the produced multimedia file;
Data storage means for receiving and storing the distributed multimedia file, singing sound cancellation means for canceling the singing sound from the audio data included in the multimedia file, and reproducing the audio data after canceling the singing sound An audio playback device comprising: audio playback means; and drawing control means for controlling display of the text data based on sequence data included in the multimedia file when the audio data is played back by the audio playback means. An audio distribution system characterized by that. Storage means for storing audio data and text data, singing sound cancellation means for canceling a singing sound from the audio data, one or more text data and the text data for the audio data after canceling the singing sound A multimedia file production means for producing a multimedia file to which sequence data indicating sequence information for display is added; a distribution server comprising a distribution means for distributing the produced multimedia file;
Data storage means for receiving and storing the distributed multimedia file; audio playback means for playing back audio data in the multimedia file; and audio data playback by the audio playback means included in the multimedia file An audio distribution system comprising: an audio playback device comprising drawing control means for controlling display of the text data based on sequence data. On the computer,
A data storage procedure for storing audio data, one or more text data, and sequence data indicating sequence information for displaying the text data;
Singing sound cancellation procedure for canceling the singing sound from the audio data;
An audio playback procedure for playing back the audio data after canceling the singing sound;
An audio reproduction program for executing a drawing control procedure for controlling display of the text data based on the sequence data during reproduction of audio data in the audio reproduction procedure. On the computer,
An authoring program for executing a procedure for adding one or more text data and sequence data indicating sequence information for displaying the text data to audio data. On the computer,
Singing sound cancellation procedure for canceling singing sound from audio data,
An authoring program for executing a procedure for adding one or more text data and sequence data indicating sequence information for displaying the text data to the audio data after canceling the singing sound.

Images