JP2006215460A - Karaoke sound transmitting and receiving system and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a Karaoke sound transmitting and receiving system which reduces the load on channels by suppressing the communication amount and reproduce the high quality Karaoke sound of accompaniment data and vocal data put together. <P>SOLUTION: The control server (a) distributes accompaniment data to a singer terminal (b) and a listener terminal (c), respectively. The singer terminal (b) reproduces the accompaniment data. The singer listens to the reproduced accompaniment and starts to sing following it. The singer terminal (b) generates sound packets based on the singing voice inputted through a microphone. The information of the timing with the accompaniment data is attached to the sound packets. The singer terminal (b) sequentially transmits the generated sound packets to the listener terminal (c). The listener terminal (c) superimposes the accompaniment data and the sound data by synchronizing them on the basis of the timing information in the sound packets and outputs the reproduced Karaoke sound. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a karaoke voice transmission / reception system realized on a network between a singer who sings karaoke and a listener who views the karaoke.

  A technique for distributing karaoke accompaniment and singing voice using a communication technique called voice chat is known. For example, when a singer's terminal plays a karaoke accompaniment and the singer sings along with it, the karaoke accompaniment and the singing are synthesized and distributed in real time to the listener's terminal via a network such as the Internet. . Thereby, a virtual karaoke box can be realized on the Internet.

  On the other hand, when chorus singing data created by a singer with his own voice is recorded in advance on a server, and the karaoke device plays back accompaniment data of karaoke, the chorus singing data acquired from the server is played back simultaneously. A karaoke system (see, for example, Patent Document 1) is known. Thereby, the chorus performance using the voice of the person can be performed at the time of karaoke accompaniment.

JP 2004-53736 A

  However, in the voice chat described above, the singer-side terminal simultaneously distributes karaoke accompaniment data and singer singing data to the listener-side terminal, which increases the amount of communication and increases the line load on the network. There was a problem that became high. For this reason, depending on the network environment to which the listener side terminal is connected, the sound quality of the karaoke performance by the singer is lowered, and there is a problem that the listener side is not entertaining.

  On the other hand, in the karaoke system that can perform the chorus performance using the voice of the person described above, it is premised that the singing data for chorus is transmitted in a batch and reproduced simultaneously with the karaoke accompaniment, so that it is distributed in real time. There was no consideration about the traffic and line load. For this reason, it was not premised on realizing a virtual karaoke box on the network.

  The present invention has been made to solve the above-mentioned problems, and reduces the line load by suppressing the communication amount, and karaoke performance in which accompaniment data and singing data are synthesized with high sound quality. An object is to provide a karaoke voice transmission / reception system that can be played.

  (1) to (7) (18) A karaoke voice transmission / reception system according to the present invention is a karaoke voice transmission / reception system including a management server, at least one singer terminal and at least one listener terminal, and (a) a management server (A1) Accompaniment data recording means for recording accompaniment data related to a plurality of karaoke songs; (a2) Upon receiving a request from the singer terminal, accompaniment data related to the karaoke song corresponding to the request is sent to the singer terminal and the listener terminal. And (b) a singer terminal that receives an operation from the singer and requests a management server to request a singer-desired karaoke song, and (b2) the management server. Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from (B3) accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment; (b4) singing voice input means for inputting a voice sung by a singer in accordance with the reproduced karaoke accompaniment; and (b5 ) Based on the input voice, voice packet generation means for generating a voice packet to which synchronization information with the karaoke accompaniment is added; and (b6) voice packet transmission means for transmitting the generated voice packet to the listener terminal. (C) The listener terminal (c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke music distributed from the management server; (c2) Acquire a voice packet transmitted from the singer terminal Voice packet acquisition means, and (c3) when synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet; In, characterized in that and a performance output means for outputting them as combined and karaoke performance.

  Therefore, accompaniment data and singing voice can be transmitted separately. For example, accompaniment data is distributed in advance, and the singing voice is transmitted in real time, whereby the communication load can be suppressed and the line load can be reduced. Moreover, since only the singing voice needs to be transmitted in real time, the listener can reproduce and output a high-quality karaoke performance.

  (8) to (14) (19) A karaoke voice transmission / reception system according to the present invention is a karaoke voice transmission / reception system including a management server, at least one singer terminal and at least one listener terminal, and (a) a management server (A1) Accompaniment data recording means for recording accompaniment data related to a plurality of karaoke songs; (a2) Upon receiving a request from the singer terminal, accompaniment data related to the karaoke song corresponding to the request is sent to the singer terminal and the listener terminal. Accompaniment data distribution means for distribution; (a3) voice packet acquisition means for acquiring voice packets transmitted from the singer terminal; (a4) voice packet transmission means for transmitting the acquired voice packets to the listener terminal; (B) The singer terminal receives the operation from the singer and (b1) the singer desired karaoke in the management server. (B2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the request distributed from the management server; (b3) Playing the acquired accompaniment data as a karaoke accompaniment (B4) singing voice input means for inputting a voice sung by a singer in accordance with the reproduced and outputted karaoke accompaniment; and (b5) synchronization with the karaoke accompaniment based on the input voice. Voice packet generating means for generating a voice packet to which information is added; and (b6 ′) voice packet transmitting means for transmitting the generated voice packet to the management server, and (c) the listener terminal is (c1) management Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the server, and (c2 ′) transmitted from the management server Voice packet acquisition means for acquiring voice packets; and (c3) synchronizing the accompaniment data and the voice packets based on the synchronization information added to the voice packets and combining them to output as a karaoke performance And a performance output means.

  Therefore, accompaniment data and singing voice can be transmitted separately. For example, accompaniment data is distributed in advance, and the singing voice is transmitted in real time, whereby the communication load can be suppressed and the line load can be reduced. Moreover, since only the singing voice needs to be transmitted in real time, the listener can reproduce and output a high-quality karaoke performance. Moreover, since the singer terminal should just transmit song voice data only to a management server, the load in a singer terminal can be reduced. That is, the communication speed in the network environment of the singer terminal can be lowered.

  (15) In the karaoke voice transmitting / receiving system according to the present invention, the voice packet generating means of the singer terminal generates a voice packet only when the voice sung by the singer is detected. Therefore, it is possible to prevent the generation of voice packets in the interlude part and the silent part and to effectively use the network band.

  (16) In the karaoke voice transmission / reception system according to the present invention, the voice packet generation means of the singer terminal plays at least one of key information, tempo information, lyrics timing information, and volume information related to karaoke accompaniment set by the singer. Append to packet. Therefore, even if the key, tempo, lyrics timing, volume, etc. are changed in the singer terminal, the listener terminal can reliably reproduce the change and reproduce and output the karaoke performance. Thereby, a karaoke voice transmission / reception system with a strong sense of reality can be realized.

  (17) In the karaoke voice transmission / reception system according to the present invention, the voice packet acquisition means of the listener terminal compares the performance time output by the performance output means with the synchronization information added to the voice packet and acquires the voice It is determined whether or not the packet is in time for output, and if the output is not in time, the voice packet is discarded. Therefore, voice packets that are not expected to be used can be discarded at an early stage to eliminate useless processing.

  The “accompaniment data recording means” in the management server corresponds to the accompaniment data database 2093 shown in FIG. 12 in the embodiment. The “accompaniment data distribution means” corresponds to the function of step S423 shown in FIG. 4 in the embodiment. In the embodiment, “voice packet receiving means” corresponds to the function of step S551 shown in FIG. In the embodiment, “voice packet transmitting means” corresponds to the function of step S553 shown in FIG. 5a.

  In the embodiment, the “request means” in the singer terminal corresponds to the function of step S405 shown in FIG. The “accompaniment data acquisition means” corresponds to the function of step S407 in the embodiment. The “accompaniment data reproducing means” corresponds to the functions of steps S409 to S4011 shown in FIG. 4 in the embodiment. In the embodiment, the “singing voice input means” corresponds to the function of step S501 shown in FIG. In the embodiment, the “voice packet generation means” corresponds to the functions of steps S502 to S509 shown in FIG. In the embodiment, the “voice packet transmitting means” corresponds to the function of step S511 shown in FIG.

  In the embodiment, the “accompaniment data acquisition means” in the listener terminal corresponds to the function of step S445 shown in FIG. In the embodiment, the “voice packet acquisition unit” corresponds to the function of step S521 shown in FIG.

The “performance output means” corresponds to the function of steps S523 to 531 shown in FIG. 5 in the embodiment.

  In the present invention, “to means” is a concept including a function of a CPU realized by a program. Here, the “program” is a concept including not only a program that can be directly executed by the CPU but also a program in a source format, a compressed program, an encrypted program, and the like.

1. First embodiment 1-1. Overview FIG. 1 is a functional block diagram of a karaoke voice transmitting / receiving system according to an embodiment of the present invention. The management server a is a computer device installed in a company that operates a website or the like that implements a karaoke voice transmission / reception system. The singer terminal b is a computer device installed on the singer side who sings karaoke performance in the karaoke voice transmission / reception system. The listener terminal c is a computer device installed on the listener side who views the singer's karaoke performance in the karaoke voice transmission / reception system. The management server a, the singer terminal b, and the listener terminal c are each connected to a network 0 such as the Internet, and can communicate with each other. There may be a plurality of singer terminals b and listener terminals c.

  The listener terminal c accesses the management server a indicated by the URL via the Internet. A virtual karaoke stage is prepared on the website of the management server a, and the listener can view an arbitrary stage via the network.

  The singer of the singer terminal b accesses the management server a indicated by the URL via the Internet. In order to perform a karaoke performance, when a singer inputs a request for a desired karaoke accompaniment to the singer terminal b, the singer terminal b transmits the request to the management server a. Upon receiving the transmission, the management server a reads the accompaniment data relating to the request from the database, and distributes the read accompaniment data to each of the singer terminal b and the listener terminal c.

  Receiving the accompaniment data, the singer terminal b reproduces accompaniment data for the desired karaoke accompaniment. The singer starts singing along with the karaoke accompaniment while viewing the reproduced karaoke accompaniment. The singer terminal b inputs a singing voice via a microphone. In response to the input, the singer terminal b generates a voice packet based on the singing voice. At this time, timing information with accompaniment data is added to the voice packet. The singer terminal b sequentially transmits the generated voice packets to the listener terminal c. Upon receiving the transmission, the listener terminal c superimposes the previously distributed accompaniment data and the voice packet on the basis of the timing information of the voice packet, and reproduces and outputs it as a singer's karaoke performance.

1-2. Functional Block Diagram The request unit b1 of the singer terminal b is for receiving a request for a singer desired karaoke accompaniment from the singer and transmitting the request to the management server a. The accompaniment data acquisition means b2 is for acquiring accompaniment data for a request for a karaoke accompaniment transmitted to the management server a from the management server a. The accompaniment data reproducing means b3 is for reproducing the accompaniment data acquired above as a karaoke accompaniment. The singing voice input means b4 is for inputting the voice sung by the singer along with the reproduced karaoke accompaniment via the microphone. The voice packet generation means b5 is for generating a voice packet to which timing information with karaoke accompaniment is added based on the input singer's singing voice. The voice packet transmitting means b6 is for transmitting the generated voice packet to another device such as a listener terminal.

  Accompaniment data acquisition means c1 of the listener terminal is for acquiring accompaniment data from the management server a in response to a request for a karaoke accompaniment transmitted from the singer terminal to the management server a. The voice packet acquisition unit c2 is for acquiring voice packets sequentially transmitted from other devices such as a singer terminal. The performance output means c3 reproduces and outputs the singer's karaoke performance at the listener terminal c by superimposing the accompaniment data and the voice packet in synchronism based on the timing information added to the voice packet.

  The accompaniment data recording means a1 of the management server a is for recording a plurality of accompaniment data for karaoke accompaniment music. The accompaniment data distribution means a2 is for receiving a request from the singer terminal b and transmitting accompaniment data corresponding to the request to the singer terminal and the listener terminal.

1-3. Hardware configuration 1-3-1. Management Server FIG. 2 shows an example of a hardware configuration that implements the management server a shown in FIG. 1 using a CPU. The management server a includes a display 201, a CPU 203, a memory 205, a keyboard / mouse 207, a hard disk 209, a CD-ROM drive 211, and a communication circuit 215.

  On the hard disk 209, a virtual karaoke server program 2091 and an accompaniment data database 2093 are recorded. The virtual karaoke server program 2091 receives various accesses from the singer terminal b and the listener terminal c, and performs various processes related to the karaoke voice transmission / reception system according to the present invention. This program is installed by reading the program recorded on the CD-ROM 212 via the CD-ROM drive 211. The installation may be performed using a program downloaded from the network 0 such as the Internet using the communication circuit 215.

  Accompaniment data distribution means a2 constituting the management server a is realized by each function of the virtual karaoke server program 2091. The accompaniment data recording means a1 is realized by an accompaniment data database 2093.

1-3-2. Singer terminal / listener terminal FIG. 3 shows an example of a hardware configuration in which the singer terminal b and the listener terminal c shown in FIG. 1 are realized using a CPU. The singer terminal b / listener terminal c includes a display 301, a CPU 303, a memory 305, a keyboard / mouse 307, a speaker 308, a hard disk 309, a CD-ROM drive 311, a communication circuit 315, and a microphone 317.

  A virtual karaoke client program 3091 is recorded on the hard disk 309. The virtual karaoke client program 3091 accesses the management server a or the listener terminal c via the network 0, and performs various processes related to the karaoke voice transmission / reception system according to the present invention.

  The request means b1, accompaniment data acquisition means b2, accompaniment data reproduction means b3, singing voice input means b4, voice packet generation means b5, and voice packet transmission means b6 constituting the singer terminal b are functions by the virtual karaoke client program 3091. It is realized by. In addition, the accompaniment data acquisition unit c1, the voice packet acquisition unit c2, and the performance output unit c3 constituting the listener terminal c are similarly realized by the functions of the virtual karaoke client program 3091.

1-4. Processing Details Processing details in the present embodiment will be described with reference to FIGS. In this embodiment, the singer terminal of the singer (singer) is realized by sequentially transmitting voice data generated by singing the singer toward the listener terminal of each listener (viewer) via the Internet. A karaoke voice transmitting / receiving system will be described. The singer's singer terminal accesses the virtual karaoke server (management server) and sends a request for a karaoke song desired by the singer. Has been delivered.

1-4-1. Overall Flowchart FIG. 4 is an overall flowchart according to the karaoke voice transmitting / receiving system in the present embodiment. The singer or listener operates the keyboard / mouse 307 of the singer terminal b or listener terminal c to activate the virtual karaoke client program 3091 recorded on the hard disk 309. In response to the activation operation, the CPU 303 of the singer terminal b or the listener end terminal c executes the virtual karaoke client program 3091 using the memory 305 or the like and accesses the management server a via the Internet based on a predetermined URL (step) S401-403, steps S441-443).

  Upon receiving the access, the CPU 203 of the management server a transmits the main screen in the karaoke voice transmission / reception system to the singer terminal b or the listener terminal c (step S421). FIG. 11A shows an example of the main screen 110 displayed on the display 301 of the singer terminal b or the listener terminal c. On the main screen 110, a requested song input column 1101, a reservation status display column 1103, a singing status display column 1105, and the like are displayed.

  When a singer who is going to sing karaoke inputs a song name of a desired requested song in the requested song input field 1101 on the main screen 110 and presses the send button 1102, the CPU 303 of the singer terminal b will enter the requested song. The song name is transmitted to the management server a (step S405). For example, when the singer inputs the requested song “AAA” and presses the send button 1102, the CPU 303 sends “AAA” as a request to the management server a.

  Upon receipt of the transmission, the CPU 203 of the management server b reads karaoke accompaniment data corresponding to the requested song from the accompaniment data database 2093 (step S423). FIG. 12 shows an example of the accompaniment data database 2093. In the accompaniment data database 2093, the requested music piece 121 and the accompaniment data 123 are recorded in association with each other. For example, the CPU 203 reads accompaniment data “0100ABC (binary data)” corresponding to the request “AAA”.

  The management server a distributes the read accompaniment data to the singer terminal b and the listener terminal c (step S425). For example, accompaniment data “0100ABC... (Binary data)” is distributed via the Internet. The singer terminal b and the listener terminal c both receive accompaniment data distributed from the management server a (step S406).

  The management server a transmits the listener list only to the singer terminal after distribution of the accompaniment data (step S427). Here, the listener list is a list of IP addresses related to the listener terminal c accessing the management server a. That is, the listener list allows the singer terminal b to recognize the listener terminal c to which karaoke song voice data should be transmitted.

  When the reception of the accompaniment data and the listener list is completed, the singer terminal b transmits a notification of the start of karaoke performance to each listener terminal c and reproduces the received accompaniment data (steps S409 and S411). Upon receiving the start notification, the listener terminal c starts reproducing the accompaniment data on condition that reception of the accompaniment data has been completed (steps S447 to S449). The reason for the completion of accompaniment data reception is that the network environment may differ between listener terminals or singer terminals. That is, even if reception of accompaniment data has been completed at the singer terminal b, there is no guarantee that reception of the accompaniment data has been completed at each listener terminal c at that time, and karaoke accompaniment cannot be started at the listener terminal c. Because there is.

  When the reproduction of the karaoke song based on the accompaniment data is started at the singer terminal b, the singer performs a karaoke song using the microphone 317. At this time, the CPU 303 of the singer terminal b sequentially generates voice packets based on the singer's singing voice inputted via the microphone 317 and sequentially transmits the voice packets to the listener terminals c. (Step S413). Details of this process will be described below.

1-4-2. Voice Packet Transmission / Reception Processing FIG. 5 shows a flowchart of voice packet transmission / reception processing executed between the singer terminal b and each listener terminal c.

1-4-2-1. Singer terminal (voice packet transmission processing)
FIG. 6 shows the relationship between the respective constituent units (modules) constituting the virtual karaoke client program 3091 executed in the singer terminal b. Further, FIG. 13 schematically shows a generation process of a voice packet generated in this process.

  As shown in FIG. 6, the CPU 303 of the singer terminal b reads the MIDI data 601 that is accompaniment data received from the management server a, and the MIDI data reproduction unit 603 reproduces the accompaniment data. The reproduced accompaniment data is reproduced and output from the speaker 308 via a D / A converter (not shown) controlled by the audio output device control unit 609. The singer starts singing karaoke using the microphone 317 while listening to the karaoke accompaniment that is the accompaniment data reproduced and output. Further, a screen as shown in FIG. 11B is displayed on the display 301 of the singer terminal b. The singer displays “key” 1111, “tempo” 1112, “lyric timing” 1113, and “volume” related to accompaniment data. 1114 and the like can be freely changed during reproduction. Further, a screen as shown in FIG. 11C is displayed on the display 301 of the listener terminal c.

  In the flowchart shown in FIG. 5, the singer terminal b inputs the singer's karaoke singing voice input via the microphone 317 via the audio input device control unit 605 (FIG. 6) (step S501). Note that the singer's karaoke voice input via the audio input device control unit 605 (FIG. 6) is synthesized with the reproduced MIDI data and provided to the audio output device control unit 609 (FIG. 6). The sound is output from the speaker 308 via a converter (not shown).

  The CPU 303 of the singer terminal b digitizes the karaoke voice by an A / D converter (not shown) based on the analog signal input via the microphone 307 in the audio input device control unit 605 (FIG. 6). (Step S502). The digitization of the karaoke voice may be performed by a PCM (Pulse Code Modulation) system in which the sound is digitized (sampled) and recorded at regular intervals. In FIG. 13, an analog signal 131 input as singer's karaoke sound is converted into a code 132 and converted into PCM.

  When CPU 303 determines that karaoke singing is being performed by a singer speaking based on a PCM signal input within a predetermined time (for example, 100 milliseconds) (step S503, YES), audio data compression is performed. The unit 607 (FIG. 6) compresses the digitized karaoke sound using a predetermined codec based on the PCM signal input within the predetermined time (step S505). Note that, for the predetermined time, one voice packet may be generated every about 50 milliseconds to 200 milliseconds.

  For example, whether or not karaoke singing is performed may be determined based on whether or not the numerical value or rate of change of the PCM signal of the input karaoke voice is within the range of the human musical vocal range. In addition, as a codec for performing audio compression, ITU-T G.723.1 and G.729 Annex A, which are low-band codecs, or MP3 (MPEG Audio Layer-3), AAC (Advanced Audio Coding), etc., which are wideband codecs, are used. That's fine. In FIG. 13, the digitized karaoke voice is converted into compressed data 133 using MP3.

  The CPU 303 adds the time stamp 135 of the accompaniment data during reproduction acquired from the audio output device controller 609 to the compressed karaoke voice data 133 compressed in the above in the voice packet generator 611 (step S507). The time stamp is synchronization timing information with karaoke accompaniment reproduced as MIDI data in the singer terminal b. That is, it is information indicating at what timing the karaoke voice is input in the karaoke accompaniment. By referring to the time stamp, the MIDI data of the karaoke accompaniment and the karaoke voice data can be reproduced in synchronization at the listener terminal c.

  FIG. 8 shows an example of the data structure in the MIDI data 601. This data structure is based on the format 0 standard of SMF (Standard MIDI File). The MIDI data 601 includes a header chunk 81 and a track chunk 83. In the header chunk 81, a MIDI data identifier 811 and header information 813 are stored. The track chunk 83 stores a track identifier 831, a track size 832, a delta time 833, a MIDI event 834, a delta time 833, a MIDI event 834,. Here, the delta time 833 is a relative representation of the time difference from the previous MIDI event to the next MIDI event. For example, when the delta time 835 relating to the MIDI event 836 is “240 (Chix)”, the MIDI event 836 is started 240 Chicks after the end of the MIDI event 834. The unit chicks represents the length of a sound obtained by decomposing quarter notes with an accuracy of 480. Therefore, the time stamp indicating the reproduction time of the MIDI data 601 represents the accumulated time from the beginning of the karaoke accompaniment generated based on the delta time and the MIDI event.

  Following the addition of the time stamp, the CPU 303 generates a voice packet by adding other information in the voice packet generation unit 611 (step S509). For example, in addition to the “time stamp” 135, “key information” 136A, “tempo information” 136B, “effect information” 136C, and the like for indicating the accompaniment data playback method during playback are added. Also, a “packet type” 136D for indicating that the packet is a voice packet, a “checksum” 136E for use in error processing of the entire packet data, a “packet size” 136F indicating the size of the packet, and a packet identification Information such as “packet identifier” 136G is added to generate the entire voice packet.

  FIG. 9 shows the data structure of the voice packet generated in the above. The voice packet includes a header portion 91 and a data portion 93. The header portion includes a “packet identifier” 136G, a “packet size” 136F, a “checksum” 136E, a “time stamp” 135, and a “packet type” 136D. "Key information" 136A, "Tempo information" 136B, and "Effect information" 136C are recorded. In the data portion, compressed audio data 133 is recorded.

  In the voice packet transmitting unit 613, the CPU 303 performs a process of adding a UDP header 137A (UDP packetization) and a process of adding an IP header 137B (IP packetization) to the voice packet. The packet is transmitted to the network 0 (step S511). That is, the generated voice packet is transmitted to each listener terminal c specified based on the IP address recorded in the listener list (step S511). In the present embodiment, since the real-time property of the voice packet is emphasized, UDP (User Datagram Protocol) having a high transfer rate is adopted without performing packet retransmission processing.

  The CPU 303 repeats the above steps S501 to 511 until the end of karaoke is detected (step S513). Note that the end of karaoke is a predetermined control signal that is generated when the reproduction of accompaniment data is completed by the MIDI data reproducing unit 603 or when the accompaniment data reproduction is stopped by a singer. . Therefore, the CPU 303 of the singer terminal b can perform voice packet generation processing based on the singer's karaoke voice only during the reproduction of the accompaniment data, and transmit this to each listener terminal c.

1-4-2-2. Listener terminal (voice packet reception processing)
FIG. 7 shows the relationship between the respective constituent units (modules) constituting the virtual karaoke client program 3091 executed in the listener terminal c.

  As shown in FIG. 7, the CPU 303 of the listener terminal c reads the MIDI data 701 that is the accompaniment data received from the management server a, and the MIDI data reproducing unit 703 reproduces the accompaniment data. The reproduced accompaniment data is reproduced and output from the speaker 308 via a D / A converter (not shown) controlled by the audio output device controller 711. Further, as described above, the CPU 303 records the accumulated time from the beginning of the karaoke accompaniment generated based on the delta time and the MIDI event as the reproduction time of the MIDI data 701.

  In the flowchart shown in FIG. 5, the CPU 303 of the listener terminal c acquires the voice packet 70 received via the Internet in the voice packet acquisition unit 705 (step S521). At this time, the voice packet acquisition unit 705 determines whether the packet should be acquired based on the data stored in the packet type 136G of the voice packet. Further, the validity of the acquired packet may be determined based on the packet size 136F and the checksum 136E. The voice packet 70 is the same as the voice packet 70 before packet transmission shown in FIG. That is, the UDP header or IP header indicating the destination or the like is removed by another communication program or the like in the listener terminal c.

  In the audio packet determination unit 707, the CPU 303 compares the time stamp 135 of the audio packet 70 with the reproduction time of the accompaniment data being reproduced (step S523). If the time stamp 135 is greater than the playback time, the CPU 303 extracts the compressed data 133 stored in the audio packet 70 and performs data expansion in the audio packet expansion unit 709 (step S525). For example, if the time stamp 135 is “10000 milliseconds” and the reproduction time is “9990 milliseconds”, the compressed data 133 of the voice packet is expanded. On the other hand, if the time stamp 135 is less than or equal to the reproduction time, the voice packet is discarded (step S530). For example, if the time stamp 135 is “10000 milliseconds” and the reproduction time is “10100 milliseconds”, the voice packet is discarded.

  The CPU 303 that has completed the data expansion acquires the karaoke voice in consideration of the other data stored in the voice packet 70 (step S527). For example, if the packet type indicates “voice”, the expanded data is regarded as karaoke voice and karaoke voice as a PCM signal is acquired.

  In the audio output device controller 711, the CPU 303 reproduces and outputs the PCM data of the karaoke voice acquired in the above in synchronization with the MIDI accompaniment data (step S529). For example, if the time stamp of the karaoke voice is “10000 milliseconds”, the accompaniment data is superimposed and reproduced and output simultaneously from the speaker 308 when the accompaniment data reaches “10000 milliseconds”.

  Further, the CPU 303 changes the accompaniment data playback method based on the key information 136A, the tempo information 136B, and the effect information C stored simultaneously in the audio packet 70. For example, if the key, tempo, and the like have been changed during the playback time of the audio packet 70 (for example, 100 milliseconds), the accompaniment data key and tempo are changed and played back for the same time.

  The CPU 303 repeats the above steps S521 to 529 until it receives the end packet (step S531). The end packet receives a predetermined control signal generated when the singer terminal b completes the reproduction of the accompaniment data or when the singer performs an operation for stopping the reproduction of the accompaniment data. It is a packet that is sent. For example, when the end packet and the voice packet have the same structure, the CPU 303 may determine whether or not the packet is the end packet based on the data stored in the packet type 136D.

1-5. Summary As described above, in the karaoke voice transmission / reception system of the present invention, the performance data of karaoke is separated into accompaniment data distributed in advance and singing data distributed in real time, so the communication volume is reduced. This can reduce the line load on the network. Thereby, irrespective of the network environment to which the terminal on the listener side is connected, the sound quality of karaoke performance by a singer can be improved, and the entertainment of virtual karaoke can be improved.

  In addition, a voice packet is generated every predetermined time based on singer song data, and information (key information, tempo information, effect information, etc.) related to accompaniment data being played on the singer terminal is stored in the voice packet. is doing. For this reason, the key information changed in the singer terminal can be immediately reflected in the listener terminal.

  For example, as shown in FIG. 10, the MIDI data reproduced in the singer terminal b is changed from “key = ± 0” to “key = + 2” in the state of “time = 2”. The voice packet 100 stores “key = + 2” as key information in the state of “time = 2”. The listener terminal c sequentially receives voice packets, and the MIDI data is “key = + 2” based on the key information “key = + 2” stored in the voice packet 101 at “time = 2”. Reproduce.

  Moreover, even if a delivery delay occurs by transmitting song data as a voice packet every predetermined time, the voice is momentarily interrupted by discarding the delayed packet and processing the next packet. Even then, the singing data can be continuously reproduced by the next voice packet. For example, in FIG. 13, even when the voice packet 105 is discarded due to delay, the singing data can be continuously reproduced by processing the voice packet 101.

2. Second Embodiment In the first embodiment, the singer terminal b is configured to sequentially transmit voice packets to the listener terminal c connected to the management server a. However, a voice packet may be transmitted from the singer terminal b to the management server a, and the voice packet may be distributed from the management server a to each listener terminal c. Thereby, the client load of the singer terminal b can be reduced, and the number of listener terminals that can be connected simultaneously can be increased. In the present embodiment, an example in which a voice packet is transmitted from the management server to each listener terminal as described above will be described.

2-1. Functional Block Diagram A functional block diagram of the karaoke voice transmission / reception system according to the present embodiment is shown in FIG. 1a. In FIG. 1a, the configurations of the singer terminal b and the listener terminal c are the same as those in FIG. The management server a includes an audio packet acquisition unit a3 and an audio packet transmission unit a4 in addition to the accompaniment data recording unit a1 and the accompaniment data distribution unit a2. The voice packet acquisition means a3 is for acquiring voice packets transmitted sequentially from the singer terminal. The voice packet transmitting means a4 is for transmitting the acquired voice packet to each listener terminal c.

2-2. Hardware Configuration The hardware configuration of each device of the karaoke voice transmitting / receiving system according to the present embodiment is the same as that shown in FIGS.

2-3. Processing Details Processing details in the present embodiment will be described with reference to FIGS. 4A and 5A. In the flowchart of FIG. 4a, the processes of steps S401 to S405 of the singer terminal b, steps S421, S423 and S425 of the management server a, and steps S441 to S447 of the listener terminal c are the same as those shown in the first embodiment. is there.

  In step S406, the CPU 303 of the singer terminal b notifies the management server a of the start of karaoke performance. In response to this, the management server a transmits a karaoke performance start notification to the listener terminal c related to the list of IP addresses related to the listener terminal c accessing the management server a (step S428). Upon receiving the start notification, the listener terminal c starts reproducing the accompaniment data on condition that reception of the accompaniment data has been completed (steps S447 to S449).

  FIG. 5a shows a flowchart of voice packet transmission / reception processing executed between the singer terminal b, the management server a, and each listener terminal c. In FIG. 5a, the processes in steps S501 to S509 and S513 for the singer terminal b and steps S521 to S531 for the listener terminal c are the same as those shown in the first embodiment.

  In step S511, the CPU 303 of the singer terminal b sequentially transmits the voice packet 70 to the management server a. Upon receiving the transmission, the management server a sequentially transmits voice packets to the listener terminal c recorded in the listener list (steps S551 and S553). Each listener terminal receives the voice packet 70 relayed by the management server a. Thereby, the singer terminal b has only to transmit the voice packet 70 only to the management server a.

  Similarly to the case of transmitting a voice packet, when transmitting an end packet, the CPU 303 of the singer terminal b transmits the end packet only to the management server a (step S515). Upon receipt of the transmission, the management server a transmits an end packet to each listener terminal c (steps S555 and S557). Each listener terminal receives an end packet relayed by the management server a.

3. Other Embodiments (1) In the above embodiment, the karaoke voice transmitting / receiving system is configured by causing the singer terminal b or the listener terminal c to execute the virtual karaoke client program 3091. However, a general browser program and its plug-in You may make it implement | achieve the function equivalent to the said virtual karaoke client program 3091 with a program. The plug-in program may be downloaded from the management server a to the singer terminal b or the listener terminal c.

(2) In the above embodiment, the voice packet or the end packet is transmitted from the singer terminal b to the listener terminal c, but other information packets are transmitted from the listener terminal c to the singer terminal b. You may do it. For example, when a listener responds to a karaoke performance at the listener terminal c while the singer's karaoke performance (accompaniment data + singing data) is being reproduced at the listener terminal c, an information packet regarding the reaction information is transmitted from the listener terminal c. The information may be transmitted to the singer terminal b and the reaction information based on the reaction packet may be reproduced or displayed at the singer terminal b. Thereby, the entertainment property in a virtual karaoke box can be improved.

(3) In the above embodiment, in order to realize each function shown in FIG. 1, this is realized by software using a CPU. However, some or all of them may be realized by hardware such as a logic circuit. In addition, you may make it make an operating system (OS) process a part of program further.

It is a functional block diagram of a karaoke voice transmission / reception system. It is a functional block diagram of a karaoke voice transmission / reception system. It is a figure which shows the hardware constitutions of a management server. It is a figure which shows the hardware constitutions of a singer terminal / listener terminal. It is a whole flowchart concerning a karaoke voice transmission / reception system. It is a whole flowchart concerning a karaoke voice transmission / reception system. It is a flowchart in the transmission / reception process of an audio | voice packet. It is a flowchart in the transmission / reception process of an audio | voice packet. It is a figure which shows the relationship of each structural unit in the singer terminal b. It is a figure which shows the relationship of each structural unit in the listener terminal c. It is a figure which shows the example of the data structure of MIDI data. It is a figure which shows the data structure of an audio | voice packet. It is a figure which shows the timing of the audio | voice data reproduced | regenerated in the singer terminal b and the listener terminal c. It is a figure which shows the example of the screen displayed on the display of a singer terminal / listener terminal. It is a figure which shows an accompaniment data database. It is a figure which shows typically the production | generation process of an audio | voice packet.

Explanation of symbols

a Management server b Singer terminal c Listener terminal

Claims (19)

A karaoke voice transmitting / receiving system comprising a management server, at least one singer terminal and at least one listener terminal,
(a) The management server
(a1) accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs;
(a2) an accompaniment data distribution means for receiving accompaniment data relating to a karaoke song corresponding to the request to the singer terminal and the listener terminal in response to a request from the singer terminal;
(b) Singer terminal
(b1) In response to an operation from the singer, a request means for making a request for a singer desired karaoke song to the management server;
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) singing voice input means for inputting the voice sung by the singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, voice packet generating means for generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6) voice packet transmission means for transmitting the generated voice packet to the listener terminal,
(c) The listener terminal
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2) voice packet acquisition means for acquiring voice packets transmitted from the singer terminal;
(c3) A performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance is provided. Karaoke voice transmission and reception system. A management server for composing a karaoke voice transmitting / receiving system provided to be able to communicate with each of at least one singer terminal and at least one listener terminal,
(a1) accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs;
(a2) an accompaniment data distribution means for receiving accompaniment data relating to a karaoke song corresponding to the request to a singer terminal and a listener terminal in response to a request from the singer terminal; Management server for. A program for realizing, by using a computer, a management server for composing a karaoke voice transmission / reception system provided to be able to communicate with each of at least one singer terminal and at least one listener terminal. A program characterized by being configured;
(a1) Accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs,
(a2) Accompaniment data distribution means that receives a request from a singer terminal and distributes accompaniment data related to the karaoke song corresponding to the request to the singer terminal and the listener terminal. A singer terminal for composing a karaoke voice transmitting / receiving system provided to be able to communicate with each of a management server and at least one listener terminal,
(b1) In response to an operation from the singer, a request means for making a request for a singer desired karaoke song to the management server;
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) singing voice input means for inputting the voice sung by the singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, voice packet generating means for generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6) A singer terminal for a karaoke voice transmission / reception system, comprising: voice packet transmission means for transmitting the generated voice packet to a listener terminal. A program for realizing, using a computer, a singer terminal for composing a karaoke voice transmission / reception system provided to be able to communicate with each of a management server and at least one listener terminal, the computer having the following means A program characterized by
(b1) Request means for receiving a request for a singer desired karaoke song from the singer in response to an operation from the singer,
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the request distributed from the management server;
(b3) Accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) Singing voice input means for inputting the voice sung by the singer along with the reproduced karaoke accompaniment,
(b5) voice packet generation means for generating a voice packet to which synchronization information with the karaoke accompaniment is added based on the input voice;
(b6) Voice packet transmitting means for transmitting the generated voice packet to the listener terminal. A listener terminal configured to be able to communicate with each of the management server and at least one singer terminal to constitute a karaoke voice transmission / reception system,
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2) voice packet acquisition means for acquiring voice packets transmitted from the singer terminal;
(c3) A performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance is provided. Listener terminal for karaoke voice transmission / reception system. A program for realizing, using a computer, a listener terminal for composing a karaoke voice transmission / reception system provided to be able to communicate with each of a management server and at least one singer terminal, and causing the computer to configure the following means: A program characterized by
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server,
(c2) Voice packet acquisition means for acquiring a voice packet transmitted from a singer terminal,
(c3) Performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance. A karaoke voice transmitting / receiving system comprising a management server, at least one singer terminal and at least one listener terminal,
(a) The management server
(a1) accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs;
(a2) Accompaniment data distribution means for receiving accompaniment data relating to a karaoke song corresponding to the request in response to a request from the singer terminal, to the singer terminal and the listener terminal;
(a3) voice packet acquisition means for acquiring voice packets transmitted from the singer terminal;
(a4) voice packet transmission means for transmitting the acquired voice packet to the listener terminal,
(b) Singer terminal
(b1) In response to an operation from the singer, a request means for making a request for a singer desired karaoke song to the management server;
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) singing voice input means for inputting the voice sung by the singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, voice packet generating means for generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6 ′) voice packet transmission means for transmitting the generated voice packet to the management server,
(c) The listener terminal
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2 ′) voice packet acquisition means for acquiring voice packets transmitted from the management server;
(c3) A performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance is provided. Karaoke voice transmission and reception system. A management server for composing a karaoke voice transmitting / receiving system provided to be able to communicate with each of at least one singer terminal and at least one listener terminal,
(a1) accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs;
(a2) Accompaniment data distribution means for receiving accompaniment data relating to a karaoke song corresponding to the request in response to a request from the singer terminal, to the singer terminal and the listener terminal;
(a3) voice packet acquisition means for acquiring voice packets transmitted from the singer terminal;
(a4) voice packet transmission means for transmitting the acquired voice packet to the listener terminal;
A management server for a karaoke voice transmission / reception system. A program for realizing, by using a computer, a management server for composing a karaoke voice transmission / reception system provided to be able to communicate with each of at least one singer terminal and at least one listener terminal. A program characterized by being configured;
(a1) Accompaniment data recording means for recording accompaniment data relating to a plurality of karaoke songs,
(a2) Accompaniment data distribution means for receiving accompaniment data relating to a karaoke song corresponding to the request in response to the request from the singer terminal, to the singer terminal and the listener terminal;
(a3) voice packet acquisition means for acquiring a voice packet transmitted from a singer terminal;
(a4) Voice packet transmitting means for transmitting the acquired voice packet to the listener terminal. A singer terminal for configuring a karaoke voice transmission / reception system with a management server provided to be communicable and at least one listener terminal,
(b1) In response to an operation from the singer, a request means for making a request for a singer desired karaoke song to the management server;
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) singing voice input means for inputting the voice sung by the singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, voice packet generating means for generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6 ′) A singer terminal for a karaoke voice transmission / reception system, comprising: voice packet transmission means for transmitting a generated voice packet to a management server. A program for realizing a management server provided in a communicable manner and at least one listener terminal and a singer terminal for constituting a karaoke voice transmission / reception system using a computer, the computer having the following means: A program characterized by
(b1) Request means for receiving a request for a singer desired karaoke song from the singer in response to an operation from the singer,
(b2) Accompaniment data acquisition means for acquiring accompaniment data related to the karaoke song corresponding to the request distributed from the management server;
(b3) Accompaniment data reproducing means for reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) Singing voice input means for inputting the voice sung by the singer along with the reproduced karaoke accompaniment,
(b5) voice packet generation means for generating a voice packet to which synchronization information with the karaoke accompaniment is added based on the input voice;
(b6 ′) Voice packet transmitting means for transmitting the generated voice packet to the management server. A management server provided to be communicable and at least one singer terminal and a listener terminal for configuring a karaoke voice transmission / reception system,
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2 ′) voice packet acquisition means for acquiring voice packets transmitted from the management server;
(c3) A performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance is provided. Listener terminal for karaoke voice transmission / reception system. A program for realizing a management server provided in a communicable manner and a listener terminal for constituting a karaoke voice transmission / reception system with at least one singer terminal, and causing the computer to constitute the following means: A program characterized by
(c1) Accompaniment data acquisition means for acquiring accompaniment data related to karaoke songs distributed from the management server,
(c2 ') voice packet acquisition means for acquiring voice packets transmitted from the management server,
(c3) Performance output means for synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet and combining them to output as a karaoke performance. In the karaoke voice transmission / reception system, singer terminal or the program according to any one of claims 1 to 14,
The voice packet generating means of the singer terminal generates a voice packet only when a voice sung by a singer is detected. In the karaoke voice transmitting / receiving system, singer terminal or the program according to any one of claims 1 to 15,
The voice packet generating means of the singer terminal adds at least one of key information, tempo information, lyrics timing information, and volume information related to karaoke accompaniment set by the singer to the voice packet. In the karaoke voice transmitting / receiving system, listener terminal or the program according to any one of claims 1 to 16,
The voice packet acquisition means of the listener terminal compares the performance time output in the performance output means with the synchronization information added to the voice packet, determines whether the acquired voice packet is in time for output, and outputs If the voice packet is not received in time, the voice packet is discarded. A virtual karaoke method comprising a management server, at least one singer terminal and at least one listener terminal,
(a) The management server
(a1) an accompaniment data recording step for recording accompaniment data relating to a plurality of karaoke songs;
(a2) an accompaniment data distribution step of receiving accompaniment data related to the karaoke song corresponding to the request in response to the request from the singer terminal, to the singer terminal and the listener terminal,
(b) Singer terminal
(b1) A request step for receiving a request for a singer desired karaoke song from the singer in response to an operation from the singer;
(b2) Accompaniment data acquisition step for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) an accompaniment data reproduction step of reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) a singing voice input step for inputting a voice sung by a singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, a voice packet generation step of generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6) a voice packet transmission step of transmitting the generated voice packet to the listener terminal,
(c) The listener terminal
(c1) Accompaniment data acquisition step for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2) a voice packet acquisition step for acquiring a voice packet transmitted from the singer terminal;
(c3) a performance output step of synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet, and synthesizing them to output as a karaoke performance. A virtual karaoke method. A virtual karaoke method comprising a management server, at least one singer terminal and at least one listener terminal,
(a) The management server
(a1) an accompaniment data recording step for recording accompaniment data relating to a plurality of karaoke songs;
(a2) Accompaniment data distribution step of receiving accompaniment data related to the karaoke song corresponding to the request in response to the request from the singer terminal, to the singer terminal and the listener terminal;
(a3) a voice packet acquisition step for acquiring a voice packet transmitted from the singer terminal;
(a4) a voice packet transmission step of transmitting the acquired voice packet to the listener terminal,
(b) Singer terminal
(b1) A request step for receiving a request for a singer desired karaoke song from the singer in response to an operation from the singer;
(b2) Accompaniment data acquisition step for acquiring accompaniment data related to the karaoke song corresponding to the above request distributed from the management server;
(b3) an accompaniment data reproduction step of reproducing and outputting the acquired accompaniment data as a karaoke accompaniment;
(b4) a singing voice input step for inputting a voice sung by a singer in accordance with the reproduced karaoke accompaniment;
(b5) based on the input voice, a voice packet generation step of generating a voice packet to which synchronization information with the karaoke accompaniment is added;
(b6 ′) a voice packet transmission step of transmitting the generated voice packet to the management server,
(c) The listener terminal
(c1) Accompaniment data acquisition step for acquiring accompaniment data related to karaoke songs distributed from the management server;
(c2 ′) a voice packet acquisition step for acquiring a voice packet transmitted from the management server;
(c3) a performance output step of synchronizing the accompaniment data and the voice packet based on the synchronization information added to the voice packet, and synthesizing them to output as a karaoke performance. A virtual karaoke method.

Images