JP2005284041A - Method for distributing contents, contents distribution server and contents receiver - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a contents distributing method capable of transmitting contents by dynamically switching tracks and smoothly reproducing the switched position and a contents distribution server and a contents receiver in which the contents distributing method is used. <P>SOLUTION: When a transmission control part 32 transmits the file name of contents data, a track number, a contents reproducing range, and a contents reproduction request message and a transmission control part 22 receives the reproduction request message, a contents acquisition part 21 acquires the contents element of a specified track in accordance with the file name, the track number and the contents reproducing range. When the transmission control part 32 transmits the track number and a track switching request message and the transmission control part 22 receives the switching request message, the contents acquisition part 21 acquires the contents element of a specified track having time information continued to time information included in the sent contents element in accordance with the track number. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention dynamically switches and transmits tracks in a content distribution system including a content distribution server that transmits content data composed of a plurality of tracks and a content reception device that receives and reproduces the content data via a network. The present invention relates to a content distribution method capable of smoothly reproducing a switched portion, a content distribution server using the content distribution method, and a content receiving device.

  Currently, a karaoke system called a communication karaoke system is generally used. Most of this communication karaoke system employs a download system that receives music data including tone and musical score information similar to MIDI (Musical Instrument Digital Interface) before playback. MIDI is about 1/10 to 1/100 the amount of data compared to digital audio data based on sampling such as PCM (Pulse Code Modulation), and uses a narrowband communication path such as a public telephone line. Since it is suitable for performing a data distribution service, it is often employed in a communication karaoke system. In the content receiving device, the received music data (MIDI data) is played in advance by a MIDI sound source device having timbres corresponding to various musical instruments, thereby providing a service to the user.

  On the other hand, a karaoke system requires a key control function. The key control function is a function that makes it easy to sing comfortably by adjusting the pitch of the music according to the voice quality of the user. In the case of an existing communication karaoke system using MIDI, key control can be realized very easily by transposing the musical score information contained in MIDI.

  On the other hand, in the communication karaoke system using MIDI, various problems have arisen with the recent demand for higher quality. For example, when it is desired to improve the sound quality of a music piece, since the only part that can be operated by MIDI is musical score information, the sound source of the content receiving apparatus must be exchanged or modified. This is a huge cost in a communication karaoke system where users (or stores) reach hundreds of thousands of households.

  In recent years, demand for partner voices in back chorus or duet music (for example, partner voices sung by women if the user is male) is increasing in online karaoke systems. However, as described above, since MIDI mainly includes musical score information, it is necessary to transmit back chorus and partner voice data by another means in accordance with the music.

  Next, a communication karaoke system using a streaming system will be described.

  2. Description of the Related Art In recent years, streaming systems that play while receiving data without accumulating all the music data of the playback content on the receiving side are actively used on the Internet. A general streaming system often handles playback content digitized by a high-efficiency encoding method such as MPEG (Moving Picture Experts Group).

  For example, in the case of audio, PCM digital data can be compressed to about 1/3 to 1/15 by performing high-efficiency encoding processing such as MPEG Audio Layer 2, 3, AAC (Advanced Audio Coding). Real-time transmission using broadband communication channels such as ADSL (Asymmetric Digital Subscriber Line) has become possible.

  In this streaming system, since it is not necessary to store all the playback content in the content receiving device, it is possible to quickly provide the content receiving device with highly efficient encoded music data that is very close to the original sound but larger in volume than MIDI. With this feature, music data with back chorus and partner voice is provided to the user with quality close to the original sound, which can contribute to higher quality of service.

  However, in a communication karaoke system employing a streaming system, the amount of computation becomes a problem when a key control function is added. In the case of MIDI, since the original is musical score information, it is easy to transpose as if rewriting the score. However, in the case of music data such as PCM and MPEG Audio, it is necessary to obtain the same effect as transposition using the characteristics of audio.

  For example, Japanese Patent Laid-Open No. 9-185392 (Patent Document 1) discloses a pitch conversion device based on frequency conversion, which corresponds to a pitch shift in the frequency domain, and is compared with MIDI transposition. Enormous operations. Therefore, in the communication karaoke system that employs the streaming system, the content receiving device must be equipped with a pitch conversion means that can perform enormous calculations.

  Here, an asynchronous model of a streaming system used for real-time distribution of video / audio will be described.

  There is no synchronization clock between two end-to-end communication devices in a data link based on the exchange of transmission frames of small data size represented by Ethernet (registered trademark) (IEEE802.3). In most cases, IP (Internet Protocol, RFC791, etc.) is used to enable packet exchange between multiple networks with the data link as the lower layer, and IP packets are used as sessions (communication path) with IP as the lower layer. There is no synchronous clock in network protocols such as TCP (Transmission Control Protocol, RFC793) and UDP (User Datagram Protocol, RFC768) for management.

  Therefore, it is not guaranteed when and when a transmission frame transmitted from a certain communication device arrives at the target communication device. Also, the time delay that occurs from transmission to reception of the transmission frame is not constant, and time fluctuation (jitter) occurs. Also, transmission frames may be lost on the transmission path, and even if a plurality of transmission frames are transmitted along a certain time axis, the arrival order is not guaranteed.

  By the way, a streaming system is a generic name of systems that receive and reproduce content via a network and that can reproduce while receiving without accumulating all of the content on the receiving side. Since video, audio, etc. are contents with real-time properties, playback synchronization control such as playback at a predetermined number of frames per second (eg 29.97 frames / second) is essential, and the presence of a synchronization clock is suitable. It can be said that there is. For example, in TV broadcasting, there is a synchronization clock called a frame synchronization signal in the NTSC_TV signal, and the reception side can reproduce it with very high accuracy using the synchronization clock on the transmission side.

  However, in a streaming system via an IP network based on IP, there is no synchronization clock as described above, and the transmission side and the reception side cannot be synchronized. Inevitably, the playback mechanism must be realized by an asynchronous model. The asynchronous model is a configuration and programming technique that operates based on the system clock of each device itself.

  FIG. 9 shows a functional block diagram of the content distribution server 102a and the content receiving device 103a in the content distribution system 101a (asynchronous model). The content distribution server 102a and the content receiving device 103a are connected to each other by the IP network 104.

  The content distribution server 102a includes a content acquisition unit 1021, a transmission unit 1022, a system clock oscillation unit 1023, and a content storage unit 1024.

  The content acquisition unit 1021 has a function of recording content data of content to be reproduced in the content storage unit 1024 and acquiring a content fragment using an arbitrary file pointer (reading position on the content). At this time, the file pointer is any one of the content file name, the track number, and the frame number, or some combination thereof. A “content segment” refers to actual data of content having an arbitrary data length.

  The content acquisition unit 1021 reads the content segment according to the system clock signal. The content acquisition unit 1021 has initial values such as a content file name, a track number, and a frame number inside, and may automatically start reading from the time of startup, or the content file name, track number, and content from an external input Reading may be started after the reproduction start time is obtained. In addition, an external input for receiving command input is provided, and when command input = reproduction start constant (for example, “PLAY”), reproduction may be performed, or command input = reproduction stop constant (for example, “STOP”). If so, it may be stopped. Alternatively, the automatic read start operation and the external read start operation may be automatically determined.

  The system clock oscillator 1023 is a real-time clock (high-precision clock) realized by a crystal oscillator or the like, and has a function of supplying a system clock signal to the content acquisition unit.

  The transmission unit 1022 has a function of securing a virtual communication path called a session, packetizing the content fragment with an appropriate header, and transmitting the packet to the content reception device 103a via the IP network 104. Have. For example, the header includes a network time stamp.

  When transmitting a packet, the transmission unit 1022 uses a transmission address to deliver the packet to the content reception device 103a. The transmission address is composed of a combination of a transmission address and a transmission port number. The transmission address may be previously stored as a constant in the transmission unit 1022 or may be input from another block. Note that a session (communication path) for transmitting a packet is dynamically established when a transmission address is determined. Further, the content segment and the packet do not necessarily match.

  In addition, the content receiving apparatus 103a includes a receiving unit 1031, a buffer unit 1032, a system clock oscillation unit 1033, a content reading unit 1034, and a reproduction unit 1035.

  The receiving unit 1031 has a function of receiving a packet from the IP network 104, interpreting a header, extracting a content fragment, and supplying the content unit 1032 to the buffer unit 1032. The receiving unit 1031 uses a reception address in order to receive only a packet necessary for itself among the packets from the content distribution server 102a. The reception address is composed of a combination of a reception address and a reception port number. The reception address may be previously stored as a constant in the reception unit 1031 or may be input from another block.

  The buffer unit 1032 has a function of temporarily accumulating and storing content pieces together with a network time stamp (described later) obtained from the header.

  The content reading unit 1034 monitors the buffer unit 1032 and starts reading the content unit according to the system clock signal from the system clock oscillation unit 1033 from the time when it is determined that the content unit sufficient for reproduction has been accumulated. It has a function of restoring and outputting a set of segments as content.

  The playback unit 1035 has a function of performing decoding according to the input content segment and outputting an audio signal and a video signal to a predetermined output device such as a speaker. For example, when the content segment is high-efficiency encoded digital audio data similar to MPEG, the playback unit 1035 includes a high-efficiency code decoder (decoder), a D / A converter, an analog amplifier, and the like. Is done.

  The first feature of the content distribution system a (asynchronous model) shown in FIG. 9 is that the content distribution server 102a and the content reception device 103a individually have system clock oscillation units 1023 and 1033, respectively. Although the accuracy of both system clock frequencies needs to be considerably high, the IP network 104 serving as a transmission path does not include a synchronous clock transmission mechanism.

  The second feature is that the content receiving device 103a includes the buffer unit 1032 and the content reading unit 1034 is configured so that the buffer unit 1032 is not emptied. Even in the case of reproducing content having a delay, normal reproduction is continued while absorbing time delay and time fluctuation (jitter) of packets caused by the IP network 104.

  The above is the outline of the asynchronous model of the streaming system used for real-time distribution of video / audio.

  Next, playback time management in the streaming system will be described.

  First, a method for assigning content playback time will be described.

  In the above asynchronous model, in order for content to be played back as a continuous medium at the correct playback time, it is necessary to transmit the content playback time in association with the content segment. For example, in FIG. 9, in order for the content reading unit 1034 to select a content segment corresponding to the 0 second 0 frame video and output it to the playback unit 1035, the content segment is the 0 second 0 frame. Content playback time indicating data is required. Otherwise, if a time delay or the like occurs on the network, the playback time will be shifted as it is.

  In streaming systems, RTP (Real-time Transfer Protocol, RFC1889) is often used to give content playback time to content pieces (Audio-Video Transport Working Group, "RTP: A Transport Protocol for Real- Time Applications ", RFC 1889, Internet Engineering Taskforce, Jan 1996). In RTP, the content fragment is the payload (cargo), and the main part is the RTP header of at least 96 bits given before the payload. Stuff).

  The RTP time stamp is mapped in real time by the RTP time stamp frequency determined in advance by the content type. The RTP timestamp frequency is also called a network timestamp frequency. In general, in the case of content digitized by a high-efficiency encoding method defined by MPEG, 90000 (90 KHz) is often used as the RTP timestamp frequency. This means that when reading 1 second of content, the RTP timestamp is incremented by +90000. That is, the following formula is established for the network time stamp, the network time stamp frequency, and the content playback time similar to the RTP time stamp.

[Equation 1]
Content playback time = network time stamp / network time stamp frequency For example, in the case of MPEG-2 video at 29.97fps, if the RTP time stamp of the first frame is "0", the second frame is "3003", The 30th frame is “90090”.

  In the case of MPEG-4 Audio AAC, one frame is fixed at 1024 samples, and the RTP timestamp frequency uses the PCM sampling frequency of the sound source. For example, if the PCM sampling frequency is 44100Hz (44.1KHz PCM = CD Audio), the RTP timestamp frequency is 44100, the RTP timestamp of the first frame is “0”, the second frame is “1024”, approximately 1 second The 43rd frame corresponding to is “44032”.

  In this example, RTP has been described as an example, but there are other packet formats that can be used to calculate the content playback time on the receiving side from the network time stamp provided for the playback time and the corresponding network time stamp frequency. Yes, very common. For example, the MPEG-2 Transport Stream standardized by MPEG standardizes a time stamp called a “Presentation Time Stamp” having exactly the same function. The above is the outline of the content reproduction time giving method.

  Next, a data structure of a file (hereinafter referred to as a content file) in which content managed by the content distribution server 102a and transmitted to the content receiving device is recorded will be described.

  FIG. 10 shows an example of the data structure of content data managed by the content acquisition unit 1021. As shown in FIG. 10, the content data is composed of the content file name, the total number of tracks T, and the track data of each track. The track data of each track further includes a track number, a track timestamp frequency, the total number of frames, And a plurality of data frames composed of frame numbers, time information, and content pieces.

  The content file name is a name that uniquely identifies a file that stores content data on the content storage unit. The total number of tracks T records the number of tracks stored in one content data. A track refers to each of a plurality of contents stored in one file, and an arbitrary track (content) can be selected by a track number t (total number of tracks T ≧ t ≧ 1). The time information as a pointer can be taken out from an arbitrary position. The track time stamp frequency recorded in each track indicates an increment per unit time of time information.

  In addition, the total number of frames, which is the total number of data frames, is recorded in each track, but the total number of frames does not have to match in each track. Further, frame numbers are assigned to each data frame in chronological order, and time information based on the track time stamp frequency is assigned.

  The data structure of content data in which a plurality of contents are multiplexed as described above is very common as a file format for storing audio / moving images. Depending on the file format, the track time stamp frequency may not be included in the data as a system constant, or frames with the same time information may be indexed to save the time information storage area. However, the frame number and the time information are held in a mutually convertible state in common.

  Examples of such file formats include Microsoft AVI, Apple Computer QuickTime Format, and ISO / IEC 14496-1 MP4 File. In addition, in Apple Computer QuickTime Format and ISO / IEC 14496-1 MP4 File, by using Reference Atom (Data Reference Atom), all or part of the contents of multiple tracks can be made independent as external content data. However, since the frame number and time information are recorded in the main content data in a mutually convertible state, it may be regarded as having the same data structure as the content data. The above is the outline of the data structure of the content data in the content distribution server 102a.

  Next, the reproduction time management process (transmission side) of the content acquisition unit 1021 when reading content data will be described with reference to the flowchart of FIG.

  First, when a content reproduction request is received from the content receiving apparatus 103a, the content acquisition unit 1021 of the content distribution server 102a initializes the following six variables (step S101).

1: Content file name CF = Default content file name (eg "hoge.mp4")
2: Track number TN = default track number (eg "1")
3: Start frame number FN = default frame number (for example, “1”)
4: Playback start time STS = current time NTS obtained from the system clock oscillator 1023
5: Content playback time CTS = “0”
6: Playback state flag F = true value “N”
Next, the content acquisition unit 1021 refers to a predetermined automatic mode constant indicating whether or not to automatically start reproduction, and determines whether to perform automatic reproduction (step S102). When the automatic mode constant is the true value “Y”, the process proceeds to step S110, and when the automatic mode constant is the true value “N”, the process proceeds to step S103.

  In the process of step S102, when the automatic mode constant is a true / false value “N”, the content acquisition unit 1021 performs external input (command input CMD, content file name CF1, track number TN1, content reproduction start time CTS1. (Step S103), if there is an external input, the truth value is “Y”, and the process proceeds to step S104. If there is no external input, the truth value “N” "And the process proceeds to step S106.

  In the process of step S103, the content acquisition unit 1021 determines whether command input CMD = reproduction stop constant among external inputs (step S104). If command input CMD = reproduction stop constant, true / false value “Y” is determined. ", The entire operation is stopped and terminated. Otherwise, the truth value is" N ", and the process proceeds to step S105.

  In the process of step S104, when the truth value is “N”, the content acquisition unit 1021 uses the three variables of the external input (content file name CF1, track number TN1, content reproduction start time CTS1) to set 5. Two variables are initialized (step S105).

1: Content file name CF = Content file name CF1
2: Track number TN = Track number TN1
3: Playback start time STS = current time NTS obtained from the system clock oscillator 1023
4: Content playback time CTS = Content playback start time CTS1
5: Playback state flag F = true value “Y”
Further, the content acquisition unit 1021 initializes the start frame number FN using the content reproduction time CTS, and calculates transmission start time information StartTS as a previous step. The calculation formula of the transmission start time information StartTS is shown below.

[Equation 2]
StartTS = CTS × TRS
Here, the track time stamp frequency TRS is the track time stamp frequency of the track indicated by the track number TN in the content file name CF.

  Next, the content acquisition unit 1021 searches the frame number of the track indicated by the track number TN in the content file name CF in order from 1 to obtain the frame number NFN of the point where time information ≧ StartTS is satisfied, and the frame number NFN. Is set to the start frame number FN.

  If the automatic mode constant is a true / false value “N” in the process of step S103, the content acquisition unit 1021 determines whether the value of the playback state flag is a true / false value “Y” ( In step S106), if the value is a true / false value “Y”, the process proceeds to step S107. If the value is a true / false value “N”, the process proceeds to step S103.

  In the case where the value of the reproduction state flag is the true / false value “Y” in the process of step S106, or in the case of the process continued from step S105, the content acquisition unit 1021 obtains the current time obtained from the system clock oscillation unit 1023. The elapsed time is calculated from NTS, playback start time STS, and system clock frequency SS, and the transmittable range is searched by calculating sendable time information LastTS from the elapsed time, content playback time CTS, and track timestamp frequency TRS. (Step S107). The formula for calculating the sendable time information LastTS is shown below.

[Equation 3]
LastTS = (NTS−STS) / SS × TRS
Here, the system clock frequency SS is a constant for converting (current time NTS−reproduction start time STS) into seconds. For example, if the system clock has an accuracy of 1/1000000 second, it is 1000000.

  Next, the content acquisition unit 1021 sequentially searches the frame number from the start frame number FN in the track indicated by the track number TN in the content file name CF, and acquires the frame number NFN at which time information ≧ LastTS is satisfied. To do. When the frame number NFN that satisfies the condition is not found, the content acquisition unit 1021 sets the termination constant “−1” to the frame number NFN.

  Next, the content acquisition unit 1021 determines whether or not transmission is possible by comparing the start frame number FN and the frame number NFN (step S108). If FN <NFN is satisfied, the truth value “Y” is obtained. The process proceeds to S109. If not established, the truth value is “N”, and the process proceeds to Step S103.

  If the value is true or false in the process of step S108, the content acquisition unit 1021 determines that the track number TN in the track indicated by the track number TN in the content file name CF is less than the frame number NFN from the start frame number FN. For each content segment corresponding to each frame number, and for each content segment, the network time stamp PTS generated according to the following equation from the corresponding time information is output to the transmission unit 1022 (step S109). The transmission unit 1022 generates an appropriate header using the network time stamp PTS and adds it to the content fragment.

[Equation 4]
After outputting PTS = time information / TRS × network time stamp frequency, the content acquisition unit 1021 sets start frame number FN = frame number NFN to shift the start frame number FN to the transmitted position.

  If the automatic mode constant is the true / false value “Y” in the process of step S102, the content acquisition unit 1021 sets the reproduction state flag F = the true / false value “Y” and the reproduction start time STS. = The current time NTS obtained from the system clock oscillator 1023 is set, and the setting for starting automatic reproduction is performed (step S110). The above is the outline of the reproduction time management process (transmission side) of the content acquisition unit 1021.

  Next, the reproduction time management processing (reception side) of the content reading unit 1034 using the network time stamp will be described with reference to the flowchart of FIG.

  When the content reading unit 1034 starts reproduction of the content, the content reading unit 1034 monitors the accumulation amount until a certain amount of content pieces and the received network time stamp PTS associated therewith are accumulated in the buffer unit 1032 (step S121). When the threshold value of the predetermined accumulation amount is exceeded, the truth value becomes “Y”, and the process proceeds to step S122. The threshold value of the accumulation amount may be simply half of the total buffer amount (Half Full), or when the bit rate of the content is known in advance, the buffer unit 1032 causes a buffer underflow. You may decide not to.

  Next, the content reading unit 1034 refers to the system clock oscillating unit 1033, sets the reproduction start time STS = the current time NTS obtained from the system clock oscillating unit 1023, and performs settings for starting reading (step S122). . The content reading unit 1034 sets a timeout variable TOUT = timeout constant (for example, “10”).

  Next, the content reading unit 1034 updates the current time NTS again with reference to the system clock oscillation unit 1033, and calculates a network time stamp PTS to be read according to the following equation (step S123).

[Equation 5]
PTS = (NTS−STS) / SS × network time stamp frequency Next, the content reading unit 1034 searches the buffer unit 1032 based on the network time stamp PTS, and the network time stamp PTS is equal to or smaller than the network time stamp PTS. The content segment associated with is taken out (step S124). If no content fragment is found, the truth value “N” is obtained, and the content reading unit 1034 proceeds to the processing of step S126. If the content fragment is found, the truth value “Y” is obtained, and a time-out variable is set. A timeout constant (for example, “10”) is set in TOUT, and the process proceeds to step S125.

  In the case where the truth value is “N” in step S124, the content reading unit 1034 waits for a certain time (for example, about 0.5 seconds), and then subtracts the time-out variable TOUT by “1” to determine whether a time-out has occurred. Determination is made (step S125). When the subtracted TOUT is less than 0, the true / false value “Y” is obtained, and the content reading unit 1034 ends the process. When the subtracted TOUT becomes the true / false value “N”, the process proceeds to step S123. As a result, if the packet does not arrive for 5 seconds (0.5 × 10), the end operation is performed.

  If the value is true / false “Y” in step S124, the content reading unit 1034 outputs the content segment extracted in step S124 to the reproduction unit 1035 (step S126). At this time, the content pieces may be rearranged by an appropriate means so as to meet the request of the playback unit 1035, or the content pieces may be merged. For example, content pieces having the same network time stamp (RTP time stamp) may be combined after rearrangement using the sequence number in the RTP header.

  For the convenience of explanation, it was assumed that the network time stamp given to the received packet always starts from 0. However, the network time at the start of transmission is preliminarily determined by other communication procedures such as RTSP described later. When the stamp initial value OFS is known, the equation 105 can be transformed into the equation 106.

[Equation 6]
PTS = (NTS-STS) / SS x network time stamp frequency + OFS
The above is the outline of the reproduction time management process (reception side) of the content reading unit 1034 using the network time stamp.

  Next, a transmission control mechanism in the streaming system will be described.

  In a streaming system, it is necessary to select / play / stop operations such as arbitrary content from user input processing, and to dynamically connect / disconnect a session such as TCP / UDP according to the operation. This is called a transmission control mechanism. FIG. 13 shows a functional block diagram of a content distribution server 102b and a content receiving device 103b that constitute a content distribution system 101b having a transmission control mechanism. The content distribution server 102b and the content receiving device 103b are connected to each other by the IP network 104.

  The content distribution server 102b includes a content acquisition unit 1021, a transmission unit 1022, a system clock oscillation unit 1023, a content storage unit 1024, and a transmission control unit 1025. The content reception device 103b includes a reception unit 1031 and a buffer unit 1032. A system clock oscillation unit 1033, a content reading unit 1034, a reproduction unit 1035, a transmission control unit 1036, and an input unit 1037. In addition, the same number is attached | subjected about the same thing as FIG. 9, and the description is abbreviate | omitted.

  When the transmission control unit 1025 of the content distribution server 102b receives transmission control information including instructions for selecting content, playing / stopping the content, etc. from the content receiving device 103b, the transmission control unit 1025 sends the transmission control information to the content acquisition unit 1021. A function of notifying the transmission unit 1022; Also, the transmission control unit 1025 refers to the content storage unit 1024, acquires the content file name, track number, and content playback start time, inputs them to the content acquisition unit 1021, and transmits them to the transmission unit 1022 It has a function to input an address.

  The protocol to be applied is assumed to be a real-time data transmission control protocol typified by RTSP (Real Time Streaming Protocol) defined in RFC2326, and uses methods such as Setup, Play, Pause, Teardown, Describe, etc. Yes (H. Schulzrinne etal, "Real Time Streaming Protocol", RFC 2326, Internet Engineering Taskforce, Apr 1998).

  Further, the transmission control unit 1036 of the content receiving device 103b transmits the transmission control information from the input unit 1037 to the transmission control unit 1025 of the content distribution server 102b, receives the response, analyzes the response, and extracts the received address. And has a function of inputting to the receiving unit 1031.

  Further, the input unit 1037 of the content receiving apparatus 103b has a function of accepting transmission control information input processing such as content selection, content playback, and stop from the user. For example, using a bitmap display and a keyboard, GUI parts such as a play button, a stop button, and a content selection dialog are provided to the user.

  Next, operations of transmission control section 1025 and transmission control section 1036 at the time of transmission / reception of transmission control information will be described using the sequence diagram of FIG. FIG. 14 shows processing performed until one content is selected using RTSP, a session for transmission is established, playback is started, and stopped. Here, the host name of the content distribution server 102b is “server.jvc-victor.jp”, the content file name storing the content is “hoge.mp4”, and “hoge.mp4” contains 1 Assume that there are two MPEG-4 AAC audio tracks, the track number is 1, and the playback time is 235 seconds.

  RTSP applies content URIs (Uniform Resource Identifiers) as resource identifiers for uniquely identifying content on the network (T. Berners-Lee, "Uniform Resource Identifiers (URI): Generic Syntax", RFC2396, Internet Engineering Taskforce, Aug 1998). The content “hoge.mp4” is represented by the following content URI.

“Rtsp: //server.jvc-victor.jp/hoge.mp4”
First, the transmission control unit 1036 requests a session description using the content URI (step S131). The session description is text data indicating what kind of session (communication path) can be established for the content data associated with the content URI. As a session description format, Session Description Protocol (SDP) is applied (M. Handley, “SDP: Session Description Protocol”, RFC2327, Internet Engineering Taskforce, April 1998). Use the DESCRIBE method to request a session description in RTSP. Each method and its response message (including other accompanying information) are inserted and received in the header of the request message and the header of the response message, respectively.

“DESCRIBE rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
The transmission control unit 1025 transmits a response including the session description (step S132). The session description includes a playback time and a media description. This playback time is the maximum playback time of continuous media associated with the specified content URI. There are many types of playback time formats, but the simplest format is NPT (Normal Play Time, ISO8601) in which the start time and end time are expressed in seconds using floating point. For example, the reproduction time of a content URI indicating a content file including an MPEG-4 AAC audio track for 235 seconds is expressed as follows using NPT.

“A = range: npt = 0.0-235.0”
Also, the media description includes information such as content type and network time stamp frequency as prior information for establishing a session.

“M = audio 0 RTP / AVP / UDP 96”
“A = rtpmap: 96 mpeg4-generic / 48000/2”
“A = control: rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1”
“A = fmtp: 96 streamtype = 5; profile-level-id = 15; mode = AAC-hbr; config = 1190; SizeLength = 13; IndexLength = 3; IndexDeltaLength = 3; Profile = 1;”
The above media description expresses that stereo sound of 48000 Hz (48 KHz) PCM sample frequency encoded by the MPEG-4 AAC Hi-bitrate encoding method is included. Also, it is shown that when establishing a session, UDP must be used as a lower layer network protocol and RTP should be applied for transmission.

  Also, it is shown that the following control URI is used for the preparation request of this session. The control URI is given a track number (trackID = 1) to identify a plurality of tracks in one content data.

“Rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1”
After analyzing the session description, the transmission control unit 1036 determines to establish a session for transmitting MPEG-4 AAC stereo audio, determines a reception address of the session, and transmits a session establishment preparation request to the transmission control unit 1025. (Step S133). The SETUP method is used for a session establishment preparation request in RTSP. In the following example, the reception address is the reception address (136.198.190.100) and the reception port number (6668-6669) that the transmission control unit 1036 has.

“SETUP rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1 RTSP / 1.0”
“Transport: RTP / AVP / UDP; unicast; destination = 136.198.190.100; client_port = 6668-
6669 ”
Upon receiving the session establishment preparation request correctly, the transmission control unit 1025 newly determines a transmission address, inputs the transmission address to the transmission unit 1022, and transmits session information to the transmission control unit 1036 (step S134). The session information includes a transmission address (136.198.190.1 in the example) and a transmission port number (19000 to 19001 in the following example) used for distribution.

“Transport: RTP / AVP / UDP; unicast; source = 136.198.190.1; server_port = 19000-19001”
In addition, the transmission control unit 1036 inputs the previously determined reception address to the reception unit 1031 when the session information is received. A new session is established when the processing of both the transmission unit 1022 and the reception unit 1031 is completed.

  Next, the transmission control unit 1036 transmits a reproduction start request (step S135). In the playback start request, the playback range can be specified using NPT. In the example below, the content is specified from the beginning (0.0 seconds) to the end (235.0 seconds). The PLAY method is used for a playback start request in RTSP.

“PLAY rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
“Range: npt = 0.0-235.0”
The transmission control unit 1025 transmits a response to the reproduction start request (step S136). The transmission control unit 1025 that has received the reproduction start request, for all the sessions that have already been prepared, from the control URI of each session and the reproduction range included in the reproduction start request, the content file name / track number / content reproduction start The time is calculated and input to the content acquisition unit 1021 together with command input = reproduction start constant (for example, “PLAY”), and transmission of the RTP packet is started.

  This response also includes RTP packet information necessary for the operation of the content reading unit 1034 of the content receiving device 103b. For example, in this session, the RTP time stamp (network time stamp) of the first RTP packet sent is included (0000000 in the following example).

“RTP-Info: url = rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1; rtptime = 0000000;”
The transmission control unit 1036 transmits a stop request when the user makes a stop input (step S137). In RTSP, the stop request is the TEARDOWN method.

“TEARDOWN rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
The transmission control unit 1025 that has received the stop request controls the transmission unit 1022 to stop the transmission of the RTP packet, disconnect the session, and notify that the transmission is stopped (step S138). In addition, the transmission control unit 1025 inputs command input = reproduction stop constant (for example, “STOP”) to the content acquisition unit 1021 and stops reading.

  For simplification of explanation, it is assumed that only one content is included in one content data. However, the transmission control mechanism can be easily extended to synchronous reproduction of a plurality of contents. For example, it is possible to easily perform synchronized playback of a plurality of contents simply by listing media descriptions regarding a plurality of contents in step S132 and repeating the preparation requests in steps S133 to S134 for the media descriptions. The above is the outline of the transmission control mechanism in the streaming system.

  Here, when trying to switch between audio and video dynamically and smoothly during content playback, a streaming system that employs an asynchronous model and has a transmission control mechanism with dynamic session generation, It is not possible to perform a bitstream switching operation smoothly during playback of two or more contents.

  For example, out of two contents (track numbers 1 and 2) having a playback time of 235 seconds, the content of track number 1 is being played back, and the user moves to the content of track number 2 in the 100th second. It is assumed that a switching instruction is input to the input unit 1037.

  Since the content receiving apparatus 103b employs an asynchronous model, the 100th second seen from the user is a time difference (current time NTS−reproduction start time STS) calculated inside the content reading unit 1034. In the buffer unit 1032, the content pieces after the 100th second already transmitted from the content distribution server 102 b have already been accumulated, or a part of them has stayed in the router or switching hub on the IP network 104. it is conceivable that.

  In this situation, when the session of track number 1 is disconnected (step S137) and a series of communication procedures from step S133 to step S135 are performed on the content of track number 2, the reproduction range in step S135 is as follows. It must be specified like this.

“PLAY rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
“Range: npt = 100.0-235.0”
FIG. 15 shows a permutation of the content pieces stored in the buffer unit 1032 immediately after the process of step S136. Let T1 be the time information of content with a playback time of 100 seconds, and T2, T3, and T4 in chronological order. FIG. 15 shows that the content of track number 1 is accumulated up to the content segment 4, and the content of track number 2 is subsequently accumulated from the content segment 1. When the user views in this permutation of content pieces, the content returns from T4 to T1, so it feels like it has been wound for a moment, and it does not switch smoothly, and a severe discomfort occurs.

The time taken from the time when the session with track number 1 is disconnected in the process of step S137 until the request for preparation for session establishment with track number 2 is received is the total playback time of packets already accumulated in the buffer unit 1032. If it is longer than this, a buffer underflow occurs, and when the timeout variable TOUT of the content reading unit 1034 expires, the reproduction stops.
Similarly, the session of track number 1 is disconnected in the process of step S137, the time taken for a series of communication procedures from step S133 to step S135 is measured for the content of track number 2, and the track number in the process of step S135 is measured. A method to reduce the sense of incongruity by correcting the NPT of 2 can be considered. For example, if it takes 5.5 seconds for a series of communication procedures, it is corrected to npt = 105.5-235.0.

  However, even if this method is applied, it is not possible to consider the time delay until the reproduction start request for track number 2 reaches the content distribution server 102b and the packet transmission is actually started. It is difficult to configure the content playback time for packets to be completely continuous.

  Japanese Patent Laid-Open No. 2002-118592 (Patent Document 2) is disclosed as a first improvement method for coping with the above problem.

  This is because a secondary delivery server is provided between the delivery server and the client, and the secondary delivery server buffers the bitstream after switching to some extent and then switches, so that the client buffer is not emptied and the content is It can be switched without interruption.

  However, in Patent Document 2, each of bitstreams distributed from a plurality of distribution servers is temporally independent from each other, and frame boundaries are not synchronized. That is, buffer underflow can be prevented, but the content rewinding feeling is ineffective, and it is difficult to configure the content playback time to be completely continuous.

  As a second improvement method for coping with the above problem, many streaming systems adopt a multi-track simultaneous reception configuration. As an example, FIG. 16 shows a functional block diagram of the content distribution server 102c and the content receiving device 103c that constitute the content distribution system 101c capable of receiving two tracks simultaneously.

  The content distribution server 102c includes a content acquisition unit 1021A, a transmission unit 1022A, a content acquisition unit 1021B, a transmission unit 1022B, a system clock oscillation unit 1023, and a content storage unit 1024. In addition, the content receiving device 103c includes a receiving unit 1031A, a buffer unit 1032A, a content reading unit 1034A, a receiving unit 1031B, a buffer unit 1032B, a content reading unit 1034B, a system clock oscillation unit 1033, a reproduction unit 1035, a transmission control unit 1036, an input Part 1037 and switching part 1038. In addition, the same number is attached | subjected about the same thing as FIG. 13, and the description is abbreviate | omitted.

  The switching unit 1038 has a function of adopting one of the two content segments and inputting it to the playback unit 1035 in accordance with the track switching input. Although the content piece that has not been adopted is read, it is discarded as it is.

  Note that the input unit 1037 accepts a track switching input for switching the track number 1 or 2 input by the user.

  When there are a plurality of synchronized tracks, the content receiving apparatus 103c repeats a series of preparation requests / responses corresponding to steps S133 to S134 for the number of tracks, and then starts a reproduction request corresponding to step S135. This is dealt with by providing the content receiving apparatus 103c with a switching unit 1038 that receives a stream corresponding to the number of tracks and reproduces one of a plurality of tracks on the receiving side.

  FIG. 17 is a flowchart showing the operation of the content receiving apparatus 103c having a multiple-track simultaneous reception configuration. Since all processes (steps S142 to S146) other than the process of step S141 are the same as steps S122 to S126 of FIG. 12, the description thereof is omitted.

  The content reading unit 1034 (A or B) on the side that receives the content of the track that is selected for playback monitors the buffer unit 1032 (A or B), waits until a predetermined amount is accumulated, and is set in advance. When the threshold value of the accumulation amount is exceeded, a true / false value “Y” is generated, and the process proceeds to the next process (step S141). This is intended that the reproduction start times STS between a plurality of tracks are exactly the same.

  The above is a conventional streaming system with a bitstream switching operation used for real-time distribution of video and audio.

  The communication karaoke system employing such a streaming system has the following problems.

  First, as a first problem, in order to realize a key control function similar to that of a communication karaoke system employing MIDI in a communication karaoke system employing a streaming system, the content receiving device side has a high computing capability. It has been necessary to mount the pitch conversion means, which has been a cause of cost increase.

  The second problem is that when a relatively small device such as a mobile phone or a PDA (Personal Digital Assistant) is used as the content receiving device and the board area is limited, a pitch conversion means is used. There was a situation that could not be installed.

The third problem is that when the pitch conversion means is realized by software, the amount of calculation increases as the quality is improved, so that the mobile phone or PDA having a central processing unit that is ineffective as a content receiving device. Then, it was not possible to adopt a high-quality pitch conversion algorithm.
JP-A-9-185392 JP 2002-118592 A

  The present invention has been made in view of the above circumstances, and a content distribution system comprising a content distribution server that transmits content data composed of a plurality of tracks via a network, and a content reception device that receives and reproduces the content data. The present invention relates to a content distribution method, a content distribution server using the content distribution method, and a content receiving apparatus that can dynamically switch and transmit tracks and smoothly reproduce the switched portion.

  In order to achieve the above object, a content distribution method according to claim 1 includes a content distribution server storing content data including a plurality of tracks having a plurality of content segment data, and the content distribution server. On the other hand, a content distribution device comprising: a content receiving device that makes a content reproduction request via a network and receives and reproduces the content fragment data transmitted from the content distribution server via the network in response to the reproduction request In the content distribution method in the system, the content data stored in the content distribution server is content data having a file name and a plurality of tracks, and the track data of each track includes a track number, a track time stamp Frequency, total number of frames Each track in the same frame number includes track key information indicating the degree of transposition of the pitch, track duet information and frame number indicating presence / absence of partner audio, time information indicating timing of reproduction, and content fragment data. The data frames have the same time information, and the content receiving device establishes a session between the content receiving device and the content distribution server to receive the content fragment data Requesting a session description, which is prior information, transmitting the session description including at least the track number, the track key information, and the track duet information for each of the tracks, and the content The receiving device is When receiving the description of the application, the step of displaying the track key information and the track duet information in association with the track number, and the content receiving device, together with the file name of the content data, the track number, and the playback range of the content Transmitting a content reproduction request message; and when the content distribution server receives the reproduction request message, the content element of the designated track according to the file name, the track number, and the content reproduction range. Obtaining and transmitting one piece of data, and when switching the track, the content receiving device transmits a track switching request message together with the track number, and the content distribution server includes the switching Request When the message is received, the designated track having time information to be reproduced continuously in accordance with the track number and the content fragment data whose reproduction order is the last among the transmitted content fragment data And acquiring and transmitting the content segment data.

  The content distribution server according to claim 2 selects content segment data requested for reproduction from content data including a plurality of tracks having a plurality of content segment data in accordance with a reproduction request from the content receiving device. A content distribution server for transmitting the selected content fragment data to the content receiving device via a network, the content data having a file name and a plurality of tracks, wherein the track data of each track has a track number Each of the data frames includes a track number, a track time stamp frequency, a total number of frames, track key information indicating the degree of pitch transposition, track duet information indicating the presence or absence of partner audio, and a frame. Number, playback timing Content storage means for storing content data having the same time information in which the data frame of each track in the same frame number has the time information and the content segment data. When a session description, which is prior information for establishing a session between the content distribution server and receiving the content fragment data, is requested, at least the track number, the track key information, Transmission of the session description including the track duet information, and reception of a content reproduction request message together with the file name of the content data, the track number, and a content reproduction range, and a track switching request message together with the track number. When the control means and the transmission control means receive the playback request message, the content segment data of the designated track is sent from the content storage means according to the file name, the track number, and the playback range of the content. And when the transmission control means receives the switching request message, the content unit data that has been played in the last order among the content unit data that has already been sent is reproduced in accordance with the track number. Content acquisition means for acquiring the content fragment data of the designated track having time information to be acquired from the content storage means.

  According to a third aspect of the present invention, there is provided a content receiving apparatus that makes a content reproduction request via a network to a content distribution server storing content data having a plurality of tracks having a plurality of content segment data. A content receiving apparatus that receives and plays back content fragment data transmitted from the content distribution server via the network in response to the playback request, and the content data stored in the content distribution server includes a file name and a plurality of data The track data of each track has a track number and a plurality of data frames, and each data frame has a track number and a plurality of data frames. Data frame is track number, track time stamp It has the number of waves, total number of frames, track key information indicating the degree of transposition of pitch, track duet information and frame number indicating presence / absence of partner audio, time information indicating playback timing, and content fragment data, and the same The data frame of each track in the frame number has the same time information, and a session is established between the content receiving device and the content distribution server with respect to the content distribution server. Requesting a session description, which is prior information for receiving the content fragment data, and receiving the session description, display the track key information and the track duet information in association with the track number, and reproduce the content In the case of A transmission control means for transmitting a content reproduction request message together with the track number, the track number, and the content reproduction range, and transmitting a track switching request message together with the track number when switching tracks. Features.

  In the communication karaoke system according to the present invention, data including back chorus and partner voice can be provided to the user with quality close to the original sound, and the pitch can be dynamically switched according to the user's request. Contributes to high quality.

  In addition, there is no need to install pitch conversion means on the receiving client side, and pitch conversion is performed at the time of authoring playback content, so it is possible to use low-speed and higher-quality pitch conversion means than before, improving service quality. Can be made.

  In addition, it is not necessary to install pitch conversion means on the receiving client side, and pitch conversion is performed at the time of authoring the playback content. Therefore, new pitch conversion means can be applied at any time after the receiving client is shipped, and the resulting receiving client The effect of the pitch change in is independent of the hardware production time and shipping time of the receiving client, and the same effect can be obtained in all receiving clients.

  Also, during authoring of the playback content, a certain part of a plurality of pitch ranges is converted by a mechanical process, and a certain part is actually played by a performer / singer using a transposed score. -By using the sung voice data, it is possible to realize a high quality service that is impossible with the conventional method.

  In addition, for playback content that the author of the playback content does not consent to music processing such as pitch conversion, it is easy without adding special information to all receiving clients without adding a track that has been pitch-converted during authoring. In addition, it is possible to prohibit pitch conversion playback for specific playback content.

  An embodiment of the present invention will be described with reference to FIGS.

<< First Embodiment >>
FIG. 1 shows a system configuration of the content distribution system 1a (communication karaoke system). The content distribution system 1a includes a content distribution server 2a, a content reception device 3a (karaoke device), and an authoring device 5a. The content distribution server 2a and the content reception device 3a are connected to each other by an IP network 4.

  The content distribution server 2a includes a content acquisition unit 21, a transmission control unit 22a, a transmission unit 23, a system clock oscillation unit 24, and a content storage unit 25.

  The content acquisition unit 21 records content data of the content to be reproduced in the content storage unit 25, and acquires a content fragment using an arbitrary pointer (read position on the content) based on the transmission control information in accordance with the system clock signal. Has the function of This pointer is any one of the content file name, the track number, and the frame number, or some combination thereof. The “content segment” refers to a data block having an arbitrary data length.

  The content acquisition unit 21 has initial values such as a content file name, a track number, and a frame number inside, and may automatically start reading from the time of start-up, or the content file name, track number, and content from an external input Reading may be started after the reproduction start time is obtained. In addition, an external input for receiving command input is provided, and when command input = reproduction start constant (for example, “PLAY”), reproduction may be performed, or command input = reproduction stop constant (for example, “STOP”). If so, it may be stopped. Alternatively, the automatic read start operation and the external read start operation may be automatically determined.

  When the transmission control unit 22a receives transmission control information including instructions such as content selection, content playback / stop, and track switching request from the content receiving device 3a, the transmission control unit 22a returns a response to the transmission control information. It has a function to notify the acquisition unit 21 and the transmission unit 23. Also, the transmission control unit 22 a refers to the content storage unit 25, acquires the content file name, track number, and content playback start time, inputs them to the content acquisition unit 21, and transmits them to the transmission unit 23. It has a function to input an address.

  As a protocol to be applied, a real-time data transmission control protocol typified by RTSP defined in RFC2326 is assumed, and methods such as Setup, Play, Pause, Teardown, and Describe can be used.

  The transmission unit 23 has a function of securing a virtual communication path called a session, packetizing the content fragment with an appropriate header, and transmitting the packet to the content reception device 3a via the IP network 4. Have. For example, the header includes a network time stamp.

  When transmitting a packet, the transmission unit 23 uses a transmission address to deliver the packet to the content receiving device 3a. The transmission address is composed of a combination of a transmission address and a transmission port number. The transmission address may be previously stored as a constant in the transmission unit 23, or an input may be received from another block. Note that a session (communication path) for transmitting a packet is dynamically established when a transmission address is determined. Further, the content segment and the packet do not necessarily match.

  The system clock oscillator 24 is a real-time clock (high-precision clock) realized by a crystal oscillator or the like, and has a function of supplying a system clock signal to the content acquisition unit 21.

  The content storage unit 25 has a function of storing content data in a predetermined file format.

  Further, the content receiving device 3a includes a receiving unit 31, a transmission control unit 32a, a buffer unit 33, a system clock oscillation unit 34, a content reading unit 35, a reproducing unit 36, an input unit 37, and a microphone input unit 38.

  The receiving unit 31 has a function of receiving a packet from the IP network 4, interpreting the header, extracting a content fragment, and supplying the extracted fragment. The receiving unit 31 uses the reception address in order to receive only the packets necessary for itself among the packets from the content distribution server 2a. The reception address is composed of a combination of a reception address and a reception port number. The reception address may be previously stored as a constant inside the reception unit 31, or input may be received from another block.

  The transmission control unit 32a transmits the transmission control information from the input unit 37 to the transmission control unit 22a of the content distribution server 2a, receives the response, analyzes the response, extracts the reception address, and inputs the received address to the reception unit 31. It has the function to do.

  The buffer unit 33 has a function of temporarily accumulating and storing the content pieces together with a network time stamp (described later) obtained from the header.

  The content reading unit 35 monitors the buffer unit 33 and starts reading the content unit in accordance with the system clock signal from the system clock oscillation unit 34 when it is determined that the content unit sufficient for reproduction has been accumulated. It has a function of restoring and outputting a set of segments as content.

  The playback unit 36 performs decoding according to the input content fragment, combines it with the audio signal from the microphone input unit, and then outputs the audio signal and the video signal to a predetermined output device such as a speaker. Have For example, when the content segment is digital audio data encoded with high efficiency similar to MPEG, the playback unit 36 includes a high efficiency code decoder (decoder), a D / A converter, an analog amplifier, and the like. Is done.

  Here, the minimum requirement for composition is to reproduce two inputs from the content reading unit 35 and the microphone input unit 38 almost simultaneously. If the content segment type is audio, appropriate mixing synthesis and volume adjustment may be performed so as not to be unnatural with the audio from the microphone input unit 38.

  The input unit 37 has a function of accepting transmission control information input processing such as content selection, content playback / stop, and track switching from the user. For example, using a bitmap display and a keyboard, GUI parts such as a play button, a stop button, and a content selection dialog are provided to the user, and a track number is input using a numeric keypad and several stages of slide switches. Provide an operation switch that accepts.

  The microphone input unit 38 has a function of receiving an input of an audio signal captured by an audio capturing device such as a microphone and converting the audio signal into a data format handled by the playback unit 36b.

  The authoring device 5 a includes an authoring unit 51 a, a pitch conversion unit 52, a content transmission unit 53, an input unit 54, a microphone input unit 55, and a content primary storage unit 56. The authoring device 5a may be housed in the same housing as the content distribution server 2a or in a separate housing. For convenience, the user who uses the authoring device 5a is called an editor.

  The authoring unit 51a uses the instruction input from the input unit 54 and the audio data from the microphone input unit 55 to create and edit content data, such as creating new content data, adding tracks, deleting content data, etc. It has a function of recording edited content data in the content primary storage unit 56.

  The pitch conversion unit 52 performs a pitch conversion for at least a semitone or a full tone on the original sound data (hereinafter referred to as source sound data) constituting the content (for example, frequency using sound frequency domain conversion). Using a shift algorithm).

  Here, an integer variable in which the scale of the reference source audio data is “0”, “+1” when shifted upward by a semitone, and “−1” when shifted downward by a semitone is referred to as a key variable k. When realizing pitch conversion in a general karaoke device, the key variable k can take a range of at least “−6 to 0 to +6”.

  For example, when the key variable k = “+ 1”, this corresponds to a transposition of the second minor, and when the original music is in C major, the key after the transposition is a D major. Further, when the key variable k = “− 6”, it corresponds to a decrease of 5 degrees, and when the original music is in C major, the key after transposition is in F major.

  Note that the reason why the key variable k is less than “−6” (or greater than +6) is not to use, for example, to reduce it by −6 in a semitone shift, and to 1 after a +6 (increase of 4 degrees) in a semitone shift. This is because it corresponds to a transposition to lower the octave. In the case of karaoke, as long as it matches the key intended by the user, it is possible to sing comfortably even if the performance octave is different.

  When the content file movement instruction and the destination distribution server name are input from the input unit 54, the content transmission unit 53 stores the content data stored in the content primary storage unit 56 in the distribution server indicated by the destination distribution server name. The content storage unit 25 has a function of transmitting via the network.

  The content transmission unit 53 has a function of recording content data on a recording medium such as a CD-R or a DVD-R. When content data is not transmitted via a network, the content data is provided on a recording medium.

  The input unit 54 has a function for the editor to accept each instruction input of a content data new creation instruction, a track addition instruction, a content data movement instruction, and a content data deletion instruction. For example, using a bitmap display and a keyboard, GUI parts such as a play button, a stop button, and a content selection dialog are provided to the editor, and a track number, a slide switch, etc. Provided is an operation switch for receiving an input of a key variable k for pitch conversion.

  The microphone input unit 55 has a function of receiving input of an audio signal captured by an audio capturing device such as a microphone and converting the audio signal into a data format handled by the playback unit 36b to generate source audio data. The microphone input unit 55 may include an interface for receiving source audio data from the outside such as a CD-ROM drive or a network. Further, the microphone input unit 55 may include a microphone and a memory, and may record an audio signal for a predetermined time and convert it into source audio data. When the editor inputs a track addition instruction, source audio data is simultaneously taken into the authoring device 5a through the microphone input unit 55.

  The content primary storage unit 56 has a function of temporarily storing content data created and edited by the authoring unit 51a.

  Next, the operation of the authoring device 5a will be described using the flowchart of FIG.

(1) New creation of content data When the content data new creation instruction and the content file name CF1 are input from the input unit 54, the authoring unit 51a stores content data that does not include any track in the content primary storage unit 56. Create with content file name CF1.

(2) Deletion of Content Data When the content data deletion instruction and the content file name CF1 are input from the input unit 54, the authoring unit 51a deletes the content data of the content file name CF1 recorded in the content primary storage unit 56. delete.

(3) Track addition The authoring unit 51a receives the track addition instruction, the content file name CF1, the track number TN1, and the key variable k from the input unit 54, and acquires the source audio data input from the microphone input unit 55 (step First, it is determined whether or not the key variable k is “0” (step S02). If the key variable k is other than “0”, the pitch conversion process is performed on the input source voice data by the pitch conversion unit 52 (step S03), and the source voice data subjected to the pitch conversion process on the authoring unit 51a. Supply. If the key variable k is “0”, the process proceeds to step S04.

  Next, the authoring unit 51a opens the content data indicated by the content file name CF1, adds +1 to the total number of tracks, adds the track data indicated by the track number TN1, and sets the source audio data for each appropriate number of samples. Divide into segments (step S04). Next, the authoring unit 51a performs high-efficiency encoding if necessary, and assigns and multiplexes the frame number and time information to generate track data (step S05). The track time stamp frequency is written by measuring the sampling frequency of the audio data.

  Basically, like the conventional content data shown in FIG. 10, the content data in this embodiment is composed of a content file name, the total number of tracks T, and track data of each track, and further, track data of each track. Is composed of a track number, a track time stamp frequency, the total number of frames, and a plurality of data frames, and each data frame is composed of a frame number, time information, and a content fragment.

  In order to continue playback smoothly when content pieces for a plurality of tracks have arrived from the content distribution server 2a to the content receiving device 3a, at least a plurality of tracks in which a track can be switched in the content file are mutually connected. In each data frame to which a piece is assigned, the time boundaries on the content reproduction time must match (content pieces having the same frame number are reproduced at the same reproduction time).

  In other words, when the same frame numbers FNA and FNB exist in a certain track A and a certain track B, the time information accompanying the FNA / track time stamp frequency of the track A and the time information accompanying the FNB / the time information of the track A The track timestamp frequency must match.

  FIG. 3 shows a data structure of content data having a plurality of tracks (arranged along the time axis). The horizontal axis represents the content playback time (each time information / track time stamp frequency).

  In FIG. 3, content segments, FN1, FN2,... FNn, FNm, etc. are frame numbers assigned to the content segments. The total number of frame numbers of one track does not need to match, and in FIG. 3, track 1 has a total number of frames n and track 2 has a total number of frames m. If the content data has been subjected to high-efficiency encoding such as MPEG-4 AAC hi-bitrate encoding, a single content segment includes a plurality of high-efficiency encoded compressed packets. It doesn't matter.

  However, the total time length when reproducing a plurality of compressed packets included in one content segment must match when content segments having the same frame number are extracted from two or more tracks. Therefore, the time information x of the content segment x of each track in the same frame number FNx always has the same value if the track timestamp frequency of each track is the same value.

  Therefore, as shown by the arrow (A) in FIG. 3, even when switching from the track 1 to the track 2 in the middle of the reproduction at the frame number FN2, the reproduction can be continued smoothly. The authoring unit 51a described above is suitable for generating content data. If the same source audio data is sequentially stored in each track by designating different key variables, all the audio data is divided into content segments according to the same rule in step S04, so that the above-described data structure is provided. .

  In this way, a plurality of tracks having different pitches are created, and the authoring device 5a supplies the created content data to the content distribution server 2a, so that high-quality pitch conversion can be performed during content authoring. Quality can be improved.

  Next, operations of the transmission control unit 22a of the content distribution server 2a and the transmission control unit 32a of the content receiving device 3a at the time of transmission / reception of transmission control information will be described with reference to the sequence diagram of FIG. FIG. 4 shows a process that is performed until one content is selected using RTSP, a session for transmission is established, playback is started, and stopped. Included in the information.

  Here, the host name of the content distribution server 2a is “server.jvc-victor.jp”, the content file name in which the content is stored is “hoge.mp4”, and “hoge.mp4” contains 1 Assume that there are two MPEG-4 AAC audio tracks, the track number is 1, and the playback time is 235 seconds.

  RTSP applies the content URI as a resource identifier for uniquely identifying the content on the network. The content “hoge.mp4” is represented by the following content URI.

“Rtsp: //server.jvc-victor.jp/hoge.mp4”
First, the transmission control unit 32a requests a session description by transmitting a content URI (step S11). The session description is text information indicating what kind of session (communication path) can be established for the content data associated with the content URI. SDP is applied as the session description format. Use the DESCRIBE method to request a session description in RTSP. Each method and its response message (including other accompanying information) are inserted and received in the header of the request message and the header of the response message, respectively.

“DESCRIBE rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
The transmission control unit 22a transmits a response including the session description to the transmission control unit 32a (step S12). The session description includes a playback time and a media description. This playback time is the maximum playback time of continuous media associated with the specified content URI. There are many different playback time formats, but the simplest format is NPT, which represents the start time and end time in seconds using floating point. For example, the playback time of a content URI indicating content data including an MPEG-4 AAC audio track for 235 seconds is expressed as follows using NPT.

“A = range: npt = 0.0-235.0”
Also, the media description includes information such as content type and network time stamp frequency as prior information for establishing a session.

“M = audio 0 RTP / AVP / UDP 96”
“A = rtpmap: 96 mpeg4-generic / 48000/2”
“A = control: rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1”
“A = fmtp: 96 streamtype = 5; profile-level-id = 15; mode = AAC-hbr; config = 1190; SizeLength = 13; IndexLength = 3; IndexDeltaLength = 3; Profile = 1;”
The above media description expresses that stereo sound of 48000 Hz (48 KHz) PCM sample frequency encoded by the MPEG-4 AAC Hi-bitrate encoding method is included. Also, it is shown that when establishing a session, UDP must be used as a lower layer network protocol and RTP should be applied for transmission.

  Also, it is shown that the following control URI is used for the preparation request of this session. The control URI is given a track number (trackID = 1) to identify a plurality of tracks in one content data.

“Rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1”
The transmission control unit 32a that has analyzed the session description determines to establish a session for transmitting MPEG-4 AAC stereo audio, determines a reception address of the session, and sends a request for session establishment preparation to the transmission control unit 22a. Transmit (step S13). The SETUP method is used for a session establishment preparation request in RTSP. In the following example, the reception address is the reception address (136.198.190.100) and the reception port number (6668-6669) of the transmission control unit 32a.

“SETUP rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1 RTSP / 1.0”
“Transport: RTP / AVP / UDP; unicast; destination = 136.198.190.100; client_port = 6668-
6669 ”
The transmission control unit 22a that has correctly received the session establishment preparation request newly determines a transmission address, inputs the transmission address to the transmission unit 23, and transmits session information to the transmission control unit 32a (step S14). The session information includes a transmission address (136.198.190.1 in the example) and a transmission port number (19000 to 19001 in the following example) used for distribution.

“Transport: RTP / AVP / UDP; unicast; source = 136.198.190.1; server_port = 19000-19001”
In addition, the transmission control unit 32 a inputs the previously determined reception address to the reception unit 31 when the session information is received. A new session is established when the processing of both the transmission unit 23 and the reception unit 31 is completed.

  Next, the transmission control unit 32a transmits to the transmission control unit 22a a track switching request (CHANGE method) that designates a track of content to be played back in accordance with a user input operation (step S15). In the track number designation (TrackID =), the track number to be switched next is set.

“CHANGE rtsp: //server.jvc-victor.jp/hoge.mp4/TrackID=2 RTSP / 1.0”
Upon receiving the track switching request, the transmission control unit 22a refers to the content storage unit 25, and if the track corresponding to the track number to be switched next exists in the content hoge.mp4, transmits the normal response below. (Step S16).

“RTSP / 1.0 200 OK”
If the track corresponding to the track number to be switched next does not exist in the content hoge.mp4, the transmission control unit 22a transmits the following abnormal response.

“RTSP / 1.0 404 NOT FOUND”
Note that the track switching request and the track switching response are examples, and uniqueness is not lost due to a slight difference in the character string expression. In this embodiment, RTSP is extended, but the track switching request may be rewritten as follows using the GET method of HTTP (Hyper Text Transfer Protocol) (Network Working Group, “Hypertext Transfer Protocol—HTTP / 1.1 ", RFC2616, The Internet Society, June 1999).

“GET /hoge.mp4?TrackID=2 HTTP / 1.1”
Although the GET method does not include the host name of the content distribution server 2a, the content file name and the track number to be switched next can be included in the same manner as RTSP.

  Next, the transmission control unit 32a transmits a reproduction start request (step S17). In the playback start request, the playback range can be specified using NPT. In the example below, the content is specified from the beginning (0.0 seconds) to the end (235.0 seconds). The PLAY method is used for a playback start request in RTSP.

“PLAY rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
“Range: npt = 0.0-235.0”
The transmission control unit 22a transmits a response to the reproduction start request (Step S18). The transmission control unit 22a that has received the reproduction start request, for all the sessions that have already been prepared, from the control URI of each session and the reproduction range included in the reproduction start request, the content file name / track number / content reproduction start The time is calculated and input to the content acquisition unit 21 together with command input = reproduction start constant (for example, “PLAY”), and transmission of the RTP packet is started.

  This response includes RTP packet information necessary for the operation of the content reading unit 35 of the content receiving device 3a. For example, in this session, the RTP time stamp (network time stamp) of the first RTP packet sent is included (0000000 in the following example).

“RTP-Info: url = rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1; rtptime = 0000000;”
The track switching request may be transmitted at any time after the preparation request response in step S14 and before the stop. Accordingly, when track switching is input during playback of a certain track, the transmission control unit 32a transmits the track switching request transmitted in step S15 (step S19), and the transmission control unit 22a receives the track switching request. Then, the track number to be switched next is input to the content acquisition unit 21. The content acquisition unit 21 confirms the presence of the track number to be switched next, and inputs the switching result (= normal or = abnormal) to the transmission control unit 22a. When the switching result is input, the transmission control unit 22a transmits a track switching response (a normal response if the switching result = normal, an abnormal response if the switching result = abnormal) to the transmission control unit 32a (step S20). ).

  Further, the content acquisition unit 21 acquires a content fragment of the designated track from the content storage unit 25 and transmits it to the content reception device 3a via the transmission unit 23 (details will be described later).

  Moreover, the transmission control part 32a transmits a stop request | requirement at the time of a user performing stop input (step S21). In RTSP, the stop request is the TEARDOWN method.

“TEARDOWN rtsp: //server.jvc-victor.jp/hoge.mp4 RTSP / 1.0”
The transmission control unit 22a that has received the stop request controls the transmission unit 23 to stop transmission of the RTP packet, disconnect the session, and notify that the transmission has been stopped (step S22). Further, the transmission control unit 22a inputs a command input = reproduction stop constant (for example, “STOP”) to the content acquisition unit 21, and stops reading.

  For simplification of explanation, it is assumed that only one content is included in one content data. However, the transmission control mechanism can be easily extended to synchronous reproduction of a plurality of contents. For example, synchronized playback of a plurality of contents can be easily performed by simply listing media descriptions regarding a plurality of contents in step S12 and repeating the preparation requests in steps S13 to S14 for the media descriptions.

  Next, the reproduction time management process (transmission side) of the content acquisition unit 21 when reading content data will be described with reference to the flowchart of FIG.

  First, when a content reproduction request is made from the content receiving device 3a, the content acquisition unit 21 of the content distribution server 2a initializes the following six variables (step S31).

1: Content file name CF = Default content file name (eg "hoge.mp4")
2: Track number TN = default track number (eg "1")
3: Start frame number FN = default frame number (for example, “1”)
4: Playback start time STS = current time NTS obtained from the system clock oscillator 24
5: Content playback time CTS = “0”
6: Playback state flag F = true value “N”
Next, the content acquisition unit 21 refers to a predetermined automatic mode constant indicating whether or not to automatically start reproduction, and determines whether to perform automatic reproduction (step S32). If the automatic mode constant is a true / false value “Y”, the process proceeds to step S44. If the automatic mode constant is a true / false value “N”, the process proceeds to step S33.

  In the process of step S32, if the automatic mode constant is a true / false value “N”, the content acquisition unit 21 performs external input (command input CMD, content file name CF1, track number TN1, content playback start time CTS1. (Step S33), if there is an external input, the truth value is “Y”, and the process proceeds to step S34. If there is no external input, the truth value “N” "And the process proceeds to step S40.

  In the process of step S33, if the truth value is “Y”, the content acquisition unit 21 determines whether command input CMD = track switching constant (eg, “CHANGE”) among external inputs (step S33). S34) If the track switching constant, the true / false value is “Y”, and the process proceeds to step S35. If the true / false value is “N”, the process proceeds to step S38.

  Also, in the process of step S34, if the truth value is “Y”, the content acquisition unit 21 uses the external input (content file name CF1, track number TN1) to make CF and CF1 equal. In addition, it is determined whether or not the track indicated by the track number TN1 exists in the content file name CF and whether the track can be switched (step S35). If possible, the true value is “Y”, and the process proceeds to step S36. After substituting TN = track number TN1 and performing track switching processing, the transmission control unit 22a is notified of the switching result = normal (step S36). If the track is not present and cannot be switched, the truth value “N” is obtained, and the content acquisition unit 21 proceeds to the process of step S37 and notifies the transmission control unit 22a of the switching result = abnormal (step). S37).

  In the process of step S34, if the truth value is “N”, the content acquisition unit 21 determines whether command input CMD = reproduction stop constant among the external inputs (step S38), and the command When the input CMD = reproduction stop constant, the truth value “Y” is obtained, and the entire operation is stopped. Otherwise, the truth value “N” is obtained, and the process proceeds to step S39.

  In the process of step S38, if the truth value is “N”, the content acquisition unit 21 uses the three variables of the external input (content file name CF1, track number TN1, content playback start time CTS1). Two variables are initialized (step S39).

1: Content file name CF = Content file name CF1
2: Track number TN = Track number TN1
3: Playback start time STS = current time NTS obtained from the system clock oscillator 24
4: Content playback time CTS = Content playback start time CTS1
5: Playback state flag F = true value “Y”
Further, the content acquisition unit 21 initializes the start frame number FN using the content reproduction time CTS, and calculates transmission start time information StartTS as a previous step. The calculation formula of the transmission start time information StartTS is shown below.

[Equation 7]
StartTS = CTS × TRS
Here, the track time stamp frequency TRS is the track time stamp frequency of the track indicated by the track number TN in the content file name CF.

  Next, the content acquisition unit 21 sequentially searches the frame number of the track indicated by the track number TN in the content file name CF from 1 to obtain the frame number NFN of the point where time information ≧ StartTS is satisfied, and the frame number NFN. Is set to the start frame number FN.

  Also, in the process of step S33, the content acquisition unit 21 determines whether or not the value of the reproduction state flag F is a true / false value “Y” (step S40), and if the value is a true / false value “Y”, step The process proceeds to S41, and if the value is “N”, the process proceeds to Step S33.

  In the case where the value of the reproduction state flag is the true / false value “Y” in the process of step S40, or in the case of the process continued from steps S36, S37, and S39, the content acquisition unit 21 starts from the system clock oscillation unit 24. Transmission is possible by calculating the elapsed time from the obtained current time NTS, playback start time STS, and system clock frequency SS, and calculating sendable time information LastTS from the elapsed time, content playback time CTS, and track time stamp frequency TRS A range is searched (step S41). The formula for calculating the sendable time information LastTS is shown below.

[Equation 8]
LastTS = (NTS−STS) / SS × TRS
Here, the system clock frequency SS is a constant for converting (current time NTS−reproduction start time STS) into seconds. For example, if the system clock has an accuracy of 1/1000000 second, it is 1000000.

  Next, the content acquisition unit 21 sequentially searches the frame number from the start frame number FN in the track indicated by the track number TN in the content file name CF, and acquires the frame number NFN at which time information ≧ LastTS is satisfied. To do. When the frame number NFN that satisfies the condition is not found, the content acquisition unit 21 sets the termination constant “−1” to the frame number NFN.

  Next, the content acquisition unit 21 determines whether or not transmission is possible by comparing the start frame number FN and the frame number NFN (step S42). If FN <NFN is satisfied, the truth value “Y” is obtained. The process proceeds to S43. If not established, the truth value is “N”, and the process proceeds to Step S33.

  If the value in the process of step S42 is true / false “Y”, the content acquisition unit 21 is less than the frame number NFN from the start frame number FN in the track indicated by the track number TN in the content file name CF. Each content unit corresponding to each frame number and the network time stamp PTS generated from the corresponding time information according to the following formula for each content unit are output to the transmitter 23 (step S43).

[Equation 9]
After outputting PTS = time information / TRS × network time stamp frequency, the content acquisition unit 21 sets start frame number FN = frame number NFN to shift the start frame number FN to the transmitted position.

  If the automatic mode constant is the true / false value “Y” in the process of step S32, the content acquisition unit 21 sets the reproduction state flag F = the true / false value “Y” and the reproduction start time STS. = Set the current time NTS obtained from the system clock oscillating unit 24 and set for automatic reproduction start (step S44).

  In this way, a track switching message is transmitted and received on the control line established by the transmission control unit 22a and the transmission control unit 32a, and the content piece of the destination track switched based on the time information is acquired and transmitted according to the message. Therefore, the content receiving device 3a can perform smooth reproduction even when the track is switched.

  Note that communication between the transmission unit 23 and the reception unit 31 may be performed in accordance with RTP (Real-time Transfer Protocol, RFC1889) rules.

  In the content distribution system 1a of FIG. 1, the content distribution server 2a, the content receiving device 3a, and the authoring device 5a have only one configuration. However, as shown in FIG. The content distribution server 2a may be connected, and the content receiving device 3b may be connected to each content distribution server 2a.

  This is suitable as an example of a communication karaoke system that can provide services to a large number of users. Receiving each content only by copying the same content data to a plurality of content distribution servers 2a by preparing content data in which content data having different pitches are recorded on a plurality of tracks by performing a pitch conversion process in the authoring device 5a. Even if the device 3a is not equipped with means for performing pitch conversion, it is possible to perform dynamic pitch conversion during playback by changing the track. In addition, even if the number of users increases and the number of content distribution servers 2a and content receivers 3a increases, the time and calculation costs required for the pitch conversion process are always constant, and dramatic cost reductions are possible. It becomes.

<< Second embodiment: Switching partner audio in duet >>
In the above embodiment, it is assumed that you know whether what pitch of the content data in advance which tracks are recorded. That is, it can be said that the track number and the key variable are associated one by one.

  However, in the case of a duet song, when the accompaniment pitch is changed, it is necessary to change the pitch of the partner voice as well. Therefore, in the second embodiment, a content distribution system 1b that simultaneously realizes partner voice switching and key control will be described.

  FIG. 7 shows a system configuration of the content distribution system 1b (communication karaoke system). The content distribution system 1b includes a content distribution server 2b, a content reception device 3b (karaoke device), and an authoring device 5b. The content distribution server 2b and the content reception device 3b are connected to each other by an IP network 4. In addition, the same number is attached | subjected about the same thing as 1st Embodiment, and the detailed description is abbreviate | omitted.

  The content distribution server 2b includes a content acquisition unit 21, a transmission control unit 22b, a transmission unit 23, a system clock oscillation unit 24, and a content storage unit 25.

  When the transmission control unit 22b receives transmission control information including instructions such as content selection, content playback / stop, and track switching request from the content receiving device 3b, the response including the track number, track key information, and track duet information is received. And transmitting the transmission control information to the content acquisition unit 21 and the transmission unit 23. Also, the transmission control unit 22 b refers to the content storage unit 25 to acquire the content file name, track number, and content playback start time, inputs them to the content acquisition unit 21, and transmits them to the transmission unit 23. It has a function to input an address.

  The content receiving device 3b includes a receiving unit 31, a transmission control unit 32b, a buffer unit 33, a system clock oscillation unit 34, a content reading unit 35, a reproduction unit 36, an input unit 37, a microphone input unit 38, and an output unit 39. Have.

  The transmission control unit 32b transmits the transmission control information from the input unit 37 to the transmission control unit 22b of the content distribution server 2b, receives the response, analyzes the response, extracts the received address, and inputs it to the reception unit 31. The track number, track key information, and track duet information are extracted and notified to the output unit 39.

  The output unit 39 is connected to a display device such as a monitor, and has a function of outputting track key information and track duet information extracted from a response including a session description to the display device.

  The authoring device 5 b includes an authoring unit 51 b, a pitch conversion unit 52, a content transmission unit 53, an input unit 54, a microphone input unit 55, and a content primary storage unit 56.

  The authoring unit 51b uses the instruction input from the input unit 54 and the audio data from the microphone input unit 55 to create and edit content data, such as creating new content data, adding tracks, deleting content data, etc. It has a function of adding track key information and track duet information to the edited content data and recording them in the content primary storage unit 56.

  In the second embodiment, a duet variable d indicating whether partner audio is included in the content data of one track is provided. When the content data of one track has no partner audio, the duet variable d = “0”, when the male partner audio is included, the duet variable d = “1”, and when the female partner audio is input, the duet variable d = “2”. A track with only partner audio without accompaniment may be prepared.

  FIG. 8 shows the data structure of content data in the second embodiment. The difference from the first embodiment is that track key information (1 to t) and track duet information (1 to t) are added to the content data as information of each track. At this time, the maximum value of the subscript t is equal to the value of the total number of tracks T.

  The track key information is information indicating how much the content data of the track is transposed, and the value of the key variable k to be referred to when the authoring unit 51b of the authoring device 5b generates the content data is recorded. The The track duet information is information indicating the presence / absence of partner voice, and the value of the duet variable d is recorded therein.

  For example, the track number 1 in FIG. 8 has a track key information 1 value of “0” and a track duet information 1 value of “1”. It turns out that it is content data.

  Next, operations of the transmission control unit 22b of the content distribution server 2b and the transmission control unit 32b of the content receiving device 3b at the time of transmission / reception of transmission control information will be described.

  When the transmission control unit 32b requests a session description by transmitting a content URI, the transmission control unit 32b transmits a response including the following media description and including the session description to the transmission control unit 32a.

  For example, the content data prepared in the content storage unit 25 has two tracks, the track key information of the track with the track number 1 is 0, the track duet information is 1, and the track key of the track with the track number 2 If the information is 1 and the track duet information is 2, the media description is as follows.

“M = audio 0 RTP / AVP / UDP 96”
“A = rtpmap: 96 mpeg4-generic / 48000/2”
“A = control: rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=1”
“A = fmtp: 96 streamtype = 5; profile-level-id = 15; mode = AAC-hbr; config = 1190; SizeLength = 13; IndexLength = 3; IndexDeltaLength = 3; Profile = 1;”
“A = karaoke: 96 keyinfo = 0; duetinfo = 1”
“M = audio 0 RTP / AVP / UDP 97”
“A = rtpmap: 97 mpeg4-generic / 48000/2”
“A = control: rtsp: //server.jvc-victor.jp/hoge.mp4/trackID=2”
“A = fmtp: 97 streamtype = 5; profile-level-id = 15; mode = AAC-hbr; config = 1190; SizeLength = 13; IndexLength = 3; IndexDeltaLength = 3; Profile = 1;”
“A = karaoke: 97 keyinfo = 0; duetinfo = 2”
Of these, keyinfo is a variable in which track key information is set, and duetinfo is a variable in which track duet information is set. The numerical values “96” and “97” are called fmt_lits in the SDP, and are values for identifying each media description.

  The transmission control unit 32b that has analyzed the session description acquires the track number, track key information, and track duet information from the media description, notifies the output unit 39, and establishes a session for transmitting MPEG-4 AAC stereo audio. The session reception address is determined, and a request for session establishment preparation is transmitted to the transmission control unit 22b.

  The output unit 39 outputs the track number, the track key information, and the track duet information in association with each other to the monitor. The display form at this time may display a designed image such as a note icon or a male / female icon instead of the numbers of the key variable k and the duet variable d. When the user selects a track to be changed based on the track number, track key information, and track duet information displayed on the monitor, the transmission control unit 32b inputs the track number input by the user's input operation. The track switching process acquired from the unit 37 and shown in the first embodiment is performed.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these. All or part of the configuration requirements of the content distribution server 2, the content receiving device 3, and the authoring device 5 are realized as a computer-executable program, and are recorded in a computer-readable recording medium and provided in advance. Is also possible.

  A plurality of programs corresponding to the content distribution server 2 may be executed by a single physical computer. Further, a plurality of programs corresponding to the content receiving device 3 may be executed by a single physical computer.

  Further, a program corresponding to the content distribution server 2 and a program corresponding to the content receiving device 3 may be executed by a single computer. Furthermore, the program corresponding to the content distribution server 2 and the program corresponding to the authoring device 5 may be executed by a single physical computer.

It is a functional block diagram of the content distribution server 2a, the content receiver 3a, and the authoring apparatus 5a in the content distribution system 1a (asynchronous model). It is a flowchart which shows the process sequence at the time of authoring. It is a figure which shows the data structure of the content data which has several tracks. It is a sequence diagram which shows operation | movement of the transmission control part 22a of the content delivery server 2a at the time of transmission / reception of transmission control information, and the transmission control part 32a of the content receiver 3a. It is a flowchart which shows the reproduction time management process (transmission side) of the content acquisition part 21 at the time of reading content data. It is a figure which shows the structural example which connected the several content delivery server 2a to the authoring apparatus 5a. It is a functional block diagram of the content distribution server 2b, the content receiver 3b, and the authoring apparatus 5b in the content distribution system 1b (asynchronous model). It is a figure which shows an example of the data structure of content data. It is a functional block diagram of the content delivery server 102a and the content receiver 103a in an asynchronous model. It is a figure which shows an example of the data structure of content data. It is a flowchart which shows the reproduction time management process (transmission side) of the content acquisition part 1021 at the time of reading a content file. It is a flowchart which shows the reproduction time management process (reception side) of the content reading part 1034 using a network time stamp. It is a functional block diagram of the content delivery server 102b and the content receiver 103b which comprise the content delivery system 101b provided with a transmission control mechanism. It is a sequence diagram which shows operation | movement of the transmission control part 1025 and the transmission control part 1036 at the time of transmission / reception of transmission control information. It is a figure which shows the permutation of the content piece accumulate | stored in the buffer part 1032. It is a functional block diagram of the content distribution server 102c and the content receiver 103c which comprise the content distribution system 101c which can receive 2 tracks simultaneously. It is a flowchart which shows operation | movement of the content receiver 103c which has a multi-track simultaneous reception structure.

Explanation of symbols

1a, b ... content delivery system, 2a, b ... content delivery server, 3a, b ... content receiving device, 4 ... IP network, 5a, b ... authoring device, 21 ... content acquisition unit, 22a, b ... transmission control unit, 23 ... Transmitter, 24 ... System clock oscillator, 25 ... Content storage, 31 ... Receiver, 32a, b ... Transmission controller, 33 ... Buffer, 34 ... System clock oscillator, 35 ... Content reader, 36 ... Playback unit 37 ... Input unit 38 ... Microphone input unit 39 ... Output unit 51a, b ... Authoring unit 52 ... Pitch conversion unit 53 ... Content transmission unit 54 ... Input unit 55 ... Microphone input unit 56 ... Content primary storage unit, 101a, b, c ... Content distribution system, 102a, b, c ... Content distribution server, 103a, b, c ... Tens receiving device, 104 ... network, 1021 ... content acquisition unit (A, B), 1022 ... transmission unit (A, B), 1023 ... system clock oscillation unit, 1024 ... content storage unit, 1025 ... transmission control unit, 1031 ... Receiving part (A, B), 1032 ... Buffer part (A, B), 1033 ... System clock oscillation part, 1034 ... Content reading part (A, B), 1035 ... Reproduction part, 1036 ... Transmission control part, 1037 ... Input Part, 1038... Switching part

Claims (3)

A content distribution server storing content data including a plurality of tracks having a plurality of content segment data and a content reproduction request to the content distribution server via the network, and responding to the reproduction request A content distribution method in a content distribution system comprising a content receiving device that receives and reproduces the content fragment data transmitted from a content distribution server via a network,
The content data stored in the content distribution server is content data having a file name and a plurality of tracks, and the track data of each track includes a track number, a track timestamp frequency, the total number of frames, and a pitch transposition. Track key information indicating the degree of sound, track duet information and frame number indicating presence / absence of partner audio, time information indicating playback timing, and content segment data, and the data of each track in the same frame number Frames have the same time information;
The content receiving device requesting a session description that is prior information for establishing a session between the content receiving device and the content distribution server and receiving the content fragment data;
The content distribution server transmitting the session description including at least the track number, the track key information, and the track duet information for each track;
When the content receiving device receives the session description, the track number information and the track duet information are associated with the track number and displayed.
The content receiving device transmitting a content reproduction request message together with a file name of the content data, the track number, and a content reproduction range;
When the content distribution server receives the playback request message, the content distribution server acquires and transmits the content segment data of the designated track according to the file name, the track number, and the playback range of the content;
Have
When switching between the tracks,
The content receiving device transmitting a track switching request message together with the track number;
When the content distribution server receives the switching request message, it should be played back in succession to the content segment data whose playback order is the last among the transmitted content segment data according to the track number. Obtaining and transmitting the content fragment data of the designated track having time information; and
A content distribution method characterized by comprising: In accordance with the playback request from the content receiving device, the content segment data requested to be played back is selected from the content data having a plurality of tracks having a plurality of content segment data, and the selected content segment data is transmitted via the network. A content distribution server for transmitting to the content receiving device,
Content data having a file name and a plurality of tracks, the track data of each track having a track number and a plurality of data frames, each data frame including a track number, a track timestamp frequency, and a total frame Number, track key information indicating the degree of transposition of pitch, track duet information and frame number indicating presence / absence of partner sound, time information indicating playback timing, and content segment data, and the same frame number Content storage means for storing content data in which the data frame of each track has the same time information;
When a session description that is prior information for establishing a session between the content receiving device and the content distribution server and receiving the content fragment data is requested, at least the track number for each track, Transmitting the session description including the track key information and the track duet information;
A transmission control means for receiving a content reproduction request message together with a file name of the content data, the track number, and a reproduction range of the content, and a track switching request message together with the track number;
When the transmission control means receives the playback request message, the content control unit obtains the content fragment data of the designated track from the content storage means according to the file name, the track number, and the playback range of the content, When the transmission control means receives the switching request message, time information that should be continuously reproduced in accordance with the track number, in the content segment data whose reproduction order is the last among the transmitted content segment data Content acquisition means for acquiring the content fragment data of the designated track having the content storage means from the content storage means;
A content distribution server comprising: A content distribution server that stores content data including a plurality of tracks having a plurality of content segment data is requested to reproduce the content via the network, and the network is connected from the content distribution server according to the reproduction request. A content receiving device for receiving and playing back content fragment data transmitted via
The content data stored in the content distribution server is content data having a file name and a plurality of tracks, and the track data of each track has a track number and a plurality of data frames, and each data frame Has a track number and a plurality of data frames, and each data frame has a track number, a track time stamp frequency, the total number of frames, track key information indicating the degree of pitch transposition, and a track indicating the presence or absence of partner audio. It has duet information and frame number, time information indicating the timing of reproduction, and content fragment data, and the data frame of each track in the same frame number has the same time information,
For the content distribution server,
Requesting a session description which is prior information for establishing a session between the content receiving device and the content distribution server and receiving the content fragment data;
When the session description is received, the track key information and the track duet information are displayed in association with the track number,
When playing back content, a content playback request message is transmitted together with the file name of the content data, the track number, and the playback range of the content. When switching tracks, a track switching request message is sent together with the track number. A content receiving apparatus comprising transmission control means for transmitting a message.

Images