JP2006135569A - Data distributing method and information processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data distributing method and an information processing apparatus that enable streaming distribution regardless of whether a file includes control information for distribution. <P>SOLUTION: The processing method of content data comprises: analyzing the structure of the content data to be distributed (step S202); generating distribution information on the basis of the analysis result (step S203) when it is decided that the content data include no description for distribution control; and distributing the content data to a network on the basis of the generated distribution control information (step S204). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a data distribution method, apparatus, and program suitable for processing multimedia files. In particular, it is suitable for data distribution processing of multimedia files described in MP4 file format or similar file format, including communication control information for streaming distribution and object descriptor information that describes the relationship between content data. is there.

  With the progress of video and audio compression encoding technology, various compression encoding formats have been standardized as standards and are currently used. In addition, so-called multimedia content in which a plurality of types of content data are associated with each other and handled as a single content, rather than handling content data such as video and audio alone, is also generally used. A typical example of such a multimedia content compression coding format is the Moving Picture Expert Group (MPEG), which is a video / audio coding format established by ISO (International Organization for Standardization).

  In the above-mentioned ISO, a general-purpose file format called “ISO Base Media file format” is standardized in order to record moving image / audio content data such as MPEG (ISO / IEC 14496-12, Non-Patent Document 1). reference). Further, based on this file format, a file format extended to record data of a specific encoding format is also standardized. The “MP4 file format” (ISO / IEC 14496-14; see Non-Patent Document 2), which is a standard file format for recording MPEG-4 video / audio encoded data, is a typical example.

  In devices such as digital cameras and mobile phones, video / audio data is encoded in MPEG-4 format, and the encoded data is multiplexed for recording in a file, and the multiplexed encoded data is separated. Extracted from the file and played. In order to multiplex and demultiplex such encoded data, an increasing number of cases adopt the above-mentioned MP4 file format.

  On the other hand, with the development of the Internet environment and the communication environment, usage forms such as multimedia content recorded in the file format as described above are not only reproduced within the device but also streamed via a network are increasing. . Therefore, in the above-mentioned ISO Base Media file format, an additional data structure called “hint track” for supporting streaming distribution of moving image / audio data in the file is defined.

<Hint track>
The data unit used in the streaming distribution process is a “packet” of the communication protocol used, and does not necessarily match the data unit of the moving image / audio encoded data. Therefore, at the time of distribution, it is necessary to divide and combine the moving image / audio encoded data into units that can be transmitted as communication protocol packets. Further, additional information for communication control (for example, protocol header information, hereinafter referred to as communication control information) may need to be added to the communication packet.

  In the hint track, an instruction for performing processing such as division / combination of these content data and addition of communication control information is described. The instruction information as described above is recorded as a hint track in a file for streaming distribution, and a distribution apparatus (hereinafter referred to as a streaming server) for streaming extracts a hint track corresponding to the distribution protocol from the file and is described. By performing packetization processing in accordance with the instruction, a packet is generated in a form conforming to the communication protocol.

  In general, the packetization method differs depending on the communication protocol used and the encoding format of the content data to be distributed. For example, when RTP (see Non-Patent Document 4) is used as a communication protocol and MPEG-4 is used as an encoding format, packetization methods such as RFC3640 (see Non-Patent Document 5) and RFC3016 (see Non-Patent Document 6) are standardized. ing. In the above-mentioned ISO Base Media file format standard, a hint track format corresponding to the case where RTP is used as a communication protocol is standardized.

  The concept and usage of the hint track are also mentioned in Patent Document 1 and Patent Document 2.

<Object descriptor>
In addition to encoding individual content data such as video and audio, MPEG-4 treats each content data as an object (object) that constitutes a scene and encodes it in units of objects. It also has the feature of being. Therefore, in MPEG-4, in order to associate the attribute information of each object such as video and audio with the encoded data corresponding to each object, information for describing the relationship between both is called an “object descriptor”. Used. By this object descriptor, the attribute of the content data can be handled correctly. This object descriptor is one part of MPEG-4, and is standardized by MPEG-4 Systems (see Non-Patent Document 3) that defines a standard for control systems such as synchronization control.

  In MPEG-4, content data is independently encoded as a data stream called an elementary stream (hereinafter referred to as ES). The content data handled as ES is not limited to video and audio, but also includes the object descriptors described above. In order to uniquely identify this ES, each ES is assigned a unique identifier called an elementary stream ID (described as ES_ID). In MPEG-4, content data included in a specific ES can be referred to by using ES_ID. That is, by describing this ES_ID in the object descriptor, association between the object descriptor and the content data can be realized.

  The object descriptor has two types of data structures: an InitialObjectDescriptor (hereinafter referred to as IOD) used as initial data when playback is started and an ObjectDescriptor (hereinafter referred to as OD) used after the playback operation is started. Is defined. Normally, OD is recorded as ES data, and IOD is treated as independent data different from ES.

  The IOD and OD include a data structure called ES_Descriptor (hereinafter referred to as ESD) for describing the attribute information of the referenced ES. The ESD includes ES_ID together with the type of stream encoding format, bit rate, and data called DecoderSpecificInfo for initializing the decoder. Therefore, ES can be referenced from both IOD and OD.

  In a typical MPEG-4 content, as shown in FIG. 1, an IOD including an ES_ID indicating an ES of an OD is first processed. First, OD data is referenced from ES_ID described in IOD. Similarly, the process is performed by referring to the content data from the ES_ID described in the OD.

<BIFS>
Furthermore, in MPEG-4, a concept called scene description is introduced in order to compose a single scene by synthesizing objects. The scene description is information describing a spatial attribute indicating the position of the object in the scene and a temporal attribute indicating the timing of appearance and disappearance of the object separately from the content data of the object such as video and audio. Used to synthesize scenes from In MPEG-4, scene description information is described as encoded data called BIFS (Binary Format for Scene). General MP4 file format content includes the IOD, OD, and BIFS data described above.

<ISMA>
The above-mentioned data such as hint track, OD, and BIFS are industry groups in which companies such as US Apple Computer and Philips are participating as specifications to define the communication means for streaming MPEG-4 content. It is also defined in the standard specification “Internet Streaming Media Alliance Implementation Specification” (see Non-Patent Document 7, hereinafter referred to as ISMA specification) defined by ISMA (Internet Streaming Media Alliance).

  An outline of how hint tracks, OD, and BIFS are used in this ISMA specification will be described with reference to FIG.

  The ISMA specification uses RTSP (see Non-Patent Document 8) and SDP (see Non-Patent Document 9) as session control protocols, as well as many video distribution methods, and streaming video / audio encoded data.・ The above-mentioned RTP is used as a protocol.

  The streaming server first sends session description information described in the SDP format to the receiving terminal using a protocol such as RTSP (21). At this time, IOD, OD, and BIFS information is sent as part of the session description information as text data encoded in Base64 format. That is, if this session description information is described in the SDP format in the file to be distributed, it is acquired from the file by the streaming server at the time of distribution and transmitted.

  After the session is established, the video / audio data stream information described in the SDP format is transmitted from the hint track recorded in the file in advance by RTSP, and the RTP session for streaming is established (22) ). Then, the video / audio data is divided and packetized on the basis of the contents of the hint track, and the distribution process is performed by the procedure of streaming to the receiving terminal on the established RTP session (23).

As described above, by recording the hint description and session description information including IOD, OD, and BIFS in advance in a file containing MPEG-4 content, the information is used by the streaming server for distribution processing. The
US Pat. No. 6,717,952 Special table 2004-505519 ISO / IEC 14496-12; “Information technology-Coding of audio-visual objects-Part 12: ISO base media file format”; ISO / IEC; 2004-01-23 ISO / IEC 14496-14; “Information technology-Coding of audio-visual objects-Part 14: MP4 file format”; ISO / IEC; 2003-11-24 ISO / IEC 14496-1; “Information technology-Coding of audio-visual objects-Part 1: Systems”; ISO / IEC; 2003-02-20 RFC1889; “RTP: A Transport Protocol for Real-Time Applications”; H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson; January 1996; IETF RFC3640; “RTP Payload Format for Transport of MPEG-4 Elementary Streams”; J.van der Meer, D.Mackie, V.Swaminathan, D.Singer, P.Gentric; November 2003; IETF RFC3016; “RTP Payload Format for MPEG-4 Audio / Visual Stream”; Y.Kikuchi, T.Nomura, S.Fukunaga, Y.Matsui, H.Kimata; November 2000; IETF “Internet Streaming Media Alliance Implementation Specification Version 1.0”; Internet Streaming Media Alliance; 2001-08-28 RFC2326; “Real Time Streaming Protocol (RTSP)”; H.schlzrinne, A.Rao, R.Lanphier; April.1998; IETF RFC2327; “SDP: Session Description Protocol”; M.Handley, V.Jacobson; April.1998; IETF 3GPP TS 26.244 “Technical Specification Group Services and System Aspects Transparent end-to-end packet switched streaming service (PSS); 3GPP file format (3GP) (Release 6)” 3rd Generation Partnership Project; 2003-02-28

  The above streaming server processing is based on the premise that streaming information such as hint description, session description information including IOD, OD, and BIFS is included in the file. In other words, in order to perform streaming, such information had to be recorded in advance in a file to be distributed.

  However, such streaming information is not always described in the file. In fact, in most cases, such streaming information is not added to a file created by an imaging device such as a digital camera or a camera-equipped mobile phone in order to suppress data redundancy.

  Therefore, as a preparation for performing streaming distribution, the content creator needs to add the streaming information to the file created by the imaging device or the recording device and then upload it to the streaming server.

  For this reason, the content creator has to deal with two types of files depending on whether or not streaming is performed, which causes a problem that the management of the files becomes complicated. In addition, additional work for adding streaming information to a file occurs, which increases the workload of the content creator, and the amount of data increases due to the addition of streaming information. Such a problem has occurred, and the work for content distribution has become complicated and inefficient.

  As a proposal for solving the above-described problem, there is a method in which streaming information such as a hint track is given regardless of whether or not streaming is performed in the file creation processing of the imaging device / recording device.

  However, according to this method, when streaming distribution is not performed, the added streaming information is left in the file as redundant data that is not used, and thus data efficiency is deteriorated. In addition, it is necessary to allocate processing capacity and memory resources for performing video recording processing that should be originally performed by the imaging device / recording device to the streaming information recording. Challenges will occur. Furthermore, even if this method is used, the above-described problem that it takes a long time to upload to the streaming server cannot be solved.

  Further, the information such as IOD, OD, and BIFS described above is not always held in a file in all file formats that can handle MPEG-4. For example, the 3GPP file format defined by 3GPP (3rd Generation Partnership Project) for 3G mobile phones (see Non-Patent Document 10) and the AMC file format used as a video recording format for mobile phones by KDDI Corporation Then, this information is not recorded in the file. Therefore, IOD, OD, and BIFS data cannot be sent to files in the above format in the format specified by the ISMA specification, so it was not possible to comply with the specification.

  The present invention has been made in view of the above problems, and a data distribution method and an information processing apparatus capable of performing streaming distribution regardless of whether or not a file includes control information for distribution. Is intended to provide.

In order to achieve the above object, a data processing method according to the present invention comprises:
A method of processing content data,
An analysis process for analyzing the structure of the content data to be distributed;
A generation step of generating distribution control information based on an analysis result of the analysis step when it is determined that a description for distribution control is not included in the content data according to an analysis result of the analysis step;
A distribution step of distributing the content data to a network based on the distribution control information generated in the generation step.

An information processing apparatus according to the present invention for achieving the above object is
Storage means for storing content data in memory;
Analysis means for analyzing the structure of content data to be distributed among the content data stored in the memory;
Generating means for generating distribution control information based on the analysis result of the analysis means when it is determined by the analysis result of the analysis means that the description for distribution control is not included in the content data;
Distribution means for distributing the content data to a network based on the distribution control information generated by the generation means.

  According to the present invention, streaming distribution can be performed regardless of whether or not the file includes control information for distribution.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  In the following embodiment, the case where the content data is in the MP4 file format will be mainly described. However, the present invention can be applied not only to MP4 but also to a case using a similar file format and architecture. For example, ISO standards such as “Motion JPEG 2000 file format” (ISO / IEC 15444-3) and “AVC file format” (ISO / IEC 14496-15) are established as standards that use the same architecture as MP4. ing. The present invention can also be applied to these standards and standards adopting a file format and architecture similar to those defined in MP4, such as the 3GPP file format described above. In addition, any video encoding format and audio encoding format may be used as long as they are defined for use in the file format standard applied to the present embodiment. For example, when the 3GPP file format is applied as the file format, an encoding format such as H.263 as the video encoding format and AMR (Adaptive Multi-Rate) as the audio encoding format may be used.

  Further, the state “content data is described in the MP4 file format” described in this specification does not indicate that the substance of the data to be handled is a physical file. The present invention can also be applied to data transmitted via a network and data stored on a memory.

<First Embodiment>
FIG. 3 is a block diagram illustrating a configuration of a network camera that realizes multimedia content distribution processing according to the first embodiment. The network camera according to the first embodiment has a function of storing captured video as an MP4 file and a function of streaming the stored MP4 file via a network.

  In the network camera of FIG. 3, video information at the time of shooting is input from the camera 100 and passed to the video capture controller 104 via the video I / F 102. Similarly, audio information at the time of shooting is input from the microphone 101 and passed to the audio capture controller 105 via the audio I / F 103. The video capture controller 104 converts the input video information into a digital video signal and passes it as an input of the video encoder 106. Similarly, the audio capture controller 105 converts the input audio signal into a digital audio signal and passes it to the audio encoder 107.

  The video encoder 106 performs compression encoding on the input video signal to generate MPEG-4 video encoded data. Similarly, the audio encoder 107 compresses and encodes the input audio signal to generate MPEG-4 audio encoded data. The encoded data generated by the video encoder 106 and the audio encoder 107 are multiplexed into the MP4 file format by the file multiplexing controller 108 and output as a file. At this time, the file multiplexing controller 108 writes the MP4 file data to be output to the external storage 112 via the external storage I / F 111. In the present embodiment, the external storage 112 is assumed to be a non-volatile memory card such as an SD card, but a recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc) may be used, or a hard disk Or a storage device such as

  Further, the network camera of FIG. 3 includes a streaming controller 113 and a network I / F 114 in order to realize a streaming distribution function. The streaming controller 113 provides a function of streaming the contents of the MP4 file using the above-described RTP and RTSP. When the streaming controller 113 receives an MP4 file distribution request from the distribution destination terminal via the network I / F 114, the streaming controller 113 acquires the requested MP4 file from the external storage 112 via the external storage I / F 111. Then, an RTP packet is generated from the encoded data obtained by demultiplexing, and streaming distribution is performed by RTP to the distribution destination terminal via the network I / F 114.

  3 are connected so that they can communicate with each other via an internal bus 118, and the processing of each component is controlled by the system controller 115. In addition, fixed information of the camera system is stored in the system ROM 116, and information such as the system state is stored in the system RAM 117. However, large-capacity data such as video data and audio data is held in dedicated memories such as the video frame memory 109 and the audio frame memory 110. Data is exchanged between the components via the system RAM 117, the video frame memory 109, and the audio frame memory 110.

  FIG. 4 is a flowchart for explaining a schematic procedure of processing executed by the streaming controller 113 in the network camera of the first embodiment.

  In the flowchart of FIG. 4, first, in step S201, the streaming controller 113 acquires the contents of the MP4 file to be distributed. In the present embodiment, the streaming controller 113 acquires information regarding a file distribution request sent from the distribution destination terminal via the network I / F 114, and stores the requested file contents via the external storage I / F 111. 112. It is assumed that the file distribution request from the distribution destination terminal is transmitted using the above RTSP.

  Next, in step S202, the streaming controller 113 analyzes the internal structure of the MP4 file to be distributed, and extracts information necessary for subsequent processing such as distribution of encoded data obtained by demultiplexing. The extracted information is held on the system RAM 117 and managed by the streaming controller 113.

  Next, in step S203, streaming information is generated based on the information extracted in step S202 described above. Streaming information typically includes session description information necessary to establish a communication session with a delivery destination terminal, information necessary for generating a packet header such as a time stamp, and encoded data serving as a packet payload. Information for extracting from the MP4 file, division information for dividing the encoded data into packet sizes suitable for the distribution protocol and distribution conditions, or encoded data itself as a payload. In the case of this embodiment, it can be said that these streaming information is SDP information and RTP hint track information recorded in the MP4 file.

  In step S204, the streaming controller 113 establishes a communication session by distributing the session description information included in the streaming information generated by the above-described processing by RTSP, and then performs the RTP packetizing process based on the streaming information. To generate an RTP packet including the encoded data of the distribution file. The RTP packet generated in this way is transmitted over the network via the network I / F 114, and is stream-distributed by RTP to the distribution destination terminal. That is, in step S204, the session description information in the SDP format is transmitted as a packet of the RTSP protocol, and the encoded A / V data is transmitted as a packet of the RTP protocol. As described in the second embodiment, in the case of distribution according to the ISMA specification, the session description information is transmitted in a form including the contents defined by the ISMA specification.

  Next, in the first embodiment, the streaming information generation process performed in step S203 of FIG. 4 will be described in detail with reference to the flowchart of FIG. FIG. 5 is a flowchart for explaining a streaming information generation processing procedure in the first embodiment.

  First, in step S301, the streaming controller 113 refers to the analysis result of the contents of the MP4 file to be distributed. This analysis result is assumed to be held in the system (system RAM 117) in a state where it can be referred to in advance by the analysis processing in step S202 described above.

  Next, in step S302, it is checked whether or not it is necessary to generate streaming information in order to perform distribution processing of the MP4 file to be distributed. In the present embodiment, it is determined that the generation of streaming information is necessary when the streaming information used in the protocol for distribution, such as RTP hint track information and SDP information, is not described in the MP4 file. Accordingly, in step S302, it is confirmed whether the MP4 file to be distributed matches this condition, and if the generation of streaming information is necessary, the streaming information generation process in step S303 and subsequent steps is executed. If there is streaming information, the processing from step S303 to step S305 is skipped, and in step S306, the streaming information included in the MP4 file is transferred to the subsequent step S204 as it is to perform distribution processing. Note that the processing in this case is the same processing as a general streaming server.

  If it is necessary to generate streaming information, first, in step S303, the data structure in the MP4 file related to the data items constituting the streaming information is referred to (for specific examples of data items constituting the streaming information, The data items shown in FIGS. 6 and 7 will be described later as an example). Needless to say, in order to perform this processing, in step S202, the internal structure of the MP4 file to be distributed is analyzed in advance, and the result is stored in the system (system RAM 117) in a state that the streaming controller 113 can refer to. It is necessary.

  Next, in step S304, streaming information is set according to the contents of the data structure of the MP4 file referred to in step S303 (a specific example of setting will be described later). The set contents are held without being transferred to the subsequent step S306 until the setting of all the data items of the streaming information is completed.

  The processes in steps S303 and S304 described above are executed for all data items constituting the generated streaming information. That is, in step S305, it is determined whether or not the processing in steps S303 and S304 has been executed for all data items, and the processing in steps S303 and S304 is repeated until all items are processed.

  When the setting of all data items of the streaming information is completed, the process proceeds from step S305 to step S306, and the contents of the streaming information obtained by executing the above steps S303 to S305 are transferred to the subsequent step S204.

  Through the above procedure, streaming information used for distribution in the network camera of this embodiment can be obtained.

  Next, a more detailed embodiment will be described with a specific example. As a specific example, details of processing performed in steps S303 and S304 will be described in the case of streaming delivery of MPEG-4 Video moving image data included in content data to be processed using RTSP and RTP. In this example, in steps S303 and S304, the video / audio data stream information described in the SDP format is generated as streaming information in RTSP communication, and the RTP packet in the payload format specified by RFC3016 is generated as streaming information in RTP communication. The In the following description, how these two types of information are generated will be described step by step.

[Generation of video / audio data stream information]
First, a procedure for generating video / audio data stream information in the SDP format transmitted by RTSP will be described.

  FIG. 6 is a diagram showing a typical example of video / audio data stream information described in the SDP format. The contents of the session description shown in FIG. 6 are described according to the specification shown in RFC3016. This information is usually described in the hint track of the MP4 file as data included in the hint track for the video / audio data to be distributed. Note that the information configuration shown in FIG. 6 is an example, and the data, format, and order used in the actual distribution processing do not necessarily match the contents shown in this figure. Therefore, in this case, the generation procedure is changed so that data is output in accordance with the actual distribution processing specifications.

  First, the media information shown in the first line (61) of FIG. 6 is set. As this information, “video” indicating that the video is handled as a media type, the port number of the RTP session used in the subsequent streaming process, and a value “RTP / AVP 96” indicating the payload format of MPEG-4 Video in RFC3016 ”Is set.

  Next, the bandwidth information shown in the second row (62) of FIG. 6 is set. “AS” is an abbreviation for Application-Specific. That is, this information depends on the application and the environment, and an arbitrary content is set.

  Next, the media subtype and time stamp resolution shown in the third line (63) of FIG. 6 are set. As the media subtype, “MP4V-ES” indicating MPEG-4 Video in RFC3016 is set. The time stamp resolution is set to 90000 (90 kHz) unless otherwise specified.

  Next, a track identifier for performing an operation on the hint track in the ISMA specification shown in the fourth line (64) of FIG. 6 is set. However, since there is no hint track in this example, “trackID = 5” is set as an identifier indicating a virtual hint track for the video track holding the data of the video stream to be distributed. This content indicates that “the track having the ID No. 5 is a hint track of the video stream to be distributed”. However, since such a track does not actually exist, when “trackID = 5” is designated, it behaves so as to perform an operation on a virtually associated video track. The value of this ID does not overlap with the IDs of other existing tracks and virtual tracks, and is arbitrary within the range permitted by the specification as long as the relationship between tracks can be managed uniquely. A value can be used. Note that the content of this line is essential for a device that conforms to the ISMA specification described in the second embodiment to be described later, and therefore, when the second embodiment is implemented, it must be set according to this description.

  Next, the media format type and profile level, and decoder setting information shown in the fifth line (65) of FIG. 6 are set. The format type specifies a value 96 indicating the MPEG-4 Video format in RFC3016. The profile level (profile-level-id) is the profile level value described in the DecoderSpecificInfo included in the ESD of the video track in the input file, and the decoder setting information (config) is the entire DecoderSpecificInfo Is converted into hexadecimal text and set.

  Next, the elementary stream ID of the MPEG-4 Video stream in the ISMA specification shown in the sixth line (66) of FIG. 6 is set. This elementary stream ID is the ISMA specification if the ES_ID value described in the ESD of the video track in the input file is acquired and set, or if it conforms to the ISMA specification described later in the second embodiment. To set a fixed value (201 in this case) set in advance for the video stream. Note that the content of this line is essential for a device that conforms to the ISMA specification described in the second embodiment to be described later, and therefore, when the second embodiment is implemented, it must be set according to this description.

  Data set in accordance with the above procedure is shaped into the format shown in FIG. 6 and is generated as session description information constituting streaming information.

[Generate RTP packet]
Next, a procedure for generating a packet of the format indicated by RFC3016 transmitted by RTP will be described.

  FIG. 7 is a diagram showing a layout of each item of a packet defined by RFC3016. The leftmost column is a column indicating the byte offset N of each item in units of 4 bytes, and the upper row is a row indicating an offset in units of bytes with respect to the byte offset N. That is, for example, the item “Timestamp” indicates that the packet is arranged at the fifth byte from the head of the packet (N = 4 and arranged from N + 1). In this embodiment, it is assumed that the extension header, SSRC, and CSRC are not used.

  First, 0x02 is set in the item Version (V). This item indicates the protocol version of RTP, because 2 is currently used. The next item Padding Flag (P) is set at the end of a series of processing and is not set here. The item Extension Flag (X) indicates whether or not an extension header is added to the RTP header. In this embodiment, it is assumed that no extension header is used, so 0 is specified. The item CCRC Count (CC) indicates the number of subsequent CSRC entries. In this embodiment, it is assumed that CSRC is not used, so 0 is specified.

  The next item Marker Bit (M) is set at the end of the series of processing and is not set here. The item Payload Type (PT) indicates the payload type of the packet. In the present embodiment, when MP4VSample Entry (“mp4v”) which is a data structure indicating that MPEG-4 Video stream data is stored is present in the input data, or this MP4VSample Entry is configured. When the value 0x20 indicating MPEG-4 Visual is set in the item objectTypeIndication included in the ES_Descriptor, a value 96 indicating the MPEG-4 Video payload format in RFC3016 is set in the item PT.

  The item “Sequence Number” is a sequential number that starts from a certain random value and is incremented every time an RTP packet is transmitted. An arbitrary initial value is selected at the start of the RTP session. In the present embodiment, the initial value of the Sequence Number selected at the start of the RTP session is incremented each time an RTP packet is transmitted.

  The item Timestamp is a time stamp indicating the composite time. In the present embodiment, when CompositionTimeToSampleBox (“ctts”), which is a data structure describing the composite time, exists in the input MP4 file, the composite time of the transmission target frame is acquired from this, and set in Timestamp. However, if CompositionTimeToSampleBox does not exist in the input MP4 file, the decoding time of the transmission target frame is acquired from TimeToSampleBox (“stts”), which is a data structure describing the decoding time, and set as the composite time instead. When the transmission target frame is divided into a plurality of packets, the same Timestamp value as that of the first packet is applied to the second and subsequent packets indicating the same frame.

  The item SSRC (Syncronization Source Identifier) is an identifier assigned so that a plurality of streams handled in synchronization share the same parameter. In this embodiment, it is assumed that SSRC is not used, so 0 is specified.

  The item CSRC (Contributing Source Identifiers) is an identifier indicating the provider of the stream after processing when the stream is transformed or edited. In the present embodiment, since it is assumed that CSRC is not used (CC = 0), CSRC is not set.

  Following the above packet header items, the frame data to be transmitted is set as payload data. In this embodiment, it is assumed that the frame data is divided every 1488 bytes and used as payload data.

  Note that 1488 bytes, which is the frame data division unit, is a value set on the assumption that IEEE 802.3 (Ethernet (registered trademark)), which is mainly used in a LAN environment, is used as a transmission medium. FIG. 8 is a diagram illustrating a layout of data transmission units in Ethernet (registered trademark). In the figure, the size of a data block that can be transmitted by Ethernet (registered trademark) is 48 to 1500 bytes. That is, the aforementioned 1488 bytes is obtained by subtracting 12 bytes, which is the size of the RTP header described in FIG. 7, from 1500 bytes, which is the maximum MTU (Maximum Transmission Unit Size) when Ethernet (registered trademark) is used. This is a value indicating the maximum payload size when an RTP packet is transmitted on Ethernet (registered trademark).

  As described above, when the size of the payload data generated by dividing the frame data is less than a multiple of 4 bytes, a padding bit is added to the end of the payload, and the header item Padding Bit (P ) Is set to 1. For the last packet generated from one frame data, 1 is set in the header item Marker Bit (M).

  In the above procedure, the transmission packet information is generated in a form in which the divided encoded data is included in the packet as a payload. However, it is not necessary to generate the packet information in a form including the contents of the encoded data. Rather, the packet information may be composed of a set of reference information such as an offset position and a size for acquiring the encoded data set as the packet header information and payload. In that case, in subsequent step S204, it is necessary to refer to the encoded data from the MP4 file based on the reference information of the payload included in the packet information to acquire the data. The related information for referring to the encoded data set as the payload is described in the MP4 file as a track reference described later.

  According to the procedure as described above, information necessary for generating a transmission packet is generated in step S203.

  Through the above procedure, the session description information and the transmission packet generated in step S203 are passed to step S204 as streaming information. In step S204, the information is transmitted via the network I / F 114 according to the RTSP and RTP transmission procedures. Sent.

  As described above, the network camera shown in the present embodiment sequentially generates streaming information necessary for distribution and performs streaming distribution even when distribution of an MP4 file not including a hint track or session description information is requested. Can do.

Second Embodiment
Next, as a second embodiment, a network camera that complies with the communication method defined by the aforementioned ISMA specification will be described as an example.

In the ISMA specification, as described in FIG. 2, it is defined that IOD, OD, and BIFS data are transmitted when a communication session is started by RTSP. As shown in FIG. 9, these data entities are described as data on a line starting with “a = mpeg4-iod:” in the RTSP header item described in the SDP format. The data of this line is expressed as URL (Uniform Resource Locator), and it is specified that IOD data encoded in Base64 format, which is one of the text encoding formats of binary data, is set at the end of the URL. is doing. In addition, OD and BIFS stream data are encoded in the same URL format as IOD and embedded in IOD data.In other words, MP4 is used to support communication processing in the format specified by the ISMA specification. Processing such as encoding IOD, OD, and BIFS data contained in the file into the URL format specified by the ISMA specification and sending it as an RTSP header in the SDP format at the start of the RTSP session is required. In the present embodiment, details of processing performed to generate session description information including such IOD, OD, and BIFS data will be described.

[Generate session description information including IOD, OD, and BIFS data]
When performing distribution according to the ISMA specification, it is checked in step S302 in FIG. 5 whether the content data to be distributed satisfies the requirements defined in the ISMA specification. For example, since a case in which a plurality of video / audio streams exist is out of the requirement, control is performed such as canceling the subsequent processing as an error or selecting and using only a stream that meets the requirement. Note that details of the requirements are defined in the ISMA specification document, and will not be described in this specification.

  Thereafter, the session description information shown in FIG. 9 is generated by the processing of steps S303 and S304. Note that the information configuration shown in FIG. 9 is an example, and the data, format, and order used in the actual distribution process do not necessarily match the contents shown in FIG. Therefore, in this case, the generation procedure is changed so that data is output in accordance with the actual distribution processing specifications.

  First, the bandwidth information shown in the first line (91) of FIG. 9 is set. This information depends on the application and the environment, and arbitrary contents are set.

  Next, as shown in the second line (92) of FIG. 9, a character string “isma-compliance: 1,1.0,1” indicating that the ISMA specification is compliant is set. Next, IOD data including OD and BIFS encoded in the format shown in the ISMA specification in the format shown in the third line (93) of FIG. 9 is set.

  However, since the IOD and OD data included in the MP4 file has a different structure from the IOD and OD data defined in the ISMA specifications, the data recorded in the MP4 file is used as it is. Cannot be done, and data structure conversion processing is required. Prior to explaining the processing, the structure of the IOD and OD data will be explained.

[Data structure of IOD and OD]
As shown in FIG. 1, the IOD and OD defined by the ISMA specification describe the relationship with the elementary stream to be described using ES_ID which is an identifier for uniquely identifying the elementary stream. In the case of FIG. 1, it is shown that the ES_ID of the OD stream is described in the IOD, and the ES_ID of the video or audio stream is described in the OD.

  However, the data structure of the IOD and OD recorded in the MP4 file is slightly different from that shown in FIG. The data structure of IOD and OD when recorded in an MP4 file will be described with reference to FIG.

  As shown in FIG. 10, data recorded in an MP4 file is structured and described by a so-called nested data structure called a “Box”. The content information is structured so as to include a box called “track” (trak) corresponding to each elementary stream in a box called “movie” (moov) indicating the whole content. The encoded data of the elementary stream corresponding to the track is stored in a box called “media data” (mdat).

  In order to be able to uniquely identify a track, each track is assigned an identifier called “Track ID” (Track ID), and the track ID is described in a box called “Track Reference” (tref), It is possible to define relationships between multiple tracks. In order to be able to define a plurality of relationships having different meanings, the track reference can have type information indicating the meaning of the relationship. The related information of the stream referenced from the OD is described as a track reference having the type mpod. The related information of the track including the encoded data referred from the hint track is described as a track reference having a type of hint.

  As shown in FIG. 10, in the case of the MP4 file format, the elementary stream is referred to using this track reference. Therefore, the related description method is clearly different from that in FIG.

  First, the IOD has no ESD, only the track ID of the track corresponding to the referenced ES. The ESD originally included in the IOD is recorded as data of a reference destination track as data independent of the IOD.

  Also, OD has no ESD, and only numbers starting from 1 indicating the index of data included in mpod type track references are recorded. The track ID of the track corresponding to the ES to be referenced is described in the data at the position indicated by the index number in the track reference data of the type indicating the relationship between OD and ES, and the ES referenced by the OD is described by this track ID. Is identified. Similar to the IOD, the ESD originally included in the OD is recorded as data independent of the OD in the reference destination track.

  In this way, in the case of MP4 file format, the data format corresponding to IOD, OD, and ESD is different from the usual, so the tag value assigned to each also has a special value for MP4 file format. used. As shown in FIG. 10, 0x10 (MP4_IOD_Tag) is used for the data corresponding to the IOD, and 0x0E (ES_ID_IncTag) is used for the data replacing the ESD included therein. In addition, 0x11 (MP4_OD_Tag) is used for data corresponding to OD, and 0x0F (ES_ID_RefTag) is used for data replacing ESD contained therein.

  Because there are differences in the IOD and OD description formats as described above, to generate IOD and OD data according to the ISMA specifications, the MP4 file box structure is analyzed and the reference relationship of each track is correctly reflected. IOD and OD data must be generated in a form that includes the ES_ID and the ESD of each track.

  Hereinafter, the conversion process of the IOD data structure in the format stored in the MP4 file will be described with reference to the flowchart of FIG. Note that the flowchart of FIG. 11 explains the IOD setting process in step S304 executed by the streaming controller 113 in more detail.

[IOD conversion processing]
First, in step S401, it is confirmed whether or not IOD data is included in the file. Normally, the MP4 file contains IOD data, but if not, the process proceeds to step S402 to execute processing for creating default IOD data and skip subsequent processing. Details of the default IOD generation processing in step S402 will be described later.

  When IOD data is included in the file, the process proceeds to step S403, and IOD binary data in a format stored in the MP4 file is acquired. It is assumed that this IOD data is held in a state that can be acquired by the streaming controller 113 (held in the system RAM 117) as a result obtained by analyzing the MP4 file in step S202 in advance.

  Next, in step S404, ES_ID_IncTag in the IOD is replaced with ES_DescrTag. At the same time, the ESD data is acquired from the “esds” box of the track referred to by the track ID following ES_ID_IncTag, and set at the position indicated by ES_DescrTag in the IOD. Referring to FIG. 12 as an example, in the MP4 file, the track 502 is designated by the Track ID in the IOD 501. ES_ID_IncTag indicating this Track ID is replaced with ES_DescrTag indicating ESD. Then, the ESD 504 in the “esds” box 503 in the designated track 502 is acquired, and set to the position of the replaced ES_DescrTag. In this way, it is converted into an IOD 501 ′ including the ESD 504.

  In step S405, it is checked whether ESD setting processing has been performed for all elementary streams referenced from the IOD. If all the elementary streams have not been processed, the process returns to step S404 to process the unprocessed elementary streams.

  With the conversion process as described above, IOD data in a format stored in an MP4 file can be appropriately converted.

[OD conversion processing]
The procedure for converting MP4 file format OD data will be described with reference to FIG. Note that the flowchart in FIG. 13 explains the OD setting process performed in step S304 by the streaming controller 113 in more detail.

  Unlike IOD, OD data is recorded in an MP4 file as elementary stream data. That is, OD data is handled as a “sample” that is a unit of media data, as in other data formats such as video and audio. Therefore, in order to process the OD data, an analysis process must be performed in step S202 so that the sample data of the OD stream can be referred to as it is. Furthermore, in order to acquire OD data, information for determining whether or not the track from which sample data is to be acquired is a track of OD data is necessary. Whether or not the track is a track of OD data can be determined based on whether or not the streamType of DecoderConfigDescriptor included in the ESD of the track is 0x01 (ObjectDescriptorStream). Therefore, it is necessary to store the streamType of the track in the system RAM 117 in a state where it can be referred to in the process of step S202.

  In the following OD conversion processing, the content of the streamType of the track is confirmed. If streamType = 0x01, it is determined that the track is an OD stream, and the data conversion processing of the track is performed.

  Like the IOD, the ESD included in the OD is represented and recorded as a separate tag and reference format within the MP4 file. Specifically, the ESD included in the OD is held as ES_ID_Ref that holds an index of “mpod” track reference in the MP4 file. Therefore, conversion of OD data in MP4 file format can be performed with the same processing procedure as IOD conversion.

  First, in step S601, it is confirmed whether or not OD data is included in the file. The presence / absence of OD data can be confirmed by checking whether the track has a track type stream type of 0x01 in the ESD. Normally, OD data is included in the MP4 file, but if it is not included, default IOD data creation processing is executed in step S402, and default OD data is created together with the IOD. Details of the default IOD generation processing in step S402 will be described later.

  If there is OD data, in step S602, OD data in a format stored in the MP4 file is acquired. In step S603, ES_ID_RefTag in the OD data is replaced with ES_DescrTag. At the same time, ESD data is acquired from the “esds” box of the track referenced by the track ID at the index number position following ES_ID_RefTag in the “mpod” type track reference data included in the track from which the OD data was acquired, and OD Set to the position indicated by ES_DescrTag.

  For example, in the case of FIG. 14, when the streamType of DecoderConfigDescriptor is 0x01 in the track 702 and the elementary stream corresponding to the track 702 is OD701, the area 710 indicated by ES_ID_RefTag in the OD701 is replaced with ES_DescrTag. Then, the track ID 706 indicated by the index 711 in the area 710 is obtained from the “mpod” type track reference 705 included in the OD track, and the ESD 709 is obtained from the esds box 708 of the OD reference track 707 indicated by the track ID 706. To do. Then, OD data 701 ′ in which the ESD 709 is incorporated in the ES_DescrTag portion is generated.

  In step S604, it is checked whether ESD setting processing has been performed for all elementary streams referred to by the OD. If there is an unprocessed elementary stream, the process returns to step S603.

[Conversion of IOD data to ISMA format]
In the case of IOD data used for distribution in the ISMA specification, it is necessary to convert the IOD data and OD data in the format stored in the MP4 file into the format specified in the ISMA specification by the above processing.

  The difference in data structure between the IOD and OD converted by the above processing and the ISMA specification IOD and OD is shown in FIG. In the case of the format obtained by the above conversion, the IOD 801 in FIG. 15 typically includes an ESD 802 corresponding to the OD stream 804 and an ESD 803 corresponding to the BIFS stream 805. However, the stream data itself of the OD stream 804 and the BIFS stream 805 exists as data independent of the IOD 801.

  On the other hand, in the case of the ISMA format, the IOD format is a format such as IOD 801 ′ in which OD data 804 and BIFS data 805 are incorporated. Therefore, in order to create ISMA-compliant streaming information, the IOD obtained by the above processing must be further converted into the ISMA format. The procedure of the conversion process will be described with reference to the flowchart of FIG.

  First, in step S901, IOD data is acquired from the MP4 file and converted according to the procedure shown in the flowchart of FIG. In step S902, OD data is acquired from the MP4 file and converted according to the procedure shown in FIG.

  Next, in step S903, processing for acquiring BIFS data from the MP4 file is performed. In order to acquire BIFS data, a track having an ESD whose streamType of DecoderConfigDescriptor is 0x03 (SceneDescriptionStream) is searched, and the data of the track is acquired. Since MP4 files usually have a BIFS track, this example does not consider the case where no BIFS track exists.

  In step S904, the OD data and BIFS data obtained in steps S902 and S903 are incorporated into the IOD data obtained in step S901 in a format defined by the ISMA specification. According to the ISMA specification, OD and BIFS data included in the IOD are text-encoded in Base64 format and preceded by “data: application / mpeg4-od-au; base64,” (for OD) or “data: application / mpeg4-bifs-au; base64, ”(in the case of BIFS), it is stored in the URL URL field of the ESD in the form of text. IOD, OD, and BIFS data are integrated and encoded in the above format, and IOD data in the form defined by the ISMA specification is generated.

  By performing the processing described above, system data such as IOD, OD, and BIFS contained in MP4 files can be converted into a format that conforms to the ISMA specification. As a result, even if the MP4 file does not contain ISMA-compliant IOD data, IOD data in a format compatible with ISMA-compliant streaming delivery can be generated and transmitted.

[Default IOD Generation Process (Step S402)]
As described above, the process of converting the IOD and OD data acquired from the MP4 file into the format of the ISMA specification has been described, but data such as IOD and OD is not necessarily included in any file. For example, a 3GPP file is different from an MP4 file and does not include data such as IOD, BIFS, and OD. Therefore, when handling a file such as a 3GPP file, it is necessary to create a default IOD without acquiring data such as IOD from the file.

  The default IOD data is prepared in advance as fixed data in the system ROM 116 of the network camera of this embodiment. Alternatively, fixed IOD data held in the system ROM 116 is used as a “template”, and IOD data obtained by rewriting some data items that vary depending on conditions such as bit rate, decode buffer size, and DecoderSpecificInfo among the contents of the IOD data in the system ROM 116. Processing may be performed such that data is stored in the system RAM 117 and used. The contents of the default IOD vary depending on the stream configuration of the file to be distributed (for example, a combination of streams such as video only, audio only, and video / audio). Therefore, it is necessary to prepare a plurality of default IODs for each configuration.

  Hereinafter, processing for generating a default IOD when a 3GPP file having no IOD data in the file will be described with reference to the flowchart of FIG. This flowchart describes in detail the processing in step S402 when a 3GPP file is handled in the processing of FIGS. 11 and 13 described above.

  First, in step S1001, it is confirmed whether the stream structure of the 3GPP file to be processed is video only, audio only, or a combination of video and audio. Next, in step S1002, default IOD data corresponding to the stream configuration of the 3GPP file confirmed in step S1001 is acquired from the system ROM 116 and stored in the system RAM 117. In step S1003, the default IOD data held in the system RAM 117 includes data that changes according to the encoding condition such as bit rate, decode buffer size, and DecoderSpecificInfo, and data contents that can be acquired from the file such as ES_ID. It is acquired from the file and rewritten by overwriting the default IOD data on the system RAM 117.

  By performing such processing, even if the file does not contain IOD, OD, or BIFS data, such as 3GPP files, IOD data defined by the ISMA specifications can be generated and used. It becomes possible.

  As described above, in the network camera of the present embodiment, streaming information can be created in step S203 through various processes as described above, and streaming distribution can be performed using the streaming information. As described in the above description, in particular, the network camera shown in the second embodiment is a content file that does not include session description information as shown in FIG. 9 for streaming delivery, or an IOD like a 3GPP file format. , OD and BIFS files can be streamed by generating session description information that conforms to the ISMA specification.

<Third Embodiment>
In the third embodiment, a network camera that outputs streaming information generated during distribution as a file will be described as an example.

  In the first embodiment, the streaming information generated in step S203 of FIG. 4 is discarded without being saved after being used in the streaming distribution process immediately after creation. In the third embodiment, when the output data is held and managed in the device in the form of a file so that it can be used even after the distribution process is completed, and distribution of a file that has been distributed before is requested again By reusing this output data, it is possible to distribute without having to perform streaming information generation processing one by one.

[Streaming information file output processing]
Hereinafter, processing for outputting and recording streaming information generated at the time of distribution as a file in the network camera of the third embodiment will be described with reference to the flowchart of FIG.

  The process of the flowchart in FIG. 18 is almost the same as that in FIG. 4, and the processes in steps S1101 to S1104 are the same as those in steps S201 to S204 in FIG. In the flowchart of FIG. 18, after the processing of steps S1101 to S1104 is performed, the output processing of the streaming information generated in step S1103 is performed in step S1105.

  In step S1105, the streaming controller 113 records the streaming information generated in step S1103 as a file in the external storage 112 via the external storage I / F 111. The streaming information only needs to be recorded so that the correspondence with the content data from which the streaming information is generated can be read out later. In this embodiment, streaming information is stored in the form of a file as shown in FIG. In this example, when the MP4 file to be distributed has a file name “MOV0001.MP4”, the information output in step S1105 is the base name “MOV0001” of the file name of the distribution target file. Output as a file name with an extension indicating the type. Further, information on each track of the file is output as a file name in which a base number “MOV0001” is added with a sequential number for identifying the track. For example, the movie session description information is “MOV0001.SDP”, and the hint track information given to the first track is streaming information with a file name “MOV0001-1.HNT”.

[Delivery process using streaming information file]
Next, the procedure of the distribution process using the streaming information recorded in the file by the above process will be described with reference to the flowchart of FIG.

  First, in step S1201, the streaming controller 113 receives the MP4 file distribution request sent from the distribution destination terminal via the network I / F 114. In the present embodiment, it is assumed that the MP4 file distribution request is sent from the distribution destination terminal in the format shown in FIG. 21 using RTSP. In FIG. 21, the MP4 file to be distributed is the part of “MOV0001.MP4” in the URL character string “rtsp: //192.168.0.1/MOV0001.MP4” specified by the DESCRIBE command (211) of RTSP. Indicated.

  In step S1202, the streaming controller 113 acquires a file name to be distributed included in the received distribution request. In the case of the example in FIG. 22, the part “MOV0001.MP4” is obtained by analyzing the URL.

  In step S1203, the streaming controller 113 checks whether the requested file exists. In the case of the example in FIG. 21, it is checked whether or not a file with the file name “MOV0001.MP4” exists in the external storage 112. If not, the error processing shown in step S1204 is performed, and the subsequent processing is not performed. The process for the distribution request is terminated.

  If the requested file exists, in step S1205, the streaming controller 113 analyzes the content of the requested MP4 file. However, in the case of the present embodiment, the analysis result is used for processing such as distribution of subsequent encoded data, but is not used for generating streaming information. Since this process is the same as the process in step S202 of FIG. 4 described above, description thereof is omitted here. Next, in step S1206, the streaming controller 113 checks whether there is a file in which streaming information corresponding to the requested MP4 file is recorded. In the case of the example in FIG. 21, since the requested file name is “MOV0001.MP4”, a streaming information file group having “MOV0001” as the base name of the file as shown in FIG. Check whether or not to do.

  If there is a streaming information file corresponding to the request file, the content of the streaming information file is acquired from the external storage 112 in step S1207. If there is no streaming information file, streaming information generation processing is performed in step S1208. Since this generation processing is the same as the processing in step S203 in FIG. 4 described above, description thereof is omitted here.

  In step S1209, the streaming controller 113 performs streaming distribution of the requested encoded data of the MP4 file via the network I / F 114 based on the streaming information acquired in step S1207 or generated in step S1208.

  In this way, when a file is generated in a form associated with a file to be distributed, when a file distribution is requested later, a streaming information file with the requested file base name exists. Can perform streaming distribution processing using the information of the file.

  In the present embodiment, the description format of the streaming information output to the file in step S1105 may be described in a standard file format such as the MP4 file format or may be described in any unique format.

  In this embodiment, the correlation between the MP4 file to be distributed and the streaming information file is defined by using the file name. However, for management to define the relationship between the MP4 file to be distributed and the streaming information file Data may be generated separately, or the streaming information itself may be additionally recorded in the distribution target MP4 file.

<Fourth embodiment>
As a fourth embodiment, a network camera that can change the method of generating streaming information statically or dynamically at the time of execution will be described as an example.

  In the present embodiment, it is assumed that the streaming controller 113 can change the generation policy and the generation form of the streaming information by providing the streaming controller 113 with the setting information regarding the generation method of the streaming control information.

  An example of the setting information given to the streaming controller 113 is shown in FIG. For example, “max_pdu_size” shown in the first line (221) of FIG. 22 indicates the maximum size of a packet generated by the streaming controller 113. The streaming controller 113 controls the division of the payload data based on this value. “Repeat” shown in the second line (222) indicates whether or not to transmit a packet a plurality of times in preparation for a transmission error. The streaming controller 113 sets a retransmission flag or the like in the packet based on this value. “Approval_packet_loss” in the third line (223) indicates an allowable packet loss rate (%). When the packet loss rate detected in the network I / F 114 exceeds the designated packet loss rate, The streaming controller 113 performs control for increasing resistance to errors, such as setting a packet retransmission flag.

  The setting information may be held as fixed data on the system ROM 116 or may be held as rewritable data in the external storage 112 or the system RAM 117. A plurality of setting information may be stored as a table, and the setting conditions may be selected adaptively according to the distribution conditions such as the mode at the time of distribution, the degree of network congestion, and the number of sessions. By doing in this way, the network camera shown by this embodiment can perform adaptive streaming delivery, even if streaming information is not compatible with the network environment to which streaming information is actually applied.

<Fifth Embodiment>
As a fifth embodiment, a network camera that generates streaming information at the time when an MP4 file to be distributed is registered will be described as an example.

  It is assumed that the network camera of this embodiment is used as a component of a content distribution system as shown in FIG. FIG. 23 shows a schematic configuration of a typical content distribution system. In FIG. 23, it is assumed that a content creator can register an MP4 file created from the file creation terminal 1303 as a file to be distributed by uploading it to the network camera 1301 serving as a distribution server. The file creation terminal 1303 may be any device that can transmit a content file to the network camera 1301, such as a personal computer, a mobile phone, a PDA, or another network camera. The communication path for uploading may be a public network such as the Internet, a wireless communication path such as Bluetooth, UWB, or IEEE 802.11, or an external bus for inter-device connection such as USB or IEEE1394. Good.

  The MP4 file registered in the network camera 1301 by the upload process described above can be viewed and requested for distribution from the distribution destination terminal 1302. That is, the network camera 1301 performs streaming delivery of the MP4 file in response to a request from the delivery destination terminal 1302.

  In the content distribution system of FIG. 23, when receiving the MP4 file uploaded from the file creation terminal 1303, the network camera 1301 performs a process of outputting streaming information to a file as shown in the third embodiment. In this embodiment, when an MP4 file is registered in the network camera 1301, only streaming information is generated, and the registered file is not distributed. After receiving the distribution request from the distribution destination terminal 1302, the network camera 1301 distributes the requested file using the streaming information generated when registering the MP4 file.

[Streaming information generation process during file registration]
Hereinafter, the procedure of the streaming information generation process performed at the time of file registration in the network camera of this embodiment will be described with reference to the flowchart of FIG.

  First, in step S1401, the network camera 1301 receives the MP4 file uploaded from the file creation terminal 1303 from the network I / F 114, and stores it in the external storage 112 via the external storage I / F 112. Note that this process need not be performed in conjunction with the streaming distribution process, and therefore may be performed not by the streaming controller 113 but by the system controller 115 or other control module (not shown). However, in this case, since the subsequent processing is executed by the streaming controller 113, the module that executed the processing of step S1401 needs to notify the streaming controller 113 that the file has been registered by some means.

  Next, the streaming controller 113 acquires the contents of the MP4 file registered in step S1401 from the external storage 112, analyzes the MP4 file acquired in step S1403, and generates streaming information of the MP4 file in step S1404. Note that the processing from step S1402 to step S1404 is the same as the processing from step S201 to step S203 in FIG.

  In step S1405, the streaming controller 113 outputs the generated streaming information as a file. As described in the third embodiment, this file is used for streaming distribution control when a distribution request for an MP4 file to be processed occurs. Note that the processing in step S1405 is the same as the processing in step S1105 in FIG.

  As described above, with the network camera shown in the fifth embodiment, even an MP4 file that has not been created for streaming in advance can be streamed without additional work such as adding control information for streaming to the MP4 file. At that time, since the data amount of the uploaded MP4 file can be small, the load and time required for the upload can be reduced.

<Sixth Embodiment>
FIG. 25 is a block diagram showing a configuration of a streaming server that provides a streaming distribution function of multimedia contents according to the sixth embodiment. The streaming server of the present embodiment has a function of acquiring an MP4 file from a connected external device and performing streaming distribution over a network.

  In FIG. 25, the streaming server 1501 includes a streaming controller 113, a network I / F 114, a system controller 115, a system ROM 116, and a system RAM 117, similarly to the network camera of the first embodiment. In addition, each component is connected by an internal bus 118, and processing of each unit is controlled by the system controller 115. In addition, the streaming server 1501 includes an external bus I / F 1502 that provides a data input / output function with a connected external device. In this embodiment, it is assumed that a digital video camera 1503 compatible with the USB mass storage class is connected as an external device.

  The streaming controller 113 acquires the MP4 file to be distributed from the digital video camera 1503 via the external bus I / F 1502, and, as in the first embodiment, converts the RTP packet from the encoded data obtained by demultiplexing. Generate and perform streaming delivery by RTP to the delivery destination terminal via the network I / F 114.

  Note that the processing procedure described in the first to fifth embodiments is applied to the external device as it is even in the device configuration in which the MP4 file as described above is stored in the external device (streaming server 1501). be able to.

<Seventh embodiment>
FIG. 26 is a block diagram showing a configuration of a streaming server according to the seventh embodiment. The streaming server 1601 of the present embodiment has a function of multiplexing video data and audio data transmitted from a connected imaging device and recording them as an MP4 file, and a function of streaming the recorded MP4 file via a network. With.

  In addition to the configuration of the streaming server 1501 of the sixth embodiment, the streaming server 1601 further includes a file multiplexing controller 108, an external storage I / F 111, and an external storage 112 similar to those shown in the network camera of the first embodiment. Shall be provided. Also, in this embodiment, it is assumed that a digital video camera 1602 that supports the USB video class is connected as an imaging device, and the video and audio data shot are MPEG-4 encoded and transmitted isochronously via USB. To do.

  When encoded data of video and audio captured by the digital video camera 1602 is input to the streaming server 1601 via the external bus I / F 1502, the file multiplexing controller 108 is similar to the network camera of the first embodiment. The encoded data is multiplexed into the MP4 file format, and the MP4 file is output to the external storage 112 via the external storage I / F 111.

  Furthermore, in the streaming controller 113, as in the first embodiment, the MP4 file is acquired from the external storage I / F 111, demultiplexed to generate an RTP packet, and is sent to the distribution destination terminal via the network I / F 114 by RTP. Stream it.

  As described above, the processing procedure described in the first to fifth embodiments can be applied in the same manner as it is even in a device configuration (streaming server 1601) that does not have a shooting function.

<Other embodiments>
The information generated by the present invention is processed into a format compatible with an arbitrary communication means (inter-device protocol) and data format, so that the present embodiment also applies to other devices via a transmission line such as a network. It is possible to provide an output. In other words, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.

  Another object of the present invention is to supply a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

  As described above, according to each of the above-described embodiments, the content creator only needs to handle a single file regardless of whether or not streaming is performed, so that file management can be simplified. Further, since the additional work for adding the streaming information to the file is not necessary, the work load of the content creator can be reduced. Also, since redundant data is not added, the time required for uploading to the streaming server can be shortened. Furthermore, for example, even files that do not contain IOD, OD, and BIFS data can be distributed in a format defined by the ISMA specification, and compatibility can be maintained.

It is a figure explaining the relevant definition of the media data using the object descriptor information in MPEG-4. It is a figure explaining the procedure of the streaming process in ISMA specification. It is a block diagram which shows the structure of the network camera which implement | achieves the process by 1st Embodiment. It is a flowchart which shows the procedure of the delivery process of MP4 file in 1st Embodiment. It is a flowchart explaining the streaming information generation process by 1st Embodiment. It is a figure which shows the example of the information of the video / audio data stream described in the SDP format. It is a figure which shows the layout of each item of the packet prescribed | regulated by RFC3016. It is a figure which shows the layout of the data transmission unit in Ethernet (trademark). It is a figure which shows the example of the RTSP session description information described in the SDP format. It is a figure explaining the related definition of the media data in an MP4 file. It is a flowchart which shows the IOD data conversion processing procedure of the format stored in an MP4 file. It is a figure explaining an IOD data conversion process. It is a flowchart which shows the OD data conversion processing procedure of the format stored in an MP4 file. It is a figure explaining an IOD data conversion process. It is a figure explaining the conversion process to the ISMA format of IOD data. It is a flowchart which shows the conversion processing procedure to the ISMA format of IOD data. It is a flowchart which shows a default IOD production | generation processing procedure. It is a flowchart which shows the process sequence of 3rd Embodiment. It is a figure which shows the structural example of a streaming information file. It is a flowchart which shows a streaming information output processing procedure. It is a figure which shows an example of the file request of RTSP. It is a figure which shows the example of the setting information used for the setting of streaming information production | generation operation | movement. It is a figure which shows schematic structure of a typical content delivery system. It is a flowchart which shows the process sequence of 5th Embodiment. It is a block diagram which shows the structure of the streaming server which implement | achieves the process by 6th Embodiment. It is a block diagram which shows the structure of the streaming server which implement | achieves the process by 7th Embodiment.

Claims (13)

A method of processing content data,
An analysis process for analyzing the structure of the content data to be distributed;
A generation step of generating distribution control information based on the analysis result of the analysis step when it is determined from the analysis result of the analysis step that the description for distribution control is not included in the content data;
A data processing method comprising: a distribution step of distributing the content data to a network based on the distribution control information generated in the generation step. The delivery control information includes information for establishing a session with a communication destination,
The distribution step includes establishing a session with the communication destination based on the distribution control information generated in the generation step, and distributing the content data to the communication destination via the network. The data processing method according to 1.   3. The method according to claim 1, wherein the generation step includes converting a description related to an object included in the content data into a description adapted to a distribution protocol in the distribution step based on the analysis result. Data processing method.   In the generation step, when a description corresponding to the distribution protocol does not exist in the content data, a description adapted to the distribution protocol is obtained by correcting a default description prepared in advance based on the analysis result. The data processing method according to claim 3, wherein:   5. The data distribution method according to claim 1, wherein the generation step generates the distribution control information when transmission of the multimedia content data is requested and distribution is performed. Further comprising a holding step of executing the generation step for the multimedia content in advance, and storing the generated distribution control information in a memory in a form capable of specifying the association with the multimedia content;
The distribution step reads out related distribution control information from the memory in accordance with an instruction to distribute the multimedia content, and distributes the multimedia content based on the read distribution control information. Item 5. A data processing method according to any one of Items 1 to 4.   The data distribution method according to claim 1, wherein the generation step changes a generation operation of the distribution control information based on setting information.   8. The data distribution method according to claim 1, wherein the distribution step generates a packet based on the content of the distribution control information and distributes the packet. Storage means for storing content data in memory;
Analysis means for analyzing the structure of content data to be distributed among the content data stored in the memory;
Generating means for generating distribution control information based on the analysis result of the analysis means when it is determined by the analysis result of the analysis means that the description for distribution control is not included in the content data;
An information processing apparatus comprising: distribution means for distributing the content data to a network based on the distribution control information generated by the generation means. And further comprising an acquisition means for acquiring image data and / or audio data obtained by the recording means and / or the recording means,
The information processing apparatus according to claim 9, wherein the storage unit stores image data and / or audio data obtained by the acquisition unit in a memory as content data. An interface for inputting image data and / or audio data from an external device;
The information processing apparatus according to claim 9, wherein the storage unit stores data input through the interface as content data in a memory.   A control program for causing a computer to execute the data processing method according to claim 1.   A storage medium storing a control program for causing a computer to execute the data processing method according to claim 1.

Images