JP2011060338A - Information processing device, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method for executing data multiplexing including image and voice data in parallel. <P>SOLUTION: Each of divided data such as image data and voice data are processed in parallel by a plurality of multiplexing parts to generate a plurality of pieces of multiplexed data as a plurality of sub clips. Furthermore, with respect to packets included in the multiplexed data, auxiliary information containing sequence numbers in the sub clips for each data kind and packet kind information. Thereafter, the auxiliary information is applied, with respect to the packets constituting the multiplexed data included in the sub clips, to calculate and set continuous count values which are the sequence numbers for each data kind. High-speed multiplexing can be enabled. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program. In particular, the present invention relates to an information processing apparatus, an information processing method, and a program for performing multiplexing processing of data recorded on an information recording medium such as a disk.

  In recent years, DVD (Digital Versatile Disc), Blu-ray Disc (registered trademark), and the like are widely used as data recording media (information recording media). These media are used for recording and reproducing various contents such as movies and music.

  Media types include read-only media such as DVD-ROM and BD-ROM, in which data is recorded in advance and new data cannot be written, DVD-RAM / R / RW, DVD + RW / + R, and BD-R / RE. There are rewritable media on which data can be written by the user.

  The read-only media is provided to users by recording various contents such as movies and music in a disc factory, for example. In addition, the rewritable media can be recorded by a user device such as a PC or a recorder after being purchased by a user and downloaded content or broadcast content via a network.

  Recording data according to a predetermined format is recorded on these discs. For example, in the BD-ROM standard, which is a ROM-type Blu-ray Disc (registered trademark) data recording format standard, an MPEG-2 transport stream (TS) is used to record video and audio on a disk as a single stream. The format is adopted. This BD-ROM standard is described in Non-Patent Document 1, for example.

  In order to generate a transport stream (TS) that is recording data for a BD-ROM, a process of multiplexing each stream such as video and audio (hereinafter referred to as elementary stream (ES)) is required. In this multiplexing process, it is necessary to order video and audio that need to be reproduced together in the reproduction process so that they are not separated from each other. For example, Patent Document 1 (Japanese Patent Laid-Open No. 8-116534) and Patent Document 2 (Japanese Patent Laid-Open No. 8-116266) are known as conventional techniques describing data stream multiplexing processing and ordering setting methods.

  In the current BD-ROM standard, the recording capacity of a BD-ROM disc is about 50 GB at the maximum. It takes a considerable amount of time to generate a transport stream equivalent to 50 GB, that is, to multiplex elementary streams such as video and audio. Therefore, in the authoring process, which is an editing process of recorded data on the BD-ROM disc, there is a problem of shortening the time required for the multiplexing process. In order to improve the authoring efficiency, it is important to shorten the multiplexing process.

  In general, multiplexing processing for generating a transport stream used as broadcast data is often performed in real time including encoding (and re-encoding). On the other hand, the multiplexing process of the transport stream for the BD-ROM disc is generally started in a state where all the elementary streams are encoded and prepared. This means that in the multiplexing process in the recording data generation for the BD-ROM, the transport stream can be divided and the multiplexing process can be performed in parallel.

  However, in order to perform parallel multiplexing processing, it is necessary that the divided streams can be multiplexed independently without interdependence. In the MPEG-2 transport stream, it is necessary to assign a continuous count value throughout the entire stream called a continuity count (continuity_counter) for each transport packet generated from each elementary stream.

  However, it is technically difficult to determine the value of the continuous counter in advance before starting the multiplexing processing of the divided packets of each stream. This is a major factor that hinders parallel multiplexing and makes it difficult to achieve high speed multiplexing processing.

JP-A-8-116534 JP-A-8-116266

ISO / IEC 13818-1 Information technology Generic coding of moving pictures and associated audio information: Systems

  As described above, in order to generate recording data for a BD-ROM disc, it is necessary to multiplex each stream (elementary stream) such as video and audio. Yes. An object of the present invention is to provide an information processing apparatus, an information processing method, and a program for realizing the multiplexing processing at high speed.

  For example, in one configuration example of the present invention, parallel multiplexing processing is performed in a state where the continuous count value (continuity_counter) is indeterminate, and multiplexing is performed by setting the continuous count value by applying auxiliary information after completion of the parallel multiplexing processing. Provided are an information processing apparatus, an information processing method, and a program that realize a shortening of the digitization process.

The first aspect of the present invention is:
A control unit that divides image data and audio data, which are recording data materials of the information recording medium, according to the number of multiplexing units that can be processed in parallel;
A plurality of multiplexing units that receive each of the divided data divided by the control unit, and multiplex the image packets storing the received image data and audio data and packets of different data types including audio packets in a mixed manner It has a plurality of multiplexing units that execute processing in parallel and generate a sub clip including multiplexed data that is a processing result,
Each of the plurality of multiplexing units is
During the sub-clip generation process, auxiliary information to be applied to the calculation of a continuous count value that is a packet sequence number for each data type for the packets constituting the plurality of multiplexed data generated by the plurality of multiplexing units is generated. It is in the information processing device.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, each of the plurality of multiplexing units includes an auxiliary sub-clip sequence number for each data type and a packet type information for a packet included in the multiplexed data. Generate information.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the information processing apparatus further includes a data type for stored packets of different data types included in the plurality of sub-clips generated by the plurality of multiplexing units. A count value setting unit configured to set a continuous count value which is another sequence number, and the count value setting unit applies the auxiliary information to configure a plurality of multiplexed data included in a plurality of sub-clips A continuous count value that is a sequence number for each data type is calculated for the packet.

Furthermore, in an embodiment of the information processing device of the present invention, the count value setting unit calculates the number of packets for each data type included in each sub clip by applying the auxiliary information, and is included in the auxiliary information. Add the total number of packets of the same data type of all preceding sub clips to the sequence number in the sub clip for each data type, and calculate the overall sequence number for each data type included in all the sub clips,
The processing for calculating the continuous count value for each data type is performed by applying the entire sequence number for each data type.

Furthermore, in one embodiment of the information processing apparatus of the present invention, the count value setting unit is configured to set the continuous count value as repeated data of 0 to 15, the overall sequence number is [q], and the continuous When the count value is [p],
p = qmod16
P calculated by the above calculation formula is calculated as the continuous count value.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, each of the plurality of multiplexing units includes an image packet storing image data, an audio packet storing audio data, and a system packet storing management information. A sub clip including multiplexed data arranged in a mixed manner is generated, and the count value setting unit applies the auxiliary information to form an image packet constituting a plurality of multiplexed data included in the plurality of sub clips, The continuous count value is calculated for each of the voice packet and the system packet.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the control unit, the plurality of multiplexing units, the count value setting unit, and the file server that stores the image data, audio data, and sub-clip are a network. Each of the plurality of multiplexing units acquires the image data and audio data from the file server via the network, and sends the generated sub clip and auxiliary information via the network. And the count value setting unit performs processing for acquiring the sub clip and auxiliary information from the file server via the network.

Furthermore, the second aspect of the present invention provides
An information processing method executed in an information processing apparatus,
A step of dividing the image data and audio data, which are the materials of the recording data of the information recording medium, according to the number of multiplexing units capable of parallel processing;
Each of a plurality of multiplexing units receives each of the divided data divided by the control unit;
Each of the plurality of multiplexing units executes, in parallel, a multiplexing process in which image packets storing received image data and audio data and packets of different data types including audio packets are mixed and arranged. A sub-clip generation step for generating a sub-clip including multiplexed data;
The sub-clip generation step includes
The information processing method includes a step of generating auxiliary information to be applied to calculation of a continuous count value that is a packet sequence number for each data type for packets constituting a plurality of multiplexed data generated by the plurality of multiplexing units. .

Furthermore, the third aspect of the present invention provides
A program for executing information processing in an information processing apparatus;
A step of causing the control unit to divide image data and audio data, which are recording data materials of the information recording medium, according to the number of multiplexing units capable of parallel processing;
Causing each of a plurality of multiplexing units to receive each of the divided data divided by the control unit;
Each of the plurality of multiplexing units executes in parallel a multiplexing process in which image packets storing received image data and audio data and packets of different data types including audio packets are mixed and arranged. A sub-clip generation step for generating a sub-clip including multiplexed data;
The sub-clip generation step includes
The program includes a step of generating auxiliary information to be applied to calculation of a continuous count value which is a packet sequence number for each data type for packets constituting a plurality of multiplexed data generated by the plurality of multiplexing units.

  The program of the present invention is, for example, a program that can be provided by a storage medium or a communication medium provided in a computer-readable format to an information processing apparatus or a computer system that can execute various program codes. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the information processing apparatus or the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to an embodiment of the present invention, there is provided an apparatus and method for executing data multiplexing processing including image and audio data in parallel. That is, each of the divided data of the image data and the audio data is processed in parallel in a plurality of multiplexing units to generate a plurality of multiplexed data as a plurality of sub clips. Further, auxiliary information including a sub-clip sequence number for each data type and packet type information is generated for the packet included in the multiplexed data. Thereafter, auxiliary information is applied to calculate and set a continuous count value, which is a sequence number for each data type, for packets constituting a plurality of multiplexed data included in a plurality of sub-clips. With this configuration, speeding up of the multiplexing process is realized.

It is a figure explaining the structural example and process of the information processing apparatus of this invention. It is a figure explaining the setting process of the continuous count value which the information processing apparatus of this invention performs. It is a figure explaining the setting process of the continuous count value which the information processing apparatus of this invention performs. It is a figure explaining the example of the auxiliary information applied to the setting of the continuous count value which the information processing apparatus of this invention performs. It is a figure explaining the example of the auxiliary information applied to the setting of the continuous count value which the information processing apparatus of this invention performs. It is a figure explaining the example of the auxiliary information applied to the setting of the continuous count value which the information processing apparatus of this invention performs. It is a figure which shows the flowchart explaining the process sequence which the information processing apparatus of this invention performs. It is a figure which shows the flowchart explaining the process sequence which the information processing apparatus of this invention performs. It is a figure which shows the flowchart explaining the process sequence which the information processing apparatus of this invention performs.

The details of the information processing apparatus, information processing method, and program of the present invention will be described below with reference to the drawings. Examples of the present invention will be described below according to the following items.
1. 1. Configuration and processing of information processing apparatus of the present invention Processing sequence executed by the information processing apparatus of the present invention

[1. Configuration and processing of information processing apparatus of the present invention]
FIG. 1 shows a configuration example of an information processing apparatus 100 according to an embodiment of the present invention. The information processing apparatus 100 executes a recording content editing process on an information recording medium such as a BD-ROM disc. The content is data including image data and audio data.

  As described above, for example, the recording data for the BD-ROM is a transport stream (TS) composed of MPEG data or the like. The transport stream (TS) is generated by multiplexing the stream (elementary stream) constituting each of the image data and the audio data which are the content data. The information processing apparatus 100 performs a multiplexing process of these elementary streams using each elementary stream (ES) of image data and audio data, and transport stream (TS) that is recording data for the BD-ROM. Is generated.

  When multiplexing each elementary stream (ES) of image data and audio data, as described above, it is necessary to order video and audio that need to be reproduced together in the reproduction process so as not to be separated from each other. . Since this order setting is necessary, it is difficult to parallelize the multiplexing process, and there is a problem that a long time is required for the transport stream generation process.

  The transport stream finally generated by the multiplexing process is stored in a clip file. A clip file stores a plurality of transport stream packets (TS packets). In each TS packet, a continuous count value (continuity_counter) indicating the above-described order is recorded as header information.

  As the continuous count value (continuity_counter), for example, 0000 to 1111 are repeatedly used as a 4-bit value. When expressed in decimal, it is a count value of 0 to 15. This count value is recorded as identification information indicating the order of TS packets. A number of TS packets included in the finally generated clip file include 0 (0000) to 15 (1111), 0 (0000) to 15 (1111), 0 (0000) to 15 (1111) from the top of the content. )... Will be set regularly in order. (Xxxx) is data represented as a 4-bit expression.

  The continuous count value to be set for each TS packet cannot be determined unless the finally generated TS packets are arranged in order. This hinders the parallelization of the multiplexing process, and causes a reduction in the efficiency of the recording data generation process, that is, the content editing (authoring) process.

  The information processing apparatus 100 according to the present invention performs parallel multiplexing processing of image and audio elementary streams in a state where the value of the continuous count value (continuity_counter) is indefinite. High speed processing is realized by this parallel processing.

  The information processing apparatus 100 of the present invention generates an auxiliary information file having information necessary for setting a continuous count value (continuity_counter) in the parallel multiplexing process. When a plurality of sub-clip files finally generated by parallel multiplexing processing are combined into one clip file, a value of a continuous count value (continuity_counter) corresponding to a TS packet is determined with reference to the auxiliary information file, A clip file storing the determined TS packet is generated.

Components of the information processing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.
File server 101
The file server 101 includes elementary streams (ES) 111, 121, and 122 such as images and sounds that are materials for editing (authoring), and a multiplexed transport stream generated by editing (authoring) processing. The storage unit stores the clip file 150 and the sub clips 130 (1) to (N) including auxiliary information (index information) generated by the parallel multiplexing process.

Image ES (Video Elementary Stream) 111
This is image data as a material for editing (authoring), and is, for example, a video elementary stream (ES) that has been subjected to MPEG encoding processing and can be used for multiplexing processing.

Audio ES (Audio 1 elementary stream) 121, 122
It is audio data used as material for editing (authoring), and is an audio elementary stream (ES) that has been encoded and can be used for multiplexing processing.
FIG. 1 shows a first audio ES 121 and a second audio ES 122. These are, for example, Japanese voice data and English voice data. In addition, there may be third and fourth voice ESs.

Controller 102
It is a control part which controls content edit processing. For example, the control for executing the multiplexing process of the image ES and the audio ES in the plurality of multiplexing units 103 (1) to (N) and the process control of the count value setting unit 104 are performed. The control unit 102 has, for example, a CPU as a program execution unit, and performs content editing control by executing a program stored in a storage unit (not shown).

Multiplexers (Mux) 1 to N, 103 (1) to (N)
A plurality of multiplexing processing units that execute in parallel a multiplexing process in which packets of different data types such as a packet storing an image ES and an audio ES are arranged and multiplexed.

The information processing apparatus 100 may be configured as a single apparatus,
A device having a control unit 102;
N devices (n = 1 to N) having a multiplexing unit n;
Count value setting unit 104,
File server 101,
These may be configured as a system connected by a network.
In this case, the control unit 102, N devices (n = 1 to N) having the multiplexing unit n, and the count value setting unit 104 access the file server 101 via the network, and acquire and store information. To do.

Multiplexers (Mux) 1 to N and 103 (1) to (N) shown in the figure can operate in parallel, and can perform a maximum of N parallel multiplexing processes.
Each of these multiplexing units (Mux) 1 to N, 103 (1) to (N) performs the multiplexing process of the service data on the image ES111 and the audio ES121 and 122, and the sub clip n & shown in FIG. The auxiliary information 130 (1) to (N) is generated and stored in the file server 101.

Count value setting unit 104
The sub clip n & auxiliary information 130 (1) to (N) generated by the multiplexing units (Mux) 1 to N and 103 (1) to (N) are acquired, and the final information is obtained while referring to the sub clip and the auxiliary information file. A continuous count value (continuity_counter) set in a packet which is typical multiplexed data is determined, and a TS file in the sub clip file is combined to generate a clip file 150.

  Steps S11 to S15 shown in FIG. 1 show a sequence of content editing (authoring) processing. These processing controls are executed in the control unit 102. Each of these processes will be described.

Step S11
This is a process of providing the image ES111 and the audio ES121 and 122 from the file server 101 to each of the multiplexing units (Mux) 1 to N and 103 (1) to (N).
The control unit 102 divides the image ES 111 and audio ES 121 and 122 to be edited and provides them to the multiplexing units (Mux) 1 to N and 103 (1) to (N).
The number of divisions is the number of multiplexing units (Mux) 1 to N, 103 (1) to (N) that can be processed in parallel. In this example, the number of multiplexing units (Mux) that can be processed in parallel is N. , Division number = N.

The control unit 102 acquires the time stamp information of the image ES111, and divides the image ES111 into N almost evenly based on the time stamp information. Further, the audio ESs 121 and 122 are also divided at a position corresponding to the time stamp of the image ES 111 that is the division position, and N division data is generated.
The N-divided image ES111 and audio ES121 and 122 are provided to the multiplexing units (Mux) 1 to N and 103 (1) to (N). In parallel multiplexing processing, a single elementary stream file is simultaneously read and accessed from a plurality of multiplexing units.

Step S12
This is a multiplexing process step in the multiplexing units (Mux) 1 to N and 103 (1) to (N).
Each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) receives the N-divided image ES111 and audio ES121 and 122, and an image and audio elementary stream included in the received data. Multiplex processing is performed. This multiplexing process is executed with the value of the continuous count value (continuity_counter) being indefinite. Further, in this process, each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) creates an auxiliary information file having information necessary for setting a continuous count value (continuity_counter). Generate.

Step S13
Processing for generating sub-clip n & auxiliary information 130 (1) to (N) generated in the multiplexing processing in the multiplexing units (Mux) 1 to N and 103 (1) to (N) and storing them in the file server 101 It is a step.
Multiplexers 103 (1) to (N) generate sub-clips and auxiliary information files in parallel.

Step S14
In the step in which the count value setting unit 104 acquires the sub clip n & auxiliary information 130 (1) to (N) generated by the multiplexing units (Mux) 1 to N and 103 (1) to (N) from the file server 101. is there.

Step S15
The count value setting unit 104 determines a continuous count value (continuity_counter) value to be set in the TS packet while referring to the sub clip and the auxiliary information file, and performs a process of combining the TS packets in the sub clip file to perform clipping. This is a step of generating the file 150.

Step S16
In this step, the clip file 150 generated by the count value setting unit 104 is stored in the file server 101.

  The content editing process is executed in the procedure of steps S11 to S16, and a clip 150 including a transport stream storing the image ES111 and the audio ES121 and 122 as material data is generated. The clip 150 is used as recording data on an information recording medium such as a BD-ROM.

  Referring to FIGS. 2 to 4, each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) is included in the sub clip generated and the multiplexed data in the final clip 150. A mechanism for determining a continuous count value (continuity_counter) set in a packet will be described.

FIG. 2 is an example in which the number of emotional parts that can be operated in parallel: N = 3. Subclips 1 and 2 are subclips generated by each of the three multiplexing units (Mux) 1 to 3. The sub-clip 3 is shown. The sub clip is composed of a plurality of packets. Each packet contains
An image packet containing image data,
A first voice packet storing first voice data;
A second voice packet storing second voice data;
System packet that stores management information,
These packets are included. These packet types can be identified by PID (Packet Identification) which is packet identification information recorded in the header of each packet.

  The multiplexed data included in the sub-clip is set as mixed data of packets storing these various data. In FIG. 2, the sub-clip 1 is described in order to explain the mechanism for determining the continuous count value (continuity_counter). Each of sub-clip 2 and sub-clip 3 will be described on the assumption that all of them are image packets storing only image data.

  The final continuous count value (continuity_counter) value to be set for the image packet is the value shown in FIG. That is, a decimal value of 0 to 15 is set continuously from the beginning. This corresponds to a process of repeatedly setting the 4-bit data 0000 to 1111.

  However, the value of the continuous count value (continuity_counter) is not a value that each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) shown in FIG.

  For example, if the multiplexing unit (Mux) 2 that generates the sub-clip 2 shown in FIG. 2 cannot know the value of the last continuous counter of the sub-clip 1 using the preceding image packet, The value of the continuous counter set in the configuration packet cannot be determined. This is shown by the data (undefined) in FIG.

  Since the multiplexing unit (Mux) 1 that generates the sub clip 1 is the first divided data of the packet received by itself, the value of the continuous count value (continuity_counter) is the same as the packet = P1 to P9 shown in the figure. The continuous count values can be set in the order of the continuous counter values 0, 1, 2, 3,.

  However, the multiplexing unit (Mux) 2 that generates the sub-clip 2 does not know the value of the last continuous counter of the sub-clip 1 using the preceding image packet, instead of the packet received by the multiplexing unit (Mux) 2 itself. Therefore, an accurate continuous count value cannot be set at this point. The same applies to the multiplexing unit (Mux) 3 that generates the sub clip 3.

  However, each multiplexing unit (Mux) can set the index value in order from the top in accordance with the number of packets included in the sub clip generated by itself. This is the index value data shown in FIG. This is a sequence number within a sub clip.

  Each multiplexing unit (Mux) generates an index value (sequence number in sub-clip) shown in FIG. 2 (3) as auxiliary information in the multiplexing process. The generated auxiliary information is stored in the file server 101 together with the sub clip generated by each multiplexing unit (Mux). This is the sub clip n & auxiliary information 130 (n) shown in the file server 101 of FIG.

The count value setting unit 104 shown in FIG. 1 sets a final continuous count value using this auxiliary information.
First, the number of constituent packets of each sub clip is obtained from the index value (sub-clip sequence number) shown in FIG. It is the data of FIG. The number of packets included in each sub clip is as follows.
Sub clip 1 = 9 Sub clip 2 = 10 Sub clip 3 = 6

Next, the number of packets in the sub-clip shown in FIG. 2 (4) is added to the index value in the sub-clip shown in FIG. 2 (3). The addition target is the total number of packets of the preceding subclip. As a result, the values 0 to 24 shown in FIG.
The value shown in FIG. 2 (5) is the entire sequence number corresponding to all the packets constituting the sub clip.

Next, a modulo 16 operation is performed on the entire sequence number. Assuming that the overall sequence number corresponding to all packets shown in FIG. 2 (5) is [q] and the continuous count value to be calculated is [p], the remainder is obtained by dividing the overall sequence number [q] by 16 by the following calculation. This is designated as [p].
p = qmod16
This result is the value shown in FIG.

  The values shown in FIG. 2 (6) are the same as the values shown in FIG. That is, the final continuous count value (continuity_counter) value to be set for the image packet is obtained.

  The count value setting unit 104 shown in FIG. 1 uses the auxiliary information to calculate the value shown in FIG. 2 (6), and this calculated value is included in each sub clip as the value of the continuous count value (continuity_counter). Set to packet.

  This is a basic processing sequence of the setting processing of the continuous count value (continuity_counter) after the parallel multiplexing processing executed by the information processing apparatus 100 of the present invention.

However, as described above, the packet included in the sub-clip generated by each of the multiplexing units (Mux) is
An image packet containing image data,
A first voice packet storing first voice data;
A second voice packet storing second voice data;
System packet that stores management information,
These packets are included.
The multiplexed data of the sub clip is set as mixed data of packets storing these various data.
In the clip 150 to be finally generated, four types of packets of different image packets, first audio packets, second audio packets, and system packets are mixed.

The continuous count value (continuity_counter) needs to be set as a continuous value in the type unit (PID unit) of these packets. That is, it is necessary to set the following count values 0 to 15 for each packet type. Note that (xxxx) is data expressed in binary.
Image packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
First voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
Second voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
System packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
In this way, individual count values are set for each packet type.
This process will be described with reference to FIG. 3, FIG. 4, and FIG.

FIG. 3 shows an example of a sub clip (sub clip N) generated by one of the multiplexing units 103 shown in FIG.
As described above, the sub-clip N is composed of four different types of packets: the image packet (V), the first audio packet (A1), the second audio packet (A2), and the system packet (S). In the following, assuming that PID indicating an image packet is V, PID of the first voice packet is A1, PID of the second voice packet is A2, and PID of the system packet is S, the type of the packet can be identified by these PIDs. It will be explained as being.

The image packet (V) is a packet in which image data included in the image ES11 acquired from the file server 101 illustrated in FIG. 1 is stored.
The first audio packet (A1) is a packet in which the first audio data included in the image ES121 acquired from the file server 101 illustrated in FIG. 1 is stored.
The second audio packet (A2) is a packet in which the second audio data included in the image ES122 acquired from the file server 101 illustrated in FIG. 1 is stored.
The system packet (S) is a packet storing management information generated in the control unit 102 or the multiplexing unit 103.

  Each of the multiplexing units 103 shown in FIG. 1 generates a sub clip N shown in FIG. 3 by executing a multiplexing process in which packets storing these four types of different data are mixed and arranged.

  The auxiliary information generated by each of the multiplexing units 103 during this multiplexing process is index information in units of packet IDs (PID) consisting of FIGS. 3B to 3E, that is, data composed of index values for each packet type. It is.

  FIG. 3A corresponds to the data shown in FIG. That is, the index value is set in order from the beginning of the packet included in the sub clip generated by itself. This index value is called a global index value. The global index value is a sequence number that is set according to the number of packets from the top of the sub clip without determining the type of packet.

Each multiplexing unit (Mux) 103 further generates a sequence number (sequence number in sub-clip) in units of packet types shown in FIGS. 3B to 3E, that is, PID units. In particular,
(B) Index value of image packet in subclip (c) Index value of first audio packet in subclip (d) Index value of second audio packet in subclip (e) System packet in subclip Index value Each of these index values.

In the auxiliary information file,
(A) Global index (b) to (e) PID unit index in units of packet types (per PID) (sequence number in sub-clip for each data type)
These multiple index values are included.

  A configuration example of auxiliary information generated by each multiplexing unit (Mux) 103 is shown in FIGS. The first auxiliary information file shown in FIG. 4 corresponds to the global index shown in FIG. 3 (a) and the PID unit index (sequence number in sub-clip for each data type) shown in FIGS. 3 (b) to 3 (e). Have data. Only the data corresponding to the data range 201 shown in FIG. 3 is shown. That is, the auxiliary information file shown in FIG. 4 includes the index values (0, 0, 0, 0, 1) for each PID in FIGS. 3B to 3E corresponding to the global index values 0 to 9 in FIG. , 2, 3, 1,...) In order.

  The second auxiliary information shown in FIG. 5 is data in which the global index shown in FIG. 3A is associated with the PID type information indicating the type of each packet.

  The example of the auxiliary information shown in FIGS. 4 and 5 is an example, and may be set to have other configurations. For example, the global index included in the first auxiliary information corresponds to the number of packets from the top of the sub-clip, and can be omitted because it can be calculated by counting the number of packets. Further, the second auxiliary information is not limited to the example illustrated in FIG. 5, and may be configured to include information such as a PID list of packets included in the sub clip and the total number of packets in PID units.

  Each multiplexing unit (Mux) 103 shown in FIG. 1 creates a sub clip, first auxiliary information shown in FIGS. 4 and 5, and second auxiliary information, and stores them in the file server 101.

The count value setting unit 104 shown in FIG. 1 acquires the sub clip generated by each multiplexing unit 103, the first auxiliary information, and the second auxiliary information from the file server 101, generates a clip 150, and finally Set the continuous count value. The count value setting unit 104 sets a continuous count value in PID units. That is, as mentioned above,
Image packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
First voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
Second voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
System packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
In this way, individual count values are set in packet type units (PID units).

  The count value setting unit 104 shown in FIG. 1 calculates a continuous count value for each PID by executing the same method as described above with reference to FIG. 2 for each packet type.

For example, an example of processing for calculating a continuous count value for an image packet (PID = V) will be described.
First, the number of constituent packets of the image packet (PID = V) included in each sub-clip is obtained from the auxiliary information shown in FIGS.
Note that the information shown in FIG. 6 is obtained by combining the first auxiliary information shown in FIG. 4 and the second auxiliary information shown in FIG. For example, by selecting only the image packet (PID = V) from the information shown in FIG. 6, the maximum value of the index in PID units is obtained, and the maximum value + 1 adds the image packet (PID = V) included in the sub clip. The total number is obtained. In FIG. 6, only the entry of the image packet (PID = V) is shown in white, and the entries of other packets are shown by black diagonal lines. As described above, if the total packet number information for each PID included in the sub-clip is set in advance as the second auxiliary information, the information can be used.

As a result, for example, the data of FIG. 2 (4) described above is obtained.
This will be described using the example of FIG.
The number of image packets (PID = V) included in each sub clip is as follows.
Number of image packets (PID = V) included in sub clip 1 = 9 Number of image packets (PID = V) included in sub clip 2 = 10 Number of image packets (PID = V) included in sub clip 3 = 6 Pieces

  Next, the total number of image packets (PID = V) of the preceding sub clip is added to the index value of the image packet (PID = V) included in the sub clip shown in FIG. This process corresponds to the process of adding the number of packets in the sub-clip shown in FIG. 2 (4) to the index value in the sub-clip shown in FIG. 2 (2) described above with reference to FIG.

  As a result, the values 0 to 24 shown in FIG. The values shown in FIG. 2 (5) are PID unit whole sequence numbers corresponding to all image packets (PID = V) included in all sub-clips.

Next, a modulo 16 operation is performed on the entire sequence number of the PID unit. Assuming that the sequence number shown in FIG. 2 (5) is [q] and the continuous count value to be calculated is [p], a remainder obtained by dividing the sequence number [q] by 16 is obtained by the following calculation. To do.
p = qmod16
This result is the value shown in FIG.

  Through these processes, a continuous count value (continuity_counter) corresponding to an image packet (PID = V) included in a sub clip in which various packets are mixed is calculated.

  The count value setting unit 104 shown in FIG. 1 calculates the continuous count value (continuity_counter) corresponding to the image packet (PID = V) using the auxiliary information in this way, and the image packet (PID = V) included in the clip 150. ).

The same processing is executed for the first audio packet (A1), the second audio packet (A2), and the system packet (S) other than the image packet (PID = V), and a continuous count value corresponding to each packet is obtained. Set. That is, as mentioned above,
Image packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
First voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
Second voice packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
System packet: 0 (0000) to 15 (1111), 0 (0000) to 15 (1111)...
In this way, individual count values are set in packet type units (PID units).

[2. Processing sequence executed by information processing apparatus of the present invention]
Next, a processing sequence executed by the information processing apparatus according to the present invention will be described with reference to flowcharts shown in FIGS.
FIG. 7 is a flowchart showing an overall sequence of processing executed by the information processing apparatus 100 shown in FIG.
FIG. 8 is a flowchart for explaining details of processing executed by each of the multiplexing units 1031 to N and 103 (1) to (N) of the information processing apparatus 100 shown in FIG. This process corresponds to each process of steps S102-1 to N in the flow of FIG.
FIG. 9 is a flowchart for explaining details of processing executed by the count value setting unit 104 of the information processing apparatus 100 shown in FIG. This process corresponds to the process of step S103 in the flow of FIG.

First, a flowchart illustrating an overall sequence of processing executed by the information processing apparatus 100 illustrated in FIG. 7 will be described. This process is executed under the control of the control unit 102 shown in FIG.
First, in step S101, the number of divisions and the division position of the material to be edited are determined. This process is executed by the control unit 102. Specifically, the number of divisions and the division positions of the image elementary stream 111, the first audio elementary stream 121, and the first audio elementary stream 122 shown in FIG. 1 are determined.

  As described above with reference to FIG. 1, the control unit 102 divides the image ES 111 and the audio ES 121 and 122 to be edited and multiplexes (Mux) 1 to N, 103 (1) to (N ) To provide. The number of divisions is the number of multiplexing units (Mux) 1 to N, 103 (1) to (N) that can be processed in parallel. In this example, the number of divisions is N.

  As described above, the control unit 102 determines the division position as follows, for example. Time stamp information of the image ES111 is acquired, and the image ES111 is divided into N almost evenly based on the time stamp information. Further, the audio ESs 121 and 122 are also divided at a position corresponding to the time stamp of the image ES 111 that is the division position, and N division data is generated. The N-divided image ES111 and audio ES121 and 122 are provided to the multiplexing units (Mux) 1 to N and 103 (1) to (N).

  Next, the processing of steps S102-1 to S102-N in the flow shown in FIG. 7 is executed. This process is a multiplexing process step performed in the multiplexing units (Mux) 1 to N and 103 (1) to (N) shown in FIG.

  Each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) receives the N-divided image ES111 and audio ES121 and 122, and an image and audio elementary stream included in the received data. Multiplex processing is performed. This multiplexing process is executed with the value of the continuous count value (continuity_counter) being indefinite. Further, in this process, each of the multiplexing units (Mux) 1 to N and 103 (1) to (N) creates an auxiliary information file having information necessary for setting a continuous count value (continuity_counter). Generate. This specific processing will be described later with reference to the flowchart of FIG.

  When the processing of steps S102-1 to S102-N in FIG. 7 is completed, the sub clips and auxiliary information files generated by the multiplexing units (Mux) 1 to N and 103 (1) to (N) are shown in FIG. Stored in the server 101.

  In step S103 of FIG. 7, the count value setting unit 104 acquires the sub clips and auxiliary information generated by the multiplexing units (Mux) 1 to N and 103 (1) to (N) from the file server 101, and continuously This is a step of determining the value of the count value (continuity_counter) and combining the TS packets in the sub clip file to generate the clip file 150. Details of this processing will be described later with reference to a flowchart shown in FIG.

  Through these processes, the multiplexing processes 103 (1) to (N) shown in FIG. 1 can execute the parallel process of the multiplexing process, and the accurate continuous count value (continuity_counter) value in PID units can be obtained. It is possible to set, and high-speed and highly accurate content editing processing is realized.

  Next, details of processing executed by each of the multiplexing units 1031 to N and 103 (1) to (N) of the information processing apparatus 100 illustrated in FIG. 1 will be described with reference to a flowchart illustrated in FIG. This process corresponds to the first half of the process described above with reference to FIGS.

The process of each step of the flowchart shown in FIG. 8 will be described.
In step S201, a counter that holds an index value is initialized (reset) for each packet type (PID type) constituting the sub clip. Specifically, as described with reference to FIG. 3, the image packet (V), the first audio packet (A1), the second audio packet (A2), and the system packet (S) constituting the sub clip are different. A counter for holding index values (data shown in FIGS. 3B to 3E) for each of the four types of packets is prepared and initialized.

  For example, the count value is set to Null (invalid value) by the initialization processing of the counter, and the number of packets is counted as Null → 0 → 1 → 2 → 3. The counter is provided for each of the image packet (V), the first voice packet (A1), the second voice packet (A2), the system packet (S), and each of these packets. In response, the number of packets is individually counted.

  In step S202, sub-clip generation by multiplexing processing is started, and packet types (PID types) are determined sequentially from the top of the sub-clip. The packet type (PID type) is an image packet (V), a first voice packet (A1), a second voice packet (A2), a system packet (S), and these packet types.

  Next, in step S203, the value of the corresponding counter is updated based on the packet type discrimination information of the packets constituting the sub clip. Specifically, the packet type of the packet constituting the sub clip is an image packet (V), a first audio packet (A1), a second audio packet (A2), a system packet (S), or any of these packets. The counter for each corresponding packet type is updated in accordance with the packet type information indicating whether or not, and the count value is incremented.

Next, in step S204, first auxiliary information and second auxiliary information are created using the count value and packet type (PID) information.
The first auxiliary information has a data structure shown in FIG. 4, for example. That is, it has correspondence data between the global index shown in FIG. 3A and the PID unit indexes shown in FIGS.
The second auxiliary information has, for example, a data configuration shown in FIG. That is, it is data in which the global index shown in FIG. 3A is associated with the PID type information indicating the type of each packet. Note that the second auxiliary information is not limited to the example shown in FIG. 5 as described above, and may include a PID list of packets included in the sub clip and information such as the total number of packets in PID units. .

  Next, in step S205, it is determined whether or not sub-clip generation has been completed. That is, it is confirmed whether or not an unprocessed packet remains. When the unprocessed packet remains and the generation of the sub-clip is not completed, the process returns to step S202, and the processes after step S202 are repeated for the unprocessed packet.

If it is determined in step S205 that the sub-clip generation has been completed, the process proceeds to step S206.
In step S206, the generated sub clip, the first auxiliary information, and the second auxiliary information are output to the file server.

  The processing according to the flowchart shown in FIG. 8 is processing performed in parallel in the multiplexing units 103 (1) to (N) shown in FIG. By this processing, N sub-clips & auxiliary information 130 (1) to (N) are stored in the file server 101 shown in FIG.

  Next, the process of step S103 in the flow of FIG. 7 will be described with reference to the flowchart shown in FIG. This process is a process executed by the count value setting unit 104 of the information processing apparatus 100 shown in FIG. This process corresponds to the latter half of the process described above with reference to FIGS.

The process of each step of the flowchart shown in FIG. 9 will be described.
In step S301, the sub clip generated by each multiplexing unit and the first auxiliary information file and the second auxiliary information file are acquired. This process is a process for acquiring N sub-clips & auxiliary information 130 (1) to (N) stored in the file server 101 shown in FIG.

  Next, in step S302, a packet type (PID type) to be processed is selected. Specifically, as packet types for setting the value of the continuous count value (continuity_counter), the image packet (V), the first voice packet (A1), the second voice packet (A2), the system packet (S), and these Select one of the packets.

  Next, in step S303, referring to the second auxiliary information (see FIG. 5), a packet corresponding to the processing target PID type is selected from the sub-clip. For example, when the processing target is an image packet (V), only the image packet included in the sub clip is selected. Note that the sub-clips to be processed here are all sub-clips generated by each multiplexing unit.

  Next, in step S304, the PID correspondence index value corresponding to the selected packet is acquired with reference to the first auxiliary information (see FIG. 4). As described above with reference to FIG. 6, this process is a process for acquiring a PID-compatible index value corresponding to a specific type of packet. This corresponds to processing for acquiring any one of the index values in FIGS.

  Next, in step S305, the total number of packets of the same type preceding the acquired PID-compatible index value is added to calculate a PID-compatible overall sequence number of the same packet type.

  This process corresponds to the process of adding the number of packets in the sub-clip shown in FIG. 2 (4) to the index value in the sub-clip shown in FIG. 2 (2) described above with reference to FIG. As a result, the values 0 to 24 shown in FIG. The value shown in FIG. 2 (5) is a specific packet type included in all the sub-clips, for example, an entire sequence number of PID units corresponding to all packets of image packets (PID = V).

Next, in step S306, the obtained PID unit whole sequence number [q] is applied to perform the following calculation to calculate a continuous count value [p].
p = q mod 16
The above operation is executed to calculate a continuous count value (continuity_counter). This result corresponds to the value shown in FIG.

  Next, in step S307, the continuous count value calculated in step S306 is set to a packet of the packet type targeted for processing.

  Next, in step S308, it is determined whether or not all PID types have been processed. That is, it is determined whether or not an unprocessed PID remains. Specifically, the image packet (V), the first audio packet (A1), the second audio packet (A2), the system packet (S), and the continuous count value (continuity_counter) for all types of these packets. It is determined whether the calculation is completed.

  If there is an unprocessed packet type (PID), the processing from step S302 is executed on the unprocessed packet type (PID).

  If it is determined in step S308 that the processing for all PID types has been completed, the processing ends. Note that the created clip is stored in the file server when the process ends.

  In this way, parallel processing of multiplexing processing can be executed in the plurality of multiplexing units 103 (1) to (N) shown in FIG. 1, and an accurate continuous count value (continuity_counter) value in PID units is set. It is possible to set, and high-speed and highly accurate content editing processing is realized.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet and can be installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of an embodiment of the present invention, an apparatus and method for executing data multiplexing processing including image and audio data in parallel are provided. That is, each of the divided data of the image data and the audio data is processed in parallel in a plurality of multiplexing units to generate a plurality of multiplexed data as a plurality of sub clips. Further, auxiliary information including a sub-clip sequence number for each data type and packet type information is generated for the packet included in the multiplexed data. Thereafter, auxiliary information is applied to calculate and set a continuous count value, which is a sequence number for each data type, for packets constituting a plurality of multiplexed data included in a plurality of sub-clips. With this configuration, speeding up of the multiplexing process is realized.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 101 File server 102 Control part (Controller)
103 Multiplexing unit 104 Count value setting unit 111 Image elementary stream 121, 122 Audio elementary stream 130 Sub clip & auxiliary information 150 clip

Claims (9)

A control unit that divides image data and audio data, which are recording data materials of the information recording medium, according to the number of multiplexing units that can be processed in parallel;
A plurality of multiplexing units that receive each of the divided data divided by the control unit, and multiplex the image packets storing the received image data and audio data and packets of different data types including audio packets in a mixed manner It has a plurality of multiplexing units that execute processing in parallel and generate a sub clip including multiplexed data that is a processing result,
Each of the plurality of multiplexing units is
During the sub-clip generation process, auxiliary information to be applied to the calculation of a continuous count value that is a packet sequence number for each data type for the packets constituting the plurality of multiplexed data generated by the plurality of multiplexing units is generated. Information processing device. Each of the plurality of multiplexing units is
The information processing apparatus according to claim 1, wherein auxiliary information including a sub-clip sequence number for each data type and packet type information for a packet included in the multiplexed data is generated. The information processing apparatus further includes:
A count value setting unit that sets a continuous count value that is a sequence number for each data type for storage packets of different data types included in a plurality of sub-clips generated by the plurality of multiplexing units;
The count value setting unit
The information processing according to claim 1 or 2, wherein the auxiliary information is applied to calculate a continuous count value, which is a sequence number for each data type, for a packet constituting a plurality of multiplexed data included in a plurality of sub-clips. apparatus. The count value setting unit
Applying the auxiliary information to calculate the number of packets for each data type included in each sub clip,
The total number of packets of the same data type of all preceding sub-clips is added to the sequence number in the sub-clip for each data type included in the auxiliary information, and the entire sequence for each data type included in all the sub-clips Calculate the number,
4. The information processing apparatus according to claim 3, wherein a calculation process of a continuous count value for each data type is performed by applying the entire sequence number for each data type. The count value setting unit
The continuous count value is set as repeated data of 0 to 15,
When the overall sequence number is [q] and the continuous count value [p],
p = qmod16
The information processing apparatus according to claim 4, wherein p calculated by the calculation formula is calculated as the continuous count value. Each of the plurality of multiplexing units generates a sub clip including multiplexed data in which image packets storing image data, audio packets storing audio data, and system packets storing management information are mixed and arranged. And
The count value setting unit
6. The continuous count value is calculated for each of an image packet, an audio packet, and a system packet constituting a plurality of multiplexed data included in a plurality of sub-clips by applying the auxiliary information. The information processing apparatus described in 1. The control unit, the plurality of multiplexing units, the count value setting unit, and the file server that stores the image data, audio data, and sub-clips are connected via a network,
Each of the plurality of multiplexing units acquires the image data and audio data from the file server via the network, and stores the generated sub clip and auxiliary information in the file server via the network. ,
The information processing apparatus according to claim 3, wherein the count value setting unit performs a process of acquiring the sub clip and auxiliary information from the file server via the network. An information processing method executed in an information processing apparatus,
A step of dividing the image data and audio data, which are the materials of the recording data of the information recording medium, according to the number of multiplexing units capable of parallel processing;
Each of a plurality of multiplexing units receives each of the divided data divided by the control unit;
Each of the plurality of multiplexing units executes, in parallel, a multiplexing process in which image packets storing received image data and audio data and packets of different data types including audio packets are mixed and arranged. A sub-clip generation step for generating a sub-clip including multiplexed data;
The sub-clip generation step includes
An information processing method including a step of generating auxiliary information to be applied to calculation of a continuous count value that is a packet sequence number for each data type for a packet constituting a plurality of multiplexed data generated by the plurality of multiplexing units. A program for executing information processing in an information processing apparatus;
A step of causing the control unit to divide image data and audio data, which are recording data materials of the information recording medium, according to the number of multiplexing units capable of parallel processing;
Causing each of a plurality of multiplexing units to receive each of the divided data divided by the control unit;
Each of the plurality of multiplexing units executes in parallel a multiplexing process in which image packets storing received image data and audio data and packets of different data types including audio packets are mixed and arranged. A sub-clip generation step for generating a sub-clip including multiplexed data;
The sub-clip generation step includes
A program including a step of generating auxiliary information to be applied to calculation of a continuous count value that is a packet sequence number for each data type for a packet constituting a plurality of multiplexed data generated by the plurality of multiplexing units.