JP4214816B2 - Media information processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a media information processing method and apparatus for efficiently converting media information into fixed-length packets.
[0002]
[Prior art]
In recent years, the multimedia era has come to handle voice, image, and other pixel values in an integrated manner. It has come to be taken up as a target. In general, multimedia refers to not only characters but also figures, sounds, especially images, etc. that are associated with each other at the same time. It is an essential condition.
[0003]
However, when the information amount of each information medium is estimated as a digital information amount, the amount of information per character is 1 to 2 bytes in the case of characters, whereas 64 Kbits (phone quality) per second in the case of speech. In addition, for a moving image, an information amount of 100 Mbits (current television reception quality) or more per second is required, and it is not realistic to handle the enormous amount of information in the digital format as it is with the information medium. For example, a videophone is a service integrated digital network (ISDN) having a transmission rate of 64 Kbit / s to 1.5 Mbits / s. Integrated Although it has already been put into practical use by Services Digital Network), it is impossible to send the video of the TV camera as it is by ISDN.
[0004]
Therefore, what is required is information compression technology. For example, in the case of a videophone, H.264 has been internationally standardized by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.264. H.263 standard video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to put image information together with audio information on a normal music CD (compact disc).
[0005]
Here, MPEG (Moving Picture Experts Group) is an international standard for moving picture signal compression, and MPEG-1 is up to 1.5 Mbps for moving picture signals, that is, up to about 1 / 100th of information of television signals. It is a standard to compress. In addition, since the transmission speed for the MPEG-1 standard is mainly limited to about 1.5 Mbps, MPEG-2 standardized to meet the demand for higher image quality has 2 to 2 video signals. Compressed to 15 Mbps. Furthermore, at present, the working group (ISO / IEC JTC1 / SC29 / WG11), which has been standardizing with MPEG-1 and MPEG-2, has achieved a compression ratio that exceeds MPEG-1 and MPEG-2, and further, on a per-object basis. MPEG-4 has been standardized that enables encoding, decoding, and operation and realizing new functions necessary in the multimedia era. In MPEG-4, it was originally aimed at standardizing a low bit rate encoding method, but now it has been extended to a more general encoding including a high bit rate including interlaced images.
[0006]
Media information such as video and audio (hereinafter referred to as multimedia information) is generated in parallel. In order to transmit multimedia information through a communication channel and reproduce it on the receiving side, it is necessary to multiplex it into one stream together with synchronization information. One of international standards for realizing such multiplexing is MPEG-2 system TS (Transport Stream, hereinafter referred to as MPEG-2 TS). In MPEG-2 TS, encoded media information is packetized in an appropriate unit for each medium to form a variable-length PES (Packetized Elementary Stream) packet, and further, a fixed-length TS (Transport Stream) packet unit Multiplex after dividing into.
[0007]
In order to construct and transmit a 1TS packet, it is necessary to have all the encoded data for constructing the 1TS packet. Therefore, especially in low bit rate encoding, packetization processing delay becomes a problem. . In order to reduce this processing delay, the conventional multiplexing method inserts redundant data (stuffing data) of the same amount as the amount of data that is insufficient to form one TS packet in the multiplexing layer, and encodes the encoded data. And 1TS packet (see, for example, Patent Document 1). Since the PES packet has a variable length, the amount of stuffing data is not constant.
[0008]
When the bit rate of the communication channel is fixed, it is necessary to control the bit rate so that the fixed bit rate is set for each piece of media information or for each piece of media information. When media information encoded at a fixed bit rate BR0 is divided into fixed-length packets for multiplexing such as TS packets by the above conventional multiplexing method, the amount of stuffing data is not constant. This media information has a variable bit rate. As a method of fixing the bit rate of media information after packetization, a bit rate with a margin equal to or greater than the fluctuation of the bit rate is assigned to the media information, and the bit rate to which the bit rate after packetization is assigned When there is a shortage of data, the amount of data is controlled so that a fixed bit rate is achieved by multiplexing invalid packets, which have no meaning for media information called NULL packets, with media information packets. There is a way to do it. The fluctuation of the bit rate corresponds to the bit rate BR1 of the stuffing data when the maximum stuffing data that can be generated (maximum 183 bytes in the case of MPEG-2 TS) is generated continuously, and the stuffing data of one time It is obtained by the product of the maximum amount of occurrence and the number of stuffing occurrences per second. By providing a margin for this bit rate BR1, it can be ensured that the media information after packetization does not exceed the allocated bit rate.
[0009]
FIG. 15 shows an example of a data structure of multiplexed data using the above packetization, and is an example of multiplexing one video signal and one audio signal. In general, additional information such as information for synchronization is also multiplexed, but in this example, it is omitted for simplicity. In the figure, video and audio are stuffed in a multiplexing layer so as to have a data size that can be converted into a fixed-length packet for each predetermined processing unit. When the maximum number of stuffing bits is not continuous, the amount of data does not satisfy the predetermined bit rate (BR0 + BR1), so a NULL packet is inserted. As the predetermined processing unit, in the case of video, methods such as every n frames (n is an integer of 1 or more), GOP (Group Of Pictures), GOV (Group Of Vops), and the like are conceivable. . In the case of audio, a method such as m ADTS frames (Audio Data Transport Stream) is conceivable.
[0010]
FIG. 16 is a block diagram showing an example of a conventional media information processing apparatus that fixes the bit rate of media information after packetization by the above method.
[0011]
The conventional media information processing apparatus encodes the media information 101 while controlling the bit rate by feedback control, and outputs the encoded media information 103a, and the encoded media while stuffing every predetermined processing unit. A packetizer 202d that packetizes the information 103a, a NULL packet generator 116 that generates a NULL packet 117, and a plurality of channelized encoded media information CH-1 to CH-n and a NULL packet 117; Multiplexing means 115 for multiplexing by a method and outputting multiplexed data 118 is configured.
[0012]
The encoding device 201d encodes the media information 101 according to the code amount control signal 107 and outputs the encoded media information 103a, and the code for each predetermined encoding processing unit from the encoded media information 103a. The code amount detecting unit 104 detects the amount and outputs the code amount 105, and the code amount control unit 106c outputs the code amount control signal 107 for controlling the code amount of the encoded media information 103a based on the code amount 105. ing. The encoded media information 103a is an occupation amount of the reception buffer model as shown in FIG. 17 so that the reception buffer of the decoder that decodes the encoded media information 103a transmitted at the fixed bit rate BR0 does not overflow or underflow. Encoded assuming transition. Times t0 to t3 represent the timing of decoding or playback / display of the encoded media information 103a. Under this assumption, data flows into the reception buffer at a fixed bit rate BR0, and the code amount BF of the encoded media information 103a consumed at the decoding or playback / display timing is instantaneously extracted from the reception buffer.
[0013]
The packetizer 202d detects a code amount for each predetermined packetization processing unit from the encoded media information 103a, and outputs a code amount 109. Stage 1 08 and a compensation amount control means for determining the stuffing amount 111a necessary for dividing the code amount 109 into a predetermined packet size 110a And supplementing means 112 for inserting stuffing bits into the encoded media information 103a according to the stuffing amount 111a, and packet dividing means 114 for dividing the stuffed encoded media information 113 into packets.
[0014]
Multiplexing means 115 checks whether or not the data amount is sufficient for a predetermined bit rate (BR0 + BR1) for each of packetized encoded media information CH-1 to CH-n. Multiplexes CH-1 to CH-n while inserting a NULL packet 117.
[0015]
As described above, in the conventional method, encoded multimedia information can be multiplexed.
[0016]
[Patent Document 1]
JP 9-74546 A (page 4-7, FIG. 1)
[0017]
[Problems to be solved by the invention]
FIG. 18 shows a breakdown of the transmission band when one video signal and one audio signal are multiplexed by a conventional method. In the conventional method, in order to protect the entire bit rate, a band BR1 for guaranteeing the maximum value of stuffing data is added to the band BR0 of encoded media information such as encoded video information and encoded audio information. There is a need to. However, since the amount of stuffing data is actually a random event, only about half of the band BR1 is statistically generated, and NULL packets are multiplexed in the remaining band corresponding to about half of the band BR1. . That is, the conventional method has an inefficiency that requires a bandwidth for transmitting invalid packets having no meaning for media information in order to guarantee the overall bit rate.
[0018]
The present invention has been made in view of the above points, and intends to propose a media information processing method and apparatus that efficiently multiplex media information.
[0019]
[Means for Solving the Problems]
In order to solve this problem, the first invention provides media information. As input, the media information was encoded Encoded media information Every predetermined processing unit Packetize to a predetermined data size Used in media information devices A media information processing method, The encoding means encodes the input media information in accordance with the code amount control signal and outputs the encoded media information; and the first code amount detection means includes a first code amount based on the encoded media information. A first code amount detection step for detecting a code amount of the first code amount, and a code amount control means are determined at the time of packetizing the first code amount and the immediately preceding processing unit; Multiplexing layer stuffing inserted for packetization With quantity So that the total data amount becomes a predetermined bit rate. Determine the code amount control signal Do A code amount control step; a second code amount detection unit for detecting a second code amount from the encoded media information; and a compensation amount control unit for detecting the second code amount. A compensation amount control step for determining the stuffing amount based on the stuffing amount and outputting the determined stuffing amount to the code amount control means; and a compensation means for inserting stuffing data into the encoded media information according to the stuffing amount A packet dividing step in which the packet dividing means divides the encoded media information into which the stuffing data has been inserted into packets of a predetermined length and outputs them. This is a media information processing method characterized by the above.
[0020]
The second invention provides media information As input, the media information was encoded Encoded media information Every predetermined processing unit Packetize to a predetermined data size Used in media information devices A media information processing method, The encoding means encodes the input media information in accordance with the code amount control signal and outputs the encoded media information; and the first code amount detection means includes a first code amount based on the encoded media information. A first code amount detecting step for detecting a code amount of the first code amount, and a first compensation amount control means, wherein the data amount obtained by combining the first code amount and the first stuffing amount of the current processing unit is: A first compensation amount control step for determining a first stuffing amount of the current processing unit so as to obtain a maximum data size not exceeding an integer multiple of the predetermined data size of packetization; and a code amount control means, A code amount control step for determining the code amount control signal in accordance with the first stuffing amount and the first code amount; and a first compensation means, wherein the encoding medium is in accordance with the first stuffing amount. A first compensation step for inserting the first stuffing data into the information, and a second code amount detection means for detecting a second code amount from the encoded media information into which the first stuffing data is inserted. A second compensation amount control step, a second compensation amount control means for determining a second stuffing amount of the current processing unit from the second code amount, and a second compensation amount control step. Means for inserting second stuffing data into the encoded media information into which the first stuffing data has been inserted in accordance with the second stuffing amount; and a packet dividing means comprising: A packet dividing step of dividing the encoded media information into which the stuffing data is inserted into packets of a predetermined length and outputting them. This is a media information processing method characterized by the above.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to FIGS.
[0022]
(Embodiment 1)
FIG. 1 is a block diagram showing an example of a media information processing apparatus according to Embodiment 1 of the present invention. The conventional media information processing apparatus is different from the conventional media information processing apparatus in that the stuffing amount 111a is output from the packetizing device 202a to the code amount control means 106a of the encoding device 201a.
[0023]
Hereinafter, the media information is encoded by the media information processing apparatus according to the first embodiment of the present invention in the band (BR0 + BR1) required when the media information is encoded and multiplexed by the band BR0 by the conventional media information processing apparatus. An example in the case of multiplex and multiplexing will be described. The BR1 is a band necessary for guaranteeing the maximum value of the stuffing data of the multiplexing layer, which is necessary for multiplexing by the conventional media information processing apparatus.
[0024]
According to the media information processing apparatus of Embodiment 1 of the present invention, the bit rate of the encoded media information 103a can be controlled in consideration of the stuffing amount 111a. Therefore, when the stuffing amount 111a is small, the encoded media information When the bit rate of 103a is increased and the stuffing amount 111a is large, the bit rate of the encoded media information 103a is decreased, so that the packetized encoded media information CH-1 can be set to a fixed bit rate (BR0 + BR1). . Therefore, in the conventional media information processing apparatus, NULL packet insertion, which is necessary to absorb the fluctuation of the stuffing amount 111a, becomes unnecessary in the media information processing apparatus of the first embodiment.
[0025]
FIG. 2 shows an example of the data structure of multiplexed data by the media information processing apparatus of the first embodiment. In this example, one video signal and one audio signal are multiplexed. In general, additional information such as information for synchronization is also multiplexed, but in this example, it is omitted for simplicity. is doing. In the figure, video and audio are stuffed in a multiplexing layer so as to have a data size that can be converted into a fixed-length packet for each predetermined processing unit. Unlike the data structure of multiplexed data by the conventional media information processing apparatus shown in FIG. 15, a NULL packet is not inserted to control the amount of encoded data so as to match a predetermined bit rate in accordance with the number of stuffing bits. .
[0026]
Since the stuffing amount 111a is a random event, only about half of the bandwidth BR1 necessary to guarantee the maximum value of the stuffing data of the multiplexed layer is statistically generated. In the conventional media information processing apparatus, as shown in FIG. 18, it is necessary to insert a NULL packet into the remaining band corresponding to about half of the band BR1, but as shown in FIG. In this media information processing apparatus, since it is not necessary to insert a NULL packet, the remaining bandwidth corresponding to about half of BR1 is a surplus bandwidth and can be allocated to the encoded media information 103a. That is, the effective bit rate of the encoded media information 103a can be increased from BR0, and as a result, the reproduction quality of the media information can be improved or the amount of information can be increased.
[0027]
The surplus band is not necessarily allocated to the encoded media information 103a, but is allocated to other media information multiplexed together or other media information multiplexed together with the encoded media information. It may be distributed and allocated to 103a.
[0028]
Next, the operation of the media information processing apparatus according to the first embodiment of the present invention will be described only with respect to differences from the conventional media information processing apparatus.
[0029]
Based on the code amount 105 and the stuffing amount 111a, the code amount control means 106a outputs a code amount control signal 107 for controlling the code amount of the encoded media information 103a at a variable bit rate with a peak rate of (BR0 + BR1). .
[0030]
The solid line in FIG. 4 is a diagram illustrating the occupation amount transition of the encoded media information 103a in the reception buffer model assumed in the media information processing apparatus of the first embodiment. Times t0 to t3 represent the timing of decoding or playback / display of the encoded media information 103a. In the transition of the buffer occupancy indicated by the solid line in the figure, the data amount flows into the reception buffer at a variable bit rate, and the code amount BF of the encoded media information 103a consumed at the decoding or playback / display timing is It is instantaneously extracted from the reception buffer. The inflow data amount at each moment is the amount obtained by subtracting the actually generated stuffing data amount BSS from the data amount buffered at the peak rate (BR0 + BR1). Specifically, the inflow data amount from time t0 to t1 is represented by {(BR0 + BR1) × (t1−t0) −BSS (t0)}. Note that stuffing data does not occur at the timing of decoding (or playback / display) of the encoded media information 103a that is not the packetization timing or the timing at which the encoded media information 103a can be divided into packets. The bit rate is the peak rate (BR0 + BR1). Conversely, when the maximum stuffing data is generated, the band BR1 is used by the stuffing data, so the instantaneous bit rate becomes equal to the band BR0 for media information by the conventional media information processing apparatus.
[0031]
As described above, according to the media information processing apparatus of Embodiment 1 of the present invention, compared with the conventional media information processing apparatus, it is possible to effectively widen the band of encoded media information, and as a result, the media The reproduction quality of information can be improved or the amount of information can be increased.
[0032]
Note that the media information processing apparatus shown in FIG. 5 also effectively widens the band of encoded media information in the same band as the conventional media information processing apparatus, similarly to the media information processing apparatus of the first embodiment. As a result, it is possible to improve the reproduction quality of media information or increase the amount of information. The media information processing apparatus of FIG. 1 uses the stuffing amount 111a output from the compensation amount control means 110a of the packetizing apparatus 202a, and the code amount control means 106a of the encoding apparatus 201a encodes encoded media information at a variable bit rate. The media information processing apparatus shown in FIG. 5 includes a compensation amount control unit 110b that determines and outputs a stuffing amount by the same method as the compensation amount control unit 110a. Using the stuffing amount 111b output from the amount control unit 110b, the code amount control unit 106a performs control of the code amount of the encoded media information 103a at a variable bit rate. The other configuration and operation of the media information processing apparatus of FIG. 5 are the same as those of the media information processing apparatus of FIG. According to the configuration of FIG. 5, since it is not necessary to output the stuffing amount from the packetizer 202b to the encoder 201b, the independence of the encoder 201b and the packetizer 202b is enhanced, and the maintainability is improved. Can also be obtained.
[0033]
(Embodiment 2)
FIG. 6 is a block diagram showing an example of the media information processing apparatus according to the second embodiment of the present invention. The internal configuration of the encoding device 201c is different from the conventional media information processing device. Specifically, the encoding apparatus 201c determines and outputs the stuffing amount 111b necessary for dividing the code amount 105 into a predetermined packet size by the same method as the compensation amount control unit 110a. And supplementary means 119 for inserting the stuffing bits of the encoded media information layer into the encoded media information 103a according to the stuffing amount 111b, and the encoded media information 103b based on the code amount 105 and the stuffing amount 111b The difference is that a code amount control unit 106b for controlling the code amount of the encoded media information 103a is provided, and the configuration and operation other than the above are the same as those of the conventional encoding device.
[0034]
Hereinafter, the media information is encoded by the media information processing apparatus according to the second embodiment of the present invention in the band (BR0 + BR1) required when the media information is encoded and multiplexed by the band BR0 by the conventional media information processing apparatus. An example in the case of multiplex and multiplexing will be described. The BR1 is a band necessary for guaranteeing the maximum value of the stuffing data of the multiplexing layer, which is necessary for multiplexing by the conventional media information processing apparatus.
[0035]
According to the media information processing apparatus of the second embodiment of the present invention, since the stuffing bits are inserted into the encoded media information layer in the compensation means 119 in accordance with the stuffing amount 111b, encoding is performed using the stuffing bits in the encoded media information layer. When the media information 103b can be adjusted to an arbitrary size, the data size can be adjusted to an integer multiple of the data size of the fixed-length packet as in the video processing unit 1 in FIG. Is no longer necessary. However, in general, encoded media information is handled in units of 1 byte (8 bits), but in the case of MPEG-4 video (encoded media information), the minimum unit for stuffing is from 9 bits to 32 bits and 1 byte. Therefore, the code amount may not be adjusted well to a data size that is an integral multiple of the data size of the fixed-length packet. In such a case, as in the video processing unit 2 in FIG. 7, until the data size of the encoded media information reaches a maximum size not exceeding an integer multiple of the data size of the fixed-length packet, the encoded media information layer The stuffing data should be inserted. In this way, the maximum value of the stuffing data amount in the multiplexing layer can be kept smaller than the minimum unit of the stuffing data in the encoded media information layer. As described above, according to the configuration of the media information processing apparatus of the second embodiment of the present invention, the maximum value of the stuffing data amount of one multiplexed layer is set as the minimum unit of stuffing data of the encoded media information layer. Since the bandwidth BR2 for guaranteeing the maximum value of the stuffing data of the multiplexed layer by the media information processing apparatus according to the second embodiment of the present invention can be suppressed to be smaller, the bandwidth BR2 is a multiplexing layer by the conventional media information processing apparatus. Can be made smaller than the bandwidth BR1 for guaranteeing the maximum value of the stuffing data. Specifically, when MPEG-4 video is multiplexed with MPEG-2 TS, the maximum value of the stuffing data amount of the multiplexing layer by the conventional media information processing apparatus and the minimum value of the stuffing data amount of MPEG-4 video Therefore, BR2 / BR1 = 1/183 to 3/183. The data structure of FIG. 7 is an example in which one video signal and one audio signal are multiplexed. In general, additional information such as information for synchronization is also multiplexed. Is omitted for simplicity.
[0036]
According to the media information processing apparatus of the second embodiment, since the stuffing amount 111b of the encoded media information layer is a random event, the maximum value of the stuffing data of the multiplexing layer by the conventional media information processing apparatus is guaranteed. Only about half of the bandwidth BR1 necessary for the generation is statistically generated. Further, according to the media information processing apparatus of the second embodiment, the band BR2 for guaranteeing the maximum value of the stuffing data of the multiplexed layer is 1/183 to 3/183 of the band BR1 as in the above example. Since only a certain extent occurs, approximately half of BR1 is the surplus bandwidth. Therefore, according to the media information processing apparatus of the second embodiment, as shown in FIG. 8, the band of encoded media information that can be multiplexed in the band of (BR0 + BR1) is effectively approximately (BR0 + BR1). / 2-BR2), that is, the bandwidth BR0 of the encoded media information that can be multiplexed by the conventional media information processing apparatus can be effectively increased by about (BR1 / 2-BR2).
[0037]
The surplus band is not necessarily allocated to the encoded media information 103a, but is allocated to other media information multiplexed together or other media information multiplexed together with the encoded media information. It may be distributed and allocated to 103a.
[0038]
Next, a code amount control method in the code amount control means 106b of the media information processing apparatus according to the second embodiment will be described.
[0039]
FIG. 9 is a diagram showing a change in the occupation amount of the encoded media information 103b in the reception buffer model assumed in the media information processing apparatus according to the second embodiment. Times t0 to t3 represent the timing of decoding or reproducing / displaying the encoded media information 103a (103b). In the transition of the buffer occupancy indicated by the solid line in the figure, the encoded media information 103b that flows into the reception buffer at a fixed bit rate (BR0 + BR1-BR2) and is consumed at the timing of decoding or playback / display. Code amount (BF + BSM) is extracted from the reception buffer instantaneously. The code amount of the encoded media information 103b is obtained by adding the stuffing data amount BSM of the encoded media information layer inserted at each decoding (or reproduction / display) timing to the code amount BF of the encoded media information 103a. . Since the stuffing data amount BSM of the encoded media layer is a random event, the encoded media information 103b has a fixed bit rate, but the encoded media information 103a has a variable bit rate. Specifically, at the timing of decoding (or reproduction / display) of the encoded media information 103a (103b) that is not at the packetization timing, or at the timing at which the encoded media information 103a can be divided into packets, the stuffing data is Since it does not occur, BSM = 0 and the instantaneous bit rate of the encoded media information 103a is (BR0 + BR1-BR2). On the other hand, when the maximum stuffing data is generated, the band BR1 is used by the stuffing data, so the instantaneous bit rate is the band (BR0-BR2).
[0040]
As described above, according to the media information processing apparatus of Embodiment 1 of the present invention, it is possible to effectively widen the band of encoded media information as compared with the conventional media information processing apparatus. Reproduction quality can be improved, or the amount of information can be increased.
[0041]
In addition, since it is not necessary to output the stuffing amount from the packetizer 202c to the encoder 201c, the independence of the encoder 201c and the packetizer 202c is increased, and the maintainability is improved.
[0042]
(Embodiment 3)
Further, by recording the program for realizing the media information processing method shown in each embodiment on a recording medium such as a flexible disk, the processing shown in each embodiment can be performed independently. It can be easily implemented in a computer system.
[0043]
FIG. 10 is an explanatory diagram when the media information processing method of each of the above embodiments is implemented by a computer system using a program recorded on a recording medium such as a flexible disk.
[0044]
FIG. 10B shows an appearance, a cross-sectional structure, and a flexible disk as viewed from the front of the flexible disk, and FIG. 10A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, the program is recorded in an area allocated on the flexible disk FD.
[0045]
FIG. 10C shows a configuration for recording and reproducing the program on the flexible disk FD. When recording the program for realizing the media information processing method on the flexible disk FD, the program is written from the computer system Cs via the flexible disk drive. When the media information processing method that implements the media information processing method using a program in a flexible disk is built in a computer system, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.
[0046]
In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.
[0047]
(Embodiment 4)
Further, an application example of the media information processing method shown in the above embodiment and a system using the same will be described.
[0048]
FIG. 11 is a block diagram showing the overall configuration of a content supply system ex100 that implements a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
[0049]
The content supply system ex100 includes, for example, a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.
[0050]
However, the content supply system ex100 is not limited to the combination as shown in FIG. 11, and any of the combinations may be connected. In addition, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
[0051]
The camera ex113 is a device capable of moving image shooting such as a digital video camera. In addition, the mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global Mobile System). Alternatively, PHS (Personal Handyphone System) or the like may be used.
[0052]
Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the encoded data transmitted by the user using the camera ex113 can be performed. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server that performs data transmission processing. The moving image data shot by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, the moving image data may be encoded by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated in some storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile phone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.
[0053]
In this content supply system ex100, the content (for example, video shot of music live) taken by the user with the camera ex113, camera ex116, etc. is subjected to media information processing and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decrypt the data subjected to the media information processing. In this way, the content supply system ex100 can receive and play back the encoded data at the client, and can also receive a private broadcast by receiving, decoding, and playing back at the client in real time. It is a system that becomes possible.
[0054]
The media information processing apparatus described in each of the above embodiments may be used for media information processing of each device constituting this system.
[0055]
A mobile phone will be described as an example.
[0056]
FIG. 12 is a diagram illustrating the mobile phone ex115 using the media information processing method described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video from a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording media ex207 and mobile phone ex115 with recording media ex207 And a slot portion ex206 to ability. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a non-volatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.
[0057]
Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. The power supply circuit unit ex310, the operation input control unit ex304, and the image encoding Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / playback unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.
[0058]
When the end of call and the power key are turned on by the user's operation, the power supply circuit unit ex310 starts up the camera-equipped digital cellular phone ex115 in an operable state by supplying power from the battery pack to each unit. .
[0059]
The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, ROM, RAM, and the like. The modulation / demodulation circuit unit ex306 performs spectrum spread processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After conversion into a signal, this is output via the audio output unit ex208.
[0060]
Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.
[0061]
When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.
[0062]
The image encoding unit ex312 and the demultiplexing unit ex308 are configured to include the media information processing apparatus described in the present invention, and the image data supplied from the camera unit ex203 is added to the media information processing apparatus described in the above embodiment. It is converted into packetized encoded image data by compression encoding and packetizing according to the media information processing method used. At the same time, the cellular phone ex115 sends the voice collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 via the voice processing unit ex305 as digital voice data.
[0063]
The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit Spread spectrum processing is performed in ex306, digital analog conversion processing and frequency conversion processing are performed in the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.
[0064]
When receiving data of a moving image file linked to a home page or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Data is sent to the demultiplexing unit ex308.
[0065]
In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into the encoded image data and the audio data, and transmits the encoded data via the synchronization bus ex313. The encoded image data is supplied to the image decoding unit ex309 and the audio data is supplied to the audio processing unit ex305.
[0066]
Next, the image decoding unit ex309 generates reproduction moving image data by decoding the encoded image data, and supplies the reproduced moving image data to the display unit ex202 via the LCD control unit ex302. The moving image data included in the recorded moving image file is displayed. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The
[0067]
It should be noted that the present invention is not limited to the above system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic. As shown in FIG. 14, the media information processing apparatus of the above embodiment is also incorporated into a digital broadcasting system. Can do. Specifically, in the broadcasting station ex409, the encoded bit stream of the video information is transmitted to the communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. Also, the playback device ex403 reads and decodes the encoded bit stream recorded on the storage medium ex402, which is a recording medium, and displays the played video signal on the monitor ex404. Further, a configuration in which an image decoding apparatus is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television is also conceivable. At this time, an image encoding device may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce the moving image on a display device such as the car navigation ex413 that the car ex412 has.
[0068]
For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203 and the camera interface unit ex303 in the configuration illustrated in FIG. 13, and the same may be applied to the computer ex111, the television (receiver) ex401, and the like. . In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats, that is, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.
[0069]
As described above, the media information processing method described in the above embodiment can be used in any of the devices and systems described above, and by doing so, the effects described in the above embodiment can be obtained.
[0070]
【The invention's effect】
As described above, according to the media information processing method of the present invention, it is possible to effectively widen the band of encoded media information as compared with the conventional media information processing apparatus, and as a result, the reproduction quality of the media information is improved. Or the amount of information can be increased, and its practical value is high.
[Brief description of the drawings]
FIG. 1 is a block diagram of a media information processing apparatus according to the present invention.
FIG. 2 is a diagram showing an example of a data structure according to the media information processing method of the present invention.
FIG. 3 is a diagram showing an example of a band breakdown according to the media information processing method of the present invention.
FIG. 4 is a diagram showing an example of a reception buffer occupation amount transition in the media information processing method of the present invention;
FIG. 5 is a block diagram of a media information processing apparatus according to the present invention.
FIG. 6 is a block diagram of a media information processing apparatus according to the present invention.
FIG. 7 is a diagram showing an example of a data structure according to the media information processing method of the present invention.
FIG. 8 is a diagram showing an example of a band breakdown according to the media information processing method of the present invention.
FIG. 9 is a diagram showing an example of a reception buffer occupation amount transition in the media information processing method of the present invention;
FIG. 10 is an explanatory diagram of a recording medium for storing a program for realizing the media information processing method of each of the embodiments by a computer system.
FIG. 11 is a block diagram showing the overall configuration of a content supply system
FIG. 12 is a diagram showing an example of a mobile phone using the media information processing method
FIG. 13 is a block diagram of a mobile phone
FIG. 14 is a diagram showing an example of a digital broadcasting system
FIG. 15 is a diagram showing an example of a data structure according to a conventional media information processing method
FIG. 16 is a block diagram of a conventional media information processing apparatus.
FIG. 17 is a diagram showing an example of transition of reception buffer occupancy in a conventional media information processing method
FIG. 18 is a diagram showing an example of bandwidth breakdown by a conventional media information processing method
[Explanation of symbols]
101 Media information
111a, 111b Stuffing amount
Cs computer system
FD flexible disk
FDD flexible disk drive
102 Coding means
103a Encoded media information
104 Encoding detection means
105 Code amount
106a Compensation amount control means
107 Code amount control signal
108 Code amount detection means
109 Code amount
110a Compensation amount control means
112 Compensation means
113 Encoded media information
114 Packet dividing means
115 Multiplexing means
116 NULL packet generation means
117 NULL packet
118 Multiplexed data
201a encoding apparatus
202a Packetizer

Claims (8)

A media information processing method used in a media information device that receives media information and packetizes the encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit ,
An encoding step for encoding the input media information according to the code amount control signal and outputting the encoded media information;
A first code amount detection step in which a first code amount detection unit detects a first code amount from the encoded media information;
A data amount combining the first code amount and the stuffing amount of the multiplexing layer determined for packetization of the current processing unit and inserted for packetization of the current processing unit by the code amount control means A code amount control step for determining the code amount control signal such that the code amount control signal becomes a predetermined bit rate ;
A second code amount detecting step in which a second code amount detecting means detects a second code amount from the encoded media information;
A compensation amount control means for determining the stuffing amount based on the second code amount, and outputting the determined stuffing amount to the code amount control means;
A compensation means for inserting stuffing data into the encoded media information according to the stuffing amount; and
A packet dividing step for dividing and outputting the encoded media information into which the stuffing data is inserted into packets of a predetermined length;
Media information processing method characterized in that it comprises a. The media information processing method according to claim 1,
Wherein the code amount control step, the media processing method characterized by controlling the code amount of the coded media information at a variable bit rate to the maximum bit rate of the predetermined bit rate. A media information processing method used in a media information device that receives media information and packetizes the encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit ,
An encoding step for encoding the input media information according to the code amount control signal and outputting the encoded media information;
A first code amount detection step in which a first code amount detection unit detects a first code amount from the encoded media information;
The first compensation amount control means is configured such that a data amount obtained by combining the first code amount and the first stuffing amount of the current processing unit does not exceed an integer multiple of a predetermined data size of the packetization. A first compensation amount control step for determining a first stuffing amount of the current processing unit so as to obtain a data size of
A code amount control means for determining a code amount control signal in accordance with the first stuffing amount and the first code amount;
A first compensation step, wherein a first compensation means inserts first stuffing data into the encoded media information according to the first stuffing amount;
A second code amount detecting step for detecting a second code amount from the encoded media information in which the first stuffing data is inserted;
A second compensation amount control means for determining a second stuffing amount of the current processing unit from the second code amount;
A second compensation step, wherein the second compensation means inserts the second stuffing data into the encoded media information into which the first stuffing data is inserted according to the second stuffing amount;
A packet dividing step, wherein the packet dividing means divides the encoded media information into which the second stuffing data has been inserted into packets of a predetermined length and outputs them;
Media information processing method characterized in that it comprises a. A media information processing method according to claim 3,
In the first compensation amount control step, the media processing method characterized by inserting stuffing data of the coded media information layer when the data size is insufficient. A media information processing apparatus which receives media information as input, and packetizes encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit, and outputs the packetized data.
Encoding means for encoding input media information according to a code amount control signal and outputting encoded media information;
First code amount detection means for detecting a first code amount from the encoded media information;
The amount of data that combines the first code amount and the stuffing amount of the multiplexing layer that is determined for packetization of the immediately previous processing unit and is inserted for packetization of the current processing unit becomes a predetermined bit rate. a code amount control means for determining the code amount control signal such that,
Second code amount detection means for detecting a second code amount from the encoded media information;
A compensation amount control means for determining the stuffing amount based on the second code amount, and outputting the determined stuffing amount to the code amount control means ;
Compensation means for inserting stuffing data into the encoded media information according to the stuffing amount;
A media information processing apparatus comprising: packet dividing means for dividing the encoded media information into which the stuffing data is inserted into packets of a predetermined length and outputting the packets. A media information processing apparatus which receives media information as input, and packetizes encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit, and outputs the packetized data.
Encoding means for encoding input media information according to a code amount control signal and outputting encoded media information;
First code amount detection means for detecting a first code amount from the encoded media information;
The current data amount is set so that a data amount obtained by combining the first code amount and the first stuffing amount of the current processing unit does not exceed an integer multiple of the predetermined data size of the packetization. First compensation amount control means for determining a first stuffing amount of a processing unit ;
Code amount control means for determining the code amount control signal according to the first stuffing amount and the first code amount;
First compensation means for inserting first stuffing data into the encoded media information according to the first stuffing amount;
Second code amount detection means for detecting a second code amount from the encoded media information into which the first stuffing data is inserted;
Second compensation amount control means for determining a second stuffing amount of the current processing unit from the second code amount;
Second compensation means for inserting second stuffing data into the encoded media information into which the first stuffing data has been inserted according to the second stuffing amount;
A media information processing apparatus comprising: packet dividing means for dividing the encoded media information into which the second stuffing data is inserted into packets of a predetermined length and outputting them. A media information processing program used in a media information device that receives media information and packetizes the encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit ,
The program is stored on a computer.
An encoding step for encoding the input media information according to the code amount control signal and outputting the encoded media information;
A first code amount detection step in which a first code amount detection unit detects a first code amount from the encoded media information;
A data amount combining the first code amount and the stuffing amount of the multiplexing layer determined for packetization of the current processing unit and inserted for packetization of the current processing unit by the code amount control means A code amount control step for determining the code amount control signal such that the code amount control signal becomes a predetermined bit rate ;
A second code amount detecting step in which a second code amount detecting means detects a second code amount from the encoded media information;
A compensation amount control means for determining the stuffing amount based on the second code amount, and outputting the determined stuffing amount to the code amount control means;
A compensation means for inserting stuffing data into the encoded media information according to the stuffing amount; and
A program, characterized in that a packet division means performs a packet division step of dividing and outputting the encoded media information into which the stuffing data is inserted into packets of a predetermined length . A media information processing program used in a media information device that receives media information and packetizes the encoded media information obtained by encoding the media information into a predetermined data size for each predetermined processing unit ,
The program is stored on a computer.
An encoding step for encoding the input media information according to the code amount control signal and outputting the encoded media information;
A first code amount detection step in which a first code amount detection unit detects a first code amount from the encoded media information;
The first compensation amount control means is configured such that a data amount obtained by combining the first code amount and the first stuffing amount of the current processing unit does not exceed an integer multiple of a predetermined data size of the packetization. A first compensation amount control step for determining a first stuffing amount of the current processing unit so as to obtain a data size of
A code amount control means for determining a code amount control signal in accordance with the first stuffing amount and the first code amount;
A first compensation step, wherein a first compensation means inserts first stuffing data into the encoded media information according to the first stuffing amount;
A second code amount detecting step for detecting a second code amount from the encoded media information in which the first stuffing data is inserted;
A second compensation amount control means for determining a second stuffing amount of the current processing unit from the second code amount;
A second compensation step, wherein the second compensation means inserts the second stuffing data into the encoded media information into which the first stuffing data is inserted according to the second stuffing amount;
A program, characterized in that the packet division means performs a packet division step of dividing and outputting the encoded media information into which the second stuffing data is inserted into packets of a predetermined length .

Images