JPH10290453A - Video transmission method and its system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a system to select optimum image quality data and to reproduce the selected data in response to reproduction processing capability even when data of the same video image with different image quality are transmitted like in simultaneous broadcast. SOLUTION: A base image is given to a base coding section 1, and a high resolution signal is given to a reinforcement signal coding section 2 codes a difference of the base image. Each encoder section transfers a macro block number coded without skipping I frame each to a coding macro block number detection sections 5, 6, and a multiplexer section 3 applies TS-stream multiplex to base and reinforcement signal data and each coded macro block number and transmits the resulting data. An MPEG decode processing section 7 at a receiver side gives information of current arithmetic processing capability to a multiplexer/demultiplexer section 4, where the received data are demultiplexed into the base data and the reinforcement signal and selects the data depending on the macro block processing capability and gives the selected data to the MPEG decode processing section 7.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to digital compression encoding, transmission and reproduction of video signals.
The present invention relates to a technique for simultaneously transmitting the same video to recipients having different reproduction processing capabilities as represented by simultaneous broadcasting.

[0002]

2. Description of the Related Art When transmitting video to a large number of recipients represented by television broadcasting, when providing a video service of higher quality than ever, all the recipients reproduce high quality data. It is difficult to change to a receiver that can be used, and during the transition period, simultaneous broadcasting may be performed so that the same broadcast can be received by both the new and old receivers so that the conventional receiver can also provide service.

[0003] Techniques for performing such simultaneous broadcasting include:
Simply encode video data like each playback device and send them both for each recipient, or send data that can be played back with conventional receivers, reinforcement necessary to play back with high image quality There are methods for generating and sending signals.

[0004] Conventional digital video transmission methods include:
For example, ISO / IEC 13818-2 (commonly known as MPEG2) has an international standard format of digital compression encoded data, and a description of scalable encoding therein.

FIG. 10 shows the configuration of a conventional video transmission device. In FIG. 10, reference numeral 101 denotes a base encoding unit that encodes a base layer of MPEG2, 102 denotes a reinforcement signal encoding unit that encodes a reinforcement signal, and 103 multiplexes data from both encoding units into a system stream. A multiplexing / demultiplexing unit; 104, a demultiplexing unit; 105, a base reproduction processing unit for reproducing a base layer; 106, a reinforcement signal reproduction processing unit for reproducing a reinforcement signal; 107, a base reproduction processing unit 105; A synthesis processing unit that synthesizes a reproduction signal of the reproduction processing unit 106.

The operation of the above-described conventional video transmission device will be described below. As input signals, video signals having the same contents but different resolutions are input. This can be generated by down-sampling the same material. The low-resolution signal is input to the base encoding unit 101 and the high-resolution signal is input to the reinforcement signal encoding unit 102.
In the base encoding unit 101, encoding based on the MPEG2 standard is performed, and compression encoding is performed. Further, the reproduced signal of the encoded video at that time is sent to the reinforcement signal encoding unit 102. The reinforcement signal encoding unit 102 calculates a difference between the high resolution input signal and the upsampled data of the video data encoded by the base layer from the base encoding unit 101, and obtains the difference signal or the original input signal. Compression encoding is performed by selecting one having a smaller code amount. This signal is encoded according to the MPEG2 spatial scalable protocol. The coded base and reinforcement signal data are multiplexed by the multiplexing unit 103 and transmitted. On the receiving side, the signal is demultiplexed into a base signal and a reinforcement signal by the demultiplexing unit 4, and the base reproduction unit 105 and the reinforcement signal reproduction unit 106 respectively.
Sent to In the synthesis processing unit 107, the base reproduction unit 10
5 and the reproduction signal of the reinforcement signal reproduction unit 106 are synthesized to reproduce a high-resolution image. At this time, the reinforcing signal reproducing unit 10
The receiver having no 6 can reproduce only the low-resolution video encoded by the base reproduction processing unit 105. Therefore, in the case where a broadcast is originally performed by a receiver having only a low-resolution reproduction section, a high-resolution broadcast can be realized by adopting such a configuration.

[0007]

However, in the above-described configuration, when performing high-resolution video reproduction, the receiving side needs a reproduction processing unit capable of sufficiently processing a processing amount necessary for possible reproduction. . MPEG2 video data employs predictive coding, and if the prediction error is sufficiently small, the data may not be coded and may be skipped, so that even if the resolution is the same, the amount of calculation required to reproduce the contents differs depending on the content. . On the other hand, a device that reproduces MPEG2 video data also includes a configuration using a programmable arithmetic unit such as a DSP or a CPU. If high-resolution data can be played back but only low resolution can be played back, or in the case of playback on a multitasking computer, if the computing capacity is reduced due to interruption from other processes, etc. There has been a problem in that an insufficient image that has not been completely reproduced because the signal for high resolution cannot be processed is displayed.

[0008] The present invention solves the above-mentioned problems, and a video transmission method and a video coding method for reproducing a video with the highest possible resolution in accordance with the computing power when the reproduction side reproduces using a CPU of a personal computer. It is an object to provide a device and a playback device.

[0009]

In order to achieve the above object, the present invention provides a method for compressing and encoding the same image into data of two or more hierarchies having different computational amounts required for reproduction. The amount of computation required to reproduce the data within the period is calculated for each layer in the data, and the computational amount information is transmitted together. The receiving side can reproduce the hierarchical data that can be reproduced based on the computational amount information in the previous period and its own reproduction capability. Is selected and reproduced.

Further, the number of macro blocks in the encoded data is transmitted as the operation amount information.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION

(Embodiment 1) Hereinafter, Embodiment 1 of the present invention will be described with reference to FIGS.

FIG. 1 shows an image coding apparatus according to a first embodiment of the present invention. In FIG. 1, 1 is a base encoding processing unit, 2 is a reinforcement signal encoding unit, 3 is a multiplexing processing unit,
Reference numeral 4 denotes a number-of-encoded macroblocks detection unit of the reinforcement signal encoding unit, reference numeral 5 denotes a number-of-encoded macroblocks detection unit of the base encoding unit, reference numeral 6 denotes a demultiplexing unit, and reference numeral 7 denotes an MPEG video reproduction processing unit. .

The operation of the above configuration will be described below. First, the data structure of an image at the time of encoding will be described with reference to FIG. Each picture of data to be encoded is divided into macroblocks and encoded. This macro block is data of a 16 × 16 pixel area, and the luminance and chrominance signals are further divided into 8 × 8 blocks, which are basic encoding processing units, and encoded. In the macroblock, the coding method and the quantization width of the blocks in the macroblock are determined.

A slice constitutes one picture by a plurality of slices in a data unit including a plurality of macroblocks. A picture is encoded by a picture to be encoded in a picture (intra picture: hereinafter, I picture), a picture to be predictively encoded from a temporally past picture (hereinafter, P picture), or a temporally past picture. There is a picture to be coded by prediction from both future pictures (hereinafter referred to as a B picture).
The arrangement of pictures in FIG. 2 is a typical example, in which the first I picture is used for predictive coding of the next three pictures, and the pictures included between them are bidirectionally predicted. Therefore, when encoding, first I
It is necessary to encode a picture and then change the arrangement of the P picture and then the B picture in the original time direction. When each macroblock in a picture accompanied by prediction is substantially equal to a predicted image, the macroblock is skipped without being encoded.

Further, a GOP (group of pictures) is composed of a plurality of pictures starting from an I picture, and one video sequence is composed of an arbitrary number of GOPs.

Returning to FIG. 1, as input signals, video signals having the same contents but different resolutions are input. This can be generated by down-sampling the same material.
The low resolution signal is input to the base encoding unit 1 and the high resolution signal is input to the reinforcement signal encoding unit 2. The base encoding unit 1 performs encoding based on the MPEG2 standard and performs compression encoding. At this time, a sequence header is attached to each I frame, and the number of macroblocks coded without skipping between them is determined by a coded macroblock number detector 5.
And record the transfer. At this time, the encoded video reproduction signal is sent to the reinforcement signal encoding unit 2. The reinforcement signal encoding unit 2 calculates the difference between the high-resolution input signal and the upsampled data of the video data encoded by the base layer from the base encoding unit 1, and calculates the difference signal or
One of the original input signals having a smaller code amount is selected and compression-encoded. This signal is encoded according to the MPEG2 spatial scalable protocol. At this time, the structure of the picture is the same as that of the base layer, and a sequence header is attached to each I frame, and the number of coded macroblocks without skipping between them is transferred and recorded to the coded macroblock number detection unit 4. The multiplexing unit 3 multiplexes the coded base and reinforcing signal data and the signals from the coded macroblock number detecting units 4 and 5 according to the MPEG2 transport stream protocol. FIG. 3 is a diagram illustrating the multiplexing process. Each data is divided into 188-byte packets and multiplexed. At this time, the packet including the sequence header of the base and the auxiliary signal is continuously multiplexed in the order of the packet including the base sequence header and the packet including the sequence header of the auxiliary signal. Furthermore, the number of encoding macros is written and transmitted at the private data position of the adaptation field portion in the packet. On the receiving side, the MPE
The G decode processing unit 7 converts the current arithmetic processing capacity into the number of macroblocks that can be processed and sends it to the demultiplexing unit 6. The demultiplexing unit 6 separates the data into a base signal and a reinforcement signal, selects data according to the macroblock processing capacity,
It is sent to the decode processing unit 7. FIG. 4 is a diagram illustrating the determination process. The number of coded macroblocks in the beginning of the sequence is detected, the base and the number of coded macroblocks of the auxiliary signal are added, and if smaller than the arithmetic processing capability, data including the auxiliary signal is sent to the MPEG2 decoding processing unit 7; Play high-resolution images. If it is large,
Only the base data is sent and only the base image is reproduced. MP
The EG decode processing unit 7 is provided with a conventional base playback unit 105.
In addition, the same processing as the processing in the reinforcement signal reproduction unit 106 and the synthesis processing unit 7 is performed.

FIG. 5 is a diagram showing an example in which the base image is a 720 × 480 video signal and the high-resolution image is a 1280 × 720 video signal, and the number of encoded macroblocks in each sequence changes. 15 pictures per sequence
In this case, a maximum of 20250 macroblocks in the base video and 54000 macroblocks in the high-resolution layer are considered. Therefore, in order to always reproduce the base video, a reproduction processing capability of 20250 macroblocks is required. In the case of a reproducing apparatus having an MPEG decoding processing unit 7 having the same processing capability, high-resolution video can be reproduced in the third and fourth sequences as in the path 1. Therefore, for example, in scenes that are relatively close to still images such as works of art and scenery, the number of coded macroblocks is small, and the processing capability is sufficient for scenes in which high-resolution video appears effectively. , It is possible to reproduce at a high resolution even with a reproducing device which is not.

(Embodiment 2) Next, Embodiment 2 of the present invention will be described with reference to FIGS.

FIG. 6 shows the configuration of the video transmission apparatus according to the second embodiment of the present invention. 6, reference numeral 8 denotes a high-resolution encoding processing unit that encodes a high-resolution video, 9 denotes a low-resolution encoding processing unit that encodes a low-resolution video, and 10 denotes a multiplexing unit. Is a coded macroblock detection processing unit, 1
3 is a demultiplexing unit, and 14 is an MPEG reproduction processing unit.

The operation of the above configuration will be described below. As input signals, video signals having the same contents but different resolutions are input. This can be generated by down-sampling the same material. The low-resolution signal is input to the low-resolution encoding unit 9 and the high-resolution signal is input to the high-resolution encoding unit 10.
Encoding conforming to the two standards is performed. At this time, a sequence header is attached to each I frame, and the number of coded macroblocks without skipping between the headers is transferred and recorded to the coded macroblock number detection units 11 and 12. At this time, the encoded video reproduction signal is sent to the reinforcement signal encoding unit 2.

The multiplexing unit 10 multiplexes the encoded high-resolution low-resolution video signal according to the MPEG2 transport stream protocol. FIG. 7 is a diagram illustrating the multiplexing process. Each data is divided into 188-byte packets and multiplexed. At this time, packets including a sequence header are multiplexed adjacently in the order of high resolution and low resolution. Further, the number of encoding macros is written and transmitted at the private data position of the adaptation field portion in the packet. On the receiving side,
The MPEG decoding unit 14 converts the current arithmetic processing capacity into the number of macroblocks that can be processed, and
Send to The demultiplexing unit 13 selects either high-resolution or low-resolution data according to the macroblock processing capability of the playback device, and sends it to the MPEG playback processing unit 14. FIG. 8 is a diagram illustrating the determination process. The number of encoded macroblocks of the packet including the head data of the high-resolution side sequence is detected. If the number of encoded macroblocks is smaller than the processing capacity, the high-resolution data is separated and sent to the MPEG2 reproduction processing unit 14 to reproduce the high-resolution video. If it is large,
The low-resolution side data is demultiplexed and sent to reproduce the low-resolution video. The MPEG decode processing unit 14 reproduces a video signal according to MPEG2 video rules. At this time, if the resolution changes from high resolution to low resolution between the sequences, the low-resolution image is up-sampled and reproduced with continuity in the shape on the screen.

FIG. 9 shows a 720 × 480 low-resolution image.
FIG. 7 is a diagram illustrating an example in which a high-resolution video is a video signal having a spatial resolution of 1280 × 720 and the number of encoded macroblocks in each sequence changes. If the number of pictures per sequence is 15, to always play back,
Up to 2025 for low-resolution video per sequence
0, a processing capacity of 54000 macroblocks in a high-resolution image is required.

Path 1 is a low-resolution reproduction processing capacity 202
This shows a stream selected in a playback device having only 50 macroblock processing capabilities. In a scene where the number of encoded macroblocks is relatively small, high-resolution reproduction becomes possible. The path 2 is a selection path in the case where the high-resolution image reproduction capability is provided (54000 macroblocks per sequence), but the processing capability is temporarily reduced due to interrupt processing of another process or the like. If the processing capacity after the third sequence is reduced to about 30,000 macroblocks, if the high-resolution side is selected as it is, the processing cannot be completed in the fifth sequence. Since a low resolution is selected, data can be correctly reproduced.

In the present embodiment, the reproduction processing capability of the reproduction apparatus is calculated by the number of macroblocks. Since the amount of calculation required for the processing is different, more accurate control can be performed by using the index of the processing capability weighted by each selected encoding. In addition, another index such as the number of times for each operation module such as DCT may be displayed in addition to the number of macro blocks.

In the present embodiment, the present invention is used for transmitting a video having a changed spatial resolution. However, the present invention can be similarly used for transmitting a video having a different image quality such as a time resolution.

In this embodiment, two different image quality transmissions are used. However, three or more transmissions can be used. Further, the switching position of the reproduced image is set for each sequence. However, in the configuration as in the first embodiment, the switching from the high resolution to the low resolution can be performed even in the middle of the sequence, and the processing amount is displayed in a picture unit. It is also possible to switch by picture
The period indicating the reproduction processing capability and the switching position may be performed at another position.

[0027]

As described above, according to the present invention, the amount of arithmetic processing required for the reproduction process is displayed, and the stream corresponding to the capability is selected on the reproduction side, so that the reproduction device can perform the processing in accordance with the capability of the reproduction apparatus. It is possible to reproduce an optimal image. This makes it possible to shift to higher-quality services even in services such as broadcasting. Also, in particular, when the receiver or the like targets a computer or the like that operates in multitasking such as a PC and the processing capacity fluctuates, the reproduced video can be controlled according to the fluctuation in the processing capacity. The practical effects of the present invention are great.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image transmission device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of a data structure of an MPEG video.

FIG. 3 is an explanatory diagram of data generated by a multiplexing unit according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram of a selection process in a demultiplexing unit according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram of an example of selection of reproduction data on the reproduction side according to the first embodiment of the present invention.

FIG. 6 is a configuration diagram of a video transmission device according to a second embodiment of the present invention.

FIG. 7 is an explanatory diagram of processing of a multiplex processing unit according to Embodiment 2 of the present invention.

FIG. 8 is an explanatory diagram of a data selection determination process in a demultiplexing unit according to the second embodiment of the present invention.

FIG. 9 is an explanatory diagram of an example of selection of reproduction data on the reproduction side according to the embodiment of the present invention.

FIG. 10 is a configuration diagram of a conventional image encoding device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Base encoding processing part 2 Reinforcement signal encoding part 3 Multiplexing processing part 4 Encoding macroblock number detection part 5 Encoding macroblock number detection part 6 Demultiplexing part 7 MPEG video reproduction processing part 8 High resolution encoding processing part Reference Signs List 9 low-resolution encoding processing unit 10 multiplexing unit 11, 12 encoded macroblock detection processing unit 13 multiplex separation unit 14 MPEG reproduction processing unit 101 base encoding unit 102 reinforcement signal encoding unit 103 multiplex processing unit 104 multiplex separation unit 105 Base reproduction processing unit 106 Reinforcement signal reproduction processing unit 107 Synthesis processing unit

Claims (10)

[Claims] 1. The compression encoding of the same image into data of two or more hierarchies different in the amount of computation required for reproduction, and the transmission side calculates the amount of computation necessary for reproducing in a predetermined period in each layer. A video transmission method, wherein data is obtained in step (1) and the operation amount information is transmitted together, and the receiving side selects and reproduces reproducible hierarchical data based on the operation amount information and its reproduction capability. 2. The transmission data is a video signal compressed and coded to a data format of the MPEG2 standard, and the receiving side transmits the number of coded macroblocks in the video data as information of operation amount, and transmits the data. 2. The video transmission method according to claim 1, wherein the receiving side reads the operation amount information, selects hierarchical data whose value is smaller than the number of reproducible macroblocks, and performs a reproducing process. 3. The transmitting side sets the number of encoded macroblocks to G
3. The video transmission method according to claim 2, wherein the GOP data is calculated for each OP and transmitted as transmission data before the arrival time of the GOP data at the receiving side. 4. A video reproducing method comprising: reading computation amount information necessary for reproduction; selecting reproducible data for reproduction; 5. The same image is compression-encoded into data of two or more hierarchies having different computational amounts required for reproduction, and for each hierarchical data, a computational amount required for reproducing within a predetermined period is obtained. An image encoding method characterized in that the operation amount information is multiplexed together. 6. The image encoding method according to claim 5, wherein the same image is compression-encoded into two or more hierarchical data having different resolutions. 7. A computing amount information reading processing unit for reading computation amount information necessary for reproduction and a video reproduction processing unit, wherein the computing amount information reading processing unit
A video playback device, wherein the video playback processing unit selects hierarchical data having a calculation amount that can be played back by the video playback processing unit, and the video playback processing unit plays back the selected hierarchical data. 8. The operation amount reading processing unit further includes an MPE
8. The image reproducing apparatus according to claim 7, further comprising a separation processing unit for performing a desired stream separation process from G2 standard system stream data, separating the selected video data from the system stream, and sending the separated video data to the video reproduction processing unit. 9. A video compression encoding unit comprising a video compression encoding unit and a multiplex processing unit, wherein the video compression encoding unit compresses and encodes the same image into data of two or more hierarchies having different computational amounts required for reproduction. Means, and for each coded hierarchical data, a calculation amount calculation means required to reproduce the data within a predetermined period, wherein the multiplex processing unit multiplexes the calculation amount information on the encoded data. Video encoding device. 10. A video encoding device encodes the same image having different resolutions into a base video and a reinforcement signal by MPEG2 standard spatial scalable encoding, and a calculation amount calculating unit encodes each video sequence. 10. The image encoding apparatus according to claim 9, wherein the number of macroblocks is calculated as operation amount information.