JP2004120415A - Device and method for converting moving picture data stream - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the processing costs in converting an encoded stream of an MPEG-2 system having a higher frame rate (for example, 30 fps) into an encoded stream of an MPEG-4 system having a lower frame rate (10 fps, for example). <P>SOLUTION: A stream converting device 4 receives the encoded stream of the 30 fps MPEG-2 system from a content server 2, converts the frame rate and the encoding system of the encoding stream, and transmits the encoded stream of the 10 fps MPEG-4 system via a network 3. In these processes, the stream converting device 4 does not decode a frame which is not encoded in the encoding stream of MPEG4-4 system from the encoding stream of the MPEG-2 system. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a moving image data stream conversion device and method for performing a moving image data frame rate conversion process and a moving image data encoding method conversion process.
[0002]
[Prior art]
Real-time distribution of moving image data via a network such as the Internet has begun to be performed. When real-time distribution of moving image data is performed, usually, moving image data as a source is recorded in a content server, and is transmitted from the content server to a client via a network. In addition, the moving image data recorded in the content server is used not only for real-time distribution, but also for direct display without using a network, or for use as a material video of another content.
[0003]
By the way, when real-time distribution of moving image data is performed via a network, a transmission rate of the network is limited. Generally, it is transmitted at, for example, about 10 fps). However, in consideration of the fact that moving image data recorded on a content server is directly displayed without going through a network or used as a material video of another content, the display rate of a normal moving image is reduced. It may be recorded at 30 fps. When moving image data having a frame rate of 30 fps is recorded in the content server in this way, in order to transmit the moving image data via a network, the frame rate must be down-converted by real-time processing and distributed. No.
[0004]
When recording moving image data distributed via a network on a recording medium such as a hard disk, recording at a normal moving image display rate (30 fps) is better than recording at a frame rate as it is. It may be desirable in consideration of reproduction processing and the like. In such a case, it is necessary to record the frame rate of the received moving image data while upconverting the frame rate by real-time processing.
[0005]
In recent years, a standard called MPEG-4 has been proposed as a moving image compression method suitable for network distribution. However, most of the conventional video assets are MPEG-2 format data. Considering that video is directly reproduced from a recording medium without passing through a network, moving image data encoded in the MPEG-4 format is considered. In some cases, it may be more convenient to record moving image data encoded by the MPEG-2 system on a recording medium than to record the moving image data on a recording medium. Therefore, even when the moving image data of the MPEG-4 system is distributed via the network, the moving image data of the MPEG-2 system is recorded on the content server, and the encoding system is appropriately converted. It is considered that they are often distributed.
[0006]
In view of the above situation, when performing network distribution, a stream converter capable of performing frame rate conversion processing and conversion processing of an encoding method between MPEG-2 and MPEG-4 is required. It is thought that it becomes.
[0007]
FIG. 8 shows a configuration diagram of a conventionally proposed stream converter.
[0008]
A stream converter 110 shown in FIG. 8 performs encoding and decoding of an MPEG-2 encoded stream, and performs encoding and decoding of an MPEG-4 encoded stream. An MPEG-4 encoding / decoding unit 112 for performing the operation and a frame memory 113 are provided.
[0009]
The stream converter 110 converts an MPEG-2 encoded stream with a frame rate of 30 fps stored in the content server 101 into an MPEG-4 encoded stream with a frame rate of 10 fps. In the case of transmission via an external device, an operation as shown in FIG. 9 is performed.
[0010]
The stream converter 110 reads an MPEG-2 encoded stream of 30 fps as shown in FIG. 9A from the content server 101 and decodes the stream by the MPEG-2 encoding / decoding unit 111. 30 fps baseband moving image data as shown is generated. The generated 30 fps baseband moving image data is temporarily stored in the frame memory 113. Subsequently, the stream converter 110 performs a thinning process of an arbitrary frame on the frame memory 113 as shown in FIG. 9C, and baseband moving image data having a frame rate of 10 fps as shown in FIG. 9D. Generate Then, the stream converter 110 reads the baseband moving image data with the frame rate of 10 fps shown in FIG. 9D from the frame memory 110, encodes the read moving image data by the MPEG-4 encoding / decoding unit 112, and outputs And generates an MPEG-4 encoded stream as shown in FIG.
[0011]
Also, the stream converter 110 converts the MPEG-4 coded stream having a frame rate of 10 fps received via the network 102 into an MPEG-2 coded stream having a frame rate of 30 fps. When the data is stored in the memory 101, an operation as shown in FIG. 10 is performed.
[0012]
The stream converter 110 receives an MPEG-4 encoded stream having a frame rate of 10 fps from the network 102 as shown in FIG. 10A, and decodes the stream by the MPEG-4 encoder / decoder 112. 10B, the baseband moving image data of 10 fps is generated. The generated 10 fps baseband moving image data is temporarily stored in the frame memory 113. Subsequently, the stream converter 110 performs frame copy processing on the frame memory 113 as shown in FIG. 10C, and generates baseband moving image data with a frame rate of 30 fps as shown in D in FIG. . Then, the stream converter 110 reads the baseband moving image data having the frame rate of 30 fps shown in D of FIG. 10 from the frame memory 110, encodes the read data by the MPEG-2 encoding / decoding unit 111, and outputs And generates an MPEG-4 encoded stream as shown in FIG.
[0013]
[Problems to be solved by the invention]
However, when the frame rate is converted by the stream converter 110 as described above, the processing amount obtained by adding the decoding processing at 30 fps and the encoding processing at 10 fps at the time of down-conversion from 30 fps to 10 fps is a processing cost. Will be spent. Also, at the time of up-conversion from 10 fps to 30 fps, the processing amount obtained by adding the decoding processing at 10 fps and the encoding processing at 30 fps is consumed as the processing cost.
[0014]
The present invention has been proposed in view of such an actual situation, and the present invention has been proposed in accordance with the first data stream obtained by encoding a moving image signal of a first frame rate using a first encoding method. When converting a moving image data stream that generates a second data stream obtained by encoding the moving image signal of the second frame rate lower than the frame rate of It is an object of the present invention to provide a stream conversion apparatus and method for reducing the number of streams.
[0015]
In addition, from the first data stream obtained by encoding the video signal of the first frame rate using the first encoding method, the video signal of the second frame rate higher than the first frame rate is obtained. It is an object of the present invention to provide a stream conversion apparatus and a stream conversion method that reduce processing costs when converting a moving image data stream that generates a second data stream encoded using the second encoding method.
[0016]
[Means for Solving the Problems]
A stream conversion device and method according to the present invention are configured to convert a video signal of a first frame rate from a first data stream encoded using a first encoding scheme to a second data stream lower than the first frame rate. A second data stream is generated by encoding the moving image signal having the frame rate of the above by using the second encoding method. In the present invention, the video signal is decoded from the first data stream without decoding a frame that is not coded in the second data stream, and the decoded video signal is converted to a second coding scheme. And generating a second data stream by encoding the data stream.
[0017]
In addition, the stream conversion device and method according to the present invention are configured such that a moving image signal having a first frame rate is higher than the first frame rate from a first data stream obtained by encoding the moving image signal using a first encoding method. A second data stream is generated by encoding the video signal at the second frame rate using a second encoding scheme. In the present invention, the moving image signal is decoded from the first data stream, and the decoded moving image signal is encoded using a second encoding scheme to generate a second data stream. In order to increase the frame rate by repeatedly inserting the frame image, the encoding process is performed on the inserted frame image to indicate that the frame image is a repetition image from the previous image.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
A stream converter to which the present invention is applied will be described as an embodiment of the present invention. A stream converter according to an embodiment of the present invention converts an encoded stream of a moving image encoded by the MPEG-2 method into an encoded stream of a moving image encoded by the MPEG-4 method, and An encoding system conversion process for performing the inverse conversion is possible. Further, the stream conversion device of the present embodiment can convert the frame rate of the encoded stream of the moving image.
[0019]
In the stream conversion device according to the embodiment of the present invention, for example, as shown in FIG. 1, an MPEG-2 coded stream of a first frame rate (for example, 30 fps) is input from a content server 2 as an input stream S1. By converting the encoding scheme and the frame rate of the input stream S1, an MPEG-4 encoded stream having a second frame rate (for example, 10 fps) lower than the first frame rate is generated, and the generated code is generated. The present invention is applied to a stream conversion device 1 that distributes a converted stream via a network 3 as an output stream S2.
[0020]
In addition, for example, as shown in FIG. 2, the stream conversion apparatus according to the embodiment of the present invention converts the MPEG-4 encoded stream of the second frame rate (for example, 10 fps) from the network 3 as the input stream S1. By converting the encoding method and the frame rate of the input stream S1 that has been input, the MPEG-2 encoded stream of the first frame rate (for example, 30 fps) is generated, and the generated encoding is performed. The present invention is applied to a stream conversion device 1 that stores a stream as an output stream S2 in a content server 2.
[0021]
FIG. 3 shows an internal configuration diagram of the stream conversion device 1 described above.
[0022]
The stream converter 1 includes an input unit 11 that performs an input process on an input stream S1 input from the content server 2 or the network 3, an output unit 12 that outputs an output stream S2 that is output to the content server 2 or the network 3, An image input / output unit 13 for performing input / output processing of baseband moving image data, an encoding / decoding unit 14 for performing encoding and decoding of an encoded stream, and a frame memory 15 for storing baseband moving image data. , A memory interface 16 serving as an interface of the frame memory 15.
[0023]
The stream converter 1 inputs and outputs an MPEG-2 encoded stream or an MPEG-4 encoded stream as an input stream S1 and an output stream S2.
[0024]
The input unit 11 receives an encoded stream of MPEG-2 or MPEG-4 from the content server 2 or the network 3 and supplies the stream to the encoding / decoding unit 14.
[0025]
The output unit 12 transmits the encoded stream of MPEG-2 or MPEG-4 output from the encoding / decoding unit 14 to the content server 2 or the network 3.
[0026]
The image input / output unit 13 transmits / receives baseband moving image data to / from an external device.
[0027]
The encoding / decoding unit 14 is a shared circuit that can perform encoding and decoding processing for both MPEG-2 and MPEG-4. Further, the encoding / decoding unit 14 can perform, for example, time-division processing or the like based on control from an external controller or the like, and can perform encoding and decoding simultaneously and in parallel. The processing for streams of different modes of −4 can also be performed simultaneously in parallel. Also, for example, a plurality of encoding processes and a plurality of decoding processes for a plurality of encoded streams can be performed.
[0028]
The frame memory 15 stores baseband moving image data. The frame memory 15 reads and writes moving image data via the memory interface 16. The encoding / decoding unit 14 stores moving image data generated by decoding the encoded stream in the frame memory 15 in frame units. Further, the encoding / decoding unit 14 reads out the moving image data stored in the frame memory 15 on a frame basis and performs an encoding process. Further, the image input / output unit 13 reads the baseband moving image data stored in the frame memory 15 and outputs it to an external device, and also outputs the baseband moving image data input from the external device to the frame memory 15. To be stored.
[0029]
Next, the configuration of the encoding / decoding unit 14 will be described.
[0030]
The encoding / decoding unit 14 includes a variable-length encoding / decoding unit 21, a quantization / inverse quantization unit 22, a DCT / IDCT processing unit 23, a motion compensation unit 24, a subtractor 25, and an adder 26. , A motion detecting section 27 and a controller 28.
[0031]
The variable length encoding / decoding unit 21 performs encoding and decoding of a variable length code and a fixed length code. When performing the encoding process, the variable-length encoding / decoding unit 21 receives the quantized DCT coefficient data from the quantization / inverse quantization unit 22 and receives various types of synthesizing data such as motion vectors from the controller 28. Taxes are input, and a variable-length code or a fixed-length code is encoded for them. The variable-length encoding / decoding unit 21 performs a variable-length code and a fixed-length code encoding process, forms a stream according to the MPEG-2 or MPEG-4 standard, and outputs the formed stream to the output unit 12. Supply. When performing a decoding process, the variable-length encoding / decoding unit 21 receives an encoded stream from the input unit 11, analyzes the structure of the encoded stream that has been input, and divides the stream. A variable-length code decoding process or a fixed-length code decoding process is performed on the obtained data. The variable length encoding / decoding unit 21 supplies the DCT coefficient to the quantization / inverse quantization unit 22 and supplies the syntax such as the motion vector to the controller 28.
[0032]
The quantization / inverse quantization unit 22 performs a quantization process and an inverse quantization process. When performing the encoding process, the quantization / inverse quantization unit 22 receives DCT coefficient data from the DCT / IDCT processing unit 23 and uses a predetermined quantization parameter for the input DCT coefficient data. The quantization processing is performed, and the quantized DCT coefficient data is supplied to the variable-length encoding / decoding unit 21. When performing a decoding process, the quantization / inverse quantization unit 22 receives DCT coefficient data from the variable-length encoding / decoding unit 21 and performs a predetermined quantization parameter on the input DCT coefficient data. , And supplies the inversely quantized DCT coefficient data to the DCT / IDCT processing unit 23.
[0033]
The DCT / IDCT processing unit 23 performs a discrete cosine transform process and an inverse discrete cosine transform process. When performing the encoding process, the DCT / IDCT processing unit 23 inputs the image data to the spatial region output from the subtracter 25 in frame units, performs a discrete cosine transform process on the input image data. , DCT coefficient data. The DCT / IDCT processing unit 23 supplies the generated DCT coefficient data to the quantization / inverse quantization processing unit 22 during the encoding processing. When performing a decoding process, the DCT / IDCT processing unit 23 receives DCT coefficient data from the quantization / inverse quantization unit 22, performs inverse discrete cosine transform on the input DCT coefficient data, and performs spatial domain Is generated. When performing the decoding process, the DCT / IDCT processing unit 23 supplies the generated image data of the spatial region to the adder 26.
[0034]
The motion compensator 24 reads the reference frame stored in the frame memory 15, performs motion compensation on the reference frame according to the motion vector, and generates a predicted frame.
[0035]
Here, when performing the encoding processing, the encoding / decoding unit 14 performs local decoding processing such as inverse quantization, IDCT processing, and motion compensation on the DCT coefficient data once subjected to the quantization processing. , And stores the locally decoded image data in the frame memory 16. Further, the motion compensating unit 24 generates a predicted frame using the locally decoded image data as a reference frame. When performing the decoding process, the motion compensation unit 24 reads the already decoded reference frame from the frame memory 15 and generates a predicted frame. The motion compensation unit 24 supplies the predicted frame to the subtractor 25 when performing the encoding process, and supplies the predicted frame to the adder 26 when performing the decoding process.
[0036]
When performing the encoding process, the subtracter 25 receives a frame to be encoded from the frame memory 16. The subtractor 25 subtracts the predicted frame supplied from the motion compensation unit 24 from the frame to be encoded. The subtracter 25 supplies the frame after the subtraction to the DCT / IDCT processing unit 23.
[0037]
When performing a decoding process, the adder 26 receives image data from the DCT / IDCT processing unit 23 in frame units, and adds a prediction frame supplied from the motion compensation unit 24 to the input image data. The adder 26 stores the added frame in the frame memory 15.
[0038]
The motion detection unit 27 calculates a motion vector between frames on a macroblock basis based on image data in the frame memory 16. The calculated motion vector is supplied to the motion compensator 24 and also to the controller 28.
[0039]
The controller 28 generates a syntax other than the image data included in the encoded stream, and controls each circuit in the encoding / decoding unit 14.
[0040]
Next, an MPEG-2 encoded stream with a frame rate of 30 fps stored in the content server 2 is converted to an MPEG-4 encoded stream with a frame rate of 10 fps, and transmitted via the network 3. In this case, the operation of the stream converter 1 will be described with reference to FIG.
[0041]
The stream converter 1 reads an MPEG-2 coded stream of 30 fps as shown in FIG. The read encoded stream of the MPEG-2 system is supplied to the encoding / decoding unit 14 via the input unit 11. The encoding / decoding unit 14 decodes the input MPEG-2 encoded stream to generate baseband moving image data. The generated baseband moving image data is stored in the frame memory 15.
[0042]
At this time, the encoding / decoding unit 14 performs a decoding process on only a frame necessary for MPEG-4 encoding, and discards an uncoded frame without decoding. It is desirable that the frame to be discarded does not affect the decoding process of another frame even if it is deleted, such as a B picture. For example, in the case of a coded stream having a general GOP configuration in which an I-picture or a P-picture appears once every three frames, as shown in B in FIG. 4, the stream is discarded without decoding the B-picture. , I or P pictures only. By performing decoding in this manner, baseband moving image data with a frame rate of 10 fps can be generated.
[0043]
Then, the encoding / decoding unit 14 reads the moving image data of the baseband having the frame rate of 10 fps shown in FIG. 4B from the frame memory 15 and encodes the moving image data by the MPEG-4 method, as shown in FIG. An MPEG-4 encoded stream is generated and distributed to the network 3.
[0044]
By performing the above-described processing, the stream conversion apparatus 1 can reduce the amount of decoding processing and reduce the overall processing cost.
[0045]
Next, a case in which an MPEG-4 encoded stream at a frame rate of 10 fps received via the network 3 is converted into an MPEG-2 encoded stream at a frame rate of 30 fps and stored in the content server 2 The operation of the stream converter 1 will be described with reference to FIG.
[0046]
The stream converter 1 receives an MPEG-4 coded stream having a frame rate of 10 fps from the network 3 as shown in FIG. The received MPEG-4 encoded stream is supplied to the encoding / decoding unit 14 via the input unit 11. The encoding / decoding unit 14 decodes the input MPEG-4 encoded stream to generate 10 fps baseband moving image data as shown in FIG. 5B. The generated baseband moving image data is stored in the frame memory 15. Subsequently, the encoding / decoding unit 14 reads and encodes 10 fps baseband moving image data stored in the frame memory 15 to generate an MPEG-2 encoded stream.
[0047]
At this time, the encoding / decoding unit 14 inserts a new frame at an arbitrary position to change the frame rate from 10 fps to 30 fps. As a new frame to be inserted, a past frame is used repeatedly as it is. The encoding / decoding unit 14 performs encoding such that a new frame used repeatedly is constituted by a skipped macroblock defined by the MPEG-2 encoding method. A skipped macroblock is a macroblock in which, when the current image data is the same as the data of the previous macroblock in time, the macroblock is skipped and the actual image data is not transmitted. For example, as shown in FIG. 5C, images B0, B1, B3, and B4 are assumed to be frames composed of skipped macroblocks. By using a frame composed of skipped macroblocks in this way, it is not necessary to perform an operation such as DCT processing on the frame, and the amount of processing can be reduced. Then, the encoding / decoding unit 14 stores the encoded MPEG-2 stream of 30 fps as shown in FIG. 5C in the content server 2 via the output unit 12 as described above.
[0048]
By performing the above-described processing, the stream conversion device 1 can reduce the processing amount of the encoding processing and reduce the overall processing cost.
[0049]
As described above, the stream converter 1 according to the embodiment of the present invention can reduce the amount of processing when converting the frame rate as compared with the conventional stream converter.
[0050]
Note that the stream conversion apparatus 1 according to the embodiment of the present invention not only converts the applications shown in FIGS. 1 and 2 but also, for example, as shown in FIG. 6, an MPEG-4 encoded stream having a frame rate of 10 fps. An MPEG-2 encoded stream received at the network 3 and having a frame rate of 30 fps is read from the content server 2 and the read encoded stream is converted into an MPEG-4 encoded stream having a frame rate of 10 fps. It can be converted and distributed via the network 3.
[0051]
For example, as shown in FIG. 7, an MPEG-4 encoded stream having a frame rate of 10 fps is received via the network 3, and the received encoded stream is converted into two 30 fps of MPEG-1 and MPEG-2. , An MPEG-1 encoded stream can be recorded on a removable medium such as the flash memory 4, and an MPEG-2 encoded stream can be recorded on the content server 2.
[0052]
【The invention's effect】
In the stream conversion apparatus and method according to the present invention, a second data stream lower than the first frame rate is converted from a first data stream obtained by encoding a moving image signal of a first frame rate using a first coding scheme. When generating a second data stream obtained by encoding the moving image signal having the frame rate of the second data stream using a second encoding scheme, the decoding apparatus does not decode a frame that is not encoded in the second data stream. The moving image signal is decoded from the first data stream, and the decoded moving image signal is encoded using a second encoding scheme to generate a second data stream.
[0053]
Thus, the stream conversion apparatus and method according to the present invention can reduce the processing cost.
[0054]
In addition, the stream conversion device and method according to the present invention are configured such that a moving image signal having a first frame rate is higher than the first frame rate from a first data stream obtained by encoding the moving image signal using a first encoding method. When generating a second data stream obtained by encoding the video signal of the second frame rate using the second encoding scheme, when increasing the frame rate by repeatedly inserting the same frame image Performs encoding processing on the inserted frame image to indicate that the frame image is a repetitive image from the previous image.
[0055]
Thus, the stream conversion apparatus and method according to the present invention can reduce the processing cost.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a first application example of a stream conversion device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a second application example of the stream conversion device according to the embodiment of the present invention.
FIG. 3 is a block diagram of a stream converter according to the embodiment of the present invention.
FIG. 4 is a diagram illustrating a case where an MPEG-2 encoded stream with a frame rate of 30 fps is converted into an MPEG-4 encoded stream with a frame rate of 10 fps by the stream converter according to the embodiment of the present invention. It is a figure which shows the data etc. which are performed.
FIG. 5 is a diagram illustrating a case where an MPEG-4 encoded stream with a frame rate of 10 fps is converted into an MPEG-2 encoded stream with a frame rate of 30 fps by the stream converter according to the embodiment of the present invention. It is a figure which shows the data etc. which are performed.
FIG. 6 is a diagram showing a first application example of the stream conversion device according to the embodiment of the present invention.
FIG. 7 is a diagram illustrating a second application example of the stream conversion device according to the embodiment of the present invention.
FIG. 8 is a block diagram of a conventional stream converter.
FIG. 9 shows data and the like generated when a conventional stream converter converts an MPEG-2 encoded stream having a frame rate of 30 fps into an MPEG-4 encoded stream having a frame rate of 10 fps. FIG.
FIG. 10 shows data generated when a conventional stream converter converts an MPEG-4 encoded stream at a frame rate of 10 fps into an MPEG-2 encoded stream at a frame rate of 30 fps. FIG.
[Explanation of symbols]
1 stream converter, 2 content server, 3 network, 11 input unit, 12 output unit, 13 image input / output unit, 14 encoding / decoding unit, 15 frame memory, 16 memory interface

Claims (4)

Decoding means for receiving a first data stream obtained by encoding a moving image signal of a first frame rate using a first encoding method, and decoding the moving image signal from the input first data stream; ,
The moving image signal decoded by the decoding means is encoded at a second frame rate lower than the first frame rate using a second encoding method to generate a second data stream. Encoding means for outputting a second data stream,
The decoding device according to claim 1, wherein the decoding unit does not decode a frame that is not encoded by the encoding unit. Decoding means for receiving a first data stream obtained by encoding a moving image signal of a first frame rate using a first encoding method, and decoding the moving image signal from the input first data stream; ,
The moving picture signal decoded by the decoding means is encoded at a second frame rate higher than the first frame rate by using a second encoding method, and a second data stream is generated. Encoding means for outputting a data stream of
When the encoding unit increases the frame rate by repeatedly inserting the same frame image, the encoding unit performs an encoding process on the inserted frame image to indicate that the image is a repetition image from the previous image. A moving image data stream conversion device, characterized in that: From a first data stream obtained by encoding a video signal of a first frame rate using a first encoding scheme, the video signal of a second frame rate lower than the first frame rate is converted into a second video stream. A method of converting a moving image data stream that generates a second data stream encoded using the encoding method of
Decoding the video signal from the first data stream without decoding frames that are not encoded in the second data stream;
A method of converting a moving image data stream, comprising: coding the decoded moving image signal using a second coding method to generate a second data stream. From a first data stream obtained by encoding a video signal of a first frame rate using a first encoding scheme, the video signal of a second frame rate higher than the first frame rate is converted to a second video stream. A method of converting a moving image data stream that generates a second data stream encoded using the encoding method of
Decoding the video signal from the first data stream;
If the decoded video signal is encoded using a second encoding scheme to generate a second data stream and the same frame image is repeatedly inserted to increase the frame rate, A moving image data stream conversion method comprising: performing a coding process on a frame image to be performed to indicate that the frame image is a repetitive image from a previous image.

Images