JP3934772B2 - Variable transfer rate encoding method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a video signal encoding method for encoding a moving image signal (video signal) and a corresponding video signal encoding device, and more particularly to a video signal encoding method and device for performing encoding using orthogonal transform and quantization. Applicable toEncode encoded data at a fixed transfer rateConvert to variable rate encoded data after temporary recording(Re-encoding)Variable transfer rate encoding method and deviceIn placeRelated.
[0002]
[Prior art]
  According to the technique described in Japanese Patent Laid-Open No. 7-284097 as an example of a conventional variable transfer rate encoding technique, a video signal is divided into a first pass and a second pass, and the first pass is encoded. Information necessary for encoding the second pass is generated and output. A so-called MPEG system is used as the encoding system at this time.
[0003]
  As for MPEG, ISO-IEC11172-2, ITU-T H.264, etc. Since the detailed description is made in H.262 / ISO-IEC13818-2, only the outline will be described here.
[0004]
  MPEG is the name of the organization that examines the video coding standard established in 1988 by ISO / IEC JTC1 / SC2 (International Organization for Standardization / International Electrotechnical Standards Meeting Technical Committee 1 / Technical Committee 2, Current SC29) ( Abbreviation for Moving Pictures Expert Group. MPEG1 (MPEG Phase 1) is a standard for storage media of about 1.5 Mbps. JPEG (Joint Photographic Coding Experts Group) for the purpose of still image coding and ISDN (IntegratedServicesDigitalNetwork (Integrated Service Digital Communication Network) for video conferencing for video conferences and low transfer rates for videophones. It inherits the basic technology of H.261 (CCITT SGXV, standardized by the current ITU-T SG15), and introduces a new technology for storage media. These were established in August 1993 as ISO / IEC 11172.
[0005]
  MPEG1 is created by combining several technologies.
[0006]
  From the input image signal, the time redundant portion is reduced by taking the difference between the image signal decoded by the motion compensator and the input image signal.
[0007]
  The prediction method has three basic modes: a mode for performing prediction from a past image, a mode for performing prediction from a future image, and a mode for performing prediction from both past and future images. Exists. In addition, these modes are a 16 pixel × 16 pixel macroblock (MB: Macro).Bcan be switched for each lock). The prediction method is determined by the picture type (Picture_Type) given to the input image. There are two modes: a mode in which prediction is performed from a past image and a mode in which the macroblock is independently encoded without prediction, and a unidirectional inter-picture prediction encoded image (P picture: P-picture). In addition, a mode for predicting from future images, a mode for predicting from past images,
There are four modes: a mode that performs prediction from both past and future images, and a mode that independently encodes without prediction. Bidirectional inter-picture predictive encoded images (B-picture: B-Picture) ). In addition, it is an intra-picture independent encoded image (I-picture) that encodes all macroblocks independently. Note that the intra-picture independent coded image is called an intra picture, and therefore, the unidirectional inter-picture predictive coded image and the bidirectional inter-picture predictive coded image can be called non-intra pictures.
[0008]
  In motion compensation, the motion region is subjected to pattern matching for each macroblock, a motion vector is detected with half-pel accuracy, and the motion vector is predicted by shifting by the detected motion vector. The motion vector has a motion vector in the horizontal direction and the vertical direction, and is transmitted as additional information of the macroblock together with an MC (Motion Compensation) mode indicating where the motion vector is predicted.
[0009]
  FIG. 8 shows a basic configuration of a video signal encoding apparatus to which MPEG1 is applied.
[0010]
  In FIG. 8, an input image signal is supplied to an input terminal 101, and this input image signal is sent to a calculator 102 and a motion compensation predictor 111 described later.
[0011]
  The computing unit 102 obtains the difference between the image signal decoded by the motion compensation predictor 111 and the input image signal, and sends the difference image signal to the DCT unit 103.
[0012]
  The DCT unit 103 orthogonally transforms the supplied difference image signal. Here, DCT (Discrete Cosine Transform) is an orthogonal transformation in which an integral transformation using a cosine function as an integral kernel is a discrete transformation into a finite space. In MPEG, two-dimensional DCT is performed on an 8 × 8 DCT block obtained by dividing a macroblock into four. In general, a video signal has many low-frequency components and few high-frequency components. Therefore, when DCT is performed, coefficients are concentrated in a low frequency.
[0013]
  Data (DCT coefficient) obtained by DCT in the DCT unit 103 is quantized in the quantizer 104. In the quantization in the quantizer 104, a quantized value is obtained by multiplying a value obtained by weighting an 8 × 8 two-dimensional frequency called a quantization matrix with a visual characteristic and a value called a quantization scale for multiplying the whole by a scalar. Then, the DCT coefficient is divided by the quantized value.
[0014]
  When the encoded data after being encoded by the video signal encoding device is decoded and dequantized later by a video signal decoding device (decoder) (not shown), the video signal encoding device By multiplying by the used quantized value, a value approximating the original DCT coefficient can be obtained.
[0015]
  The data quantized by the quantizer 104 is variable length encoded by the VLC unit 105. In the VLC unit 105, among the quantized values, the direct current (DC) component is subjected to DPCM (one of predictive coding).DifferencialPulseCodeModulation). Also, for alternating current (AC) components, a so-called zigzag scan is performed from low to high, and the zero run length and effective coefficient value are considered as one event, with the highest occurrence probability. So-called Huffman coding is performed, in which codes having a short code length are assigned.
[0016]
  Data variable-length encoded by the VLC unit 105 is temporarily stored in the buffer memory 106, then read out from the buffer memory 106 at a predetermined transfer rate, and output as encoded data (encoded bit stream). Output from the terminal 107.
[0017]
  The generated code amount for each macroblock in the output encoded data is the buffer manager 11 described later.4Is transmitted to the code amount controller 113. The code amount controller 113 obtains a difference between the generated code amount for each macroblock and the target code amount, generates a code amount control signal corresponding to the difference, and feeds back to the quantizer 104, thereby generating the generated code. Perform quantity control. The code amount control signal fed back to the quantizer 104 for the code amount control is a signal for controlling the quantization scale in the quantizer 104. A specific code amount control method will be described later.
[0018]
  On the other hand, the quantized image data is sent to the inverse quantizer 108 where it is inversely quantized.
[0019]
  Further, the DCT coefficient data obtained by the inverse quantization is sent to the inverse DCT unit 109 and subjected to inverse DCT, and then the arithmetic difference unit 112 adds the prediction difference image from the motion compensation predictor 111 to generate an image signal. Is restored.
[0020]
  The restored image signal is temporarily stored in the image memory 110 and then sent to the motion compensation predictor 111. The image signal sent from the image memory 110 to the motion compensated predictor 111 is
It is used for generating a reference decoded image for calculating a difference image in the computing unit 102.
[0021]
  An encoded bit stream that is encoded data output from the output terminal 107 is:
In the case of a video signal, each picture has a variable length code amount. This is because MPEG uses information conversion such as DCT, quantization, and Huffman coding, as well as adaptively changing the code amount allocated to each picture in order to improve image quality. That is,
In MPEG, since motion compensation prediction is performed, the encoded image itself is encoded such that the input image signal is encoded as it is in some cases and the difference image signal that is the difference between the predicted image and the input image signal is encoded in some cases. This is because the entropy of a large change. In this case, in many cases, the code amount control is performed while allocating to the image entropy ratio and keeping the buffer memory capacity limit.
[0022]
  Therefore, the buffer manager 114Monitors the relationship between the code amount generated by encoding and the usable encoding rate, and sets the target code amount in the buffer memory 106 so as to be within a predetermined buffer capacity.
[0023]
  Information corresponding to the difference of the actual generated code amount with respect to the target code amount is fed back to the variable length encoder 105 and enters the code amount controller 113. The code amount controller 113 suppresses the generated code amount by increasing the value of the quantization scale set in the quantizer 106, or conversely reduces the generated code amount by decreasing the value of the quantization scale. The code amount control signal is generated.
[0024]
  As described above, when variable length data is transferred as fixed transfer rate data (fixed transfer rate encoded data), the maximum buffer amount on the video signal decoding device side that receives the data is This is the upper limit value of the generated code amount in the video signal encoding device. That is, the encoded data is input at a constant speed and stored at a predetermined value, and then a predetermined time (in the case of an NTSC video signal, which is one of the television standard broadcasting systems, 1 / 29.97 sec unit). In this case, a virtual decoder model that performs decoding in an instant is used, and encoding is performed on the encoding device side so that neither overflow nor underflow occurs in the virtual buffer memory (so-called VBV buffer) of the model. It is defined by MPEG. If this rule is observed, the rate in the VBV buffer changes locally, but if the observation time is long, it becomes a fixed transfer rate, and MPEG defines this as a fixed rate.
[0025]
  Here, in the case of the fixed transfer rate defined as described above, when the generated code amount is small on the encoding device side, the buffer occupancy amount sticks to the upper limit value on the decoding device side. In this case, for example, the amount of code must be increased on the encoding device side so that an invalid bit is added and the buffer (VBV buffer) does not overflow.
[0026]
  On the other hand, in the case of the variable transfer rate, the definition of this fixed transfer rate is expanded, and when the buffer occupancy reaches the upper limit value, reading in the decoding device is stopped, so that overflow does not occur in principle. Is defined as Therefore, in the case of the variable transfer rate, even if the amount of generated code is very small, reading of the decoding device is stopped, so there is no need to insert invalid bits as in the case of the fixed transfer rate. For this reason, in the case of a variable transfer rate, encoding is performed so that only underflow does not occur.
[0027]
  Against the background of such a technique, in a conventional variable transfer rate encoding, a video signal is encoded by being divided into a first pass and a second pass, and information necessary for encoding the second pass in the first pass. Will be described.
[0028]
  FIG. 9 shows a configuration for encoding the first pass of the conventional example. In FIG. 9, parts that operate in the same manner as the components of the basic configuration shown in FIG.
[0029]
  In the first path configuration shown in FIG. 9, a video signal is reproduced and input to the input terminal 101. At this time, generated code amount information generated for each short section with respect to the entire video sequence is added to the encoded information.
[0030]
  This generated code amount is calculated by the code amount counter 121 after VLC in the VLC unit 105, and is sent to the storage circuit 122 for storage.
[0031]
  The storage circuit 122 may be anything such as a high-speed storage medium such as a hard disk or an optical disk.
[0032]
  Since the first pass is used to monitor whether or not accurate compression has been performed, the buffer memory 106 and the output of the encoded data thereafter are shown in FIG. If the code amount can be counted, it is not always necessary.
[0033]
  By the way, in the encoding method represented by MPEG, since variable length encoding is performed, if the first pass encoding is performed with a fixed quantization width, the complexity of the encoded image and motion compensation are reduced. The generated code amount increases in accordance with the difference (residual component) amount.
[0034]
  Therefore, if the generated code amount is distributed using this property, the image quality can be made substantially uniform. In the second pass encoding described later, the generated code amount must be controlled to the entire target code amount while maintaining the code amount distribution ratio.
[0035]
  The generated code amount is detected in units of short sections generated in the first pass encoding, and the information is stored in the storage circuit 122. As an example of the short interval, 1 GOP (group of pictures) of about 15 pictures can be considered, which can be divided for each intra-picture independent coding. in this case,
The amount of generated code for each GOP unit is stored in the storage circuit 122.
[0036]
  In the first pass encoding, it is common to generate a larger amount of code than the final code amount output in the second pass, generally with a small and fixed quantization width. Thus, the reason why the quantization width is reduced in the first pass encoding is that it is necessary to finely decompose the information up to the high frequency component of the image and detect the characteristics of the image.
[0037]
  Here, it is assumed that the code amount generated in the i-th short interval in the first pass is PS1B (i). The ratio of the code amount PS1B (i) generated in each short section is substantially the same in the second pass, and the final target total code amount is set as the target code amount in each short section in the second pass. .
[0038]
  For example, if the short interval is 1 GOP, the code can be controlled while maintaining a certain level of image quality by the following method.
[0039]
  Next, FIG. 10 shows a configuration for performing the second pass encoding of the conventional example. In FIG. 10, parts that operate in the same manner as the components of the basic configuration shown in FIG. 8 are given the same reference numerals, and descriptions thereof are omitted.
[0040]
  In the configuration of the second pass shown in FIG. 10, the quantization scale is decreased to increase the generated code amount, while the quantization scale is increased to decrease the generated code amount.
[0041]
  A method for controlling the quantization width based on the occupation ratio of the buffer memory 106 using this principle can be considered. It should be noted that an image (P picture) to be unidirectionally inter-picture predictively encoded is predicted from the previous intra-picture independent encoded image (I picture) or P picture, and between two-way pictures. The predicted coded image (B picture) is predicted from the I picture and P picture on both sides in terms of time, so that, for example, when the I picture deteriorates, the other pictures also deteriorate in conjunction.
[0042]
  In the following method example, the entire code amount control is realized while considering the code amount distribution for these pictures.
[0043]
  First, the target code amount determination circuit 124 uses the generated code amount PS1B (i) obtained in the first pass from the storage circuit 122, and uses the generated code amount PS1B (i) of each GOP as shown in the following equation. )
  PS2B (i) = final target total code amount × PS1B (i) / ΣPS1B (i)
[0044]
  Here, when the target code amount PS2B (i) to be given to one GOP is R, the specific code amount control is performed by the code amount controller 113 by an algorithm such as the following first and second steps. .
[0045]
(A) First step
  In the first step, as shown in the following equation, the allocated code amount for each picture of the GOP is distributed with a certain weight assigned to the pictures that have not yet been encoded in the GOP.
      Xi = Si × Qi
      Xp = Sp × Qp
      Xb = Sb × Qb
[0046]
  Here, X is called a global complexity measure, which is defined by the product of the generated code amount S of the previous encoding result of the same picture type and the average quantization scale Q, i represents an I picture, p represents a P picture, and b represents a B picture. In addition, the quantization scale that achieves ideal image quality has a ratio of Kp = 1.0 with respect to the P picture when the I picture is the reference, and a ratio with the B picture when the I picture is also used as the reference. Assume that Kb = 1.4.
[0047]
  At this time, the code amounts Ti, Tp, and Tb of each picture assigned in the first step are obtained by the following equations.
    Ti = MAX {R / (1+ (NpXp / XiKp) + NbXb / XiKb)), br / (8 * pr)}
    Tp = MAX {R / (Np + (NpKpXb / KbXp)), br / (8 * pr)}
    Tb = MAX {R / (Nb + (NpKbXp / KbXp)), br / (8 * pr)}
[0048]
  In each equation, Ti represents the code amount of the I picture, Tp represents the code amount of the P picture, Tb represents the code amount of the B picture, MAX is the maximum, and R is the initial value given to the GOP. Np represents the number of uncoded pictures of P pictures in the GOP, Nb represents the number of uncoded pictures of B pictures in the GOP, pr represents the picture rate, and br represents the bit rate.
[0049]
  Here, the initial code amount R is updated as follows each time encoding progresses in the GOP.
      R = R-Si, p, b
[0050]
(B) Second step
  In the second step, the code amount (Ti, Tp, Tb) of each picture assigned in the first step is matched with the actual generated code amount, so that it is generated for each macroblock as shown in the following equations. While adding the code amount, the difference between the target code amount and the predicted target code amount halfway is fed back to the quantization scale in units of macroblocks.
      dji = dOi + Bj-1− (Ti (j-1) / MB_cnt)
      djp = dOp + Bj-1− (Tp (j-1) / MB_cnt)
      djb = dOp + Bj-1− (Tb (j-1) / MB_cnt)
[0051]
  In each equation, dOi is the initial occupation amount of the virtual buffer (VBV buffer) in the I picture, dOp is the initial occupation amount of the virtual buffer (VBV buffer) in the P picture, and dOb is the virtual buffer (VBV) in the B picture. Buffer), j is a number indicating the number of each picture counted from the top of each picture counted by the code quantity counter 121, and Bj is the jth number from the top of each picture counted by the code quantity counter 121. MB_cnt represents the number of macroblocks in one picture, dji represents the feedback amount in the I picture, djp represents the feedback amount in the P picture, and djb represents the feedback amount in the B picture.
[0052]
  Further, the average quantization scale Q is obtained by the following equation.
      Q = dj × 31 / r
      R = 2 × br / pr
  In the equation, Q is a quantization scale, and r is a parameter that determines the response speed of feedback.
[0053]
  In the conventional configuration, it is possible to perform code amount control by performing each calculation as described above.
[0054]
  As for MPEG, ISO-IEC 11172-2, ITU-T H.264, etc. Since detailed description is made in H.262 / ISO-IEC13818-2, further description is omitted here.
[0055]
[Problems to be solved by the invention]
  Thus, in the conventional configuration, in order to realize variable transfer rate encoding, the above-described two-pass encoding must be passed.
[0056]
  That is, as shown in FIGS. 9 and 10, the video signal to be encoded has to be input twice to the video signal encoding device. For this reason, variable transfer rate encoding cannot be performed on an image signal that is transmitted only once, such as a moving image signal transmitted in real time by broadcasting or communication.
[0057]
  Also, for example, when editing encoded data, it must be re-encoded, or in order to make the encoded data compliant with MPEG, in addition to the part that is actually edited, There were problems that required a lot of work, such as having to partially correct the frame.
[0058]
  The present invention has been made in view of the above-described problems. For example, it is possible to perform encoding at a variable transfer rate even for a moving image signal transmitted in real time by broadcasting or communication. Variable transfer rate encoding method and device that are easy to editSetFor the purpose of provision.
[0059]
[Means for Solving the Problems]
  In order to solve the above problems, the present invention provides the following method and apparatus.
(1) A variable transfer rate encoding method for encoding a video signal using orthogonal transform and quantization and outputting the video signal at a variable transfer rate,
  Encoding an incoming video signal using orthogonal transform and quantization;
  Simultaneously with the encoding of the incoming video signal, the encoded encoded dataFor each image unitGenerated code amount andAverage per image unitQuantization width and detectedFor each image unitGenerated code amount andAverage per image unitGenerating encoded information having information on quantization width;
  Recording the encoded encoded data at a fixed transfer rate and recording the encoded information;
  Decoding the recorded encoded data;
  Based on the recorded encoded information and the reproduction time of the encoded data and the target average variable transfer rate, or based on the encoded information and the target code amount,For each image unitSet a new target code amount,For each image unitA variable transfer rate encoding step for variable-length encoding the data obtained by decoding the encoded data so that the generated code amount becomes the new target code amount;
A variable transfer rate encoding method comprising:
(2) In the recording step, parameters related to motion compensation at the time of encoding in the encoding step are recorded,
  In the variable transfer rate encoding step, when the data obtained by decoding the recorded encoded data is variable-length encoded, the recorded parameter for motion compensation is used as a parameter for motion compensation. The variable transfer rate encoding method according to (1) above.
(3) Provide a step for entering editing information,
  In the step of generating the encoding information, the encoding information of only a necessary section based on the editing information is generated,
  In the decoding step, the data of only the necessary section based on the editing information is decoded,
  In the variable transfer rate encoding step, the encoding information of only the necessary section based on the editing information is used as the encoding information, and the data decoded only in the necessary section based on the editing information is encoded. The variable transfer rate encoding method according to (1) or (2) above, wherein the variable transfer rate encoding method is used.
(4) A variable transfer rate encoding device that encodes a video signal using orthogonal transform and quantization and outputs the video signal at a variable transfer rate,
  An incoming video signal encoding means for encoding the incoming video signal using orthogonal transform and quantization;
  Simultaneously with the encoding of the incoming video signal, the encoded encoded dataFor each image unitGenerated code amount andAverage per image unitQuantization width and detectedFor each image unitGenerated code amount andAverage per image unitEncoded information detecting means for generating encoded information having information on the quantization width;
  Recording means for recording the encoded data at a fixed transfer rate and recording the encoded information;
  Decoding means for decoding the recorded encoded data;
  Based on the recorded encoded information and the reproduction time of the encoded data and the target average variable transfer rate, or based on the encoded information and the target code amount,For each image unitSet a new target code amount,For each image unitVariable transfer rate encoding means for variable-length encoding data obtained by decoding the encoded data so that the generated code amount becomes the new target code amount;
A variable transfer rate encoding device comprising:
(5) The incoming video signal encoding means outputs a parameter relating to motion compensation during encoding,
  The recording means records parameters relating to the motion compensation;
  The variable transfer rate encoding means uses the recorded parameter for motion compensation as a parameter for motion compensation when variable length encoding is performed on data obtained by decoding the recorded encoded data. The variable transfer rate encoding apparatus according to (4), characterized in that:
(6) Editing information input means for inputting editing information;
  Detection control means for controlling the encoded information detection means based on the editing information;
  A decoding control means for controlling the decoding means based on the editing information;
  In the encoded information detection means, under the control of the detection control means, generate encoded information for only the necessary section based on the edit information,
  In the decoding means, under the control of the decoding control means, the encoded data is decoded only in a necessary section based on the editing information,
  In the variable transfer rate encoding means, encoded information of only a necessary section based on the editing information is used as the encoding information, and only a necessary section is decoded based on the editing information as data to be encoded. The variable transfer rate encoding device according to (4) or (5), wherein
[0060]
DETAILED DESCRIPTION OF THE INVENTION
  Hereinafter, a variable transfer rate encoding method and apparatus according to the present invention will be described.SetPreferred embodiments will be described in detail with reference to the drawings.
[0061]
  FIG. 1 shows a variable transfer rate encoding method and apparatus according to the present invention.Reference exampleShows a basic configuration of a video signal encoding apparatus. In addition,Reference exampleNow, an example using MPEG1 encoding will be described as an example of a moving image signal encoding method.
[0062]
  This smellAnd biThe input image signal input to the input terminal 1 of the video signal encoding device is a video signal composed of a luminance signal and a color difference signal, and after being digitized, the images are rearranged according to the picture type. It is what. The input image signal is recorded in the storage circuit 22 as encoded data, and further, a schematic configuration example until the encoded data recorded in the storage circuit 22 is converted (re-encoded) into variable transfer rate encoded data. This will be described with reference to FIG.
[0063]
  The input image signal supplied to the input terminal 1 is sent to the calculator 2 and the motion compensation predictor 11.
[0064]
  The motion compensated predictor 11 performs motion compensation prediction on the input image signal in the coding order, and the computing unit 2 calculates the difference between the input image signal and the predicted image from the motion compensated predictor 11.
[0065]
  The DCT unit 3 performs DCT on the difference image data obtained by the calculation in the calculator 2.
[0066]
  The DCT coefficient from the DCT unit 3 is quantized by the quantizer 4. The quantized data is sent to the VLC unit 5 together with the motion vector and the encoding mode from the motion compensated predictor 11, and is variable length encoded (VLC) by the VLC unit 5.
[0067]
  The encoded data obtained by the variable length encoding in the VLC unit 5 is temporarily stored in the buffer memory 6 and then output from the buffer memory 6 as an MPEG video stream.
[0068]
  In addition, the rate controller 23 monitors the sufficiency of the buffer memory 6, and basically,
An encoding control signal that coarsens the quantization when the degree of fullness of the buffer memory 6 increases and finer the quantization when the buffer memory 6 decreases is fed back to the quantizer 4.
That is, the encoding control signal fed back to the quantizer 4 is a control signal for controlling the quantization width.
[0069]
  here,In the apparatus shown in FIG.However, as described above, the input image signal is encoded at a fixed transfer rate defined by MPEG. The encoding rate depends on the capacity of the recording medium and the reproduction time of the input image signal to be recorded. However, it is desirable that the encoding rate be as high as possible within the conditions. In other words, better image quality is desirable. This is because the encoding rate becomes equal to the maximum transfer rate under the condition that is encoded in the variable transfer rate encoded data conversion described later.
[0070]
  For this reason,Device shown in FIG.Then, the encoded data output from the buffer memory 6 is recorded in the storage circuit 22. In addition,Device shown in FIG.The storage circuit 22 used in 1 can be anything such as a recordable / reproducible hard disk, optical disk, or high-speed storage medium.
[0071]
  On the other hand, since the I picture and P picture need to be used later as reference images for motion compensation prediction, the quantized data of the I picture and P picture output from the quantizer 4 is transmitted to the inverse quantizer 8 and the subsequent ones. Is also sent.
[0072]
  That is, the DCT coefficient data obtained by the inverse quantization in the inverse quantizer 8 is sent to the inverse DCT unit 9 and subjected to inverse DCT, and then the prediction difference from the motion compensated predictor 11 in the arithmetic unit 12. The images are added to restore the image signal.
[0073]
  The restored image signal is temporarily stored in the image memory 10. The restored image signal stored in the image memory 10 is the same image signal that is reproduced in a later video signal decoding apparatus.
[0074]
  The image signal stored in the image memory 10 is sent to the motion compensation predictor 11 and becomes a reference image for the next motion compensation prediction. That is, the image signal sent from the image memory 10 to the motion compensation predictor 11 is used for generating a reference decoded image for calculating a difference image in the computing unit 2.
[0075]
  next,As shown in FIG.In the video signal encoding apparatus, the encoded data recorded in the storage circuit 22 is sent to the encoded information detector 24. The encoded information detector 24 detects the generated code amount and quantization width of each picture from the encoded data, and sends the generated code amount and quantization width to the storage circuit 22 for recording again. Specific coding information recorded in the storage circuit 22 may include picture information having a format as shown in FIG. The picture information portion is recorded in the storage circuit 22 in the order of encoding for the number of pictures.
[0076]
  Here, compressed data by MPEG encoding is determined so that a 4-byte picture start code is attached to the head of a picture. This picture start code is a byte-aligned unique code that can be distinguished from other data such as “0 × 00000100”.detectionThe device 24 detects the picture start code first, and then counts the code amount until the picture start code is detected, thereby calculating the generated code amount of the picture.
[0077]
  Further, although the generated code amount depends on the encoding rate, it corresponds to 15 Mbps, has a maximum of about 1.75 Mbits, and needs only about 1000 bits of accuracy.
Accordingly, the storage circuit 22 is configured to record about 2 bytes of information in the encoded picture order, considering byte alignment with information of about 11 to 12 bits / picture.
[0078]
  Further, as the quantization width information, it is ideal to obtain the sum or average value of quantization widths determined for each block of 16 pixels × 16 pixels called a macroblock, but the so-called NTSC broadcast system 720 × 480 is used. In the case of a picture of pixels, the average of 1350 macroblocks must be taken. In this case, the compressed data is transferred to the relatively deep part of the MPEG layer, which is originally a macroblock layer.CAlthough it is necessary to perform (modulation decoding), the slice quantization width at the head of the slice layer for one column of the macroblock can be used as the quantization width information in order to increase the speed. There are 30 slice quantization widths (480/16) in the case of a picture of 720 × 480 pixels in the NTSC broadcasting system, and the slice start code “0 × 00000101” which is a unique code as with the picture start code. It is possible to detect between “0 × 0000011E”. In addition,
The slice quantization width is indicated by 5 bits immediately after the slice start code.
[0079]
  Encoding informationdetectionIn the unit 24, the generated code amount and the quantization width value described above are 30.detectionAnd the sum or average of them is taken. Since the quantization width is indicated by 5 bits from 1 to 31 in both the macroblock layer and the slice layer, the quantization width information is a value that can be expressed by 2 bytes. For this reason, in the memory circuit 22, the quantization width information is added after the generated code amount information and recorded.
[0080]
  In the configuration of FIG. 1, after encodingMarksGenerated code amount and quantization width information from encoded datadetectionAs an example, the generated code amount and quantization width information are simultaneously obtained with the encoding in the VLC unit 5.detectionYou may make it do.
[0081]
  Simultaneously with the encoding by the VLC unit 5, the generated code amount and the quantization width information are encoded information.detectionIn vessel 24detectionFIG. 3 shows a basic configuration for doing so. In FIG. 3, parts that operate in the same manner as the components of the configuration shown in FIG.
[0082]
  Encoding information having the configuration shown in FIG.detectionIn the unit 24, for example, by using the encoded data when the VLC unit 5 performs variable length encoding, for example, by counting the code between picture start codes, quantization for each macroblock at the time of encoding is performed. Sum or average widthInspection OutLike to do. Of course, the encoding informationdetectionIn the unit 24, instead of obtaining the quantization width between the picture start codes, as described above, it is also possible to obtain the slice quantization width between the slice start codes. The encoded informationdetectionIn vessel 24detectionThe generated generated code amount and quantization width information are recorded in the storage circuit 22.
[0083]
  In the configuration of FIG. 1 and FIG. 3 described above, generated code amount information and quantization width information recorded in the storage circuit 22, MarksUse encoded data, etc., as described later.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)Like to do.
[0084]
  That is, the basic configuration of FIGS. 1 and 3 is recorded in the storage circuit 22 in addition to the configuration described above.TickEncoded dataRecoverOn the basis of the decoder 40 that converts the image data into the generated image data, the generated code amount information and the quantization width information that are also recorded in the storage circuit 22, and the target average variable transfer rate or information about the target code amount, which will be described later, SoRecoveryA variable transfer rate encoder 50 for converting the image data into variable transfer rate encoded data and outputting it from the output terminal 7. The variable transfer rate encoded data output from the output terminal 7 is not shown.NoteIt will be recorded on the recording medium.
[0085]
  FIG. 4 shows the generated code amount information and quantization width information recorded in the storage circuit 22., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)More specifically, the configuration of the first specific example to be performed, that is, the decoder 40 and the variable transfer rate encoder 50 of FIG. 1 and FIG. 3 and their peripheral circuits (not shown in FIG. 1 and FIG. 3). A typical configuration is shown. In FIG. 4, the storage circuit 22 is also shown for convenience of explanation.
[0086]
  In FIG. 4, it is read from the memory circuit 22TickThe encoded data is decoded by the decoder 40., RecoveryIt is sent to the variable transfer rate encoder 50 as signal image data.
[0087]
  This variable transfer rate encoder 50 is suppliedRecoveryThe image data is sent to the calculator 52 and the motion compensation predictor 61.
[0088]
  In the motion compensated predictor 61, RecoveryMotion compensation prediction is performed on the image data in the order of encoding., RecoveryThe difference between the image data and the predicted image from the motion compensated predictor 61 is calculated.
[0089]
  The DCT unit 53 performs DCT on the difference image data obtained by the calculation in the calculator 52.
[0090]
  The DCT coefficient from the DCT unit 53 is quantized by the quantizer 54. The quantized data is sent to the VLC unit 55 together with the motion vector and the encoding mode from the motion compensated predictor 61, and is variable length encoded (VLC) by the VLC unit 55.
[0091]
  The encoded data obtained by the variable length encoding in the VLC unit 55 is temporarily stored in the buffer memory 56, and then output from the output terminal 57 as the variable transfer rate encoded data. The output of FIG. 1 or FIG. Sent to terminal 7. The output variable transfer rate encoded data is recorded on a recording medium (not shown).
[0092]
  On the other hand, since the I picture and P picture need to be used later as a reference image for motion compensation prediction, the quantized data of the I picture and P picture output from the quantizer 54 is transmitted to the inverse quantizer 58 and later. Is also sent.
[0093]
  That is, the DCT coefficient data of the I picture and P picture obtained by the inverse quantization by the inverse quantizer 58 is sent to the inverse DCT unit 59 and subjected to inverse DCT, and then the motion compensation prediction is performed by the arithmetic unit 62. The prediction difference image from the unit 61 is added to restore the image signal.
[0094]
  The restored image signal is temporarily stored in the image memory 60. The restored image signal stored in the image memory 60 is the same image signal that is reproduced in a later video signal decoding apparatus.
[0095]
  The image signal stored in the image memory 60 is sent to the motion compensation predictor 61 to be a reference image for the next motion compensation prediction. In other words, the image signal sent from the image memory 60 to the motion compensation predictor 61 is used by the calculator 52 to generate a reference decoded image for calculating a difference image.
[0096]
  The code amount counter 62 calculates a generated code amount after VLC in the VLC unit 55 and supplies picture code amount information indicating the generated code amount to the code amount control circuit 63.
[0097]
  The code amount control circuit 63 controls the quantization step in the quantizer 54 based on picture code amount information from the code amount counter 62 and picture target code amount information described later.
[0098]
  On the other hand, the encoded information including the generated code amount information and quantization width information read from the storage circuit 22 is input to the target code amount determination circuit 26 not shown in FIGS.
[0099]
  The target code amount determination circuit 26 is set with a target code amount of variable transfer rate encoded data from, for example, a CPU (central processing unit) 25. The CPU 25 is provided outside, but can also be provided inside. The target code amount set by the CPU 25 is going to be converted into variable transfer rate encoded data.MarkIf the reproduction time of the encoded data is known, it is equivalent to inputting the average variable transfer rate.
[0100]
  Here, an example of the algorithm for determining the coding amount in the target code amount determination circuit 26 will be described below.
[0101]
  For example, as described above, encoded informationdetectionContainer 24 isdetectionThe generated code amount for each picture is BIT (i), and the average quantization width of the entire picture at that time is Q (i), MarksConvert encoded data to variable transfer rate encoded data(Re-encoding)After that, the entire target code amount is set to TB. Note that (i) indicates a picture number. Using these, give each picture of variable transfer rate encoded dataEyesThe standard code amount TG (i) can be obtained by the following equation.
    EN (i) = BIT (i)^0.8× Q (i)
    TG (i) = TB × EN (i) / ΣEN (i)
[0102]
  Given to each picture of variable transfer rate encoded data obtained by this formulaEyesThe target code amount (picture target code amount information) TG (i) is sent to the code amount control circuit 63 of the variable transfer rate encoder 50. Note that EN in the formula isdetectionThis is almost proportional to the picture complexity and the amount of error at the time of motion compensation, and represents the difficulty of encoding. By increasing the code amount when the EN value is high and decreasing the code amount when the EN value is small, it is possible to assign the code amount with a constant image quality. Give to each picture of variable transfer rate encoded dataEyesStandard code amount TG(I)In this ratio, the entire target code amount TB after being converted into the variable transfer rate encoded data is distributed. Similarly to the conventional example described above, the code amount ratio may be simply distributed to the generated code amount ratio, and the code amount control in the picture can be realized by the method after the second step described above.
[0103]
  In the code amount control circuit 27, the picture target code amount TG (i) obtained as described above,
The code amount is controlled by controlling the quantization scale in the quantizer 54 based on the picture code amount information counted by the code amount counter 62.
[0104]
  According to the video signal encoding apparatus of the present embodiment, the above processing is performed.TickConvert encoded data to variable transfer rate encoded data(Re-encoding)Making it possible.
[0105]
  1 and 3, the configuration shown in FIG. 5 is used instead of the configuration shown in FIG., MarksConvert encoded data to variable transfer rate encoded data(Re-encoding)It is also possible to do.
[0106]
  Ie,FIG. 5 shows generated code amount information and quantization width information recorded in the storage circuit 22., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)The structure of the 2nd specific example for doing is shown. In FIG. 5, parts that operate in the same manner as the components shown in FIG. 4 are given the same reference numerals, and descriptions thereof are omitted.
[0107]
  In the configuration of FIG. 5, it is recorded in the storage circuit 22 as in the example of FIG.TickEncoding information based on encoded datadetectionThe generated code amount and quantization width of each picture by the unit 24detectionOr encoding the generated code amount and quantization width information while encoding as in the example of FIG.detectionIn vessel 24detectionIn the configuration toMarkUsed inMarksParameters related to motion compensation for each macroblock described in the encoded data are recorded in the storage circuit 22, and are stored in the variable transfer rate encoder 50.RecoveryWhen the signal image data is subjected to variable transfer rate encoding, the parameters relating to motion compensation recorded in the storage circuit 22 are used.
[0108]
  The difference between the configuration of FIG. 5 and the configuration of FIG. 4 is that the parameters related to motion compensation read from the storage circuit 22 are input to the encoded information input unit 64 of the variable transfer rate encoder 50 as motion compensation related information. Thus, the motion compensation predictor 61 of the variable transfer rate encoder 50 uses this motion compensation related information to perform motion compensation prediction.
[0109]
  Here, specific examples of the parameters relating to motion compensation recorded in the storage circuit 22 include a motion vector for each macroblock and a motion compensation type. That is, specific coding information stored in the storage circuit 22 of the configuration example shown in FIG. 5 includes macro information as shown in FIG. 6 in addition to the picture information in the format as shown in FIG. The block information portion is recorded in the raster order from the upper left to the lower right of the picture by the number of macro blocks.
[0110]
  By doing so, it is recorded in the memory circuit 22.MarkEven if the encoded data has some degree of encoding degradation, for example,MarksWhen the motion compensated predictor 61 obtains the motion vector using the decoded image obtained by decoding the encoded data by the decoder 40, it is not disturbed by the encoding degradation noise. In addition, the amount of processing for obtaining a motion vector can be reduced. In addition to this, for example, in the case of extending to so-called MPEG2, it is sufficiently effective to record information such as adaptive switching between frames and fields.
[0111]
  Further, in the configuration shown in FIGS. 1 and 3, the configuration shown in FIG. 7 is used instead of the configuration shown in FIGS., MarksConvert encoded data to variable transfer rate encoded data(Re-encoding)It is also possible to do.
[0112]
  Ie,FIG. 7 shows the generated code amount information and quantization width information recorded in the storage circuit 22., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)The structure of the 3rd specific example for doing is shown. In FIG. 7, parts that operate in the same manner as the components shown in FIG. 4 and FIG.
[0113]
  That is, in the configuration of FIG. 7, FIG.MarksObtained by encodingTickFor example, when the encoded data is subjected to an editing process, the variable transfer rate encoder 50 performs the encoding according to the editing information used for the editing process at the time of the variable transfer rate encoding. To. As specific editing information, for example, information on a format consisting of hours: minutes: seconds: frames indicating time information of an image for which editing is to be enabled can be cited.
[0114]
  More specifically, in FIG. 7, edit information generated by combining the edit start point and the edit end point is input from the edit information input device 41, and the edit information is sent to the decoding controller.
[0115]
  The decoding controller 42 controls the decoder 40 so as to decode and reproduce only the time portion of the image for which editing is to be enabled, which is described in the editing information. Accordingly, the decoder 40 at this time is supplied from the storage circuit 22.MarkOf the encoded data, only the time portion of the image for which the editing described in the editing information is to be validated is decoded and reproduced.
[0116]
  In the case of the configuration of FIG. 7, the editing information from the editing information input device 41 isdetectionIt is also input to the controller 43. thisdetectionThe controller 43 sets the generated code amount and quantization width of each picture only for the time portion of the image for which editing is to be enabled.detectionEncoding informationdetectionThe device 24 is controlled.
Therefore, the generated code amount information and quantization width information of each picture for only the time portion of the image for which editing is to be enabled are recorded as encoded information in the storage circuit 22 at this time.
[0117]
  Further, in the target code amount determination circuit 26 in the case of the configuration of FIG. 7, the generated code amount information and quantization width information of each picture for the time portion of the image for which editing is to be enabled, stored in the storage circuit 22. Based on the encoded information and the target code amount or average variable transfer rate from the CPU 25, the picture target code amount for only the time portion of the image for which editing is to be validated is calculated.
[0118]
  For this reason, the variable transfer rate encoder 50 can perform variable transfer rate encoding only for the time portion of the image for which editing is to be enabled.
[0119]
  In such an editing process, for example, when the editing point is an image portion of a P picture, the image needs to be encoded as an I picture. In addition, when the edit point is an image portion of a B picture, the motion compensation information before and after the edit point needs to be invalidated. Therefore, in such a case, processing such as encoding the related picture as an I picture is necessary.
[0120]
  According to the video signal encoding apparatus of the present embodiment to which the variable transfer rate encoding method and apparatus of the present invention is applied, the configuration as described above is used.TickConvert encoded data to variable transfer rate encoded data(Re-encoding)It is possible to do.
[0121]
  Also,BookThe variable transfer rate encoded data generated as described above by the video signal encoding apparatus of the embodimentOn a recording medium (not shown)RecordDoAs a result, the capacity of the recording medium is effectively utilized, and even when the variable transfer rate encoded data is decoded later, good decoded data can be obtained.
[0122]
  The present invention is not limited to the above-described embodiment, and various modifications can be made according to the design and the like as long as the technical idea of the present invention is not deviated from. The encoding method is not limited to MPEG1 described above.
[0123]
【The invention's effect】
  As described above, according to the variable transfer rate encoding method and apparatus of the present invention, it is not necessary to input the same image to the encoding apparatus twice as in the conventional variable transfer rate encoding apparatus. That is, as a moving image signal to be encoded, an image signal that is transmitted only once, such as a moving image signal transmitted in real time from broadcasting or communication, can be used at a high rate.With a markAnd then temporarily record the data on a writable disc or tape media.
Thereafter, the data is used to perform variable transfer rate encoding with high image quality, and finally necessary variable transfer rate encoded data can be generated and recorded, for example.
  Also, record temporarilyTickPerforms variable transfer rate encoding using parameters for motion compensation of encoded dataIf you doWhen obtaining a motion vector, it is not disturbed by encoding degradation noise.
  In addition, record temporarilyTickPerforms variable transfer rate encoding using the editing information of the encoded data such as the part to be cut.If you doThus, variable transfer rate encoding reflecting the editing information can be performed.
[Brief description of the drawings]
FIG. 1 shows a variable transfer rate encoding method and apparatus according to the present invention.As a reference exampleFIG. 3 is a block diagram illustrating a schematic configuration when encoded data is detected by an encoding information detector and converted into variable transfer rate encoded data (re-encoding) in the video signal encoding apparatus.
FIG. 2 is a diagram illustrating a format of encoded information (picture information) recorded in a storage circuit of the video signal encoding apparatus according to the present embodiment.
FIG. 3 is a video signal encoding device according to the present embodiment.detectionIn a vesseldetectionShiTickConvert encoded data to variable-length encoded data(Re-encoding)It is a block diagram which shows the example of schematic structure in the case of doing.
FIG. 4 shows generated code amount information and quantization width information recorded in a storage circuit in the video signal encoding apparatus of the present embodiment., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)It is a block diagram which shows the structure of the 1st specific example for doing.
FIG. 5 shows generated code amount information and quantization width information recorded in a storage circuit in the video signal encoding apparatus of the present embodiment., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)It is a block diagram which shows the structure of the 2nd specific example for doing.
FIG. 6 is a diagram illustrating a format of encoded information (macroblock information) recorded in a storage circuit of the video signal encoding apparatus according to the present embodiment.
FIG. 7 shows generated code amount information and quantization width information recorded in a storage circuit in the video signal encoding apparatus according to the present embodiment., MarksUsing encoded data, etc.The markConvert encoded data to variable transfer rate encoded data(Re-encoding)It is a block diagram which shows the structure of the 3rd specific example for doing.
FIG. 8 is a block diagram showing a basic configuration of a conventional video signal encoding device.
[Fig. 9] Fig. 9 is a block diagram illustrating a configuration in the case where the first pass encoding is performed in the conventional video signal encoding device.
FIG. 10 is a block diagram showing a configuration when performing a second pass encoding in a video signal encoding device of a conventional example.
[Explanation of symbols]
  DESCRIPTION OF SYMBOLS 1 ... Input terminal, 2,52 ... Operation unit, 3,53 ... DCT device, 4,54 ... Quantizer,
5, 55 ... VLC unit, 6, 56 ... Buffer memory, 7, 57 ... Output terminal,
8, 58 ... inverse quantizer, 9, 59 ... inverse DCT, 10, 60 ... image memory,
11, 61 ... motion compensation predictor, 12, 62 ... arithmetic unit, 62 ... code amount counter,
63 ... Code amount control circuit, 22 ... Memory circuit, 24 ... Encoding informationdetectionDevice, 25 ... CPU,
26 ... target code amount determination circuit, 40 ... decoder, 41 ... editing information input device,
42 ... Decoding controller, 43 ...detectionController, 50... Variable transfer rate encoder.

Claims (6)

A variable transfer rate encoding method that encodes a video signal using orthogonal transform and quantization and outputs the video signal at a variable transfer rate,
Encoding an incoming video signal using orthogonal transform and quantization;
Simultaneously with the encoding of the incoming video signal, a generated code amount for each image unit and an average quantization width for each image unit of the encoded data are detected, and each detected image unit is detected. the method comprising the generated code amount and generate encoded information having information and the average quantization scale of each image unit,
Recording the encoded encoded data at a fixed transfer rate and recording the encoded information;
Decoding the recorded encoded data;
Based on the recorded encoded information, the reproduction time of the encoded data, and the target average variable transfer rate, or based on the encoded information and the target code amount, a new one for each image unit A variable transfer rate encoding step for variable-length encoding the data obtained by decoding the encoded data so that the generated code amount for each image unit becomes the new target code amount;
A variable transfer rate encoding method comprising: In the recording step, parameters relating to motion compensation at the time of encoding in the encoding step are recorded,
In the variable transfer rate encoding step, when the data obtained by decoding the recorded encoded data is variable-length encoded, the recorded parameter for motion compensation is used as a parameter for motion compensation. The variable transfer rate encoding method according to claim 1. Provide a step for entering editing information,
In the step of generating the encoding information, the encoding information of only a necessary section based on the editing information is generated,
In the decoding step, the data of only the necessary section based on the editing information is decoded,
In the variable transfer rate encoding step, encoding information of only a necessary section based on editing information is used as the encoding information, and data decoded only in a necessary section based on the editing information is encoded. 3. The variable transfer rate encoding method according to claim 1, wherein the variable transfer rate encoding method is used. A variable transfer rate encoding device that encodes a video signal using orthogonal transform and quantization and outputs the video signal at a variable transfer rate,
An incoming video signal encoding means for encoding the incoming video signal using orthogonal transform and quantization;
Simultaneously with the encoding of the incoming video signal, a generated code amount for each image unit and an average quantization width for each image unit of the encoded data are detected, and each detected image unit is detected. Encoding information detecting means for generating encoded information having information on the generated code amount and the average quantization width for each image unit ;
Recording means for recording the encoded data at a fixed transfer rate and recording the encoded information;
Decoding means for decoding the recorded encoded data;
Based on the recorded encoded information, the reproduction time of the encoded data, and the target average variable transfer rate, or based on the encoded information and the target code amount, a new one for each image unit Variable transfer rate encoding means for variable-length encoding the data obtained by decoding the encoded data so that the generated code amount for each image unit becomes the new target code amount.
A variable transfer rate encoding device comprising: The incoming video signal encoding means outputs a parameter relating to motion compensation during encoding,
The recording means records parameters relating to the motion compensation;
The variable transfer rate encoding means uses the recorded parameter for motion compensation as a parameter for motion compensation when variable length encoding is performed on data obtained by decoding the recorded encoded data. The variable transfer rate encoding apparatus according to claim 4. Editing information input means for inputting editing information;
Detection control means for controlling the encoded information detection means based on the editing information;
A decoding control means for controlling the decoding means based on the editing information;
In the encoded information detection means, under the control of the detection control means, generate encoded information for only the necessary section based on the edit information,
In the decoding means, under the control of the decoding control means, the encoded data is decoded only in a necessary section based on the editing information,
In the variable transfer rate encoding means, encoded information of only a necessary section based on the editing information is used as the encoding information, and only a necessary section is decoded based on the editing information as data to be encoded. 6. The variable transfer rate encoding device according to claim 4 or 5, wherein:

Images