JP2010050833A - Video coding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain a coding processing efficiently using a memory transmission band of a shared memory without wrecking the coding processing. <P>SOLUTION: In the video coding device, a video input section 12, a predictive coding method determining section 18, a predictive coding processing section 20 and an entropy coding section 22 share a memory 16. The predictive coding method determining section 18 evaluates the amount of codes to be generated by applying simple coding processing to image data to be coded in the memory 16 and notifies the predictive coding processing section 20 of a prediction system that optimizes coding efficiency. The predictive coding processing section 20 performs predictive coding on the image data to be coded in the memory 16 in accordance with the notified prediction system. The predictive coding processing section 20 stores processing time information in macro block units. A CPU 10 controls prediction method determination processing conditions of the predictive coding method determining section 18 so as to converge the processing time information of the predictive coding processing section 20 within the time corresponding to a requested processing time. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video encoding apparatus.

  In recent moving image compression techniques, a hybrid coding method that compresses information from both the redundancy in the space domain and the redundancy in the time domain is the mainstream. The former compresses information by converting video data into the frequency domain by orthogonal transform processing such as DCT (Discrete Cosine Transform), and reducing the bit length of higher frequency components that have less influence on human vision. The latter is to reduce the amount of information by encoding the difference between the screens of a plurality of images that are temporally mixed, and further increases the encoding efficiency by applying motion compensation.

  In the calculation of moving image compression processing, the amount of motion search processing in motion compensation is dominant. Motion compensation accuracy in video compression has also been improved. That is, the motion estimation accuracy of 1 pixel accuracy in the unit of 16 × 16 pixel blocks is improved to half pixel accuracy in the MPEG-1 system. Furthermore, in MPEG-4 AVC (ISO / IEC 14496-10), the accuracy is improved to 1/4 pixel with respect to a minimum of 4 × 4 pixels. As for the inter-screen prediction itself, the prediction is expanded from only the forward prediction to the response to the backward prediction, and bidirectional prediction referring to two screens arbitrarily from the front and back. The amount of motion search processing increases with the improvement of motion estimation accuracy.

Japanese Patent Application Laid-Open No. 2004-151867 describes a technique for performing motion estimation with high accuracy and efficiency. In this technique, first, predicted motion information VIp with an accuracy of integer pixel positions is obtained from a plurality of reference frames. Then, predicted motion information VHp, VQp with other pixel position accuracy is obtained from one reference frame referred to at that time, and one of the obtained VIp, VHp, VQp is selected.
Japanese Patent Laid-Open No. 2006-093881

  How accurately the motion detection is performed within the range defined by the moving image compression method greatly affects the image quality. It is important to perform flexible processing within a limited transmission band.

  Generally, motion estimation can be performed with higher accuracy by performing motion search processing as widely and as accurately as possible. However, in actual implementation, it is important how accurate motion estimation can be performed with a small amount of calculation. For this reason, a design that secures a memory transmission band in consideration of the worst case is necessary so that the moving image compression process does not fail under a condition in which the motion search range and accuracy are limited. In practice, even when there is a margin in the memory transmission band, processing is often performed with limited motion search accuracy.

  Therefore, there is a demand for a moving image compression processing method that efficiently uses the memory transmission band without causing the encoding processing to fail.

  It is an object of the present invention to provide a video encoding apparatus and method that satisfy such a demand.

  A video encoding apparatus according to the present invention is a video encoding apparatus that encodes a screen constituting a moving image in units of macroblocks, and includes a predictive encoding method determining unit that determines a predictive encoding method of an input video signal; Predictive encoding processing means for predictively encoding the input video signal in accordance with the predictive encoding method determined by the predictive encoding method determining means, and predictive encoding processing means for holding processing time information in units of macroblocks And a memory shared by the predictive encoding method determining means and the predictive encoding processing means, and a control means for controlling the prediction method determination processing conditions of the predictive encoding method determining means according to the processing time information. It is characterized by that.

  According to the present invention, the prediction encoding processing time in units of macroblocks is stored, the processing time required for the entire screen is evaluated in predetermined time units, and the prediction method determination processing conditions are controlled. As a result, the encoding process time is adjusted, and the encoding process using the memory transmission band efficiently can be performed within the limited memory transmission band without causing the encoding process to fail.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of the present invention. The video encoding apparatus according to the present embodiment encodes a video signal using the MPEG-4 AVC method. It is assumed that the input video signal is a 4: 2: 0 component signal having a field frequency of 59.94 Hz and 1920 × 1088 pixels.

  In the present embodiment, an encoding target screen of a video signal representing a moving image is divided into 16 × 16 pixel blocks, and encoding processing is executed in units of macro blocks. The CPU 10 functions as a control unit that generates encoding parameters necessary for video encoding and controls each unit. The video input unit 12 writes the input video signal to the memory 16 via the memory I / F 14. The memory I / F 14 arbitrates access requests from the processing units to the memory 16 and controls reading and writing of the memory 16.

  The predictive encoding method determination unit 18 determines a predictive encoding method for each macroblock in the encoding target screen. Specifically, a prediction method in which the amount of generated code is evaluated by applying a simple intra-screen encoding process or inter-screen predictive encoding process to the encoding target image data in the memory 16 and the encoding efficiency is optimized. decide. For example, simple encoding processing based on various prediction method determination processing conditions including a motion search range, motion vector accuracy, and the number of motion vectors is used. The predictive coding method determining unit 18 transmits the predictive coding processing information necessary for the determined predictive coding process to the predictive coding processing unit 20. The CPU 10 can set the prediction method determination processing condition in the prediction encoding method determination unit 18, and thereby controls the content of the prediction method determination processing condition by the prediction encoding method determination unit 18.

  The predictive encoding processing unit 20 predictively encodes the encoding target image data in the memory 16 in accordance with the predictive encoding processing information specified by the predictive encoding method determining unit 18. Specifically, predicted image data is generated from the encoded image data read from the memory 16. Then, orthogonal transform processing such as integer precision discrete cosine transform is performed on the prediction residual signal that is the difference between the encoding target image data and the prediction image data in units of 8 × 8 pixels or 4 × 4 pixel blocks. However, the DC (direct current) component as a result of integer-precision discrete cosine transform of each pixel block on the luminance signal or color difference signal subjected to the intra-screen prediction processing in units of 16 × 16 pixel blocks is further discrete. Hadamard transformation is performed.

  The orthogonally transformed transform coefficient is quantized in a quantization step corresponding to the designated quantization parameter, and the obtained quantized coefficient data is supplied to the entropy encoding unit 22. The quantized coefficient data is simultaneously decoded locally by adding an inverse quantization process, an inverse orthogonal transform process (inverse discrete Hadamard transform and inverse integer precision discrete cosine transform), and predicted image data. The locally decoded image data is subjected to deblocking filter processing, and the decoded image data before and after the deblocking filter processing is written back to the memory 16.

  The predictive coding processing unit 20 can hold processing time information required for the series of processes, and notifies the CPU 10 of the time information.

  The entropy encoding unit 22 entropy encodes the quantized coefficient data from the predictive encoding processing unit 20. The entropy encoding unit 22 performs encoding according to CABAC (Context-based Adaptive Variable Length Coding) as an entropy encoding method. Alternatively, CAVLC (Context-based Adaptive Binary Arithmetic Coding) may be used as the entropy coding method. The entropy encoding unit 22 multiplexes system data (sequence information, picture information, slice information, etc.) on code data obtained by entropy encoding, and outputs the result as encoded data.

  Considering the case where the video encoding process is performed in real time, the encoding process for one field of video must be completed within one field time. FIG. 2 shows a processing sequence of this embodiment. When an image for one field is composed of (1920 × 544) / (16 × 16) = 4080 macroblocks, the required processing time per macroblock is (1 / 59.94 Hz) / 4080 MB≈about 4 μs. Become.

  The predictive coding method determining unit 18, the predictive coding processing unit 20, and the entropy coding unit 22 are pipeline processing in units of one macroblock, and one macroblock processing of each processing unit 18, 20, and 22 is completed. Transitions to the next macroblock processing at a stage. Since the processing units 12, 18, 20, and 22 share the memory 16, the memory accesses of the processing units 12, 18, 20, and 22 affect each other. If the encoding process cannot be completed within 4 μs on average per macroblock, the processing delay accumulates, the memory area for holding the input image data overflows, and the video encoding process may fail.

  Consider the processing contents of each processing unit 12, 18, 20, 22 and the frequency of memory access to the memory 16. The video input unit 12 is a fixed process for writing the image data of the input video signal to the memory 16, and the memory access is also a fixed length periodic access. However, since the video signal is input in real time, the allowable latency for memory access is small, and the priority for the memory I / F 14 is set to be the highest.

  Access to the memory 16 by the predictive encoding method determination unit 18 largely depends on the prediction method determination processing conditions. In order to determine a prediction method for one macroblock of an encoding target image, it is necessary to read a large number of image regions by motion search or the like. For this reason, access to the memory 16 by the predictive coding method determination unit 18 occupies most of the entire memory access.

  The processing content of the predictive encoding processing unit 20 is fixed processing for each predictive encoding method, and the memory access is fixed and periodic for each predictive encoding method. The processing content of the entropy encoding unit 22 depends on the input quantized coefficient data, and the generated encoded data length also varies. However, since the compressed data after encoding is written into the memory 16, the memory access is also small and less affected than other processing units.

  From the above, in this embodiment, paying attention to the encoding processing time in units of one macroblock, the memory access is controlled or adjusted as follows. That is, the processing time of the entire encoding process is grasped based on the processing time information of the predictive encoding processing unit 20 in which the processing content and memory access are fixed. Then, the CPU 10 periodically evaluates the processing time information obtained from the prediction encoding processing unit 20 and controls the prediction method determination processing condition of the prediction encoding method determination unit 18. As a result, it is possible to perform an encoding process that effectively uses the transmission band of the memory 16 without breaking the encoding process. The control method will be described below.

  The predictive coding processing unit 20 monitors the processing time per macroblock, and adds the number of macroblocks as processing time information when the actual processing time exceeds the above-described required processing time. The predictive coding processing unit 20 also subtracts the number of macroblocks as processing time information when the actual processing time per macroblock is less than the required processing time. That is, in this embodiment, the number of macroblocks is increased or decreased depending on whether the actual processing time exceeds the requested processing time. Hereinafter, the count value of this macroblock is expressed as OVER_MB. The CPU 10 refers to the processing time information held by the predictive coding processing unit 20 as appropriate.

  FIG. 3 shows a flowchart of the CPU 10 that controls the prediction method determination processing conditions of the prediction encoding method determination unit 18 in accordance with the processing time information from the prediction encoding processing unit 20.

  The CPU 10 initializes OVER_MB in the predictive coding processing unit 20 and MB_COUNT indicating the number of macroblocks that have been processed (S1), and commands the start of coding (S2). Thereafter, the CPU 10 periodically acquires MB_COUNT (S3). When the macroblock processing number MB_COUNT reaches a predetermined number (S5), OVER_MB is acquired, and it is evaluated whether the OVER_MB value is 0, a positive value, or a negative value (S6). Here, 1 MB line: 120 MB unit.

  When the OVER_MB value is a positive value (S6), the OVER_MB value is compared with an OVER_LIMIT value that is a threshold for changing the prediction method determination processing condition of the prediction encoding method determination unit 18 set in advance (S7). When the OVER_MB value exceeds the OVER_LIMIT value (S7), it is determined that the memory transmission band is insufficient, and the prediction method determination processing condition of the prediction encoding method determination unit 18 is changed to a direction to limit (S8).

  Specifically, one of the reduction of the motion search range, the reduction of motion vector accuracy (for example, from 1/4 pixel to 1 pixel accuracy), and the limitation of the number of motion vectors (for example, 16 to 8), or a combination thereof Thus, the load of determining the prediction method is reduced. By reducing the motion search range, it is possible to reduce the amount of reference image read by the reduced area. By reducing the motion vector accuracy, it is possible to reduce the amount of reference image read accompanying the predictive interpolation process for generating half pixels or ¼ pixels. By limiting the number of motion vectors, it is possible to reduce the number of times of performing motion search within a macroblock, and to reduce the amount of reference images read. By controlling the prediction method determination processing conditions of the prediction encoding method determination unit 18 in this way, memory access of the prediction encoding method determination unit 18 can be suppressed. As a result, it is possible to reduce the memory access waiting of the predictive encoding processing unit 20 thereafter, and to reduce or eliminate the delay of the entire encoding process.

  When the OVER_MB value is a negative value (S6), it indicates that the encoding process can be executed within the requested time. Therefore, it can be considered that there is a margin in the memory transmission band. Therefore, in this case, it is possible to change the prediction method determination processing condition of the prediction encoding method determination unit 18 so as to relax (S9). For example, the determination accuracy of the predictive coding method determination unit 18 is improved by expanding the motion search range, improving the motion vector accuracy, increasing the number of motion vectors, or a combination thereof. Of course, if the relaxation of the prediction method determination processing condition has reached the limit, it is not necessary to change the prediction method determination processing condition.

  If OVER_MB is 0 (S6), it indicates that the prediction process itself can be processed with an average required time, so the condition is not changed.

  Thereafter, OVER_MB is evaluated in predetermined MB units (S5), and the same processing is repeated. When the number of processed macroblocks reaches one field (4080 macroblocks) (S4), the encoding process for one field is terminated.

  As described above, the prediction method determination processing condition of the prediction encoding method determination unit 18 is applied based on the time information of the actual processing time exceeding / less than the requested processing time based on the processing time of the prediction encoding processing unit. Control. By this adaptive control, the encoding process can be prevented from failing while effectively using the transmission band of the shared memory 16.

  By adaptively controlling access to the shared memory as described above, the transmission bandwidth of the shared memory can be effectively used, and the memory 16 can be shared by the recording processing unit 24 as shown in FIG. Become.

  An operation for controlling the prediction method determination processing condition of the prediction encoding method determination unit 18 in accordance with the processing time information held by the prediction encoding processing unit 20 when the video signal is encoded by the MPEG-4 AVC method will be described. FIG. 5 shows the control flowchart.

  In the control flow shown in FIG. 5, the predictive coding processing unit 20 holds the accumulated value of the difference between the actual processing time per macroblock and the requested processing time as the processing time information. Assuming that the operating frequency of the predictive coding processing unit 20 is 200 MHz, the required processing time is (4 μs) / (1/200 MHz) ≈800 T in terms of the number of clock cycles (T). The predictive coding processing unit 20 holds the cumulative addition value OVER_T of the number of clock cycles exceeding and below the requested processing clock cycle number (800T).

  The CPU 10 initializes OVER_T in the predictive coding processing unit 20 and MB_COUNT indicating the number of macroblocks that have been processed (S11), and commands the start of coding (S12). Thereafter, the CPU 10 periodically acquires MB_COUNT (S13). When the macroblock processing number MB_COUNT reaches a predetermined number (S15), OVER_T is acquired, and the OVER_T value is evaluated as 0, positive value, or negative value (S16). Here, 1 MB line: 120 MB unit.

  When the OVER_T value is a positive value (S16), the OVER_T value is compared with an OVER_LIMIT value that is a threshold for changing the prediction method determination processing condition of the prediction encoding method determination unit 18 set in advance (S17). When the OVER_T value exceeds the OVER_LIMIT value (S17), it is determined that the memory transmission bandwidth is insufficient, and the prediction method determination processing condition of the prediction encoding method determination unit 18 is changed to a direction that limits (S18). In order to limit the prediction method determination processing condition, the method described with reference to FIG. 3 may be employed.

  When the OVER_T value is a negative value (S16), it indicates that the encoding process can be executed within the required time. Therefore, it can be considered that there is a margin in the memory transmission band. Therefore, in this case, it is possible to change the prediction method determination processing condition of the prediction encoding method determination unit 18 so as to relax (S19). For example, the determination accuracy of the predictive coding method determination unit 18 is improved by expanding the motion search range, improving the motion vector accuracy, increasing the number of motion vectors, or a combination thereof. Of course, if the relaxation of the prediction method determination processing condition has reached the limit, the prediction method determination processing condition may not be changed.

  If OVER_T is 0 (S16), it indicates that the prediction process itself can be processed with an average required time, so the condition is not changed.

  Thereafter, OVER_T is evaluated in predetermined MB units (S15), and the same processing is repeated. When the number of processed macroblocks reaches one field (4080 macroblocks) (S14), the encoding process for one field is terminated.

  As described above, the prediction method determination processing condition of the prediction encoding method determination unit 18 is applied based on the time information of the actual processing time exceeding / less than the requested processing time based on the processing time of the prediction encoding processing unit. Control. By this adaptive control, the encoding process can be prevented from failing while effectively using the transmission band of the shared memory 16.

  By adaptively controlling access to the shared memory as described above, the transmission bandwidth of the shared memory can be effectively used, and the memory 16 can be shared by the recording processing unit 24 as shown in FIG. Become.

It is a schematic block diagram of one Example of this invention. It is a figure which shows the encoding process timing of a macroblock unit. It is a flowchart which shows the control method of the prediction determination process conditions based on process time information (OVER_MB). It is a schematic block diagram of an embodiment in which the recording processing unit is changed. It is a flowchart which shows the control method of the prediction determination process condition based on process time information (OVER_T).

Explanation of symbols

10: CPU
12: Video input unit 14: Memory I / F
16: Memory 18: Predictive encoding method determination unit 20: Predictive encoding processing unit 22: Entropy encoding unit 24: Recording processing unit

Claims (4)

A video encoding device that encodes a screen constituting a moving image in units of macroblocks,
A predictive encoding method determining means for determining a predictive encoding method of an input video signal;
Predictive encoding processing means for predictively encoding the input video signal according to the predictive encoding method determined by the predictive encoding method determining means, and predictive encoding processing means for holding processing time information in units of macroblocks; ,
A memory shared by the predictive encoding method determining means and the predictive encoding processing means;
A video encoding apparatus comprising: a control unit that controls a prediction method determination processing condition of the prediction encoding method determination unit according to the processing time information. The processing time information is a count value that is added when the actual processing time per macroblock exceeds the required processing time, and subtracted when the actual processing time is less than the required processing time.
The video encoding apparatus according to claim 1, wherein the control unit controls the prediction method determination processing condition according to a size of the count value. The processing time information is a cumulative addition value of the time when the processing time per macroblock is increased or decreased with respect to the required processing time,
The video encoding apparatus according to claim 1, wherein the control unit controls the prediction method determination processing condition according to a size of the cumulative addition value.   The video according to any one of claims 1 to 3, wherein the prediction method determination processing condition includes any one of a motion search range, the number of motion vectors, pixel accuracy of motion vectors, and a combination thereof. Encoding device.

Images