JPWO2006004065A1 - Moving picture coding processing system, moving picture coding or decoding processing system, moving picture coding processing method, and moving picture coding or decoding processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture coding or decoding processing system and a moving picture coding or decoding processing method capable of avoiding a failure phenomenon while achieving low power consumption. SOLUTION: The necessary calculation amount calculation means 2 for calculating a necessary calculation amount necessary for encoding or decoding a predetermined frame, and the time Te previously allocated for the encoding process or the decoding process of the predetermined frame are described above. An operation power supply voltage capable of encoding or decoding a required amount of calculation, an operation determining means 3 for calculating a substrate bias voltage, and an operating frequency, and the processor 1 performs the constant operating frequency calculated during the time Te. And when performing the encoding or decoding processing of a predetermined frame while operating with the operating power supply voltage and the substrate bias voltage, and when the required calculation amount calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount. A failure avoidance means for avoiding a failure phenomenon that occurs is provided. [Selection diagram]

Description

  The present invention uses a processor capable of changing an operating frequency and an operating power supply voltage and/or a substrate bias voltage to sequentially encode or decode a moving image composed of a plurality of consecutive frames on a frame-by-frame basis. The present invention relates to a moving picture coding processing system, a moving picture coding or decoding processing system, a moving picture coding processing method, and a moving picture coding or decoding processing method.

  In recent years, it has become possible to send and receive moving images via a transmission path and to store moving images in storage media. Generally, a moving image has a large amount of information, and therefore, when transmitting a moving image using a transmission path with a limited transmission bit rate or storing the moving image in a storage medium with a limited storage capacity, Technology for encoding and decoding images is indispensable. As a moving picture encoding/decoding method, there are MPEG (Moving Picture Experts Group) and H.26X, which are being standardized by ISO/IEC. These are for encoding or decoding a plurality of consecutive frames that make up a moving image, and reduce the redundancy by using the temporal correlation and spatial correlation of the moving image. Is a technique for reducing the amount of information of the above, encoding, and decoding the encoded moving image again to the original moving image.

  Such encoding/decoding technology is applied to information terminal devices such as mobile phones incorporating a personal computer or a microcomputer, and operates a processor of a computer based on a program describing encoding/decoding means. By doing so, it functions as a moving image coding processing system when transmitting a moving image, and as a moving image decoding processing system when receiving a moving image. However, such a moving image encoding or decoding process tends to consume a large amount of power due to a relatively large amount of calculation, and the encoding/decoding process is performed by using software having higher versatility than hardware. It is a major issue to reduce the power consumption in the.

  The conventional means for reducing power consumption in a moving picture coding or decoding system using software will be described below. A conventional means for reducing power consumption is disclosed in, for example, Non-Patent Document 1 below.

Proceedings of IEEE International Symposium on Circuits and System 2001 (May, 2001) pp918-921 “An LSI for Vdd-Hopping and MPEG4 System Based on the Chip”(H. Kawaguchi, G. Zhang, S. Lee, and T. Sakurai)

  FIG. 22 is a diagram showing a conventional method for reducing power consumption in the moving image (moving image) encoding processing system shown in Non-Patent Document 1. The means for reducing power consumption is the same in the moving image decoding processing system.

  In Non-Patent Document 1, an operating power supply voltage and an operating frequency for reducing power consumption when processing moving image coding (especially MPEG) on a processor capable of dynamically changing the operating power supply voltage and the operating frequency. It shows the control method of. That is, according to the invention of Non-Patent Document 1, as shown in FIG. 23, when performing moving image encoding, the amount of calculation of moving image encoding or decoding differs for each frame depending on the intensity of motion in the moving image. With this in mind, the power consumption is reduced by controlling the operating frequency and operating power supply voltage of the processor.

  In the encoding process, the processing time of one frame is restricted to the time Tf due to the regulations of the encoding method (MPEG, etc.), and it is necessary to complete the encoding process of one frame within the processing time Tf. It The processing time Tf (second) of one frame is divided into N intervals at a constant interval, and each interval (time) is defined as a time slot Tslot (Tslot=Tf/N). The remaining time TRi at the time when the time slot Tsloti ends from the slot Tslot1 is defined as TRi=Tf-Tslot×i. Let R (that is, R×N be the number of blocks of one frame) be the number of blocks of a moving image to be processed in one time slot Tslot (encoding of moving images is performed in block units), and let (R×i ) Let Tacc(i+1) be the time taken for block processing (that is, the time actually taken for the block group to be processed from time slot Tslot1 to time slot Tsloti). Let Trd be the time until the operating power supply voltage and operating frequency stabilize when the voltage is changed. The real time slot RTsloti indicates the processing time actually required for the processing to be completed within the time slot Tsloti. In FIG. 22, first, with respect to the processing of the block group allocated to the time slot Tslot1 and the time slot Tslot2, at the clock frequency Fmax at which the processing can be sufficiently completed within the time slot Tslot1 and Tslot2 even when the load is maximum. To operate. If the time Tacc3 required for the processing is Tacc3<(Tf-TR2), that is, if the processing of the allocated block group is completed in the time slots Tslot1 and Tslot2, the block allocated to the next time slot Tslot3. The processing time Ttar3 that can be used for processing the group is Ttar3=Tf-Tacc3-TR3-Trd, and the processing of the block group assigned to Tslot3 should be completed within this processing time Ttar3. Lower the operating frequency. Processing times Tf1, Tf2, and Tf3 in FIG. 22 represent processing times when operating at each operating frequency F1, F2, and F3 when the load is maximum in the time slot Tslot3. As the operating frequency, if the operating frequency of F2=Fmax/2 in FIG. 22 is selected, the processing time to be completed from time slot Tslot1 to time slot Tslot3 within (Tf-TR3) even when the load is maximum. Processing does not enter a certain next time slot Tslot4. On the other hand, when the operating frequency F3=Fmax/3 is selected, the processing time Tf3 exceeds the processing time Ttar3. Therefore, the block group to be processed in this time slot Tslot3 is operated at the operating frequency of F2=Fmax/2 and the operating power supply voltage suitable for the operating frequency. Similarly, this processing is performed for each time slot Tslot.

  As a result, when dynamically changing the operating clock frequency and the operating power supply voltage, by selecting the minimum operating frequency that can process a predetermined number of block groups within a predetermined time, the overall operating frequency Also, the power consumption is reduced by lowering the operating power supply voltage for operation and controlling the voltage according to the required processing.

  By the way, for a process to be completed within a certain processing time (for example, one frame processing time Tf here) (for example, one frame processing here), the processor is operated at a constant operation through the one frame processing time. It is preferable to operate by processing at the power supply voltage and the operating frequency. That is, assuming that the processing time for one frame is Tf (seconds), the calculation amount is Kf (cycle), and the operating frequency is Ff, the operating frequency Ff=Kf/Tf (cycle/second) is set, and the processing time Tf for one frame is set. Through the operation of the processor at a constant operating frequency Ff, the power consumption can be further reduced as compared with the case where the operating frequency Ff is changed many times within the processing time Tf.

  However, in Non-Patent Document 1, even though the unit of synchronization of the processing time Tf is one frame, the operating power supply voltage and the operating frequency are changed up to N times within one frame, which results in low power consumption. Was not planned sufficiently. That is, the low power consumption of the moving picture coding or decoding process in the processor capable of controlling the operating power supply voltage and the operating frequency in multiple stages as in the conventional example is to reduce the operating power supply many times during the processing of one frame. It was necessary to change the voltage and operating frequency. On the other hand, as described above, since the unit of constraint of processing time is a frame, it is preferable to control at a minimum fixed frequency during which one frame is processed. Therefore, in the conventional example in which the operating power supply voltage and the operating frequency are changed up to N times during the processing of one frame, the power consumption cannot be sufficiently reduced.

Therefore, the inventors of the present application completed the invention of Patent Document 1 below. The present invention is intended to reduce power consumption by performing encoding or decoding processing while operating the processor 1 at a constant operating power supply voltage and operating frequency.
Japanese Patent Application 2003-48535

  Further, one of the other factors that hinder the reduction of the power consumption of the processor is the subthreshold leak current of the MOS transistor that constitutes the processor. The subthreshold leak current is a minute current that flows when the gate voltage of the MOS transistor formed on the semiconductor substrate is equal to or lower than the threshold voltage. The power consumption due to this subthreshold leakage current tends to become dominant as the miniaturization of MOS transistors increases, and a moving image is encoded or decoded using a processor in which MOS transistors are integrated on a semiconductor substrate. This is one of the factors that hinder the reduction of power consumption in a moving image encoding or decoding system.

  This subthreshold leakage current is reduced by operating at a constant operating frequency Ff throughout the processing time Tf, as compared with the case where the operating frequency Ff of the processor is changed many times within the processing time Tf of one frame. It is possible to reduce the power consumption. In the invention of Non-Patent Document 1 described above, the operating frequency is changed up to N times within one frame even though the unit of synchronization of the processing time Tf is one frame. It was also unfavorable from the viewpoint of subthreshold leakage current. On the other hand, regarding the MOS transistor, it is known that the subthreshold leakage current can be controlled by controlling the substrate bias voltage of the semiconductor region in which the MOS transistor is formed.

Therefore, the inventors of the present application completed the invention of Patent Document 2 below. The present invention calculates a calculation amount required for encoding or decoding for each frame (hereinafter, referred to as a necessary calculation amount), and calculates a constant substrate bias voltage/operating frequency or a constant substrate bias voltage/operating power supply voltage/ The power consumption is reduced by performing the encoding or decoding process while operating the processor 1 at the operating frequency.
Japanese Patent Application 2003-409641

  However, in the above-mentioned Patent Documents 1 and 2, when the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the calculation amount actually required for the encoding or decoding processing, one frame processing is performed. There is a possibility that a failure phenomenon may occur in which the encoding process or the decoding process cannot be completed within the allocated time Te. Although Patent Documents 1 and 2 also disclose the failure avoidance means for avoiding the failure phenomenon, several failure avoidance means are conceivable, and in particular, it is strongly desired to improve the deterioration of image quality due to the failure avoidance means.

  Therefore, the present invention is to solve the above-mentioned problems, and it is possible to avoid the breakdown phenomenon while realizing low power consumption by controlling the substrate bias voltage, the operating power supply voltage, and the operating frequency. Another object is to propose a moving picture coding processing system, a moving picture coding or decoding processing system, a moving picture coding processing method, and a moving picture coding or decoding processing method.

The moving picture coding processing system of the present invention includes a processor that functions as a moving picture coding unit that codes a moving picture composed of a plurality of consecutive frames in frame units,
Necessary calculation amount calculation means for calculating the necessary calculation amount Kp required for encoding one frame, and the necessary calculation amount Kp within the time Te pre-allocated for the encoding process of the one frame. An operation determining means for determining a possible operating frequency,
The moving image encoding means operates while the processor operates at the operating frequency determined by the operation determining means, and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance. Is a moving image coding processing system that performs coding processing of the one frame by
It is characterized by comprising a first failure avoiding means for reducing the actual calculation amount of the encoding process at a predetermined timing.
Further, the moving picture coding processing method of the present invention comprises a moving picture coding step of coding a moving picture composed of a plurality of consecutive frames using a processor on a frame-by-frame basis, and coding a single frame. The necessary calculation amount calculation step of calculating the necessary calculation amount Kp and the operating frequency at which the necessary calculation amount Kp can be coded within the time Te pre-allocated for the coding process of the one frame are determined. And an operation determining step,
While the processor is operating at the operating frequency obtained in the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image encoding or It is a moving picture coding processing method for carrying out the coding processing of the one frame in the decoding step,
It is characterized by including a first failure avoidance step of reducing the actual calculation amount of the encoding process at a predetermined timing.
The first failure avoidance means and the first failure avoidance step reduce the actual calculation amount by changing a part of the encoding process to a process having a smaller calculation amount.
Here, the “encoding process” refers to the process of the encoding process unit/encoding process step performed after the process of the action determining unit/action determining step, and the “decoding process” refers to the action determining unit/ This refers to the processing of the decoding processing means/decoding processing step performed after the processing of the operation determining step.

  According to the moving picture coding system and the moving picture coding method of the present invention, the necessary calculation amount Kp necessary for the coding processing of one frame to be completed within the time Te is calculated, and the processor is operated within the time Te. And the minimum operation frequency at which one frame can be coded within the time Te (or close to the minimum), and an operation power supply voltage suitable for the operation frequency. Since the encoding process is performed with the substrate bias voltage, the power consumption can be reduced. Even if the required calculation amount Kp calculated by the required calculation amount calculation unit is smaller than the actually required calculation amount, the actual calculation amount of the encoding process is the first failure avoiding unit or the first calculation unit. Since it is reduced by the failure avoidance step, the failure phenomenon can be avoided.

  The first failure avoidance means is characterized in that, among macroblocks that have not been coded, blocks having information on color difference signals are processed as invalid blocks.

  According to the present invention, only a color difference block is set for a macro block composed of a block having luminance signal information (hereinafter referred to as a luminance block) and a block having color difference signal information (hereinafter referred to as a color difference block). Since the invalid blocks are processed and the luminance blocks are coded in the same way as usual, compared to the case where all the blocks of the macro blocks are processed as the invalid blocks, the image quality is improved while avoiding the failure phenomenon. Can be secured. Here, the reason why the target of the invalid block formation processing is not the luminance block but the color difference block is that the invalid block formation processing of the color difference block has less influence on the image quality deterioration.

  The first failure avoiding means is characterized by performing an intra-frame coding process on a macro block that has not been coded.

  The amount of calculation in the intraframe coding process is smaller than that in the interframe coding process. According to the present invention, by performing the intra-frame coding process on a macro block that has not been coded, the calculation amount actually required for the coding process is calculated by the necessary calculation amount calculation means. The value can be set smaller than the calculation amount Kp or close to the required calculation amount Kp. This makes it possible to avoid the failure phenomenon.

  The first failure avoiding means is characterized by performing inter-frame coding processing on a macro block for which coding has not been completed, by setting the motion vector to 0 without performing motion vector detection.

  The calculation amount required for the motion vector detection process in the inter-frame encoding process is an element that occupies most of the calculation amount of the encoding process. According to the present invention, since the motion vector is forcibly set to 0 without performing the motion vector detection process and the interframe coding process is performed, the calculation amount is smaller than that of the normal interframe coding process in which the motion vector detection process is performed. Therefore, the actual calculation amount of the encoding process can be set to a value smaller than or close to the required calculation amount Kp. This makes it possible to avoid the failure phenomenon.

  The first failure avoidance means is characterized by performing a coding process by increasing the quantization step size for a macroblock that has not been coded.

  According to the present invention, the number of effective blocks and the number of effective coefficients generated in the macroblock are suppressed by increasing the quantization step size and performing the encoding process on the macroblock that has not been encoded yet. However, the number of executions of the processing performed on the effective block and the effective coefficient (for example, when MPEG4 is used as the moving image encoding method, IDCT processing, IQ processing, VLC processing) is reduced, and the actual calculation of the encoding processing is performed. The amount can be set smaller than the required calculation amount Kp or close to the required calculation amount Kp. This makes it possible to avoid the failure phenomenon.

  Further, the moving picture coding system according to the present invention comprises mitigating means for mitigating an influence given to the coding processing of the subsequent frame coded after the one frame by the first failure avoiding means. Is characterized by.

  The first failure avoidance means changes the processing content of the encoding process that should normally be performed to perform the failure avoidance. Therefore, when the first failure avoidance means is executed in one frame, The image quality of the frame is not a little affected by the processing by the first failure avoidance means. According to the present invention, when the processing by the first failure avoiding means is executed in one frame, the relaxing processing is executed by the relaxing means for relaxing the influence on the encoding processing of the subsequent frame, so that the code of the subsequent frame is It is possible to suppress deterioration of image quality due to the digitization processing.

  The easing unit reduces the quantization step size for a macroblock of a subsequent frame corresponding to the macroblock on which the processing by the first failure avoiding unit is executed in the one frame.

  According to the present invention, by reducing the quantization step size, the code amount assigned to the corresponding macroblock can be increased to improve the image quality, and the failure avoidance processing by the first failure avoidance means is performed. Even in the case, it is possible to mitigate the influence on the subsequent frames encoded after the one frame for which the failure avoidance processing has been performed.

  Further, when the calculation amount of the encoding process of the one frame is reduced by the first failure avoiding means, the necessary calculation amount of the subsequent frame to be encoded after the one frame is increased. Is characterized by. For example, for the required amount of calculation of the subsequent frame or an element used to calculate the required amount of calculation, the subsequent frame is added by adding m times (m is a real number of 1 or more) or a real number n larger than 0. Increase the required calculation amount of.

  When the processing of the first failure avoidance means or the first failure avoidance step is executed, there is a high probability that the required calculation amount Kp for the subsequent frame will be smaller than the actually required calculation amount. According to the present invention, after the processing of the first failure avoidance means and the first failure avoidance step, the necessary calculation amount of the subsequent frame is increased, so that the necessary calculation amount of the subsequent frame can satisfy the actual calculation amount. Therefore, the failure phenomenon in the subsequent frame can be avoided.

A moving image encoding or decoding processing system of the present invention includes a processor that functions as a moving image encoding or decoding unit that encodes or decodes a moving image composed of a plurality of continuous frames in frame units,
Necessary calculation amount calculation means for calculating a necessary calculation amount Kp necessary for encoding or decoding one frame, and the necessary calculation within a time Te pre-allocated for the encoding or decoding process of the one frame. An operation determining means for determining an operating frequency capable of encoding or decoding the quantity Kp,
The moving image encoding is performed while the processor operates at the operation frequency determined by the operation determining means and the operation power supply voltage and/or the substrate bias voltage suitable for the operation frequency within the time Te that is assigned in advance. Or a moving picture coding or decoding processing system for performing coding or decoding processing of the one frame by a decoding means,
When the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount, the necessary calculation amount of a subsequent frame encoded or decoded after the one frame is increased. It is characterized in that it is provided with a failure avoidance means of 2.
Further, the moving picture coding or decoding processing method of the present invention is a moving picture coding or decoding step for coding or decoding a moving picture composed of a plurality of consecutive frames using a processor in frame units. And a necessary calculation amount calculation step of calculating a necessary calculation amount Kp necessary for encoding or decoding one frame, and within the time Te pre-allocated for the encoding or decoding process of the one frame. An operation deciding step of deciding an operating frequency capable of encoding or decoding the required calculation amount Kp,
While the processor is operating at the operating frequency obtained by the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image coding or A moving image encoding or decoding method that performs encoding or decoding of the one frame in the decoding step,
When the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount, the necessary calculation amount of a subsequent frame encoded or decoded after the one frame is increased. It is characterized by including the failure avoidance step of 2.

  When the required calculation amount Kp is smaller than the actually required calculation amount for one frame, there is a high probability that the required calculation amount Kp will be smaller than the actually required calculation amount for the subsequent frames as well. According to the present invention, since the second failure avoidance means and the second failure avoidance step increase the required calculation amount of the subsequent frame, it is highly possible that the necessary calculation amount of the subsequent frame satisfies the actual calculation amount. Therefore, the failure phenomenon in the subsequent frame can be avoided. The process of the second failure avoidance means/step is performed when the required operation amount Kp of the predetermined frame is smaller than the actual operation amount regardless of the presence or absence of the first failure avoidance means/step.

  In the second failure avoidance means and the second failure avoidance step, the necessary calculation amount of the subsequent frame or the element used for calculating the necessary calculation amount is multiplied by m (m is a real number of 1 or more), or , And a real number n larger than 0 is added.

  According to the present invention, the value of the necessary calculation amount calculated by the necessary calculation amount calculation means in the processing of the subsequent frame or the value of the element used for calculating the necessary calculation amount is multiplied by m (m is 1 or more). By adding a real number) or a real number n larger than 0, it is possible to increase the necessary calculation amount Kp of the subsequent frame and avoid the occurrence of the breakdown phenomenon in the subsequent frame. As the elements to be increased and the values of m and n, it is preferable to derive appropriate elements and values by experiments beforehand and set them in the system.

A moving image encoding or decoding processing system of the present invention includes a processor that functions as a moving image encoding or decoding unit that encodes or decodes a moving image composed of a plurality of continuous frames in frame units,
Necessary calculation amount calculation means for calculating a necessary calculation amount Kp necessary for encoding or decoding one frame, and the necessary calculation within a time Te pre-allocated for the encoding or decoding process of the one frame. An operation determining means for determining an operating frequency capable of encoding or decoding the quantity Kp,
Within the time Te pre-allocated by the processor, moving image coding or while operating at the operating frequency determined by the operation determining means, and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency. A moving image encoding or decoding processing system for performing encoding or decoding processing of the one frame by a decoding means,
A third failure avoiding means for extending the time Te at a predetermined timing is provided.
Further, a moving picture coding or decoding processing method of the present invention includes a moving picture coding step of coding or decoding a moving picture composed of a plurality of consecutive frames using a processor in frame units. Required calculation amount calculation step for calculating the required calculation amount Kp required for encoding or decoding of the frame, and the required calculation amount within the time Te pre-allocated for the encoding or decoding processing of the one frame. An operation determining step of determining an operating frequency at which Kp can be encoded or decoded.
While the processor is operating at the operating frequency obtained in the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image encoding or A moving image encoding or decoding method that performs encoding or decoding of the one frame in the decoding step,
It is characterized by including a third failure avoidance step of extending the time Te at a predetermined timing.

  According to the present invention, when the encoding or decoding processing of one frame cannot be completed within the time Te allocated in advance, the processing of the one frame is performed using the time allocated to the processing of the next subsequent frame. The time Te is extended and the encoding or decoding process of the one frame is completed. As a result, it is possible to avoid the failure phenomenon without reducing the actual processing amount of encoding or decoding and without deteriorating the image quality.

  Furthermore, the third failure avoidance means may be configured to detect a subsequent frame that is encoded or decoded after the one frame when the time Te that is previously assigned to the encoding process of the one frame is extended. It is characterized in that the time Te previously assigned to the encoding process or the decoding process of the subsequent frame is changed.

  According to the present invention, by extending the time Te of one frame, the time assigned to the subsequent frame is shortened. However, regarding the encoding or decoding processing of the next subsequent frame, The time Te, which is obtained by subtracting the extension processing time assigned to the one frame from the time assigned to the encoding or decoding processing, is used as the time Te for the encoding or decoding processing of the subsequent frame. The operating frequency of the processor capable of performing the encoding or decoding processing of the subsequent frame is determined within the changed time according to the extension time, and the operating frequency and the substrate bias voltage or the operating power supply voltage suitable for the operating frequency determine the processor. Works. Therefore, low power consumption is realized also in the encoding or decoding process of the subsequent frame. Even if the time Te assigned to the subsequent frame is shortened by extending the time Te assigned to one frame, changing the time Te of the subsequent frame may cause a failure phenomenon in the processing of the subsequent frame. Can be lowered.

  The third failure avoidance means, when the time Te previously assigned to the encoding process of the one frame is extended, stores the frame in the input frame memory that stores the frame to be encoded next. The feature is that the writing destination is changed.

  According to the present invention, when the time Te is extended by the third failure avoidance means, the sequentially input frames are stored by changing the write destination without overwriting one frame. The one frame can be held in the frame memory, and the encoding process can be continued and completed.

  The third failure avoidance means, when the time Te previously assigned to the encoding process of the one frame is extended, determines that the input frame memory storing the frame to be encoded next cannot be written. It is characterized by doing.

  According to the present invention, when the time Te is extended by the third failure avoiding means, the input frame memory cannot write the frame, so that one frame is held without being overwritten by the subsequent frame. Therefore, the encoding process can be continued and completed. Further, when the frame is encoded every plural frames, the time Te is not extended if the input frame memory becomes writable before the frame to be encoded is input to the input frame memory. It is possible to perform the encoding process on the same frame as the case.

  In the moving picture coding system and the moving picture coding or decoding system, when the frame to be coded before the one frame is a previous frame, the required calculation amount is required when performing the moving picture coding process. The calculating means calculates a motion amount between the predetermined frame and the previous frame, an activity amount of the predetermined frame, an activity amount of the previous frame, an average value of the quantization step size of the previous frame, and an average value of the quantization step size of the previous frame. The difference in the average quantization step size of the previous frame, the number of macroblock matchings in the previous frame, the number of valid blocks in the previous frame, the number of valid coefficients in the previous frame, and the actual coding required for the previous frame The amount of calculation, the number of bits generated in the previous frame, the coding bit rate of the predetermined frame, the type of intra-frame coding or inter-frame coding for the predetermined frame, and the previous frame calculated by the necessary calculation amount calculation means. It is preferable to calculate the required calculation amount using one or more elements of the required calculation amount.

  In the moving picture encoding or decoding system, when the frame to be decoded before the one frame is a previous frame, the necessary calculation amount calculation means is configured to set a predetermined frame when performing the moving picture decoding process. Number of bits of the encoded data, the type of whether the predetermined frame is intra-frame encoded or inter-frame encoded, the average value of the magnitudes of the motion vectors of the predetermined frame or the previous frame , Variance of magnitude of motion vector of predetermined frame or previous frame, number of effective blocks of predetermined frame or previous frame, number of effective coefficients of predetermined frame or previous frame, bit rate of predetermined frame or previous frame, predetermined frame or previous frame Code amount, the average value of the quantization step sizes of the predetermined frame or the previous frame, and the difference of the average value of the quantization step sizes (the difference between the quantization step size of the predetermined frame and the previous frame, or the previous frame Difference between the quantization step size and the quantization step size of the frame two frames before), the amount of calculation actually required for decoding the previous frame, and the required calculation amount of the previous frame calculated by the necessary calculation amount calculation means. It is preferred to calculate the required amount of computation using one or more elements.

Each of the plurality of elements is an element that affects the required calculation amount Kp in the encoding or decoding process. According to the present invention, one or more of the above elements are used as elements of the required calculation amount calculation means to calculate the required calculation amount Kp. Therefore, the required calculation amount Kp calculated by the required calculation amount calculation means is a reality. The value becomes closer to the calculation amount when the encoding or decoding process is performed. Therefore, it is less likely that the calculated required calculation amount Kp is too large as compared with the actual calculation amount and the reduction in power consumption is hindered, and the required calculation amount Kp is smaller than the actual calculation amount and is encoded or The failure phenomenon that the decryption process is not completed in time hardly occurs.
For example, the required calculation amount of the encoding process for one frame is determined by the number of times element processing such as macroblock matching is executed. Element processing is classified into processing repeatedly executed by the number of macroblock matching times, processing repeatedly executed by the number of valid blocks, and processing repeatedly executed by the number of valid coefficients. When all the numbers are the maximum, the amount of calculation required for the encoding process of one frame is the maximum (worst). In the actual encoding process, the number of macroblock matchings, the number of effective blocks, and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of calculation required for the encoding process fluctuates for each frame. In the present invention, although the calculation amount required for the encoding process differs for each frame, the number of macroblock matching times, the number of effective blocks, the number of effective coefficients, and the calculation amount required for the encoding process are close to each other between frames that are close in time. Paying attention to this property, as the parameters used to predict the required calculation amount of the coding process, the number of macroblock matching, the number of valid blocks, and the valid block of the previous frame, which has already completed the coding process at the time of prediction. By adopting the number of coefficients, the required calculation amount of the encoding process, and the like, the required calculation amount K of each frame can be set to a value approximate to the calculation amount when the encoding process is actually performed.
Similarly, the required calculation amount of the decoding process for one frame is also determined by the number of times the element process of the decoding process is executed. Element processing is classified into processing that is repeatedly executed by the number of valid blocks and processing that is repeatedly executed by the number of valid coefficients. When all of the number of valid blocks and the number of valid coefficients are maximum, the processing is performed as one frame of decoding processing. The required amount of calculation is maximum (worst). In the actual decoding process, the number of effective blocks and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of calculation required for the decoding process fluctuates for each frame. In the present invention, the calculation amount required for the decoding process is different for each frame, but attention is paid to the property that the number of effective blocks, the number of effective coefficients, and the required calculation amount of the decoding process are close values between temporally close frames. , The number of effective blocks of the previous frame that has already completed the decoding process at the time of prediction, the number of effective coefficients, and the necessary calculation of the encoding process, as the parameters used to predict the required calculation amount of the decoding process By adopting the amount or the like, the required calculation amount K of each frame can be set to a value approximate to the calculation amount when the decoding process is actually performed.

  As described above, according to the moving picture coding processing system, the moving picture coding or decoding processing system, the moving picture coding processing method, and the moving picture coding or decoding processing method of the present invention, the coding For one frame to be encoded or decoded (frame to be encoded or decoded in the future), calculation for predicting the necessary calculation amount Kp required for encoding or decoding is performed, and the frame is assigned to the processing of the predetermined frame. Within the time Te, the operating frequency and the operating power supply voltage, or the operating frequency and the substrate bias voltage, or the operating frequency and the substrate bias are set to the minimum or near the operating frequency necessary for processing the required calculation amount Kp in frame units. Since the voltage and the operating power supply voltage are dynamically controlled, low power consumption can be realized.

  Further, since the failure avoidance means is provided, it is possible to avoid the failure phenomenon that occurs when the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount, and the encoding or decoding process is performed. It is possible to prevent the generated moving image from being deteriorated. Further, when the processing by the failure avoidance means is executed, the mitigation processing is mitigated by the mitigation means for mitigating the influence on the encoding processing of the subsequent frame, thereby suppressing the deterioration of the image quality due to the encoding processing of the subsequent frame. it can.

The schematic block diagram which showed operation|movement of the moving image coding processing system of the 1st Embodiment of this invention. The figure which shows the implementation example of the moving image encoding processing system of the 1st Embodiment of this invention. The figure which shows the schematic flowchart of the moving image encoding processing program which makes a computer function as the moving image encoding processing system of the said embodiment. Sectional drawing which shows a triple well structure. FIG. 3 is a conceptual diagram showing an operating power supply voltage, a substrate bias voltage, and an operating frequency of a processor used in the moving picture coding system according to the above embodiment. Explanatory drawing explaining that low power consumption can be achieved by making an operating power supply voltage and an operating frequency constant. FIG. 6 is an explanatory diagram illustrating a relationship between a time when performing an interrupt and a calculation remaining amount in the above-described embodiment. The schematic block diagram which showed operation|movement of the moving image coding processing system of the 2nd Embodiment of this invention. The figure which shows the schematic flowchart of the moving image encoding processing program which makes a computer function as the moving image encoding processing system of the said embodiment. The schematic block diagram which showed operation|movement of the moving image decoding processing system of the 3rd Embodiment of this invention. The figure which shows the schematic flowchart of the moving image encoding treatment program which makes a computer function as the moving image encoding processing system of the said embodiment. The figure which shows the process time transition of normal encoding processing. The figure which shows the process time transition of the encoding process in 3rd Embodiment. The figure which shows the process time transition of the encoding process in 3rd Embodiment. The figure which shows the process time transition of the encoding process in 3rd Embodiment. The schematic block diagram which showed operation|movement of the moving image decoding processing system of the 4th Embodiment of this invention. The schematic block diagram which showed operation|movement of the moving image coding processing system of the 5th Embodiment of this invention. FIG. 3 is a conceptual diagram illustrating a relationship between an operating power supply voltage and an operating frequency of a processor used in the moving image coding processing system according to the above embodiment. The schematic block diagram which showed operation|movement of the moving image coding processing system of the 6th Embodiment of this invention. FIG. 3 is a conceptual diagram illustrating a relationship between a substrate bias voltage and an operating frequency of a processor used in the moving image coding processing system according to the above embodiment. FIG. 6 is a diagram showing an example of the relationship between the operating frequency of the processor, the operating power supply voltage, and the substrate bias voltage in the embodiment. The figure which showed the method of performing the conventional low power consumption about a moving image coding processing system. FIG. 6 is a conceptual diagram showing a state in which the amount of calculation for moving image encoding or decoding differs for each frame.

Explanation of symbols

S1 (S11, S12, S13, S14), S2, S3, S5, S6 Moving picture coding processing system S4 Moving picture decoding processing system 1, 51, 61 Processor 2, 42 Necessary calculation amount calculating means 3, 53, 63 Operation determining means 4, 54, 64 Operation control means 5 Moving picture coding means 45 Moving picture decoding means 6 Local decoding frame memory 7 Input frame memory 8 Element memories 91, 92, 93, 94, 95 First failure avoiding means 96 Second failure avoidance means 97, 98 Third failure avoidance means 10 Processed macroblock number register 101 Input image data 102 Operating power supply voltage/substrate bias voltage/operating frequency instruction 103 Local decoding data 104 Operating power supply voltage/substrate bias Voltage/operating frequency instruction 105 Operating power supply voltage/substrate bias voltage/operating frequency supply 106 Encoded data 107 Average value of quantization step size of previous frame,
108 Type of intra-frame coding or inter-frame coding for each frame
109 moving image coding bit rate 110 amount of activity of previous frame (past frame) 111 number of macroblock matching of previous frame 112 number of effective blocks of previous frame 113 number of effective coefficients of previous frame 114 quantization step of previous frame The difference between the average value of the sizes and the average value of the quantization step size of the frame immediately before that 115 115 The amount of processing actually required for encoding the previous frame 116 The required amount of calculation of the previous frame calculated by the required amount of calculation calculation means 117 Number of processed macroblocks 45 Moving picture decoding means 46 Local decoding frame memory 401 Input coded data 406 Decoded data 502 Operating power supply voltage/operating frequency instruction 505 Operating power supply voltage/operating frequency supply
602 Substrate bias voltage/operating frequency instruction 605 Substrate bias voltage/operating frequency supply
p-sub p-type semiconductor substrate n-well n-type well p-well p-type well

  Hereinafter, a moving picture coding or decoding processing system and a moving picture coding or decoding processing method according to the present invention will be described. In the moving picture coding or decoding system of the present invention, the processor 1 described later performs the moving picture coding processing and the moving picture decoding processing. When performing the moving picture coding, it is used as a moving picture coding processing system. It functions and functions as a moving picture decoding processing system when performing moving picture decoding. For example, the moving picture coding or decoding processing system of the present invention may be one that performs coding or decoding in frame units or time units, or may be one that performs only decoding processing or only coding processing. .. Further, the processor 1 is an arithmetic unit (instruction) dedicated to moving image processing for efficiently realizing moving image encoding processing or moving image decoding processing with a smaller number of cycles, smaller power consumption, and smaller program code amount. May be provided. As an example of an arithmetic unit (instruction) dedicated to moving image processing, a product-sum arithmetic unit (product-sum arithmetic instruction) used in matrix operation such as discrete cosine transform processing, a difference used in operation such as block matching operation in motion vector detection processing Absolute value sum calculator (difference absolute value sum instruction) and so on. Hereinafter, for convenience of description, a case where encoding is performed is referred to as a moving image encoding system, and a case where decoding is performed is referred to as a moving image decoding system, and the moving image encoding process and the moving image decoding process will be separately described in detail.

(First embodiment)
The moving picture coding system S1 according to the first embodiment of the present invention has a predetermined frame which occurs when the necessary calculation amount Kp calculated by the necessary calculation amount calculation means 2 is smaller than the actual calculation amount of the predetermined frame. This is to solve the problem of the failure phenomenon in which the processing cannot be completed within the time allotted to the processing of, and the coding processing of the predetermined frame is completed within the time previously allocated to the coding processing of the predetermined frame. If it is determined that the failure does not occur, the first failure avoiding means 9 that performs processing to avoid the failure phenomenon is provided, and the operating frequency, the substrate bias voltage, and the operating power supply voltage are controlled to be constant on a frame-by-frame basis (that is, the code of a predetermined frame). The processor operates at a constant operating frequency/constant substrate bias voltage/constant operating power supply voltage within the time allotted to the encoding or decoding process, and the operating frequency or By not changing the substrate bias voltage or the operating power supply voltage), the subthreshold leakage current and the charging/discharging current are appropriately suppressed, and the power consumption is reduced. The system S1 is realized by a computer that is an information terminal device such as a mobile phone or a personal computer having a microcomputer built therein, and is a system that functions as a part of a multimedia signal processing unit in the computer. , A system that sequentially encodes a moving image composed of a predetermined number of consecutive frames on a frame-by-frame basis.

  FIG. 1 is a schematic block diagram showing the operation of the moving image coding processing system S1 of the present embodiment, and FIG. 3 is a moving image coding processing method realized by the system S1. In the moving image coding processing system S1, the operating power supply voltage Vdd, the substrate bias voltages Vbn and Vbp, and the operating frequency F are variable in r steps (r is an integer of 2 or more) (that is, the operating power supply voltage Vdd in r steps and A processor 1 capable of operating at a substrate bias voltage Vbn, Vbp and an operating frequency F) and capable of changing an operating power supply voltage, a substrate bias voltage and an operating frequency by a program, and an operating power supply voltage, a substrate bias voltage and an operating frequency of the processor 1. A computer (in particular, a multimedia signal processing unit in the computer) that includes at least an operation control unit 4 for controlling the above, a local decoding frame memory 6 that is a storage area for storing predetermined data, an input frame memory 7, and an element memory 8 is there. Here, Vbn is the substrate bias voltage of the n-channel MOS transistor, and Vbp is the substrate bias voltage of the p-channel MOS transistor. In the local decoding memory 6, the input frame memory 7, and the like, the operation voltage, the operation frequency, and the substrate bias voltage may be controlled by the operation control unit 4 similarly to the processor 1. In the present embodiment, regarding the elements (processor 1, local decoding frame memory 6, element memory 8, processed macroblock number register 10, input frame memories 7a, 7b, etc.) included in the control area CA indicated by the dotted line, The operating frequency, operating voltage, and substrate bias voltage are controlled.

The processor 1 is a semiconductor element having a triple well structure, and the substrate bias voltage can be controlled for both the nMOS transistor and the pMOS transistor. The local decoding memory 6 and the input frame memory 7 are semiconductor memory elements, and the operation control means 4 controls the operating power supply voltage, the substrate bias voltage, and the operating frequency in the same manner as the processor 1.

  FIG. 4 is a partial sectional view of the processor 1 having the triple well structure. The processor 1 has a triple well structure by forming an n-type well n-well on a P-type semiconductor substrate p-sub and further forming a p-type well p-well on the n-type well n-well. .. An n-channel MOS transistor and a p-type well contact layer p-Contact are formed in the p-type well p-well. The n-channel MOS transistor has source/drain layers S and D made of n-type impurity layers and a gate electrode G. A p-channel MOS transistor and an n-type well contact layer n-Contact are formed in the n-type well n-well. The n-channel MOS transistor has source/drain layers S and D made of a p-type impurity layer and a gate electrode G. A substrate bias voltage Vbn is applied to a p-type well p-well, which is a semiconductor region in which an n-channel MOS transistor is formed, via a p-type well contact layer p-Contact. A substrate bias voltage Vbp is applied to an n-type well n-well, which is a semiconductor region in which a p-channel MOS transistor is formed, via an n-type well contact layer n-Contact.

  The operation control unit 4 includes an operation power supply voltage control unit 4c including a DC-DC converter, a substrate bias voltage Vbn generation unit 4d for controlling a substrate bias voltage of an n-channel MOS transistor, and a substrate of a p-channel MOS transistor. It comprises a substrate bias voltage Vbp generating means 4e for controlling the bias voltage, and an operating frequency control means 4b having a PLL and the like. However, each element of the operation control means 4 may exist outside the moving image coding processing system S1, and the operating power supply voltage, the substrate bias voltage, or the operating frequency may be controlled from outside the moving image coding processing system S1. The processor 1, the memories 6 and 7, and the operation control unit 4 are connected to each other via wiring.

  The processor 1 includes, as means for operating on the processor 1, a necessary calculation amount calculation means 2, an operation determination means 3, a moving image coding means 5, and a first failure avoidance means 9. Reference numeral 101 is input image data, reference numeral 102 is an operating power supply voltage/substrate bias voltage and operating frequency instruction, reference numeral 103 is locally decoded data of the previous frame, reference numeral 105 is an operating power supply voltage/substrate bias voltage/operating frequency supply, reference numeral Reference numeral 106 is coded data of a frame, reference numeral 107 is information on the average value of the quantization step size of the previous frame, reference numeral 108 is a type of each frame, that is, intra-frame coding or inter-frame coding, and reference numeral 109 is Coded bit rate information of moving image, code 110 is the amount of activity of the previous frame, code 111 is the number of macroblock matching of the previous frame, code 112 is the number of valid blocks of the previous frame, and code 113 is the number of valid coefficients of the previous frame. , Reference numeral 114 represents the difference between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the immediately preceding frame, reference numeral 115 represents the processing amount actually required for the encoding of the previous frame, reference numeral 116 Is the required calculation amount of the previous frame calculated by the required calculation amount calculation means 2. The element memory 8 is a part of a plurality of elements used in a necessary calculation amount calculation unit 2 described later (a type 108 of intra-frame coding or inter-frame coding, a coding bit rate). 109, a frame activity amount 110, and a necessary calculation amount 116 calculated by the necessary calculation amount calculation unit 2) are storage areas. The processed macroblock number register 10 is a register that temporarily stores information on the number of macroblocks 117 that have been encoded. Although MPEG-4 is used as the encoding method for the moving image encoding means 5, the H.264 standard is used. Other encoding methods such as 26X, MPEG-1, MPEG-2 may be used.

  FIG. 2 shows an implementation example of the moving image coding processing system S1. The system S1 mainly includes the processor 1, various memories MR, 7a and 7b as peripheral devices, various interfaces CI, DI and BI, operation control circuits 4a, PLL4b, DC-DC converter 4c, substrate bias voltage generating circuits 4d and 4e. It is realized by hardware including the above. The above-mentioned components can communicate with each other via buses B1, B2 and the like.

  The processor 1 includes a processor core 1a, an instruction cache memory 1b, a data cache memory 1c, and a bus controller BC. The required calculation amount calculation means 2, the operation determination means 3, the moving image coding means 5, and the failure avoidance means 9 and 11 are realized by executing the program stored in the memory MR on the processor core 1a as necessary. To be done. The instruction cache memory 1b and the data cache memory 1c are cache memories provided to speed up the processing of the program executed on the processor core 1a.

  The locally decoded frame memory 6, the element memory 8, and the processed macroblock number register 10 are aggregated in the memory MR of FIG. 2, and the average value 107 of the quantization step size of the previous frame and the intraframe encoding for each frame are performed. Type 108 whether there is inter-frame coding or not, bit rate 109 of moving picture coding, amount of activity 110 of previous frame (past frame), number of macroblock matching 111 of previous frame, number of effective blocks of previous frame 112, the number 113 of effective coefficients of the previous frame, the difference 114 between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the previous frame 114, and the actual coding of the previous frame The processing amount 115, the required calculation amount 116 of the previous frame calculated by the necessary calculation amount calculation means, and the processed macroblock number 117 are stored in the memory MR as data. The locally decoded data 103 is transmitted/received as signals 100j, 100k, 100l between the memory MR and the processor core 1a via the bus controller BC.

  The two input frame memories 7a and 7b correspond to the input frame memory 7 of FIG. Video data (input image data 101) input from the camera interface CI is input to the input frame memory 7a (or the input frame memory 7b) via the bus B2. The applications of the input frame memory (#0) 7a and the input frame memory (#1) 7b are switched each time one frame is processed. That is, in the processing of the i-th frame, the input image data is written in the input frame memory (#1) 7b by the signal 100h, and the input frame memory (# by the signal 100o for the encoding processing by the moving image encoding processing means. 0) When the input image data is read from 7a, in the processing of the (i+1)th frame, the input image data is written in the input frame memory (#0) 7a by the signal 100i, and the moving image coding processing means For the encoding processing, the input image data is read from the input frame memory (#1) 7b by the signal 100p. Therefore, the signal 100p does not occur when the input image data is written in the input frame memory (#1) 7b by the signal 100h, and conversely, the signal 100h does not occur when the image is read by the signal 100p. .. Similarly, when the input image data is written in the input frame memory (#0) 7a by the signal 100i, the signal 100o is not generated, and the input image data is read from the input frame memory (#0) 7a by the signal 100o. Signal 100i is not generated when the signal is being output. At this time, in the processing of the i-th frame, the input frame memory (#0) 7a, and in the processing of the (i+1)th frame, the input frame memory (#1) 7b stores the operating frequency, the substrate bias voltage, and the operating voltage. It becomes a control target. As described above, by preparing the input frame memories for two frames so that the respective operating frequencies can be set independently, the operation of writing the input image data from the camera interface CI, which is always the constant operating frequency, can be performed. It is possible to execute the read operation of the input image data in which the operating frequency fluctuates based on the calculated value of the necessary calculation amount, without interfering with each other.

  The operation control circuit 4a can send and receive signals to and from the PLL 4b, the DC-DC converter 4c, and the substrate bias voltage generation circuits 4d and 4e, and these functions as the operation control means 4. The operation control circuit 4a receives the operating power supply voltage/substrate bias voltage/operating frequency instruction 102 by the signal 100e from the processor core 1a, transmits the signal 100u to the PLL 4b based on the instruction 102, and then the DC-DC converter 4c. To the substrate bias voltage generating circuit 4d, and a signal 100x to the substrate bias voltage generating circuit 4e. The PLL 4b transmits the operating frequency signal 100a based on the signal 100u, the DC-DC converter 4c supplies the operating power supply voltage 100b based on the signal 100v, and the substrate bias voltage generating circuit 4d outputs the nMOS substrate bias voltage based on the signal 100w. 100c, and the substrate bias voltage generation circuit 4e supplies the pMOS substrate bias voltage 100d based on the signal 100x. As a result, the operating frequency, the substrate bias voltage, and the operating power supply voltage are controlled for the elements (processor 1, memory MR, input frame memories 7a and 7b, bus controller BC, etc.) included in the control area CA indicated by the dotted line in FIG. To be done. The signals 100e, 100j, 100k, 100l, 100m, 100o, 100p, 100q, 100r, 100s are the operating frequency signal 100a output by the PLL 4b, the power supply voltage supply 100b output by the DC-DC converter 4c, and the substrate bias voltage generating circuit 4d. Of the nMOS substrate bias voltage 100c output by the above and the pMOS substrate bias voltage 100d output by the substrate bias voltage generation circuit 4e, the frequency and the signal level change.

  The coded data 106 coded by the moving picture coding means 5 operating on the processor 1 is transmitted as a signal 100m to the bit stream interface BI via the bus B1 and is output as a signal 100n. The locally decoded data 106 generated in 1 is transmitted as a signal 100j to the memory MR functioning as the locally decoded frame memory 6. Further, image data and the like are read out as a signal 100q from the memory via the bus B1 and transmitted to the display interface DI. The signal 100q received by the display interface DI is output as video data by the signal 100t. The video data is output/displayed as a moving image via a monitor connected to the display interface DI.

  The operation control circuit 4a, the display interface DI, and the bit stream interface BI always operate at a constant operating power supply voltage, but the signals 100e, 100q, and 100m transmitted and received among them are elements included in the control area CA (processor 1 or The signal level fluctuates according to changes in the substrate bias voltage of the memory MR and the input frame memories 7a and 7b) and the operating power supply voltage. In order to absorb this influence, the operation control circuit 4a, the display interface DI, and the bit stream interface BI preferably include a level converter that corrects the signal levels of the received signals 100e, 100q, 100m.

  Next, the operation of the moving picture coding system S1 of the present embodiment will be described. The moving image coding processing system S1 is realized by causing the computer (in particular, the multimedia signal processing unit in the computer) to function as the following predetermined means by the moving image coding processing program Prg1. Further, the present system S1 realizes the moving picture coding method of the present invention including the following steps. Hereinafter, an arbitrary one frame to be coded from among sequentially coded frames is a predetermined frame (that is, a frame to be coded next when a certain frame is coded as a reference, in other words, , A frame that has not been coded at that point in time and is scheduled to be coded in the future (also called a current frame), one frame coded before a predetermined frame (coded in the past) A process of encoding a predetermined frame with (frame) as the previous frame will be described, but the same process is performed for any frame.

  The moving image coding processing program Prg1 causes the computer to function as described below in steps 1 to 5 described later. (Step 1) Input image information of a predetermined frame into the input frame memory 7. (Step 2) Function as the required calculation amount calculation means 2 for calculating the required calculation amount Kp of a predetermined frame. (Step 3) Functions as the operation determining unit 3 that determines the operating frequency F, the operating power supply voltage Vdd, and the substrate bias voltages Vbn and Vbp according to the calculated required calculation amount Kp. (Step 4) It functions as the operation control means 4 for controlling the operation of the processor 1 with the calculated operating frequency F, operating power supply voltage Vdd, and substrate bias voltages Vbn, Vbp. (Step 5) It is made to function as the moving picture coding means 5 for coding the image information of a predetermined frame. As described above, the processing of steps 1 to 5 is performed for all the frames in the order of the frames input to the input frame memory 7 (that is, the encoding order), thereby encoding the moving image. The details will be described below.

  (Step 1) The input input image data is temporarily stored in the input frame memory 7, which is a storage area for temporarily storing the frame, in order to synchronize the frame.

  (Step 2: Required calculation amount calculation step) The required calculation amount calculation means 2 accesses the input frame memory 7 to acquire the input image data 101 of a predetermined frame, and the necessary calculation amount Kp required for the encoding process of the predetermined frame. To calculate. There are various possible methods for calculating the required calculation amount Kp. For example, it is desirable to use one or more elements that affect the calculation amount of the encoding process of a predetermined frame. As an element, for example, in the moving image encoding process, when the processing content is motion compensation, paying attention to the fact that the amount of calculation is large for a video with a lot of motion, while the amount of calculation is small for a video with little motion. The distortion value calculated as the sum of absolute differences as the amount of motion between the predetermined frame and the previous frame, and the value calculated as the sum of absolute differences of adjacent pixels as the amount of activity of each frame, the macroblock matching count, and The number of effective blocks, the number of effective coefficients, the encoding bit rate, the number of generated bits, the amount of calculation actually required for encoding the previous frame, and the need for the previous frame calculated by the required calculation amount calculation means 2. The amount of calculation is included. Here, for each element, assuming that only the value of one element changes and the value of the other element does not change, the required amount of calculation is larger when the value of that one element is larger than when it is small. Is relatively large, and when the value of one of the elements is small, the required calculation amount is relatively small as compared with the case where it is large. In addition, when the predetermined frame is intra-frame coding, the required calculation amount Kp is relatively smaller than when it is inter-frame coding, and when it is inter-frame coding, it is intra-frame coding. By comparison, the required calculation amount Kp is made relatively large. That is, since these plurality of elements affect the required calculation amount required for the encoding process of the predetermined frame, the necessary calculation amount calculation means 2 determines the required calculation amount Kp( By performing the calculation so as to increase or decrease the number of cycles, the required calculation amount Kp calculated by the necessary calculation amount calculation means 2 becomes a value closer to the calculation amount when the encoding process is actually performed.

  For example, in the present embodiment, the input image data 101 of a predetermined frame calculated by using the function G and stored in the input frame memory 7 and the decoded previous image stored in the local decoding frame memory 6 are stored. The size of motion of the input image is predicted (calculated) by comparing with the locally decoded data 103 of the frame. The locally decoded data 103 of the previous frame is the encoded data 106 of the previous frame that is formed by encoding the previous frame in the encoding process of the previous frame that is encoded before the predetermined frame is decoded by the local decoder. And is stored in the local decoding frame memory 6. As an example of prediction (calculation) of the magnitude of motion, for example, the sum of absolute differences is used. Hereinafter, a method of obtaining the sum of absolute differences Σ and the required calculation amount Kp will be described. As the image data of the previous frame, the locally decoded data 106 decoded by the local decoder after encoding may be used, or the input image data of the input previous frame may be used as it is.

  The input image data 101 of a predetermined frame stored in the input frame memory 7 is stored in a local decoding frame memory 6 described later, where X(i,j) (i is the horizontal coordinate of the image and j is the vertical coordinate). If the locally decoded data 103 of the previous frame is Y(i,j) (i is the coordinate in the horizontal direction of the image and j is the coordinate in the vertical direction), the motion amount between the predetermined frame and the previous frame is the sum of absolute difference values. Calculate Z=Σ|X(i,j)-Y(i,j)| for all (or sampled) pixels. The value of this sum of absolute differences is Z. On the other hand, in the activity amount of the frame, the sum W of adjacent pixel differences in X(i,j), that is, horizontal direction Wh=Σ|X(i,j)−X(i-1,j)|, vertical It is obtained by calculating the direction Wv=Σ|X(i,j)-X(i,j-1)|, and is calculated for all (or sampled) input images. The value of the sum of the absolute values of the adjacent pixel differences (that is, the activity amount of each frame) is W.

The sum of absolute differences is Z, the activity amount of a predetermined frame is Wa, the activity amount of the previous frame (past frame) is Wb, the average quantization step size of the previous frame (average value of the quantization step size) is Qprev, and the previous frame , M is the number of effective blocks of the previous frame, C is the number of effective coefficients of the previous frame, S is the processing amount actually required for encoding the previous frame, and S is the encoding bit rate of the predetermined frame. BR, ΔQprev is the difference between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the previous frame, D is the actual number of generated bits of the previous frame, and is calculated by the necessary calculation amount calculation means. Assuming that the required calculation amount of the previous frame is Kp′, the required calculation amount Kp can be calculated by using one or more of these elements.
Kp=G(Z, Wa, Wb, Qprev, M, B, C, S, BR, ΔQprev, D, Kp')
Calculated by. However, G is a function derived from one or more elements of Z, Wa, Wb, Qprev, M, B, C, S, BR, ΔQprev, D, Kp′. As an example,
Kp=j+αM+βB+γC+δZ+εΔQprev
However, the present invention is not limited to this. Further, as an element used for calculating the required calculation amount Kp, type I is used, which indicates whether the predetermined frame is intra-frame coding or inter-frame coding. The required calculation amount Kp is a small value when the predetermined frame is intra-frame coding, and is a large value when the predetermined frame is the inter-frame coding. That is, when using the difference absolute value sum Z, the necessary calculation amount calculation means 2 calculates the difference absolute value sum Z=Σ|Xij−Yij|, and then the necessary calculation amount Kp=G(Z, Wa, Wb, Qprev, M, B, C, S, BR, ΔQprev, D, Kp') are calculated.

  The function G will be described below. When the image change between the previous frame and the predetermined frame is large (small), that is, when the absolute difference sum Z is large (small), the number of macroblock matching executed in the predetermined frame becomes large (small), The amount of calculation required for frame motion detection processing (depending on the number of times macroblock matching is performed) becomes large (small). Further, if the activity amount Wa of the predetermined frame is large (small), it means that the predetermined frame contains a large amount (small) of high frequency components of the image. In this case, the number of effective blocks generated in the encoding process of the predetermined frame, The number of effective coefficients becomes large (small), the amount of calculation required for IDCT processing of a predetermined frame (depends on the number of effective blocks generated), and the amount of calculation required for IQ processing (depends on the number of effective coefficients generated). ), the amount of calculation required for VLC processing (depending on the number of effective coefficients generated) becomes large (small). Therefore, the function G is configured to set Kp large (small) when parameters such as Z and Wa are large (small).

  Since a moving image has a large correlation between consecutive frames, the number of macroblock matchings executed in the encoding process, the number of effective blocks generated in the encoding process, the number of effective coefficients, and the amount of calculation required in the encoding process , The activity amount becomes a very close value between temporally consecutive frames. Therefore, when M, B, C, S, and Wb are large (small), the number of macroblock matchings, the number of effective blocks, the number of effective coefficients, the amount of calculation required for encoding processing, and the amount of activity are large even in a predetermined frame. There is a high probability that it will become smaller. Furthermore, when the predicted calculation amount calculated by the necessary calculation amount calculation means becomes a value close to the calculation amount required for the actual encoding process, S≈Kp′. Therefore, the function G is configured to set Kp large (small) when parameters such as M, B, C, S, Wb, and Kp' are large (small).

  When the target bit rate is large (small), the value of the quantization step size is set small (large), and as a result, the number of effective blocks and the number of effective coefficients generated in the encoding process become large (small). When the number of generated bits in the previous frame is large (small) as compared with the target bit rate, the value of the quantization step size of a predetermined frame is set small (large), and the number of effective blocks generated in the encoding process, The number of effective coefficients becomes small (large). Therefore, when the coding bit rate BR of a predetermined frame is large (small), the above function G is such that the actual number of generated bits D of the previous frame is large (small) as compared with BR so that Kp is set large (small). In this case, Kp is set to be small (large). Further, by considering the average quantization step size Qprev of the previous frame and the difference ΔQprev between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the immediately preceding frame, the above function G is The calculated Kp can be set to a value close to the amount of calculation required to actually encode the predetermined frame.

  It should be noted that the moving image coding bit rate 109, the type 108 of whether the predetermined frame and the previous frame are intra-frame coding or inter-frame coding, the activity amount 110 of the previous frame, and the necessary calculation amount calculation. The required calculation amount 116 of the previous frame calculated by the means is previously stored in the element memory 8 which is a storage area for storing elements, and is read and used by the required calculation amount calculation means 2 when calculating the required calculation amount Kp. To be done. Average value 107 of quantization step size of previous frame, number of macroblock matching 111 of previous frame, number 112 of effective blocks of previous frame, number 113 of effective coefficients of previous frame, average value of quantization step size of previous frame and its The difference 114 from the average value of the quantization step size of the immediately preceding frame and the processing amount 115 actually required for the encoding of the previous frame are the moving image encoding means when the encoding process of the previous frame is performed. 5 is fed back to the required calculation amount calculation means 2. In the required calculation amount calculation means 2, only one of these elements may be used, or a plurality of elements may be used in combination.

  (Step 3: Operation Determining Step) The operation determining means 3 performs a calculation for predicting the operating frequency Fe (cycles/second) for the processing of the predetermined frame, based on the value of the required calculation amount Kp. That is, the minimum unit in which the processing time is defined by the encoding method is one frame, and assuming that the time assigned to the encoding processing of the predetermined frame is Te (second), the operating frequency Fe required for the predetermined frame is Fe. (Cycle/second), that is, the operating frequency Fe (cycle/second) capable of encoding the required calculation amount Kp within the time Te (second) is represented by Fe=Kp/Te, and therefore the operation determining means 3 Calculates the operating frequency Fe=Kp/Te. However, the time Te assigned to the encoding process of the predetermined frame is the time Tp for predicting the amount of calculation for the predetermined frame from the time limit Tf of the processing of one frame, and the operating frequency/operating power supply voltage/substrate bias voltage It is the time obtained by subtracting the change time Ts. As shown in FIG. 5, the operating power supply voltage, the substrate bias voltage, and the operating frequency supported by the peripheral device including the processor 1 and/or the local decoding memory 6 can be changed in r stages (r is an integer of 2 or more). In this case, the operation determining means 3 performs a calculation to select the operating frequency F(n) that satisfies F(n)>Fe and Fe>F(n−1) as the operating frequency for performing the encoding process of the predetermined frame, A calculation for selecting the operating power supply voltage Vdd(n) and the substrate bias voltages Vbn, Vbp(n) suitable for the operating frequency F(n) is performed, and the peripheral device including the processor 1 and/or the local decoding memory 6 and the like is determined. The operating power supply voltage, the substrate bias voltage, and the operating frequency are instructed to the operation control means 4 so that the operating frequency F(n), the operating power supply voltage Vdd(n), and the substrate bias voltages Vbn, Vbp(n) are used ( Reference numeral 102). Note that n is an integer of 1 or more and r or less.

  FIG. 5 is a diagram showing a relationship among operating frequency, operating power supply voltage, and substrate bias voltage. In the operation determining means 3, for each operating frequency, the operating power supply voltage and the substrate bias voltage are adjusted so that the current consumed by the processor 1 or peripheral devices including the processor 1 and the local decoding memory 6 and the like becomes equal to or less than a predetermined value. Is preset. For example, from the relationship between the subthreshold leakage current Ist and the charge/discharge current Icd, the operating power supply voltage Vdd and the substrate bias voltages Vbn and Vbp that minimize the power consumption P for each operating frequency are obtained by experiments and calculations, and this operation is performed. It is desirable to use a combination of the power supply voltage Vdd and the substrate bias voltages Vbn and Vbp. Here, when the current is minimized, the current obtained by using one or more of each current element is used for the calculation. It should be noted that the substrate bias voltage may be automatically calculated for the operating power supply voltage according to the operating frequency by the hardware and/or the program built in the operation determining means 3. Further, the operating power supply voltage and the substrate bias voltage may be calculated for the operating frequency by the hardware and/or the program built in the operation determining means 3.

  (Step 4) The operation control means 4 uses the values of the operation power supply voltage Vdd(n) and the substrate bias voltages Vbn(n) and Vbp(n) and the operation frequency F(n) instructed by the operation determination means 3 as a processor. 1 and/or a peripheral device including the local decoding memory 6 and the like (reference numeral 105), and its operating power supply voltage Vdd(n) and substrate bias voltages Vbn(n), Vbp(n) and operating frequency F(n). ) Controls the processor 1 and peripheral devices to operate constantly. As a result, the peripheral device including the processor 1 and/or the local decoding memory 6 and the like operates at a constant operating power supply voltage Vdd(n) and substrate bias voltages Vbn(n), Vbp(n) and operating frequency F(n). Will work with. Specifically, the operation power supply voltage control means 4c built in the operation control means 4 controls the processor 1 to operate at a constant operation power supply voltage Vdd(n), and the substrate bias voltage Vbn control means 4d controls the n-channel MOS. Control is performed to operate the processor 1 with a constant substrate bias voltage Vbn(n) for the transistor, and the processor 1 is operated with the constant substrate bias voltage Vbp(n) for the p-channel MOS transistor by the substrate bias voltage Vbp control means 4b. The operating frequency control means (PLL) 4b controls the processor 1 to operate at a constant operating frequency F(n).

The substrate bias voltage control is performed by applying an appropriate substrate bias voltage Vbn(n) to the n-channel MOS transistor with respect to the substrate bias voltages Vbn(n) and Vbp(n), and an appropriate substrate bias voltage to the p-channel MOS transistor. It is performed by applying the substrate bias voltage Vbp(n). Specifically, the potential difference between the substrate bias voltage Vbn(n) for the n-channel MOS transistor and the ground potential Vss is set to Vbbn(n), and the substrate bias voltage Vbp(n) and the operating power supply voltage Vdd for the p-channel MOS transistor are set. The potential difference from (n) is Vbbp(n). That is,
Vbbn(n)=Vbbn(n)+Vss
Vbp(n)=Vbbp(n)+Vdd(n)
The relationship is established. The voltages Vbbn(n) and Vbbp(n) and the operating power supply voltage Vdd(n) can be set independently. However, Vbbn(n) is a voltage applied to the drain-source pn junction of the n-channel MOS transistor, and this voltage should not exceed the diffusion potential Vφ, and Vbbp(n) is the p-channel transistor. Applied to the pn junction between the drain and the source of the above, so that this voltage does not fall below the diffusion potential −Vφ. The diffusion potential Vφ is usually 0.6V.

  (Step 5: Moving picture coding step) The moving picture coding means 5 is realized by the moving picture coding processing program Prg1 on the processor 1 of the computer, and is stored in the input frame memory 7 using the processor 1. This is means for accessing the stored input image data in units of moving image encoding and performing encoding processing. That is, the moving picture coding means 5 acquires the input image data 101 of a predetermined frame from the input frame memory 7 and codes it to generate the coded data 106. In step 4, the peripheral devices including the processor 1 and/or the local decoding memory 6 etc. are supplied with constant operating power supply voltage Vdd(n) and substrate bias voltages Vbn(n), Vbp(n). ) And the operating frequency F(n), the operation control means 4 operates in step 5 at the constant operating frequency F(n), operating power supply voltage Vdd(n), and substrate bias voltage. A predetermined moving picture coding means 5 is used to perform coding by using the processor 1 while operating the peripheral devices including the processor 1 and/or the local decoding memory 6 and the like at Vbn(n) and Vbp(n). The frame will be encoded. For example, for an image with a large amount of motion (input image data 101 of a predetermined frame), peripheral devices including the processor 1 and/or the local decoding memory 6 are constantly operated at a high frequency, and for an image with little motion. It is possible to reduce the power consumption by operating constantly at a low frequency. Further, the moving picture coding means 5 is provided with a local decoder having a function of decoding the coded data 106, and the coded data 106 of a predetermined frame is decoded by the local decoder and stored in the locally decoded frame memory 6 as the locally decoded data. It is stored as 103. The locally decoded data 103 of the predetermined frame is used when calculating the necessary calculation amount Kp for the frame to be encoded next to the predetermined frame. The encoded data 106 of a predetermined frame is transmitted through a transmission path or stored in a storage medium.

  The control of the operating power supply voltage and the substrate bias voltage is performed according to the operating frequency F by at least one of the operating power supply voltage Vdd, the substrate bias voltage Vbp of the p-channel MOS transistor, and the substrate bias voltage Vbn of the n-channel MOS transistor. You may control only a voltage.

  Further, not only the operating frequency, the operating power supply voltage, and the substrate bias voltage used for the processing of one frame are constantly operated, but also the operating frequency, the operating power supply voltage, and the substrate bias voltage are set to N sets (N is positive) by the operation determining means 3. Integer) may be selected. In this case, the operating time of each of the N sets of operating frequencies, operating power supply voltages, and substrate bias voltages is calculated, and the processor 1 is operated.

(First failure avoidance step)
The encoding processing system S1 includes a first failure avoiding means for avoiding a failure phenomenon, and the first failure avoiding step is executed by interrupting the moving image encoding step. When it is determined that the encoding process of the predetermined frame cannot be completed within the allocated time Te, the first failure avoidance unit performs normal encoding so that the encoding process can be completed within the allocated time Te. The failure phenomenon is avoided by switching to processing that can reduce the amount of calculation of processing. At this time, the timing Ti at which the process is switched is calculated by performing the interrupt process and temporarily interrupting the encoding process. Further, in order to minimize the number of macroblocks that perform the process of reducing the calculation amount, the timing Ti may be updated according to the encoded process.

(Calculation/determination of switching timing Ti)
The determination of the timing Ti at which the first failure avoidance means is executed will be specifically described. FIG. 7 shows the relationship between the time when an interrupt is made and the remaining calculation amount. The time Te allocated to the encoding processing of one frame is MB, the number of macroblocks in one frame is MB, and the maximum operation amount of one macroblock is Kw. One macroblock is required for the processing by the first failure avoidance means. Let Ks be a calculation amount. The processor 1 reads the number MBi (reference numeral 117) of the macroblocks for which the encoding processing has been completed from the processed macroblock number register 10 and calculates the switching timing Ti by the following mathematical expression.
Ti=Te-(Ks x (MB-MBi))/F
However, in (step 5), Ti is calculated in advance with MBi=0 before starting the encoding process. The update of Ti is performed at a time Ti calculated in advance, at which time the encoding process is temporarily interrupted by the interrupt process and whether Ti is updated is Kw×(MB-MBi)>F×(Te- Ti) or Kw*(MB-MBi)<F*(Te-Ti). Here, MBi≠MB as a premise for the determination. If Kw×(MB−MBi)<F×(Te−Ti), it is determined that there is a margin in the amount of calculation, and Ti is updated by the above formula. If Kw×(MB-MBi)>F×(Te-Ti), it is determined that there is no margin in the amount of calculation, and the first failure avoiding means 9 switches to the process of reducing the amount of calculation. If MBi=MB when deciding whether to update Ti, the encoding processing routine is ended without switching the processing. Examples of the first failure avoidance means include Examples 1 to 4 below.

  (Example 1 of First Failure Avoidance Means) In the first failure avoidance means 91 of Example 1, the moving picture coding means 5 executes the coding processing routine of the input image data 101 of a predetermined frame in step 5. At this time, at the timing Ti determined as described above, a process of forcibly converting only the color difference block into an invalid block is performed on the macro block that has not been encoded. By performing processing to forcibly invalidate only the color difference blocks of macro blocks and performing encoding processing for luminance blocks in the same way as usual, it is possible to prevent deterioration of image quality and force all blocks. It is possible to improve the image quality rather than to invalidate it to avoid the failure.

  (Example 2 of First Failure Avoidance Means) The first failure avoidance means 92 of Example 2 performs the encoding at the timing Ti determined above while executing the encoding processing routine of the predetermined frame. The intra-frame coding process is forcibly performed on the unfinished macroblock. As compared with the case of performing inter-frame coding, since the motion vector detection processing that occupies the most processing amount is not performed, the actual calculation amount can be reduced. In addition, since the coding process is performed up to the last macroblock, the number of corresponding macroblocks is increased as compared with Example 1, but the image quality can be further improved.

  (Example 3 of First Failure Avoidance Means) The first failure avoidance means 93 of Example 3 performs the encoding at the timing Ti determined above while executing the encoding processing routine of the predetermined frame. Inter-frame coding processing is performed on the unfinished macroblock by forcibly setting the magnitude of the motion vector to 0. In the example 3, since the inter-frame coding process is performed without performing the motion vector detection process, the reduction amount of the calculation amount is small and the number of applicable macroblocks is increased, but the influence on the coding process of the next frame is an example. It is smaller than 2 and the deterioration of the image quality can be further improved.

  (Example 4 of First Failure Avoidance Means) The first failure avoidance means 94 of Example 4 forcibly performs a quantization step on a macroblock that has not been encoded at the timing Ti determined above. Encoding is performed by increasing the size. As a result, the number of valid blocks and the number of valid coefficients are reduced for the macroblock, and among the coding processes, the process performed on the valid block and the variable length coding process performed on the valid coefficient. The number of times can be reduced.

  In each of the examples of the first failure avoidance means described above, the content of normal encoding processing is omitted, and the motion vector is forcibly set to 0, or the number of effective blocks and the generation of effective coefficients are suppressed, thereby reducing the required calculation amount. is doing. Therefore, by performing the failure avoidance processing by the first failure avoidance means in a predetermined frame, in the processing of the next frame, the macro block of the next frame corresponding to the macro block for which the failure avoidance processing is executed in the predetermined frame is performed. When the inter-frame coding process is performed, the image quality may be affected. For this reason, there is a possibility that the image quality will deteriorate in the next frame even though the image quality deterioration is prevented in the predetermined frame by the failure avoidance processing.

  Therefore, in order to mitigate the influence on the encoding process of the macroblock of the next frame corresponding to the macroblock for which the first failure avoidance means is executed in the predetermined frame, the quantization step size is reduced for the macroblock. The relaxation processing is performed by the relaxation means 95 that performs the encoding processing. By this processing, the generated code amount assigned to the corresponding macroblock is increased, the information obtained from the macroblock is increased, and the image quality can be improved.

(Proof 1)
Below, it will be proved that the present invention can further reduce the power consumption due to the subthreshold leakage current, as compared with the conventional technique in which one frame is encoded while changing the operating frequency of the processor a plurality of times. For example, the substrate bias voltage and the operating frequency of the processor 1 are variable in r stages as shown in FIG. 5, the required calculation amount of any one frame is Kt, and the time allocated to the processing of that frame is Tt. As shown in FIG. 6A, the operating frequency is set to Ft, the substrate bias voltage when operating the processor 1 at the operating frequency Ft is set to Vbp and Vbn, and the threshold voltage suitable for the substrate bias voltages Vbp and Vbn is set. Is set to Vt, the case where the processing of the required calculation amount Kt ends at time Tt is set to Case 1, and the operating frequency of the initial value is set to h*Ft as shown in FIG. The substrate bias voltage when operating at Ft is Vbp1 and Vbn1, the threshold voltage suitable for the substrate bias voltages Vbp1 and Vbn1 is Vt1, and the operating frequency of the processor is changed to h*Ft/2 when time T1 elapses. Then, the substrate bias voltage when operating the processor 1 at the operating frequency h*Ft/2 is set to Vbp2 and Vbn2, the threshold voltage suitable for the substrate bias voltages Vbp2 and Vbn2 is set to Vt2, and the required calculation amount Kt is calculated at the time T1+T2. Consider the case where the processing is terminated as Case 2, and the case where the arbitrary one frame is encoded for each of Case 1 and Case 2. However, the threshold voltage is Vt1>Vt>Vt2, and the power consumption due to the subthreshold leakage current is
Pst=Vdd×I ₀ ×10^(-Vt/S)
I ₀ : constant, Vdd: operating power supply voltage, Vgs: gate-source voltage,
Vt: threshold voltage, S: subthreshold swing
Is expressed as Using this, the power consumption Pst1 due to the sub-threshold leakage current of Case1 and the power consumption Pst2 due to the sub-threshold leakage current of Case2 are calculated,
_{Pst1 = Vdd × I 0 × 10} ^ (- Vt / S) × Tt
_{Pst2 = Vdd × I 0 × 10} ^ (- Vt1 / S) × T1 + I 0 × 10 ^ (- Vt2 / S) × T2
Next to
Pst1:Pst2=10^(-Vt/S)*Tt: (10^(-Vt1/S)*T1+10^(-Vt1/S)*T2)
Becomes Here, for example, if h=1.5, Ta=1/3×Tt, Tb=2/3×Tt, Vt1=3×S, Vt2=S, Vt=2×S,
Pst1:Pst2=10 ⁻² :(10 ⁻³ /3+10 ⁻¹ ×2/3)
≈0.01:0.07
And Pst1<Pst2. That is, when processing a predetermined amount of calculation for a certain period of time, despite the same amount of calculation Kt, as in the case of Case 1, the minimum operating frequency at which the process can be completed within that period of time results in the entire processing time. It can be seen that operating the substrate bias voltage of the processor at a constant level consumes less power than in the case of Case 2 in which the operating frequency is changed during the processing time as in the conventional case. Therefore, according to the present invention, in which the encoding process for one frame is performed while operating the processor 1 at a constant substrate bias voltage and operating frequency, the substrate bias voltage and operating frequency are determined for each block, so that one frame is processed. It can be seen that the power consumption is reduced as compared with the conventional technology in which the operating frequency is changed many times during encoding.

(Proof 2)
Below, it will be proved that the present invention can achieve lower power consumption as compared with the conventional technology in which one frame is encoded while changing the operating power supply voltage and operating frequency of the processor a plurality of times. For example, when a certain amount of calculation Kt is performed at a certain time Tt, control is performed at the same frequency during the certain time, and the frequency Ft is Ft=Kt/Tt.
When set to, low power consumption can be realized. For example, the operating power supply voltage and operating frequency of the processor 1 are variable in r stages as shown in FIG. 5, the required calculation amount of any one frame is Kt, and the time allocated to the processing of that frame is Tt. As shown in FIG. 6A, when the operating frequency is set to Ft, the operating power supply voltage when operating the processor 1 at the operating frequency Ft is set to Vdd, and the processing of the required calculation amount Kt ends at time Tt ( That is, (when the operating frequency is constant) is set to Case 1, and the operating frequency of the initial value is set to h*Ft as shown in FIG. 6B, and the operating power supply for operating the processor at the operating frequency h*Ft. The voltage is V1, the operating frequency of the processor is changed to h*Ft/2 when the time T1 has elapsed, the operating power supply voltage when operating the processor 1 at the operating frequency h*Ft/2 is V2, and the time T1+T2 Consider that the case where the processing of the required calculation amount Kt is completed (that is, the operation frequency is switched once) is set as Case 2 and the arbitrary one frame is encoded for each of Case 1 and Case 2. Both have the same calculation amount, that is, Kt (cycle). On the other hand, the power consumption is
P=α×C×f×Vdd ² ×t
α: coefficient, C: number of transistors of processor f: operating frequency, Vdd: operating power supply voltage, t: operating time. When the power consumption Pa of Case 1 and the power consumption Pb of Case 2 are calculated using this,
Pa=α×C×Ft×V ² ×Tt
Pb=α×C×(h×Ft)×V1 ² ×T1+α×C×(h×Ft/2)×V2 ² ×T2
Next to
Pa: Pb=V ² ×Tt: (h×V1 ² ×T1+(h/2)×V2 ² ×T2)
Becomes Here, for example, if h=1.5, T1=1/3×Tt, Tb=2/3×Tt, V=1, V1=1.5, V2=0.75,
Pa:Pb=1 ² : (1.5×1.5 ² /3+(1.5/ ² )×0.75 ² ×(2/3)
≈ 1:1.41
And Pa<Pb. That is, when processing a predetermined amount of calculation for a fixed time, despite the same amount of calculation Kt, as in Case 1, the minimum operation frequency at which the processing can be completed within that time is used throughout the processing time. It can be seen that the constant power consumption of the processor is lower than that of the case 2 in which the operating frequency is changed during the processing time as in the conventional case. Therefore, according to the present invention in which the encoding processing of one frame is performed while operating the processor 1 at a constant operating power supply voltage and operating frequency, the operating power supply voltage and operating frequency are determined for each block, so that one frame is processed. It can be seen that the power consumption can be reduced as compared with the conventional technique in which the operating power supply voltage and the operating frequency are changed many times during encoding.

  In the operating frequency prepared for the processor, select an operating frequency that is equal to or higher than the predicted operating frequency and an operating frequency that is lower than the predicted operating frequency, interrupt at a predetermined timing, and operate in two steps. Etc. may be controlled. At the time of the interruption, the remaining amount of the necessary calculation amount of the predetermined frame calculated by the necessary calculation amount calculation means is the amount of calculation actually required for the encoding or decoding processing of the predetermined frame by the encoding or decoding processing means. If it is smaller than the remaining amount, the switching timing may be advanced to prolong the time for operating at a high operating frequency.

Furthermore, the moving image coding processing system S1 requires the necessary calculation of the subsequent frame that is coded after the predetermined frame when the calculation amount of the coding process of the predetermined frame is reduced by the first failure avoidance means. The amount may be increased. The process of increasing the necessary calculation amount of the subsequent frame is a process of increasing the necessary calculation amount Kp of the subsequent frame by a predetermined value in the process of the necessary calculation amount calculation unit 2 of the subsequent frame in which the encoding process is performed after the predetermined frame. And is executed in the necessary calculation amount calculation step (Step 2) of the subsequent frame. Specifically, the necessary calculation amount Kp is multiplied by m (m is a real number of 1 or more) in the necessary calculation amount calculation step of the subsequent frame performed after the encoding process of the predetermined frame. For example, if m=1.1, it is possible to allow (increase) a 10% margin with respect to the calculated required calculation amount Kp. Further, a real number n (n is a real number of 0 or more) may be added to the required calculation amount Kp, and a margin can be provided (increased) at a constant value regardless of the calculated value of the required calculation amount. .. Using the above example, the required calculation amount Kp finally calculated is
Kp=G(Z)×m (Formula 1)
Kp=G(Z)+n (Equation 2)
Required by. Combining the two formulas,
Kp=G(Z)×m+n (Equation 3)
May be

Further, when the required calculation amount of the predetermined frame is smaller than the actual calculation amount and the first failure avoidance means is executed, the function G() for calculating the required calculation amount in the necessary calculation amount calculation step of the subsequent frame. The coefficient of any element used for Z) is multiplied by m (m is a real number of 1 or more). For example, when the macroblock matching count M becomes small due to the execution of the failure avoidance process, using the above example, the required calculation amount Kp calculated in the next frame is
Kp=j+(α×m)×M+βB+γC+δZ+εΔQprev
Kp=j+(α+n)×M+βB+γC+δZ+εΔQprev
Required by. Combining two equations Kp=j+(α×m+n)×M+βB+γC+δZ+εΔQprev
May be Further, the number of elements is not limited to one and may be increased. All elements used in the function may be multiplied by m (m is a real number of 1 or more), or all elements may be added with a real number n (n is a real number of 0 or more). ), (3) may be increased by a constant value regardless of the value of the required calculation amount calculated.

(Second embodiment)
FIG. 8 is a schematic block diagram of the moving picture coding processing system S2 of the present embodiment, and FIG. 9 is a schematic flowchart explaining a moving picture coding method realized by this system S2. The moving image coding processing system S2 of the present embodiment includes a second failure avoiding means 96. The second failure avoidance means 96, after the encoding process of the predetermined frame is completed, the required operation amount Kp calculated by the required operation amount calculation means in the predetermined frame and the operation amount actually required for the encoding process of the predetermined frame. In the case of "required calculation amount Kp<actually required calculation amount", the following calculation is required in the processing of the calculation amount required calculation means of the subsequent frame in which the encoding process is performed after the predetermined frame. It has a function of increasing the calculation amount Kp. The other points are similar to those of the first embodiment. The moving image coding processing system S2 may include both the first failure avoidance means/steps and the second failure avoidance means/steps, or may include only the second failure avoidance means/steps. good.

(Second failure avoidance step)
The second failure avoidance unit 96 is a unit that functions as a part of the required calculation amount calculation unit 2, and is executed within the required calculation amount calculation step (Step 2). Specifically, the second failure avoidance unit 96 compares the required calculation amount Kp calculated by the necessary calculation amount calculation unit in a predetermined frame with the calculation amount actually required for the encoding process of the predetermined frame, When “required calculation amount Kp<actual calculation amount required”, the necessary calculation amount Kp of the subsequent frame is increased in the process of the necessary calculation amount calculation means of the subsequent frame in which the encoding process is performed after the predetermined frame. .. In order to increase the required calculation amount Kp of the subsequent frame, for example, the required calculation amount Kp is multiplied by m (m is a real number of 1 or more) in the calculation step of the required calculation amount of the subsequent frame which is encoded after the predetermined frame. ) Do. If m=1.1, it is possible to allow (increase) a margin of 10% for the calculated required calculation amount Kp. Further, a real number n (n is a real number of 0 or more) may be added to the required calculation amount Kp, and a margin can be provided (increased) at a constant value regardless of the calculated value of the required calculation amount. .. Using the above example, the required calculation amount Kp finally calculated is
Kp=G(Z)×m (Formula 1)
Kp=G(Z)+n (Equation 2)
Required by. Combining the two formulas,
Kp=G(Z)×m+n (Equation 3)
May be

Further, when the required calculation amount of the predetermined frame is smaller than the actual calculation amount and the first failure avoidance means is executed, the function G() for calculating the required calculation amount in the necessary calculation amount calculation step of the subsequent frame. The coefficient of any element used for Z) is multiplied by m (m is a real number of 1 or more). For example, when the macroblock matching count M becomes small due to the execution of the failure avoidance process, using the above example, the required calculation amount Kp calculated in the next frame is
Kp=j+(α×m)×M+βB+γC+δZ+εΔQprev
Kp=j+(α+n)×M+βB+γC+δZ+εΔQprev
Required by. Combining two equations Kp=j+(α×m+n)×M+βB+γC+δZ+εΔQprev
May be Further, the number of elements is not limited to one and may be increased. All elements used in the function may be multiplied by m (m is a real number of 1 or more), or all elements may be added with a real number n (n is a real number of 0 or more). ), (3) may be increased by a constant value regardless of the value of the required calculation amount calculated.

(Third Embodiment)
The moving picture coding processing system S3 according to the third embodiment of the present invention completes the coding processing when the time allotted for the coding processing of a predetermined frame in place of the first failure avoidance means has elapsed. If not, a third failure avoiding means for performing processing for avoiding the failure phenomenon is provided. The other points are similar to those of the second embodiment.

(Third failure avoidance step)
FIG. 10 is a schematic block diagram of a moving picture coding processing system S3 including the third failure avoiding means 97, and FIG. 11 is a schematic flowchart explaining a moving picture coding processing method realized by this system S3. is there. The third failure avoidance means 97 executes the encoding processing routine when the moving image encoding means 5 executes the encoding processing routine of the input image data 101 of the predetermined frame in step 5 and the assigned time Te has elapsed. If the coding process is not finished by judging whether it is finished or not, the processing time of the predetermined frame is extended until the coding process is completed by using the time allocated to the next frame. By doing so, the failure phenomenon that the encoding process cannot be completed can be avoided.

(Input frame memory)
The third failure avoidance means 97 will be specifically described below. First, as one of the reasons that the processing time of one frame is limited to the time Tf due to the regulation of the encoding method (MPEG, etc.), the data of the input image is stored in the input frame memory at intervals of the frame rate in the input frame memory. That is, the data in the input frame memory is updated frame by frame at intervals of the frame rate of the input image. The interval at which the data in the input frame memory is updated differs depending on the frame rate, and the encoding processing time for one frame is assigned a time limited by the frame rate of the input image. Here, the input frame memory 7 is a memory capable of storing data for a plurality of frames (here, two frames), the start address of the data is designated by the address of the memory, and the data is stored from the designated address. It shall be stored sequentially.

(Processing time control)
FIG. 12 is a diagram showing the transition of processing time in the case of normal encoding processing, and FIGS. 13, 14, and 15 are cases of encoding processing in which the third failure avoidance step is executed by the third failure avoidance means. It is a figure which shows the transition of the processing time of. For example, in real-time processing, if 30 image data are input per second, the data in the external memory is updated every 1/30 second. When 10 images are coded per second by the moving picture coding processing system, the input image is coded at a ratio of 1 in 3 images. Here, the time assigned to one frame to be encoded is Tf=1/10 seconds, and the frame numbers of the input image are #1, #2, #3, #4,... The numbers of the normally encoded frames are #1, #4, #7, #10,... As shown in FIG. The third failure avoidance means, when the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount Km in a predetermined frame, that is, the time allotted for the encoding process has elapsed. However, even if the coding process is not completed, the processing time Te of the predetermined frame is extended by using the coding process time assigned to the next frame, and the coding process of the predetermined frame is continued. It is a means of controlling the status signal of the input frame memory to complete the process. Examples of the third failure avoidance means include Examples 1 and 2 below.

  (Example 1 of Third Failure Avoidance Means) The third failure avoidance means 97 has a function of executing procedures 1 to 4 described later. (Procedure 1) The state Te of the input frame memory is controlled to extend the time Te allocated to the encoding process in the predetermined frame, and the encoding process of the predetermined frame is continued. (Procedure 2) When the encoding process of the predetermined frame is completed, the time during which the encoding process of the next frame is possible is calculated. (Procedure 3) Input frame memory state switching control is performed. Hereinafter, the procedures 1 to 3 will be described in detail.

  (Procedure 1: Extension of time Te) FIG. 13 shows the transition of the encoding processing time by the failure avoiding means 97. Using the above example, the #4 frame data is stored in the input frame memory 7b, and the #7 frame data is stored in the input frame memory 7b. Here, when the coding of #4 cannot be completed in the time Te allocated to the coding process, in order to extend the coding processing time Te of #4 and continue the process, the frame data of #4 is used. Need to hold. Therefore, the state control of the input frame memory is performed, the input frame memory 7b is controlled to the unwritable state, the input frame memory 7a is maintained in the writable state, and the writing destination of the frame data #8 is set to the input frame memory 7a. Change to. Since the input frame memory 7b can hold the #4 frame data and the #4 coding process can be continued, the time can be extended until the #4 coding process is completed.

  (Procedure 2: Calculation of Next Frame Processing Time) When the coding process of #4 is completed, the extended time To spent in the coding process of #4 is obtained after the allocated time Te has elapsed. The encoding processing time Tl of the next frame to be encoded is calculated by Tl=Te-To. Here, for the time Tl, the required calculation amount is calculated by the necessary calculation amount calculation means for the next frame to be encoded, and the operation frequency/operation power supply voltage and the substrate bias voltage are determined by the operation determination means. The time Tl is calculated in advance because it will be used when performing. The time Tl may be calculated when the necessary calculation amount calculation means for the next frame is executed.

  (Procedure 3: Memory State Switching Control) In the extension time To calculated in Procedure 2, if To<Tf/3, the next encoding target frame is #8. By switching the state of the input frame memory 7 when the writing of the #8 frame data is completed, the input frame memory 7a is in the reading state of the #8 frame data and the input frame memory 7b is in the next frame. Then, the writing state of #9 is changed to the coding step of #8. When Tf/3<To<2Tf/3, as shown in FIG. 14, the next encoding target frame is #9, and when the writing of the frame data of #9 ends, the input frame memory 7 The state control is switched in the same manner, and the process proceeds to the encoding step of #9. By controlling the state of the input frame memory according to the above procedure, the processing time of a predetermined frame can be changed and the time can be extended. By completing the encoding processing of the predetermined frame, the failure can be completely avoided. it can.

  In addition, by calculating the encoding processing time Tl of the next frame, it is possible to determine the optimum operating frequency, operating power supply voltage and substrate bias voltage for the encoding of the next frame, and it is possible to efficiently reduce the operating frequency to the subsequent frames. Power consumption can be reduced. The time Tl may be calculated when the necessary calculation amount calculation means of the encoding step of the next frame to be encoded is executed.

  (Example 2 of Third Failure Avoidance Means) The third failure avoidance means 98 differs from the third failure avoidance means 97 only in the state control method of the input frame memory, and other points are the same as the third failure avoidance means 97. This is similar to the failure avoidance means 97. FIG. 15 shows the time transition of the encoding process by the third failure avoiding means 98. In the state control of the input frame memory, the third failure avoiding means 98, when the processing cannot be completed within the encoding processing time Te assigned to #4, at least for the extension time To, the input frame memories 7a and 7b. Both are controlled so that they cannot be written, and the time Te is extended by the time To. The input frame memory is in a state in which the #4 frame data stored in the input frame memory 7b and the #7 frame data stored in the input frame memory 7a are held. For the frame of #4, the breakdown phenomenon is avoided by extending the time Te. The frame of #8 input during the extension time To is not written in the input frame memory. When the coding process of #4 is completed, the write/read state of the input frame memories 7a and 7b is switched. The input frame memory 7b, which is controlled to the write state, can write the input frame data. The input frame memory 7a reads the #7 frame data, and the #9 and #10 frames are sequentially written to the input frame memory 7b. Regardless of the extension time To, after the encoding process of the #4 frame, the encoding processes of the #7 and #10 frame data are sequentially performed. Since the same frame number as the normal encoding process is encoded, the image quality equivalent to that of the normal encoding process can be obtained.

  If the frame rate of the input image data is equal to the frame rate of the moving image coding processing system, the frames to be coded are #1, #2, #3, #4, #5, #6, #7, ... and the range in which the encoding processing time can be extended is the maximum Tf. When the extension time of the encoding process of #4 exceeds Tf, #6 may be encoded as the next frame.

(Fourth Embodiment)
The moving image decoding processing system S4 according to the fourth embodiment of the present invention is a system for decoding encoded moving images. FIG. 16 is a schematic block diagram showing the operation of the moving picture decoding processing system S4. The moving image decoding processing system S4 of the present embodiment is provided with operating power supply voltage, substrate bias voltage, and operating frequency in r stages (r is an integer of 2 or more), and the operating power supply voltage, substrate bias voltage, and operation are performed by a program. It comprises a processor 1 whose frequency can be changed, an operation control means 4 for controlling the operating power supply voltage, the substrate bias voltage and the operating frequency of the processor 1, and a local decoding frame memory 46 for storing the decoded data of the previous frame. Further, in the local decoding memory 46, the operation power supply voltage, the substrate bias voltage, and the operation frequency may be controlled by the operation control means 4 similarly to the processor 1. The processor 1 includes, as means for operating on the processor 1, a necessary calculation amount calculating means 42, an operation determining means 3, a moving image decoding means 35, and a third failure avoiding means 97. Reference numeral 401 is input encoded data, reference numeral 102 is an operating power supply voltage/substrate bias voltage/operating frequency instruction, reference numeral 105 is an operating power supply voltage/substrate bias voltage/operating frequency supply, and reference numeral 406 is decoded data. The same reference numerals as those in the embodiment are parts having the same function or equivalent functions. The point that decoding is performed instead of encoding and the points other than the following are the same as in the third embodiment.

The operation of the moving picture decoding processing system S4 will be described with reference to FIG. Hereinafter, any one of the frames that will be decoded in the future (that is, the frame that will be decoded next based on the time when a certain frame is decoded, in other words, that time) , The frame that has not been decoded yet and is scheduled to be decoded in the future) is the predetermined frame, and the one frame that was decoded before the predetermined frame (the frame that was decoded in the past) is the previous frame. The process of decoding a predetermined frame will be described, but the same process is performed on any frame. The moving picture decoding processing program Prg4 that causes the computer to function as the moving picture decoding processing system S4 is almost the same as the moving picture coding processing program Prg1, but in step 5, it decodes the coded data of a predetermined frame. A computer (specifically, the processor 1 incorporated in the computer) is made to function as the moving image decoding means 45. The input coded data 401 input to the moving image decoding processing system S4 is input to the necessary calculation amount calculation means 42. The required calculation amount calculation means 42 calculates the generated information amount (the number of bits) FB of one frame of the encoded data 401 (that is, the encoded data 401 of a predetermined frame), and performs the calculation to predict the required calculation amount Kp ( Required calculation amount calculation step). The required calculation amount Kp is
Kp=G (FB, MVa, MVv, B, C, BR, Q, ΔQ, I, E, P)
It is represented by. Here, FB is the number of bits of coded data of the predetermined frame or the previous frame, MVa is the average value of the motion vector magnitudes of the predetermined frame or the previous frame, and MVv is the variance of the motion vector magnitudes of the predetermined frame or the previous frame. , B is the number of effective blocks of a predetermined frame or the previous frame, C is the number of effective coefficients of the predetermined frame or the previous frame, BR is the bit rate of the predetermined frame or the previous frame, and Q is the quantization step size of the predetermined frame or the previous frame. The average value, ΔQ is the difference between the average values of the quantization step sizes of the predetermined frame and the previous frame or the difference between the average values of the quantization step sizes of the previous frame and the previous frame, and I is whether the predetermined frame is an I picture or P The type is a picture or a B picture, E is the amount of calculation required for decoding the previous frame, and P is the required amount of calculation for the previous frame calculated by the required amount calculation unit. The function G will be described below.

  The amount of calculation required for decoding a predetermined frame depends on the number of times the IDCT process, IQ process, and VLD process are executed in the decoding of the predetermined frame. The number of executions of the IDCT process depends on the number of valid blocks included in a predetermined frame, and the number of executions of the IQ process and the VLD process depends on the number of effective coefficients included in the predetermined frame. That is, when the number of effective blocks or the number of effective coefficients included in a predetermined frame is large (small), the amount of calculation required for the decoding process becomes large (small). Therefore, the function G is configured to set Kp large (small) when B and C are large (small).

  When the change in the image between the previous frame and the predetermined frame is large (small), the average value MVa of the motion vector magnitudes and the variance MVv of the motion vector magnitudes are large (small). The number of blocks and the number of effective coefficients are large (small), and the amount of calculation required for encoding processing is large (small). Therefore, the function G is configured to set Kp large (small) when MVa or MVv is large (small).

  When the predetermined frame is an I picture, it is not necessary to add the predicted image and the difference image when generating the decoded data, and thus the amount of calculation required for the decoding process becomes small. Therefore, the function G is configured to set Kp small when the predetermined frame is an I picture.

  When the number of bits FB of the encoded data and the frame rate BR are large (small), the number of effective blocks and the number of effective coefficients become large (small). Therefore, the function G is configured to set Kp large (small) when FB and BR are large (small). Since the value of the quantization step size is changed when controlling the bit rate, the function G is calculated by considering the average value Q of the quantization step sizes and the difference ΔQ between the average values of the quantization step sizes. Kp can be set to a value close to the amount of calculation required to actually decode the predetermined frame.

  Since a moving image has a large correlation between consecutive frames, MVa, MVv, B, C, BR, FB, and Q have a close value between a predetermined frame and the previous frame. Therefore, when these parameters are used in the function G, the values in the predetermined frame may be used, or the values in the previous frame may be used. When using the value in a predetermined frame, after receiving the input encoded data, a part of this data is decoded and the value is extracted and used. At this time, there is an advantage that the predicted calculation amount Kp can be made closer to the calculation amount necessary for the actual decoding process by using the value in the predetermined frame. When the value in the previous frame is used, the predictive calculation amount Kp can be calculated before receiving the input coded data of a predetermined frame. Therefore, while the input coded data is being received, the received data is decoded. There is a merit that processing can be performed simultaneously.

  Further, since a moving image has a large correlation between consecutive frames, the calculation amount required for the decoding process of a predetermined frame is a value close to the calculation amount E actually required for the decoding process of the previous frame. Further, when the predicted calculation amount calculated by the necessary calculation amount calculation means is a value close to the calculation amount required for the actual decoding process, P≈E. Therefore, by taking E and P into consideration, Kp calculated by the function G can be set to a value close to the amount of calculation required to actually decode the predetermined frame. FB is the generated information amount (bit number) for one frame. However, the function G is a function derived using one or more of the elements FB, MVa, MVv, B, C, BR, D, Q, ΔQprev, I, E, and P. The required calculation amount Kp is a calculation performance (frequency, cycle) predicted to be necessary for a predetermined frame, and has a high value if the number of bits FB in the predetermined frame is large, and a low value if the number of bits FB is small. Further, as the element of the required calculation amount calculation means 42 which is a calculation for predicting the required calculation amount Kp, it is possible to use the type of whether the predetermined frame is intra-frame coding or inter-frame coding, The required calculation amount Kp is a small value when the predetermined frame is intra-frame coding, and is a large value when the predetermined frame is the inter-frame coding. Further, the required calculation amount Kp is the average value of the motion vector magnitudes (of the frame to be decoded or in the previous frame) MVa, the variance of the motion vector magnitudes (of the frame to be decoded or in the previous frame). MVv, the number of effective blocks (of the frame to be decoded or in the previous frame) B, the number of effective coefficients (of the frame to be decoded in the previous frame or of the previous frame) C, bit rate (decoding from now on) BR of the frame to be reproduced, or of the previous frame), generated information amount (of the frame to be decoded or of the previous frame) FB, average value of the quantization step size (of the frame to be decoded or of the previous frame) Q), the difference in the average value of the quantization step size (the difference between the Q of the frame to be decoded and the Q of the frame one before, or the difference between the Q of the frame one before and the Q of the frame two before). ) ΔQ, type I whether it is an I picture, a P picture or a B picture, the amount of calculation E actually required for decoding the previous frame, the predicted value of the amount of calculation required for decoding the previous frame ( That is, the necessary calculation amount of the preceding frame calculated by the necessary calculation amount calculation unit P is also influenced, and these may be used as elements in the necessary calculation amount calculation unit 42. For example, the average value of the motion vector size (of the frame to be decoded or in the previous frame) MVa, the variance of the motion vector size (of the frame to be decoded in the previous frame, or of the previous frame) MVv, valid For the number of blocks (of the frame to be decoded or in the previous frame) B and the number of effective coefficients (of the frame to be decoded in the previous frame or of the previous frame) C, for each element, the value of the other element When the value of the element is large, the required calculation amount Kp is relatively large compared to the case where the element value is small, and when the element value is small, the necessary calculation amount is compared to the case where the element value is large. The quantity Kp is made relatively small. In the necessary calculation amount calculating means 42, only one of these elements may be used, or a plurality of elements may be used in combination. That is, since these plurality of elements affect the required calculation amount required for the decoding processing of the predetermined frame, the necessary calculation amount calculation means 42 determines the necessary calculation amount Kp( By performing the calculation so as to increase or decrease the number of cycles, the required calculation amount Kp calculated by the necessary calculation amount calculation means 42 becomes a value closer to the calculation amount when the decoding process is actually performed.

  The operation determining means 3 (operation determining step) and the operation control means 4 are the same as those in the first embodiment. The moving picture decoding means 45 decodes the input coded data 401 of a predetermined frame to generate decoded data 406 (moving picture decoding step). In the decoding processing by the moving image decoding means 45, the operation controlling means 4 performs the decoding processing while operating the processor 1 at a constant operating power supply voltage, substrate bias voltage and operating frequency. For each frame, the necessary amount of calculation required before the decoding processing of that frame is calculated, and while operating the processor at a constant operating frequency, operating power supply voltage and substrate bias voltage according to the required amount of calculation, Since decoding is performed, low power consumption can be achieved. The decrypted data 406 is displayed as a moving image on an image display unit of a mobile phone or a personal computer, or stored in a storage medium such as a hard disk. In the operating frequency prepared for the processor, select an operating frequency that is equal to or higher than the predicted operating frequency and an operating frequency that is lower than the predicted operating frequency, interrupt at a predetermined timing, and operate in two steps. Etc. may be controlled.

  It is preferable that the moving picture decoding system S4 also includes the third failure avoidance means 97, 98. Each failure avoiding means is almost the same as that of the third embodiment, but is different in that it judges not the amount of calculation of the encoding process but the amount of calculation of the decoding process. The third failure avoidance means 97, 98 can avoid the failure phenomenon.

  The moving picture coding processing system according to the present invention includes the first failure avoiding means 91, 92, 93, 94, the first failure avoiding means with the mitigating means 95, the second failure avoiding means 96, and the third failure avoiding means. The means 97 and 98 may be provided individually, or each means may be appropriately combined and provided. The decryption processing system may include each of the second failure avoidance means 96 and the third failure avoidance means 97 and 98 independently, or may appropriately include the respective means in combination. For example, the encoding processing system includes any one of the first failure avoiding means 91, 92, 93, 94, the first failure avoiding means with the mitigating means 95, and the second failure avoiding means 96. In this way, if the failure cannot be avoided even if the required calculation amount is increased by the second failure avoiding means 96, the first failure avoiding means 91 performs the invalid blocking processing only on the color difference blocks, or Whether the intra-frame coding process is performed by the corruption avoiding unit 92, or the inter-frame coding process is performed by setting the magnitude of the motion vector to 0 by the first corruption avoiding unit 93, or the first corruption avoiding unit 94. Therefore, the quantization step size may be increased to perform the encoding process. In addition, the moving picture encoding or decoding processing program may realize low power consumption by controlling the operating frequency, the substrate bias voltage, and the operating voltage in two steps, or a hardware having the same function as the program. It may be realized by wear.

(Fifth Embodiment)
The first to fourth embodiments described above control the operating power supply voltage, the substrate bias voltage, and the operating frequency, but the present embodiment controls the operating power supply voltage and the operating frequency. Is intended to reduce power consumption. Note that this embodiment is not limited to a system integrated in a semiconductor. FIG. 17 is a schematic block diagram showing the operation of the moving picture coding system S5 of the present embodiment, and FIG. 18 is a conceptual diagram showing the relationship between the operating power supply voltage and operating frequency of the processor 51. The moving picture coding system S5 of the present embodiment is variable in the operating power supply voltage Vdd and the operating frequency in r stages (r is an integer on 2) instead of the processor 1 of the first embodiment. The processor 51 is capable of operating at the r-stage operating power supply voltage Vdd and operating frequency and capable of changing the operating power supply voltage and operating frequency by a program. The operation control means 54 controls the operation power supply voltage and the operation frequency of the processor 1. The processor 51, or the processor 1 and the peripheral devices (the local decoding memory 6, the input frame memory 7, etc.) are controlled by the operation control means 52 in terms of operating power supply voltage and operating frequency.

  The operation determining means 53 performs a calculation for selecting an operation frequency F(n) that satisfies F(n)>Fe and Fe>F(n−1) as an operation frequency for performing the encoding process of a predetermined frame, A calculation for selecting an operating power supply voltage Vdd(n) suitable for the operating frequency F(n) is performed, and the peripheral frequencies including the processor 1 and/or the local decoding memory 6 are set to the operating frequency F(n) and the operating power supply voltage. The operation control means 54 is instructed to operate constantly at Vdd(n) (reference numeral 502). The operation control means 54 supplies the values of the operating frequency F(n) and the operating power supply voltage Vdd(n) instructed by the operation determining means 53 to the peripheral devices including the processor 1 and/or the local decoding memory 6 and the like. (Reference numeral 505), the control is performed so that the processor 1 is constantly operated with the operating frequency F(n) and the operating power supply voltage Vdd(n). As a result, the peripheral device including the processor 1 and/or the local decoding memory 6 and the like operates at the operating power supply voltage Vdd(n) and the operating frequency F(n). Other points are almost the same as those of the first embodiment.

  The moving image encoding system S5 also includes the first failure avoidance means 91, 92, 93, 94, 95, the second failure avoidance means 96, or the third failure avoidance means 97, 98. Preferably.

(Sixth Embodiment)
Although the first to fourth embodiments described above control the operating power supply voltage, the substrate bias voltage and the operating frequency, the present embodiment controls the substrate bias voltage and the operating frequency. Is intended to reduce power consumption. FIG. 19 is a schematic block diagram showing the operation of the moving picture coding system S6 of the present embodiment, and FIG. 20 is a conceptual diagram showing the relationship between the substrate bias voltage and the operating frequency of the processor 61. The moving picture coding processing system S6 of the present embodiment, in place of the processor 1 of the first embodiment, can change the substrate bias voltages Vbn, Vbp and the operating frequency in r stages (r is an integer of 2 or more). (That is, it can operate with the substrate bias voltages Vbn, Vbp and the operating frequency in the r stage) and can change the substrate bias voltage and the operating frequency by a program. Further, the operation control means 64 controls the substrate bias voltage and the operating frequency of the processor 1. The processor 61, or the processor 61 and peripheral devices (the local decoding memory 6, the input frame memory 7, etc.) are controlled by the operation control means 64 in terms of the substrate bias voltage and the operating frequency.

  The operation determining means 63 performs a calculation to select an operation frequency F(n) that satisfies F(n)>Fe and Fe>F(n−1) as an operation frequency for performing encoding processing of a predetermined frame, A calculation for selecting the substrate bias voltages Vbn(n) and Vbp(n) suitable for the operating frequency F(n) is performed, and the peripheral devices including the processor 1 and/or the local decoding memory 6 are operated at the operating frequency F(n). ) And the substrate bias voltages Vbn(n) and Vbp(n) to instruct the operation control means 64 about the substrate bias voltage/operating frequency (reference numeral 602). The operation control means 64 includes the values of the operating frequency F(n) and the substrate bias voltages Vbn(n) and Vbp(n) instructed by the operation determining means 63 in the processor 61 and/or the local decoding memory 6 and the like. It is supplied to the peripheral device (reference numeral 605), and the processor 61 is controlled to operate constantly at the operating frequency F(n) and the substrate bias voltages Vbn(n) and Vbp(n). As a result, the peripheral device including the processor 61 and/or the local decoding memory 6 and the like operates at the constant substrate bias voltages Vbn(n) and Vbp(n) and the operating frequency F(n). Other points are almost the same as those of the first embodiment.

  The moving picture coding system S6 also includes the first failure avoidance means 91, 92, 93, 94, 95 or the third failure avoidance means 97, 98 and the second failure avoidance means 96. Is preferred.

  Also in the moving picture decoding system S4 of the fourth embodiment, instead of controlling the operating power supply voltage and the substrate bias voltage suitable for the operating frequency, the operating power supply voltage or the substrate bias voltage suitable for the operating frequency is used. It is possible to control

(Example)
An example will be described. The present example is an example of a system in which the moving picture coding system 2 of the second embodiment includes a first failure avoiding means 91 and a second failure avoiding means. An example will be described in which moving image data composed of 75 frames is used as an encoding target, and the 35th frame is used as an encoded frame. Each frame is composed of a pixel array of 144 rows and 176 columns. MPEG-4 is used as the encoding process. FIG. 21 shows an example of the relationship between the operating frequency, the operating power supply voltage, and the substrate bias voltage in the processor 1 of the moving image coding system S2. The processor 1 of the moving image coding system S2 has an operating frequency of 50 MHz to 250 MHz, an operating power supply voltage of 0.5 V to 1.0 V, and a substrate bias voltage of −1.0 V to 0.5 V, which is variable in 5 steps. ..

First, the moving image coding system S2 accesses the input frame memory 7 to acquire the 35th frame, and the necessary calculation amount calculation means 2 calculates the necessary calculation amount Kp of the frame. Specifically, as the required calculation amount Kp, first, the 34th frame is used as the previous frame, and the sum Z of absolute differences is calculated by the following formula.
Z=Σ|Xij-Yij|=41541
Next, the number M of macroblock matchings of the previous frame is M=308, the quantization step size of the previous frame (average value of the quantization step size) Qprev=3, the number of effective blocks of the previous frame B=109, and the effective coefficient of the previous frame. The number C=620, the amount of processing S actually required for encoding the previous frame S=3561303, and the encoding bit rate BR=65536 for a predetermined frame are obtained. Further, the difference ΔQprev=0 between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the immediately preceding 33rd frame is calculated. Also, the actual number of generated bits in the previous frame D=56797 is obtained. Next, the required calculation amount Kp is calculated by using the following formula using each element.
Kp=j+αM+βB+γC+δZ+εΔQprev
As described above, in the first embodiment, the required calculation amount Kp=6958217 is obtained.

Next, the operating frequency is calculated by the following formula.
Fe=Kp/Te=6958217/(1/15)=104 MHz
Calculate F(n) such that F(n)>Fe and Fe>F(n-1), and select the operating frequency F(3)=150 MHz from among the five operating frequency variable of the processor 1. Then, the operating power supply voltage Vdd(3)=0.8V, the substrate bias voltage Vbn(3)=0V, and Vbp(3)=0.8V suitable for the operating frequency F(3)=150 MHz are determined. The operation control unit 4 is instructed to operate the processor 1 at the operating frequency F=150 MHz, the operating power supply voltage Vdd=0.8V, the substrate bias voltage Vbn=0V, and Vbp=0.8V. The operation control unit 4 controls at least the processor 1 to operate at a constant operating power supply voltage F=150 MHz, operating power supply voltage Vdd=0.8 V, substrate bias voltage Vbn=0 V, and Vbp=0.8 V. Further, when the failure avoidance means 91 is provided, the switching timing time is calculated by the following mathematical formula.
Ti=Te-(Ks x (MB-MBi))/F

= 0.066666-(4750 x (99-0))/(150000000)
≈ 0.06353
Here, Ks is the number of cycles required to process only the color difference block as an invalid block in one macroblock. The moving image coding means 5 uses the processor 1 in a state in which the operating frequency F=150 MHz, the operating power supply voltage Vdd=0.9 V, the substrate bias voltage Vbn=0 V, and Vbp=0.8 V are constantly operated. Then, encoding processing is performed to generate encoded data.

Furthermore, when the first failure avoiding means 91 is provided during execution of the encoding processing routine, Kw×(MB-MBi)>F×(Te-Ti) at the timing when the Ti has elapsed. Determine whether there is. In the first embodiment, Kw x (MB-MBi) <F x (Te-Ti) at the timing of Ti = 0.06353, it is determined that there is a margin in the amount of calculation, and the switching timing is updated. At this time, since MBi=83, Ti is updated by the following formula.
Ti=Te-(Ks x (MB-MBi))/F
= 0.066666-(4750 x (99-83))/(150000000)
≈ 0.06662
At the above timing, since MBi≠MB and Kw×(MB-MBi)>F×(Te-Ti), all remaining macroblocks are switched to invalid block processing only for color difference blocks.

The step of the 35th frame ends, and the process proceeds to the step of the next 36th frame. In the case where the second failure avoidance means 96 is provided, in the necessary calculation amount calculation means, the macroblock matching count M of the previous frame is M=327, the quantization step size of the previous frame (average value of the quantization step size) Qprev=3, The number of effective blocks in the previous frame B=121, the number of effective coefficients in the previous frame C=749, the amount of processing S actually required for encoding the previous frame S=7031333, the number of actually generated bits in the previous frame D=83942 Then, it is checked whether or not the failure avoidance processing has been executed in the immediately preceding frame. Since the failure avoidance processing has been executed, the second failure avoidance means 96 performs processing for increasing the predicted value of the required calculation amount. In the present embodiment, the number of macroblock matching times M, the number of effective blocks B, and the number C of effective coefficients are multiplied by 1.1, and the necessary calculation amount Kp is calculated by the following formula.
Kp=j+(α×1.1)M+(β×1.1)B+(γ×1.1)C+δZ+εΔQprev
From the above, the required calculation amount Kp=7101472 is obtained.

  Further, the quantization step sizes Q=4 to Q=3 in the macroblock of the predetermined frame corresponding to the macroblock for which the corruption avoidance processing has been executed in the previous frame by the first corruption avoiding means 95. Thereby, the influence of the image quality of the previous frame can be mitigated and the image quality can be further improved.

Claims (18)

A processor functioning as a moving picture coding means for coding a moving picture composed of a plurality of consecutive frames in frame units;
Necessary calculation amount calculation means for calculating the necessary calculation amount Kp required for encoding one frame, and the necessary calculation amount Kp within the time Te pre-allocated for the encoding process of the one frame. An operation determining means for determining a possible operating frequency,
The moving image encoding means operates while the processor operates at the operating frequency determined by the operation determining means, and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance. Is a moving image coding processing system that performs coding processing of the one frame by
A moving image coding processing system, comprising: a first failure avoiding means for reducing a calculation amount of coding processing at a predetermined timing.   2. The moving picture coding process according to claim 1, wherein the first failure avoiding means processes a block having color difference signal information among macro blocks that have not been coded as an invalid block. system.   2. The moving picture coding processing system according to claim 1, wherein the first failure avoiding means performs intraframe coding processing on macroblocks that have not been coded.   The first failure avoiding means performs inter-frame coding processing on a macroblock for which coding has not been completed by setting a motion vector to 0 without performing motion vector detection. Video coding processing system.   2. The moving picture coding system according to claim 1, wherein the first failure avoiding means performs a coding process with a large quantization step size for a macro block that has not been coded. ..   6. The mitigation means for mitigating the influence of the first failure avoidance means on the coding process of the subsequent frame coded after the one frame, according to any one of claims 1 to 5. The moving image coding processing system according to item 1.   7. The relaxation unit reduces the quantization step size for a macroblock of a subsequent frame corresponding to the macroblock on which the processing by the first failure avoidance unit is executed in the one frame. The moving image coding processing system according to item 1.   When the calculation amount of the coding process of the one frame is reduced by the first failure avoiding means, the necessary calculation amount of a subsequent frame to be coded after the one frame is increased. The moving image coding processing system according to any one of claims 1 to 7.   The required amount of the subsequent frame is calculated by adding m times (m is a real number greater than or equal to 1) or a real number n larger than 0 to the required amount of operation of the subsequent frame or an element used to calculate the required amount of operation. 9. The moving picture coding or decoding processing system according to claim 8, wherein the amount of calculation is increased. A processor functioning as a moving picture coding or decoding means for coding or decoding a moving picture composed of a plurality of continuous frames in frame units is provided.
Necessary calculation amount calculation means for calculating a necessary calculation amount Kp necessary for encoding or decoding one frame, and the necessary calculation within a time Te pre-allocated for the encoding or decoding process of the one frame. An operation determining means for determining an operating frequency capable of encoding or decoding the quantity Kp,
The moving image encoding is performed while the processor operates at the operation frequency determined by the operation determining means and the operation power supply voltage and/or the substrate bias voltage suitable for the operation frequency within the time Te that is assigned in advance. Or a moving picture coding or decoding processing system for performing coding or decoding processing of the one frame by a decoding means,
When the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount, the necessary calculation amount of a subsequent frame encoded or decoded after the one frame is increased. 2. A moving picture coding or decoding processing system, characterized in that it comprises a failure avoiding means (2).   The second failure avoidance means adds m times (m is a real number of 1 or more) or a real number n larger than 0 to the required calculation amount of the subsequent frame or an element used for calculating the necessary calculation amount. 11. The moving picture coding or decoding processing system according to claim 10. A processor functioning as a moving picture coding or decoding means for coding or decoding a moving picture composed of a plurality of continuous frames in frame units is provided.
Necessary calculation amount calculation means for calculating a necessary calculation amount Kp necessary for encoding or decoding one frame, and the necessary calculation within a time Te pre-allocated for the encoding or decoding process of the one frame. An operation determining means for determining an operating frequency capable of encoding or decoding the quantity Kp,
Within the time Te pre-allocated by the processor, moving image coding or while operating at the operating frequency determined by the operation determining means, and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency. A moving image encoding or decoding processing system for performing encoding or decoding processing of the one frame by a decoding means,
A moving picture encoding or decoding processing system comprising a third failure avoiding means for extending the time Te at a predetermined timing.   The third failure avoidance means may be configured to detect a subsequent frame that is encoded or decoded after the one frame when the time Te previously assigned to the encoding process of the one frame is extended. 13. The moving picture coding or decoding processing system according to claim 12, wherein the time Te which is previously allocated to the coding processing or the decoding processing of the subsequent frame is changed.   The third corruption avoiding means, when the time Te pre-allocated for the encoding processing of the one frame is extended, stores the frame in the input frame memory for storing the next frame to be encoded. 14. The moving image coding processing system according to claim 12, wherein the writing destination is changed.   The third failure avoidance means, when the time Te previously assigned to the encoding processing of the one frame is extended, determines that the input frame memory storing the next encoded frame cannot be written. 14. The moving picture coding system according to claim 12 or claim 13. A moving picture coding step of coding a moving picture composed of a plurality of consecutive frames using a processor in frame units, and a necessary calculation quantity calculation for calculating a necessary calculation quantity Kp necessary for coding one frame And an operation determining step of determining an operating frequency at which the necessary calculation amount Kp can be encoded within a time Te pre-allocated for encoding the one frame.
While the processor is operating at the operating frequency obtained in the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image encoding or It is a moving picture coding processing method for performing the coding processing of the one frame in the decoding step,
A moving picture coding method, comprising a first failure avoidance step of reducing an actual calculation amount of the coding processing at a predetermined timing. A moving picture coding or decoding step of coding or decoding a moving picture composed of a plurality of consecutive frames using a processor in frame units, and necessary operations necessary for coding or decoding one frame A necessary calculation amount calculation step of calculating the amount Kp, and an operating frequency at which the necessary calculation amount Kp can be encoded or decoded within a time Te pre-allocated for the encoding or decoding process of the one frame. And a motion determining step for determining,
While the processor is operating at the operating frequency obtained by the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image coding or A moving image encoding or decoding method that performs encoding or decoding of the one frame in the decoding step,
When the required calculation amount Kp calculated by the necessary calculation amount calculation means is smaller than the actually required calculation amount, the necessary calculation amount of a subsequent frame encoded or decoded after the one frame is increased. 2. A moving picture coding or decoding processing method, characterized by comprising the step 2 of avoiding a failure. A moving image coding step of coding or decoding a moving image composed of a plurality of consecutive frames by using a processor in frame units, and a necessary calculation amount Kp necessary for coding or decoding one frame. A necessary calculation amount calculation step of calculating, and an operation of determining an operating frequency at which the necessary calculation amount Kp can be encoded or decoded within a time Te pre-allocated to the encoding or decoding process of the one frame. And a decision step,
While the processor is operating at the operating frequency obtained in the operation determining step and the operating power supply voltage and/or the substrate bias voltage suitable for the operating frequency within the time Te that is assigned in advance, moving image encoding or A moving image encoding or decoding method that performs encoding or decoding of the one frame in the decoding step,
A moving picture coding or decoding processing method comprising a third failure avoidance step of extending the time Te at a predetermined timing.

Images