JP5263967B2 - Movie stream processing apparatus and movie stream processing program - Google Patents

Description

  The present invention relates to a moving image stream processing apparatus and a moving image stream processing program for processing a moving image stream.

  As this type of technology, for example, a method of thinning out a picture having a moving image stream has been proposed (Patent Document 1). In this patent, the number of decoding processes is reduced by thinning out B-pic (Bi-directionally predictive-coded picture) in an image.

JP 2005-64569 A

  The technique of Patent Document 1 can reduce the burden of decoding processing by reducing the number of decoding sheets, but does not display a frame (B-pic) that has not been decoded. For this reason, since the frame rate of the video (the number of frames displayed per second) is lowered, there is a problem that the video quality of the terrestrial digital broadcasting is lowered.

  The present invention has been made in view of the above points, and it is an object of the present invention to provide a moving image stream processing apparatus and a moving image stream processing program capable of reducing the degradation of the image quality of moving images while reducing the burden of decoding processing.

(1) In order to achieve the above object, a video stream processing apparatus according to the first aspect of the present invention provides:
In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Have a, a frame generating means for generating a new frame to compensate for the frame to decoding processing unit does not decoded using the information of the frame which the decoding means has decoded,
The new frame generated by the frame generation means is a frame different from the decoded frame .
(2) In order to achieve the above object, a video stream processing apparatus according to the second aspect of the present invention provides:
In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame that is not decoded by the decoding processing means is a frame that satisfies at least a ratio between an inter-frame coding mode and an intra-frame coding mode that can be selected for each macroblock that is a processing unit at the time of decoding,
It is characterized by that.
(3) In order to achieve the above object, a video stream processing apparatus according to the third aspect of the present invention provides:
In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame that is not decoded by the decoding processing means is a frame that satisfies the predetermined amount of error in the frame when performing inter-frame encoding at least in the moving image stream of the frame.
It is characterized by that.
(4) In order to achieve the above object, a video stream processing apparatus according to the fourth aspect of the present invention provides:
In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame generation means generates the new frame that compensates for a frame that has not been decoded by the decoding processing means, using vector information of a frame that has not been decoded by the decoding processing means.
It is characterized by that.
(5) In order to achieve the above object, a video stream processing apparatus according to the fifth aspect of the present invention provides:
In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
Frames other than the some of the plurality of frames included in the video stream are frames that decode a part and do not decode the remaining part,
The frame generation means generates the new frame by supplementing a portion of the frame that has not been decoded using information of the frame decoded by the decoding processing means.
It is characterized by that.

Moreover, the moving image stream processing program according to the sixth to tenth aspects of the present invention causes a computer to execute the main functions of each of the moving image stream processing apparatuses according to the first to fifth aspects .

  According to the present invention, it is possible to reduce the degradation of video image quality while reducing the burden of decoding processing.

It is a figure for demonstrating the basic view which concerns on one Embodiment of this invention. It is a figure for demonstrating the basic view which concerns on one Embodiment of this invention. It is a figure explaining the idea which concerns on one Embodiment of this invention, and is a figure which shows the data structure of the moving image stream of MPEG2. It is a figure explaining the idea which concerns on one Embodiment of this invention, and is a figure which shows the data structure of I-pic of MPEG2. It is a figure explaining the idea which concerns on one Embodiment of this invention, and is a figure which shows the data structure of I-pic or B-pic of MPEG2. It is a block diagram which shows the structure of the moving image stream processing apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the decoder part and control part of the moving image stream processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the state in each process of the moving image stream processing apparatus which concerns on one Embodiment of this invention.

  Embodiments of the present invention will be described with reference to the drawings. In addition, this invention is not limited by the following embodiment and drawing. It goes without saying that the following embodiments and drawings can be modified without changing the gist of the present invention.

  In this embodiment, the input is terrestrial digital broadcast (especially high-definition size broadcast), the video stream included in the broadcast is decoded, and displayed on the display device in a size smaller than the decode size. Show. The input need not be limited to terrestrial digital broadcasting, but may be any other stream as long as the video is compressed. In addition, moving picture compression will be described by taking MPEG (Moving Picture Experts Group) 2 which is an international moving picture standard system as an example. Other compression methods such as H.264 and MPEG4 may be used.

  First, a basic concept of processing performed by the moving image stream processing apparatus according to the present embodiment will be described with reference to FIGS. The moving image stream of digital terrestrial broadcasting includes an I-pic (Intra-coded Picture) configured only by intra-screen coding and a P- that has been inter-coded using temporally past images. Three types of frames are included: pic (Predictive-coded picture) and B-pic (Bi-directionally predictive-coded picture) that has been inter-coded using temporally past and future images. A moving image is represented by these frames.

  The video stream processing apparatus in the present embodiment does not decode frames (B-pics 12 and 13 in FIG. 1) that satisfy a certain standard among these frames included in the video stream, and does not decode the standard. The unsatisfied frames (I-pic11 and P-pic14 in FIG. 1) are decoded. Thereby, the processing load can be reduced. This also reduces power consumption. Also, new frames (frames 22 and 23 in FIG. 2) are generated from the decoded I-pic 21 and decoded P-pic 24 frames. Therefore, if the generated frame is used for reproduction of a moving image, it is possible to reduce the deterioration of the image quality of the moving image. As a generation method, for example, there is a method using FRC (Frame Rate Control) processing, but the generation method is not limited to this method. Also, the frame generated here supplements the frame that has not been decoded, and reduces the deterioration of the image quality of the moving image. The image of the frame to be generated need not be the same as the image of the frame that has not been decoded. Further, the frames that have not been decoded and the frames to be generated need not have the same number. Note that the frame may be thinned out regardless of the content of the image. That is, some frames may not be decoded regardless of the content of the image. This eliminates the need for complicated processing.

  In recent years, mobile devices such as mobile phones have been able to handle moving images, but the display size of a display device (display unit) for displaying moving images does not correspond to the high-definition size. Assuming that a high-definition terrestrial digital broadcast is viewed on such a mobile phone, the video size larger than the display size of the display device is decoded, and the decoded video size is reduced to the display size. Therefore, it is necessary to watch the video, a lot of processing costs are required, and it is considered that a lot of power consumption is required in an actual product. For this reason, it is preferable that the decoded frame is once reduced according to the size of the display screen, and a new frame is generated based on the reduced decoded frame. As a result, the data handled in the FRC process is reduced by performing the FRC process after reducing the decoded frame, thereby realizing a reduction in processing load and power consumption. Each frame may be reduced after a new frame is generated before reduction. Furthermore, the degradation of image quality can be reduced by generating a new frame. In this way, it is possible to view video content having a size larger than the display size with low power consumption while maintaining the image quality and the frame rate.

  The processing described above is performed by the video stream processing device, decoding some frames of the plurality of frames included in the video stream, and using the decoded frame information, the decoding processing means does not decode the frames. It is an example of the process which produces | generates the new frame to supplement. Note that the decoded frame information includes information on the decoded frame itself, and also information on the frame after processing by reducing the frame.

  Next, the moving picture stream processing apparatus and the processing performed by this apparatus in this embodiment will be described. As shown in FIG. 6, the moving image stream processing apparatus 1 includes a control unit 101, a display unit 103, a storage unit 105, an operation unit 107, a decoder unit 109, an audio output unit 111, an audio input unit 113, a communication unit 115, and a bus. 140. The moving image stream processing apparatus 1 is configured by, for example, a mobile phone capable of receiving and reproducing terrestrial digital broadcasting.

  The control unit 101 includes, for example, a CPU (Central Processing Unit) and a RAM (Random Access Memory), and controls the entire operation of the moving image stream processing apparatus 1 (operation of each unit). For example, the control unit 101 performs a process of displaying and reproducing terrestrial digital broadcasting.

  The storage unit 105 includes an appropriate storage device such as a secondary storage device, and stores various data and a moving image stream processing program. The moving image stream processing program 150 is expanded in the above-described RAM or the like. The moving image stream processing program 150 causes the computer to perform processing described later (processing performed by the moving image stream processing device 1). The moving image stream processing program 150 may be installed in the moving image stream processing device 1 from a portable storage medium via various reading devices. The video stream processing program 150 may be downloaded and installed in the video stream processing apparatus 1 from various networks such as the Internet via various communication units.

  The operation unit 107 includes a keyboard and the like, and inputs an operation signal corresponding to an operation from the user to the control unit 101.

  The display unit 103 includes a display panel 104 and a driver circuit, and displays an image on the display panel 104 under the control of the control unit 11.

  The configuration of the decoder unit 109 will be described later.

  The audio output unit 111 includes a speaker for calling and the like. For example, the audio output unit 111 performs DA conversion on the audio signal of the content reproduced by the control unit 101 and emits sound through the speaker.

  The voice input unit 113 collects voice signals and supplies them to the communication unit during a call or the like.

  The communication unit 115 transmits / receives call voice, various data, and the like via the base station.

  The bus 140 transmits data between the units.

  The decoder unit 109 will be described with a focus on moving image processing. 7, the decoder unit 109 includes a receiving unit 201, a Demux unit 203, a decoder (an example of a decoding processing unit) 205, a scaler unit (an example of a reduction unit) 207, an FRC unit (a frame generation unit). An example) 209 and an output unit 211 are provided. Each of these units is controlled by the control unit 101 (see the dotted arrows in FIG. 7, where the solid arrows indicate the outline of the data path).

  The receiving unit 201 includes an antenna. This antenna has a shape necessary for receiving a digital broadcast, for example, a UHF band radio wave or a VHF band radio wave. The receiving unit 201 extracts a reception stream specified by the control unit 101 from radio waves input from the antenna (for example, a reception stream corresponding to a channel specified by the user) by a known method, and outputs the received stream to the Demux unit 203.

  The Demux unit 203 decomposes the received stream into a moving image stream and an audio stream and performs stream analysis of the moving image stream.

  The decoder 205 decodes the moving image stream. The audio stream is subjected to processing such as decoding by a decoding unit (not shown), and audio data is extracted and output to the control unit 101 as appropriate.

  The scaler unit 207 enlarges / reduces the frame decoded by the decoder 205 to a display size.

  The FRC unit 209 creates frames between a plurality of temporally different frames. In this manner, the FRC unit 209 generates a new frame that supplements a frame that has not been decoded using the information on the frame that the decoder 205 has decoded.

  The output unit 211 rearranges the decoded frames and the generated frames in time series and outputs the rearranged frames and generated frames to the control unit 101 as moving image data.

  The control unit 101 performs predetermined processing on the input moving image data and audio data as necessary, outputs the processed data to the display unit 103 and the audio output unit 111, and displays the moving image and audio represented by this data. Are reproduced on the display unit 103 and the audio output unit 111.

  As described above, the Demux unit 203 separates a stream acquired from a broadcast into a moving image and a sound, and analyzes moving image data. By this analysis, it is determined whether or not to perform decoding processing for each frame from various information of the stream. The Demux unit 203 is a case where the frames to be judged that are sequentially input are frames that satisfy one or more of the following criteria (which criteria are to be adopted are determined in advance). Then, it is determined not to perform this frame decoding process. Then, the Demux unit 203 outputs an analysis result specifying whether or not to decode the target frame to the control unit 101.

(1) Frame in which the code amount of the frame satisfies the first predetermined condition (2) The ratio between the inter-frame coding mode and the intra-frame coding mode that can be selected for each macroblock that is a processing unit when decoding is the second (3) Frame that satisfies the predetermined condition of (3) Frame in which the error amount of the pixel value of each frame satisfies the third predetermined condition when performing inter-frame coding in the moving image stream of this frame (4) The moving image stream Frames that are not referenced between frames from temporally different frames when decoding

  The reference will be described in more detail with reference to FIGS. As shown in FIG. 3, the MPEG2 moving image stream has a sequence header and a group (Group Of Pictures) header at the top, followed by information of each picture (frame). Each frame has a frame header at the head, and data (picture header) that can determine whether the frame is one of I / P / B-pic. As shown in FIGS. 4 and 5, the inside of each frame is represented by the picture header, slice header, MB header, vector information (vector information (v)), and DCT (Discrete Cosine Transform) coefficient (r). Configured in order.

  Judgment criterion (1) focuses on the fact that a frame with a small code amount (for example, the total amount or average value of code amounts in a frame) representing a frame can be estimated to be similar to a temporally adjacent frame. It is a standard. If they are similar, it can be determined that the FRC unit 209 in the subsequent stage can create a similar frame, and therefore it can be predicted that the frame to be determined does not need to be decoded. If a frame not to be decoded is determined based on such a criterion, it is possible to efficiently reduce the deterioration of image quality. The frame (1) is, for example, a frame whose frame code amount is less than the first threshold. The Demux unit 203 measures the code amount of the current processing target frame (any of I / P / B-pic) in the moving image stream, and obtains and determines the code amount (see FIG. 3).

  The criterion (2) is a criterion focusing on the fact that an image of a frame in which an intra-frame coding mode (for example, Intra coding mode) is frequently used can be estimated to have low correlation with surrounding images. In this mode, information between screens is not used, and encoding is performed using only frame information in the screen. When this mode is frequently used for a certain frame, this frame is used by the FRC unit 209 in the subsequent stage. It can be predicted that it is difficult to create a frame corresponding to. If a frame not to be decoded is determined based on such a criterion, it is possible to efficiently reduce the deterioration of image quality. The frame (2) is, for example, a frame whose Intra coding mode value is less than the second threshold value, or a frame whose Intra / Inter ratio value is less than the third threshold value. The Demux unit 203 measures the Intra / Inter ratio information included in the MB header of the P / B-pic, and obtains and determines the value of the Intra coding mode or the value of the Intra / Inter ratio from the measured value. (See FIG. 5).

  The criterion (3) focuses on the fact that an image of a frame with a large amount of error representing the frame (for example, the total amount or average value of DCT coefficients in the frame) can be estimated to have low correlation with surrounding images. It is a standard. When the value of the error amount is large, it can be predicted that it is difficult for the FRC unit 209 in the subsequent stage to create a frame corresponding to this frame. If a frame not to be decoded is determined based on such a criterion, it is possible to efficiently reduce the deterioration of image quality. The frame (3) is, for example, a frame whose error amount is less than the fourth threshold value. The Demux unit 203 measures the DCT coefficient of I / P / B-pic, and obtains and determines an error amount value from the measured value (see FIG. 5).

  The criterion (4) is that a frame that is referenced between frames from a temporally different frame when decoding a video stream (a frame that is referenced in inter-frame prediction during decoding of other frames) is correlated with surrounding images. It is a standard that focuses on the fact that it is high. Since the frame referred to in inter-frame prediction is required when decoding other frames, decoding is preferable. If a frame not to be decoded is determined based on such a criterion, it is possible to efficiently reduce the deterioration of image quality. The frame (4) is, for example, the last P-pic in a B-pic or a gop. This depends on the format of the stream. In MPEG2, B-pic is not referenced between frames. The Demux unit 203 observes the picture header, obtains an attribute of the frame (which is I / P / B-pic), and determines (see FIG. 4).

  It should be noted that, according to at least one of the above criteria (1) to (4), a frame that is likely to be created by the FRC process is not decoded, and a frame or the like in which the screen is switched (FRC) There is a high possibility that a frame that is difficult to generate by processing is not combined. As a result, it is possible to reduce image quality degradation caused by not performing decoding.

  In the present embodiment, the stream analysis is performed by the Demux unit 203. In many devices, since the processing cost of the decoder 205 is high, many parts are implemented as hardware, and flexible stream analysis is often difficult. On the other hand, since the Demux unit 203 corresponds to various standards and formats and is often configured to be able to perform processing flexibly with software, in this embodiment, the Demux unit 203 performs stream analysis. It is configured. In this way, the decoder unit according to the present embodiment is configured by forming the frame specifying unit configured by the Demux unit 203 and the control unit 101 to analyze a moving image stream and specify a frame that is not decoded outside the decoder 205. 109 manufacturing cost and the like can be suppressed. In addition, power consumption of the decoder 205 can be suppressed.

  Note that at least a part related to the above determination in the stream analysis may be performed by the control unit 101. Even in this case, the Demux unit 203 and the control unit 101 constitute a frame specifying unit. Further, the decoder 205 and / or the control unit 101 may perform processing similar to the stream analysis processing. In this case, the decoder 205 and / or the control unit 101 constitute a frame specifying unit. The frame specifying unit that analyzes the moving image stream and specifies a frame that is not decoded may have another configuration.

  As described above, the decoder 205 receives an instruction of a frame to be decoded from the control unit 101, and performs decoding only for the frame. As described above, the decoder 205 decodes some of the plurality of frames included in the moving image stream.

  As described above, the scaler unit 207 receives information on the decoded frame from the control unit 101, and performs enlargement / reduction processing only on the frame. For example, when the broadcast moving image size is 1920 × 1080 pixels and the display size of the display panel in FIG. 6 is 854 × 480, the reduction processing is performed so that the broadcast moving image size of the processing target frame becomes the display size. Apply to the target frame. In broadcasting, the size of an image to be displayed is determined by the size and aspect ratio used for moving image compression. The above size is an example of the size for this display.

  Note that it is preferable to stop the overall operation of the Scaler unit 207 during the period when the Scaler unit 207 does not perform enlargement / reduction processing so that power consumption is not as much as possible. For example, when the scaler unit 207 is hardware when processing is stopped, the control unit 101 stops the clock supplied to the scaler unit 207, or SRAM (Static Random Access Memory) or SDRAM (Synchronous Dynamic Random Access). Memory access to (Memory) etc. is stopped. Note that when the operation of any of the decoder 205, the scaler unit 207, and the FRC unit 209 is stopped, the clock to be supplied is stopped, or the memory access to the SRAM, SDRAM, or the like is stopped. As a result (especially by stopping the clock), power consumption can be reduced efficiently.

  Note that this image size conversion processing (particularly reduction processing) based on the size and aspect ratio at the time of moving image compression may be performed after the FRC processing described below.

  The FRC unit 209 performs FRC processing for generating a new frame using information on frames that are temporally different (for example, information on decoded frames (particularly, information on reduced frames)). At this time, the temporally different frame (reference frame) may be either a decoded image or an image created by FRC processing. However, in the first process, the decoded frame information is always used. If the reference frame of the decoded image is also traced back, it becomes a frame generated using the information of the decoded frame. For this reason, even if there is a decoded image in at least one of the reference frames, the FRC unit 209 generates a new frame using the decoded frame information.

As a method of the FRC processing, any of the following methods or a combination method is used.
(A) Created by interpolation of reference frames.
(B) A motion vector between reference frames is calculated and supplemented.
(C) Make up using morphing.
(D) Using the vector information (for example, motion vector) of the frame that has not been decoded, a new frame that supplements the frame that has not been decoded is generated. For example, vector information is extracted from a stream of frames that have not been decoded, and the vector is used for FRC processing (for example, used auxiliary).

  In (d) above, since the vector information of the original frame is used, a frame closer to the original frame can be generated, and the deterioration in image quality can be reduced.

  In the above description, an example of performing decoding processing or FRC processing on the entire screen (entire frame) is shown, but the processing may be divided inside an image. That is, a frame that is not decoded may be a frame that does not decode at least a portion, and a portion that has not been decoded in the frame is supplemented using information on the frame that the decoder 205 has decoded. For example, the image is analyzed at the time of decoding, the decoding process is performed on areas that are considered to be important parts in the screen (for example, the person area and the caption area), and the decoding process is not performed on the other areas. Create by processing. Thereby, it is possible to efficiently reduce power consumption and image quality.

  Finally, the operation of each part will be described with reference to FIGS. FIG. 8 shows the structure of data used for each process in blocks along the time axis for each frame, where I is I-pic, P is P-pic, and B is B-pic. . The numbers in the blocks indicate the display order of images in actual reproduction. A small block is a frame whose data amount is reduced by reduction or the like, and a shaded block is a new frame generated by the FRC process. Further, in order to facilitate understanding, the delay time generated by the processing of each unit is ignored in FIG. 8, but it is considered that some delay time actually occurs. However, the basic idea is the same.

  The reception unit 201 outputs the reception stream by sequentially outputting the data constituting the reception stream to the demux unit 203.

  The reception stream is input to the Demux unit 203 by sequentially inputting data constituting the reception stream output from the reception unit 201. The Demux unit 203 sequentially extracts data included in the moving image stream from the input data and analyzes the data to analyze the moving image stream. This analysis is mainly performed for each frame data constituting the moving image stream. Then, for example, the Demux unit 203 determines whether the current analysis target frame should be decoded (see above), and outputs the determined determination result to the control unit 101 as an analysis result.

  When the analysis result is input to the control unit 101, the control unit 101 controls the operations of the decoder 205, the scaler unit 207, and the FRC unit 209 based on the analysis result.

  When the analysis result is to be decoded (in FIG. 8, in the case of I-pic and P-pic), the control unit 101 operates the decoder 205 and the scaler unit 207, and stops the FRC unit 209. The frame is input to the decoder 205 and decoded there, and the decoded frame is input to the scaler unit 207 and reduced. The Scaler unit 207 holds a frame that is appropriately reduced.

  When the analysis result is not decoded (in the case of B-pic in FIG. 8), the control unit 101 stops the decoder 205 and the scaler unit 207 and operates the FRC unit 209. At this time, the control unit 101 causes the scaler unit 207 to output the frame held by the scaler unit 207 to the FRC unit 209. The FRC unit 209 performs FRC processing using the frame information input to the FRC unit 209, and generates new frames (shaded blocks B1, B2, B4, and B5 in FIG. 8) (for example, , B1 and B2 are generated using I0 and P3 input and held from the Scaler unit 207). The control unit 101 causes the FRC unit 209 to output the frame input to the FRC unit 209 and the frame generated by the FRC unit 209 at an appropriate timing.

  The input frame and the generated frame output from the FRC unit 209 are input to the output unit 211. The output unit 211 arranges the input frames in the display order and outputs them to the control unit 101 (see FIG. 8). Note that the output unit 211 recognizes the display order of frames based on data in the frame or a command from the control unit 101.

  Note that, under certain conditions, the decoder 205 may perform the above-described process of not decoding some frames. This is performed by, for example, instructing the decoder 205 by the control unit 101 configuring the frame specifying unit. Alternatively, for example, only when the certain condition is satisfied, the Demux unit 203 outputs the analysis result of the stream (for example, information specifying which frame is not decoded) to the control unit 101. .

  As an example of the predetermined condition, when the remaining battery level for driving the video stream processing apparatus 1 satisfies the fourth predetermined condition (when the remaining battery level is low, for example, the remaining battery level is 5) or when the reception state of the radio wave of the video stream is the fifth predetermined condition (when the reception state is poor, for example, the bit error rate of the video stream is equal to or higher than the sixth threshold). Or when the electric field strength falls below the seventh threshold value). In this case, a process of decoding a part of a plurality of frames included in the moving image stream is performed. For the remaining battery level, bit error rate, or electric field strength, these measuring instruments are provided at appropriate locations. The measurement results measured by these measuring devices are output to the control unit 101, for example, and the above conditions are determined by the control unit 101, for example. When the predetermined condition is satisfied, the control unit 101 causes the decoder to perform a process that does not decode some frames, or causes the Demux unit 203 to output an analysis result.

  Thus, the power consumption can be reduced at a suitable timing such as when the remaining battery level is low, or when the reception quality is poor, or when the image quality is poor and degradation of the image quality is not a problem.

DESCRIPTION OF SYMBOLS 1 ... Movie stream processing apparatus, 11, 21 ... I-pic, 12 ... B-pic, 13 ... B-pic, 14, 24 ... P-pic, 22 ... Frame, 23 ... Frame, 24 ... P-pic, 101 ... Control part 103 ... Display part 105 ... Storage part 107 ... Operation part 109 ... Decoder part 111 ... Audio output part 113 ... Audio input part 115 ... Communication part 140 ... Bus 150 ... Video stream processing Program 201 201 Receiving unit 203 Demux unit 205 Decoder 207 Scaler 209 FRC unit 211 Output unit

Claims (21)

In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Have a, a frame generating means for generating a new frame to compensate for the frame to decoding processing unit does not decoded using the information of the frame which the decoding means has decoded,
The new frame generated by the frame generation means is a frame different from the decoded frame.
A moving picture stream processing apparatus.   2. The moving picture stream processing apparatus according to claim 1, wherein the frame that is not decoded by the decoding processing means is a frame in which at least a code amount of the frame satisfies a first predetermined condition.   Frames that are not decoded by the decoding processing means are frames that satisfy at least the second predetermined ratio between the inter-frame coding mode and the intra-frame coding mode that can be selected for each macroblock that is a processing unit at the time of decoding. The moving picture stream processing apparatus according to claim 1, wherein   The frame that is not decoded by the decoding processing means is a frame that satisfies a third predetermined condition for an error amount of the frame when performing inter-frame encoding at least in a moving picture stream of the frame. Item 4. The moving image stream processing device according to Item 1.   2. The moving picture stream processing apparatus according to claim 1, wherein the frame that is not decoded by the decoding processing means is a frame that is not referenced between frames from at least a temporally different frame when the moving picture stream is decoded. A reduction means for reducing the frame decoded by the decoding processing means;
6. The moving picture stream processing apparatus according to claim 1, wherein the information on the frame is information on the reduced frame.   2. The frame generation unit generates the new frame that supplements a frame that has not been decoded by the decoding processing unit, using vector information of a frame that has not been decoded by the decoding processing unit. 7. The moving picture stream processing device according to any one of items 1 to 6. The frame that is not decoded by the decoding processing means is a frame that is not decoded at least in part,
2. The frame generating unit generates the new frame by supplementing a part of the frame that has not been decoded with information on a frame decoded by the decoding processing unit. 8. The moving picture stream processing device according to any one of items 7 to 9. A frame specifying unit that analyzes the video stream and specifies a frame that is not decoded;
9. The moving picture stream processing apparatus according to claim 1, wherein the decoding processing unit decodes the partial frame without decoding the frame specified by the frame specifying unit. The decoding processing means is constituted by a decoder that decodes the partial frame without decoding the frame specified by the frame specifying means,
10. The moving picture stream processing apparatus according to claim 9, wherein the frame specifying means is formed outside the decoder.   The decoding processing means decodes a part of a plurality of frames included in the moving picture stream when a remaining amount of a battery driving the moving picture stream processing apparatus satisfies a fourth predetermined condition. The moving picture stream processing apparatus according to claim 1, wherein the moving picture stream processing apparatus is a moving picture stream processing apparatus.   The decoding processing means decodes a part of a plurality of frames included in the moving picture stream when a radio wave reception state of the moving picture stream satisfies a fifth predetermined condition. Item 12. The moving image stream processing device according to any one of Items 1 to 11. On the computer that processes the video stream,
A decoding process step of decoding some of the plurality of frames included in the video stream;
Have a, a frame generation step of generating a new frame to supplement the frames were decoded at the decoding step by using the information of a frame decoded by the decoding step,
The new frame generated by the frame generation step is a frame different from the decoded frame.
A moving image stream processing program characterized in that processing is performed. In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame that is not decoded by the decoding processing means is a frame that satisfies at least a ratio between an inter-frame coding mode and an intra-frame coding mode that can be selected for each macroblock that is a processing unit at the time of decoding,
A moving picture stream processing apparatus. In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame that is not decoded by the decoding processing means is a frame that satisfies the predetermined amount of error in the frame when performing inter-frame encoding at least in the moving image stream of the frame.
A moving picture stream processing apparatus. In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
The frame generation means generates the new frame that compensates for a frame that has not been decoded by the decoding processing means, using vector information of a frame that has not been decoded by the decoding processing means.
A moving picture stream processing apparatus. In a video stream processing apparatus that processes video streams,
Decoding processing means for decoding some of the plurality of frames included in the video stream;
Frame generating means for generating a new frame that compensates for a frame that has not been decoded by the decoding processing means using information of the frame decoded by the decoding processing means,
Frames other than the some of the plurality of frames included in the video stream are frames that decode a part and do not decode the remaining part,
The frame generation means generates the new frame by supplementing a portion of the frame that has not been decoded using information of the frame decoded by the decoding processing means.
A moving picture stream processing apparatus. On the computer that processes the video stream,
A decoding process step of decoding some of the plurality of frames included in the video stream;
A frame generation step of generating a new frame that supplements a frame that has not been decoded in the decoding processing step by using the information of the frame decoded in the decoding processing step;
The frame that is not decoded in the decoding processing step is a frame that satisfies at least a ratio between an inter-frame coding mode and an intra-frame coding mode that can be selected for each macroblock that is a processing unit at the time of decoding,
A moving image stream processing program characterized in that processing is performed. On the computer that processes the video stream,
A decoding process step of decoding some of the plurality of frames included in the video stream;
A frame generation step of generating a new frame that supplements a frame that has not been decoded in the decoding processing step by using the information of the frame decoded in the decoding processing step;
The frame that is not decoded in the decoding processing step is a frame that satisfies the predetermined amount of error in the frame when performing inter-frame encoding at least in the moving image stream of the frame.
A moving image stream processing program characterized in that processing is performed. On the computer that processes the video stream,
A decoding process step of decoding some of the plurality of frames included in the video stream;
A frame generation step of generating a new frame that supplements a frame that has not been decoded in the decoding processing step by using the information of the frame decoded in the decoding processing step;
The frame generation step is a step of generating the new frame that supplements the frame that has not been decoded in the decoding processing step by using vector information of the frame that has not been decoded in the decoding processing step.
A moving image stream processing program characterized in that processing is performed. On the computer that processes the video stream,
A decoding process step of decoding some of the plurality of frames included in the video stream;
A frame generation step of generating a new frame that supplements a frame that has not been decoded in the decoding processing step by using the information of the frame decoded in the decoding processing step;
Frames other than the some of the plurality of frames included in the video stream are frames that decode a part and do not decode the remaining part,
The frame generation step is a step of generating the new frame by supplementing a portion that has not been decoded in the frame with information of the frame decoded in the decoding processing step.
A moving image stream processing program characterized in that processing is performed.

Images