JP4399744B2 - Program, information processing apparatus, information processing method, and recording medium - Google Patents

Description

  The present invention relates to a program, an information processing apparatus, an information processing method, and a recording medium, and particularly to a program, an information processing apparatus, an information processing method, and a recording medium that are suitable for random playback.

MPEG (Moving picture experts gro) is a typical encoding method for moving image data.
up), a motion-compensated predictive coding method typified by a coding method.

  Motion-compensated predictive coding schemes are roughly classified into two types of coding schemes: intra coding (intraframe coding) and inter coding (interframe reference coding). In intra coding, coding is performed based on the correlation in units of macroblocks in an encoded image. In inter coding, an image that temporally precedes and follows an encoded image is used as a reference image, and a difference between a predicted image generated from the reference image and the encoded image is encoded (for example, Patent Documents). 1).

JP-A-8-10951

  In the conventional MPEG decoding, when a P frame is decoded, the immediately preceding I frame or P frame as a reference image is decoded to generate a reference image, and the target P frame is decoded based on this. For this reason, when the P frame located at the end of the GOP is decoded, it is necessary to decode a plurality of frames in order to decode the target P frame.

  Further, when a B frame is decoded, it is necessary to use the P frames before and after that as reference images, and before and after decoding the B frame to be decoded, it is necessary to decode the previous and subsequent P frames.

  Therefore, in the case of random playback (also called scrub playback) or the like, when it is desired to play back P frames and B frames that are temporally dissociated in the stream, it takes time to decode other frames as reference images. The necessary video cannot be decoded and output in a short time.

  The present invention has been made in view of such circumstances, and is intended to be able to output a necessary video in a short time when performing random reproduction.

The program according to one aspect of the present invention is a decodable number that can execute decoding processing within a display time of an image corresponding to one frame for a bitstream composed of a plurality of GOPs (Group Of Pictures). Based on the number of frames and the number of P pictures in one GOP, a continuous number calculating step for calculating the continuous number of P pictures existing between two P pictures to be decoded, and calculation by the processing of the continuous number calculating step based on the number of successive P-pictures, which are, from each of the plurality of GOP constituting the bit stream, and P-picture decoding step of decoding each of the P-picture selected predetermined number to be decoded, the P picture decoded each frame in which the P picture that has been selected a predetermined number by the processing steps produce results that are decoded respectively, each mark as an I-picture An encoding step of reduction, by analyzing the bit stream, an analysis step of acquiring information including the order of the picture, the analysis step coded information on the generated encoded data by processing of the coding step processing based on said information Ri obtained by the, to execute processing including the attribute information generating step of generating attribute information available to decode the bit stream to the computer.
When a predetermined number larger than the number of P pictures is set as a threshold and the number of decodable frames is equal to or greater than the threshold, execution of the process of the P picture decoding step is prohibited, and the number of decodable frames is If it is equal to or less than the threshold value, execution of the process of the P picture decoding step is permitted.
The threshold is set based on the time required for the decoding process of the frame that takes the longest time to decode in the bitstream.
In the processing of the continuous number calculating step, a value smaller than the number of decodable frames is determined as a candidate for the continuous number of the P picture, and when the candidate is smaller than the P picture number, the candidate is determined as the continuous number of P pictures. When the candidate is larger than the number of P pictures, a predetermined value other than the candidate is adopted as the number of consecutive P pictures.
The predetermined value adopted as the number of consecutive P pictures when the candidate is larger than the number of P pictures is determined based on a value obtained by dividing the number of P pictures by 2.

  A bitstream decoding step for decoding a bitstream may be further included based on the encoded data encoded by the encoding step processing and the attribute information generated by the attribute information generation step processing. it can.

The processing of the bit stream decoding step includes a determining step of determining a coding type of a frame to be decoded by the processing of said determining step, it is determined that the frame to be decoded is an I-picture of said bitstream If a first decoding step of decoding the I picture to be decoded of said bit stream, the processing of the determining step, it is determined that the frame to be decoded is a P-picture of said bitstream If, among the encoded data encoded by the processing in the encoding step, a second decoding step of decoding the coded data corresponding to the P picture to be decoded by the processing of said determining step, decrypting If the frame to be is determined to be a B picture of the bitstream, the Either generate a reference image by decoding the I-picture of said bitstream required to decode the picture, or, among the encoded data encoded by the processing in the encoding step, the decoding of the B picture Referring to the reference image generating step of generating a reference picture by decoding the coded data corresponding to the P picture of the bitstream required, the reference image generated by the processing of the reference image generation step, And a third decoding step of decoding the B picture of the bitstream.

Bit rate of the I-picture obtained as a result of the processing in the encoding step can be to a fixed rate <br/> so.

Bit rate of the I-picture obtained as a result of the processing in the encoding step can be to a variable rate <br/> so.

It is possible to further include a decoding process executable frame number calculating step for calculating the number of decodable frames .
The encoded data generated by the process of the encoding step is decoded by a predetermined number of the P pictures selected by the process of the P picture decoding step, and each frame obtained as a result is processed by the process of the encoding step. The encoded information relating to the encoded data may include a picture size and an address for each of the encoded data.
The encoding method employed for the processing executed by the computer may be a moving picture experts group (MPEG) encoding method.

The information processing apparatus according to an aspect of the present invention provides a number of frames that can be decoded within a display time of an image corresponding to one frame for the bitstream including a plurality of GOPs (Group Of Pictures). And a continuous number calculating means for calculating the continuous number of P pictures existing between two P pictures to be decoded based on the number of P pictures in one GOP and the processing of the continuous number calculating step. P picture decoding means for selecting and decoding a predetermined number of P pictures to be decoded from each of the plurality of GOPs constituting the bit stream based on the number of consecutive P pictures, and the P picture decoding means encoding means for said P-picture selected predetermined number of frames obtained as a result of the decoding, respectively, to each encoded as an I-picture by the It analyzes Tsu bets stream, obtains information including the order of pictures, based on the encoded information about the generated coded data by acquired the information, and the encoding means by analyzing the bit stream, the bit Attribute information generating means for generating attribute information usable for decoding the stream.

  Based on the encoded data generated by the encoding means and the attribute information generated by the attribute information generating means, a bitstream decoding means for decoding the bitstream can be further provided.

According to an information processing method of one aspect of the present invention, the number of frames in which decoding processing can be executed within a display time of an image corresponding to one frame for the bitstream including a plurality of GOPs (Group Of Pictures). And a continuous number calculating step for calculating the continuous number of P pictures existing between two P pictures to be decoded based on the number of P pictures in one GOP and the processing of the continuous number calculating step. A P picture decoding step for selecting and decoding a predetermined number of P pictures to be decoded from each of the plurality of GOPs constituting the bit stream based on the number of consecutive P pictures; and the P picture decoding step the P-picture selected predetermined number by processing each frame obtained as a result of the decoding, respectively, to each encoded as an I picture And Goka step, by analyzing the bit stream, the processing of the analysis step and the analysis step and encoding information about the coded data generated by the processing of the encoding step of obtaining information including the order of pictures based on said information Ri obtained by the, including an attribute information generating step of generating attribute information available to decode the bit stream.

In one aspect of the present invention , a decodable frame that is a number that can be decoded within a display time of an image corresponding to one frame for a bitstream composed of a plurality of GOPs (Group Of Pictures) The number of consecutive P pictures existing between two P pictures to be decoded is calculated based on the number of P pictures in one GOP and the number of P pictures calculated by the processing of the number of consecutive steps calculation step. A predetermined number of P pictures to be decoded are selected and decoded from each of the plurality of GOPs constituting the bitstream based on the continuous number, and the predetermined number of P pictures selected by the process of the P picture decoding step are selected. Each frame obtained as a result of decoding a picture is encoded as an I picture, the bit stream is analyzed, and a picture is decoded. The bitstream is decoded based on the encoded information related to the encoded data generated by the process of the encoding step and the information acquired by the process of the analyzing step. Attribute information that can be used to do this is generated.

  According to an aspect of the present invention, bitstream attribute information can be generated, and in particular, a forward prediction encoded frame is decoded, and the decoded forward prediction encoded frame is intraframe encoded. Therefore, the bit stream can be randomly reproduced at high speed using the intra-coded frame and the attribute information.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  A program or an information processing method according to one aspect of the present invention includes a forward predictive encoded frame decoding step (for example, step S5 in FIG. 11) for decoding a forward predictive encoded frame (eg, a P picture) in a bitstream. Processing), an encoding step (for example, processing in step S7 in FIG. 11) for intra-coding the forward prediction encoded frame decoded by the processing of the forward prediction encoded frame decoding step, and analyzing the bit stream Analysis step (for example, the process of step S3 in FIG. 11), encoding information (for example, Proxy File Picture Size, Proxy File Address in FIG. 3) and the analysis step regarding the encoded data generated by the encoding step process Attribute information that can be used to decode the bitstream based on the analysis result of Attribute information generating step of generating an index file) (for example, to execute processing including the processing in step S11) in FIG. 11 to the computer.

  Based on the encoded data encoded by the encoding step process and the attribute information generated by the attribute information generation step process, a bit stream decoding step (for example, steps S43 to S43 in FIG. 12) is performed. The process of step S50 can further be included.

  The process of the bit stream decoding step includes a determination step of determining the encoding type of the frame to be decoded (for example, the process of step S43 or step S44 in FIG. 12), and the frame to be decoded by the process of the determination step. Is determined to be an intra-frame encoded frame (for example, an I picture), a first decoding step of decoding a decoded intra-frame encoded frame of the bit stream (in step S46 of FIG. 12). Processing) and the determination step processing, when it is determined that the frame to be decoded is a forward prediction encoded frame (for example, a P picture) in the bitstream, it is encoded by the encoding step processing. Encoding corresponding to the forward prediction encoded frame to be decoded among the encoded data The frame decoded by the second decoding step (step S45 in FIG. 12) for decoding the data and the determination step processing is a bidirectionally predictive encoded frame (for example, B picture) in the bit stream. If it is determined that the code is encoded by decoding the intra-frame encoded frame of the bit stream necessary for decoding the bi-directional predictive encoded frame, or by generating a reference image Of the encoded data corresponding to the forward predictive encoded frame of the bitstream necessary for decoding the bidirectional predictive encoded frame to generate a reference image (step S48 in FIG. 12). Processing) and the reference image generated by the process of the reference image generation step, May include a third decoding step of decoding the forward prediction encoded frame (processing of step S49 in FIG. 12).

  In the encoding step processing, the forward predictive encoded frame decoded by the forward predictive encoded frame decoding step is encoded into fixed-rate intraframe encoded data (for example, as shown in FIG. 4). Can be encoded).

  In the encoding step processing, the forward prediction encoded frame decoded by the forward prediction encoded frame decoding step processing is encoded into intra-frame encoded data at a variable rate (for example, as shown in FIG. 5). Can be encoded).

  In the process of the forward predictive encoded frame decoding step, a predetermined number of forward predictive encoded frames in the bitstream are selectively decoded (for example, as described with reference to FIGS. 14 to 18), In the process of the encoding step, a predetermined number of forward predictive encoded frames decoded by the process of the forward predictive encoded frame decoding step can be intra-frame encoded.

  Forward predictive encoded frame decoding based on the number of frames (for example, the number of decodable frames) that can be decoded within the display of an image corresponding to one frame and the number of forward predictive encoded frames It may further include a setting step (for example, the process shown in FIG. 19) for setting a predetermined number of forward prediction encoded frames that are selectively decoded by the process of the step.

  It further includes a calculating step (for example, the process of step S81 in FIG. 19) for calculating the number of frames (for example, the number of decodable frames) that can be decoded within the display time of the image corresponding to one frame. Can do.

  In the process of the forward predictive encoded frame decoding step, the consecutive number of the predetermined number of forward predictive encoded frames in the bit stream is minimized (for example, as described with reference to FIGS. 14 to 18). ), A predetermined number of forward prediction encoded frames to be decoded can be selectively decoded.

  An information processing apparatus according to one aspect of the present invention is an information processing apparatus (for example, the playback apparatus 1 in FIG. 1 or the personal computer 201 in FIG. 9) that generates information that can be used to decode a bitstream, and includes a bit The forward prediction encoded frame decoding means (for example, the decoder 34 in FIG. 2) for decoding the forward prediction encoded frame (P picture) in the stream, and the forward direction decoded by the forward prediction encoded frame decoding means. Encoding means (for example, the encoder 36 in FIG. 2) for intra-encoding the predicted encoded frame, and the bit stream analysis result, the encoding information relating to the bit stream analysis result and the encoded data generated by the encoding means (For example, based on Proxy File Picture Size, Proxy File Address in Fig. 3) Can be used to decode bitstream Comprises an attribute information (for example, index file) attribute information generating means for generating (e.g., the bit stream analyzing unit 32 of FIG. 2) and.

  Based on the encoded data generated by the encoding means and the attribute information generated by the attribute information generating means, further provided is a bit stream decoding means (for example, the playback unit 12 in FIG. 6) for decoding the bit stream. Can do.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration example of a playback apparatus 1 to which the present invention is applied.

  The playback device 1 includes a conversion unit 11 and a playback unit 12.

  The conversion unit 11 is, for example, a bitstream (I picture (intraframe encoded frame), P picture (forward prediction encoded frame), and B picture) encoded by interframe reference encoding such as an Mpeg2 LongGOP stream. (Bit stream composed of bi-directional predictive encoded frames) is generated, and an index file that is referred to when the playback unit 12 decodes (decodes) the generated index file. A proxy file is generated by encoding (encoding) a P picture and converting it into an I picture, and the index file and the proxy file are supplied to the playback unit 12. Details of the conversion unit 11 will be described later with reference to FIG.

  The playback unit 12 receives a bitstream including an I picture, a P picture, and a B picture, and receives an index file and a proxy file from the conversion unit 11, and based on a user operation input, The bit stream is decoded and reproduced from the user's desired position. Details of the reproduction unit 12 will be described later with reference to FIG.

  FIG. 2 is a block diagram illustrating a more detailed configuration example of the conversion unit 11 of FIG.

  The bitstream acquisition unit 31 receives a bitstream including an I picture, a P picture, and a B picture and supplies the bitstream to the bitstream analysis unit 32.

  The bit stream analysis unit 32 receives a bit stream including an I picture, a P picture, and a B picture, and supplies the I picture and the P picture to the decoder 34.

  In addition, the bit stream analysis unit 32 controls the decoder 34, the encoder 36, and the proxy file storage unit 37. Further, the bit stream analysis unit 32 analyzes the supplied bit stream, analyzes the result of processing executed by the encoder 36, and information stored in the proxy index storage unit 38, which will be described later. The index file shown is generated and supplied to the index file storage unit 33.

  The index file shown in FIG. 3 describes information that is referred to when the original bitstream acquired by the bitstream acquisition unit 31 and supplied to the bitstream analysis unit 32 is decoded. Specifically, in the index file, the order of pictures in the display order (Display Order), the order of pictures in the stream order (Stream Order), and the order of pictures from the display order to the stream order are converted. The amount of deviation of the picture (Temporal Offset) is described. Also, in the index file, information indicating the position of the sequence header (Sequence Header) in the stream order is described as “1” indicating that the sequence header is present and “0” indicating that the sequence header is present. Information indicating presence / absence (Forward / Backward Prediction (Picture Type)) is described.

  In this example, the information indicating the presence / absence of the reference image in the forward direction or the backward direction assumes that the supplied bitstream is in the OpenGOP format, and the forward reference frame (that is, P picture) is “10”. It is assumed that the bidirectional reference frame (that is, B picture) is described as “11”, and the intra-frame encoded data (that is, I picture) is described as “00”. When the supplied bit stream is in Closed GOP format, the two B pictures following the intra-frame encoded data (ie, I picture) refer only to the backward direction, so that “01” is described. To do.

  The index file describes the picture size of each picture and the address (Address) within the picture stream or recording area. The index file also contains information (Key Frame Offset (decimal)) indicating the distance between the picture and the picture that is the key frame (here, the I picture on which intraframe coding processing is performed), the decoder's VbvDelay (VBV: Video buffer verifier) indicating the amount of accumulation in the virtual input buffer in 90 KHz clock time is described. The index file also includes depth information (Depth of Past Frames for Depth of Past Frames for the number of frames between the frames farthest from the playback frame in the forward image required to decode the picture. Decoding) is described.

  Further, the index file includes a picture size (Proxy File Picture Size) as encoding information regarding an I picture which is a frame corresponding to a P picture in the original stream and is encoded by the encoder 36, which is included in the proxy file. The addresses (Proxy File Address) of these frames are described. These pieces of information are stored in a proxy index storage unit 38 to be described later.

  Details of the proxy file will be described later.

  The index file storage unit 33 stores the index file described with reference to FIG. 3 supplied from the bitstream analysis unit 32 and outputs the index file to the reproduction unit 12.

  The decoder 34 decodes the I picture and the P picture supplied from the bit stream analysis unit 32 to generate an uncompressed baseband image, and supplies uncompressed baseband image data corresponding to the P picture to the encoder 36. At this time, when the generated uncompressed baseband image is required as a reference image of a frame to be decoded thereafter, the decoder 34 also supplies the generated uncompressed baseband image to the reference image memory 35. When storing and decoding the P picture, the reference image stored in the reference image memory 35 is appropriately referred to.

  The encoder 36 encodes the uncompressed baseband image data corresponding to the supplied P picture as an I picture by intra-frame coding, and supplies the generated I picture to the proxy file storage unit 37. Further, the encoder 36 can set, for example, a compression rate as a parameter at the time of compression processing (I picture generation).

  A case where the bit rate of the I picture output from the encoder 36 is fixed will be described with reference to FIG.

  When the bit rate of the output I picture is fixed, the encoder 36 generates the I picture so as to have a predetermined bit rate according to the storage capacity of the proxy file storage unit 37, for example. In general, a P picture using forward reference has a smaller amount of information than an I picture, so the bit rate when re-compressing a P picture and encoding it into an I picture is the value before conversion (P picture). It is preferable to set the bit rate higher than the bit rate because it is easy to maintain the image quality. The bit rate may be set by the user.

  Next, a case where the bit rate of the I picture output from the encoder 36 is changed will be described with reference to FIG.

  The encoder 36 acquires the rate of the I picture (first I picture in GOP units) of the original bit stream from the bit stream analysis unit 32 or the decoder 34, and the bit rate according to the value is obtained. An I picture is generated. In general, the bit rate of the I picture of the original compressed stream reflects the complexity of the video in that GOP. Therefore, the bit rate of the I picture generated by conversion from the P picture is also changed in accordance with the bit rate of the leading I picture that reflects the complexity of the video in the GOP, thereby preventing image quality deterioration. It becomes possible.

  Then, the proxy file storage unit 37 is generated as described with reference to FIG. 4 or FIG. 5 supplied from the encoder 36, and based on the I picture data supplied from the encoder 36, the proxy file (proxy That is, a proxy data file) is generated and stored, and is output to the reproducing unit 12.

  The proxy index storage unit 38 generates an I picture each time an I picture is generated by encoding the uncompressed baseband image data corresponding to the supplied P picture as an I picture by intraframe coding. As encoding information about a picture, a picture size (Proxy File Picture Size) and an address of those frames (Proxy File Address) are acquired and stored as a proxy index, and supplied to the bitstream analysis unit 32.

  FIG. 6 is a block diagram showing a more detailed configuration example of the playback unit 12 of FIG.

  The playback unit 12 includes an operation input acquisition unit 51, a decode processing unit 52, and a playback video output unit 53.

  The operation input acquisition unit 51 receives a user operation input and supplies a signal indicating the user operation input to the decode processing unit 52.

  The decode processing unit 52 includes an index file acquisition unit 61, a decode control unit 62, a bit stream acquisition unit 63, a proxy file acquisition unit 64, a switch 65, a decoder 66, and a reference image memory 67, and an operation input acquisition unit. Based on the user operation input supplied from 51, the decoding process is executed and supplied to the playback video output unit 53.

  The index file acquisition unit 61 acquires the index file output from the index file storage unit 33 of the conversion unit 11 and supplies it to the decode control unit 62.

  The bitstream acquisition unit 63 acquires the original bitstream supplied to the playback device 1 and plays back or outputs a frame based on the user's operation input supplied from the operation input acquisition unit 51 or playback output A frame necessary for decoding the received frame is output to the switch 65.

  The proxy file acquisition unit 64 acquires the proxy file output from the proxy file storage unit 37 of the conversion unit 11, that is, the data of the I picture generated by conversion from the P picture, and is supplied from the operation input acquisition unit 51. Based on a user operation input, a frame to be reproduced and output or a frame necessary for decoding the frame to be reproduced and output is output to the switch 65.

  The switch 65 is an I picture generated by conversion from the original bit stream supplied from the bit stream acquisition unit 63 or the P picture supplied from the proxy file acquisition unit 64 based on the control of the decode control unit 62. Is supplied to the decoder 66.

  The decode control unit 62 controls the bit stream supplied to the decoder 66 by switching the switch 65 based on the index file supplied from the index file acquisition unit 61. Specifically, when the frame to be reproduced and output is an I picture in the original bitstream, the decode control unit 62 supplies the original bitstream supplied to the reproduction apparatus 1 supplied from the bitstream acquisition unit 63. The switch 65 is controlled so that the I picture is supplied to the decoder 66. In addition, when the frame to be reproduced and output is a P picture in the original bit stream, the decode control unit 62 supplies the I picture data generated by conversion from the P picture supplied from the proxy file acquisition unit 64 to the decoder. The switch 65 is controlled to be supplied to 66. In addition, when the frame to be reproduced and output is a B picture in the original bit stream, the decode control unit 62 supplies the I picture data generated by conversion from the P picture supplied from the proxy file acquisition unit 64 to the decoder. The switch 65 is controlled so as to be supplied to 66, and after these are decoded and a reference image is generated, the switch 65 is controlled so that the B picture of the original bit stream is supplied to the decoder 66.

  The decoder 66 decodes the compressed frame image data supplied from the switch 65, generates an uncompressed baseband image, and supplies it to the reproduction video output unit 53. At this time, the decoder 34 also supplies the generated uncompressed baseband image to the reference image memory 67 when the generated uncompressed baseband image is necessary as a reference image of a frame to be decoded thereafter. When storing and decoding a P picture or B picture, the reference image stored in the reference image memory 67 is appropriately referred to.

  The reproduction video output unit 53 applies, for example, color correction, size correction, and slow reproduction field to the decoded uncompressed image data (baseband image data) supplied from the decoder 66 as necessary. Various corrections such as control are performed so that the decoded image is correctly reproduced and output, and the corrected output baseband image data is output.

  Next, with reference to FIG. 7, a case where the playback unit 12 decodes a P picture in an original bit stream will be described.

  For example, in the case of reproducing and outputting a P11 frame in an Mpeg2 LongGOP bitstream, conventionally, as shown in FIG. 7A, the I2 frame which is the first I picture of the GOP including P11 is decoded, and then P5 After generating the reference frame image data by decoding the P8 frame, the target P11 frame can be decoded.

  On the other hand, when the reproduction unit 12 to which the present invention is applied reproduces and outputs the P11 frame, as shown in FIG. 7B, the I11 frame, which is an I picture compressed in the frame corresponding to the P11 frame, is obtained from the proxy file. What is necessary is just to extract, decode, and reproduce and output.

  Therefore, when the reproduction unit 12 to which the present invention is applied reproduces and outputs the P11 frame, the number of decodes is only one compared to the conventional number of decodes of four, and the designated frame can be reproduced and output at high speed. it can.

  Next, with reference to FIG. 8, a case where the playback unit 12 decodes a B picture in an original bit stream will be described.

  For example, when a B12 frame is reproduced and output in a bitstream of Mpeg2 LongGOP, conventionally, as shown in FIG. 8A, the I2 frame which is the first I picture of the GOP including B12 is decoded, and then P5 After decoding P8, P11, and P14 frames to generate reference frame image data, the target B12 frame can be decoded.

  On the other hand, when the reproduction unit 12 to which the present invention is applied reproduces and outputs the B12 frame, as shown in FIG. 8B, intra-frame compression corresponding to the P11 frame and the P14 frame necessary as the reference image of the B12 frame. It is only necessary to decode the I11 frame and the I14 frame, which are the I pictures, and to decode and reproduce the target B12 frame using these as reference images.

  Therefore, when the reproduction unit 12 to which the present invention is applied reproduces and outputs the B12 frame, the number of decoding is 3 times compared to the conventional number of decoding times of 6, and the designated frame can be reproduced and output at high speed. .

  The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, the playback device 61 is configured by a personal computer 201 as shown in FIG.

  In FIG. 9, a CPU (Central Processing Unit) 221 executes various processes according to a program stored in a ROM (Read Only Memory) 222 or a program loaded from an HDD 226 to a RAM (Random Access Memory) 223. The RAM 223 also appropriately stores data necessary for the CPU 221 to execute various processes.

  The CPU 221, ROM 222, and RAM 223 are connected to each other via a bus 224. Also connected to the bus 224 are interfaces (I / F) 225-1 to 225-3, an HDD (hard disc drive) 226, a video special effect audio mixing processing unit 227, and a signal processing unit 228.

  Input devices such as a keyboard 202 and a mouse 203 are connected to the interface 225-1. The storage device 204 is connected to the interface 225-2 so that information can be exchanged. The interface 225-3 is connected to external video recording / reproducing apparatuses 205-1 to 205-m so that information can be exchanged. The HDD 226 can drive a hard disk and store various information.

  The video special effect audio mixing processing unit 227 is also connected to the signal processing unit 228, the storage device 204, and the video recording / playback devices 205-1 to 205-m, and the storage device 204 and the video recording / playback devices 205-1 to 205-m. The video signal supplied from any one of m or from the HDD 226 via the bus 224 is subjected to special effects, mixed with audio, etc., and supplied to the signal processing unit 228 for output. Then, the data is supplied to and stored in any of the storage device 204 and the video recording / reproducing devices 205-1 to 205-m.

  The signal processing unit 228 is also connected to the display 229 and the speaker 230. For example, the video signal supplied from the video special effect audio mixing processing unit 227 is supplied to the display 229 for display, or the audio signal is displayed on the speaker 230. To output sound.

  The display 229 includes, for example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal display), and the like, and displays an image supplied from the signal processing unit 228. The speaker reproduces and outputs the sound supplied from the signal processing unit 228.

  A drive 231 is also connected to the bus 224 as necessary, and a removable medium 206 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately installed, and a computer program read from the medium is required. Accordingly, it is installed in the HDD 226.

  FIG. 10 is a functional block diagram for explaining an example of functions when the processing to which the present invention is applied is executed by software in the personal computer 201 described with reference to FIG.

  In addition, the same code | symbol is attached | subjected to the part corresponding to the case in FIG.2, FIG.6 or FIG.9, The description is abbreviate | omitted suitably.

  When the CPU 221 executes a predetermined program, the personal computer 201 has the bit stream analysis unit 32, the decoder 34, and the encoder 36 shown in FIG. 2, and the decode control unit 62, the switch 65, the decoder 66, and the like shown in FIG. The playback video output unit 53 has the same function.

  The CPU 221 stores a bitstream corresponding to an area of the storage device 204, the external video recording / playback device 205, or the HDD 226 based on a user operation input by an input device such as the mouse 202 or the keyboard 203. The I picture and the P picture in the bit stream recorded in the unit 281 are decoded using the area of the RAM 223 corresponding to the reference image memory 35, and only the P picture is encoded as the I picture, thereby proximate. A file is generated and stored in the proxy file storage unit 37 corresponding to any area of the storage device 204, the external video recording / playback device 205, or the HDD 226.

  Further, every time the uncompressed baseband image data corresponding to the P picture is encoded as an I picture by intra-frame coding and an I picture is generated, the CPU 221 uses the picture size as encoding information regarding the generated I picture. (Proxy File Picture Size) and addresses (Proxy File Address) of those frames are stored as a proxy index in the area of the RAM 223 corresponding to the proxy index storage unit 38, the bit stream is analyzed, and the proxy index storage unit 3 is obtained, the index file described with reference to FIG. 3 is generated, and corresponds to one of the areas of the storage device 204, the external video recording / playback device 205, or the HDD 226. Index file notation It is stored in the section 33.

  The CPU 221 receives a frame command to be reproduced and output based on a user operation input by an input device such as the mouse 202 or the keyboard 203, and the CPU 221 includes the storage device 204, the external video recording / reproduction device 205, or the HDD 226. Referring to the index file stored in the index file storage unit 33 corresponding to any one of the areas, the RAM 223 corresponding to the reference image memory 67 in the same manner as described with reference to FIGS. Perform decoding using the area of.

  Specifically, when the frame to be reproduced and output is an I picture in the original bit stream, the decode control unit 62 of the CPU 221 selects a corresponding I picture in the bit stream stored in the bit stream storage unit 381. , Supplied to the decoder 66 for decoding. Then, when the frame to be reproduced and output is a P picture in the original bit stream, the decode control unit 62 of the CPU 221 converts the corresponding I picture in the proxy file stored in the proxy file storage unit 37 to the decoder 66. To be decoded. In addition, when the frame to be reproduced and output is a B picture in the original bit stream, the decoding control unit 62 of the CPU 221 determines the I and streams before and after the B picture to be reproduced in the bit stream stored in the bit stream storage unit 381. The picture or the I picture corresponding to the P picture before and after the B picture to be reproduced in the proxy file stored in the proxy file storage unit 37 is supplied to the decoder 66 to be decoded, and the reference picture memory 67 Are stored in the area of the RAM 223 corresponding to the B picture, and the B picture to be reproduced in the original bit stream is supplied to the decoder 66 as a reference image and decoded.

  The decoded frame image data is subjected to various processes by the reproduction video output unit 53 of the CPU 221 and is output to the display 229 for display.

  Next, referring to the flowchart of FIG. 11, the conversion unit 11 described with reference to FIG. 2 of the playback apparatus 1 of FIG. 1 or the CPU 221 of the personal computer described with reference to FIGS. The proxy file generation process 1 will be described.

  In step S1, the bitstream acquisition unit 31 (CPU 221) acquires the original bitstream and supplies it to the bitstream analysis unit 32 (bitstream analysis unit 32 of the CPU 221).

  In step S2, the bit stream analyzing unit 32 (the bit stream analyzing unit 32 of the CPU 221) reads one picture of the supplied original bit stream.

  In step S3, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) analyzes the read picture. In other words, the bit stream analysis unit 32 acquires information of a corresponding picture in the index file described with reference to FIG.

  In step S4, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) determines whether or not the read picture is an I picture or a P picture. If it is determined in step S4 that the picture is not an I picture or a P picture, that is, a B picture, the process proceeds to step S10 described later.

  If it is determined in step S4 that the picture is an I picture or a P picture, in step S5, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) supplies the read I picture or P picture to the decoder 34. . The decoder 34 decodes the supplied I picture or P picture and stores it in the reference picture memory 35.

  In step S6, the decoder 34 (decoder 34 of the CPU 221) determines whether or not the decoded picture is a P picture. If it is determined in step S6 that the decoded picture is not a P picture, that is, an I picture, the process proceeds to step S10 described later.

  If it is determined in step S6 that the decoded picture is a P picture, in step S7, the decoder 34 (decoder 34 of the CPU 221) selects an uncompressed image frame corresponding to the decoded P picture as an encoder 36 (CPU 221). Encoder 36). The encoder 36 encodes the supplied uncompressed image frame as an I picture, and a proxy file corresponding to any one of the proxy file storage unit 37 (the storage device 204, the external video recording / playback device 205, or the HDD 226). To the storage unit 37).

  In step S8, the proxy file storage unit 37 (the storage unit 204, the external video recording / playback unit 205, or the proxy file storage unit 37 corresponding to any area of the HDD 226) generates the encoded I picture. Is stored.

  In step S9, the encoder 36 obtains index information of the picture size (Proxy File Picture Size) and the addresses of those frames (Proxy File Address), that is, the proxy index, as the encoding information about the generated I picture. The proxy index storage unit 38 is supplied. The proxy index storage unit 38 stores the proxy index of the corresponding picture.

  If it is determined in step S4 that the read picture is not an I picture or a P picture, that is, a B picture, in step S6, the decoded picture is not a P picture, that is, an I picture. If it is determined, or after the processing of step S9 is completed, in step S10, the bit stream analysis unit 32 (CPU 221) determines whether or not the processing of all the pictures has been completed. If it is determined in step S10 that all the pictures have not been processed, the process returns to step S2, and the subsequent processing is repeated.

  When it is determined in step S10 that all the pictures have been processed, in step S11, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) analyzes the analysis result of each picture and the proxy index storage unit. 38, based on the picture size (Proxy File Picture Size) and address (Proxy File Address) of the frame encoded as an I picture by the encoder 36, that is, the frame converted from the P picture to the I picture. The index file described with reference to FIG. 3 is generated and stored in the index file storage unit 33 (the storage device 204, the external video recording / playback device 205, or the index file storage unit 33 corresponding to any area of the HDD 226). Supply and remember The process is terminated.

  By such processing, the P picture in the original bit stream is decoded and then encoded as an I picture, and the I picture (I picture (included in the original bit stream and proxy file) An index file in which information relating to P pictures in the original bitstream is described is generated.

  Next, referring to the flowchart of FIG. 12, the playback unit 12 described with reference to FIG. 6 of the playback apparatus 1 of FIG. 1 or the CPU 221 of the personal computer described with reference to FIGS. The random reproduction process 1 will be described.

  In step S41, the bit stream acquisition unit 63 (CPU 221) acquires the original bit stream, the proxy file acquisition unit 64 (CPU 221) acquires the proxy file generated by the conversion unit 11, and the index file acquisition unit 61. (CPU 221) acquires the index file generated in the conversion unit 11.

  In step S <b> 42, the decode control unit 62 (decode control unit 62 of the CPU 221) reads the index data for one picture to be reproduced next based on the user operation input acquired by the operation input acquisition unit 51.

  In step S43, the decode control unit 62 (decode control unit 62 of the CPU 221) determines whether or not the next picture to be reproduced is a B picture.

  If it is determined in step S43 that the next picture to be reproduced is not a B picture, that is, an I picture or a P picture, in step S44, the decode control unit 62 (decode control unit 62 of the CPU 221) It is determined whether or not the picture to be reproduced is a P picture in the original stream.

  When it is determined in step S44 that the next picture to be reproduced is a P picture in the original stream, in step S45, the decode control unit 62 (decode control unit 62 of the CPU 221) switches to the switch 65 (switch 65 of the CPU 221). And the I picture included in the proxy file acquired by the proxy file acquisition unit 64, that is, the I picture corresponding to the P picture of the original stream is supplied to the decoder 66 (decoder 66 of the CPU 221). The decoder 66 decodes the supplied I picture, supplies the decoded I picture to the reproduced video output unit 53 (the reproduced video output unit 53 of the CPU 221), and the process proceeds to step S50 described later.

  If it is determined in step S44 that the next picture to be reproduced is not a P picture in the original stream, that is, an I picture in the original stream, in step S46, the decoding control unit 62 (decoding control unit 62 of the CPU 221). Controls the switch 65 to supply a corresponding I picture of the original bitstream acquired by the bitstream acquisition unit 63 to the decoder 66. The decoder 66 decodes the supplied I picture, supplies the decoded I picture to the reproduced video output unit 53 (the reproduced video output unit 53 of the CPU 221), and the process proceeds to step S50 described later.

  If it is determined in step S43 that the next picture to be reproduced is a B picture, the decode control unit 62 (decode control unit 62 of the CPU 221) determines that the index file acquired by the index file acquisition unit 61 in step S47. The original bit stream I picture before and after the B picture to be decoded or the I picture of the proxy file is supplied to the decoder 66 by controlling the switch 65. The decoder 66 reads the supplied I picture.

  In step S48, the decoder 66 (decoder 66 of the CPU 221) decodes the read I picture and stores it in the reference image memory 67 (an area corresponding to the reference image memory 67 of the RAM 223).

  In step S <b> 49, the decode control unit 62 (decode control unit 62 of the CPU 221) controls the switch 65 so that the B picture to be reproduced in the original bit stream acquired by the bit stream acquisition unit 63 is sent to the decoder 66. Supply. The decoder 66 decodes the supplied B picture with reference to the reference image stored in the reference image memory 67, and supplies the B picture to the reproduction video output unit 53 (the reproduction video output unit 53 of the CPU 221) for output. .

  After the process of step S45, step S46, or step S49 is completed, in step S50, the decode control unit 62 (decode control unit 62 of the CPU 221) determines whether or not the processing of all the pictures that have received the reproduction command has been completed. Determine whether. If it is determined in step S50 that the processing for all the pictures that have been instructed to be reproduced has not been completed, the processing returns to step S42, and the subsequent processing is repeated. If it is determined in step S50 that the processing for all the pictures that have been instructed to be reproduced has been completed, the processing ends.

  By such processing, the time taken to decode the P picture and B picture is reduced as compared with the conventional case as described with reference to FIGS.

  As described above, the playback apparatus 1 to which the present invention is applied receives supply of a compressed bitstream, analyzes this to generate an index file, and converts a P picture to an I picture to generate a proxy file. The converting unit 11 and a reproducing unit 12 that realizes a predetermined frame decoding process at a high speed according to a user operation input based on the compressed bit stream, index file, and proxy file.

  Further, the personal computer 201 to which the present invention is applied can have the functions of the conversion unit 11 and the reproduction unit 12 similar to those of the reproduction apparatus 1 in FIG. 1 by executing a predetermined program.

  In the conversion unit 11, only the I picture and the P picture are decoded from the supplied original bit stream, and only the frame corresponding to the P picture is encoded again as the I picture, thereby generating a proxy file. Is done.

  At this time, the data rate of the I picture of the generated proxy file may be a fixed rate, or may be changed according to the frame rate of the first I picture of the GOP corresponding to the original bitstream (variable). Good as a rate).

  The index file generated by the conversion unit 11 includes not only information necessary for decoding the original bitstream but also an I picture included in the proxy file, that is, a frame that was a P picture in the original bitstream. , Information such as a picture size and an address relating to a frame encoded as an I picture is included.

  When the playback unit 12 refers to the index file and decodes and reproduces an I picture in the original bit stream, the corresponding I picture in the original bit stream is decoded and P in the original bit stream is decoded. When a picture is decoded and reproduced and output, the corresponding I picture (P picture in the original bitstream) of the proxy file is decoded, and when the B picture in the original bitstream is decoded and reproduced and output, The I picture of the original bit stream before and after the B picture to be reproduced and output or the I picture of the proxy file (P picture in the original bit stream) is decoded and used as a reference image. B-pictures of the original bit stream is adapted to be decoded.

  Thereby, the decoding time in random generation can be shortened.

  In addition, as described above, PI conversion for converting a P picture into an I picture is performed, and the decoding time is reduced and random access is performed by switching the stream before conversion and the converted part for use in decoding processing. The performance can be improved. However, it takes time to convert the P picture portion of the original stream to an I picture before editing and playback. For example, if there are 5 P pictures in a stream with 15 frames per 1 GOP, it takes time to decode all 5 frames and convert them into I pictures (decode and re-encode).

  Therefore, when performing PI conversion, the bit stream analysis unit 32 in FIG. 2 does not convert all P pictures into I pictures, but converts only some P pictures into I pictures as necessary. The decoder 34, the encoder 36, and the proxy file storage unit 37 may be controlled so that the proxy file is generated and stored.

  The processing time for performing PI conversion depends on the number of P pictures that are converted into I pictures (decoded and then re-encoded). Therefore, if the number of pictures to be converted can be reduced, the processing time (proxy file generation time) Can be shortened. It is preferable that the bit stream analysis unit 32 determines that the maximum number of frames necessary for decoding the converted stream does not exceed the capability of the decoder as a criterion for determining the number of pictures to be converted. The maximum number of frames necessary for decoding the converted stream (hereinafter also referred to as the number of decodable frames) varies depending on, for example, whether the playback mode is scrub playback or normal playback.

  Specifically, the number of decodable frames may be set in advance (the number of frames is directly specified) or may be specified by a higher-level application, for example. For example, the number of decodable frames may be set according to the number of decoders and CPUs used for decoding processing and the clock frequency. In such a case, the bit stream analysis unit 32 does not need to calculate the number of decodable frames, and may acquire the designated decodable frame number.

  On the other hand, for example, when only the time distribution T (for example, 20 ms) that can be given to the decoding process from the upper application is designated, the bit stream analysis unit 32 needs to calculate the number of decodable frames. is there.

  At this time, if the bit stream analyzing unit 32 selects a P picture to be converted into an I picture so that the P pictures to be converted are not as continuous as possible, the P picture to be converted into an I picture with respect to the number of decodable frames This is preferable because the number of

  With reference to FIG. 13, a variation of PI conversion in the case of 1 GOP 15 frames and the number of P pictures N = 4 (IBBPBBPBBPBBPBB) will be described.

  For example, when PI conversion is not performed, the effective sequence (picture type of 5 frames of I picture and P picture excluding B picture) is IPPPP, the longest decoding time is Open GOP, and the beginning of the stream When the two G pictures following the I picture in the stream order (for example, B0B1 in FIGS. 7 and 8) are decoded, the time required for the decoding of 7 frames is obtained.

  As described above, when all four P pictures are PI-converted, the effective sequence is IIIII, and the longest decoding time is the time required for decoding three frames when decoding any B picture. It becomes.

  On the other hand, the longest decoding time for random decoding varies as shown in FIG. 13 by not reducing all the P pictures to be converted but reducing them to a certain number. That is, when only one of the four P pictures is subjected to PI conversion, an effective sequence can be obtained by selecting a P picture to be converted into an I picture so that the P pictures to be converted are not as continuous as possible. It becomes IPPIP, and the longest decoding time is the time required for decoding for 5 frames. In addition, when only two of the four P pictures are PI-converted, by selecting a P picture to be converted into an I picture so that the P pictures to be converted are not as continuous as possible, an effective sequence is IPIIP, IIPIP or IPIPI, and the longest decoding time is the time required for decoding for 4 frames.

  Note that when only one of the four P pictures is PI-converted and when only two of the four P pictures are PI-converted, the longest decoding time becomes shorter. When a P picture to be converted to an I picture is selected so that the P pictures are not continuous as much as possible, that is, only in the effective sequence shown in FIG. This is a case where a P picture to be converted so that two frames are not consecutive in the subsequent sequence is selected. When one picture is subjected to PI conversion, the P picture is not consecutive in three frames in the converted sequence. This is a case where a P picture to be converted is selected.

  Next, with reference to FIG. 14 to FIG. 18, a specific decoding process when the P picture to be converted is not converted but reduced to a certain number will be described.

  With reference to FIG. 14, a decoding process when only one of four P pictures is subjected to PI conversion and the effective sequence is IPIPP will be described.

  As shown in FIG. 14A, in the display order, four P pictures included in 15 frames aligned with B0, B1, I2, B3, B4, P5, B6, B7, P8, B9. When the P8 picture is converted to the I picture and the I8 picture is prepared as the proxy file, the decoding time becomes the longest when B0 or B1 is decoded, and it is necessary to decode at that time. As shown in FIG. 14B, in addition to the reference images I8, P11, P14, and I2, B0 or B1 (shown as B0 in FIG. 14B) corresponds to five pictures.

  Next, with reference to FIG. 15, similarly, a decoding process when only one of four P pictures is subjected to PI conversion and the effective sequence is IPPIP will be described.

  As shown in FIG. 15A, in the display order, four P pictures included in 15 frames aligned with B0, B1, I2, B3, B4, P5, B6, B7, P8, B9. When the P11 picture is converted to the I picture and the I11 picture is prepared as the proxy file, the decoding time becomes the longest when B9 or B10 is decoded, and it is necessary to decode at that time. As shown in FIG. 15B, in addition to the reference images I2, P5, P8, and I11, B9 or B10 (shown as B9 in FIG. 15B) corresponds to five pictures.

  In all cases where only one of the four P pictures is PI-converted, the longest decoding time is not 5 pictures. That is, the longest decoding time is 5 pictures only when three converted P pictures are not consecutive as shown in FIG. 14 and FIG. In other cases, for example, when the converted sequence is IIPPP or IPPPI, the case where the longest decoding time is required for 6 pictures occurs, and the effect of PI conversion is reduced. End up.

  Next, with reference to FIG. 16, a decoding process when two of four P pictures are subjected to PI conversion and the effective sequence is IPIIP will be described.

  As shown in FIG. 16A, in the display order, four P pictures included in 15 frames aligned with B0, B1, I2, B3, B4, P5, B6, B7, P8, B9. When P8 picture and P11 picture are converted to I picture and I8 picture and I11 picture are prepared as proxy files, the decoding time is the longest when decoding B6 or B7, or B0 or B1 As shown in FIG. 16B, in addition to the reference images I2, P5, and I8, it is necessary to decode B6 or B7 (in FIG. 16B, B6 is shown. ), Or in addition to the reference images I11, P14, and I2, B0 or B1 (in FIG. 16B, 0 and the 4 picture worth of being shown).

  Similarly, FIG. 17 is a diagram for explaining a decoding process when two of four P pictures are PI-converted and an effective sequence is IIPIP. FIG. 18 is a diagram illustrating four P pictures. It is a figure for demonstrating the decoding process at the time of carrying out PI conversion of 2 sheets of these, and making an effective sequence into IPIPI.

  As shown in FIG. 17A, in the display order, four P pictures included in 15 frames aligned with B0, B1, I2, B3, B4, P5, B6, B7, P8, B9. When P5 picture and P11 picture are converted into I picture and I5 picture and I11 picture are prepared as proxy files, the decoding time is the longest when B9 or B10, or B0 or B1 is decoded As shown in FIG. 17B, it is necessary to decode at that time B9 or B10 in addition to the reference images I5, P8, and I11 (indicated as B9 in FIG. 17B). ), Or in addition to the reference images I11, P14, and I2, B0 or B1 (in FIG. 17B). Is four picture worth of being shown as B0).

  Further, as shown in FIG. 18A, in the display order, four frames included in 15 frames aligned with B0, B1, I2, B3, B4, P5, B6, B7, P8, B9. When P8 picture and P14 picture of P picture are converted to I picture and I8 picture and I14 picture are prepared as proxy files, decoding time is the longest when decoding B6 or B7, or B12 or B13 In this case, as shown in FIG. 18B, in addition to the reference images I2, P5, and I8, it is necessary to decode B6 or B7 (in FIG. 18B, it is shown as B6). In addition to the reference images I8, P11, and I14, B12 or B13 (FIG. 18). In is four picture worth of being shown as B12).

  In this case as well, the longest decoding time is not necessarily 4 pictures in all cases in which two of the four P pictures are PI-converted. That is, the longest decoding time is 4 pictures only when two converted P pictures are not consecutive as shown in FIG. 16 to FIG. In other cases, for example, when the converted sequence becomes IIPPI or IPPII, the effect of PI conversion decreases.

  14 to 18, the case where the number of B pictures between I pictures or P pictures is 2 has been described, but any number of B pictures between I pictures or P pictures is 2 or more. The number of reference images required for decoding consecutive B pictures is the same even if the number is the same, so it goes without saying that the longest decoding time in each case is the same.

  Next, the setting process of the P picture to be converted will be described with reference to the flowchart of FIG.

  In step S81, a process for calculating the number of decodable frames, which will be described later with reference to FIG. 20, is executed to obtain the number of decodable frames.

  For example, when the number of decodable frames is determined in advance or designated by a higher-level application, the bit stream analysis unit 32 of the conversion unit 11 performs the decodable frame number calculation process in step S81. The number of decodable frames that are determined in advance or specified by a higher-level application may be acquired without executing.

  In step S82, the bit stream analysis unit 32 of the conversion unit 11 satisfies N + 3> X, where N is the number of P pictures in 1 GOP of the bit stream acquired by the bit stream acquisition unit 31, and X is the number of decodable frames. Determine whether or not.

  When it is determined in step S82 that N + 3> X is not satisfied, in step S83, the bitstream analysis unit 32 assumes that PI conversion is not performed, and the process ends.

  When N + 3 ≦ X, the time required for decoding a frame that takes the longest decoding time in a bitstream that does not perform PI conversion is longer than the time required to decode the number of decodable frames X Also short. That is, in such a case, there is no need to perform PI conversion.

  If it is determined in step S82 that N + 3> X, in other words, the time required for the decoding process of the frame that takes the longest decoding time in the bitstream not subjected to the PI conversion is the decodable frame. If it is determined that the time is longer than the time required to decode the number X, in step S84, the bitstream analysis unit 32 determines that the PI sequence that is the maximum number of consecutive P pictures that can be decoded in the sequence after PI conversion. The skip number S is assumed to be X-3.

  In step S85, the bit stream analysis unit 32 determines whether N / S> 1 when S = X−3.

  When it is determined in step S85 that N / S> 1, in step S86, the bitstream analysis unit 32 sets the PI skip count S to X-3.

  When it is determined in step S85 that N / S> 1 is not satisfied, in step S87, the bit stream analysis unit 32 sets the PI skip number S to N / 2 (however, if it is not an integer, it is an integer having a close value). And

  After the process of step S86 or step S87 is completed, in step S88, the bitstream analysis unit 32 sets a P picture to be converted based on the PI skip count S, and the process is terminated.

  By such processing, based on the number of decodable frames and the number of P pictures in 1 GOP, the number of PI skips, which is the maximum number of consecutive P pictures that can be decoded in the sequence after PI conversion, is obtained. In addition, a P picture to be converted into an I picture is set. In this way, when the number of P pictures to be PI-converted is reduced as much as possible, it is possible to reduce the time for generating a proxy file as compared with the case of returning all P pictures to I pictures. It becomes. A specific setting example of the P picture converted to the I picture will be described later with reference to FIG.

  Next, the process for calculating the number of decodable frames executed in step S81 in FIG. 19 will be described with reference to the flowchart in FIG.

  In step S101, the bit stream analysis unit 32 sets the number of display frames per unit time, such as 30 frames per second or 15 frames per second.

  In step S102, the bit stream analysis unit 32 sets a time T given to the decoding process for the display of one frame by subtracting the time used for other processes in the one frame display cycle.

  Specifically, for example, if 13 ms is used per frame for processing other than decoding, 1/30 = 33 ms and 33-13 = 20 ms in the case of 30 seconds per second display, and 20 ms corresponds to the display of one frame. The time T given to the decoding process is set, and in the case of 15 frames per second display, 1/15 = 66 ms, 66-13 = 53 ms, and 53 ms is the time T given to the decoding process for one frame display. Is set.

  In step S103, the bit stream analyzing unit 32 decodes the decoding target stream by 1 GOP or more, and measures an average 1 frame decoding time A.

  In step S104, the bit stream analysis unit 32 calculates T ÷ A, sets the maximum integer value equal to or less than T ÷ A to the number of decodable frames X, and the process returns to step S81 in FIG. Proceed to

  By such processing, the number of decodable frames X is calculated and used for setting a P picture to be PI-converted.

  Using FIG. 21, the relationship between the number of decodable frames X and the number of PI skips S when the number N of P pictures included in one GOP is N = 11, and the setting of P pictures to be converted into I pictures Will be described.

  For example, when X = 4 and S = 1, P pictures after PI conversion must not exist continuously, and the sequence after PI conversion is IPIPIPIPIPIP. For example, when X = 5 and S = 2, since only two P pictures after PI conversion can be continued, the sequence after PI conversion is IPPIPPIPPIPP. For example, when X = 6 and S = 3, only three P pictures after PI conversion can be continued, so the sequence after PI conversion is IPPPIPPPIPPP.

  For example, when X = 7 and S = 4, up to four P pictures after PI conversion can be continued. Therefore, the sequence after PI conversion may simply be IPPPPIPPPPIP. Other sequences (for example, IPPPPIPPPIPP) may be used as long as the number of consecutive images is 4 or less and the number of conversions to I pictures is 2 or less. However, since it is preferable that the number of consecutive P pictures is as small as possible, when X = 7 and S = 4, as in the case of X = 6 and S = 3, the effective sequence is IPPPIPPPIPPP. This is preferable because the number of pictures to be converted is the same and the longest decoding time can be further shortened.

  For example, when X = 8 and S = 5, up to five P pictures after PI conversion can be continued, and the sequence after PI conversion is IPPPPPIPPPPP. Further, for example, when X = 9 and S = 5, it is determined that N / S> 1 in the above-described step S85, so the bitstream analysis unit 32 sets the PI skip count S to N / 2 ( However, if it is not an integer, S = 5 from the nearest integer). Even in the case of 14 (= N + 3)> X ≧ 10, it is determined that N / S> 1 in step S85 described above, so the bitstream analysis unit 32 sets the PI skip count S to S = 5. To do. At this time, the sequence after the PI conversion is IPPPPPIPPPPP.

  If X ≧ 14 (= N + 3), it is determined that PI conversion is not necessary.

  Next, referring to the flowchart of FIG. 22, the conversion unit 11 described with reference to FIG. 2 of the playback apparatus 1 of FIG. 1 or the CPU 221 of the personal computer described with reference to FIGS. The proxy file generation process 2 will be described.

  In steps S131 to S136, processing similar to that in steps S1 to S6 described with reference to FIG. 11 is executed.

  In other words, the bit stream acquisition unit 31 (CPU 221) acquires the original bit stream and supplies it to the bit stream analysis unit 32 (bit stream analysis unit 32 of the CPU 221). The bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) reads and analyzes one picture of the supplied original bit stream. In other words, the bit stream analysis unit 32 acquires information of a corresponding picture in the index file described with reference to FIG.

  Then, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) determines whether the read picture is an I picture or a P picture. If it is determined that the picture is not an I picture or a P picture, that is, a B picture, the process proceeds to step S141 described later.

  When it is determined that the picture is an I picture or a P picture, the bit stream analysis unit 32 (the bit stream analysis unit 32 of the CPU 221) supplies the read I picture or P picture to the decoder 34. The decoder 34 decodes the supplied I picture or P picture and stores it in the reference picture memory 35.

  Then, the decoder 34 (decoder 34 of the CPU 221) determines whether or not the decoded picture is a P picture. If it is determined that the decoded picture is not a P picture, that is, an I picture, the process proceeds to step S141 described later.

  If it is determined in step S136 that the decoded picture is a P picture, in step S137, the decoder 34 (decoder 34 of the CPU 221) determines that the P picture is based on the control of the bitstream analysis unit 32. It is determined whether or not the P picture is set to be converted into an I picture in the setting process of the P picture to be converted and needs to be converted. If it is determined in step S137 that the P picture is not required to be converted, the process proceeds to step S141 described later.

  If it is determined in step S137 that the P picture needs to be converted, in step S138, the decoder 34 (decoder 34 of the CPU 221) converts the uncompressed image frame corresponding to the decoded P picture to the encoder 36 (encoder of the CPU 221). 36). The encoder 36 encodes the supplied uncompressed image frame as an I picture, and a proxy file corresponding to any one of the proxy file storage unit 37 (the storage device 204, the external video recording / playback device 205, or the HDD 226). To the storage unit 37).

  In step S139, the proxy file storage unit 37 (the storage unit 204, the external video recording / playback unit 205, or the proxy file storage unit 37 corresponding to any area of the HDD 226) generates the encoded I picture. Is stored.

  In step S140, the encoder 36 uses the index information of the picture size (Proxy File Picture Size) and the addresses of those frames (Proxy File Address), ie, the proxy index, as the encoding information about the generated I picture. The proxy index storage unit 38 is supplied. The proxy index storage unit 38 stores the proxy index of the corresponding picture.

  If it is determined in step S134 that the read picture is not an I picture or a P picture, that is, a B picture, in step S136, the decoded picture is not a P picture, that is, an I picture. If it is determined, if it is determined in step S137 that it is not a P picture that needs to be converted, or after the processing of step S140 is completed, the bitstream analysis unit 32 (CPU 221) determines that all the pictures in step S141. It is determined whether or not the above process has been completed. If it is determined in step S141 that all the pictures have not been processed, the process returns to step S132, and the subsequent processing is repeated.

  When it is determined in step S141 that all the pictures have been processed, in step S142, the bitstream analysis unit 32 (the bitstream analysis unit 32 of the CPU 221) analyzes the analysis result of each picture and the proxy index storage unit. 38, based on the picture size (Proxy File Picture Size) and address (Proxy File Address) of the frame encoded as an I picture by the encoder 36, that is, the frame converted from the P picture to the I picture. The index file described with reference to FIG. 3 is generated and stored in the index file storage unit 33 (the storage device 204, the external video recording / playback device 205, or the index file storage unit 33 corresponding to any area of the HDD 226). Supply and remember Then, the process is terminated.

  Through such processing, a P-picture file generated by encoding an I picture after decoding a P picture that is set to be converted to an I picture among the P pictures in the original bitstream. And an index file in which information about an I picture (P picture in the original bit stream) included in the original bit stream and the proxy file is written.

  Next, referring to the flowchart in FIG. 23, the playback unit 12 described with reference to FIG. 6 of the playback device 1 in FIG. 1 or the CPU 221 of the personal computer described with reference to FIGS. The random reproduction process 2 will be described.

  In steps S171 to S174, the same processing as that in steps S41 to S44 described with reference to FIG. 12 is executed.

  That is, the bit stream acquisition unit 63 (CPU 221) acquires the original bit stream, the proxy file acquisition unit 64 (CPU 221) acquires the proxy file generated by the conversion unit 11, and the index file acquisition unit 61 (CPU 221). ) Obtains the index file generated in the conversion unit 11.

  Then, the decode control unit 62 (decode control unit 62 of the CPU 221) reads the index data for one picture to be reproduced next, based on the user's operation input acquired by the operation input acquisition unit 51, and then reproduces it. It is determined whether the picture to be processed is a B picture. If it is determined that the picture is not a B picture, the process proceeds to step S178 described later.

  If it is determined that the next picture to be reproduced is not a B picture, that is, an I picture or a P picture, the decode control unit 62 (decode control unit 62 of the CPU 221) It is determined whether or not it is a P picture in the stream. If it is determined that the picture is not a P picture in the original stream, the process proceeds to step S177 described later.

  If it is determined in step S174 that the next picture to be reproduced is a P picture in the original stream, in step S175, the decode control unit 62 (decode control unit 62 of the CPU 221) It is determined whether or not the picture is a P picture converted into an I picture when the proxy file is created.

  If it is determined in step S175 that the P picture to be reproduced next is a P picture converted to an I picture when the proxy file is created, in step S176, the decode control unit 62 (decode control unit 62 of the CPU 221). Controls the switch 65 (switch 65 of the CPU 221), and the decoder 66 (of the CPU 221) converts the I picture included in the proxy file acquired by the proxy file acquisition unit 64, that is, the I picture corresponding to the P picture of the original stream. To the decoder 66). The decoder 66 decodes the supplied I picture, supplies the decoded I picture to the reproduced video output unit 53 (the reproduced video output unit 53 of the CPU 221), and proceeds to step S181 described later.

  If it is determined in step S174 that the next picture to be reproduced is not a P picture in the original stream, that is, an I picture in the original stream, or in step S175, the next reproduced P picture is a proxy picture. If it is determined that it is not a P picture converted to an I picture at the time of file creation (that is, a P picture that has not been converted to an I picture at the time of proxy file creation), in step S177, the decoding control unit 62 (CPU 221). The decode control unit 62) controls the switch 65 to supply the corresponding I picture or P picture of the original bit stream acquired by the bit stream acquisition unit 63 to the decoder 66. The decoder 66 decodes the supplied I picture or P picture, supplies the decoded I picture or P picture to the reproduced video output unit 53 (the reproduced video output unit 53 of the CPU 221), and the process proceeds to step S181 described later.

  If the P picture to be referred to for decoding the P picture is a P picture converted to an I picture when the proxy file is created, the I picture included in the proxy file is used as the reference image. Needless to say.

  When it is determined in step S173 that the next picture to be reproduced is a B picture, the decode control unit 62 (decode control unit 62 of the CPU 221) determines that the index file acquired by the index file acquisition unit 61 in step S178. The original bit stream I picture before or after the B picture to be decoded, the P picture, or the I picture of the proxy file is supplied to the decoder 66 by controlling the switch 65. The decoder 66 reads the supplied I picture or P picture.

  In step S179, the decoder 66 (decoder 66 of the CPU 221) decodes the read I picture or P picture and stores it in the reference image memory 67 (an area corresponding to the reference image memory 67 in the RAM 223). .

  In step S180, the decode control unit 62 (decode control unit 62 of the CPU 221) controls the switch 65 so that the B picture to be reproduced in the original bit stream acquired by the bit stream acquisition unit 63 is sent to the decoder 66. Supply. The decoder 66 decodes the supplied B picture with reference to the reference image stored in the reference image memory 67, and supplies the B picture to the reproduction video output unit 53 (the reproduction video output unit 53 of the CPU 221) for output. .

  After the processing of step S176, step S177, or step S180 is completed, in step S181, the decode control unit 62 (decode control unit 62 of the CPU 221) determines whether or not the processing of all the pictures that have received the reproduction command has been completed. Determine whether. If it is determined in step S181 that the processing for all the pictures that have been instructed to be reproduced has not been completed, the processing returns to step S172, and the subsequent processing is repeated. If it is determined in step S181 that the processing for all the pictures that have been instructed to be reproduced has been completed, the processing ends.

  Although the time taken to generate the proxy file by such processing is shorter than the case described with reference to FIGS. 11 and 12, the time taken to decode the P picture and B picture. Is shorter than in the conventional case, so that the decoding process does not fail.

  In this way, the number of P pictures that are PI-converted (P pictures that are decoded and then re-encoded into I pictures) is determined according to the number of decodable frames, and any P picture is converted. In the decoding process, the proxy file is used while reducing the conversion time (in other words, the generation time of the proxy file) required for converting the P picture into the I picture. It is possible to reduce the longest decoding time.

  Here, the description has been given assuming that the playback device 1 having the conversion unit 11 and the playback unit 12 is configured as one device, but the conversion unit 11 and the playback unit 12 may be configured as different devices. It goes without saying that it is good.

  In addition, as described above, the series of processes described above can be executed by hardware or can be executed by software.

  When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a network or a recording medium into a general-purpose personal computer or the like.

  As shown in FIG. 9, the recording medium is distributed to provide a program to the user separately from the apparatus main body, and includes a magnetic disk (including a floppy disk) on which the program is recorded, an optical disk (CD- Not only is it composed of removable media 206 consisting of ROM (Compact Disk-Read Only Memory), DVD (Digital Versatile Disk), magneto-optical disk (including MD (Mini-Disk)), or semiconductor memory. These are configured by a ROM 222 on which a program is recorded, a hard disk included in the HDD 226, and the like provided to the user in a state of being incorporated in the apparatus main body in advance.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  In the present specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows the structure of the reproducing | regenerating apparatus to which this invention is applied. It is a block diagram which shows the structure of the conversion part of FIG. It is a figure for demonstrating an index file. It is a figure for demonstrating the case where the bit rate of I picture is fixed. It is a figure for demonstrating the case where the bit rate of I picture is made to fluctuate. It is a block diagram which shows the structure of the reproducing part of FIG. It is a figure for demonstrating the case where P picture is reproduced | regenerated at random. It is a figure for demonstrating the case where B picture is reproduced | regenerated at random. It is a block diagram which shows the structure of a personal computer. It is a functional block diagram for demonstrating the function implement | achieved in the personal computer of FIG. It is a flowchart for demonstrating the proxy file production | generation process 1. FIG. 10 is a flowchart for explaining random reproduction processing 1; It is a figure for demonstrating the case where the PI conversion number is changed. It is a figure for demonstrating the minimum decoding number. It is a figure for demonstrating the minimum decoding number. It is a figure for demonstrating the minimum decoding number. It is a figure for demonstrating the minimum decoding number. It is a figure for demonstrating the minimum decoding number. It is a flowchart for demonstrating the setting process of the P picture to convert. It is a flowchart for demonstrating the number calculation process of a decodable frame. It is a figure for demonstrating the setting of the P picture to convert. It is a flowchart for demonstrating the proxy file production | generation process 2. FIG. 10 is a flowchart for explaining random reproduction processing 2;

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Playback apparatus, 11 Conversion part, 12 Playback part, 31 Bit stream acquisition part, 32-bit stream analysis part, 33 Index file storage part, 34 Decoder, 35 Reference image memory, 36 Encoder, 37 Proxy file storage part, 51 Operation Input acquisition unit, 52 decode processing unit, 53 playback video output unit, 61 index file acquisition unit, 62 decode control unit, 63 bit stream acquisition unit, 64 proxy file acquisition unit, 65 switch, 66 decoder, 67 reference image memory, 201 personal computer, 202 mouse, 203 keyboard, 204 storage device, 205 external video recording / playback device, 221 CPU, 222 ROM, 223 RAM, 226 HDD

Claims (16)

For a bitstream composed of a plurality of GOPs (Group Of Pictures), the number of decodable frames, which is the number that can be decoded within the display time of an image corresponding to one frame, and a P picture in one GOP A continuous number calculating step for calculating a continuous number of P pictures existing between two P pictures to be decoded based on the number;
A predetermined number of P pictures to be decoded are selected from each of the plurality of GOPs constituting the bitstream based on the number of consecutive P pictures calculated by the processing of the number-of-continuations calculation step. A picture decoding step;
An encoding step of encoding each frame obtained as a result of decoding a predetermined number of the P pictures selected by the processing of the P picture decoding step as an I picture ,
Analyzing the bitstream to obtain information including a sequence of pictures ;
Based on said encoded said information Ri obtained by the step of encoding information relating to the generated encoded data by processing in the processing of the analyzing step, the attribute information available to decode the bit stream A program for causing a computer to execute processing including an attribute information generation step to be generated.   When a predetermined number larger than the number of P pictures is set as a threshold and the number of decodable frames is equal to or larger than the threshold, execution of the process of the P picture decoding step is prohibited.
  When the number of decodable frames is less than or equal to the threshold, execution of the process of the P picture decoding step is permitted
  The program according to claim 1.   The threshold is set based on the time required for decoding the frame that takes the longest time to decode in the bitstream.
  The program according to claim 2.   In the processing of the continuous number calculating step,
  A value smaller than the number of decodable frames is determined as a candidate for the number of consecutive P pictures;
  If the candidate is smaller than the number of P pictures, adopt the candidate as the number of consecutive P pictures;
  If the candidate is larger than the number of P pictures, a predetermined value other than the candidate is adopted as the number of consecutive P pictures.
  The program according to claim 1.   The predetermined value adopted as the number of consecutive P pictures when the candidate is larger than the number of P pictures is determined based on a value obtained by dividing the number of P pictures by 2
  The program according to claim 4. A process further comprising: a bitstream decoding step of decoding the bitstream based on the encoded data encoded by the process of the encoding step and the attribute information generated by the process of the attribute information generation step The program according to claim 1, which is executed by a computer. The processing of the bitstream decoding step includes
A determining step of determining an encoding type of a frame to be decoded;
A first decoding step of decoding the decoded I picture of the bitstream when the determination step determines that the decoded frame is an I picture of the bitstream;
The process of the determination step, when the frame to be decoded is determined to be a P-picture of said bitstream, of encoded data encoded by the processing in the encoding step, the P to be decoded A second decoding step of decoding the encoded data corresponding to a picture ;
When it is determined by the process of the determining step that the frame to be decoded is a B picture of the bit stream, a reference picture is decoded by decoding an I picture of the bit stream necessary for decoding the B picture . Or decoding the encoded data corresponding to the P picture of the bitstream necessary for decoding the B picture out of the encoded data encoded by the processing of the encoding step. A reference image generation step for generating a reference image;
The program according to claim 6 , further comprising: a third decoding step of decoding the B picture of the bit stream with reference to the reference image generated by the processing of the reference image generation step. Bit rate of the I-picture obtained as a result of the processing in the encoding step, program according to claim 1 which is a fixed rate. Bit rate of the I-picture obtained as a result of the processing in the encoding step, program according to claim 1 which is the fluctuation rate. The program according to claim 1 , further comprising: a decoding process executable frame number calculating step of calculating the decodable frame number .   The encoded data generated by the processing of the encoding step is
    A predetermined number of the P pictures selected by the P picture decoding step are decoded, and the resulting frames are encoded by the encoding step processing. The resulting I pictures Yes,
  The encoding information regarding the encoded data is:
    Includes picture size and address for each of the encoded data
  The program according to claim 1.   The encoding method employed for the processing executed by the computer is an MPEG (Moving picture experts group) encoding method.
  The program according to claim 1. In an information processing apparatus that generates information that can be used to decode a bitstream,
For the bitstream composed of a plurality of GOPs (Group Of Pictures), the number of frames that can be decoded within the display time of an image corresponding to one frame and the number of P pictures in one GOP A continuous number calculating means for calculating a continuous number of P pictures existing between two P pictures to be decoded;
A predetermined number of P pictures to be decoded are selected from each of the plurality of GOPs constituting the bitstream based on the number of consecutive P pictures calculated by the processing of the number-of-continuations calculation step. Picture decoding means;
Encoding means for encoding each frame obtained as a result of decoding a predetermined number of the P pictures selected by the P picture decoding means , respectively, as an I picture ;
Analyzing the bitstream , obtaining information including the arrangement order of pictures, based on the information obtained by the analysis of the bitstream and the coding information related to the coded data generated by the coding means, An information processing apparatus comprising: attribute information generating means for generating attribute information usable for decoding the bitstream. The information according to claim 13 , further comprising: a bitstream decoding unit that decodes the bitstream based on the encoded data generated by the encoding unit and the attribute information generated by the attribute information generation unit. Processing equipment. In an information processing method of an information processing apparatus for generating information usable for decoding a bitstream,
For the bitstream composed of a plurality of GOPs (Group Of Pictures), the number of frames that can be decoded within the display time of an image corresponding to one frame and the number of P pictures in one GOP A sequence number calculating step for calculating a sequence number of P pictures existing between two P pictures to be decoded,
A predetermined number of P pictures to be decoded are selected from each of the plurality of GOPs constituting the bitstream based on the number of consecutive P pictures calculated by the processing of the number-of-continuations calculation step. A picture decoding step;
An encoding step of encoding each frame obtained as a result of decoding a predetermined number of the P pictures selected by the processing of the P picture decoding step as an I picture ,
Analyzing the bitstream to obtain information including a sequence of pictures ;
Based on said encoded said information Ri obtained by the step of encoding information relating to the generated encoded data by processing in the processing of the analyzing step, the attribute information available to decode the bit stream An information processing method including an attribute information generation step to be generated.   A recording medium on which the program according to claim 1 is recorded.

Images