JP2010252394A - Image coding apparatus, image coding method, and image editing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To align phases of pictures in a coded stream before and after editing in order to suppress deterioration of the quality of re-encoded pictures even when the coded stream is edited at an arbitrary picture position. <P>SOLUTION: The image coding apparatus includes: a coding processing section which outputs coded data obtained by coding inputted frames; and a decoding processing section which decodes the coded data and outputs the decoded data including respective frames to be reproduced. The decoding processing section outputs the decoded data having been subjected to the editing, in which the frames corresponding to the editing are inserted in the respective frames to be reproduced included in the decoded data, when the editing of modifying reproduction modes of the respective frames to be reproduced included in the decoded data is carried out. The coding processing section codes the decoded data having been subjected to the editing in such a way that respective inputted frames and frames corresponding to the respective inputted frames included in the decoded data having been subjected to the editing are coded in the same picture type. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image encoding device, an image encoding method, and an image editing device that can edit, decode, and re-encode an encoded stream obtained by encoding uncompressed video data.

  With the recent development of digital technology, digital audio / video recording / reproducing apparatuses such as HDD (Hard Disc Drive) or DVD (Digital Versatile Disc) recorders and DVD players have been put into practical use. In such a digital system, the image data is compressed and streamed by the MPEG (Moving Picture Experts Group) system.

  The MPEG2 system (ISO / IEC13818-2) is an intra-frame coded image (hereinafter I picture: intra coded picture), an image that is inter-frame coded by unidirectional prediction (hereinafter P picture: predictive coded picture), and bidirectional. A coded stream is formed by an image (hereinafter referred to as B picture: bi-directionally predictive coded picture) that has been inter-frame coded by prediction.

  An MPEG-2 video stream is encoded in units of GOP (group of pictures). One GOP is usually composed of about 15 pictures, starts from an I picture, and includes a P picture or a B picture.

  An I picture is created by intra-picture coding, and does not perform predictive coding from the previous picture, that is, performs intra-picture coding without referring to other pictures. It is an image having the information. A P picture is a picture that performs inter-picture predictive coding with reference to a temporally previous I picture or P picture. Therefore, information on an already decoded I picture or P picture that precedes in the stream order is required. It is an image. A B picture is a picture that uses two I pictures or P pictures before and after (past and future) and performs predictive coding in both directions. This is an image that requires information of I picture or P picture 2 image.

  Therefore, when a B picture exists, the code image order does not match the display image order. That is, when decoding a B picture, an image that is displayed later in the playback order than that image is referred to, and therefore an I picture or P picture that is the reference image is arranged before the B picture in the code order. Therefore, when editing a video stream obtained by such an MPEG2 system, the reference picture of an image encoded into a P picture and a B picture changes due to editing. Therefore, necessary picture data is extracted. It cannot simply be connected.

  Therefore, when editing a moving image encoded by the MPEG method, it is conceivable to edit in units of GOPs. However, if there is a change in the middle of a GOP, there is a problem that editing cannot be performed at that point. Therefore, Patent Document 1 describes a video stream editing method that connects an editing point in the middle of a GOP and an editing point in the middle of a GOP.

  In the editing method described in Patent Document 1, a video stream that can be continuously played by linking a part or all of the first video stream subjected to interframe prediction encoding and a part or all of the second video stream. Get. In this case, the first partial video stream is cut out from the first video stream up to immediately before the picture encoded by intraframe prediction encoding or one-way interframe prediction. Then, from the second video stream, the second partial video stream after the picture encoded by intra-frame encoding or unidirectional inter-frame prediction is cut out. It is determined whether or not the picture displayed immediately before the second partial stream cut out from the second video stream is an intra-frame encoded picture, and is determined to be an intra-frame encoded picture. If this picture is the first intra-frame coded picture, and it is determined that the picture is not an intra-frame coded picture, the picture from the intra-frame coded picture immediately before the picture to the picture is determined. A decoded image of the picture is obtained by sequentially decoding the unidirectional inter-frame prediction coded pictures, and then the decoded picture is re-encoded by intra-frame coding processing and obtained from the first video stream. Between the first partial stream and the second partial stream obtained from the second video stream. By inserting the reduction picture, thereby enabling editing even when there is a splicing point in the middle of the GOP.

  However, when editing such an encoded stream, the entire code amount can be reduced by deleting a part of the stream, but the bit rate at the time of encoding is changed. It is difficult to adjust the code amount after editing.

  For example, uncompressed digital video data is compressed and encoded into GOP units composed of I-pictures, B-pictures, and P-pictures by a method such as MPEG, and recorded on a magneto-optical disc (MO disc). When recording on a medium, the data amount (bit amount) of the compressed video data after compression encoding is equal to or less than the recording capacity of the recording medium while maintaining high quality of the video after decompression decoding, or the transmission capacity of the communication line. Must be:

  For this reason, a moving image encoding method using prior analysis may be employed. In the encoding method using the pre-analysis, first, the uncompressed video data is preliminarily compressed and encoded in the first pass to estimate the amount of data after compression encoding, and the estimated data in the next second pass. The compression rate is adjusted based on the amount, and compression encoding is performed so that the amount of data after compression encoding is equal to or less than the recording capacity of the recording medium (hereinafter, such compression encoding method is also referred to as two-pass encoding). ).

  In this two-pass encoding, if the transition of the buffer occupancy due to the allocation of the code amount is not taken into account, the buffer will cause a buffer failure such as overflow or underflow in the actual encoding process. Even if a process for avoiding buffer failure is incorporated in the actual encoding process, the code amount generated when the image is encoded deviates from the assigned target code amount. Therefore, the code amount control after generation cannot be performed accurately. In this case, a code amount that is different from the code amount that should be assigned is assigned to an image, which causes deterioration of image quality in encoding.

  Therefore, in Patent Document 2, an image is analyzed before encoding an input image, a complexity for each image is calculated, and a code amount corresponding to the complexity is collectively allocated to images within a certain section. Therefore, by estimating the change in the amount of code occupied by the buffer, it is possible to avoid buffer failure, and by performing appropriate code allocation based on the given bit rate and buffer size, the image quality of the encoded image can be reduced. A moving picture coding apparatus using a prior analysis to improve is disclosed.

Japanese Patent Laid-Open No. 2002-300528 JP 2002-232882 A

  However, in the case of the two-pass encoding method described in Patent Document 2, if the stream encoded in the first pass is edited on a picture-by-picture basis, the picture phase after the edit point is decoded when the edited stream is decoded and re-encoded. Cannot match the picture phase of the encoded stream of the first pass. In this case, for example, a picture originally encoded as a B picture may be encoded as an I picture, and the edited image may deteriorate. Furthermore, since the picture phase of the edited stream does not match the picture phase of the first-pass encoded stream that has been analyzed in advance, it becomes impossible to refer to the complexity after the edit point, and the edited stream has two passes.・ Encoding becomes impossible.

  An image encoding device according to the present invention is an image encoding device that edits an encoded stream obtained by encoding uncompressed video data, and the encoded stream is edited at one or a plurality of editing points. An editing unit for creating an editing instruction; a decoding processing unit for decoding the encoded stream according to the editing instruction to generate an edited stream; and a code for re-encoding the edited stream to generate an edited encoded stream An encoding processing unit, and the encoding processing unit generates the edited encoded stream by aligning picture phases so that the same picture type is obtained in the same frame as the encoded stream.

  In the present invention, when an encoded stream obtained by encoding uncompressed video data is edited and re-encoded, the picture phase is aligned and re-encoded so that the same picture is obtained in the same frame as the original encoded stream. Since encoding is possible, for example, what was originally a B picture is not encoded into an I picture, and deterioration in image quality can be prevented.

  An image editing apparatus according to the present invention includes: an image encoding processing unit that edits an encoded stream obtained by encoding uncompressed video data; and two or more recording units that record the encoded stream. An encoding processing unit that generates an editing instruction so that the encoded stream recorded in one storage device is edited at one or a plurality of editing points; and decoding the encoded stream according to the editing instruction A decoding processing unit that generates an edited stream; and an encoding processing unit that re-encodes the edited stream to generate an edited encoded stream, and the encoding processing unit includes the encoded stream and The edited encoded stream is generated by aligning picture phases so that the same picture type is obtained in the same frame, and the other storage device stores the edited encoded stream. Recording the program.

  In the present invention, data edited from the original encoded stream recorded in one storage device is re-encoded with the same picture phase so that the same picture is obtained in the same frame as the original encoded stream. This can be dubbed to the other storage device without degrading the image quality. Also, since the edited stream is encoded with the same picture phase in the same frame as the encoded stream, the complexity of each frame is analyzed when the original encoded stream is generated. This will allow two-pass encoding of the edited stream.

  According to the present invention, even if editing is performed at an arbitrary picture position of the encoded stream, the picture phase of the encoded stream obtained by encoding after editing and the encoded stream before editing can be aligned, and re-encoding can be performed. It is possible to suppress degradation of the image quality. According to the present invention, even if the encoded stream is edited, the same picture phase as that of the pre-edited encoded stream can be obtained. Even so, 2-pass encoding can be performed.

It is a block diagram which shows the image coding apparatus concerning embodiment of this invention. It is a block diagram which shows the detail of the encoding process part of the image coding apparatus concerning embodiment of this invention. It is a block diagram which shows the detail of the decoding process part of the image coding apparatus concerning embodiment of this invention. 2A and 2B are diagrams illustrating an original stream editing method in the image encoding apparatus according to the embodiment of the present invention, in which FIG. 1A is a diagram illustrating the original stream, and FIG. 2B is a GOP extracted from the original stream including the editing point; (C) is a diagram showing a part of the edited playlist. It is a figure explaining the method to produce | generate the edited stream in the image coding apparatus concerning embodiment of this invention, Comprising: (a) is the total number of pictures (n + (N-m + 1) contained in a re-encoded picture group. ) <N, (b) is a diagram illustrating the case of n + (N−m + 1)> N. It is a figure explaining the method to produce | generate the edited stream in the image coding apparatus concerning embodiment of this invention, Comprising: (a) is the total number of pictures (n + (N-m + 1) contained in a re-encoded picture group. ) = 2N, (b) is a diagram illustrating a case where n + (N−m + 1) = N. FIG. 7 is a diagram for explaining a method of generating an edited stream in the image encoding device according to the embodiment of the present invention, where (a) is an edit point B located at the first GOP, and (b) is an edit It is a figure explaining the case where the point A is located in the last GOP. It is a flowchart which shows the encoding method of the 2nd pass in the image coding apparatus concerning embodiment of this invention. Similarly, it is a flowchart showing a second pass encoding method in the image encoding device according to the embodiment of the present invention. It is a flowchart which shows the method of calculating the target code amount using complexity in the edited stream. Similarly, it is a flowchart illustrating a method for calculating a target code amount using complexity in an edited stream. It is a flowchart which shows the method of calculating the complexity of the frame inserted in order to arrange a picture phase in an edited stream.

Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is applied to a moving image encoding apparatus with an editing function using a two-pass encoding method.
The image encoding apparatus according to the present embodiment has a two-pass encoding function. In 2-pass encoding, uncompressed audio / video data is preliminarily compressed and encoded to estimate the amount of data after compression encoding (first pass), and then the compression rate is adjusted based on the estimated amount of data In this method, compression encoding is performed so that the amount of data after compression encoding is less than the recording capacity of the recording medium (second pass). Therefore, in the first pass, when a title such as uncompressed video data is recorded by MPEG encoding, for example, each frame is analyzed to determine its complexity (preliminary analysis). Record along with the converted title. Then, in the second pass, a code amount is allocated and encoded according to the complexity so as to obtain a predetermined bit rate. By assigning the code amount using the complexity, it is possible to improve the image quality at a limited bit rate and prevent underflow and overflow of the buffer.

  The image encoding apparatus according to the present embodiment can perform the above-mentioned two-pass encoding even when generating a playlist or an edited title in which an MPEG-coded recorded title (program) is edited in units of pictures. It is to make.

  Usually, the pre-analyzed encoded stream requires complexity for each frame, but each frame is encoded to a predetermined picture type, and the complexity also corresponds to the picture type. ing. Therefore, if the encoded stream is edited and decoded in the middle of a GOP, when the title after editing is encoded again, the frame corresponding to the encoded stream before editing is not encoded to the same picture type. . In other words, for example, a frame that was originally encoded with a B picture may be encoded with an I picture, and the image quality may be degraded in the encoded stream after editing.

  Furthermore, when the encoded stream after editing is encoded into a picture of a different picture type in the frame (same image) corresponding to the pre-analyzed encoded stream before editing, the optimal target is determined using the pre-analysis result. Two-pass encoding in which the code amount is set and encoded becomes impossible. On the other hand, the image coding apparatus according to the present embodiment inserts a predetermined image (hereinafter referred to as an inserted image) into the coded stream after editing, so that the pre-analyzed coding before editing is performed. The picture phase is aligned so that the frame corresponding to the stream is encoded into a picture of the same picture type. In other words, aligning the picture phase means matching the picture type so that the constituent picture of the encoded stream once encoded becomes the same picture type as that of the constituent picture when it is decoded and re-encoded. Indicates encoding. Refer to the pre-analysis result of the pre-edited encoded stream even if the pre-analyzed encoded stream is edited and re-encoded with the edit point in the GOP by aligning the picture phase. And allows two-pass encoding.

  Here, the configuration of the image encoding apparatus according to the present embodiment will be described first, and then the edited MPEG stream (hereinafter referred to as the edited encoded stream ST1) is converted into the MPRG stream (hereinafter referred to as the edited encoded stream ST1). The method of matching the picture phase of the original stream ST0) will be described, and finally, the two-pass encoding method of the edited stream using the pre-analysis result of the original stream will be described.

  FIG. 1 is a block diagram showing an image encoding apparatus according to the present embodiment. The image encoding device 1 includes an encoding processing unit 2, an editing unit 3, a decoding processing unit 4, a display unit 5, and storage interfaces (I / F) 6 and 7. The display unit 5 may be a device different from the image encoding device 1. In this embodiment, the storage I / F is described as two, but it goes without saying that there may be two or more storage I / Fs. The storage I / F 6 is connected to a storage device 30 such as an HDD, and the storage I / F 7 is connected to a storage device 40 such as a DVD recorder. The storage devices 30 and 40 may be included in the image encoding device.

  In the image encoding apparatus 1 according to the present embodiment, the input non-compressed video data is MPEG encoded by the encoding processing unit 2 and recorded in the storage devices 30 and 40, or stored by the editing unit 3. The encoded stream recorded in the devices 30 and 40 is edited. Further, the encoded stream is decoded by the decoding processing unit 4 and displayed (reproduced) on the display unit 5. Here, as described above, the present image encoding apparatus 1 edits an encoded stream obtained by encoding video data by the encoding processing unit 2 (hereinafter referred to as an original stream), and an edited encoded stream. When generating ST1, it has a function of aligning the picture phases of the original stream ST0 and the edited encoded stream ST1. That is, in the original stream ST0, for example, a frame encoded into an I picture is also encoded into an I picture in the edited stream.

  Thus, when the original stream ST0 is generated, its complexity is analyzed, and if it is stored together with the original stream ST0, the complexity can be referred to when the edited stream ST1 is generated. Therefore, an optimal code amount according to the complexity can be assigned. That is, it is possible to perform two-pass encoding in which encoding is performed while controlling the amount of code when the edited encoded stream ST1 is generated. As a result, even if the bit rate is made smaller than that at the time of encoding the original stream, the deterioration of the image quality can be suppressed, and recording can be performed on a medium with limited storage capacity such as a DVD.

  Hereinafter, each block will be described in detail. FIG. 2 is a block diagram illustrating details of the encoding processing unit. As shown in FIG. 2, the encoding processing unit 2 is supplied with uncompressed video data supplied from outside or decoded data from the decoding processing unit 4, and encodes this into an encoding unit 21 and an encoding unit. It has an encoding buffer 22 for temporarily storing the data. In addition, an analysis unit 23 that analyzes various information at the time of encoding and calculates complexity is provided. Furthermore, a code amount assigning unit 24 for assigning a target code amount of each picture based on the complexity is provided, thereby enabling two-pass encoding. Furthermore, a code amount control unit 25 that controls the encoding unit 21 to perform encoding with the code amount specified by the control unit (not shown) or the code amount assigned by the code amount allocation unit 24 is provided. Furthermore, in order to make the picture phases before and after editing coincide with each other in the case of two-pass encoding, a pause / resume control unit 26 that controls the timing of pause and restart of the encoding process is provided.

  Normally, when recording a title, the encoding processing unit 2 MPEG-encodes the input uncompressed video data at, for example, a relatively high bit rate, and converts the encoded stream (original stream) to the storage I / F 6. Are stored in the storage device 30 having a large storage capacity such as an HDD. At that time, the analysis unit 23 analyzes the complexity X based on a code amount and a quantization scale generated when each frame of the video data is encoded into a predetermined picture. This complexity X indicates the complexity when each frame is encoded into a picture of a predetermined picture type, and is associated with each picture. This complexity X is stored in the storage device 30 together with the original stream. The complexity level X may be recorded not in the storage device 30 but in a memory (not shown) or the like provided in the device.

  The analysis unit 23 includes a feature amount observation unit 51 and a complexity calculation unit 52. The feature quantity observation unit 51 observes the generated code quantity and average quantization scale of the image as the feature quantity at the time of the first pass encoding. For example, when the encoding unit 21 encodes a title made up of uncompressed video data based on a specified bit rate R under the control of the code amount control unit 25, the generated code amount S [f of each frame f is encoded. ] And the average quantization scale Q [f].

The complexity calculation unit 52 calculates the complexity based on the generated code amount and the average quantization scale observed by the feature amount observation unit 51. For example, when the generated code amount is S [f], the average quantization scale is Q [f], and the complexity X [f], the complexity X [f] is
X [f] = S [f] * Q [f]
And can be calculated.

  Note that details of the calculation method of the complexity X for calculating the target code amount in the two-pass encoding are described in, for example, Patent Document 2. Normally, the code amount allocation unit 24 calculates the target code amount from the complexity calculated in this way, and encoding is performed so that the target code amount is obtained in the second pass encoding.

  Here, as will be described later, an inserted image is inserted into the edited stream as necessary before and / or after the editing point in order to match the picture phase with the original stream after the encoding. When the edited stream is encoded and the edited encoded stream ST1 is generated, the complexity calculating unit 52 in the present embodiment refers to the complexity analyzed at the time of generating the original stream ST0. Is supplied to the code amount assigning unit 24. Also, the complexity when generating the picture by encoding the inserted image (hereinafter referred to as “inserted picture”) is calculated from the complexity of the original stream and supplied to the code amount allocation unit 24.

The code amount assigning unit 24 assigns a target code amount for encoding each frame (generating each picture) based on the complexity supplied from the complexity calculating unit 52. For the target code amount, for example, the total code amount that can be used in the code amount allocation section corresponding to a predetermined GOP length or the like is distributed according to the complexity of each image. When the code amount allocation interval is L frames and the total code amount that can be allocated to the frames from the f-th frame to the (f + L-1) -th frame is Ra [f], Ra [f] is the complexity X [f]. The target code amount T [f] of each frame that is proportionally distributed in the case where the sum of the complexity X [f] of the allocated section is Xsum,
T [f] = (X [f] / Xsum) * Ra [f]
Can be calculated as

  The code amount control unit 25 controls the encoding processing unit 2 so that encoding is performed at a bit rate set in advance or instructed from the outside during the first pass encoding. In the second pass encoding, a quantization scale is calculated based on information from the code assigning unit 24, and encoding is performed using this quantization scale. If there is a difference between the generated code amount and the assigned code amount when the actually generated code amount is measured at this time, feedback control is performed so that the code amount approaches a predetermined bit rate, and the code amount is reduced. By controlling, encoding is performed so that the target code amount is obtained. Briefly, if the generated code amount exceeds the target code amount, the generation of the code is suppressed by increasing the quantization scale, and if it is lower, the quantization scale is decreased to generate the code. To increase.

  Furthermore, the code amount control unit 25 monitors the buffer occupation amount of the encoding buffer 22, and monitors and quantizes the code amount generated at the time of encoding so that overflow or underflow does not occur in the encoding buffer 22. Control scale adjustment and stuffing. For example, in order to avoid overflow of the encoding buffer 22, the quantization scale is increased to suppress the generation of codes, or the information to be encoded is deleted without encoding, and the generated code amount is suppressed. On the other hand, in order to avoid the underflow of the encoding buffer 22, the quantization scale is reduced to increase the amount of generated code, or stuffing is performed to increase the generated code amount.

  The encoding unit 21 encodes uncompressed video data supplied from the outside and decoded data sent from the decoding processing unit 4 in accordance with given parameters, and generates compressed data. The encoding unit 21 measures the generated code amount and notifies the code amount control unit 25 of the generated code amount. Further, in the first pass encoding, the generated code amount and the average quantization scale are notified to the feature amount observation unit 51.

  The encoding buffer 22 can store the data encoded by the encoding unit 21 and output it at a fixed bit rate. The encoding buffer 22 can absorb fluctuations in the generated code amount for each image.

  Returning to FIG. 1, the decoding processing unit 4 decodes the MPEG stream recorded in the storage devices 30 and 40 and displays it through the display unit 5, and supplies it to the encoding processing unit 2 for re-encoding. To do. FIG. 3 is a block diagram showing details of the decoding processing unit 4.

  The decoding processing unit 4 includes a decoding unit 61 that decodes the encoded stream encoded by the encoding processing unit 2 or the encoded stream recorded in the storage devices 30 and 40, and the decoded audio / video data. Are temporarily stored, and a pause / resume control unit 63 that controls the timing of the decoding process of the decoding unit 61.

  When executing the above-described two-pass encoding, the decoding processing unit 4 decodes the original stream ST0 encoded in the first pass. Here, when the edited encoded stream ST1 is generated by the encoding processing unit 2, GOPs are sequentially supplied from the editing unit 3 in accordance with an editing instruction (playlist) corresponding to the virtual title generated by the editing unit 3. This is decrypted. At this time, if the edit point is in the middle of the GOP as will be described later, the pause / resume control unit 63 repeatedly outputs the decoded image immediately before it or only the decoded image necessary for the edited encoded stream. Output and control. In this way, the decoded data (edited stream) decoded and output by the decoding processing unit 4 is encoded by the encoding processing unit 2 into the edited encoded stream ST1.

  Returning to FIG. 1, the editing unit 3 generates a playlist corresponding to a virtual title so that the original stream stored in the storage device 30 is edited at a desired editing point. In addition, the editing unit 3 in the present embodiment has a function of controlling the encoding processing unit 2 and the decoding processing unit 4 in order to cause the pass-edited video to be two-pass encoded. In addition, you may provide the control part for controlling the decoding process part 4 and the encoding process part 2 separately. Details of this control method will be described later.

  When the edited encoded stream ST1 is generated, for example, the editing unit 3 instructs the user to cut a desired edit point, a bit rate when generating the edited encoded stream ST1, and the like. Is instructed. The editing unit 3 edits the original stream ST0 by generating a playlist in accordance with the editing point instruction. The editing unit 3 can reproduce the edited stream by supplying the GOP of the original stream ST0 to the decoding processing unit 4 according to the playlist. When the edited stream is subjected to two-pass encoding, the decoded processing unit 4 outputs the edited stream, and the encoding processing unit 2 performs two-pass encoding. That is, the bit rate is instructed so that the encoded stream ST1 after encoding has a desired data capacity, a target code amount corresponding to the complexity X is assigned, and the edited stream is encoded and edited based on this. The encoded stream ST1 is generated.

  Next, a method of generating the edited encoded stream ST1 by 2-pass encoding will be described in detail. In the present embodiment, the image encoding device 1 performs editing by designating, for example, any two pictures with respect to an already recorded title (original stream), and dubbing the edited title. The case of performing is explained.

  FIG. 4 is a diagram showing a method for editing the original stream ST0. 4A shows the original stream, FIG. 4B shows the GOP of the original stream including the edit points, and FIG. 4C shows a part of the edited playlist. FIG.

  As shown in FIG. 4A, the original stream ST0 is composed of a plurality of GOPs # 1, # 2,... #J,. In this embodiment, for simplicity of explanation, each GOP is assumed to be composed of N (N is an integer) pictures, and for example, I, P, B, B, P,. In the following description, it is assumed that the arrangement order of pictures (encoding rules) is the same. Note that the present application can be applied if the same frame is encoded with the same picture in the original stream ST0 and the edited encoded stream ST1. That is, even if the GOP length and the coding rule are different between GOPs, if the picture phase is the same between the original stream ST0 and the edited coded stream ST1, the GOP length, the coding rule, etc. are all set. It is not necessary for GOPs to be the same.

  Further, the complexity of each picture constituting the original stream ST0 (GOP) is analyzed by the analysis unit 23 when the original stream ST0 is generated from the video data, and is stored in the storage device 30 as the complexity X. .

  In the following description, a case where the edited encoded stream ST1 is generated using the editing points A and B shown in FIG. 4 will be described. Here, the original stream ST0 is composed of GOP # s (1 ≦ s ≦ S), and each GOP # s is composed of a picture #t (1 ≦ t ≦ N). The edit point A is assumed to indicate between a picture #n (1 ≦ n ≦ N) and # n + 1 of GOP # j (1 ≦ j ≦ S). When picture # n = # N, edit point A indicates a GOP boundary. Furthermore, it is assumed that the pictures after the editing point A are cut. The edit point B is assumed to indicate between # m−1 and #m (1 ≦ m ≦ N) of GOP # k (1 ≦ k ≦ S). When picture # m = # 1, edit point B indicates a GOP boundary. Furthermore, it is assumed that the picture before the editing point B is cut.

  For example, as shown in FIG. 4B, when viewed in GOP units, the stream from the first picture # 1 of GOP #j to picture #n immediately before editing point A and the editing point B of GOP #k The stream from the next picture #m to the last picture #N is extracted and edited so that the edit points A and B are continuous as shown in FIG.

  The editing unit 3 can edit the original stream (title) by designating any two editing points A and B in the original stream. That is, the edited stream is ended at the editing point A, the edited stream is started at the editing point B, or the editing point A and the editing point B are continuously played back. be able to. When editing, a playlist corresponding to a virtual title is generated regardless of whether or not the original stream is operated. The original stream may be edited when generating the playlist. The editing unit 3 continuously reproduces, for example, the editing point A and the editing point B by supplying a stream in GOP units to the decoding processing unit 4 that is an MPEG AV decoder while referring to the playlist.

  The edited stream can be reproduced by outputting the audio and video signals output by the decoding processing unit 4 to the display unit 5, and the edited stream decoded by the decoding processing unit 4 can be reproduced by the encoding processing unit 2. 2-pass encoding is performed by referring to the complexity of the original stream when encoding is performed. Then, by supplying this to the storage device 40, the edited stream edited from the original stream can be recorded (dubbed) by two-pass encoding.

  In 2-pass encoding, the condition is that the picture type (picture phase) of each frame at the time of encoding in the second pass is the same as that at the time of encoding in the first pass. This is because the code amount is assigned according to the complexity analyzed in the first pass, and therefore, if the picture phase is different, an appropriate code amount cannot be assigned.

  Therefore, the rule of the picture structure and the GOP length (total number of pictures in the GOP) in each GOP of the edited encoded stream ST1 generated by the encoding processing unit 2 (hereinafter also referred to as a picture configuration) are 1. It is necessary to be the same as when encoding the pass. That is, it is necessary to match the picture phase of the edited encoded stream ST1 with the phase of the original stream.

  5 to 7 are diagrams illustrating a method for generating an edited stream. The editing unit 3 controls the decoding processing unit 4 and the encoding processing unit 2 in the six cases shown in FIGS. FIG. 5 shows a case where two editing points A and B are connected, and pictures # 1 to #n of GOP #j and pictures #m to #N of GOP #k shown in FIG. If the total number of pictures included in the picture group consisting of (hereinafter referred to as an edited picture group) is different from an integer multiple of N, one or more predetermined images (inserted images) are placed between the editing points A and B. The total number of pictures included in a picture group (hereinafter referred to as a re-encoded picture group) when inserted and encoded is controlled to be an integer multiple of N.

  FIG. 6 shows a case where the total number of pictures included in the edited picture group is N or 2N. In this case, the edit point B coincides with the GOP boundary in the edited encoded stream ST1, so that it is not necessary to insert an insertion image. Further, FIG. 7 shows a case where there is one edit point, and a case where the edit point B comes to the top or a case where the edited stream ends at the edit point A. An edited stream is generated by combining one or more of these six editing methods.

  First, a case where the total number of pictures included in the edited picture group does not become an integer multiple of N will be described.

(1) n + (N−m + 1) <N (see FIG. 5A)
As shown in FIG. 5A, when the total number of pictures included in the edited picture group (n + (N−m + 1)) <N, the picture #n from the first picture # 1 to the edit point A of GOP # j (A part of GOP # j) The number of pictures included in 102, the number of pictures included in the picture #m through the last picture #N (part of GOP # k) 103 (N− This is a case where the sum n + (N−m + 1) of m + 1) is less than N.

  In this case, the editing unit 3 (m−n−1) first decoded images J obtained by decoding the picture #n of GOP # j between the editing point A and the editing point B in GOP # j. The encoding processing unit 2 is controlled to insert and generate the re-encoded picture group 101. In other words, the editing unit 3 inserts (mn−1) decoded images J between the editing points A and B of the edited picture group, so that the number of pictures in the re-encoded picture group is N. As a result, the re-encoded picture group 101 has the same number of pictures as the GOP.

  By inserting (m−n−1) decoded images J, a part 102 of GOP # j and a part 103 of GOP # k can have the same picture phase as GOP # j and #k, respectively. . Thereby, the complexity X of GOP # j and #k can be referred to for the part 102 of GOP # j and the part 103 of GOP # k having the same picture phase.

  By the way, the inserted picture (first inserted picture) obtained by encoding the decoded image J is a picture that does not exist in the original stream ST0, and its complexity has not been analyzed. However, this decoded image J is an image obtained by decoding the picture #n of GOP # j, and the complexity when generating the picture #n of GOP # j from the image J is obtained by a prior analysis. Therefore, in the present embodiment, the complexity of the inserted picture generated from the decoded image J is calculated based on the complexity analyzed in advance when generating the picture #n of this GOP # j. As a result, the complexity of generating each picture of the edited stream can be obtained by reference or calculation, so that when the edited stream is generated, an optimal code amount can be assigned and two-pass encoding can be performed. . The calculation method of the complexity of the inserted picture generated from the decoded image J and the encoding process using the complexity will be described later.

(2) n + (N−m + 1)> N (see FIG. 5B)
As shown in FIG. 5B, when the total number of pictures included in the edited picture group (n + (N−m + 1))> N, pictures #n from the first picture # 1 to the edit point A of GOP # j (A part of GOP # j) The number of pictures included in 102, the number of pictures included in the picture #m through the last picture #N (part of GOP # k) 103 (N− This is a case where the sum n + (N−m + 1) of m + 1) exceeds N.

  In this case, the editing unit 3 generates ((N−m) + m−1) decoded images J obtained by decoding the picture #n of GOP # j between the editing point A and the editing point B in GOP # j. The encoding processing unit 2 is controlled to insert and generate the re-encoded picture group 111. That is, the editing unit 3 inserts ((N−m) + m−1) copies of the decoded image J of the picture #n of GOP # j between the editing points A and B of the edited picture group. The number of pictures in the coded picture group is 2N.

  By inserting ((N−m) + m−1) decoded images J, a part 102 of GOP # j and a part 103 of GOP # k have the same picture phase as GOP # j and #k, respectively. can do. As a result, for the part 102 of GOP # j and the part 103 of GOP # k having the same picture phase, it is possible to refer to the complexity X of GOP # j and #k. Further, the complexity of the inserted picture is calculated from the complexity of the picture #n of GOP # j as described above.

  Further, in the above-described case, the case where the decoded image J of the picture #n of GOP # j is inserted has been described, but the inserted image may be as follows. That is, not only the decoded image J but also the decoded image K of the picture #m of GOP # k is used as the insertion image. Then, the number of frames is set to N by inserting the inserted image J into the part 102 of GOP # j, and the number of frames is set to N by the inserted image K and the part 103 of GOP # k. A re-encoded picture group may be configured. In this case, the video decoded from the editing point A to the editing point B becomes a still image of the decoded images J and K, and a more natural editing result can be obtained than when only the decoded image J is used.

(3) n + (N−m + 1) = 2N (see FIG. 6A)
As shown in FIG. 6A, when the total number of pictures included in the edited picture group (n + (N−m + 1)) = 2N, the picture # from the first picture # 1 to the edit point A of GOP # j n (a part of GOP # j) 102, the number of pictures n, the number of pictures included in the picture #m through the final picture #N (part of GOP # k) 103 after the editing point B of GOP # k (N This is a case where the sum n + (N−m + 1) = 2N of −m + 1).

  That is, picture #n of GOP # j = picture #N, picture #m of GOP # j = picture # 1, part 102 of GOP # j matches all of GOP # j, and one of GOP # k When the section 103 matches all of GOP # k and the re-encoded picture group (= edited picture group) 121 after editing with the editing points A and B connected has the same phase as the GOP in the first pass It is. In this case, unlike the above (1) and (2), the second-pass encoding can be performed using the complexity of the GOP as it is without inserting an insertion image. In this case, a part 103 = GOP # k of GOP # k may be turned into a closed GOP.

(4) n + (N−m + 1) = N (see FIG. 6B)
As shown in FIG. 6B, the total number of pictures included in the edited picture group (n + (N−m + 1)) = 2N means that the picture # from the first picture # 1 to the edit point A of GOP # j. n (a part of GOP # j) 102, the number of pictures n, the number of pictures included in the picture #m through the final picture #N (part of GOP # k) 103 after the editing point B of GOP # k (N This is the case where the sum n + (N−m + 1) = N.

  Even in this case, the re-encoded picture group (= edited picture group) 131 after editing with the editing points A and B connected has the same picture phase as that of the GOP. The second pass encoding can be performed as it is without performing the insertion process.

(5) When GOP # k is positioned at the head (see FIG. 7A)
As shown in FIG. 7A, for example, the edited stream from the editing point B to the last picture of the original stream in FIG. 4 is used, that is, GOP # k = GOP # 1.

  In this case, the picture #m to the last picture #N (part of GOP # k) 103 after the edit point B of GOP # k are the edit picture group. If this is the first GOP as it is, the picture of GOP # k #M is encoded into an I picture, and if this is the case, the picture phase does not match during the second pass encoding, and the complexity of the original stream cannot be referred to. Therefore, by inserting (m−1) inserted images before the part 103 of GOP # k to obtain the re-encoded picture group 141, the number of pictures is N. In this case, the inserted images to be inserted in order to align the phases can be prepared as, for example, monochrome images M1 to M (m−1). For example, a decoded image K obtained by decoding picture #m of GOP # k may be used as an inserted image. When a monochrome image is inserted for phase matching, the amount of code can be reduced, so that a complexity prepared in advance for a monochrome image can be set.

  Here, the inserted picture obtained by encoding the inserted image is also a picture that does not exist in the original stream, and its complexity has not been analyzed. However, when a monochrome image is encoded to be an inserted picture, the amount of code is extremely large. The number may be small, and the complexity value can be set as appropriate. Further, when the decoded image K is an insertion image, it is calculated from the complexity when the GOP # k picture #m is generated as described above.

(6) When GOP # j is located at the end (see FIG. 7B)
As shown in FIG. 7B, for example, the edited stream from the editing point B to the last picture of the original stream in FIG. 4 is used, that is, GOP # k = GOP # S.

  In this case, the picture #n (part of GOP # j) 102 from the first picture # 1 to the editing point A of GOP # j is an edited picture group, but the number of pictures is n. In this case, as described above, (N−n) images J obtained by decoding the picture #n of the GOP # j are inserted after the part 102 of the GOP # j to form the recoded picture group 151. Thus, the GOP length can be made uniform with the total number of pictures as N.

  However, if GOP # j is the final GOP, the picture phase is aligned without necessarily inserting (N−n) inserted images, and the complexity of the first pass is referenced during the second pass encoding. can do. That is, for example, when picture #n is picture # 1 and an I picture, the complexity up to edit point A can be quoted from the complexity of original stream ST0 without inserting an insertion image, and two-pass encoding is possible. It is. If picture #n is a P picture or a B picture, two-pass encoding is possible if a minimum number of inserted images that can decode picture #n are inserted. In this case, similarly to the above, the complexity up to the editing point A refers to the complexity of the original stream ST0, and the complexity of the inserted picture may be calculated from the complexity of the picture #n of GOP # j.

  As described above, even if editing is performed in the middle of the GOP constituting the original stream, decoding is performed after the editing, and the encoded stream is generated by re-encoding, the picture type is matched for the same frame as the original stream, The picture phase can be aligned. Therefore, since the edited stream is encoded in the same picture type as the original stream, there is no deterioration in image quality. This also means that if the complexity is analyzed when the original stream is generated, the complexity can be referred to when the edited encoded stream is generated. Enables two-pass encoding with an optimal code amount assigned.

  In the above (1), (4), (5), and (6), if the total number of pictures in the re-encoded picture groups 101, 131, 141, and 151 is N, respectively (2), ( In 3), the case where the total number of pictures in the re-encoded picture groups 111 and 121 is 2N has been described, but the total number of pictures in the re-encoded picture group is not limited thereto. That is, if the total number of pictures in the re-encoded picture group is an integer multiple of N, for the editing point A in the edited encoded stream ST1, the frames before and after the editing point in the edited encoded stream ST1 are referred to as the original stream ST0. The picture phase can be matched for the same picture type.

  Next, the two-pass encoding method according to the present embodiment will be described in detail. 8 and 9 are flowcharts showing the second pass encoding method. It is assumed that the storage device 30 connected to the image encoding device 1 stores an original stream whose complexity has been analyzed in advance. It is assumed that the user creates a playlist in which the title is edited and records it in the storage device 40 using two-pass encoding.

  In the present embodiment, a case will be described in which a title that has been edited on the display unit 5 is reproduced and re-encoded and recorded in the storage device 40. However, the title is not reproduced on the display unit 5. It may be recorded. In addition, the original stream ST0 recorded in the storage device 30 is an original encoded stream of the first pass, and the edited encoded stream ST1 recorded in the storage device 40 is encoded and stored by two-pass encoding. This will be explained as an MPEG encoded stream.

  As shown in FIG. 8, first, the image encoding device 1 acquires information about the total number of pictures, the GOP playback time including the editing point, and the picture playback time of the editing point from the edited playlist (step S1). ). After acquiring various information, the edited original stream (title) is displayed on the display unit 5 (step S2). Then, the editing unit 3 determines which of the above-described (1) to (6) the editing point is applied, and controls the decoding processing unit 4 and the encoding processing unit 2 according to the determination result while 2 Causes path encoding to be performed. First, it is determined whether an edit point exists in the first GOP of the playlist (step S3). If there is no edit point in the first GOP, the process shown in FIG. 9 described later is executed.

  On the other hand, when there is an editing point in the first GOP, the editing unit 3 performs the following processing. Here, the case shown in the above (5) and FIG. 7A, that is, the case where the edit point B 1 exists in the first GOP will be described. In this case, under the control of the editing unit 3, the decoding processing unit 4 starts the decoding process from the head GOP, but does not output the GOP until the playback time of the editing point B. That is, the display unit 5 displays an inserted image such as a monochrome image prepared in advance for the reproduced image (inserted image output (video mute control)), and performs a mute process for audio (audio mute control). (Step S4). The encoding processing unit 2 encodes an insertion image such as a monochromatic image output from the decoding processing unit 4 by two-pass encoding, that is, controlling the code amount according to the complexity.

  Up to the editing point B being processed, the inserted image output from the decoding processing unit 4 is encoded into a picture of a predetermined format. In this case, it may be simply encoded so as to have the same picture configuration as that of the GOP of the original stream, but if Nm inserted images are inserted, the picture phases after the editing point B are aligned. Therefore, the picture type for encoding the inserted image may be different from the GOP picture configuration of the original stream. Since this inserted image is an image that does not exist in the original stream ST0, the complexity of the inserted image does not exist.

  Therefore, in the present embodiment, the complexity calculation unit 52 appropriately calculates the complexity of the inserted image. For example, it can be calculated based on the corresponding complexity of the first GOP of the original stream. When the inserted image is a single color image, the code amount may be extremely small, and a complexity or the like prepared in advance may be used. Further, since the inserted image arranged at the head is decoded as an I picture, the complexity thereof may be the complexity of the head picture of the head GOP of the original stream or a fraction thereof. The code amount assigning unit 24 acquires the complexity, determines a target code amount according to the complexity, and sequentially encodes the inserted image with the same picture phase as that of the GOP (step S5).

  Then, when the playback time of the editing point B is reached (step S6: Yes), the decoding processing unit 4 outputs the decoding result after the picture #m of GOP #k (decoded image output (video unmute)) / Audio unmute control) (step S7). As a result, the encoding processing unit 2 receives and encodes the decoded data of the original stream after the editing point B. Since the picture phase is the same as that of the original stream after the editing point B, the complexity calculation unit 52 reads the complexity of the original stream stored in the storage device 30, and the code amount allocation unit 24 reads the complexity. The encoding unit 2 MPEG-encodes the decoded data with this target code amount, and proceeds to step S10 described later.

  Next, if there is no edit point in the first GOP in step S3, the process proceeds to step S8 in FIG. 9, and if there is no edit point in the first GOP of the playlist, the decoding processing unit 4 starts from the first GOP. Start decryption. In the encoding processing unit 2, the complexity calculating unit 52 reads the complexity of each picture of the corresponding GOP of the original stream, the code amount assigning unit 24 calculates and assigns the target code amount, and the encoding unit 21 The decoded data output from the decoding processing unit 4 is MPEG-encoded with this target code amount (step S9).

  In this way, the image decoded by the decoding processing unit 4 is encoded by the encoding processing unit 2 until the reproduction time of the editing point A, and the two-pass encoding is performed while controlling the code amount according to the complexity. When the playback time of the editing point A is reached (step S10: Yes), the decoding processing unit 4 repeatedly decodes the decoded image J of the picture #n of GOP # j decoded immediately before the output. Pause as it is (decode / pause control). At that time, the audio is muted (audio mute control) (step S11).

  When the total number of pictures (n + (N−m + 1)) included in the edited picture group including the editing point A and the editing point B connected to the editing point A is smaller than N (step S12: Yes), that is, In the case of (1) and FIG. 5A, a process of encoding the decoded image J by mn-1 sheets is executed. In this case, the complexity calculating unit 52 of the encoding processing unit 2 calculates complexity Xpr and Xbr, which will be described later, and the code amount assigning unit 24 calculates the target code amount from the complexity Xpr and Xbr, and the encoding unit 21 Performs an encoding process so as to achieve this target code amount (step S13).

  When the encoding unit 21 encodes (m−n−1) decoded images J, that is, when (m−n−1) inserted images J are inserted (step S14: Yes). The editing unit 3 temporarily stops the encoding of the encoding unit 21 (encoding / pause control) (step S15). That is, the editing unit 3 inserts an inserted picture obtained by encoding the image J obtained by decoding the picture #n of the GOP # j between the editing points A and B so that the total number of pictures in the re-encoded picture group is N. The total number of pictures in the re-encoded picture group is set to N by inserting (mn−1) into. As a result, the picture phase up to the edit point A of the re-encoded picture group and the picture phase after the edit point A can be matched with the picture phase of the original stream.

  Next, the editing unit 3 cancels the decoding / pause control and audio mute control of the decoding processing unit 4, and decodes the rest of the GOP # j after the editing point A (step S16). When the decoding processing unit 4 decodes GOP # j to the end, the editing unit 3 then supplies GOP # k including the editing point B that is continuous with the editing point A to the decoding processing unit 4. The decoding processing unit 4 decodes GOP # k including the editing point B (step S17). When the playback time of the editing point B is reached (step S18: Yes), the editing unit 3 releases the encoding / pause control of the encoding processing unit 2 (encoding / resume control). Thereby, the encoding unit 21 starts encoding the decoded image data after the editing point B (step S19).

  Next, when the total number of pictures (n + (N−m + 1)) included in the edited picture group is larger than N (step S12: No, step S20: Yes), that is, in the case of (2) and FIG. In this case, the complexity calculator 52 of the encoding processor 2 calculates later-described complexity Xir, Xpr, and Xbr, and the code amount allocator 24 calculates the target code amount from the complexity complexity Xir, Xpr, and Xb. Then, the encoding processing unit 2 performs an encoding process so as to achieve this target code amount (step S21).

  If the encoding processing unit 2 encodes (N−n + m−1) decoded images J, that is, When (N−n + m−1) inserted pictures obtained by encoding the decoded image J are inserted (step S22: Yes), the processing from step S15 described above is executed. That is, the editing unit 3 performs the encoding / pause control of the encoding processing unit 2, causes the decoding processing unit 4 to decode the remaining GOP # j, further starts decoding from the first picture of GOP # k, and The encoding / resume control of the encoding processing unit 2 is executed at the playback time of point B (steps S15 to S19). As described above, (N−n) pictures encoding the decoded image J may be inserted, and (m−1) pictures encoding the decoded image K may be inserted.

  Next, when the total number of pictures (n + (N−m + 1)) included in the edited picture group is 2N (step S20: No, step S23: Yes), that is, the above (3) and FIG. 6 (a). If the total number of pictures (n + (N−m + 1)) included in the edited picture group is N (step S23: No), that is, in the case of (4) and FIG. Proceed to S25. If the total number of pictures (n + (N−m + 1)) is 2N (step S23: Yes), the editing unit 3 instructs the encoding processing unit 2 to close GOP # k including the editing point B. You may make it GOP.

  As described above, the editing unit 3 appropriately controls the decoding processing unit 4 and the encoding processing unit 2 according to the total number of pictures (n + (N−m + 1)) of the edited picture group to be two-pass encoded. If there is no edit point, the decoding processing unit 4 decodes the GOP specified by the playlist supplied from the editing unit 3, and the encoding processing unit 2 sequentially MPEG-decodes the decoded images. Thus, when the decoding processing unit 4 decodes the final GOP of the playlist and the encoding processing unit 2 encodes it (step S25: Yes), the editing unit 3 ends the encoding of the encoding processing unit 2 (step S25). S26).

  In the present embodiment, as described above, by inserting a picture obtained by encoding a monochrome image or a decoded picture obtained by decoding a picture immediately before or after the editing point as necessary, editing before the editing point A and / or editing is performed. The encoded picture phase after point B can be matched with the picture phase of the original stream. As a result, the complexity analyzed at the time of encoding the original stream is cited, and the target code amount is set to enable two-pass encoding.

  Here, in order to enable two-pass encoding, the decoding processing unit 4 is paused and controlled, and, for example, one or a plurality of decoded images J shown in FIG. 5 are inserted to align the picture phase. Since a picture obtained by encoding an image (decoded image J, K, monochrome image, etc.) is not in the original stream, the complexity cannot be cited. Therefore, in the present embodiment, the complexity of the inserted image is estimated and calculated from the complexity of the original encoded stream.

  Next, a method for calculating the target code amount in the code amount assigning unit of the encoding processing unit 2 and a method for calculating the complexity when encoding an inserted image that does not exist in the original stream will be described. 10 and 11 are flowcharts showing a method for calculating the target code amount using the complexity, and FIG. 12 is a flowchart showing a method for calculating the complexity of the picture to be inserted.

  Let La be the number of frames in which the input decoded image can be analyzed before encoding the frame f. As shown in FIG. 12, the complexity calculator 52 first obtains the total number of edit points, the GOP position including the edit points, and the picture position of the edit points from the playlist (step S31). The code amount assigning unit 24 initializes the frame number f of the input decoded image to -La + 1 (step S32).

  Then, the complexity calculator 52 sequentially reads the GOP complexity X [s, t] in accordance with the playlist (step S33). The complexity of the original stream ST0 is analyzed in advance and stored together with the original stream ST0, for example. Here, the complexity [s, t] indicates the complexity of the picture #t (1 ≦ t ≦ N) of GOP # s (1 ≦ s ≦ S) in the original stream ST0.

  Then, after reading up to the complexity X [s, t] = X [#j, #n] of the picture immediately before the edit point A (step S34: Yes), in this embodiment, the phase is subsequently changed as necessary. An insertion image for aligning is inserted. Therefore, the complexity of the inserted image inserted between the edit points is calculated (step S35). Details of this will be described later with reference to FIG.

  The complexity calculator 52 determines whether or not the input frame f satisfies the frame number La (step S36). When the number of frames of the input image is less than the frame number La (that is, when the frame number f of the image initialized to −La + 1 is f <0), the complexity calculation unit 52 increments the value of f. (Step S38), the complexity of the next image is read out.

  On the other hand, when the number of frames of the input image becomes the same as the frame number La (j = 0), the complexity calculating unit 52 determines whether or not the frame f is a multiple of the unit interval C in which the encoding process is performed. Is determined (step S37).

  If the frame f is not a multiple of the unit interval C on which the encoding process is performed, the complexity calculation unit 52 increments the value of the frame f (step S38), and reads the complexity of the next image.

  On the other hand, when the frame f is an integral multiple of the unit interval C in which the encoding process is performed, the code allocation unit 24 allocates a code amount to the code amount allocation interval C.

  First, the total code amount Ra in the allocation interval is calculated based on the bit rate of the second pass encoding process. At this time, it is also possible to adjust the total code amount in consideration of the buffer occupation amount BOC [f]. (Step S39)

Next, the code amount allocation unit 24 calculates a target code amount of each frame. For the target code amount T [f] of each frame, the total code amount Ra [f] that can be allocated to the code amount allocation section is proportionally distributed with the complexity X [s, t].
T [f] = (X [s, t] / Xsum) * R [f]
Can be calculated. Here, Xsum represents the total sum of the complexity X [s, t] of the allocation section. The target code amount T [f] is calculated for each of the frame f to the frame f + L−1 (step S41).

Subsequently, the code amount allocation unit 24 calculates the buffer occupation amount in the encoding buffer 22 of the allocated target code amount (step S41). For example, the buffer occupancy is BOC [f]
BOC [f] = BOC [f−1] + T [f] −Rframe
And can be calculated. Here, Rframe indicates the code amount per frame calculated from the bit rate R used in the main encoding. In addition, the buffer occupancy initial value BOC [0] = 0 is set.

  Then, the code amount allocation unit 24 determines whether or not the encoding buffer 22 causes an overflow or underflow based on the calculated buffer occupancy BOC [f]. For example, when the upper limit of the encoding buffer is B, it is determined whether or not the buffer occupation amount BOC [j] is smaller than B-Rframe.

  If the encoding buffer has underflowed (step S42: Yes), the code amount allocation unit 24 adjusts the code amount so that the encoding buffer 22 does not cause an underflow (step S43). For example, the frame fu that minimizes the code occupation amount of the encoding buffer 22 is detected, and the allocation of the codes allocated from the frames f to fu is increased so that the encoding buffer does not underflow at fu. Then, the increased code amount is reduced from the code amount allocated to the frames from fu + 1 to f + L−1.

  When the encoding buffer has overflowed (step S44: Yes), the code amount allocation unit 24 adjusts the code amount so that the encoding buffer 22 does not overflow. For example, the frame fo with the largest code buffer occupation amount is detected, and the code amount allocated from frames f to fo is reduced, so that the code buffer allocation is reduced so that the buffer does not overflow at fo. Let The reduced code amount is allocated to frames from fo + 1 to f + L−1 (step S45).

  In addition, when the allocation is an appropriate allocation that does not cause any overflow or underflow in the encoding buffer 22 (step S42: No, step S44: No), the encoding unit 21 encodes the allocation section C. This is performed (step S46). Then, the process proceeds to step S38, the value of the frame f is incremented (step S38), and the complexity calculation unit 52 reads the complexity of the next image and repeats the above-described processing.

  Next, a process for calculating the complexity of the inserted image J will be described. As shown in FIG. 12, first, it is determined whether or not the total number n + (N−m + 1) of re-encoded pictures is smaller than N (step S51). If it is smaller, s = j and t = n + 1 are set (step S52), and the complexity X [s, t] is sequentially calculated until t = m-1 is reached (step S53) (step S54). .

  Here, the period from t = n to m−1 is a process of encoding the same decoded image J. That is, in this case, as described above, at the editing point A, the decoded image J is displayed as a pause, and a new image that does not exist in the original stream of 1-pass encoding is inserted to encode this. become. For this reason, the complexity of the image is not required in the first pass encoding process. Therefore, the target code amount for each picture cannot be calculated as it is.

By the way, this inserted image is a decoded image J at t = n. Therefore, in the present embodiment, calculation is performed based on the complexity X [#j, #n] of the decoded image J according to the picture type for encoding the decoded image J. That is, if you want to encode a P picture,
Complexity Xpr = X [#j, #n] / Dp
And if it is encoded into a B picture:
Complexity Xbr = X [#j, #n] / Db
Calculate as
Here, Dp is a constant for calculating the complexity when encoding into a P picture, and Db is a constant for calculating the complexity when encoding into a B picture.

  Here, 0 <Dp ≦ Db, for example, Xpr = X [#j, #n] / 3, Xbr = X [#j, #n] / 10, or the like. For Dp and Db, if there is a repeated portion of the same picture during one-pass encoding, the complexity thereof may be referred to. The (mn-1) pictures to be inserted are inserted in order to align the phases after the editing point B, and the (mn-1) pictures are not necessarily the original stream. Need not be the same as t = n + 1 to m−1 picture types. If it is desired to increase the amount of code allocated before edit point A or after edit point B, the B picture may be increased from the original stream to reduce the complexity of the inserted picture.

  Here, the inserted image is described as the decoded image J at t = n, but as described above, the decoded image K at s = k and t = m can also be used as the inserted image. That is, the complexity of the inserted image can be calculated in the same manner as described above based on the complexity X [#k, #m] of the decoded image K obtained by decoding the picture #m of GOP # k.

  Further, the values of Dp and Db can be appropriately adjusted according to the picture type in the original stream of the decoded image J or the decoded image K. For example, when the decoded image J or the decoded image K is an I picture in the original stream ST0, Dp and Db may be increased, and when it is a B picture, Dp and Db may be relatively decreased. . That is, in the above example, Dp = 3 and Db = 10 are described. However, according to the picture type of the decoded image J or the decoded image K, Dp = 1/3, Db = 1, etc. are set, and the decoded image J or It is also possible to set the complexity equal to or higher than the complexity at the time of encoding the decoded image K.

  Then, t is sequentially incremented (step S55), and when t = m, the total number of inserted pictures = mn−1 (sheets), that is, the number of frames of the inserted image (mn−1) and f are increased. Thus, the frame f = f + (mn-1) is set (step S56), and the process proceeds to step S36 in FIG.

  When the total number n + (N−m + 1) of re-encoded pictures is larger than N and smaller than 2N (step S57), first, s = j and t = n + 1 are set (step S58), and t = N. (Step S59), the complexity levels Xpr and Xbr are calculated in the same manner as in step S54 described above while incrementing t (steps S60 and S61).

  When t is greater than N, s = k and t = 1 are set (step S62). Until t reaches m (step S63), the complexity X [s, t] is incremented while incrementing t. Calculate (steps S64 and S65). Here, since the picture arranged at s = k and t = 1 is the first picture of the GOP, it is set as an I picture. This I picture is also a still picture of the decoded picture J, but since it is an I picture, it is necessary to allocate a larger amount of code than other P pictures and B pictures. Therefore, the complexity X [#k, # 1] at s = k, t = 1 as an I picture refers to the complexity X [#k, # 1] of the original stream ST0 as it is. P pictures and B pictures after t = 1 are calculated as complexity Xpr = X [#j, #n] / Dp and complexity Xbr = X [#j, #n] / Db, as described above. When t = m is reached in this way, the total number of inserted pictures = (N−n) + m−1 (sheets), that is, (N−n) + m−1, f are increased by the number of frames of the inserted image, and the frame f = f + ( N−n) + m−1 (step S66), and the process proceeds to step S36 in FIG.

  The timing of the processing in FIGS. 10 to 12 is the target code amount prior to encoding of each frame (decoded image) in the 2-pass encoding shown in FIGS. 8 and 9 in the encoding processing unit 2. Can be calculated.

  According to the present embodiment, in the edited title (playlist) edited from the original stream in units of frames (pictures), the picture phase before and after the editing point can be matched with the picture phase of the original stream ST0. Therefore, even if the bit rate is lowered and re-encoding is performed, the degradation of image quality can be minimized. Furthermore, if the complexity is obtained by analysis when the original stream is encoded, it can be referred to, and the target code amount is calculated based on this complexity and the edited encoding is performed by two-pass encoding. Stream ST1 can be obtained. Therefore, for example, even if a high-bit-rate original stream recorded on a HDD with a large storage capacity is edited on a picture-by-picture basis, a DVD with a small storage capacity while minimizing image quality degradation by two-pass encoding. It is possible to generate an edited encoded stream ST1 for recording (dubbing).

  That is, between the edit points, the picture phase straddling the GOP boundary can be maintained at the edit point by encoding the decoded image immediately before the edit point and inserting a desired frame (inserted image). Furthermore, since the inserted frame is a decoded image immediately before the editing point in the pause display, the complexity X when encoding the decoded image is set to 1 / Dp or Db of the complexity obtained in the original stream. It becomes possible to decide. With the above processing, the complexity of each picture when the edited encoded stream ST1 is generated can be obtained, so that two-pass encoding processing is possible.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. For example, in the above-described embodiment, arbitrary processing in each block shown in FIGS. 1 to 3 can be realized by causing a CPU (Central Processing Unit) to execute a computer program. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium.

2 Encoding processing unit 3 Editing unit 4 Decoding processing unit 5 Display units 6 and 7 Storage I / F
21 encoding unit 22 encoding buffer 23 analysis unit 24 code allocation unit 25 code amount control unit 26 pause / resume control unit 30, 40 storage device 51 feature amount observation unit 52 complexity calculation unit 61 decoding unit 62 decoding buffer 63 pause / Resume control unit

Claims (4)

An encoding processing unit that outputs encoded data obtained by encoding each of the input frames;
A decoding processor that decodes the encoded data and outputs decoded data including each of the frames to be reproduced;
Have
When editing is performed to change the playback mode of each of the frames to be played back included in the decoded data,
The decoding processing unit outputs edited decoded data obtained by inserting a frame corresponding to the editing into each of the frames to be reproduced included in the decoded data,
The encoding processing unit edits each of the input frames and a frame corresponding to each of the input frames included in the edited decoded data so as to be encoded with the same picture type. An image processing apparatus, wherein the decoded data is encoded. The editing is performed such that the first frame of the frames to be played back included in the decoded data is connected to the second frame having a playback order that is two or more later than the one frame. in case of,
The image processing apparatus according to claim 1, wherein the decoding processing unit inserts a predetermined number of frames between the first frame and the second frame. In the case where the editing is performed in which the third frame, which is a frame subsequent to the first frame in the frame to be reproduced included in the decoded data, is changed to the first frame,
The image processing apparatus according to claim 1, wherein the decoding processing unit inserts a predetermined number of frames before the third frame. In the case where the editing is performed in which the fourth frame, which is the previous frame in the playback order of the last frame included in the decoded data included in the decoded data, is changed to the last frame,
The image processing apparatus according to claim 1, wherein the decoding processing unit inserts a predetermined number of frames after the fourth frame.

Images