JP2020108032A - Video code stream editing device and program - Google Patents

Abstract

To reduce deterioration of quality when partially editing a video code stream.SOLUTION: A code stream editing device 1 includes a decoder 12 that generates a pre-replacement video by decoding a GOP including a replacement section in a pre-replacement stream, a video replacement unit 13 that replaces a video in the replacement section of the video before replacement with the replacement video to generate a video after replacement, a video clipping unit 14 that generates a clipped video obtained by excluding frames that do not refer to frames after a start point frame during encoding, which are frames positioned before the start point frame of the replacement section in the display order in the post-replacement video, an encoder 15 that encodes the clipped video and generates a replacement stream, and a partial replacement unit 16 that replaces the section corresponding to the clipped video of the pre-replacement stream with the replacement stream to generate a post-replacement stream.SELECTED DRAWING: Figure 1

Description

The present invention relates to a video coded stream editing device and a program for editing a video coded stream.

At the broadcasting station, the completed program that can be broadcast as a program as it is is AVC (Advanced Video Coding)/H. File management is often performed in the form of a video coded stream coded using a video coding method such as H.264 (see Non-Patent Document 1, for example). The file of the completed program is recorded in the transmission server and waits for its transmission time (broadcast time).

When a portion requiring correction is found in the completed program recorded in the sending server, partial replacement editing is performed to replace only the portion requiring correction with the correct content. Partial replacement editing is often performed when there is a problem with a specific cut and only this cut is replaced, or when only the frame to which this telop is added due to an error in the text of the telop is corrected.

In the system disclosed in Non-Patent Document 1, as shown in Table 1, the completed program is AVC/H. In the H.264 video coding system, a video coded stream is composed of only I pictures coded using only intra-frame prediction. Since this video coded stream is composed of only I pictures that can be decoded by itself, it is not necessary to once decode and code the video coded stream when performing partial replacement editing. Therefore, it is possible to easily perform partial replacement editing of a section having an arbitrary frame as a start point and an end point.

As described above, the system disclosed in Non-Patent Document 1 adopts the structure (Intra Only) of only I pictures for ease of editing. However, although the Intra Only is easy to edit, the coding efficiency is low. On the other hand, the structure (Long GOP (Group of Pictures)) using a P picture encoded using inter-frame prediction by unidirectional prediction and a B picture encoded using inter-frame prediction by bidirectional prediction is Although it is difficult to edit, it has an advantage of high coding efficiency because the coding is performed by utilizing the correlation between frames.

The system disclosed in Non-Patent Document 1 targets HD (High Definition) video and is intended for AVC/H. It is operated at H.264, Intra Only, and 50 Mbps, but when targeting 8K Super Hi-Vision video, which is expected to spread in the future, a coding method with higher coding efficiency due to an increase in the amount of video signal information. , GOP structure is required.

In FIG. 6, an 8K Super Hi-Vision image is displayed in HEVC (High Efficiency Video Coding)/H. 265 shows a graph of bit rate and PSNR (Peak Signal Noise Ratio) when encoded with Intra Only and Long GOP. Note that the higher the PSNR value, the less the image quality deterioration. The image to be encoded used is Kaede (Fig. 6(a)) and diesel train (Fig. 6(a)), which is a test chart of the ultra-high definition/wide color gamut standard moving image-A series provided by the Institute of Image Information and Television Engineers (ITE). It is FIG.6(b)). The GOP structure of the Long GOP was M=4, N=32, and Temporal_ID=2. From this figure, an 8K Super Hi-Vision image can be recorded in HEVC/H. When encoding with H.265, it can be confirmed that Long GOP is advantageous over Intra Only.

Further, in Patent Document 1, a first partial stream from the MPEG video stream 1 to immediately before an I or P picture is cut out, a second partial stream after an I or P picture is cut out from the MPEG video stream 2, and a first partial stream is cut out. And a second partial stream are combined to generate the edited stream 3. More specifically, when the picture displayed immediately before the second partial stream is an I picture, the picture is an intra-frame coded picture. When the picture displayed immediately before the second partial stream is not the I picture, the unidirectional inter-frame predictive coded pictures from the I picture immediately before the picture to the picture are sequentially decoded, The decoded image of the picture is obtained, and the picture re-encoded by the intra-frame encoding is the intra-frame encoded picture. Finally, the intra-frame coded picture is inserted between the first partial stream and the second partial stream, and the concatenation process is performed.

JP, 2002-300258, A

“Introduction of file-based production equipment”, Niimi et al., Journal of Image Media Society, Vol.67, No. 5, pp.374-378, 2013

In the system of Non-Patent Document 1 for HD video, the completed program is encoded by Intra Only. However, when targeting large-capacity video such as 8K Super Hi-Vision video, Long is used from the viewpoint of coding efficiency. Encoding with GOP is required.

However, since P picture and B picture cannot be independently decoded, when editing a section that is not a GOP unit, the coded stream after editing cannot be correctly decoded. The reason for this will be described with reference to FIG. I, B, and P in the figure mean I picture, B picture, and P picture, respectively. When the GOP structure of the pre-replacement stream, which is the video encoded stream before partial replacement and editing, is as shown in FIG. 4A, since the second frame to the fourth frame are B pictures, decoding is performed. Refers to the first I-frame, which is the front I-picture, and the fifth P-frame, which is the rear P-picture. Since the 10th frame to the 12th frame are B pictures, when decoding, the 9th frame which is the front P picture and the 13th frame which is the rear P picture are referred to. Since the 13th frame is a P picture, the 9th frame, which is the preceding P picture, is referred to when decoding. Since the 14th frame to the 16th frame are B pictures, when decoding, the 13th frame which is the front P picture and the rear P picture or I picture are referred to.

FIG. 4B shows frames (P#5, B#6, B) from the fifth frame to the ninth frame in the display order with respect to the stream before replacement, which is the video encoded stream before partial replacement editing. #7, B#8, P#9) is partially replaced with the replacement stream (I#1', B#2', B#3', B#4', P#5'). ing. At this time, the second to fourth frames (B#2, B#3, B#4) of the post-substitution stream, which is the video encoded stream after the partial substituting edit, are the frames to be referred to P Since #5 does not exist, correct decoding cannot be performed. Similarly, from the 10th frame to the 13th frame (B#10, B#11, B#12, P#13) of the stream after replacement, there is no P#9 which is a frame to be referred to, and therefore, it is correct. Can't decrypt. Further, since the 13th frame (P#13) cannot be correctly decoded, the 14th frame to the 16th frame (B#14, B#15, B#16) with the 13th frame as the reference frame Even, it cannot be decrypted correctly.

Therefore, a method of making it possible to correctly decode the encoded stream after editing will be described with reference to FIG. FIG. 5 shows frames (P#5, B#6, B#7, P#5, B#6, B#7, P) from the fifth frame to the ninth frame in the display order with respect to the pre-replacement stream which is a video encoded stream before partial replacement editing. It shows a case where the B#8, P#9) is partially replaced by the replacement stream (I#1', B#2', B#3', B#4', P#5'). First, a GOP including a partial replacement section is decoded in GOP units to generate a pre-replacement video. After that, the pre-replacement video is edited, and the fifth to ninth frames (#5, #6, #7, #8, #9) are replaced with the replacement video (#1', #2', #3', #4', #5') to generate the post-replacement image. When the data to be replaced is given as a video encoded stream (replacement stream), it is necessary to decode the replacement stream as shown in FIG. 5 to generate a replacement video. According to this method, the video after replacement can be correctly encoded. However, the quality of the post-replacement stream is further deteriorated because the pre-replacement stream that has already undergone coding degradation is decoded and re-encoded.

Further, in the technique disclosed in Patent Document 1, the video coded stream can be cut out and concatenated by generating the intra-frame coded picture described above, but partial replacement editing of the video coded stream is performed. There was a problem that I could not.

An object of the present invention made in view of such circumstances is to provide a video coded stream editing device and a program capable of partially editing an arbitrary section of a video coded stream and reducing deterioration of quality due to partial editing. To provide.

In order to solve the above problems, a video coded stream editing device according to the present invention performs partial replacement editing on a video coded stream coded using interframe prediction, and a video before partial replacement editing is performed. A video coded stream editing apparatus for generating a post-replacement stream, which is a video coded stream after partial substitution editing, from a pre-replacement stream, which is a coded stream, wherein a replacement section of the pre-replacement stream A decoder that decodes a GOP including the following to generate a pre-replacement video, and a video replacement unit that replaces a video in the replacement section of the pre-replacement video with a replacement video to generate a post-replacement video, A cutout image is generated by excluding a frame that is positioned before the start point frame of the replacement section in the display order in the post-replacement video and does not refer to a frame after the start point frame at the time of encoding. A video cutout unit, an encoder that encodes the cutout video to generate a replacement stream, and a section corresponding to the cutout video of the pre-replacement stream is replaced with the replacement stream, and the replacement is performed. And a partial replacement unit that generates a subsequent stream.

Further, in the video coded stream editing device according to the present invention, the video cutout unit identifies a picture type of the start point frame of the replacement section of a GOP including the replacement section, and the picture type is I. When the picture is a picture, the cutout video is from the start point frame to the last frame in the post-replacement video, and when the picture type is not an I picture, the display order of the post-replacement video is displayed. Then, the frame from the frame immediately after the P picture or I picture located immediately before the start point frame to the final frame is set as the cutout image.

Further, in order to solve the above problems, a program according to the present invention causes a computer to function as the above-mentioned video coded stream editing device.

According to the present invention, it is possible to partially edit a section having an arbitrary frame of a video coded stream as a start point and an end point. Further, it is possible to reduce the quality deterioration when the video coded stream is partially edited.

It is a block diagram which shows the structural example of the video coding stream edit apparatus which concerns on one Embodiment of this invention. 6 is a flowchart showing an operation example of the video coded stream editing device according to the embodiment of the present invention. It is a figure explaining an example of processing of a video coding stream editing device concerning one embodiment of the present invention. It is a figure explaining the example which cannot correctly decode the encoding stream after edit. It is a figure explaining the method of making the coded stream after edit correctly decodable. It is a graph which shows PSNR with respect to a bit rate at the time of encoding 8K super high-definition video.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows a configuration example of a video coded stream editing device 1 according to an embodiment of the present invention. The video coded stream editing device 1 shown in FIG. 1 includes a replacement position designation unit 11, a decoder 12, a video replacement unit 13, a video cutout unit 14, an encoder 15, and a partial replacement unit 16. Equipped with.

The video coded stream editing device 1 is a device that performs partial replacement editing for replacing a part of a video coded stream coded using interframe prediction with another stream. In the following description, the video coded stream before partial replacement editing is referred to as “pre-replacement stream”, and the video coded stream after partial replacement editing is referred to as “post-replacement stream”. Since the video coded stream is coded using inter-frame prediction, the GOP structure is Long GOP. However, the present invention targets Closed GOP and does not target Open GOP.

The sending server 20 sends the pre-replacement stream to the decoder 12, the partial replacement unit 16, and the video cutout unit 14 of the video coded stream editing apparatus 1. Further, the transmission server 20 receives the post-substitution stream from the partial substitution unit 16.

When the operator of the video coded stream editing device 1 specifies the start point and the end point of the partial replacement edit, the replacement position specifying unit 11 generates start point/end point information indicating the start point and the end point of the partial replacement edit. .. Then, the replacement position designation unit 11 outputs the start point/end point information to the decoder 12, the video replacement unit 13, the video cutout unit 14, and the partial replacement unit 16. The start point and the end point can be specified using, for example, a time code in the stream or a frame number in the display order. Further, any two frames of the video coded stream can be designated as the start point and the end point, respectively, and the replacement section set at the start point and the end point need not be in GOP units.

The decoder 12 includes a GOP including a start point and an end point in the pre-replacement stream input from the transmission server 20, based on the start point/end point information input from the replacement position specifying unit 11, that is, includes a replacement section. A GOP and may be a plurality of GOPs) are decoded to generate a pre-replacement image. Then, the decoder 12 outputs the generated pre-replacement video to the video replacement unit 13.

The video replacement unit 13 acquires the replacement video from an editing device, a recording/playback device, or the like that is a peripheral device of the video coded stream editing device 1. The video replacement unit 13 also acquires the pre-replacement video from the decoder 12. Then, the video replacement unit 13 replaces the video from the start point to the end point (replacement section) of the pre-replacement video with the replacement video based on the start point/end point information input from the replacement position designation unit 11. A video after replacement is generated. The video replacement unit 13 outputs the generated post-replacement video to the video cutout unit 14.

The video cutout unit 14 is a frame that is positioned before the start point frame of the replacement section in the display order in the post-replacement video input from the video replacement unit 13, and is a frame located after the start point frame at the time of encoding. A cutout image is generated by excluding frames that do not refer to frames. Then, the video cutout unit 14 outputs the generated cutout video to the encoder 15.

The encoder 15 encodes the cutout video input from the video cutout unit 14 to generate a replacement stream. Then, the encoder 15 outputs the generated replacement stream to the partial replacement unit 16. The first frame in the decoding order of the replacement stream is the I picture.

The partial replacement unit 16 replaces the section corresponding to the cut-out video of the pre-replacement stream received from the transmission server 20 with the replacement stream input from the encoder 15 to generate the post-replacement stream. Then, the partial replacement unit 16 transmits the generated post-replacement stream to the transmission server 20.

FIG. 2 is a flowchart showing an operation example of the video coded stream editing device 1. FIG. 3 is a diagram illustrating a specific example of the process of the video coded stream editing device 1. The operation of the video coded stream editing apparatus 1 will be described with reference to FIGS. 2 and 3. In addition, in FIG. 3, it is assumed that the frames are counted in the display order. That is, the frame number in FIG. 3 means a POC (Picture Order Count) which is a display order number.

In the video coded stream editing device 1, the replacement position specifying unit 11 specifies the replacement position (step S101). In the example shown in FIG. 3, partial replacement editing of the 7th frame to the 10th frame of the nth GOP is performed on the pre-replacement stream received from the transmission server 20. In this case, the replacement position specifying unit 11 generates information indicating the 7th frame and the 10th frame of the nth GOP as the start point/end point information.

Next, the video coded stream editing apparatus 1 decodes the GOP including the replacement section in the pre-replacement stream by the decoder 12 to generate the pre-replacement video (step S102). In the example shown in FIG. 3, the decoder 12 decodes the nth GOP to generate a pre-replacement image.

Next, in the video coded stream editing apparatus 1, the video replacement unit 13 acquires the replacement video, replaces the video in the replacement section of the pre-replacement video with the replacement video, and generates the post-replacement video. Yes (step S103). In the example shown in FIG. 3, the image replacement unit 13 acquires #1', #2', #3', #4' as replacement images.

Next, the video coded stream editing device 1 confirms the GOP structure of the GOP including the replacement section by the video cutout unit 14, and identifies the picture type of the start point frame of the replacement section. When the picture type of the start point frame of the replacement section is an I picture (step S104-Yes), the video cutout unit 14 determines the cutout video from the start point frame to the final frame of the post-replacement video. (Step S105).

On the other hand, when the picture type of the start point frame of the replacement section is the P picture or the B picture (step S104-No), the video cutout unit 14 displays the pre-replacement stream of the post-replacement video in the display order. The frame position (POC) one frame after the P picture or I picture located immediately before the start point frame is specified in (POC order). Then, among the images after the replacement, the frames from the frame corresponding to the previously specified frame position (POC) to the final frame are set as the cut-out images (step S106).

In the example shown in FIG. 3, since the picture type of the start point frame (B#7) of the replacement section is B picture, the video cutout unit 14 is positioned at the position P immediately before the start point frame (B#7). The frame position (sixth frame) after the picture (P#5) is specified. Then, among the images after replacement, the sixth frame (#6) to the final frame (#12) are set as cutout images.

Next, the video coded stream editing device 1 uses the encoder 15 to code the cut video and generate a replacement stream (step S107). At that time, as shown in FIG. 3, the first frame in the decoding order of the replacement stream is the I picture. The last frame in the display order of the replacement stream is a P picture or an I picture. Other picture types of the replacement stream are arbitrary, but since the coding efficiency decreases as the number of I pictures increases, it is preferable to use B pictures or P pictures.

Next, in the video coded stream editing device 1, the partial replacement unit 16 replaces the section corresponding to the cutout video of the stream before replacement with the replacement stream to generate a post-replacement stream (step S108). In the example shown in FIG. 3, for the nGOP of the stream after replacement, the first 5 frames (I#1, B#2, B#3, B#4, P#5) and the last 4 frames (I# 13, B#14, B#15, B#16) are the same as the stream before replacement. Therefore, regarding these frames, quality deterioration due to re-encoding does not occur.

The video coded stream editing device 1 has been described above, but a computer may be used to cause the video coded stream editing device 1 to function. Such a computer stores a program describing processing contents for realizing each function of the video coded stream editing apparatus 1 in a storage unit of the computer, and causes the CPU of the computer to read and execute the program. Can be achieved with.

Further, this program may be recorded in a computer-readable medium. The computer readable medium can be used for installation on a computer. Here, the computer-readable medium in which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.

As described above, in the present invention, when partial replacement editing is performed on a coded stream having a GOP structure of Long GOP, among the frames positioned before the start point frame in the display order, the frames after the start point frame are displayed. For the frame that does not refer to the frame, the original video coded stream before editing is used as it is. Therefore, according to the present invention, it is possible to reduce the image quality deterioration due to re-encoding even when the video encoded stream is partially edited.

Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, it is possible to combine a plurality of configuration blocks described in the configuration diagram of the embodiment into one or to divide one configuration block.

DESCRIPTION OF SYMBOLS 1 Video coded stream editing apparatus 11 Replacement position designation section 12 Decoder 13 Video replacement section 14 Video cutout section 15 Encoder 16 Partial replacement section 20 Sending server

Claims (3)

Partial replacement editing is performed on a video coded stream encoded using interframe prediction, and a video code after partial replacement editing is performed from a pre-replacement stream that is a video coding stream before partial replacement editing. A video coded stream editing device for generating a post-replacement stream that is a coded stream,
A decoder that decodes a GOP including a replacement section of the pre-replacement stream to generate a pre-replacement video;
A video replacement unit that replaces the video in the replacement section of the video before replacement with a replacement video to generate a video after replacement,
In the post-substitution video, a cut-out video is generated that excludes frames that are located in the display order before the start point frame of the replacement section and that do not refer to frames after the start point frame during encoding. And the video cutout part
An encoder that encodes the cutout video to generate a replacement stream,
A partial replacement unit that replaces a section corresponding to the cutout image of the pre-replacement stream with the replacement stream to generate the post-replacement stream;
A video coded stream editing device, comprising: The video cutout unit identifies a picture type of the start point frame of the replacement section of a GOP including the replacement section,
When the picture type is an I picture, the cutout video is the video from the start point frame to the final frame in the post-replacement video,
If the picture type is not an I-picture, in the post-replacement video, the P-picture or the I-picture located immediately before the start-point frame in the display order and the frame immediately after the first picture to the last frame are switched to the cut-off picture. The video coded stream editing device according to claim 1, wherein the video coded stream editing device is an outgoing video. A program for causing a computer to function as the video coded stream editing device according to claim 1.

Images