JP2024027758A - Multi-layer encoding device, single-layer encoding device, and program - Google Patents

Abstract

An object of the present invention is to suppress an increase in development costs when using multi-layer encoding. A multi-layer encoding device 1 that performs multi-layer encoding includes a first single-layer encoding unit 10 that outputs a bitstream by encoding an input video signal; and a second single layer encoding section 20 provided at a subsequent stage. The second single layer encoding unit 20 outputs the bitstream obtained by the first single layer encoding unit 10 as a base layer bitstream, and decodes the bitstream obtained by the first single layer encoding unit 10. The input video signal is encoded using the parameters derived through the processing, and an enhancement layer bitstream is output. [Selection diagram] Figure 1

Description

The present invention relates to a multilayer encoding device, a single layer encoding device, and a program.

Broadcasting video transmission has basically used a mode in which only one video can be transmitted per channel. For example, in a coding method called HEVC (High Efficiency Video Coding) used in current 4K8K satellite broadcasting, video is transmitted using a coding tool group called Main 10 profile. Furthermore, in HEVC, there is a tool group called Scalable Main 10 profile that transmits multiple videos in layers. This is an encoding mechanism in which an additional video can be superimposed as an upper layer (enhancement layer) while transmitting a video corresponding to the Main10 profile as a base layer. Similarly, in VVC (Versatile Video Coding), which is the latest coding method, in addition to the Main 10 profile, a Multilayer Main 10 profile that realizes video transmission of multiple layers has been developed.

In such a multilayer encoding technique, the base layer is used as a basic stream, and the differential video thereof is transmitted as an enhancement layer. Multi-layer encoding is sometimes referred to as layered encoding or scalable encoding. On the decoding side, in addition to decoding the base layer alone, it is also possible to decode the base layer and enhancement layer in combination. According to multi-layer encoding, for example, decoding using bitstreams of multiple layers produces a high-resolution or high-quality video, whereas decoding using bitstreams of only some layers (base layer) Even after decoding, low resolution or low quality video can be played.

A first application example of multi-layer encoding is SNR scalable, which realizes video services of different quality (low quality, high quality) by transmitting an enhancement layer for improving quality with respect to a base layer. A second application example is spatial scalability, in which an enhancement layer (high resolution video) for improving resolution is transmitted to a base layer (low resolution video). Spatial scalability makes it possible to efficiently transmit broadcasting services of different resolutions simultaneously, such as a set of 2K video and 4K video, or a set of 4K video and 8K video.

Patent Document 1 describes a technology that applies multi-layer encoding (in particular, SNR scalable) to realize a video overlay service. Specifically, the main video is encoded as a base layer, and a video in which additional information such as subtitles and explanatory video (i.e. sub-video) is superimposed on the main video is encoded as an enhancement layer. Send each bitstream. According to such a method, a receiving device that receives only the base layer can play only the main video, while a receiving device that receives the base layer and enhancement layer can play the main video by decoding only the base layer. By decoding both the base layer and the enhancement layer, the main video with sub-video can be played, and the receiving device can select the service it wants to receive.

Japanese Patent Application Publication No. 2022-53534

In multi-layer encoding, efficient encoding is achieved by using one already encoded video (base layer: BL) for signal prediction of the other (enhancement layer: EL). Here, the number of ELs is not limited to one, and there may be a plurality of ELs with different resolutions or different qualities. Multi-layer encoding uses signal correlation between layers and is more efficient than individual encoding, but on the other hand, a single encoding device can process images of multiple resolutions or multiple qualities. must have the ability to encode

For example, when realizing a real-time encoding device, in the case of two layers consisting of one BL and one EL, a multilayer encoding device must encode two input video signals per unit time, and a non-multilayer encoding device must encode two input video signals per unit time. Double the processing power is required for encoding, increasing development costs. Thus, in multi-layer encoding, there is a problem in that a multi-layer encoding device must be developed with a degree of complexity corresponding to the number of multi-layers.

Therefore, an object of the present invention is to provide a multi-layer encoding device, a single-layer encoding device, and a program that make it possible to suppress an increase in development costs when using multi-layer encoding.

The multi-layer encoding device according to the first aspect is a device that performs multi-layer encoding. The multi-layer encoding device includes a first single-layer encoding device that outputs a bitstream by encoding an input video signal, and a second single-layer encoding device provided after the first single-layer encoding device. and means for converting. The second single layer encoding means outputs the bitstream obtained by the first single layer encoding means as a base layer bitstream, and outputs the bitstream obtained by the first single layer encoding means. The input video signal is encoded using parameters derived by performing decoding processing on the input video signal, and an enhancement layer bit stream is output.

The program according to the second aspect causes a computer to function as the multilayer encoding device according to the first aspect.

A single layer encoding device according to a third aspect includes: a decoding processing means that performs a decoding process on an inputted bitstream; a bitstream generation means that outputs a bitstream by encoding an input video signal; a switch means for switching a mode of enabling or not inputting a stream; when the bitstream input is disabled, the bitstream generating means generates a single layer without performing the decoding process; The device operates to perform encoding, and when the input of the bit stream is enabled, operates to perform multi-layer encoding by a serially connected configuration of a plurality of single layer encoding devices including the own single layer encoding device. do.

The program according to the fourth aspect causes a computer to function as the single layer encoding device according to the third aspect.

According to the present invention, it is possible to provide a multi-layer encoding device, a single-layer encoding device, and a program that make it possible to suppress an increase in development costs when using multi-layer encoding.

FIG. 1 is a diagram showing the configuration of a multilayer encoding device according to a first embodiment. FIG. 3 is a diagram showing the configuration of a second single layer encoding section according to the first embodiment. FIG. 3 is a diagram showing the configuration of a multilayer encoding device according to a modification of the first embodiment. FIG. 3 is a diagram showing the configuration of a multilayer encoding device according to a second embodiment. It is a figure showing the composition of the 2nd single layer encoding part concerning a 2nd embodiment. It is a figure which shows the structure of the multilayer encoding apparatus based on the example of a change of 2nd Embodiment. FIG. 3 is a diagram showing the configuration of a multilayer encoding device according to a third embodiment. FIG. 3 is a diagram showing the configuration of a multilayer encoding device according to a third embodiment.

Embodiments will be described with reference to the drawings. In the description of the drawings below, the same or similar parts are designated by the same or similar symbols.

(First embodiment)
FIG. 1 is a diagram showing the configuration of a multilayer encoding device 1 according to the first embodiment.

The multi-layer encoding device 1 according to the first embodiment is a device that performs multi-layer encoding. Specifically, the multilayer encoding device 1 converts a base layer (BL) bitstream (hereinafter referred to as a "BL stream") and an enhancement layer (EL) bitstream from an input video signal by multilayer encoding. (hereinafter referred to as an "EL stream") and outputs an output bitstream composed of a BL stream and an EL stream. Here, the multi-layer encoding device 1 may be configured to output a single stream including a BL stream and an EL stream, or may output each of the BL stream and n EL streams as a plurality of streams via separate routes. It is also possible to output the data as a stream. Therefore, the term "output bitstream" includes both aspects. In addition, in the case of the latter aspect, it is not always necessary to provide a stream synthesis section, which will be described later.

The multilayer encoding device 1 is configured to include a plurality of single layer encoding sections connected in series. The single layer encoding unit at the first stage generates a BL stream, and each single layer encoding unit from the second stage onward generates an EL stream. An example in which the multilayer encoding device 1 generates and outputs one EL stream will be described below, but the multilayer encoding device 1 may also be configured to generate and output two or more EL streams. good.

In this way, by configuring the multi-layer encoding device 1 with a plurality of single-layer encoding sections connected in series, each single-layer encoding section only needs to generate a single layer bitstream, and each The processing power and complexity of the single layer encoder can be reduced. Therefore, each single layer encoding unit can be developed at lower development cost than when developing a multilayer encoding unit that processes multiple layers simultaneously. Therefore, it is possible to suppress an increase in development costs when using multi-layer encoding.

The multilayer encoding device 1 according to the first embodiment performs scalable encoding using videos of the same resolution, that is, SNR scalable. Note that although an example in which VVC is used as the encoding method will be mainly described, the encoding method may be a method other than VVC, such as HEVC.

As shown in FIG. 1, the multilayer encoding device 1 according to the first embodiment includes a first single layer encoding unit 10 and a second single layer encoding unit provided after the first single layer encoding unit 10. It has a conversion section 20 and a buffer section 30. The first single layer encoding section 10 corresponds to a single layer encoding section for BL, and the second single layer encoding section 20 corresponds to a single layer encoding section for EL.

The first single layer encoding unit 10 generates a bitstream by encoding the input video signal (original image), and outputs the generated bitstream to the second single layer encoding unit 20. The first single layer encoding section 10 can be configured with a general encoding section. For example, the first single layer encoding unit 10 is an encoding unit that performs encoding using a general profile (for example, Main 10 profile) that is different from a multilayer encoding profile.

The second single layer encoding section 20 outputs the bitstream obtained by the first single layer encoding section 10 as a BL stream. The second single layer encoding unit 20 replaces high-level syntax information such as profile information included in the bitstream obtained by the first single-layer encoding unit 10 with high-level syntax information for multilayer encoding. For example, the second single layer encoding unit 20 replaces single layer encoding profile information with multilayer encoding profile information.

Further, the second single layer encoding unit 20 performs decoding processing on the bitstream obtained by the first single layer encoding unit 10, derives the decoded video and parameter information recorded in the BL stream, and inputs the decoded video and the parameter information recorded in the BL stream. A video signal is encoded as an EL signal, and an EL stream is generated and output. Specifically, the second single layer encoding section 20 performs decoding processing on the bitstream obtained by the first single layer encoding section 10, thereby decoding the bitstream necessary for multilayer encoding (interlayer encoding). A video signal and parameters are acquired, and an EL stream is generated and output using the video signals and parameters. Here, the second single layer encoding unit 20 generates an EL stream using the Multilayer Main10 profile.

The input video signal (original image) input to the second single layer encoding section 20 is an input video signal temporarily held by the buffer section 30. Specifically, the buffer section 30 buffers the input video signal for a time corresponding to the time required for encoding in the first single layer encoding section 10. The second single layer encoding unit 20 generates and outputs an EL stream by encoding the input video signal output by the buffer unit 30. The buffer unit 30 may be built in the second single layer encoding unit 20, but it absorbs the delay difference between the bitstream and the EL encoding process of the original image, and the second single layer encoding unit 20 is configured to be installed at an appropriate timing. Enables EL encoding processing.

FIG. 2 is a diagram showing the configuration of the second single layer encoding section 20 according to the first embodiment.

In addition to the input video signal (original video) output from the buffer section 30, the second single layer encoding section 20 receives as input the bit stream output from the first single layer encoding section 10. The second single layer encoding section 20 includes a decoding processing section 21 , an HLS (High Level Syntax) replacement section 22 , a bitstream generation section 200 , and a stream synthesis section 28 .

The decoding processing unit 21 performs decoding processing on the bitstream output from the first single layer encoding unit 10, outputs the decoded image to the DPB (decoded picture buffer) 24 of the bitstream generation unit 200, and stores the decoded image and motion. The vector is output to the encoding processing section 25 of the bitstream generation section 200. The decoding processing unit 21 includes, for example, a prediction unit that performs a prediction process to generate a predicted image by intra prediction or inter prediction, and an inverse quantization unit that performs an inverse quantization process on the quantized transform coefficients and outputs the transform coefficients. an inverse transformer that performs inverse transform processing on the transform coefficients and outputs a prediction residual; an addition means that adds the prediction residual to the predicted image and outputs a decoded image (reconstructed image); and a loop filter process for the decoded image. and loop filter means for performing.

The HLS replacement unit 22 replaces high-level syntax information such as profile information included in the bitstream output by the first single layer encoding unit 10 with appropriate information as a multilayer encoded signal, and converts the bitstream after the replacement into appropriate information. It is output to the stream combining unit 28 as a BL stream. For example, if BL is encoded with a profile different from multilayer encoding (for example, Main 10 profile), the HLS reassignment unit 22 changes the profile information to a multilayer profile and identifies that the stream is to be used in combination with EL. Modify various high-level syntax shown. Here, since the BL stream is only subjected to HLS replacement and no encoding processing is performed, it can be processed extremely lightweight and at high speed.

The bitstream generation section 200 generates and outputs an EL stream from the input video signal (original video) output from the buffer section 30 using various parameters (decoded image, motion vector, etc.) output from the decoding processing section 21. The bitstream generation section 200 includes a rearrangement section 23, a DPB 24, an encoding processing section 25, a decoding processing section 26, and an entropy encoding section 27.

The rearranging unit 23 rearranges the video frames (original images) in an order different from the display order in order to improve encoding efficiency, and outputs the rearranged video frames to the encoding processing unit 25.

The DPB 24 stores the decoded images output by the decoding processing units 21 and 26 as reference images, and outputs the reference images to the encoding processing unit 25.

The encoding processing unit 25 performs encoding processing on the video frame (original image) output by the rearranging unit 23 using the reference image output by the DPB 24 to generate quantized transform coefficients, and converts the quantized transform coefficients into It is output to the decoding processing section 26 and the entropy encoding section 27. The encoding processing unit 25 includes, for example, a prediction unit that performs a prediction process to generate a predicted image by intra prediction or inter prediction, a subtraction unit that generates a prediction residual of the predicted image with respect to the original image, and a conversion process for the prediction residual. and a quantization means that performs quantization processing on the transform coefficients and outputs quantized transform coefficients.

The decoding processing unit 26 performs decoding processing on the quantized transform coefficients output from the encoding processing unit 25 to generate a decoded image, and outputs the decoded image to the DPB 24 . The decoding processing unit 26 includes, for example, inverse quantization means that performs inverse quantization processing on quantized transform coefficients and outputs transform coefficients, and inverse transform means that performs inverse transform processing on transform coefficients and outputs prediction residuals. , a prediction unit that generates a prediction signal from a decoded signal, an addition unit that adds a prediction residual to the generated prediction image and outputs a decoded image (reconstructed image), and a loop filter that performs loop filter processing on the decoded image. means. The reconstructed image is output to the encoding processing unit as a prediction reference signal, and is also stored in the DPB 24 as a reference signal for inter prediction. For example, the decoding processing unit 26 performs VVC and HEVC decoding processing.

The entropy encoding unit 27 performs entropy encoding processing on the quantized transform coefficients output from the encoding processing unit 25 to generate a bit stream, and outputs the bit stream to the stream combining unit 28 as an EL stream.

The stream combining unit 28 combines the BL stream and the EL stream in the order specified by the encoding method, and outputs the combined stream as one stream.

In this way, the multilayer encoding device 1 according to the first embodiment is configured by directly connecting the first single layer encoding section 10 and the second single layer encoding section 20. Each of the first single layer encoding section 10 and the second single layer encoding section 20 only needs to generate a single layer bitstream. Therefore, compared to configuring a multi-layer encoder that processes multiple layers simultaneously, the processing capacity and complexity of each of the first single-layer encoder 10 and the second single-layer encoder 20 can be reduced. , can be developed with low development cost. Here, as the first single layer encoding section 10, a general encoding section (that is, an existing encoding section) that is not compatible with the profile of multilayer encoding can be used.

Furthermore, although the second single layer encoding unit 20 according to the first embodiment additionally requires a decoding processing unit 21 and an HLS replacement unit 22 compared to a general encoding unit, it does not include a BL encoding processing unit. There is no need to equip the device, the performance required for the device can be significantly lowered, and the development cost can be reduced.

(Example of modification of the first embodiment)
Regarding a modification of the first embodiment, differences from the above-described first embodiment will be mainly described. FIG. 3 is a diagram showing the configuration of a multilayer encoding device 1 according to a modification of the first embodiment.

In this modified example, single layer encoding sections having a common hardware configuration are used as the first single layer encoding section 10 and the second single layer encoding section 20 according to the first embodiment described above. Specifically, the second single layer encoding section 20 according to the first embodiment described above with some functions disabled is used as the first single layer encoding section 10. "Disable" means not to use the part of the function, and may also be referred to as "disable" of the part of the function. For example, this switching of invalidation is performed by the switch unit 201 (201a and 201b). The switch unit 201 (201a and 201b) is a physical switch or a logical switch, and mode switching can be realized by switching the switch unit 201 (201a and 201b).

That is, the multilayer encoding device 1 is configured by using the second single layer encoding unit 20 according to the first embodiment described above as a basic common device and connecting the basic common devices in series. This allows the multilayer encoding device 1 to be configured more efficiently. In this modification example, the basic common device used as the first single layer encoding unit 10 according to the first embodiment described above is referred to as a “single layer encoding device 20a”, and The basic common device used as the 2 single layer encoding unit 20 is referred to as a “single layer encoding device 20b”.

As shown in FIG. 3, each of the single layer encoding device 20a and the single layer encoding device 20b is an encoding device that receives a stream as an external input and has a built-in decoding process, and has a switch unit for mode switching. 201 (201a and 201b) physically or logically, and when stream input is disabled, it can operate as a single layer encoding device, while when stream input is enabled, it can operate as a multi-layer encoding device by serial connection configuration. Perform layer-compatible operations. In this modified example, the single layer encoding device 20a disables stream input by the switch unit 201a, and operates as a single layer encoding device. On the other hand, the single-layer encoding device 20b enables stream input by the switch unit 201b, and performs multi-layer compatible operations.

Specifically, single layer encoding device 20a and single layer encoding device 20b have a common configuration. The common configuration includes a decoding processing unit 21 (21a, 21b) that performs decoding processing on a bitstream output by another single layer encoding device in the preceding stage, and a decoding processing unit 21 (21a, 21b) that performs decoding processing on a bit stream output by another single layer encoding device in the previous stage, and a decoding processing unit 21 (21a, 21b) that performs decoding processing on a bit stream output by another single layer encoding device in the previous stage, and In the case where the decoding processing unit 21a is disabled and the input video signal is encoded, the bitstream generation unit 200 (200a, 200b) outputs a bitstream. Further, the common configuration includes an HLS replacement section 22 (22a, 22b). When the single layer encoding device is used for BL encoding, the HLS replacement unit 22a is disabled. Furthermore, since there is no input from the decoding processing unit 21a in the encoding processing unit, inter-layer prediction is invalidated.

On the other hand, when the single layer encoding device is used for EL encoding, the single layer encoding device enables the decoding processing unit 21 and the HLS replacement unit 22. The single layer encoding device outputs the bitstream obtained by another single layer encoding unit in the previous stage as a BL stream, and the bitstream generation unit 200 inputs the bitstream using parameters derived by performing decoding processing. An EL stream is output by encoding the video signal.

Specifically, although the BL single layer encoding device 20a includes a decoding processing unit 21a and an HLS replacement unit 22a, the decoding processing unit 21a and the HLS replacement unit 22a are disabled. The bitstream generation unit 200a of the BL single layer encoding device 20a generates and outputs a BL bitstream from an input video signal. The bitstream generation section 200a includes a rearrangement section 23a, a DPB 24a, an encoding processing section 25a, a decoding processing section 26a, and an entropy encoding section 27a. The rearrangement unit 23a rearranges the video frames (original images) in an order different from the display order in order to improve encoding efficiency, and outputs the rearranged video frames to the encoding processing unit 25a. The DPB 24a stores the decoded image output by the decoding processing unit 26a as a reference image, and outputs the reference image to the encoding processing unit 25a. The encoding processing unit 25a performs encoding processing on the video frame (original image) output by the rearranging unit 23a using the reference image output by the DPB 24a to generate quantized transform coefficients, and converts the quantized transform coefficients into It is output to the decoding processing section 26a and the entropy encoding section 27a. The decoding processing unit 26a performs decoding processing on the quantized transform coefficients output by the encoding processing unit 25a to generate a decoded image, and outputs the decoded image to the DPB 24a. The entropy encoding unit 27a performs entropy encoding processing on encoded information such as HLS related to EL, quantization parameters, quantization transform coefficients, motion vectors, and other predictive control information output by the encoding processing unit 25a to generate a bit stream. and output the bitstream.

On the other hand, in the single layer encoding device 20b for EL, the decoding processing section 21b and the HLS reassignment section 22b are enabled. The decoding processing unit 21b performs decoding processing on the bitstream output from the single layer encoding device 20a, outputs the decoded image to the DPB 24b of the bitstream generation unit 200b, and outputs the decoded image, motion vector, etc. to the bitstream generation unit 200b. It is output to the encoding processing section 25b. The HLS reassignment unit 22b replaces high-level syntax such as profile information included in the bitstream output by the single layer encoding device 20a with information indicating multilayer encoding (for example, profile information, corresponding layer information, etc.), and The bitstream after replacement is output as a BL stream. The bitstream generation unit 200b generates and outputs a BL bitstream from the input video signal. The bitstream generation section 200b includes a rearrangement section 23b, a DPB 24b, an encoding processing section 25b, a decoding processing section 26b, and an entropy encoding section 27b. The rearrangement unit 23b rearranges the video frames (original images) in an order different from the display order in order to improve encoding efficiency, and outputs the rearranged video frames to the encoding processing unit 25b. The DPB 24b accumulates the decoded images output by the decoding processing units 21 and 26 as reference images, and outputs the reference images to the encoding processing unit 25b. The encoding processing unit 25b performs encoding processing on the video frame (original image) output by the rearranging unit 23b using the reference image output by the DPB 24b to generate quantized transform coefficients, and converts the quantized transform coefficients, etc. is output to the decoding processing section 26b and the entropy encoding section 27b. The decoding processing unit 26b performs decoding processing on the quantized transform coefficients outputted by the encoding processing unit 25b, generates a decoded image, and outputs the decoded image to the DPB 24b. The entropy encoding unit 27b generates a bitstream by performing entropy encoding processing on the quantized transform coefficients and the like outputted by the encoding processing unit 25b, and outputs the bitstream as an EL stream.

In this manner, according to this modification, the multilayer encoding device 1 is configured by connecting basic common devices having a common configuration in series. This allows the multilayer encoding device 1 to be configured more efficiently. Specifically, unlike the first embodiment described above, it is no longer necessary to manufacture and manage each of the first single layer encoding section 10 and the second single layer encoding section 20 individually, and the multilayer encoding device 1 can be easily implemented.

(Second embodiment)
Regarding the second embodiment, differences from the above-described first embodiment will be mainly described. FIG. 4 is a diagram showing the configuration of a multilayer encoding device 1 according to the second embodiment. The multi-layer encoding device 1 according to the second embodiment performs scalable encoding using videos of different resolutions, that is, spatial scalability.

As shown in FIG. 4, the multilayer encoding device 1 according to the second embodiment further includes a downconverter 40 that performs downconversion to reduce the resolution of the input video signal. For example, the down-converting unit 40 down-converts an input video signal that is a 4K video to a 2K video, or down-converts an input video signal that is an 8K video to a 4K video. The first single layer encoding unit 10 outputs a bitstream by encoding the down-converted input video signal output by the down-converting unit 40.

The second single layer encoding section 20 outputs the bitstream obtained by the first single layer encoding section 10 as a BL stream. Further, the second single layer encoding unit 20 performs decoding processing on the bitstream obtained by the first single layer encoding unit 10, derives parameter information recorded in the decoded video and the BL stream, and derives the parameter information recorded in the decoded video and the BL stream. Perform upsampling processing on. The second single layer encoding section 20 encodes the input video signal output from the buffer section 30 using the result of the upsampling process, and generates and outputs an EL stream.

FIG. 5 is a diagram showing the configuration of the second single layer encoding section 20 according to the second embodiment.

The second single layer encoding unit 20 according to the second embodiment includes an upsampling unit 202 that upsamples the decoded image outputted by the decoding processing unit 21, and an upsampling unit 202 that upsamples the motion vector outputted by the decoding processing unit 21. 203. Upsampling units 202 and 203 perform upsampling processing to create a scale suitable for the difference in resolution. The upsampling unit 202 outputs the decoded image after upsampling to the DPB 24 and the encoding processing unit 25. The upsampling unit 203 outputs the decoded image after upsampling to the encoding processing unit 25. The other configurations are the same as those in the first embodiment described above.

(Example of modification of second embodiment)
Regarding the modification of the second embodiment, differences from the above-described second embodiment will be mainly described. FIG. 6 is a diagram showing the configuration of a multilayer encoding device 1 according to a modification of the second embodiment.

The multi-layer encoding device 1 according to this modification example has a common hardware configuration as the first single layer encoding section 10 and the second single layer encoding section 20, similarly to the modification example of the first embodiment described above. A single-layer encoding unit with the following functions is used. Specifically, the second single layer encoding section 20 according to the second embodiment described above with some functions disabled is used as the first single layer encoding section 10.

That is, the multilayer encoding device 1 is configured by using the second single layer encoding section 20 according to the second embodiment described above as a basic common device and connecting the basic common devices in series. In this modified example, the basic common device used as the first single layer encoding unit 10 according to the second embodiment described above is referred to as a “single layer encoding device 20a”, and The basic common device used as the 2 single layer encoding unit 20 is referred to as a “single layer encoding device 20b”.

The single layer encoding device 20a for BL includes a decoding processing section 21a, an HLS replacement section 22a, an upsampling section 202a, and an upsampling section 203a; and invalidates the upsampling unit 203a. The single layer encoding device 20a outputs a bitstream by encoding the down-converted input video signal output by the down-converting unit 40.

On the other hand, in the single layer encoding device 20b for EL, the decoding processing section 21b, the HLS replacement section 22b, the upsampling section 202b, and the upsampling section 203b are enabled. The single layer encoding device 20b outputs the bitstream obtained by the single layer encoding device 20a as a BL stream. Furthermore, in the single layer encoding device 20b, the upsampling unit 202b and the upsampling unit 203b perform upsampling processing on parameters derived by performing decoding processing on the bitstream obtained by the single layer encoding device 20a. , encodes the input video signal output by the buffer unit 30 using the result of the upsampling process, and generates and outputs an EL stream.

(Third embodiment)
In the above-described embodiments and modifications thereof, the multilayer encoding device 1 having a configuration in which two stages of single layer encoding units (single layer encoding devices) are connected in series has been mainly described. However, a configuration in which three or more stages of single layer encoding units (single layer encoding devices) are connected in series may also be used. FIGS. 7 and 8 are diagrams showing the configuration of a multilayer encoding device 1 according to the third embodiment.

As shown in FIG. 7, the multilayer encoding device 1 according to the third embodiment includes a first single layer encoding section 10 and a second single layer encoding section provided after the first single layer encoding section 10. The encoding unit 20A, a third single layer encoding unit 20B provided after the second single layer encoding unit 20A, and buffer units 30A and 30B are provided. The first single layer encoding section 10 corresponds to a single layer encoding section for BL, and the second single layer encoding section 20A and the third single layer encoding section 20B each correspond to a single layer encoding section for EL. corresponds to the section.

The first single layer encoding unit 10 generates a bitstream by encoding the input video signal (original image), and outputs the generated bitstream to the second single layer encoding unit 20A. The first single layer encoding unit 10 may encode the down-converted input video signal as in the second embodiment described above (see FIG. 8). The first single layer encoding section 10 can be configured with a general encoding section. For example, the first single layer encoding unit 10 is an encoding unit that performs encoding using a general profile (eg, Main10 profile) that is different from a multilayer encoding profile.

Alternatively, the first single layer encoding section 10 may have a common hardware configuration with each of the second single layer encoding section 20A and the third single layer encoding section 20B. The first single layer encoding unit 10 may be configured such that some functions in the common configuration are disabled (see the above-mentioned modification example).

The second single layer encoding section 20A is configured similarly to the second single layer encoding section 20 according to the embodiment described above. The second single layer encoding section 20A outputs the bitstream obtained by the first single layer encoding section 10 as a BL stream to the third single layer encoding section 20B. However, in the case where the first single layer encoding section 10 performs encoding using a different encoding profile from multilayer encoding, the second single layer encoding section 20A performs encoding based on the encoding profile obtained by the first single layer encoding section 10. High-level syntax information including profile information included in the bitstream is replaced with multilayer encoding information.

Further, the second single layer encoding section 20A uses the parameters derived by decoding the bitstream obtained by the first single layer encoding section 10 to convert the input video signal output from the buffer section 30A. EL stream 1 is encoded, and EL stream 1 is generated and output. The second single layer encoding unit 20A may encode the down-converted input video signal (see FIG. 8). The second single layer encoding unit 20A acquires parameters necessary for multilayer encoding (interlayer encoding) by performing decoding processing on the bitstream obtained by the first single layer encoding unit 10, EL stream 1 is generated and output using the parameters. For example, the second single layer encoding unit 20A generates EL stream 1 according to the Multilayer Main 10 profile, and outputs a bitstream in which the BL stream and EL stream 1 are combined.

The third single layer encoding section 20B is configured similarly to the second single layer encoding section 20 according to the above-described embodiment. The third single layer encoding section 20B outputs the BL stream output from the second single layer encoding section 20A as is. Further, the third single layer encoding section 20B encodes the input video signal output from the buffer section 30B using the parameters derived by decoding the BL stream output from the second single layer encoding section 20A. , and generates and outputs EL stream 2. Specifically, the third single layer encoding unit 20B performs decoding processing on the BL stream output by the second single layer encoding unit 20A, thereby obtaining parameters necessary for multilayer encoding (interlayer encoding). , and generates and outputs EL stream 2 using the parameters. For example, the second single layer encoding unit 20A generates EL stream 2 according to the Multilayer Main 10 profile, and outputs a bitstream that is a combination of the BL stream, EL stream 1, and EL stream 2.

The input video signal (original image) input to the second single layer encoding section 20A is an input video signal temporarily held by the buffer section 30A. Specifically, the buffer section 30A buffers the input video signal for a time corresponding to the time required for encoding in the first single layer encoding section 10. The second single layer encoding section 20A generates and outputs the EL stream 1 by encoding the input video signal output from the buffer section 30A. The buffer unit 30A may be built in the second single-layer encoding unit 20A, but it absorbs the delay difference between the bitstream and the EL encoding process of the original image, and the second single-layer encoding unit 20A can be installed at an appropriate timing. Enables EL encoding processing.

The input video signal (original image) input to the third single layer encoding section 20B is an input video signal temporarily held by the buffer section 30B. Specifically, the buffer section 30B buffers the input video signal output by the buffer section 30A for a time corresponding to the time required for encoding in the second single layer encoding section 20A. The third single layer encoding unit 20B generates and outputs the EL stream 2 by encoding the input video signal output by the buffer unit 30B. Although the buffer section 30B may be built in the third single layer encoding section 20B, it is arranged so that the third single layer encoding section 20B can perform EL encoding processing at an appropriate timing.

In the configuration example of FIG. 8, a down-converting section 40A down-converts the input video signal and outputs it to the first single layer encoding section 10, and a down-converting section 40A down-converts the input video signal outputted by the buffer section 30A and outputs the down-converting input video signal to the second single-layer encoding section 10. It has a down converter 40B that outputs to the single layer encoder 20A. For example, when the input video signal is an 8K video signal, the down-converter 40A outputs the video signal down-converted to 2K to the first single layer encoding unit 10, and the down-converter 40B outputs the video signal down-converted to 4K. The signal may be output to the second single layer encoding section 20A.

(Other embodiments)
A program that causes a computer to execute each process performed by the multilayer encoding device 1 or the single layer encoding device (single layer encoding unit) 20 may be provided. The program may be recorded on a computer readable medium. Computer-readable media allow programs to be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

For example, the program causes the computer to function as the multilayer encoding device 1. Specifically, the program includes a first single layer encoding procedure for outputting a bitstream by encoding an input video signal in a computer, and a second single layer encoding procedure provided after the first single layer encoding procedure. A single layer encoding procedure is performed. The second single layer encoding procedure includes a procedure for outputting the bitstream obtained by the first single layer encoding procedure as a BL stream, and a decoding process for the bitstream obtained by the first single layer encoding procedure. The method includes a procedure of encoding an input video signal using the derived parameters and outputting an EL stream.

Alternatively, the program may be a single layer encoding program module that causes the computer to function as the single layer encoding device 20. Specifically, the program module instructs the computer in a decoding procedure for decoding a bitstream obtained by another preceding single-layer encoding program module, and instructs the computer to perform base layer encoding by the program module. When used, the decoding processing procedure is invalidated, and a bitstream generation procedure for outputting a bitstream by encoding the input video signal is executed. When the program module is used for enhancement layer encoding, the program module outputs the bitstream obtained by another single layer encoding program module in the preceding stage as a BL stream, and validates the decoding procedure. At the same time, the input video signal is encoded using parameters derived through decoding processing, and an EL stream is output.

The circuits that execute each process performed by the multi-layer encoding device 1 or the single-layer encoding device 20 are integrated, and the multi-layer encoding device 1 or the single-layer encoding device 20 is configured as a semiconductor integrated circuit (chip set, SoC). You may.

As used in this disclosure, the terms "based on" and "depending on/in response to" refer to "based solely on" and "depending on," unless expressly stated otherwise. does not mean "only according to". Reference to "based on" means both "based solely on" and "based at least in part on." Similarly, the phrase "in accordance with" means both "in accordance with" and "in accordance with, at least in part." The terms "include", "comprise", and variations thereof do not mean to include only the listed items, but may include only the listed items or in addition to the listed items. This means that it may contain further items. Also, as used in this disclosure, the term "or" is not intended to be exclusive OR. Furthermore, any reference to elements using the designations "first," "second," etc. used in this disclosure does not generally limit the amount or order of those elements. These designations may be used herein as a convenient way of distinguishing between two or more elements. Thus, reference to a first and second element does not imply that only two elements may be employed therein or that the first element must precede the second element in any way. In this disclosure, when articles are added by translation, for example, a, an, and the in English, these articles are used in the plural unless the context clearly indicates otherwise. shall include things.

Although the embodiments have been described above in detail with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made without departing from the scope of the invention.

1: Multilayer encoding device 10: First single layer encoding section 20: Second single layer encoding section (single layer encoding device)
20A: Second single layer encoding section 20B: Third single layer encoding section 20a: Single layer encoding device 20b: Single layer encoding device 21: Decoding processing section 21a: Decoding processing section 21b: Decoding processing section 22: HLS Replacement section 22a: HLS reassignment section 22b: HLS reassignment section 23: Reordering section 23a: Reordering section 23b: Reordering section 25: Encoding processing section 25a: Encoding processing section 25b: Encoding processing section 26: Decoding processing section 26a: Decoding processing unit 26b: Decoding processing unit 27: Entropy coding unit 27a: Entropy coding unit 27b: Entropy coding unit 28: Stream synthesis unit 28a: Stream synthesis unit 28b: Stream synthesis unit 30: Buffer unit 30A: Buffer Section 30B: Buffer section 40: Down-conversion section 40A: Down-conversion section 200: Bitstream generation section 200a: Bitstream generation section 200b: Bitstream generation section 201a: Switch section 201b: Switch section 202: Upsampling section 202a: Upsampling Section 202b: Upsampling section 203: Upsampling section 203a: Upsampling section 203b: Upsampling section

Claims (12)

A multi-layer encoding device that performs multi-layer encoding,
first single layer encoding means for outputting a bitstream by encoding an input video signal;
a second single layer encoding means provided after the first single layer encoding means,
The second single layer encoding means includes:
outputting the bitstream obtained by the first single layer encoding means as a base layer bitstream;
A multilayer encoding device that encodes the input video signal using parameters derived by decoding the bitstream obtained by the first single layer encoding means, and outputs an enhancement layer bitstream. The second single layer encoding means replaces high-level syntax information including profile information included in the bitstream obtained by the first single layer encoding means with the multilayer encoding information. The multilayer encoding device described in . Further comprising a buffer means for buffering the input video signal for a time corresponding to the time required for encoding by the first single layer encoding means,
The multi-layer encoding device according to claim 1, wherein the second single layer encoding means outputs the enhancement layer bit stream by encoding the input video signal outputted by the buffer means. further comprising down-converting means for down-converting the input video signal to reduce its resolution,
The multi-layer encoding according to any one of claims 1 to 3, wherein the first single-layer encoding means outputs the bitstream by encoding the input video signal output by the down-converting means. Device. The second single layer encoding means includes:
outputting the bitstream obtained by the first single layer encoding means as the base layer bitstream;
Performing decoding processing on the bitstream obtained by the first single layer encoding means, performing upsampling processing on the parameters derived, and encoding the input video signal using the result of the upsampling processing, The multi-layer encoding device according to claim 4, wherein the multi-layer encoding device outputs a bitstream of the enhancement layer. The first single layer encoding means and the second single layer encoding means have a common configuration,
The common configuration is
a decoding processing means that performs decoding processing on a bitstream outputted by another single layer encoding means in the previous stage;
When the single layer encoding means is used for base layer encoding, bitstream generation means for disabling the decoding processing means and outputting a bitstream by encoding the input video signal; Prepare,
In a case where the single layer encoding means is used for encoding an enhancement layer,
Outputting the bitstream obtained by another single layer encoding means in the previous stage as the base layer bitstream,
Claim 1: While enabling the decoding processing means, the bitstream generation means outputs an enhancement layer bitstream by encoding the input video signal using parameters derived by performing the decoding processing. The multilayer encoding device described in . The common configuration is such that when the single layer encoding means is used for encoding the enhancement layer, the high-level synchronization method includes profile information included in the bitstream output by the other single layer encoding means at the preceding stage. further comprising a replacement means for replacing tax information with the multi-layer encoding information,
The multi-layer encoding device according to claim 6, wherein when the single layer encoding means is used for encoding the base layer, the replacing means is disabled. A program for causing a computer to function as the multilayer encoding device according to claim 1. A single layer encoding device,
a decoding processing means that performs decoding processing on the input bitstream;
a bitstream generation means for outputting a bitstream by encoding an input video signal;
A switch means for switching a mode of whether or not input of the bitstream is enabled,
When the input of the bitstream is invalidated, the bitstream generating means operates to perform single layer encoding without performing the decoding process,
When input of the bitstream is enabled, a single layer encoding device operates to perform multilayer encoding by a series connection configuration of a plurality of single layer encoding devices including the own single layer encoding device. 10. The apparatus according to claim 9, further comprising replacing means for replacing high-level syntax information including profile information included in the input bitstream with the multi-layer encoding information when the input of the bitstream is enabled. single layer encoding device. When the bitstream input is enabled,
The decoding processing means performs the decoding processing on a bitstream output by another single layer encoding device in the previous stage,
The single layer encoding device according to claim 9 or 10, wherein the bitstream generation means encodes the input video signal using parameters derived by performing the decoding process, and generates an enhancement layer bitstream. A program for causing a computer to function as the single layer encoding device according to claim 9.

Images