JP7369574B2 - Video conversion device and its program - Google Patents

Description

The present invention relates to a video conversion device and program for converting HDR video to SDR video.

With the standardization of High Dynamic Range (HDR) video formats and their introduction into broadcasting formats, the production and broadcasting of HDR programs is increasing. In HDR program production, it is possible to use video shot with an SDR (Standard Dynamic Range) camera in addition to HDR video shot with an HDR camera, and to insert previously produced SDR program materials into the HDR program. be.

Conventionally, SDR video is converted to look like HDR video and then used in an HDR program. Such a conversion method is called "Inverse Tone Mapping (ITM)," and three methods are exemplified in Non-Patent Document 1. When converted using this ITM, although it looks like HDR video, there is a possibility that the original production intention of the SDR program will be lost. Therefore, "Direct Mapping (DM)" is sometimes used so that it can be used in HDR programs while maintaining the original SDR program production intention (FIG. 9, Non-Patent Documents 2 and 3). In this direct mapping, as shown in FIG. 10, for example, a signal level of 0-100% in the original SDR video is mapped to a signal level of 0-75% in the converted HDR video. Here, this 75% signal level is defined as a reference white level for HDR video (Non-Patent Document 3).
Note that an SDR video before being converted to an HDR video will be referred to as an "original SDR video," and an SDR video that has been converted from an HDR video will be referred to as a "converted SDR video."

When broadcasting an HDR program using high-definition broadcasting (for example, digital terrestrial broadcasting) that only specifies SDR as a broadcasting method, it is necessary to convert HDR video to SDR video. A method for converting HDR video into SDR video is called "tone mapping (TM)," and three methods are exemplified in Non-Patent Document 1. Furthermore, the above-mentioned Non-Patent Document 3 describes the basic concept of tone mapping (FIG. 11). FIG. 12 shows an example of the characteristics of this tone mapping.

To summarize the above, as shown in FIG. 13, HDR video includes "HDR video produced in HDR" and "HDR video converted from SDR video." Furthermore, "direct mapping" and "inverse tone mapping" are used to convert SDR video to HDR video. Then, these different HDR images are converted into SDR images by "tone mapping."

Report ITU-R BT.2446 “Methods for conversion of high dynamic range content to standard dynamic range content and vice-versa” Report ITU-R BT.2390 “High dynamic range television for production and international program exchange” ARIB TR-B43 “Operation Guidelines for Program Production Using High Dynamic Range Video”

When HDR videos of different origins are uniformly converted into SDR videos using "tone mapping," there is a problem in that the converted SDR video looks different from the original SDR video. FIG. 14 illustrates the relationship between signal levels between the original SDR video and the converted SDR video. If the gradation characteristics of the original SDR video are correctly reproduced, the signal levels of both SDR videos will have a linear relationship as shown by the broken line in FIG. The gradation characteristics match.

Here, consider a case where an SDR video is converted to an HDR video using "direct mapping", and this HDR video is converted to an SDR video using "tone mapping". In this case, the signal levels of the original SDR video and the converted SDR video have a nonlinear relationship as shown by the solid line in Figure 14, and the gradation characteristics of the original SDR video and the converted SDR video are different. . In other words, the converted SDR video is overall brighter than the original SDR video, and the gradation of the bright parts is lost. In this way, the gradation characteristics of the SDR video after conversion have changed from the gradation characteristics of the original SDR video, and the SDR video after conversion looks different from the original SDR video.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a video conversion device and a program for the same, in which a converted SDR video looks the same as the original SDR video.

In view of the above-mentioned problems, the video conversion device according to the present invention includes a video source identifier indicating whether the HDR video was produced in HDR or converted from the original SDR video, and a video source identifier indicating whether the HDR video was converted from the original SDR video. The present invention is a video conversion device that converts an HDR video into a converted SDR video based on a conversion method and a conversion method/characteristic identifier indicating the conversion characteristics of the case, and includes a video conversion means.

According to this configuration, the video conversion means receives the video source identifier and the conversion method/characteristic identifier along with the HDR video, and identifies the conversion method and conversion characteristics based on the input video source identifier and conversion method/characteristic identifier. , converts the HDR video into a converted SDR video using the identified conversion method and conversion characteristics. The video conversion means converts the HDR video into a converted SDR video by tone mapping when the video source identifier indicates that the video was produced in HDR, and the video source identifier indicates that the video was converted from the original SDR video, and , if the conversion method/characteristic identifier indicates that the conversion was performed by inverse tone mapping, the HDR video is converted to the converted SDR video by tone mapping that gives the inverse characteristics of the inverse tone mapping, and the video source identifier is the original SDR video. If the conversion method/characteristic identifier indicates that the conversion was performed using direct mapping, the HDR video is converted to the converted SDR video using a conversion function that provides the inverse characteristics of direct mapping. In this way, the video conversion device converts the HDR video into the converted SDR video, taking into consideration the conversion method and conversion characteristics of the original SDR video.

Further, in view of the above-mentioned problems, a video conversion device according to the present invention is a video conversion device that converts an HDR video to a converted SDR video based on an HDR video level, and includes a video determination unit, a video conversion device, and a video conversion device. The configuration includes a means.

According to this configuration, the video determining means receives the HDR video and determines whether or not the input HDR video does not include a component higher than a predetermined video level.
The video conversion means determines the origin of the HDR video based on the determination result of the video determination means, and converts the HDR video into a converted SDR video based on the determined origin of the HDR video.

Here, since the HDR video converted by direct mapping does not contain components higher than a predetermined video level (generally the HDR reference level), the conversion method of the original SDR video depends on the presence or absence of components higher than the predetermined video level. can be determined. The video converting means converts the HDR video into a converted SDR video using a conversion function that provides an inverse characteristic of direct mapping when the determination result indicates that the HDR video does not contain components higher than the predetermined video level; If the determination result indicates that the HDR video contains components of a predetermined video level or higher, the HDR video is converted to a converted SDR video by tone mapping. In this way, the video conversion device converts the HDR video into the converted SDR video, taking into consideration the conversion method of the original SDR video.

Note that the present invention can also be realized by a program that causes hardware resources such as a CPU, memory, and hard disk included in a computer to operate as the above-mentioned video conversion device.

According to the present invention, the SDR video after conversion looks the same as the original SDR video.

FIG. 1 is a block diagram showing the configuration of a video conversion device according to a first embodiment. 2 is an explanatory diagram illustrating the HDR video production means of FIG. 1. FIG. FIG. 2 is an explanatory diagram illustrating the SDR video conversion means of FIG. 1; 3 is a flowchart showing the operation of the video conversion device according to the first embodiment. FIG. 7 is an explanatory diagram illustrating a video conversion device according to a modified example. FIG. 2 is a block diagram showing the configuration of a video conversion device according to a second embodiment. 7 is an explanatory diagram illustrating the SDR video conversion means of FIG. 6. FIG. 7 is a flowchart showing the operation of the video conversion device according to the second embodiment. It is an explanatory diagram explaining conventional direct mapping. It is a graph showing the characteristics of conventional direct mapping. FIG. 2 is an explanatory diagram illustrating conventional tone mapping. It is a graph showing the characteristics of conventional tone mapping. It is an explanatory diagram explaining conventional HDR program production and conversion to SDR video. 2 is a graph showing a relationship between signal levels between an original SDR video and an SDR video converted from an HDR video.

(First embodiment)
[Configuration of video conversion device]
Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings as appropriate. In addition, in each embodiment, the same code|symbol was attached|subjected to the same means, and description was abbreviate|omitted.
With reference to FIG. 1, the configuration of a video conversion device 1 according to the first embodiment will be described.
The video conversion device 1 converts HDR video into SDR video, and includes HDR video production means 10, storage means 20, and SDR video conversion means 30, as shown in FIG.

<HDR video production means>
The HDR video production means 10 is for producing an HDR program from video materials such as SDR video and HDR video, and includes an HDR conversion means 11, a switching means 12, and an identifier adding means 13, as shown in FIG. . In addition, in FIG. 2, the HDR conversion means 11 is illustrated as "SDR→HDR conversion", and the identifier addition means 13 is illustrated as "identifier addition".

The HDR conversion means 11 converts the SDR video into an HDR video when an SDR video is input as a video material. For example, the HDR conversion means 11 receives an instruction for a conversion method from SDR video to HDR video from the user of the video conversion device 1, and converts the SDR video to HDR video using the specified conversion method. This conversion method is either "inverse tone mapping" or "direct mapping." Here, there are multiple methods for "inverse tone mapping" as shown in Non-Patent Document 1, and each method allows setting of different conversion characteristics, so it is necessary to identify the conversion characteristics as well. be. Even in "direct mapping," the conversion characteristics change depending on the setting of the reference level, so it is necessary to identify the conversion characteristics. Then, the HDR conversion means 11 outputs the converted HDR video to the switching means 12, generates the above-mentioned conversion method and conversion characteristics, and outputs them to the identifier addition means 13.

The switching means 12 switches the HDR video converted by the HDR converting means 11 or the HDR video directly input to the switching means 12 and outputs it to the identifier adding means 13. Here, the switching of HDR video by the switching means 12 is an operation performed by a program producer in the process of producing a program. When the switching means 12 switches to the HDR video from the HDR converting means 11, it generates a video source identifier indicating that the HDR video is converted from the original SDR video, and outputs it to the identifier adding means 13. On the other hand, when switching to directly input HDR video, the switching means 12 generates a video source identifier indicating that the video is an HDR video produced by HDR, and outputs it to the identifier adding means 13 .

The identifier adding means 13 adds a video source identifier and a conversion method/characteristic identifier to each frame of the HDR video input from the switching means 12. Then, the identifier adding means 13 writes the HDR video to which the video source identifier and the conversion method/characteristic identifier have been added into the storage means 20 (FIG. 1).
Note that when the video source identifier indicates that the video is an HDR video produced using HDR, the conversion method/characteristic identifier is set to indicate "no conversion."

Returning to FIG. 1, the description of the configuration of the video conversion device 1 will be continued.
The storage means 20 is a storage device such as a memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive) that stores an HDR video to which a video source identifier and a conversion method/characteristic identifier are added. This HDR video is referred to by SDR video conversion means 30, which will be described later.

<SDR video conversion means>
The SDR video conversion means 30 converts the HDR video stored in the storage means 20 into SDR video. As shown in FIG. 3, the SDR video conversion means 30 includes an identifier extraction means 31, a video conversion means 32, and a switching means 33.
In addition, in FIG. 3, the identifier extracting means 31 is illustrated as "identifier extraction", and the SDR video converted by tone mapping is illustrated as "SDR _{TM video", and the SDR video converted by tone mapping is shown as "SDR TM} video", and the SDR video converted by tone mapping is shown as "SDR TM video". The SDR video is illustrated as "SDR _DM video."

The identifier extracting means 31 extracts a video source identifier and a conversion method/characteristic identifier added to an HDR video. Then, the identifier extracting means 31 outputs the HDR video, the video source identifier, and the conversion method/characteristic identifier to the video converting means 32.

The video conversion means 32 identifies a conversion method and conversion characteristics based on the video source identifier and conversion method/characteristic identifier input from the identifier extraction means 31, and converts the HDR video into an SDR video using the identified conversion method and conversion characteristics. It is something to do. In this embodiment, there are three conversion methods, so they will be explained one by one.

<<First conversion method: For HDR video produced by HDR>>
In the first conversion method, the video source identifier indicates that HDR production has been performed. In this case, the video conversion means 32 converts the HDR video into an SDR video by tone mapping. Here, in the video conversion means 32, arbitrary characteristics are set in advance so that the HDR video can be converted into an optimal SDR video, and tone mapping is performed using the characteristics.
Note that in the first conversion method, the conversion method/characteristic identifier indicates "no conversion", and the video conversion means 32 does not need to refer to the conversion method/characteristic identifier.

<<Second conversion method: For HDR video derived from inverse tone mapping>>
In the second conversion method, the video source identifier indicates that the SDR video was converted, and the conversion method/characteristic identifier indicates that the conversion was performed by inverse tone mapping. In this case, the video converting means 32 converts the HDR video into an SDR video by tone mapping that provides the inverse characteristics of the inverse tone mapping. The inverse characteristics of this inverse tone mapping are determined and set sequentially based on the inverse tone mapping characteristics indicated by the conversion method/characteristic identifier. If the characteristics of the inverse tone mapping are known, the inverse characteristics may be set in advance.

<<Third conversion method: For HDR video derived from direct mapping>>
The third conversion method indicates that the video source identifier was converted from SDR, and that the conversion method/characteristic identifier was converted by direct mapping. In this case, the video conversion means 32 converts the HDR video into an SDR video using a conversion function that provides the inverse characteristics of direct mapping. For example, if the characteristics of direct mapping are shown in FIG. 10, the inverse function of FIG. 10 may be set in advance in the video conversion means 32 as a conversion function that provides the inverse characteristics of direct mapping.

The switching unit 33 switches the SDR video (SDR _TM video, SDR _DM video) converted by the video converting unit 32 frame by frame according to the video source identifier, and outputs it to the outside (for example, a program transmission device not shown). It is something.
Note that in FIG. 3, the SDR video converted using the first or second conversion method becomes the SDR _TM video, and the SDR video converted using the third conversion method becomes the SDR _DM video. Furthermore, although the first and second conversion methods are the same in that they perform conversion using tone mapping, the characteristics when performing conversion using tone mapping are different.

[Operation of video conversion device]
The operation of the video conversion device 1 will be described with reference to FIG. 4.
As shown in FIG. 4, in step S1, when an SDR video is input as a video material, the HDR converter 11 converts the SDR video into an HDR video. Note that if an HDR video is input as the video material, the process may proceed to step S2 without performing the process of step S1.

In step S2, the identifier adding means 13 adds a video source identifier and a conversion method/characteristic identifier to the HDR video. The HDR video to which the video source identifier and the conversion method/characteristic identifier have been added is stored in the storage means 20.
In step S3, the identifier extraction means 31 reads the HDR video from the storage means 20 and extracts the video source identifier and conversion method/characteristic identifier added to the HDR video.

In step S4, the video conversion means 32 determines a conversion method based on the video source identifier and the conversion method/characteristic identifier, and converts the HDR video into an SDR video using the determined conversion method. As described above, the video converting means 32 converts HDR video into SDR video using any of the first to third conversion methods.

[Action/Effect]
As described above, in the video conversion device 1, when HDR video is converted from SDR, the conversion method of the original SDR video is considered and the HDR video is converted to SDR video. The result will be similar to the original SDR image. In other words, in the video conversion device 1, there is no difference in brightness or gradation reproduction characteristics, regardless of whether the video material is produced in HDR or converted from SDR video, or whether it is converted by inverse tone mapping or direct mapping. There is no difference in appearance between the original SDR video and the converted SDR video. In this way, the video conversion device 1 can produce high-quality SDR programs.

Although the first embodiment has been described in detail above, the present invention is not limited to the first embodiment described above, and includes design changes within a range that does not depart from the gist of the present invention.
Although in the first embodiment described above, the video source identifier and the conversion method/characteristic identifier are separate identifiers, the present invention is not limited to this. That is, in the video conversion device 1, the video source identifier and the conversion method/characteristic identifier may be combined into one video source/conversion method/characteristic identifier.

In the first embodiment described above, the video source identifier and the conversion method/characteristic identifier are added to the HDR video, and the HDR video to which the identifier is added is temporarily stored in the storage unit 20, but the present invention is not limited to this. For example, the HDR video production means and the SDR video conversion means may be directly connected via a digital interface, and the video source identifier and conversion method/characteristic identifier may be multiplexed into the auxiliary data area of the digital interface. Further, as shown in FIG. 5, the video conversion device 1B directly sends the HDR video and video source identifier output from the switching means 12 and the conversion method/characteristic identifier output from the HDR conversion means 11 to the video conversion means 32. You can also enter it. In this case, the video conversion device 1B does not need to add the video source identifier and the conversion method/characteristic identifier to the HDR video. In addition, in FIG. 5, the video conversion means 32 is illustrated as "HDR→SDR conversion".

(Second embodiment)
[Configuration of video conversion device]
With reference to FIG. 6, differences from the first embodiment regarding a video conversion device 1C according to the second embodiment will be described.
The second embodiment differs from the first embodiment in that the conversion method is determined based on the HDR video level without using a video source identifier or a conversion method/characteristic identifier.

As shown in FIG. 6, the video conversion device 1C includes a video source analysis/determination means (video determination means) 40 and an SDR video conversion means (video conversion means) 50.
Note that the HDR video input to the video conversion device 1C is equivalent to the HDR video output from the switching means 12 in the first embodiment, and includes a mixture of HDR video and video converted from SDR video. are doing. Note that the method for converting SDR video into HDR video is the same as that in the first embodiment, so the explanation will be omitted.

<Video source analysis/judgment means>
The video source analysis/determination means 40 receives an HDR video and determines for each frame of the HDR video whether or not the input HDR video contains a component of a predetermined video level or higher. In other words, the video source analysis/determination means 40 analyzes the signal level of the HDR video using a known method, and determines whether a component higher than a predetermined video level is included. Then, the video source analysis/judgment means 40 outputs the determination result to the SDR video conversion means 50 together with the HDR video.

Here, it is preferable to set the predetermined video level as the HDR reference level. The HDR reference level is the reference white level of HDR video defined in Non-Patent Document 3 (in the case of HDR video using the HLG method, HLG reference white of 75% signal level). This predetermined video level is set in advance in the video source analysis/judgment means 40.

<SDR video conversion means>
The SDR video conversion means 50 determines the origin of the HDR video based on the determination result input from the video source analysis/determination means 40, and converts the HDR video into an SDR video based on the determined origin of the HDR video. It is. As shown in FIG. 7, the SDR video conversion means 50 includes a converted video output means 51 and a switching means 52.

Converted video output means 51 converts HDR video into SDR video and outputs the converted SDR video to switching means 52. In this embodiment, there are two conversion methods, so they will be explained one by one.

<<First conversion method: For HDR video derived from direct mapping>>
The first conversion method corresponds to a case where HDR video is converted from SDR video by direct mapping. In this case, since it is determined that components of a predetermined video level or higher are not included, it can be presumed that direct mapping was used to convert the HDR video to the SDR video. Therefore, the converted video output means 51 converts the HDR video into an SDR video using a conversion function that provides the inverse characteristics of direct mapping.

<<Second conversion method: For HDR video not derived from direct mapping>>
The second conversion method corresponds to the case where the HDR video is not converted from the SDR video by direct mapping, that is, the HDR video is produced in HDR, or the HDR video is converted from the SDR video by reverse tone mapping. In this case, since it is determined that components of a predetermined video level or higher are included, it can be presumed that a method other than direct mapping was used to convert the HDR video to the SDR video. Therefore, the converted video output means 51 converts the HDR video into an SDR video by tone mapping.

Note that, unlike the first embodiment, the video conversion device 1C cannot determine whether the video is an HDR video produced in HDR or an HDR video converted using inverse tone mapping. For this reason, the video conversion device 1C converts both the HDR video produced in HDR and the HDR video converted by inverse tone mapping into SDR video using tone mapping with the same characteristics.

The switching means 52 switches the SDR video input from the converted video output means 51 based on the determination result of the video source analysis/determination means 40 and outputs it to the outside (for example, a program transmission device not shown). be. Specifically, when the switching means 52 determines that a component higher than a predetermined video level is not included, the switching means 52 outputs an SDR _DM video that has been converted from an HDR video to an SDR video using a conversion function that provides the inverse characteristics of direct mapping. Switch to On the other hand, if the switching unit 52 determines that a component of a predetermined video level or higher is included, the switching unit 52 switches to the SDR _TM video output converted from the HDR video to the SDR video by tone mapping.
Note that in FIG. 7, the SDR video converted using the first conversion method becomes the SDR _DM video, and the SDR video converted using the second conversion method becomes the SDR _TM video.

[Operation of video conversion device]
Referring to FIG. 8, the operation of the video conversion device 1C will be described.
As shown in FIG. 8, in step S10, the video source analysis/determination means 40 analyzes the signal level of the HDR video.
In step S11, the video source analysis/determination means 40 determines whether the HDR video does not include components higher than a predetermined video level.

In step S12, the converted video output means 51 converts the HDR video into an SDR video (SDR _TM video) by tone mapping, and converts the HDR video into an SDR video (SDR _DM video) by using a conversion function that provides the inverse characteristics of direct mapping. Convert.

If a component of a predetermined video level or higher is not included (Yes in step S13), it can be estimated that the conversion has been performed by direct mapping, so the switching means 52 switches to the output of SDR _DM video (step S14).
If a component of a predetermined video level or higher is included (No in step S13), it can be estimated that the conversion has not been performed by direct mapping, so the switching means 52 switches to the output of the SDR _TM video (step S15).

[Action/Effect]
As described above, in the video conversion device 1C, when HDR video is converted from SDR video, the HDR video is converted to SDR video by considering the conversion method of the original SDR video. The appearance will be similar to the original SDR image. Furthermore, the video conversion device 1C can convert an HDR video to an SDR video even if a video source identifier and a conversion method/characteristic identifier are not added to the HDR video. In this way, the video conversion device 1C can produce a high quality SDR program.

In each of the embodiments described above, the video conversion device has been described as independent hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, memory, and hard disk included in a computer to operate as the above-mentioned video conversion device. These programs may be distributed via communication lines, or may be written and distributed on recording media such as CD-ROMs and flash memories.

1, 1B, 1C Video conversion device 10 HDR video production means 11 HDR conversion means 12 Switching means 13 Identifier addition means 20 Storage means 30 SDR video conversion means 31 Identifier extraction means 32 Video conversion means 33 Switching means 40 Video source analysis/judgment means (Video judgment means)
50 SDR video conversion means (video conversion means)
51 Converted video output means 52 Switching means

Claims (4)

A video source identifier indicating whether the HDR video was produced as HDR or converted from the original SDR video, and a conversion method/characteristic identifier indicating the conversion method and conversion characteristics when the original SDR video was converted to the HDR video. A video conversion device that converts the HDR video to a converted SDR video based on,
The video source identifier and the conversion method/characteristic identifier are input together with the HDR video, and the conversion method and conversion characteristics are identified based on the input video source identifier and the conversion method/characteristic identifier, and the identified conversion method and video conversion means for converting the HDR video into the converted SDR video according to conversion characteristics;
Equipped with
The video conversion means includes:
If the video source identifier indicates that the video was produced in HDR, converting the HDR video into the converted SDR video by tone mapping;
If the video source identifier indicates that the video has been converted from the original SDR video, and the conversion method/characteristic identifier indicates that the conversion has been performed by inverse tone mapping, tone mapping that provides an inverse characteristic of the inverse tone mapping. Converting the HDR video to the converted SDR video by
If the video source identifier indicates that the original SDR video was converted, and the conversion method/characteristic identifier indicates that the conversion was performed by direct mapping, the A video conversion device that converts an HDR video into the converted SDR video . The video conversion apparatus according to claim 1 , further comprising an identifier adding means for adding the video source identifier and the conversion method/characteristic identifier to the HDR video. A video conversion device that converts an HDR video to a converted SDR video based on an HDR video level,
a video determining means for receiving the HDR video and determining whether or not the input HDR video does not include a component higher than a predetermined video level;
a video conversion unit that determines the origin of the HDR video based on a determination result of the video determination unit, and converts the HDR video to the converted SDR video based on the determined origin of the HDR video;
Equipped with
The video conversion means includes:
If the determination result indicates that the HDR video does not contain a component higher than the predetermined video level, converting the HDR video to the converted SDR video using a conversion function that provides an inverse characteristic of direct mapping;
If the determination result indicates that the HDR video contains a component higher than the predetermined video level, the HDR video is converted to the converted SDR video by tone mapping. A program for causing a computer to function as the video conversion device according to any one of claims 1 to 3 .

Images