CN109544463A - The inverse tone mapping (ITM) method of image content-based - Google Patents
The inverse tone mapping (ITM) method of image content-based Download PDFInfo
- Publication number
- CN109544463A CN109544463A CN201811208020.2A CN201811208020A CN109544463A CN 109544463 A CN109544463 A CN 109544463A CN 201811208020 A CN201811208020 A CN 201811208020A CN 109544463 A CN109544463 A CN 109544463A
- Authority
- CN
- China
- Prior art keywords
- image
- hdr
- luminance
- tone mapping
- ldr
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 title claims abstract description 13
- 238000005192 partition Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000005316 response function Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to image procossings, image display technology field, and to propose new method, realization more accurately restores HDR information from LDR image, show optimal true effect in equipment to show in HDR.Thus, the technical solution adopted by the present invention is that, the inverse tone mapping (ITM) method of image content-based, image content-based, divide the image into highlight bar and diffusing reflection area, different brightness spread functions is used to different regions, and restores high dynamic range HDR information from common LDR image, and then show in HDR equipment.Present invention is mainly applied to image procossing, image shows occasion.
Description
Technical Field
The present invention relates to the field of image processing and image display technologies, and in particular, to an inverse tone mapping method based on image content.
Background
An image in a normal 8-bit encoding format cannot provide sufficient accuracy if it is to be displayed on a High Dynamic Range (HDR) display device, so that HDR information needs to be recovered from a normal image (LDR), which is called inverse tone mapping (inverse of camera response function). Although there is a mature process for restoring camera response in the multi-exposure fusion technique. But with only a single LDR image, without any camera information and parameters and scene information, finding a suitable camera response function becomes very difficult, which is also the case for a large number of traditional movie resources. Therefore, inverse tone mapping techniques are required to convert LDR to HDR.
The inverse tone mapping basic step is an extension of the dynamic range. Based on a suitable inverse gamma correction function or inverse tone mapping function or inverse camera response function, all pixel values in the LDR image are transformed to obtain linear pixel values, and then the linear pixel values are proportionally stretched to obtain the best display effect on the displayed HDR device. Rempel [1] and massa [2] et al propose a simple inverse gamma transform with gamma 2.2 to achieve well linearized pixel values, which is directly based on the inverse of the gamma correction. Farid [3] proposes on this basis that gamma values can be estimated from a single picture in the absence of camera calibration information. Banterle [4] et al linearize the LDR pixel values by the inverse of the tone mapping function, achieving good visual results, but because the extent of the banding artifact dynamic range extension for highlight areas is very limited, it is assumed that tone mapping is primarily an exponential function and a sigmod function, which have a large inhibitory effect on the dynamic range. Lin [5] suggests that for a single LDR image, the camera response curve can be reconstructed more accurately based on the distribution of color pixels near the edge of the object.
Reference documents:
[1]Allan G.Rempel,Matthew Trentacoste,Helge Seetzen,H.David Young,Wolf-gang Heidrich, Lorne Whitehead,and Greg Ward.Ldr2Hdr:On-the-fly reversetone mapping of legacy video and photographs.ACM Transactions on Graphics(Proc.SIGGRAPH),26(3),2007.Article 39.
[2]Belen Masia,Sandra Agustin,Roland W.Fleming,Olga Sorkine,and DiegoGutierrez. Evaluation of reverse tone mapping through varying exposure condi-tions.ACM Transactions on Graphics(Proc.SIGGRAPH Asia),28(5):160:1–160:8,2009.
[3]Hany Farid.Blind inverse gamma correction.IEEE Transactions onImage Processing,pages 1428–1433,2001
[4]Francesco Banterle,Alessandro Artusi,Kurt Debattista,and AlanChalmers.Advanced High Dynamic Range Imaging:Theory and Practice.AK Peters(CRC Press),Natick,MA,USA, 2011.
[5]Stephen Lin,Jinwei Gu,Shuntaro Yamazaki,and Heung-YeungShum.Radio-metric calibration from a single image.Conference on ComputerVision and Pattern Recognition (CVPR’04),02:938–945,2004.
disclosure of Invention
To overcome the deficiencies of the prior art, the present invention aims to propose a new algorithm to enable a more accurate recovery of HDR information from LDR images, thereby displaying an optimal realistic effect on HDR display devices. Therefore, the technical scheme adopted by the invention is that an inverse tone mapping method based on image content divides an image into a highlight area and a diffuse reflection area based on the image content, adopts different brightness expansion functions for different areas, recovers high dynamic range HDR information from an ordinary LDR image, and further displays the high dynamic range HDR information on HDR equipment.
Specifically, the method comprises the following steps:
(1) inputting LDR image, based on brightness image Ld(x, y), calculating a partition threshold w,dividing the LDR image into highlight and diffusion regions;
(2) by different brightness spread functions Lw(x, y) performing brightness expansion;
if L isd(x, y) > w, the region is a highlight region, and the luminance spread function is:
Lw(x,y)＝s1w+s2(Ld(x,y)-w)
otherwise, the area is a diffuse reflection area, and the brightness expansion function is:
Lw(x,y)＝s1Ld(x,y)
wherein,ρ represents the percentage of diffuse reflectance luminance in the HDR display device luminance, a constant defined by the viewer;
(3) the output image is an HDR image containing HDR information.
Further, the calculation of the partition threshold w
Firstly, filtering the brightness image by using median filters with the size of m and the size of 2m +1 respectively, wherein m is max (image width and image height)/50, and respectively calculating the maximum value of the brightness of the filtered image as thresholds t1 and t 2; secondly, calculating a mask M with the size of 5 × 5 according to the values of the threshold values t1 and t2, and performing median filtering according to M; and finally, calculating the minimum value of the brightness of the filtered image as a partition threshold value w.
The invention has the characteristics and beneficial effects that:
based on the image content, the image is divided into a highlight area and a diffuse reflection area, different brightness expansion functions are adopted for different areas, the LDR image or video recovers HDR information through the algorithm, and a good visual effect is obtained on HDR equipment.
Description of the drawings:
figure 1 the overall solution of the invention.
Fig. 2 calculation of the partition threshold w.
Detailed Description
The invention adopts the inverse tone mapping algorithm based on the image content and adopts different linear expansion coefficients for different areas based on the image content, thereby obtaining the best display effect on HDR equipment.
The overall technical scheme of the invention is shown in figure 1.
(1) Inputting an LDR image based on the luminance image (L)d(x, y)), calculating a partitioning threshold w, dividing the LDR image into highlight and diffuse regions;
(2) by different brightness spread functions (L)w(x, y)) performing brightness expansion;
if L isd(x, y) > w, the region is a highlight region, and the luminance spread function is:
Lw(x,y)＝s1w+s2(Ld(x,y)-w)
otherwise, the area is a diffuse reflection area, and the brightness expansion function is:
Lw(x,y)＝s1Ld(x,y)
wherein,p represents the percentage of diffuse reflected luminance in the HDR display device luminance, a constant defined by the viewer.
(3) The output image is an HDR image containing HDR information.
Further:
(1) calculation of partition threshold w
As shown in fig. 2, first, the luminance image is filtered by median filters having a size of m (m ═ max (image width, image height)/50) and a size of 2m +1, and the filtered image luminance maximum value is calculated as thresholds t1, t2, respectively. Next, a mask M of size 5 × 5 is calculated with the values of the thresholds t1, t2, and median filtered with M. And finally, calculating the minimum value of the brightness of the filtered image as a partition threshold value w.
(2) Selection of rho
In luminance expansion, the percentage of diffuse reflected luminance at the HDR display device luminance, i.e., ρ, is considered. Psychology studies have shown that the human eye observes that in outdoor scenes it is inclined to assign a small fraction of the dynamic range to the highlight regions to obtain a bright overall scene, i.e. the values of p are high, whereas in indoor scenes it is inclined more to assign a larger dynamic range to the highlight regions, i.e. the values of p are small.
The invention provides an inverse tone mapping algorithm based on image content, which divides an image into a highlight area and a diffuse reflection area based on the image content, adopts different brightness expansion functions for different areas, and recovers HDR information from an LDR image, thereby obtaining the best display effect on HDR equipment.
Claims (3)
1. An inverse tone mapping method based on image content is characterized in that an image is divided into a highlight area and a diffuse reflection area based on the image content, different brightness expansion functions are adopted for different areas, high dynamic range HDR information is recovered from an ordinary LDR image, and then the high dynamic range HDR information is displayed on HDR equipment.
2. The method of inverse tone mapping based on image content according to claim 1, characterized in that in particular:
(1) inputting LDR image based on luminance mapLike Ld(x, y) calculating a partition threshold w, dividing the LDR image into highlight and diffuse regions;
(2) by different brightness spread functions Lw(x, y) performing brightness expansion;
if L isd(x, y) > w, the region is a highlight region, and the luminance spread function is:
Lw(x,y)＝s1w+s2(Ld(x,y)-w)
otherwise, the area is a diffuse reflection area, and the brightness expansion function is:
Lw(x,y)＝s1Ld(x,y)
wherein,ρ represents the percentage of diffuse reflectance luminance in the HDR display device luminance, a constant defined by the viewer;
(3) the output image is an HDR image containing HDR information.
3. The method of image content-based inverse tone mapping according to claim 1, further wherein the partition threshold w is calculated by first filtering the luminance image with median filters of size m and size 2m +1, respectively, m-max (image width, image height)/50, and calculating the maximum value of the filtered image luminance as thresholds t1, t 2; secondly, calculating a mask M with the size of 5 × 5 according to the values of the threshold values t1 and t2, and performing median filtering according to M; and finally, calculating the minimum value of the brightness of the filtered image as a partition threshold value w.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208020.2A CN109544463A (en) | 2018-10-17 | 2018-10-17 | The inverse tone mapping (ITM) method of image content-based |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208020.2A CN109544463A (en) | 2018-10-17 | 2018-10-17 | The inverse tone mapping (ITM) method of image content-based |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109544463A true CN109544463A (en) | 2019-03-29 |
Family
ID=65844315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811208020.2A Pending CN109544463A (en) | 2018-10-17 | 2018-10-17 | The inverse tone mapping (ITM) method of image content-based |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109544463A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036991A1 (en) * | 2019-08-30 | 2021-03-04 | 华为技术有限公司 | High dynamic range video generation method and device |
| CN113518185A (en) * | 2020-12-30 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Video conversion processing method and device, computer readable medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463820A (en) * | 2014-10-29 | 2015-03-25 | 广东工业大学 | Reverse tone mapping algorithm based on frequency domain |
| CN105850114A (en) * | 2013-12-27 | 2016-08-10 | 汤姆逊许可公司 | Method for inverse tone mapping of an image |
| CN108027961A (en) * | 2015-08-31 | 2018-05-11 | 汤姆逊许可公司 | Method and apparatus for inverse tone mapping (ITM) |
-
2018
- 2018-10-17 CN CN201811208020.2A patent/CN109544463A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105850114A (en) * | 2013-12-27 | 2016-08-10 | 汤姆逊许可公司 | Method for inverse tone mapping of an image |
| CN104463820A (en) * | 2014-10-29 | 2015-03-25 | 广东工业大学 | Reverse tone mapping algorithm based on frequency domain |
| CN108027961A (en) * | 2015-08-31 | 2018-05-11 | 汤姆逊许可公司 | Method and apparatus for inverse tone mapping (ITM) |
Non-Patent Citations (1)
| Title |
|---|
| LAURENCE MEYLAN ET AL.: "The Reproduction of Specular Highlights on High Dynamic Range Displays", 《IS&T/SID 14TH COLOR IMAGING CONFERENCE (CIC)》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036991A1 (en) * | 2019-08-30 | 2021-03-04 | 华为技术有限公司 | High dynamic range video generation method and device |
| CN112449120A (en) * | 2019-08-30 | 2021-03-05 | 华为技术有限公司 | High dynamic range video generation method and device |
| CN112449120B (en) * | 2019-08-30 | 2022-06-10 | 华为技术有限公司 | High dynamic range video generation method and device |
| CN113518185A (en) * | 2020-12-30 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Video conversion processing method and device, computer readable medium and electronic equipment |
| CN113518185B (en) * | 2020-12-30 | 2022-08-05 | 腾讯科技（深圳）有限公司 | Video conversion processing method and device, computer readable medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105745914B (en) | Method and system for inverse tone mapping (ITM) | |
| JP5457532B2 (en) | Increase the dynamic range of images | |
| US7724952B2 (en) | Object matting using flash and no-flash images | |
| Rao et al. | A Survey of Video Enhancement Techniques. | |
| JP5139293B2 (en) | Imaging camera processing apparatus and imaging camera processing method | |
| US8538147B2 (en) | Methods and appartuses for restoring color and enhancing electronic images | |
| Wang et al. | A fast single-image dehazing method based on a physical model and gray projection | |
| WO2019047985A1 (en) | Image processing method and device, electronic device, and computer-readable storage medium | |
| JP2007095073A (en) | Method and system for extracting alpha matte from image acquired of scene | |
| Kuo et al. | Content-adaptive inverse tone mapping | |
| WO2019056549A1 (en) | Image enhancement method, and image processing device | |
| WO2019101005A1 (en) | Pixel compensation method and apparatus, and terminal device | |
| TWI538492B (en) | Video clip search method | |
| Kinoshita et al. | Automatic exposure compensation using an image segmentation method for single-image-based multi-exposure fusion | |
| Guthier et al. | Flicker reduction in tone mapped high dynamic range video | |
| US20220351701A1 (en) | Method and device for adjusting image quality, and readable storage medium | |
| TWI426779B (en) | Object color correction method, system and its computer program products | |
| CN109544463A (en) | The inverse tone mapping (ITM) method of image content-based | |
| CN105516674B (en) | A kind of supervision equipment with HDR display functions | |
| KR102136716B1 (en) | Apparatus for Improving Image Quality and Computer-Readable Recording Medium with Program Therefor | |
| Huang et al. | High dynamic range tone mapping based on per-pixel exposure mapping | |
| Gouiffès et al. | HTRI: High time range imaging | |
| CN106651815B (en) | Method and device for processing Bayer format video image | |
| Fang et al. | Detail maintained low-light video image enhancement algorithm | |
| CN109936709B (en) | Image extraction method based on time information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190329 |