CN111292269A - Image tone mapping method, computer device and computer readable storage medium - Google Patents
Image tone mapping method, computer device and computer readable storage medium Download PDFInfo
- Publication number
- CN111292269A CN111292269A CN202010091029.0A CN202010091029A CN111292269A CN 111292269 A CN111292269 A CN 111292269A CN 202010091029 A CN202010091029 A CN 202010091029A CN 111292269 A CN111292269 A CN 111292269A
- Authority
- CN
- China
- Prior art keywords
- tone mapping
- operator
- image
- stage
- mapping operator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 347
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004364 calculation method Methods 0.000 claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 20
- 238000001914 filtration Methods 0.000 claims description 17
- 230000000694 effects Effects 0.000 description 24
- 125000001475 halogen functional group Chemical group 0.000 description 24
- 238000004422 calculation algorithm Methods 0.000 description 22
- 230000000903 blocking effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method for tone mapping of an image, a computer device and a computer readable storage medium, the method comprises the steps of obtaining an initial image, obtaining an initial brightness value of each pixel of the initial image, and carrying out tone mapping calculation on the initial brightness value of each pixel; setting more than two levels of tone mapping operators, wherein each level of tone mapping operator has fixed dynamic range mapping capability; and acquiring a bit width value of an initial brightness value of the pixel, determining the number of stages of the selected tone mapping operator according to the bit width value of the initial brightness value, and applying the determined tone mapping operator of at least one stage to perform mapping calculation on the initial brightness value. The invention also provides a computer device and a computer readable storage medium for realizing the method. The method can be suitable for tone mapping calculation of images in different scenes, and improves the image quality after tone mapping.
Description
Technical Field
The present invention relates to the technical field of image processing, and in particular, to an image tone mapping method, and a computer device and a computer-readable storage medium for implementing the method.
Background
Many existing intelligent electronic devices have an image shooting function, and for example, a smartphone, a tablet computer, a vehicle event data recorder and the like are all provided with a camera device. With the development and popularization of sensor technology, the dynamic range of images acquired by the image sensor is wider and wider. The precision of a CMOS (Complementary Metal Oxide Semiconductor) image sensor commonly used at present can usually reach 12bit to 14bit, that is, the luminance value of one pixel needs to be represented by binary data of 12bit to 14bit, while some wide dynamic images synthesized by multiple frames can reach a dynamic range close to human eyes.
However, most displays now support a codec standard of 8 bits, i.e. the luminance value of a pixel can only be represented using 8 bits of binary data. Of course, there are also a few displays that can achieve a dynamic range of 10 bits. However, even if the encoding and display dynamic range of 10 bits is supported, there is a great difference from the dynamic range of the acquisition side. At this time, in order to enable human eyes to better receive an image close to a real wide dynamic range, a tone mapping algorithm is required to perform mapping calculation on the luminance values of the pixels acquired by the cmos image sensor, that is, the luminance values with a large bit width are mapped to form the luminance values with a small bit width.
The existing tone mapping algorithms are mainly classified into three major categories, namely, global tone mapping algorithms, local tone mapping algorithms and hybrid tone mapping algorithms. The global tone mapping algorithm is that the whole image uses the same global tone mapping operator to perform tone mapping calculation, and has the advantages of high efficiency, strong scene adaptability and the like, and the defect is that the contrast of a part of brightness intervals is inevitably compressed, so that the image distortion is caused. The local tone mapping algorithm divides the whole image into a plurality of regions, and each region uses one local tone mapping operator to perform tone mapping calculation. The hybrid tone mapping algorithm is used for carrying out tone mapping calculation on an image by combining a global tone mapping operator and a local tone mapping operator, is realized by combining a global tone mapping method and a local tone mapping method, has the advantages of both the global tone mapping method and the local tone mapping method, but is only limited to a specific dynamic range mapping scene or a specific mapping operator combination for hybrid tone mapping at present and has no universality.
For example, the chinese patent application publication No. CN1740891A and the chinese patent publication No. CN101527038B disclose two tone mapping schemes consisting of fixed two-level global tone mapping operators, wherein the former generates a second-level tone mapping curve by analyzing the result of the first-level tone mapping, and further obtains a better mapping result; the latter avoids the overstretching of the image contrast by the histogram equalization operator of the second stage through the gamma operator of the first stage. Compared with the traditional global tone mapping method, the two schemes can achieve better tone mapping results, but still have the defects of a global mapping algorithm, namely the contrast of partial brightness regions is lost, and the contrast of each brightness region cannot be controlled more finely due to the limitation of a fixed series and a fixed operator.
The chinese patent inventions with publication numbers CN102682436B and CN102722868B disclose two local tone mapping algorithms based on Retinex theory, both of which reduce the halo effect of the local tone mapping operator by means of multi-scale edge-preserving filtering, but still have the problems of large computation amount and incapability of completely eliminating the halo effect.
Chinese patent publication No. CN104408752B discloses a two-stage hybrid tone mapping algorithm based on content-based global tone mapping plus bilateral filtering, and chinese patent publication No. CN103403759B discloses a two-stage hybrid tone mapping structure based on global tone mapping plus multi-scale local tone mapping, in which the former weakens the halo effect by bilateral filtering, and the latter weakens the halo effect by multi-scale mode. Compared with a single-stage global tone mapping operator, the two-stage global tone mapping operator has better local contrast maintenance; compared with a single-stage local tone mapping operator, the method has the advantages of halo effect suppression. Neither involves more levels of tone mapping schemes or how to accommodate more different dynamic range tone mapping application scenarios in a multi-level structure.
With the rise of display-side HDR display devices and encoding standards, the application of the tone mapping algorithm is expanded from an encoding side to a decoding side, the application range is wider and wider, and more dynamic range mapping scenes need to be processed. Typical application scenarios include an application at a camera head end (where an 8-bit to 24-bit luminance value needs to be mapped to an 8-bit to 10-bit luminance value), an application at a decoding end (where an 8-bit to 12-bit luminance value needs to be mapped to an 8-bit to 10-bit luminance value), an application at a mobile end (where an 8-bit to 12-bit luminance value needs to be mapped to an 8-bit to 10-bit luminance value) to save screen power consumption, an application at a professional wide dynamic image capture device (RGBE, OpenEXR, etc.) (where a luminance value exceeding 32-bit luminance value needs to be mapped to an 8-bit luminance value), and the like. The mapping range difference in the application scene is large, and the requirements for the mapping capability of the tone mapping operator are different. Because the existing tone mapping algorithm can not meet the mapping requirements under various scenes, the application scene of the tone mapping algorithm is single, and people need to set an adaptive mapping algorithm aiming at various scenes, so that the development efficiency of electronic products is influenced.
Disclosure of Invention
The invention mainly aims to provide an image tone mapping method capable of adaptively adjusting tone mapping operators according to requirements of different use scenes.
Another object of the present invention is to provide a computer apparatus for implementing the image tone mapping method.
It is still another object of the present invention to provide a computer readable storage medium for implementing the above-mentioned image tone mapping method.
In order to achieve the main object of the present invention, the image tone mapping method provided by the present invention comprises obtaining an initial image, obtaining an initial brightness value of each pixel of the initial image, and performing tone mapping calculation on the initial brightness value of each pixel; setting more than two levels of tone mapping operators, wherein each level of tone mapping operator has fixed dynamic range mapping capability; and acquiring a bit width value of an initial brightness value of the pixel, determining the number of stages of the selected tone mapping operator according to the bit width value of the initial brightness value, and applying the determined tone mapping operator of at least one stage to perform mapping calculation on the initial brightness value.
According to the scheme, the number of stages of the selected tone mapping operator can be determined according to the bit width value of the initialized brightness value of each pixel, so that multiple times of mapping calculation can be performed by using a multi-stage tone mapping operator for an image with a large bit width value, and one-stage or two-stage tone mapping operators can be performed once or twice for an image with a small bit width value. Therefore, the requirements of tone mapping calculation under different scenes can be met, and the tone mapping algorithm can be applied to electronic equipment with different types and different application scenes.
Preferably, the dynamic range mapping capability of each tone mapping operator is the same; or the dynamic range mapping capability of at least one tone mapping operator is different from the dynamic range mapping capability of another tone mapping operator.
It follows that, in order to facilitate the design of the tone mapping algorithm, the multi-level tone mapping operators can all be designed to have the same dynamic range mapping capability. In order to meet the use requirements of more scenes, the dynamic range mapping capability of the multi-level tone mapping operator can be designed to be different, so that the tone-mapped image is closer to a real color, and the distortion is smaller.
Further, the tone mapping operator is one of the following: a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator; wherein, the mixed tone mapping operator is the calculation value of the global tone mapping operator and the local tone mapping operator of the stage.
Therefore, by reasonably using the global tone mapping operator, the local tone mapping operator or the mixed tone mapping operator, the adaptability of the tone mapping algorithm under various different situations can be improved, and the use requirements under different situations can be met.
Further, the hybrid tone mapping operator is a calculation value of the global tone mapping operator and the local tone mapping operator of the level calculated by using a preset weight coefficient.
Therefore, the proportion of the global tone mapping operator and the local tone mapping operator in each level of mixed tone mapping operator can be reasonably configured by using the preset weight coefficient, the quality of the image after tone mapping calculation is improved, and the tone of the image after tone mapping is closer to the real color.
Preferably, the weight coefficients applied by the hybrid tone mapping operator at the subsequent stage are adjusted according to the gradient inversion coefficient of the mapping calculation result at the previous stage.
Therefore, the method can better solve the problem of halo effect after tone mapping, so that image halo after tone mapping is less, and the problem of halo introduced due to tone mapping is avoided.
In a further aspect, the global tone mapping operator of at least one level corresponds to a different luminance mapping curve than the global tone mapping operator of another level.
Therefore, the brightness mapping curves corresponding to the multilevel global tone mapping operators can be different, so that the contrast of different brightness intervals can be adjusted through each level of global tone mapping operators, the contrast of different intervals can be adjusted through multilevel adjustment, the brightness contrast can be adjusted more finely by tone mapping, and the tone mapping effect is more ideal.
The method comprises the following steps of carrying out mapping on the local tone of each stage of the image to obtain a mapping result, and carrying out mapping on the local tone of each stage of the image to obtain a mapping result.
Therefore, the size of the filtering window is gradually reduced, so that the problem that the image subjected to tone mapping has halo can be better controlled, and the problem that the image is distorted due to a large amount of halo effects of the image subjected to tone mapping is avoided.
The further scheme is that the block areas corresponding to the local tone mapping operators of two adjacent stages are staggered, and the block areas corresponding to the local tone mapping operators of each stage are not completely overlapped.
Therefore, the block regions corresponding to the multilevel local tone mapping operators are staggered, so that the phenomenon that the multilevel local tone mapping operators have the same block regions can be avoided, and the halo effect appears at the boundary of the blocks. Therefore, the halo effect after tone mapping can be effectively reduced by the above means.
In order to achieve the above another object, the present invention provides a computer device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program realizes the steps of the image tone mapping method when executed by the processor.
To achieve the above-mentioned further object, the present invention provides a computer program stored on a computer readable storage medium, which when executed by a processor implements the steps of the image tone mapping method.
Drawings
FIG. 1 is a flow chart of an embodiment of the image tone mapping method of the present invention.
FIG. 2 is a flow chart of an embodiment of the image tone mapping method of the present invention using a one-level mapping operator.
FIG. 3 is a flow chart of an embodiment of the image tone mapping method of the present invention using two levels of mapping operators.
FIG. 4 is a flow chart of an embodiment of the image tone mapping method of the present invention using three levels of mapping operators.
FIG. 5 is a flow chart of an embodiment of the image tone mapping method of the present invention using a four-level mapping operator.
FIG. 6 is a graph of luminance mapping corresponding to a three-level global tone mapping operator according to an embodiment of the image tone mapping method of the present invention.
FIG. 7 is a schematic diagram of a block region corresponding to two adjacent filtering operators according to an embodiment of the image tone mapping method of the present invention.
The invention is further explained with reference to the drawings and the embodiments.
Detailed Description
The image tone mapping method of the present invention is applied to an intelligent electronic device, and preferably, the intelligent electronic device is provided with an image pickup device, such as a camera, and the image pickup device has an image sensor, such as a CMOS, and the intelligent electronic device obtains an initial image by using the image pickup device. Preferably, the intelligent electronic device is provided with a processor and a memory, the memory having stored thereon a computer program, the processor implementing the image tone mapping method by executing the computer program.
Image tone mapping method embodiment:
the embodiment mainly performs tone mapping on an initial image acquired by an image sensor, that is, mapping an initial brightness value of each pixel in the image, so as to reduce a bit width of the initial brightness value, and meet display requirements of output devices, such as a display screen.
The present embodiment will be described in detail with reference to fig. 1. First, step S1 is executed to set a multilevel tone mapping operator. The tone mapping operator with fixed dynamic range mapping capability is designed as a basic unit in the embodiment, and then the tone mapping operator is mapped according to the dynamic range of the practical application. Thus, the tone mapping calculation can be determined by using one or more levels of tone mapping operators according to actual requirements. In one case, the multi-level tone mapping operator has the same dynamic range mapping capability, e.g. defines a basic tone mapping operator with a dynamic range mapping capability of 24db/4bit, i.e. the bit width of the luminance value of a pixel will be reduced by 4 bits each time a tone mapping calculation is performed one level. Therefore, for a display capable of supporting only 8-bit luminance values, if the bit width of the luminance value of each pixel of the input initial image is 12 bits, only one stage of calculation of the tone mapping operator is needed to meet the application requirement. If the bit width of the luminance value of each pixel of the input initial image is 16 bits, the calculation of the tone mapping operator needs to be performed in two stages.
Alternatively, the dynamic range mapping capabilities of the tone mapping operators of the multiple levels are different, for example, the dynamic range mapping capability of the tone mapping operator of one level is greater than the dynamic range mapping capability of the tone mapping operator of another level. In this case, the dynamic range mapping capability of the tone mapping operator of each stage is also preset.
Then, step S2 is executed to acquire an initial image and acquire an initial luminance value of each pixel. In this embodiment, the initial image is an image output by a CMOS image sensor, and the color information of the initial image may be RGB information. For RGB information, a linear calculation may be performed to obtain an image of YCbCr information, Y being the luminance value of a pixel, Cb being the blue chrominance value of a pixel, and Cr being the red chrominance value of a pixel. Since the technology of converting RGB information into YCbCr information is well known, it will not be described herein.
The bit width of the output initial brightness value is not necessarily the same for different CMOS image sensors, the bit width of the output initial brightness value of some CMOS image sensors is 24 bits, the bit width of the output initial brightness value of other CMOS image sensors is 20 bits or 16 bits, and since the display device can only process data with a brightness value of 8 bits, the mapping calculation of the initial brightness value of the image is required. This embodiment is to perform adaptive tone mapping calculation for images with different initial luminance value bit widths.
Then, step S3 is executed to acquire an initial luminance value of one pixel of the initial image, and acquire a bit width value of the initial luminance value. Generally, in the same image, the bit width value of the luminance value of each pixel is the same, so that only the initial luminance value of one of the pixels needs to be obtained, i.e. the bit width value of the initial luminance value of each pixel of the image can be determined.
Next, step S4 is executed to determine the number of stages of the tone mapping operator according to the bit width value of the initial luminance value. Since the bit width value of the luminance value of each pixel of the initial image is determined and the dynamic range mapping capability of the tone mapping operator at each level is also determined, it can be calculated how many levels of tone mapping calculation need to be performed, and therefore, step S4 can quickly calculate how many levels of tone mapping operators need to perform tone mapping calculation.
Then, step S5 is executed, and the initial luminance value of the pixel is tone-mapped using the multi-level tone mapping operator determined in step S4, for example, using a global tone mapping operator. Since the use of a tone mapping operator for tone mapping calculations is known in the art, it is not described in detail here.
Then, step S6 is executed to determine whether all pixels in the initial image have already performed the tone mapping calculation, if so, the tone mapping calculation of the image is completed, otherwise, the next pixel is obtained, i.e., step S7 is executed, and the process returns to step S5, and the tone mapping calculation is performed on the current pixel by using one or more tone mapping operators until all pixels of the initial image have been performed the tone mapping calculation.
For a display that can only handle 8 bits of luminance value, if the dynamic range mapping capability of the tone mapping operator at each level is set to 24db/4bit, the number of tone mapping operator stages used is different for different initial images. Referring to fig. 2, for the case that the luminance value of the pixel of the input initial image is only 12 bits, only one level tone mapping operator is needed to complete the tone mapping calculation. For example, step S11 is performed to obtain the initial luminance value of the pixel, where the initial luminance value is 12-bit data, and then step S12 is performed to perform the calculation of tone mapping by applying tone mapping operator 0. In the present embodiment, tone mapping operators from the first level to the fourth level are tone mapping operator 0, tone mapping operator 1, tone mapping operator 2, and tone mapping operator 3, respectively.
After the tone mapping operator 0 is applied to perform the tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 12 bits to 8 bits to meet the use requirement of the display screen, step S13 is executed to calculate the luminance gain, and finally step S14 is executed to output the mapped luminance value, where the luminance value of each pixel is 8-bit data.
If the bit width of the luminance value of each pixel of the input original image is 16 bits, the calculation of tone mapping needs to be performed using the flow shown in fig. 3. Firstly, step S21 is executed to obtain an initial brightness value of the pixel, step S22 is executed to perform the calculation of tone mapping by using the tone mapping operator 0, the bit width of the brightness value of the pixel is reduced from 16 bits to 12 bits, step S23 is executed to perform the calculation of tone mapping by using the tone mapping operator 1, the bit width of the brightness value of the pixel is reduced from 12 bits to 8 bits to meet the use requirement of the display screen, step S24 is executed to calculate the brightness gain, and step S25 is executed to output the mapped brightness value.
If the bit width of the luminance value of each pixel of the input original image is 20 bits, the calculation of tone mapping needs to be performed using the flow shown in fig. 4. Firstly, step S31 is executed to obtain an initial brightness value of the pixel, step S32 is executed to perform the calculation of tone mapping by applying the tone mapping operator 0, the bit width of the brightness value of the pixel is reduced from 20 bits to 16 bits, step S33 is executed to perform the calculation of tone mapping by applying the tone mapping operator 1, the bit width of the brightness value of the pixel is reduced from 16 bits to 12 bits, step S34 is executed to perform the calculation of tone mapping by applying the tone mapping operator 2, the bit width of the brightness value of the pixel is reduced from 12 bits to 8 bits, and the usage requirement of the display screen is met, step S35 is executed to calculate the brightness gain, and step S36 is executed to output the mapped brightness value.
If the bit width of the luminance value of each pixel of the input original image is 24 bits, the calculation of tone mapping needs to be performed using the flow shown in fig. 5. Firstly, step S41 is executed to obtain an initial brightness value of the pixel, step S42 is executed to perform the calculation of tone mapping by applying tone mapping operator 0, the bit width of the brightness value of the pixel is reduced from 24 bits to 20 bits, step S43 is executed to perform the calculation of tone mapping by applying tone mapping operator 1, the bit width of the brightness value of the pixel is reduced from 20 bits to 16 bits, step S44 is executed to perform the calculation of tone mapping by applying tone mapping operator 2, the bit width of the brightness value of the pixel is reduced from 16 bits to 12 bits, step S45 is executed to perform the calculation of tone mapping by applying tone mapping operator 3, the bit width of the brightness value of the pixel is reduced from 12 bits to 8 bits to meet the use requirement of the display screen, step S46 is executed to calculate the brightness gain, and step S47 is executed to output the mapped brightness value.
The tone mapping operator of this embodiment may be a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator. For a multi-level global tone mapping operator, different brightness mapping curves can be provided, so that the fine control of the contrast of different brightness regions is realized. FIG. 6 shows a luminance mapping curve corresponding to a global tone mapping operator with three levels of tone mapping operators, wherein FIG. 6(a) corresponds to a luminance mapping curve corresponding to tone mapping operator 0, and tone mapping operator 0 mainly improves the contrast ratio in the luminance interval [0, L0 ]; fig. 6(b) corresponds to a luminance mapping curve corresponding to tone mapping operator 1, where tone mapping operator 1 mainly improves the contrast within the luminance interval [ L0, L1 ]; fig. 6(c) corresponds to the luminance mapping curve corresponding to tone mapping operator 2, and tone mapping operator 2 mainly improves the contrast within the luminance interval [ L1, L2 ]. In practical applications, in order to obtain a better effect, the shape of the luminance mapping curve corresponding to the global tone mapping operator in the current-level tone mapping operator may be formulated by combining image content, for example, by using a histogram equalization method.
For the setting of the local tone mapping operator, the problem of balancing the halo effect and the local contrast needs to be considered. In order to balance the halo effect and improve the local contrast, the basic idea of the embodiment is to reduce or even eliminate the halo effect by designing the local filter coefficients of tone mapping operators at different levels in a differentiated manner. Specifically, the halo effect is reduced by means of local filtering, such as a retinal cortex (Retinex) theory, a tone mapping operator based on a base layer plus a detail layer, and the like, and by using different filtering operators for each stage, wherein a typical design manner is to use filtering windows with different scales for different stages. In this embodiment, each level of local tone mapping operator includes one filtering operator, and the filtering operators corresponding to each level of local tone mapping operator are different, for example, the filtering window corresponding to each level of tone mapping operator changes step by step, and as the tone mapping operator changes from the previous stage to the subsequent stage, the size of the corresponding filtering window changes from large to small, for example, the size of the filtering window corresponding to tone mapping operator 0 is 17 × 17, that is, the pixels of the filtering window are 17 rows and 17 columns; the size of the filter window for tone mapping operator 1 is 9 × 9, the size of the filter window for tone mapping operator 2 is 5 × 5, and so on.
The size of the conventional filter window is often fixed, and compared with the conventional filter window design scheme, the small-sized filter window is designed at a later stage, so that the contrast of a high-contrast area on the initial image is reduced remarkably, and the remarkable halo effect is reduced.
In practical implementation, for the local tone mapping operator based on the statistical information, such as local histogram equalization, etc., different blocking manners of different levels may be adopted to reduce the halo effect, as shown in fig. 7, in the blocking region corresponding to the local tone mapping operator of two adjacent levels, the blocking manner of the previous level is as shown in fig. 7(a), one local region is divided into 16 blocking regions, each blocking region is a square, that is, a square region with a length and a width of 2 r. The partitioning method of the next stage is, as shown in fig. 7(b), shifted from the previous stage, and for the same local region, the middle 9 partitioned regions are also square regions with length and width of 2r, but four corners are square regions with length and width of r, and the region near the borderline is a rectangular region with length of 2r and width of r.
Due to the local tone mapping algorithm, the region where the halo effect is likely to occur is often the intersection of two adjacent block regions. In the embodiment, the blocking areas of two adjacent stages are staggered with each other, and the blocking areas of each stage are not completely overlapped, that is, the areas of two adjacent stages where the halo effect may occur are staggered with each other, so that the halo effect occurring after tone mapping can be effectively reduced.
The hybrid tone mapping operator is an operator obtained by calculating a global tone mapping operator and a local tone mapping operator, and specifically, the global tone mapping operator and the local tone mapping operator at the level may be calculated by using a preset weighting coefficient. For example, assuming that I is a pixel of the initial image, G () is a global tone mapping operator of the stage, and L () is a local tone mapping operator of the stage, the hybrid tone mapping operator TM () corresponding to the stage can be calculated by the following formula:
tm (i) axg (i) +(1-a) × l (i) (formula 1)
The a is a preset weighting coefficient, which may be a global coefficient, that is, each pixel of the whole image uses the same weighting coefficient, or a local coefficient, that is, the weighting coefficient of the image is not fixed, but gradually changes according to a preset rule following the movement of the pixel.
For the mixed tone mapping algorithm with the two-stage structure, the tone mapping operator at each stage is a mixed tone mapping operator, and the mixed tone mapping operator at each stage is obtained by calculation by using a preset weighting coefficient, a global tone mapping operator and a local tone mapping operator. In addition, in the present embodiment, in the multi-level tone mapping algorithm, after the previous tone mapping operator is calculated, the weighting coefficient in the subsequent mixed tone mapping operator may be dynamically adjusted. Specifically, the weighting coefficient a of the next stage is locally and dynamically adjusted according to the gradient inversion condition of the mapping result of the previous stage, so as to obtain a better halo effect and local contrast balance result.
For example, the local gradient inversion coefficient may be defined by the following equation:
where Ω represents the window in which pixel i is located, j is the other pixels in the window, Ψ represents the condition for gradient inversion: (TM (j) -TM (i) x (I (j) -I (i) < 0). The larger the local gradient inversion coefficient is, the larger the weighting coefficient a corresponding to the next level is, so that the edge effect after tone mapping can be controlled, and the image quality after tone mapping can be improved.
Therefore, in the embodiment, by designing the multi-level tone mapping operator, each level of tone mapping operator has fixed mapping capability, and by using the multi-level tone mapping operator adaptively and flexibly, the rapid and flexible customization of different dynamic range mapping algorithms can be completed, and the requirements of image tone mapping in different scenes are met. In addition, when the global tone mapping operator is designed, the shape of the brightness mapping curve corresponding to each level of tone mapping operator is designed, so that the fine management of the contrast of each brightness interval is realized, and the tone mapping quality is improved. In addition, in the design of the local tone mapping operator, the purpose of weakening or even eliminating the halo effect is achieved through the design of the differentiation of the filter coefficients of all levels. And finally, when the mixed tone mapping operator is designed, the halo effect and the local contrast are better balanced through the combination processing of a more multi-level global tone mapping operator and a more multi-level local tone mapping operator.
The embodiment of the computer device comprises:
the computer apparatus of this embodiment may be an intelligent electronic device, and the computer apparatus includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of the image tone mapping method. Of course, the intelligent electronic device further includes a camera device for acquiring an initial image.
For example, a computer program may be partitioned into one or more modules that are stored in a memory and executed by a processor to implement the modules of the present invention. One or more of the modules may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program in the terminal device.
The Processor may be a Central Processing Unit (CPU), or may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center of the terminal device and connecting the various parts of the entire terminal device using various interfaces and lines.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
A computer-readable storage medium:
the computer program stored in the computer device may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the flow in the method according to the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium and used by a processor to implement the steps of the image tone mapping method.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.
Finally, it should be emphasized that the present invention is not limited to the above embodiments, such as the change of the weighting coefficients used by the hybrid tone mapping operator, or the change of the luminance mapping curve corresponding to the multilevel global tone operator, and the like, and these changes should also be included in the protection scope of the present invention.
Claims (10)
1. An image tone mapping method, comprising:
acquiring an initial image, acquiring an initial brightness value of each pixel of the initial image, and carrying out tone mapping calculation on the initial brightness value of each pixel;
the method is characterized in that:
setting more than two levels of tone mapping operators, wherein each level of tone mapping operator has fixed dynamic range mapping capability;
performing a tone mapping calculation on the initial luminance value of each of the pixels includes:
and acquiring a bit width value of the initial brightness value of the pixel, determining the number of stages of the selected tone mapping operator according to the bit width value of the initial brightness value, and applying the determined tone mapping operator of at least one stage to perform mapping calculation on the initial brightness value.
2. The image tone mapping method according to claim 1, characterized in that:
the dynamic range mapping capability of the tone mapping operator at each level is the same; or
The dynamic range mapping capability of at least one of said tone mapping operators is different from the dynamic range mapping capability of another of said tone mapping operators.
3. The image tone mapping method according to claim 1 or 2, characterized in that:
the tone mapping operator is one of: a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator;
wherein, the mixed tone mapping operator is a calculation value of the global tone mapping operator of the stage and the local tone mapping operator of the stage.
4. The image tone mapping method according to claim 3, characterized in that:
the mixed tone mapping operator is a calculated value of the global tone mapping operator of the level and the local tone mapping operator of the level, which is calculated by using a preset weight coefficient.
5. The image tone mapping method according to claim 4, characterized in that:
and adjusting and calculating the weight coefficient applied by the mixed tone mapping operator at the next stage according to the gradient inversion coefficient of the mapping calculation result at the previous stage.
6. The image tone mapping method according to claim 3, characterized in that:
the global tone mapping operator of at least one level corresponds to a different luminance mapping curve than the global tone mapping operator of another level.
7. The image tone mapping method according to claim 3, characterized in that:
each stage of the local tone mapping operator comprises a filtering operator, the filtering operators corresponding to the local tone mapping operators at each stage are different, and the size of a filtering window corresponding to the local tone mapping operator at a plurality of stages from a front stage to a rear stage is gradually reduced.
8. The image tone mapping method according to claim 7, characterized in that:
the block areas corresponding to the local tone mapping operators of two adjacent stages are staggered, and the block areas corresponding to the local tone mapping operators of each stage are not completely overlapped.
9. Computer arrangement, characterized in that it comprises a processor and a memory, said memory storing a computer program which, when executed by the processor, carries out the steps of the image tone mapping method according to any of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program, when executed by a processor, implements the steps of the image tone mapping method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010091029.0A CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010091029.0A CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111292269A true CN111292269A (en) | 2020-06-16 |
| CN111292269B CN111292269B (en) | 2023-09-01 |
Family
ID=71021374
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010091029.0A Active CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111292269B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052923A (en) * | 2021-03-31 | 2021-06-29 | 维沃移动通信（深圳）有限公司 | Tone mapping method, tone mapping apparatus, electronic device, and storage medium |
| CN113905183A (en) * | 2021-08-25 | 2022-01-07 | 珠海全志科技股份有限公司 | Chromatic aberration correction method and device for wide dynamic range image |
| CN115100031A (en) * | 2022-08-23 | 2022-09-23 | 摩尔线程智能科技(北京)有限责任公司 | Image processing method and image processing apparatus |
| CN115147290A (en) * | 2021-03-30 | 2022-10-04 | 北京小米移动软件有限公司 | Image processing method, apparatus and storage medium |
| CN115861096A (en) * | 2022-11-22 | 2023-03-28 | 瀚博半导体(上海)有限公司 | Image processing method and device and computer equipment |
| CN116894795A (en) * | 2023-09-11 | 2023-10-17 | 归芯科技(深圳)有限公司 | Image processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090003697A1 (en) * | 2007-06-27 | 2009-01-01 | Core Logic, Inc. | Non-linear tone mapping apparatus and method and computer readable medium stored thereon computer executable instructions for performing the method |
| CN103295194A (en) * | 2013-05-15 | 2013-09-11 | 中山大学 | Brightness-controllable and detail-preservation tone mapping method |
| CN106504278A (en) * | 2015-09-04 | 2017-03-15 | 迪士尼企业公司 | HDR tone maps |
| CN108198155A (en) * | 2017-12-27 | 2018-06-22 | 合肥君正科技有限公司 | A kind of self application tone mapping method and system |
-
2020
- 2020-02-13 CN CN202010091029.0A patent/CN111292269B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090003697A1 (en) * | 2007-06-27 | 2009-01-01 | Core Logic, Inc. | Non-linear tone mapping apparatus and method and computer readable medium stored thereon computer executable instructions for performing the method |
| CN103295194A (en) * | 2013-05-15 | 2013-09-11 | 中山大学 | Brightness-controllable and detail-preservation tone mapping method |
| CN106504278A (en) * | 2015-09-04 | 2017-03-15 | 迪士尼企业公司 | HDR tone maps |
| CN108198155A (en) * | 2017-12-27 | 2018-06-22 | 合肥君正科技有限公司 | A kind of self application tone mapping method and system |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115147290A (en) * | 2021-03-30 | 2022-10-04 | 北京小米移动软件有限公司 | Image processing method, apparatus and storage medium |
| CN113052923A (en) * | 2021-03-31 | 2021-06-29 | 维沃移动通信（深圳）有限公司 | Tone mapping method, tone mapping apparatus, electronic device, and storage medium |
| CN113052923B (en) * | 2021-03-31 | 2023-02-28 | 维沃移动通信（深圳）有限公司 | Tone mapping method, tone mapping apparatus, electronic device, and storage medium |
| CN113905183A (en) * | 2021-08-25 | 2022-01-07 | 珠海全志科技股份有限公司 | Chromatic aberration correction method and device for wide dynamic range image |
| CN113905183B (en) * | 2021-08-25 | 2024-04-16 | 珠海全志科技股份有限公司 | Color difference correction method and device for wide dynamic range image |
| CN115100031A (en) * | 2022-08-23 | 2022-09-23 | 摩尔线程智能科技(北京)有限责任公司 | Image processing method and image processing apparatus |
| CN115100031B (en) * | 2022-08-23 | 2023-03-14 | 摩尔线程智能科技(北京)有限责任公司 | Image processing method and image processing apparatus |
| CN115861096A (en) * | 2022-11-22 | 2023-03-28 | 瀚博半导体(上海)有限公司 | Image processing method and device and computer equipment |
| CN115861096B (en) * | 2022-11-22 | 2023-10-31 | 瀚博半导体(上海)有限公司 | Image processing method and device and computer equipment |
| CN116894795A (en) * | 2023-09-11 | 2023-10-17 | 归芯科技(深圳)有限公司 | Image processing method and device |
| CN116894795B (en) * | 2023-09-11 | 2023-12-26 | 归芯科技(深圳)有限公司 | Image processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111292269B (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111292269B (en) | Image tone mapping method, computer device, and computer-readable storage medium | |
| US7003153B1 (en) | Video contrast enhancement through partial histogram equalization | |
| US9501818B2 (en) | Local multiscale tone-mapping operator | |
| US6965416B2 (en) | Image processing circuit and method for processing image | |
| US8411991B2 (en) | Image processing apparatus, image processing method, and program | |
| US9076218B2 (en) | Method and image processing device for image dynamic range compression with local contrast enhancement | |
| US9147238B1 (en) | Adaptive histogram-based video contrast enhancement | |
| EP1494462A1 (en) | Image processing device and method | |
| US20110013834A1 (en) | Apparatus and method for feature-based dynamic contrast enhancement | |
| US8831346B2 (en) | Image processing apparatus and method, and program | |
| US8265391B1 (en) | Localized, adaptive video contrast enhancement using controlled histogram equalization | |
| US9177396B2 (en) | Image processing apparatus and image processing method | |
| CN111161188B (en) | Method for reducing image color noise, computer device and readable storage medium | |
| CN113935911A (en) | High dynamic range video image processing method, computer device and computer readable storage medium | |
| US11551336B2 (en) | Chrominance and luminance enhancing systems and methods | |
| WO2019104047A1 (en) | Global tone mapping | |
| CN115496668A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN114998122A (en) | Low-illumination image enhancement method | |
| CN112534466B (en) | Directional scaling system and method | |
| CN115499632A (en) | Image signal conversion processing method and device and terminal equipment | |
| JP2018019239A (en) | Imaging apparatus, control method therefor and program | |
| CN112598595A (en) | High-dynamic digital image display enhancement method and system based on gamma correction | |
| Hu et al. | Using adaptive tone mapping to enhance edge-preserving color image automatically | |
| US8587725B2 (en) | Method of digital signal processing | |
| KR101073497B1 (en) | Apparatus for enhancing image and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |