CN111292269B - Image tone mapping method, computer device, and computer-readable storage medium - Google Patents
Image tone mapping method, computer device, and computer-readable storage medium Download PDFInfo
- Publication number
- CN111292269B CN111292269B CN202010091029.0A CN202010091029A CN111292269B CN 111292269 B CN111292269 B CN 111292269B CN 202010091029 A CN202010091029 A CN 202010091029A CN 111292269 B CN111292269 B CN 111292269B
- Authority
- CN
- China
- Prior art keywords
- tone mapping
- stage
- operator
- image
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 349
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000004364 calculation method Methods 0.000 claims abstract description 49
- 238000004590 computer program Methods 0.000 claims description 21
- 238000001914 filtration Methods 0.000 claims description 14
- 238000004422 calculation algorithm Methods 0.000 description 24
- 230000000694 effects Effects 0.000 description 24
- 230000000903 blocking effect Effects 0.000 description 9
- 238000013461 design Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 125000001475 halogen functional group Chemical group 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000011284 combination treatment Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Abstract
The invention provides an image tone mapping method, a computer device and a computer readable storage medium, wherein the method comprises the steps of obtaining an initial image, obtaining an initial brightness value of each pixel of the initial image, and carrying out tone mapping calculation on the initial brightness value of each pixel; setting tone mapping operators with more than two stages, wherein each stage of tone mapping operator has fixed dynamic range mapping capability; obtaining the bit width value of the initial brightness value of the pixel, determining the number of stages of the tone mapping operators used according to the bit width value of the initial brightness value, and applying the determined tone mapping operators of at least one stage to perform mapping calculation on the initial brightness value. The invention also provides a computer device for realizing the method and a computer readable storage medium. The invention can be suitable for the tone mapping calculation of images in different scenes, and improves the image quality after tone mapping.
Description
Technical Field
The present invention relates to the technical field of image processing, and in particular, to an image tone mapping method, and a computer apparatus and a computer readable storage medium for implementing the method.
Background
Many existing intelligent electronic devices have an image shooting function, for example, a smart phone, a tablet computer, a vehicle recorder and the like are all provided with a camera device. With the development and popularization of sensor technology, the dynamic range of the image acquired by the image sensor is wider and wider. CMOS (Complementary Metal Oxide Semiconductor ) image sensors commonly used today can usually reach 12-14 bits of accuracy, i.e. the brightness value of a pixel needs to be represented by using 12-14 bits of binary data, while some wide dynamic images synthesized by multiple frames can reach a dynamic range close to the human eye.
However, most displays now support the codec standard of 8 bits, i.e. the luminance value of one pixel can only be represented using 8 bits of binary data. Of course, there are also a few displays that can achieve a 10bit dynamic range. However, even if the 10bit coding and display dynamic range is supported, there is a large difference from the dynamic range at the acquisition end. At this time, in order to make the human eye better receive the image approaching to the real wide dynamic range, it is necessary to use a tone mapping algorithm to perform a mapping calculation on the luminance values of the pixels collected by the COMS image sensor, that is, map the luminance values with larger bit width into the luminance values with smaller bit width.
Existing tone mapping algorithms are mainly divided into three categories, namely global tone mapping algorithm, local tone mapping algorithm and hybrid tone mapping algorithm. The global tone mapping algorithm is that the whole image uses the same global tone mapping operator to carry out tone mapping calculation, and has the advantages of high efficiency, strong scene adaptability and the like, and the defects are that partial brightness interval contrast is inevitably compressed, so that the image distortion is caused. The local tone mapping algorithm divides the whole image into a plurality of areas, each area uses a local tone mapping operator to carry out tone mapping calculation, the algorithm can better keep the detail contrast of each brightness interval, and the disadvantage is that the calculation amount is large and the halation effect is easy to generate. The mixed tone mapping algorithm is used for carrying out tone mapping calculation on an image by combining a global tone mapping operator and a local tone mapping operator, is realized by combining a global tone mapping method and a local tone mapping method, has the advantages of the global tone mapping operator and the local tone mapping method, is only limited to a specific dynamic range mapping scene or a specific mapping operator combination for mixed tone mapping, and has no universality.
For example, chinese patent application publication No. CN1740891a and chinese patent publication No. CN101527038B disclose two tone mapping schemes consisting of a fixed two-stage global tone mapping operator, wherein the former generates a second-stage tone mapping curve by analyzing the result of the first-stage tone mapping, and thus obtains a better mapping result; the latter avoids over stretching of the image contrast by the second level histogram equalization operator by the first level gamma operator. Compared with the traditional global tone mapping method, the two schemes can realize better tone mapping results, but still have the defect of the global mapping algorithm, namely the contrast of part of brightness areas is lost, and meanwhile, the contrast of each brightness area cannot be controlled more finely due to the fixed number of stages and fixed operators.
The Chinese patent publication No. CN102682436B, CN102722868B discloses two local tone mapping algorithms based on Retinex theory, which weaken the halation effect problem of the local tone mapping operator by a multi-scale edge-preserving filtering mode, but still have the problems of large operation amount, incapability of thoroughly eliminating the halation effect and the like.
The chinese patent publication No. CN104408752B discloses a content-based two-stage hybrid tone mapping algorithm with global tone mapping and bilateral filtering, and the chinese patent publication No. CN103403759B discloses a two-stage hybrid tone mapping structure with global tone mapping and multiscale local tone mapping, wherein the former reduces the halation effect by bilateral filtering, and the latter reduces the halation effect by multiscale. Compared with a single-stage global tone mapping operator, the two have better local contrast preservation; compared to single-stage local tone mapping operators, it is advantageous in terms of halation suppression. But neither involves more levels of tone mapping schemes or how to accommodate more different dynamic ranges of tone mapping application scenarios in a multi-level structure.
With the rise of display terminal HDR display equipment and coding standards, the application of tone mapping algorithm is extended from the coding terminal to the decoding terminal, the application range is wider and wider, and the dynamic range mapping scenes required to be processed are more and more. Typical application scenarios include applications at the camera end (which requires mapping 8-to 24-bit luminance values to 8-to 10-bit luminance values), applications at the decoding end (which requires mapping 8-to 12-bit luminance values to 8-to 10-bit luminance values), mobile end applications to save screen power consumption (which requires mapping 8-to 12-bit luminance values to 8-to 10-bit luminance values), applications at professional-wide dynamic image acquisition devices (RGBE, openEXR, etc. formats) (which require mapping luminance values exceeding 32-bit to 8-bit luminance values), etc. The mapping range in the application scene is large in difference, and the requirements for the mapping capability of the tone mapping operator are also different. Because the existing tone mapping algorithm cannot meet the mapping requirements under various scenes, the tone mapping algorithm is single in application scene, and people need to set an adaptive mapping algorithm for various scenes, so that the development efficiency of electronic products is affected.
Disclosure of Invention
The main object of the present invention is to provide an image tone mapping method capable of adaptively adjusting tone mapping operators according to the requirements of different usage scenes.
Another object of the present invention is to provide a computer apparatus for implementing the above image tone mapping method.
It is a further object of the present invention to provide a computer readable storage medium embodying the above image tone mapping method.
In order to achieve the main purpose of the invention, the image tone mapping method provided by the invention comprises the steps of obtaining an initial image, obtaining an initial brightness value of each pixel of the initial image, and carrying out tone mapping calculation on the initial brightness value of each pixel; setting tone mapping operators with more than two stages, wherein each stage of tone mapping operator has fixed dynamic range mapping capability; obtaining the bit width value of the initial brightness value of the pixel, determining the number of stages of the tone mapping operators used according to the bit width value of the initial brightness value, and applying the determined tone mapping operators of at least one stage to perform mapping calculation on the initial brightness value.
As can be seen from the above solution, the number of levels of the tone mapping operators to be used can be determined according to the bit width value of the initialized luminance value of each pixel, so that multiple mapping calculations can be performed by using the multi-level tone mapping operators for images with larger bit width values, and one or two mapping calculations can be performed by using one or two tone mapping operators for images with smaller bit width values. In this way, the requirements of tone mapping calculation under different scenes can be met, and the tone mapping algorithm can be applied to electronic equipment with a plurality of different types and different application scenes.
Preferably, the dynamic range mapping capability of each tone mapping operator is the same; or the dynamic range mapping capability of at least one level of tone mapping operator is different from the dynamic range mapping capability of another level of tone mapping operator.
It follows that to facilitate the design of the tone mapping algorithm, the multi-level tone mapping operators can all be designed to have the same dynamic range mapping capability. In order to meet the use requirements of more scenes, the dynamic range mapping capability of the multi-level tone mapping operator can be designed to be different, so that the tone mapped image is closer to the real color, and the distortion is smaller.
Further, the tone mapping operator is one of the following: a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator; wherein the blended tone mapping operator is the calculated value of the global tone mapping operator and the local tone mapping operator of the stage.
In this way, by reasonably using the global tone mapping operator, the local tone mapping operator or the mixed tone mapping operator, the adaptability of tone mapping algorithm under various different situations can be improved, and the use requirements under different scenes can be met.
Still further, the mixed tone mapping operator is a calculated value of the global tone mapping operator and the local tone mapping operator of the stage calculated using a preset weight coefficient.
Therefore, the specific gravity of the global tone mapping operator and the local tone mapping operator in each level of mixed tone mapping operator can be reasonably configured by using the preset weight coefficient, the quality of the tone mapping calculated image is improved, and the tone of the tone mapping calculated image is more similar to the real color.
In a preferred embodiment, the weighting factor applied by the mixed tone mapping operator of the subsequent stage is adjusted according to the gradient inversion factor of the mapping calculation result of the previous stage.
Therefore, the problem of halation effect after tone mapping can be better solved by the means, so that the halation of the image after tone mapping is less, and the problem of halation caused by tone mapping is avoided.
In a further aspect, the luminance mapping curve corresponding to the global tone mapping algorithm of at least one stage is different from the luminance mapping curve corresponding to the global tone mapping operator of another stage.
Therefore, the brightness mapping curves corresponding to the multi-level global tone mapping operators can be different, so that the contrast of different brightness intervals can be adjusted through each level of global tone mapping operator, and the adjustment of the tone mapping to the brightness contrast is finer through the adjustment of the multi-level contrast of different intervals, so that the tone mapping effect is more ideal.
In a further scheme, each stage of local tone mapping operator comprises a filtering operator, the filtering operators corresponding to the local tone mapping operators of each stage are different, and the size of a filtering window corresponding to the multi-stage local tone mapping operator of the front stage is gradually reduced from the front stage to the rear stage.
Therefore, the size of the filtering window is gradually reduced, so that the problem of halation of the tone-mapped image can be better controlled, and the problem of image distortion caused by a large amount of halation effect of the tone-mapped image is avoided.
In a further scheme, the partitioned areas corresponding to the local tone mapping operators of two adjacent stages are staggered, and the partitioned areas corresponding to the local tone mapping operators of each stage are not completely overlapped.
It can be seen that, since the block areas corresponding to the multi-level local tone mapping operators are staggered from each other, the multi-level local tone mapping operators have the same block area, so that the halation effect occurs at the boundary of the blocks. Therefore, the halation effect after tone mapping can be effectively reduced by the means.
To achieve the above another object, the present invention provides a computer apparatus including a processor and a memory storing a computer program which, when executed by the processor, implements the steps of the above-described image tone mapping method.
To achieve still another object of the present invention, there is provided a computer program stored on a computer readable storage medium, which when executed by a processor, implements the steps of the above-described image tone mapping method.
Drawings
FIG. 1 is a flow chart of an embodiment of an image tone mapping method of the present invention.
FIG. 2 is a flow chart of an embodiment of an image tone mapping method of the present invention using a primary mapping operator.
FIG. 3 is a flow chart of an embodiment of an image tone mapping method of the present invention using two levels of mapping operators.
FIG. 4 is a flow chart of an embodiment of an image tone mapping method of the present invention using three levels of mapping operators.
FIG. 5 is a flow chart of an embodiment of an image tone mapping method of the present invention using a four-level mapping operator.
FIG. 6 is a graph of luminance mapping corresponding to a three-level global tone mapping operator for an embodiment of an image tone mapping method of the present invention.
FIG. 7 is a schematic diagram of a block region corresponding to adjacent two-stage filtering operators according to an embodiment of the image tone mapping method of the present invention.
The invention is further described below with reference to the drawings and examples.
Detailed Description
The image tone mapping method of the present invention is applied to an intelligent electronic device, preferably, the intelligent electronic device is provided with an image pickup device, such as a camera, and the image pickup device is provided with an image sensor, such as a CMOS, and the intelligent electronic device acquires an initial image by using the image pickup device. Preferably, the intelligent electronic device is provided with a processor and a memory, the memory having stored thereon a computer program, the processor implementing the image tone mapping method by executing the computer program.
Image tone mapping method embodiment:
the embodiment mainly performs tone mapping on an initial image acquired by an image sensor, that is, maps an initial brightness value of each pixel in the image, so as to reduce the bit width of the initial brightness value and meet the display requirement of an output device, such as a display screen.
The present embodiment will be described in detail with reference to fig. 1. First, step S1 is performed to set a multi-level tone mapping operator. The present embodiment designs a tone mapping operator with a fixed dynamic range mapping capability as a basic unit, and then maps the requirement according to the dynamic range of the actual application. In this way, it may be determined to use one or more levels of tone mapping operators for tone mapping calculations based on actual requirements. One case is that the multi-level tone mapping operator has the same dynamic range mapping capability, e.g. a basic tone mapping operator is defined, which operator has a dynamic range mapping capability of 24db/4bit, i.e. the bit width of the luminance value of a pixel will be reduced by 4 bits per tone mapping calculation of one level. Therefore, for a display capable of supporting only 8-bit luminance values, if the bit width of the luminance value of each pixel of the input initial image is 12 bits, only one level of calculation of the tone mapping operator is needed to meet the application requirement. If the bit width of the luminance value of each pixel of the input initial image is 16 bits, two-stage calculation of the tone mapping operator is required.
Alternatively, the dynamic range mapping capabilities of the multiple levels of tone mapping operators are different, e.g., where the dynamic range mapping capability of one level of tone mapping operator is greater than the dynamic range mapping capability of another level of tone mapping operator. In this case, the dynamic range mapping capability of the tone mapping operator of each stage is also preset.
Then, step S2 is performed to acquire an initial image and an initial luminance value of each pixel. In this embodiment, the initial image is an image output by the CMOS image sensor, and the color information of the initial image may be RGB information. For RGB information, a linear calculation may be performed to obtain an image of YCbCr information, Y being a luminance value of a pixel, cb being a blue chrominance value of the pixel, and Cr being a red chrominance value of the pixel. Since the technology of converting RGB information into YCbCr information is a well-known technology, it will not be described in detail herein.
For different CMOS image sensors, the bit width of the initial luminance value output by some CMOS image sensors is not necessarily the same, the bit width of the initial luminance value output by some CMOS image sensors is 24 bits, and the bit width of the initial luminance value output by other CMOS image sensors is 20 bits or 16 bits, and since the display device is often capable of processing only 8 bits of data, it is necessary to perform mapping calculation on the initial luminance value of the image. The present embodiment performs adaptive tone mapping calculation for images having different bit widths of the initial luminance values.
Then, step S3 is performed to acquire an initial luminance value of one pixel of the initial image, and to acquire a bit width value of the initial luminance value. Generally, the bit width value of the luminance value of each pixel in the same image is the same, so that the bit width value of the initial luminance value of each pixel in the image can be determined only by acquiring the initial luminance value of one pixel.
Next, step S4 is performed to determine the number of tone mapping operator stages according to the bit width value of the initial luminance value. Since the bit width value of the luminance value of each pixel of the initial image is already determined and the dynamic range mapping capability of each level of tone mapping operator is also determined, it is possible to calculate how many levels of tone mapping calculation are required, and therefore, step S4 can quickly calculate how many levels of tone mapping operators are required to perform tone mapping calculation.
Then, step S5 is performed to perform tone mapping calculation on the initial luminance value of the pixel using the multi-level tone mapping operator determined in step S4, for example, using a global tone mapping operator. Since the use of tone mapping operators for tone mapping calculations is known in the art, no further description is provided herein.
Then, step S6 is executed to determine whether all pixels in the initial image have been subjected to tone mapping calculation, if so, the tone mapping calculation of the image is completed, otherwise, the next pixel is acquired, that is, step S7 is executed, and step S5 is executed again, and tone mapping calculation is performed on the current pixel by using one or more tone mapping operators until all pixels in the initial image have been subjected to tone mapping calculation.
For displays that can only handle luminance values of 8 bits, if the dynamic range mapping capability of each level of tone mapping operator is set to 24db/4bit, the tone mapping operator progression used is not the same for different initial images. Referring to fig. 2, for the case where the luminance value of the pixel of the input initial image is only 12 bits, the calculation of tone mapping can be completed by using only a primary tone mapping operator. For example, step S11 is performed first to obtain an initial luminance value of the pixel, where the initial luminance value is 12 bits of data, and then step S12 is performed to apply the tone mapping operator 0 to perform tone mapping calculation. In the present embodiment, tone mapping operators of the first stage to the fourth stage are tone mapping operator 0, tone mapping operator 1, tone mapping operator 2, and tone mapping operator 3, respectively.
After the tone mapping operator 0 is applied to perform tone mapping calculation, the bit width of the brightness value of each pixel is reduced from 12 bits to 8 bits, so that the use requirement of the display screen is met, step S13 is executed to calculate the brightness gain, and finally step S14 is executed to output the mapped brightness value, wherein the brightness value of each pixel is 8 bits of data.
If the bit width of the luminance value of each pixel of the input initial image is 16 bits, calculation of tone mapping is required using the flow shown in fig. 3. Firstly, step S21 is executed to obtain an initial luminance value of a pixel, then step S22 is executed to apply tone mapping operator 0 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 16 bits to 12 bits, then step S23 is executed to apply tone mapping operator 1 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 12 bits to 8 bits, thus meeting the use requirement of the display screen, step S24 is executed to calculate the luminance gain, and finally step S25 is executed to output the mapped luminance value.
If the bit width of the luminance value of each pixel of the input initial image is 20 bits, calculation of tone mapping is required using the flow shown in fig. 4. Firstly, step S31 is executed to obtain an initial luminance value of a pixel, then step S32 is executed to apply tone mapping operator 0 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 20 bits to 16 bits, then step S33 is executed to apply tone mapping operator 1 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 16 bits to 12 bits, then step S34 is executed to apply tone mapping operator 2 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 12 bits to 8 bits, thereby meeting the use requirement of the display screen, then step S35 is executed to calculate the luminance gain, and finally step S36 is executed to output the mapped luminance value.
If the bit width of the luminance value of each pixel of the input initial image is 24 bits, calculation of tone mapping is required using the flow shown in fig. 5. Firstly, step S41 is executed to obtain an initial luminance value of a pixel, then step S42 is executed to apply tone mapping operator 0 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 24 bits to 20 bits, then step S43 is executed to apply tone mapping operator 1 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 20 bits to 16 bits, then step S44 is executed to apply tone mapping operator 2 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 16 bits to 12 bits, then step S45 is executed to apply tone mapping operator 3 to perform tone mapping calculation, the bit width of the luminance value of the pixel is reduced from 12 bits to 8 bits, and the use requirement of the display screen is met, then step S46 is executed to calculate the luminance gain, finally step S47 is executed to output the mapped luminance value.
The tone mapping operator of this embodiment may be a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator. For a multi-level global tone mapping operator, there may be different luminance mapping curves, thereby enabling fine control of the contrast of different luminance regions. FIG. 6 shows a luminance mapping curve corresponding to a global tone-mapping operator with three levels of tone-mapping operators, where FIG. 6 (a) corresponds to a luminance mapping curve corresponding to tone-mapping operator 0, tone-mapping operator 0 improving contrast primarily in the luminance interval [0, L0 ]; FIG. 6 (b) is a luminance mapping curve corresponding to tone-mapping operator 1, tone-mapping operator 1 mainly improving contrast in the luminance range [ L0, L1 ]; fig. 6 (c) corresponds to a luminance mapping curve corresponding to the tone mapping operator 2, and the tone mapping operator 2 mainly improves the contrast in the luminance range L1, L2. In practical application, in order to obtain a better effect, the shape of the brightness mapping curve corresponding to the global tone mapping operator in the current tone mapping operator can be formulated by combining the image content, for example, the customization is performed by adopting a histogram equalization mode.
For the setting of the local tone mapping operator, the problem of balancing the halation effect with the local contrast needs to be considered. In order to balance the halation effect and improve the local contrast, the basic idea of the embodiment is to weaken or even eliminate the halation effect by carrying out differential design on local filter coefficients of tone mapping operators of each level. Specifically, the halation effect is reduced by local filtering through the use of different filtering operators at each stage, such as the retinal cortex (Retinex) theory, a base layer plus detail layer based tone mapping operator, and the like, wherein a typical design mode is that different stages use different scale filtering windows. In this embodiment, each stage of local tone mapping operator includes a filter operator, where the filter operators corresponding to each stage of local tone mapping operator are different, for example, the filter windows corresponding to each stage of tone mapping operator are changed step by step, and as each stage of tone mapping operator changes from the front stage to the back stage, the size of the corresponding filter window changes from large to small, for example, the size of the filter window corresponding to tone mapping operator 0 is 17×17, that is, the pixels of the filter window are 17 rows and 17 columns; the size of the filter window corresponding to tone mapping operator 1 is 9 x 9, the size of the filter window corresponding to tone mapping operator 2 is 5 x 5, and so on.
The size of the conventional filter window is often fixed, and compared with the design scheme of the conventional filter window, the embodiment designs the filter window with a small size at a later stage, so that the contrast of a high-contrast area on the initial image is significantly reduced, and the significant halation effect is reduced.
In practical implementation, for a local tone mapping operator based on statistical information, such as local histogram equalization, different blocking modes of different levels may be used to reduce the halation effect, as shown in fig. 7, in the blocking areas corresponding to the local tone mapping operators of two adjacent levels, the blocking mode of the previous level is shown in fig. 7 (a), and a local area is divided into 16 blocking areas, where each blocking area is square, that is, square areas with length and width of 2 r. As shown in fig. 7 (b), the blocking method of the latter stage is staggered from that of the former stage, and for the same partial region, the middle 9 blocking regions are square regions each having a length and a width of 2r, but the four vertex angles are square regions each having a length and a width of r, and the region near the edge line is a rectangular region having a length of 2r and a width of r.
Because in the local tone mapping algorithm, the region where the halo effect is likely to occur is often the intersection of two adjacent segmented regions. In this embodiment, the blocking areas of the adjacent two stages are staggered with each other, and the blocking areas of each stage are not completely overlapped, that is, the areas where the halo effect may occur in the adjacent two stages are staggered with each other, so that the halo effect occurring after tone mapping can be effectively reduced.
The mixed tone mapping operator is an operator obtained by calculating the global tone mapping operator and the local tone mapping operator, and specifically, a preset weighting coefficient can be used for calculating the global tone mapping operator and the local tone mapping operator of the level. For example, assuming that I is a pixel of the initial image, G (-) is a global tone-mapping operator for the stage, and L (-) is a local tone-mapping operator for the stage, then the blended tone-mapping operator TM (-) for the stage can be calculated by the following formula:
TM (I) =a×G (I) + (1-a) ×L (I) (formula 1)
Wherein a is a preset weighting coefficient, which may be a global coefficient, that is, the same weighting coefficient is used for each pixel of the whole image, or a local coefficient, that is, the weighting coefficient of the image is not fixed, but gradually changes along with the movement of the pixel according to a preset rule.
For the mixed tone mapping algorithm of the two-stage structure, tone mapping operators of each stage are mixed tone mapping operators, and the mixed tone mapping operators of each stage are obtained by calculation by using preset weighting coefficients, global tone mapping operators and local tone mapping operators. In addition, in the multi-level tone mapping algorithm of this embodiment, after calculating the tone mapping operator of the previous level, the weighting coefficients in the mixed tone mapping operator of the next level may be dynamically adjusted. Specifically, the weighting coefficient a of the subsequent stage is locally and dynamically adjusted according to the gradient inversion condition of the mapping result of the previous stage, so that a better halation effect and a local contrast balance result are obtained.
For example, the local gradient inversion coefficient may be defined by the following formula:
(2)
Where Ω denotes the window in which pixel i is located, j is the other pixels in the window, ψ denotes the gradient inversion condition, which is: (TM (j) -TM (I))× (I (j) -I (I)) <0. The larger the local gradient inversion coefficient is, the larger the weighting coefficient a corresponding to the next stage is, so that the edge effect after tone mapping can be controlled, and the image quality after tone mapping is improved.
Therefore, in the embodiment, the multistage tone mapping operators are designed, each stage of tone mapping operator has fixed mapping capability, and the rapid and flexible customization of different dynamic range mapping algorithms can be completed through the self-adaptive flexible use of the multistage tone mapping operators, so that the requirements of tone mapping of images in different scenes are met. In addition, when the global tone mapping operator is designed, the fine management of the contrast of each brightness interval is realized by designing the shape of the brightness mapping curve corresponding to each tone mapping operator, and the tone mapping quality is improved. In addition, in the design of the local tone mapping operator, the purpose of weakening or even eliminating the halation effect is achieved through the design of different levels of filter coefficient differentiation. Finally, when the design of the mixed tone mapping operator is carried out, the problems of halation effect and local contrast are better balanced through the combination treatment of global tone mapping operator and local tone mapping operator of more stages.
Computer apparatus embodiment:
the computer apparatus of this embodiment may be an intelligent electronic device, and the computer apparatus includes a processor, a memory, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the image tone mapping method described above. Of course, the intelligent electronic device further comprises an image capturing device for acquiring the initial image.
For example, a computer program may be split into one or more modules, which are stored in memory and executed by a processor to perform the various modules of the invention. One or more of the modules may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in the terminal device.
The processor referred to in the present invention may be a central processing unit (Central Processing Unit, CPU), or other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being a control center of the terminal device, and the various interfaces and lines being used to connect the various parts of the overall terminal device.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory, and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
Computer-readable storage medium:
the computer program stored in the above-mentioned computer means may be stored in a computer readable storage medium if it is implemented in the form of software functional units and sold or used as a separate product. Based on such understanding, the present invention may implement all or part of the above-described embodiment method, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of the above-described image tone mapping method when executed by a processor.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the content of the computer readable medium can be appropriately increased or decreased according to the requirements of the jurisdiction's jurisdiction and the patent practice, for example, in some jurisdictions, the computer readable medium does not include electrical carrier signals and telecommunication signals according to the jurisdiction and the patent practice.
Finally, it should be emphasized that the present invention is not limited to the above embodiments, for example, the change of the weighting coefficients used for calculating the hybrid tone mapping operator, or the change of the luminance mapping curve corresponding to the multi-level global tone operator, and these changes should also be included in the scope of the claims of the present invention.
Claims (10)
1. An image tone mapping method comprising:
acquiring an initial image, acquiring an initial brightness value of each pixel of the initial image, performing tone mapping calculation on the initial brightness value of each pixel, calculating brightness gain, and outputting a mapped brightness value;
the method is characterized in that:
setting tone mapping operators with more than two stages, wherein each stage of tone mapping operator has fixed dynamic range mapping capability;
performing tone mapping calculation on the initial luminance value of each of the pixels includes:
obtaining a bit width value of the initial brightness value of the pixel, determining the number of stages of the tone mapping operators used according to the bit width value of the initial brightness value, and applying the determined tone mapping operators of at least one stage to perform mapping calculation on the initial brightness value.
2. The image tone mapping method according to claim 1, wherein:
the dynamic range mapping capability of the tone mapping operator of each stage is the same; or alternatively
The dynamic range mapping capability of the tone mapping operator of at least one stage is different from the dynamic range mapping capability of the tone mapping operator of another stage.
3. The image tone mapping method according to claim 1 or 2, characterized in that:
the tone mapping operator is one of the following: a global tone mapping operator, a local tone mapping operator, or a hybrid tone mapping operator;
wherein the mixed tone mapping operator is the calculated value of the global tone mapping operator of the stage and the local tone mapping operator of the stage.
4. A method of tone mapping an image according to claim 3, wherein:
the mixed tone mapping operator is a calculated value of the global tone mapping operator of the stage and the local tone mapping operator of the stage calculated by using a preset weight coefficient.
5. The image tone mapping method of claim 4, wherein:
and adjusting the weight coefficient applied by the mixed tone mapping operator of the subsequent stage according to the gradient inversion coefficient of the mapping calculation result of the previous stage.
6. A method of tone mapping an image according to claim 3, wherein:
the luminance mapping curve corresponding to the global tone mapping operator of at least one stage is different from the luminance mapping curve corresponding to the global tone mapping operator of another stage.
7. A method of tone mapping an image according to claim 3, wherein:
the local tone mapping operators of each stage comprise filtering operators, the filtering operators corresponding to the local tone mapping operators of each stage are different, and the sizes of the filtering windows corresponding to the local tone mapping operators of the stages from the front stage to the rear stage are gradually reduced.
8. The image tone mapping method of claim 7, wherein:
the partitioned areas corresponding to the adjacent two stages of local tone mapping operators are staggered, and the partitioned areas corresponding to the local tone mapping operators of each stage are not completely overlapped.
9. A computer arrangement comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, performs the steps of the image tone mapping method according to any one of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, implements the steps of the image tone mapping method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010091029.0A CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010091029.0A CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111292269A CN111292269A (en) | 2020-06-16 |
| CN111292269B true CN111292269B (en) | 2023-09-01 |
Family
ID=71021374
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010091029.0A Active CN111292269B (en) | 2020-02-13 | 2020-02-13 | Image tone mapping method, computer device, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111292269B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052923B (en) * | 2021-03-31 | 2023-02-28 | 维沃移动通信（深圳）有限公司 | Tone mapping method, tone mapping apparatus, electronic device, and storage medium |
| CN113905183B (en) * | 2021-08-25 | 2024-04-16 | 珠海全志科技股份有限公司 | Color difference correction method and device for wide dynamic range image |
| CN115100031B (en) * | 2022-08-23 | 2023-03-14 | 摩尔线程智能科技(北京)有限责任公司 | Image processing method and image processing apparatus |
| CN115861096B (en) * | 2022-11-22 | 2023-10-31 | 瀚博半导体(上海)有限公司 | Image processing method and device and computer equipment |
| CN116894795B (en) * | 2023-09-11 | 2023-12-26 | 归芯科技(深圳)有限公司 | Image processing method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103295194A (en) * | 2013-05-15 | 2013-09-11 | 中山大学 | Brightness-controllable and detail-preservation tone mapping method |
| CN106504278A (en) * | 2015-09-04 | 2017-03-15 | 迪士尼企业公司 | HDR tone maps |
| CN108198155A (en) * | 2017-12-27 | 2018-06-22 | 合肥君正科技有限公司 | A kind of self application tone mapping method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100900694B1 (en) * | 2007-06-27 | 2009-06-04 | 주식회사 코아로직 | Apparatus and method for correcting a low tone with non-linear mapping and computer readable medium stored thereon computer executable instruction recorded with time-series data structure |
-
2020
- 2020-02-13 CN CN202010091029.0A patent/CN111292269B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103295194A (en) * | 2013-05-15 | 2013-09-11 | 中山大学 | Brightness-controllable and detail-preservation tone mapping method |
| CN106504278A (en) * | 2015-09-04 | 2017-03-15 | 迪士尼企业公司 | HDR tone maps |
| CN108198155A (en) * | 2017-12-27 | 2018-06-22 | 合肥君正科技有限公司 | A kind of self application tone mapping method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111292269A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111292269B (en) | Image tone mapping method, computer device, and computer-readable storage medium | |
| US9467704B2 (en) | Adaptive ratio images in HDR image representation | |
| US8169500B2 (en) | Dynamic range compression apparatus, dynamic range compression method, computer-readable recording medium, integrated circuit, and imaging apparatus | |
| US9978121B2 (en) | Adaptive sharpening in image processing and display | |
| US9076218B2 (en) | Method and image processing device for image dynamic range compression with local contrast enhancement | |
| US8374430B2 (en) | Apparatus and method for feature-based dynamic contrast enhancement | |
| US9177396B2 (en) | Image processing apparatus and image processing method | |
| EP1934937A2 (en) | Image detail enhancement | |
| US7889942B2 (en) | Dynamic range compensation-dependent noise reduction | |
| CN113935911A (en) | High dynamic range video image processing method, computer device and computer readable storage medium | |
| CN111724316B (en) | Method and apparatus for processing high dynamic range image | |
| WO2019104047A1 (en) | Global tone mapping | |
| CN114998122A (en) | Low-illumination image enhancement method | |
| CN106341613A (en) | Wide dynamic range imaging method | |
| Albu et al. | One scan shadow compensation and visual enhancement of color images | |
| CN112534466B (en) | Directional scaling system and method | |
| CN114223209A (en) | Adaptive image data linearization for HDR image sensors | |
| Asari et al. | Nonlinear enhancement of extremely high contrast images for visibility improvement | |
| JP2018019239A (en) | Imaging apparatus, control method therefor and program | |
| CN112598595A (en) | High-dynamic digital image display enhancement method and system based on gamma correction | |
| Kao et al. | Tone reproduction in color imaging systems by histogram equalization of macro edges | |
| Hu et al. | Using adaptive tone mapping to enhance edge-preserving color image automatically | |
| US8587725B2 (en) | Method of digital signal processing | |
| CN112954232A (en) | Moving image processing method, moving image processing apparatus, camera, and storage medium | |
| CN117876283A (en) | Automatic driving high dynamic scene image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |