RU2509366C1 - System and method of converting hue of flat and three-dimensional images - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: method involves selecting a hue conversion function based on analysis of distribution of hue of elements of the entire image, varying parameters of the hue conversion function for each image element based on analysis of said distribution of the local surrounding region, where parameters are varied gradually for neighbouring image elements; and converting hue of each image element using the hue conversion function with parameters obtained for that image element.

EFFECT: high quality of digital images by increasing global and local contrast without generating undesirable artefacts and distortions.

12 cl, 8 dwg

Description

The invention relates to the field of digital image processing and can be used to improve the quality of flat and three-dimensional images, and more specifically for adaptively increasing global and local contrast.

A large number of patents and publications are known on the transformation or correction of tones (or gradations) to improve the visual quality of 2D and 3D images. Dozens of patents describe various applications of methods based on histogram equalization for adaptive tone correction, for example, US patent 7020333 [1]. In methods based on the alignment of histograms, the tone conversion function is the result of an optimization procedure that attempts to select a conversion function that will bring the existing histogram of image tones to some desired form. However, it is not possible to establish the desired type of histogram for all possible types of images, therefore, methods based on the alignment of histograms have limited application.

Most of the existing patents and published applications are devoted to explicitly setting the tone conversion function and selecting the parameters of this function in automatic mode. For example, US Pat. No. 7,023,580 [2] describes a tone conversion method using an adaptive sigmoid function. In US Pat. No. 6,026,181 [3], a suitable conversion function common to the whole image is selected based on the classification of the scene in the image. Such a single tone conversion function cannot provide correction of local contrast in a number of image areas.

There are several patents and patent applications laid out in which the tone conversion function is set for local areas of the image. US patent 7724981 [4] claims a method comprising the following steps: splitting an image into disjoint blocks, for each block, the values of the black point (darkest tones) and white point (brightest tones) are determined, for each pixel a new value is obtained using it the previous value, its position in the image plane, the values of the black and white points for this block and for neighboring blocks.

US patent 6771814 [5] describes a method that includes dividing the image into blocks, calculating the average value for each block, determining the saturation value in order to find a possible degree of improvement in contrast, obtaining a histogram of tones for each block, generating a transformation function for each block tones based on this histogram and saturation values, the conversion of all pixels in the image using the tone conversion functions.

In US patent 8160387 [6] the image intensity varies in accordance with the gains that are set for local areas and depend on the values of the gains of neighboring areas.

U.S. Patent Laid-Open No. 20090097747 [7] describes a contrast process in which a specific tone conversion curve is used for each local area. Then, in order to suppress arising artifacts at the edges of regions, these pixels at the edges of regions are transformed using a function obtained from the transformation functions of neighboring regions.

This technical solution is selected as a prototype of the claimed invention.

All the methods mentioned here for applying level conversion functions in local areas do not perform global image correction and do not guarantee that there is no sudden jump in the tone conversion function at the boundary between local areas. Such abrupt changes in the conversion function can lead to the formation of unwanted visual distortions in the form of false contours.

The problem to which the claimed invention is directed is to automatically improve the visual quality of flat and three-dimensional images, namely to increase global and local contrast of images without generating unwanted artifacts and distortions.

The technical result is achieved by developing a method for converting tones of flat and three-dimensional images. The method of converting tones of flat and three-dimensional images is to perform the following operations:

- choose the function of the conversion of tones, based on the analysis of the distribution of tones of the elements of the whole image.

- change the parameters of the tone conversion function for each image element, based on the analysis of the distribution of tones of the elements of the local surrounding area, and so that the parameters change smoothly for neighboring image elements;

- convert the tone of each image element using the tone conversion function with the parameters obtained for this image element in the previous step.

Also, the technical result is achieved by developing a system for converting tones of flat and three-dimensional images.

The inventive system contains:

- a tone conversion function selection module, configured to select a function type and a number of parameters of a tone conversion function based on an analysis of the distribution of tones of the entire image, while the original image is input to this module, and the tone conversion function with a number of parameters is transmitted to the parameter changing module tone conversion functions and into the tone conversion module;

- a module for changing the parameters of the tone conversion function, configured to change the parameters of the tone conversion function for each image element based on the analysis of the distribution of tones of the elements of the local surrounding area, the parameters changing smoothly for neighboring image elements, the original image and the tone conversion function with a number of parameters are sent to the input of this module, and the parameters of the tone conversion function changed for each pixel are transmitted to the tone conversion module at;

- a tone conversion module, configured to convert the tones of each image element by means of a tone conversion function with parameters specific to a given image element and obtained from a module for changing the parameters of a tone conversion function, and the input image, a tone conversion function and changed for each image element is the parameters of this function.

Further, the essence of the claimed invention is explained in detail with reference to graphic materials.

Figure 1. The block diagram of the main steps of the method for converting tones of flat and three-dimensional images.

Figure 2. An illustration of a selection of parameters of a Hermite cubic slice used as a tone conversion function in a preferred embodiment of the invention.

Figure 3. A block diagram of the change stage for each image element of the parameters of the tone conversion function based on the analysis of the distribution of tones of the elements of the local surrounding area, where the parameters change smoothly for neighboring image elements.

Figure 4. An illustration for a preferred embodiment of the invention a change step for each image element of the tone conversion function parameters.

Figure 5. An example of tone conversion for a scanned image.

6. The block diagram of the system for converting tones of flat and three-dimensional images.

7. Block diagram of the module for changing the parameters of the tone conversion function.

A block diagram of the main steps of the method is shown in figure 1. At 101, a tone conversion function is selected based on an analysis of the distribution of tones of the elements of the entire image. In flat (or planar) images, image elements are pixels. The invention is applicable both for black and white (in the gray scale), and for color images. In volumetric (volumetric) images, image elements are voxels. At step 102, for each image element (pixel or voxel), the parameters of the tone conversion function are changed based on the analysis of the distribution of tones of the elements of the local surrounding area and so that the parameters change smoothly for neighboring image elements.

For volumetric images, the local region can be either flat, in any of the sections, or volumetric. At step 103, the tone of each image element is transformed using the tone conversion function with parameters specific to that image element and obtained in the previous step.

The step of selecting the tone conversion function (101) includes selecting the type of function and selecting values of the immutable parameters of the function. Many different functions can be used as a tone conversion function (functions or level transfer curves), for example, it can be piecewise linear functions, gamma correction, polynomial and S-shaped functions. S-shaped curves allow you to increase both global and local image contrast. There are dozens of different S-shaped functions, for example, a sigmoid function, hyperbolic tangent, cubic polynomials and splines. In a preferred embodiment of the invention, a Hermite cubic spline is used as an S-curve, which is defined in parametric form by the expression:

F (t) = (l-3t ² + 2t ³ ) × P ₀ + t ² × (3-2t) × P ₁ + t × (1-2t + t ² ) × Q ₀ -t ² × (1- t) × Q ₁ ,

where P ₀ is the starting point of the curve, P ₁ is the ending point, Q ₀ and Q _{1 are the} direction vectors from the starting and ending points, respectively, the parameter t varies from 0 to 1.

Figure 2 illustrates a method for selecting a number of parameters of a Hermite cubic spline based on an analysis of the distribution of tones of the elements of the entire image. Tone conversion function 201, i.e. the dependence of the value of the tone at the output on the value at the input is normalized to the range [0, 1], the y-coordinate values (the axis of the values at the output) of the start and end points of the curve are rigidly set to P _0y = 0 and P _0y = 1. In order to calculate the x-coordinate values (the axis of the input values) of the starting and ending points of the P _0x and P _1x curve, an H-global histogram of image brightness 202 is analyzed. The starting point of the P _0x curve corresponds to the darkest tones of the image (the so-called black point) :

$$P_{0x}=\min (D,\min \left\{i|\; |\; |H[i]\ge H_0\right\},\min \left\{i|\; |\; |{\displaystyle \sum _{k=0}^{i}H[k]\ge C_0}\right\})$$

where H ₀ , C ₀ and D are threshold values, respectively, by the level of the histogram, by the area of the histogram, and by brightness. Threshold D is introduced to avoid excessive dimming of the image.

The end point of the P _1x curve corresponds to the lightest tones of the image (the so-called white point):

$$P_{\mathrm{one}x}=\max (\max \left\{i|\; |\; |H[i]\ge H_{\mathrm{one}}\right\},\max \left\{i|\; |\; |{\displaystyle \sum _{k=\mathrm{one}}^{255}H[k]\ge C_{\mathrm{one}}}\right\})$$

where H ₁ and C ₁ are threshold values, respectively, in terms of level and area of the histogram.

The end and start points are fixed for the whole image. This provides an increase in global image contrast. In order for the Hermite’s cubic spline to be S-shaped, the vectors Q ₀ and Q ₁ must be directed up and down as shown in FIG. The values of these vectors vary for each image element and depend on the global and local binarization thresholds between the tones of the foreground elements and the background of the entire image and the local area. There are many ways to adaptively calculate the threshold of binarization. In a preferred embodiment of the invention, the well-known Otsu method is used according to N. Otsu, “A threshold selection method from gray level histogram”, IEEE Transactions on System Man Cybernetics, vol. 9 no. 1, 1979, pp. 62-66 [8].

FIG. 3 shows a flowchart of a step (102) of changing, for each image element, parameters of a tone conversion function based on an analysis of the distribution of tones of the elements of the local surrounding area and so that the parameters change smoothly for adjacent image elements. At step 301, the image is divided into local overlapping areas. The analysis for each local region of the distribution of tones of the foreground and background elements is performed at step 302. Step 303 is the formation of a matrix with the parameters of the tone conversion function, where each element of the matrix corresponds to the local region and depends on the result of the analysis of the distribution of tones of the foreground and background elements in this area. In step 304, the matrix with the parameters of the tone conversion function is smoothed to ensure a smooth change in tones between adjacent local areas. Extraction by interpolation of the function parameters from the smoothed matrix with the parameters of the tone conversion function for each image element is performed at step 305.

4 illustrates step 102 for a preferred embodiment of the invention. The original image 401 is divided into overlapping areas. As an example, 3 such regions are shown: 402, 403, and 404. A histogram of the luminance of tones is calculated for each local region. 4, histogram 405 corresponds to region 402, histogram 406 corresponds to region 403, and histogram 407 corresponds to region 404. For each histogram, i.e. tone distribution, the width of the tone range W1 in a given local region and the Otsu binarization threshold Tl are calculated (see [8]). If the width of the range of tones is less than a predetermined value, as, for example, in histogram 405, then it is assumed that in this local area there are elements of only the foreground or only the background. In this case, the global binarization threshold T is written in the corresponding cell of the matrix 408 with the parameters of the tone conversion function. If the width of the tone range is greater than a predetermined value, as, for example, in histograms 406 and 407, then it is assumed that there are elements and foreground and background. In this case, the local binarization threshold T1 is recorded in the corresponding cell of the matrix 408 with the parameters of the tone conversion function. Note that there is no need to determine which image elements relate to the foreground or background. It is enough to determine whether there are simultaneously elements in a given local area that belong to both of these classes or only to one of these classes. The smoothing of the matrix 408 with the parameters of the tone conversion function is performed using an averaging low-pass filter. The result is a smoothed matrix 409, the values of which affect the values of the vectors Q ₀ and Q _{1 of the} cubic spline of Hermite 411 when processing each image element 410 as follows:

If (K> 0.5), then Q _0x = 1 + A × (K-0.5); Q _0y = 0; Q _lx = l; Q _ly = 0;

otherwise Q _0x = 1; Q _0y = 0; Q _1x = 1 + A × (0.5 - K); Q _1y = 0;

where A is a coefficient in the range from 1 to 6, K is extracted from the matrix with the parameters of the tone conversion function using bilinear interpolation.

The described approach provides a different shape of the S-shaped tone conversion function for each image pixel and a smooth change of function for neighboring image pixels, which leads to an increase in global and local image contrast without generating noticeable unwanted artifacts / distortions.

An example of tone conversion for a scanned image is shown in FIG. 5, where 5.1 is the original image, 5.2 is the result of the tone conversion.

The block diagram of the system modules is shown in Fig.6. The tone conversion function selection module (601) selects the type of function and a number of parameters of the tone conversion function based on the analysis of the tone distribution of the whole image. The input image of this module receives the original image. The result of the module - the tone conversion function with a number of parameters is transmitted to the module for changing the parameters of the tone conversion function (602) and to the tone conversion module (603). The module for changing the parameters of the tone conversion function changes for each image element the parameters of the tone conversion function based on the analysis of the distribution of tones of the elements of the local surrounding area and so that the parameters change smoothly for neighboring image elements. The input image and the tone conversion function with a number of parameters are input to this module. The parameters of the tone conversion function changed for each pixel are transferred to the tone conversion module. The tone conversion module converts the tone of each image element through the tone conversion function with parameters specific to the given image element and obtained from the module for changing the parameters of the tone conversion function. The input image, the tone conversion function, and the parameters of this function changed for each image element are input to this module.

7 shows a block diagram of a module for changing the parameters of the tone conversion function (602) of the inventive system. The module for dividing the image into regions (701) divides the image into overlapping local plane / volume regions. The input image receives the original image. Local areas are transmitted to the image analysis module (702). The image analysis module analyzes for each local area the distribution of tones of the foreground and background elements. The input area of the module receives local areas of the image. The results of the analysis of the tone distribution are transmitted to the module generating the matrix of parameters of the tone conversion function (703). The module for generating the matrix of parameters for the tone conversion function generates a matrix of parameters for the tone conversion function, where each local area corresponds to a matrix element depending on the distribution of tones of the foreground and background elements in this local area. The input of the module receives the results of the analysis of the distribution of tones. The matrix of parameters of the tone conversion function is transmitted to the smoothing module (704). The smoothing module performs low-pass filtering of the matrix of parameters of the tone conversion function.

All of the listed system modules can be made in the form of a system on a chip (SoC), a programmable logic array (FPGA), or a specialized integrated circuit (ASIC). The functions of these modules are clear from their description, as well as from the description of the proposed method for converting tones of flat and three-dimensional images.

Thus, the inventive method and system combine global and local tone correction by selecting a tone conversion function, a number of parameters of which are fixed for the whole image, and the series changes for the local area depending on the distribution of tones in this area. The use of overlapping local regions, smoothing the matrix with the parameters of the tone conversion function, and applying bilinear interpolation when extracting parameters from the matrix provide a smooth change in the function both between adjacent local regions and between neighboring image elements, thus guaranteeing the absence of noticeable artifacts and distortions.

Also, unlike existing methods, the inventive method in a preferred embodiment uses an Hermite S-shaped cubic spline as a tone conversion function. Setting the start and end points of the spline that are common to the entire image provides an increase in global contrast. The control vectors depend on the distribution of tones in the local region and adaptively adjust the shape of the curve.

Moreover, most existing methods are designed to process only flat (2D) images, while the inventive method and system can be used to improve the quality of both flat and three-dimensional (volumetric) color and monochrome images.

The invention is primarily intended to improve the visual quality of scanned images in scanners and copiers. Especially effective is the use of this invention to improve copying of originals containing simultaneously low-contrast text or graphics in areas of light and dark background. Also, the invention can be used to improve the visual quality of photos in any device for photographing and viewing photos, for example in mobile devices with a camera, digital cameras, televisions and monitors. In addition, the proposed system and method can be used to improve the visual quality of volumetric medical images, for example, when visualizing tomographic images.

Although the above embodiment of the invention has been set forth for the purpose of illustration, it is clear to those skilled in the art that various modifications, additions and substitutions are possible without departing from the scope and meaning of the present invention disclosed in the attached claims.

Claims (12)

1. The method of converting tones of flat and three-dimensional images, providing for the following operations:
- choose the function of the conversion of tones based on the analysis of the distribution of tones of the elements of the whole image.
- change the parameters of the tone conversion function for each image element, based on the analysis of the distribution of tones of the elements of the local surrounding area and so that the parameters change smoothly for neighboring image elements;
- convert the tone of each image element using the tone conversion function with the parameters obtained for this image element in the previous step. 2. The method according to claim 1, characterized in that the choice of the tone conversion function involves the following operations:
- choose the type of function;
- choose the values of the immutable parameters of the function; 3. The method according to claim 1, characterized in that when choosing the type of tone conversion function, the following types of functions are considered: piecewise linear, gamma correction, polynomial and S-shaped functions; 4. The method according to claim 1, characterized in that as a function of an S-shaped shape, a Hermite cubic spline is selected:
F (t) = (1-3t ² + 2t ³ ) × P ₀ + t ² × (3-2t) × P ₁ + t × (1-2t + t ² ) × Q ₀ -t ² × (1- t) × Q ₁ ,
where the starting point P ₀ depends on the values of the darkest tones of the whole image, the end point P ₁ depends on the values of the lightest tones of the whole image, the vectors Q ₀ and Q ₁ depend on the global and local binarization thresholds between the tones of the foreground elements and the background of the whole image and local area. 5. The method according to claim 1, characterized in that the change for each image element of the parameters of the tone conversion function consists of the following steps:
- break the image into local overlapping areas;
- analyze for each local area the distribution of tones of the elements of the foreground and background;
- form a matrix with the parameters of the tone conversion function, where each matrix element corresponds to a local area and depends on the result of the analysis of the distribution of tones of the foreground elements and the background in this area;
- smooth the matrix with the parameters of the tone conversion function to ensure a smooth change of tones between neighboring local areas;
- extracting by interpolation the parameters of the function from the smoothed matrix with the parameters of the tone conversion function for each image element. 6. The method according to claim 1, characterized in that the analysis for each local region of the distribution of tones includes determining the width of the range of tones and the threshold of binarization between the elements of the foreground and background. 7. The method according to claim 1, characterized in that when forming a matrix with parameters of the tone conversion function, the following rule is used: if the width of the tone range is greater than a predetermined value, then the local binarization threshold is written to the corresponding cell of the matrix, otherwise the global binarization threshold is recorded. 8. The method according to claim 1, characterized in that the smoothing of the matrix with the parameters of the tone conversion function is performed using an averaging low-pass filter. 9. The method according to claim 1, characterized in that the function parameters are extracted from the matrix with the parameters of the tone conversion function using bilinear interpolation. 10. The method according to claim 1, characterized in that when converting the tone of each image element using a cubic Hermite curve, the following parameters are used: the starting point P ₀ depends on the value of the darkest tones of the whole image, the end point P ₁ depends on the value of the lightest tones of the whole image, vectors Q ₀ (Q _0x , Q _0y ) and Q ₁ (Q _1x , Q _1y ) depend on the values in the matrix with the parameters of the tone conversion function:
if (K> 0.5), then Q _0x = 1 + A × (K-0.5); Q _0y = 0; Q _lx = 1; Q _ly = 0; otherwise Q _0x = 1; Q _0y = 0; Q _1x = 1 + A × (0.5-K); Q _1y = 0;
where A is a coefficient in the range from 1 to 6, K is extracted from the matrix with the parameters of the tone conversion function using interpolation. 11. A system for converting tones of flat and three-dimensional images, which contains:
- a tone conversion function selection module, configured to select a function type and a number of parameters of a tone conversion function based on an analysis of the distribution of tones of the entire image, while the original image is input to this module, and the tone conversion function with a number of parameters is transmitted to the parameter changing module tone conversion functions and into the tone conversion module;
- a module for changing the parameters of the tone conversion function, configured to change the parameters of the tone conversion function for each image element based on the analysis of the distribution of tones of the elements of the local surrounding area, the parameters changing smoothly for adjacent image elements, the original image and the tone conversion function with a number of parameters are sent to the input of this module, and the parameters of the tone conversion function changed for each pixel are transferred to the tone conversion module s;
- a tone conversion module, configured to convert the tones of each image element by means of a tone conversion function with parameters specific to a given image element and obtained from a module for changing the parameters of a tone conversion function, and the input image, a tone conversion function and changed for each image element is the parameters of this function. 12. The system according to claim 11, characterized in that the module for changing the parameters of the tone conversion function contains:
- the module for dividing the image into regions divides the image into overlapping local flat / volume regions, the original image is fed to the input of the module, local regions are transmitted to the image analysis module;
- the image analysis module analyzes for each local region the distribution of tones of the foreground and background elements, the local image regions arrive at the input of the module, the results of the analysis of the distribution of tones are transmitted to the module generating the matrix of parameters of the tone conversion function;
- the module generating the matrix of parameters of the tone conversion function generates a matrix of parameters of the tone conversion function, where each local area corresponds to a matrix element that depends on the distribution of tones of the foreground and background elements in this local area, the results of the analysis of the distribution of tones, the matrix of parameters of the conversion function are input to the module tones are transmitted to the smoothing module;
- the smoothing module performs low-pass filtering of the matrix of parameters of the tone conversion function.

Images