CN107330885B - Multi-operator image redirection method for keeping aspect ratio of important content area - Google Patents

Multi-operator image redirection method for keeping aspect ratio of important content area Download PDF

Info

Publication number
CN107330885B
CN107330885B CN201710549826.7A CN201710549826A CN107330885B CN 107330885 B CN107330885 B CN 107330885B CN 201710549826 A CN201710549826 A CN 201710549826A CN 107330885 B CN107330885 B CN 107330885B
Authority
CN
China
Prior art keywords
image
size
energy
important content
dimension
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710549826.7A
Other languages
Chinese (zh)
Other versions
CN107330885A (en
Inventor
唐振华
张倩
李军
黄宝婵
覃团发
常侃
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangxi Xinhao Zhiyun Technology Co.,Ltd.
Original Assignee
Guangxi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangxi University filed Critical Guangxi University
Priority to CN201710549826.7A priority Critical patent/CN107330885B/en
Publication of CN107330885A publication Critical patent/CN107330885A/en
Application granted granted Critical
Publication of CN107330885B publication Critical patent/CN107330885B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4023—Decimation- or insertion-based scaling, e.g. pixel or line decimation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Abstract

The invention discloses a multi-operator image redirection method for keeping the aspect ratio of an important content area, which comprises the steps of firstly carrying out visual saliency detection on an input image, calculating by combining a normalized gray level image to obtain an energy spectrum, carrying out threshold segmentation and division on the energy spectrum to obtain a visual saliency object, calculating the stretching size required to be carried out according to the size of the visual saliency object in order to reduce the overall size of the visual saliency object, respectively stretching the visual saliency object and background information to different degrees, carrying out similarity transformation on the stretched image, and finally carrying out fine adjustment by using a line cutting method of adding a gradient vector stream according to the target size to obtain a final output image. The method can detect and globally protect the visual salient content, thereby improving the quality of the image redirection technology.

Description

Multi-operator image redirection method for keeping aspect ratio of important content area
Technical Field
The invention belongs to the field of image processing, and particularly relates to an image redirection method based on content perception, in particular to a multi-operator image redirection algorithm for keeping the aspect ratio of an important content area based on content perception.
Background
Compared with the traditional redirection method such as nearest neighbor interpolation, bilinear interpolation, clipping and the like, the image redirection method based on content perception, namely the line clipping method, can randomly change the aspect ratio of the image without distorting the content, and is very suitable for mobile terminal equipment with increasingly diversified display sizes at present. The following briefly introduces some existing classes of line clipping-based image redirection methods.
The first method, which uses line cropping alone to redirect the image, is to find an optimal cropping gap on the image energy map and then change the image size by inserting or removing the cropping gap. However, this method has the disadvantages that: the image size is changed only by deleting the cutting gap, the image content is easy to generate sawtooth distortion, and the integrity of the visual content is further influenced.
The second method, which performs image redirection by using multiple operations such as line cropping, scaling, and conventional cropping, includes two methods. The first method is simply combining line cropping and scaling, and performing image redirection through different amounts of line cropping and scaling, specifically determining the optimal number of line cropping by using a bidirectional similarity function of the euclidean distance of the image, dominant color description similarity, and line energy variation. The second is the regular combination of line cropping and scaling, which reduces the loss of content during image redirection by applying line cropping in a certain degree of reverse. The specific number of crops is still confirmed by using the two-way similarity function of the euclidean distance of the image. The performance of these two methods is better than that of the line clipping technique alone, but their drawbacks are: the absence of detection and protection of visually significant regions in an image is prone to cause loss of important information, resulting in distortion of the image.
In summary, the existing image redirection method cannot embody a high-quality visual effect, and the performance of the existing algorithm is to be improved.
Disclosure of Invention
The invention aims to overcome the defect that the existing image redirection method cannot embody the high-quality visual effect, provides a multi-operator image redirection method for keeping the aspect ratio of an important content area, and can obtain the higher-quality visual experience effect, thereby improving the performance of the image redirection technology.
In order to achieve the above object, the present invention provides a multi-operator image redirection method for maintaining aspect ratio of important content region, comprising:
(1) carrying out significance detection on an original image to obtain a significance map, and generating an energy map by using the significance map and the normalized gray map;
(2) carrying out binarization segmentation on the energy map to obtain an important content area and a background area;
(3) according to the size of the target image and the sizes of the important content area and the background area, calculating the sizes of the important content area and the background area which are respectively stretched in a non-reorientation dimension;
(4) according to the sizes of the important content area and the background area which are respectively stretched in the non-reorientation dimension, stretching the important content area in the non-reorientation dimension by adopting an even amplification method, stretching the background area of the image in the non-reorientation dimension by adopting a reverse line cutting method, and stretching the stretched image in the reorientation dimension by adopting an even amplification method; wherein, the redirection dimension refers to the dimension needing scaling, and the non-redirection dimension refers to the other dimension;
(5) performing similarity transformation on the stretched image obtained in the step (4), and uniformly scaling the image until the non-reorientation dimension is the same as that of the original image;
(6) and (4) scaling the scaled image obtained in the step (5) to a target size by using a reverse line clipping method for adding a gradient vector flow.
In an embodiment of the present invention, the step (3) is specifically:
(3.1) calculating the size increment of the original image in the non-reoriented dimension:
Figure GDA0002539729460000031
wherein h represents the size of the original image in the non-reoriented dimension, and w1Representing the size of the important content area in the redirection dimension, wfRepresents the size of the target image in the reorientation dimension and satisfies wf<w1The Δ h represents the size increment of the original image in the non-reoriented dimension;
(3.2) allocating different increment values according to the proportion of the important content area to the background area of the original image, wherein the increment of the important content area stretching is as follows:
Figure GDA0002539729460000032
wherein, the h1Representing the size of the important content area in the non-reoriented dimension, said β representing an adaptation parameter, said ah1A size increment representing a stretching of the important content area;
the size increments of the upper and lower background areas stretch are:
Figure GDA0002539729460000033
Figure GDA0002539729460000034
wherein, the h2Represents the size of the upper background area, said h3Represents the size of the lower background region, Δ h2Size increment representing stretching of the upper background area, Δ h3Representing the size increment of the stretching of the lower background area.
In an embodiment of the present invention, in the step (4), a reverse line cutting method is adopted to perform size stretching on the background information portion, specifically:
(4.1.1) energizing magnitude values for each pixel of the background information partial image; defining gradient values of the image as energy values:
Figure GDA0002539729460000035
wherein, x represents the original image,
Figure GDA0002539729460000036
representing a convolution, e represents a gradient map of the image.
(4.1.2) searching a cutting line with the minimum energy value by using a dynamic programming method;
(4.1.3) copy each pixel point of the crop line and insert it under the original pixel point.
In an embodiment of the present invention, the dynamic programming method includes:
(4.1.2.1) calculating a forward cumulative energy value map corresponding to each pixel point in the image, the calculation process being as follows:
M(i，j)＝e(i，j)+min(M(i-1，j-1)，M(i-1，j)，M(i-1，j+1))
wherein e (i, j) represents the energy of the pixel point, and M (i, j) represents the calculated forward accumulated energy of the pixel point;
(4.1.2.2) finding out the position point (i, j) of the minimum energy value from the last line of the forward accumulated energy map M, taking the point with the minimum accumulated energy in the adjacent three points in the previous line as the point of the line clipping route in the line each time by taking the point as the entry, namely:
seam(i-1)＝min{(i-1，j-1)，(i-1，j)，(i-1，j+1)}
wherein the seam (i-1) represents the point selected by the (i-1) th row of the cutting line;
(4.1.2.3) repeating the search up to the first line according to the rule to obtain the cutting line.
In an embodiment of the present invention, the method for uniformly amplifying in step (4) is a bilinear difference method, the bilinear interpolation method uses a gray value of the image as an energy value for calculation, and the step of uniformly scaling the image by the bilinear difference method includes:
(4.2.1) calculating the corresponding relation between the zoomed image pixel points and the original image pixel points according to the size of the zoomed target:
Figure GDA0002539729460000041
wherein, the if，jfRespectively represent the abscissa and ordinate of the scaled image, said ki，kjRespectively representing the scaling in the horizontal direction and the vertical direction, and i, j respectively representing the horizontal coordinate and the vertical coordinate of the original image;
(4.2.2) then pixel point (i)f，jf) The corresponding energy values of the four adjacent domain points are respectively as follows:
Figure GDA0002539729460000042
Figure GDA0002539729460000051
wherein, the
Figure GDA0002539729460000053
And
Figure GDA0002539729460000054
respectively represent ifAnd jfFinishing;
(4.2.3) calculation of bilinear difference:
Figure GDA0002539729460000055
wherein, said ef(i, j) represents the pixel point energy value in the zoomed image;
and (4.2.4) calculating the energy value of each pixel point of the zoomed image to obtain the zoomed image.
In an embodiment of the present invention, the step (6) is specifically:
(6.1) calculating a GVF vector GVF (i, j) of each pixel point in the image, wherein the GVF (i, j) comprises the size and the direction;
(6.2) planning downwards line by taking pixel points (1, j) in the first line of the image as initial points to obtain n line seams, wherein n is the size of the image in the reorientation dimension;
(6.3) calculating the accumulated energy value of the n seams, wherein the energy is the size of the GVF vector;
and (6.4) deleting the seam with the minimum energy value, and repeating the steps (6.1) to (6.4) until the seam is scaled to the size of the target image, so as to finish image compression.
In an embodiment of the present invention, the step (6.2) is specifically:
and planning to obtain a candidate cutting line according to the GVF vector direction of each pixel point in the image, and planning downwards if the current pixel point is (i, j), wherein the pixel points in the (i +1) th row are selected from the pixel points (i +1, j-1), (i +1, j) and (i +1, j +1), and the selection rule is as follows:
Figure GDA0002539729460000052
wherein θ [ GVF (i, j) ] represents the direction of the GVF vector of point (i, j).
In an embodiment of the present invention, the adaptive parameter β is selected according to a ratio of the important content area to the background area:
Figure GDA0002539729460000061
in an embodiment of the present invention, in the step (4), the reorientation dimension direction is stretched by using a uniform magnification method to balance the dimensional stretching of the important content area, and the stretched dimensions are as follows:
Figure GDA0002539729460000062
wherein w represents an original size of an original image, and w represents a size of the original imageenlRepresenting the image size after stretching.
In one embodiment of the invention, the energy profile is calculated by the following formula:
E＝S×Gm
wherein S represents a saliency map of an original image, GmRepresenting a normalized gray scale map, wherein:
Figure GDA0002539729460000063
wherein, G represents the gray value of the original image, the value range is (0,255), and G ismaxAnd GminRespectively the maximum and minimum of the image grey value.
In an embodiment of the present invention, the step (2) is specifically:
(2.1) selecting an adaptive threshold, and performing binarization segmentation on the energy map according to the adaptive threshold to obtain a binarization energy map; the method comprises the following steps:
(2.1.1) choosing an adaptive threshold, the binarization threshold being T that maximizes the objective function0Value, energy greater than T0The point of (1) is called foreground point, and the energy is less than T0The points of (2) are called background points, and the objective function is:
G(T0)＝ω0(U0-U)2+ω1(U1-U)2
wherein, the ω is0、ω1Respectively representing the proportion of the number of foreground points and background points to the total number of points of the whole atlas, wherein U is0、U1Respectively representing the energy average values of foreground points and background points, wherein U represents the average value of the energy of the whole spectrum;
(2.1.2) carrying out binarization segmentation according to the self-adaptive threshold, wherein the segmentation principle is as follows:
Figure GDA0002539729460000071
wherein E (x, y) and Em(x, y) respectively representing an input energy spectrum and an output binary energy spectrum;
(2.2) detecting the point with the median value of 1 in the binary energy spectrum, and selecting the uppermost (i)1，j1) The lowest part (i)2，j2) Leftmost (i)3，j3) And rightmost (i)4，j4) Four boundary points;
(2.3) a quadrilateral [ (j)1-j2)×(i4-i3)]I.e. the important content area, and the rest is the background information part.
The invention has the beneficial effects that: the method can overcome the defect that the prior image repositioning method cannot embody the high-quality visual effect, achieves the aim of protecting the important object in the reorientation process by adjusting the global proportion of the important object, improves the reorientation quality of the image, and ensures that the image reorientation technology obtains the higher-quality visual experience effect.
Drawings
FIG. 1 is a schematic flowchart of a multi-operator image redirection method for maintaining aspect ratio of important content areas according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an image change of a multi-operator image redirection process for maintaining aspect ratio of important content regions according to an embodiment of the present invention;
FIG. 3 is a schematic flowchart of a method for redirecting an image in a width direction according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the comparison of the image scaling effect with the original resolution of 400 × 344 and the target resolution of 280 × 344;
FIG. 5 is a diagram illustrating the comparison of the image scaling effect with the original resolution of 267 × 400 and the target resolution of 134 × 400;
FIG. 6 is a diagram illustrating a comparison of image scaling effects with an original resolution of 400 × 300 and a target resolution of 200 × 300;
fig. 7 is a diagram illustrating the comparison of the image scaling effect with the original resolution of 300 × 400 and the target resolution of 210 × 300.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in fig. 1 and fig. 2, the present invention provides a multi-operator image redirection method for maintaining aspect ratio of important content region, comprising:
(1) carrying out significance detection on an original image to obtain a significance map, and generating an energy map by using the significance map and the normalized gray map;
specifically, the energy profile may be calculated by the following formula:
E＝S×Gm
wherein S represents a saliency map of an original image, and G represents a saliency map of an original imagemRepresenting a normalized gray scale map, wherein:
Figure GDA0002539729460000081
wherein G represents the gray value of the original image, the value of G is (0-255), and GmaxAnd GminRespectively the maximum and minimum of the image grey value.
(2) Carrying out binarization segmentation on the energy map to obtain an important content area and a background area; the method specifically comprises the following steps:
(2.1) selecting an adaptive threshold, and performing binarization segmentation on the energy map according to the adaptive threshold to obtain a binarization energy map; the method comprises the following steps:
(2.1.1) choosing an adaptive threshold, the binarization threshold being T that maximizes the objective function0Value, energy greater than T0The point of (1) is called foreground point, and the energy is less than T0The dots of (1) are referred to as background dots. The objective function can be expressed as:
G(T0)＝ω0(U0-U)2+ω1(U1-U)2
wherein, the ω is0、ω1Respectively representing the proportion of the number of foreground points and background points to the total number of points of the whole atlas, wherein U is0、U1Respectively representing the energy average values of foreground points and background points, wherein U represents the average value of the energy of the whole spectrum;
(2.1.2) performing binary segmentation according to the adaptive threshold, wherein the segmentation principle can be expressed as:
Figure GDA0002539729460000091
wherein E (x, y) and Em(x, y) respectively representing an input energy spectrum and an output binary energy spectrum;
(2.2) detecting the point with the median value of 1 in the binary energy spectrum, and selecting the uppermost (i)1，j1) The lowest part (i)2，j2)、Leftmost (i)3，j3) And rightmost (i)4，j4) Four boundary points;
(2.3) a quadrilateral [ (j)1-j2)×(i4-i3)]Namely, the important content area, and the rest is the background information part;
(3) according to the size of the target image and the sizes of the important content area and the background area, calculating the sizes of the important content area and the background area which are respectively stretched in a non-reorientation dimension;
the proportion of important content is reduced through stretching of non-reorientation dimension, so that the aspect ratio of the important content is protected in the reorientation process, for convenience, the width reduction is taken as an example in the invention, and the stretching steps are as follows:
(3.1) calculating the increment of the non-reoriented dimension (longitudinal height) can be expressed as:
Figure GDA0002539729460000092
wherein h represents the height of the original image, and w1Represents the width of the important content area, said wfRepresents the width of the target image, and Δ h represents the increment of the longitudinal height;
(3.2) allocating different height increment values according to the proportion of the important content area to the background area of the original image, wherein the height increment of the important content area stretching is as follows:
Figure GDA0002539729460000101
wherein, the h1Representing the height of the important content area, said β representing an adaptive parameter, said ah1Representing the height increment of the important content area stretch, the adaptive parameter β is generally selected according to the ratio of the important content area to the background area, and can be expressed as:
Figure GDA0002539729460000102
the height increment of the stretching of the upper background area and the lower background area is respectively as follows:
Figure GDA0002539729460000103
Figure GDA0002539729460000104
wherein, the h2Represents the height of the upper background area, said h3Represents the height of the lower background region, Δ h2Height increment representing stretching of the upper background area, Δ h3Height increments representing stretching of the lower background area;
(4) according to the sizes of the important content area and the background area which are respectively stretched in the non-reorientation dimension, stretching the important content area in the non-reorientation dimension by adopting an even amplification method, stretching the background area of the image in the non-reorientation dimension by adopting a reverse line cutting method, and stretching the stretched image in the reorientation dimension by adopting an even amplification method; wherein, the redirection dimension refers to the dimension needing scaling, and the non-redirection dimension refers to the other dimension;
(4.1) highly stretching the background information part by adopting a reverse line cutting method; the method comprises the following steps:
(4.1.1) energizing magnitude values for each pixel of the background information partial image; during backward line clipping we define the gradient values of the image as energy values, the gradients being defined as follows:
Figure GDA0002539729460000105
then is applied to the digital image
Figure GDA0002539729460000111
Wherein, x represents the original image,
Figure GDA0002539729460000112
representing a convolution, e represents a gradient map of the image.
(4.1.2) searching a cutting line with the minimum energy value by using a dynamic programming method; specifically, the dynamic planning method comprises:
(4.1.2.1) calculating a forward cumulative energy value map corresponding to each pixel point in the image, the calculation process being as follows:
M(i，j)＝e(i，j)+min(M(i-1，j-1)，M(i-1，j)，M(i-1，j+1))
wherein e (i, j) represents the energy of the pixel point, and M (i, j) represents the calculated forward accumulated energy of the pixel point;
(4.1.2.2) finding out the position point (i, j) of the minimum energy value from the last line of the forward accumulated energy map M, taking the point with the minimum accumulated energy in the adjacent three points in the previous line as the point of the line clipping route in the line each time by taking the point as the entry, namely:
seam(i-1)＝min{(i-1，j-1)，(i-1，j)，(i-1，j+1)}
wherein the seam (i-1) represents the point selected by the (i-1) th row of the cutting line;
(4.1.2.3) repeating the search up to the first line according to the rule to obtain the cutting line.
(4.1.3) copy each pixel point of the crop line and insert it under the original pixel point.
(4.2) highly stretching the important content area by adopting a uniform amplification method;
(4.3) stretching the width by using a uniform magnification method to balance the high stretching of the important content area, wherein the stretched width can be expressed as:
Figure GDA0002539729460000113
wherein w represents an original width of an original image, and w represents an original width of an original imageenlRepresenting the image width after stretching;
further, the method for uniform amplification in (4.2) and (4.3) is a bilinear difference method, the bilinear interpolation method uses the gray value of the image as an energy value for calculation, and the step of uniform scaling of the image by the bilinear difference method is as follows:
(4.2.1) calculating the corresponding relation between the zoomed image pixel points and the original image pixel points according to the size of the zoomed target:
Figure GDA0002539729460000121
wherein, the if，jfRespectively represent the abscissa and ordinate of the scaled image, said ki，kjRespectively representing the scaling in the horizontal direction and the vertical direction, and i, j respectively representing the horizontal coordinate and the vertical coordinate of the original image;
(4.2.2) then pixel point (i)f，jf) The corresponding energy values of the four adjacent domain points are respectively as follows:
Figure GDA0002539729460000122
Figure GDA0002539729460000123
wherein, the
Figure GDA0002539729460000124
And
Figure GDA0002539729460000125
respectively represent ifAnd jfFinishing;
(4.2.3) calculation of bilinear difference:
Figure GDA0002539729460000126
wherein, said ef(i, j) represents the pixel point energy value in the zoomed image;
and (4.2.4) calculating the energy value of each pixel point of the zoomed image to obtain the zoomed image.
(5) Performing similarity transformation on the stretched image obtained in the step (4), and uniformly scaling the image until the non-reorientation dimension is the same as that of the original image;
(6) scaling the scaled image obtained in step (5) to a target size by using a reverse line clipping method adding a gradient vector stream, specifically:
(6.1) calculating a GVF vector GVF (i, j) of each pixel point in the image, wherein the GVF (i, j) comprises the size and the direction;
(6.2) with the pixel points (1, j) in the first line of the image as initial points, planning downwards line by line to obtain n line seams, wherein n is the width of the image;
specifically, a candidate cutting line is planned according to the GVF vector direction of each pixel point in the image, and if the current pixel point is (i, j), the candidate cutting line is planned downwards, and the pixel points in the (i +1) th row are selected from the pixel points (i +1, j-1), (i +1, j +1), and the selection rule is as follows:
Figure GDA0002539729460000131
wherein θ [ GVF (i, j) ] represents the direction of the GVF vector of point (i, j).
(6.3) calculating the accumulated energy value of the n seams, wherein the energy is the size of the GVF vector;
and (6.4) deleting the seam with the minimum energy value, and repeating the steps (6.1) to (6.4) until the seam is scaled to the size of the target image, so as to finish image compression.
The method of the present invention is described below with reference to a specific example, and in the following embodiment, a multi-operator image redirection method for maintaining aspect ratio of important content area is described by taking image width reduction as an example. As shown in fig. 3, the method comprises the steps of:
step 100: an original image is read in.
Step 101: and detecting the visual saliency of the input image to obtain a saliency map.
Step 102: and carrying out normalized gray scale transformation on the input image to obtain a gray scale image.
Step 103: and correspondingly multiplying the saliency map and the gray scale map to obtain an energy map of the input image.
Step 104: carrying out threshold segmentation on the energy map by the maximum inter-class variance method (OTSU) to obtain important contents:
(1041) automatically selecting a binary threshold value, wherein the threshold value is T which maximizes the objective function0Value, energy greater than T0The point of (1) is called foreground point, and the energy is less than T0The dots of (1) are referred to as background dots. The objective function can be expressed as:
G(T0)＝ω0(UO-U)2+ω1(U1-U)2
wherein, the ω is0、ω1Respectively representing the proportion of the number of foreground points and background points to the total number of points of the whole atlas, wherein U is0、U1Respectively representing the energy average values of foreground points and background points, wherein U represents the average value of the energy of the whole spectrum.
(1042) And (3) carrying out binary segmentation according to a threshold value, wherein the segmentation principle can be expressed as:
Figure GDA0002539729460000141
wherein E (x, y) and Em(x, y) respectively represent the input energy map and the output binarized energy map.
(1043) Detecting the point with the median value of 1 in the binary image, and selecting the top (i)1，j1) The lowest part (i)2，j2) Leftmost (i)3，j3) And rightmost (i)4，j4) Four boundary points, then a quadrilateral [ (j)1-j2)×(i4-i3)]I.e. important content areas.
Step 105: and calculating the stretching degree of the important content area and the background area according to the size of the divided important content, the size of the original image and the size of the target image.
(1051) The increment of the longitudinal height can be expressed as:
Figure GDA0002539729460000142
wherein h represents the height of the image, and w1Representing the width of the important content area of the image, said wfRepresents the target width of the image, said Δ h representing the increment of the longitudinal height;
(1052) different increment values are distributed according to the proportion of the important content area of the image to the background information part, wherein the height increment of the stretching of the important content area is as follows:
Figure GDA0002539729460000143
wherein, the h1Representing the height of important contents of the image, β representing an adaptive parameter, Δ h1Representing the height increment of important content stretch.
The height increments of the upper and lower background area stretch are:
Figure GDA0002539729460000151
Figure GDA0002539729460000152
wherein, the h2Represents the height of the upper background area, said h3Represents the height of the lower background region, Δ h2Height increment representing stretching of the upper background area, Δ h3Height increments representing stretching of the lower background area;
(1053) the width stretching is carried out to balance the height stretching of the important content area, and the width after stretching can be expressed as:
Figure GDA0002539729460000153
wherein w represents the original width of the image, wenlRepresenting the image width after stretching;
(1054) the value of the parameter β adaptively selected according to the ratio of the important content area to the background area can be expressed as:
Figure GDA0002539729460000154
step 106: the important content area is uniformly enlarged based on the size calculated in step 105, and the height of the background information portion is increased by reverse line cutting and the width is increased by uniform enlargement.
Step 107: and performing similarity transformation on the amplified image to the height equal to the original image.
Step 108: the image is fine-tuned to the target size using a modified line clipping algorithm with the addition of a stream of gradient vectors.
(1081) Calculating a GVF vector GVF (i, j) of each pixel point in the image, wherein the GVF vector GVF (i, j) comprises the size and the direction;
(1082) with pixel points (1, j) in a first line of the image as initial points, planning downwards line by line to obtain n line seams, wherein n is the image width, and the line-by-line planning rule is as follows:
assuming that the current pixel point is (i, j), planning downwards, and selecting the pixel point in the (i +1) th row from the points (i +1, j-1), (i +1, j) and (i +1, j +1), wherein the selection rule is as follows:
Figure GDA0002539729460000161
where θ [ GVF (i, j) ] refers to the direction of the GVF vector for point (i, j).
(1083) Calculating the accumulated energy value of n seams, wherein the energy is the size of a GVF vector;
(1084) and (4) deleting the seam with the minimum energy value and repeating the steps (1081) to (1084) until the seam is scaled to the size of the target image, and completing image compression.
Step 109: and outputting the target image.
The whole self-adaptive image redirection process is completed, the defect that the existing image redirection method cannot embody the high-quality visual effect can be overcome by executing the process, and the purpose of protecting the important object in the redirection process is achieved by adjusting the global proportion of the important object, so that the quality of image redirection is improved, and the image redirection technology obtains the higher-quality visual experience effect.
In order to test the performance of the image redirection method of the present invention, an experimental test was performed using a public image database that is specific to image redirection. In order to measure the performance of the algorithm, the method and the other four image redirection algorithms are respectively compared in performance from two aspects of image subjective effect and image objective scaling quality.
As shown in fig. 4, 5, 6, and 7, the scaling results of four images with different original resolutions and target resolutions are compared. The images scaled by other methods in fig. 4 to 7 all have different degrees of distortion, and the distortion content is marked by red boxes in each image. This is mainly because other methods only reduce the width or height of the image in a single way, and when the visually significant objects contained in the image are too large, the visually significant objects are influenced and distorted in the zooming process. The method of the invention protects the visual salient object globally, and changes the whole size of the visual salient object to adapt to the target size by stretching the width and the height, thereby completely storing the visual salient object, and generating no distortion phenomenon of other methods. The above analysis shows that the subjective effect of the image reconstructed by the method of the present invention is better than that of other methods.
The performance of the method is objectively evaluated by adopting an image scaling quality evaluation standard, the evaluation standard obtains a quality index of image quality evaluation by traversing the correlation of the original image and the scaled image in different scale spaces, the range of the quality index is [0,1], and the higher the numerical value is, the higher the matching degree of the two images is, namely, the better the quality of the scaled image is. The image quality indexes of the 15 related images were analyzed and compared as shown in table 1 below.
TABLE 1 image quality index comparison
SC GVF MO OUR
FIG. 2 0.0970 0.1358 0.1583 0.1594
FIG. 3 0.4029 0.3602 0.3840 0.4469
FIG. 4 0.3124 0.4418 0.2694 0.5478
FIG. 5 0.1498 0.1783 0.0354 0.1858
The above results show that the image quality after zooming by the method of the present invention is obviously superior to the image obtained by other methods, mainly because the method of the present invention enhances the protection of the visually significant content and ensures the integrity of the visually significant content, so that the image after zooming has high spatial correlation with the original image, and the matching degree is obviously improved. Thus, the method of the invention has better performance on objective evaluation results.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (8)

1. A multi-operator image redirection method for keeping aspect ratio of important content areas is characterized by comprising the following steps:
(1) carrying out significance detection on an original image to obtain a significance map, and generating an energy map by using the significance map and the normalized gray map;
(2) carrying out binarization segmentation on the energy map to obtain an important content area and a background area;
(3) according to the size of the target image and the sizes of the important content area and the background area, calculating the sizes of the important content area and the background area which are respectively stretched in a non-reorientation dimension; the method specifically comprises the following steps:
(3.1) calculating the size increment of the original image in the non-reoriented dimension:
Figure FDA0002539729450000011
wherein h represents the size of the original image in the non-reoriented dimension, and w1Representing the size of the important content area in the redirection dimension, wfRepresents the size of the target image in the reorientation dimension and satisfies wf<w1The Δ h represents the size increment of the original image in the non-reoriented dimension;
(3.2) allocating different increment values according to the proportion of the important content area to the background area of the original image, wherein the increment of the important content area stretching is as follows:
Figure FDA0002539729450000012
wherein, the h1Representing the size of the important content area in the non-reoriented dimension, said β representing an adaptation parameter, said ah1A size increment representing a stretching of the important content area;
the size increments of the upper and lower background areas stretch are:
Figure FDA0002539729450000013
Figure FDA0002539729450000014
wherein, the h2Represents the size of the upper background area, said h3Represents the size of the lower background region, Δ h2Size increment representing stretching of the upper background area, Δ h3Size increments representing stretching of the lower background area;
(4) according to the sizes of the important content area and the background area which are respectively stretched in the non-reorientation dimension, stretching the important content area in the non-reorientation dimension by adopting an even amplification method, stretching the background area of the image in the non-reorientation dimension by adopting a reverse line cutting method, and stretching the stretched image in the reorientation dimension by adopting an even amplification method; wherein, the redirection dimension refers to the dimension needing scaling, and the non-redirection dimension refers to the other dimension; the method for stretching the size of the background information part by adopting a reverse line cutting method comprises the following steps:
(4.1.1) energizing magnitude values for each pixel of the background information partial image; defining gradient values of the image as energy values:
Figure FDA0002539729450000021
wherein, x represents the original image,
Figure FDA0002539729450000022
representing a convolution, e representing a gradient map of the image;
(4.1.2) searching a cutting line with the minimum energy value by using a dynamic programming method;
(4.1.3) copying each pixel point of the trim line and inserting it under the original pixel point;
(5) performing similarity transformation on the stretched image obtained in the step (4), and uniformly scaling the image until the non-reorientation dimension is the same as that of the original image;
(6) and (5) scaling the scaled image obtained in the step (5) to a target size by using a line clipping method for adding a gradient vector stream.
2. The method of claim 1, wherein the dynamic programming method comprises:
(4.1.2.1) calculating a forward cumulative energy value map corresponding to each pixel point in the image, the calculation process being as follows:
M(i，j)＝e(i，j)+min(M(i-1，j-1)，M(i-1，j)，M(i-1，j+1))
wherein e (i, j) represents the energy of the pixel point, and M (i, j) represents the calculated forward accumulated energy of the pixel point;
(4.1.2.2) finding out the position point (i, j) of the minimum energy value from the last line of the forward accumulated energy map M, and taking the point with the minimum accumulated energy in the three adjacent points in the last line as the point of the cutting route of the cutting line in the line each time by taking the point as the entry, namely:
seam(i-1)＝min{(i-1，j-1)，(i-1，j)，(i-1，j+1)}
wherein the seam (i-1) represents the point selected by the (i-1) th row of the cutting line;
(4.1.2.3) repeating the search up to the first line according to the rule to obtain the cutting line.
3. The method for redirecting the multioperator image with the aspect ratio of the important content region maintained according to claim 1 or 2, wherein the method for uniformly amplifying in the step (4) is a bilinear difference method, the bilinear interpolation method uses the gray value of the image as an energy value for calculation, and the step of uniformly scaling the image by the bilinear difference method comprises the following steps:
(4.2.1) calculating the corresponding relation between the zoomed image pixel points and the original image pixel points according to the size of the zoomed target:
Figure FDA0002539729450000031
wherein, the if，jfRespectively represent the abscissa and ordinate of the scaled image, said ki，kjRespectively representing the scaling in the horizontal direction and the vertical direction, and i, j respectively representing the horizontal coordinate and the vertical coordinate of the original image;
(4.2.2) then pixel point (i)f，jf) The corresponding energy values of the four adjacent domain points are respectively as follows:
Figure FDA0002539729450000032
Figure FDA0002539729450000033
wherein, the
Figure FDA0002539729450000034
And
Figure FDA0002539729450000035
respectively represent ifAnd jfFinishing;
(4.2.3) calculation of bilinear difference:
ef(i，j)＝([jf]-jf+1)×((if-[if])×x21+([if]-if+1)×x11)+(jf-[jf])×((if-[if])×x22+([if]-if+1)×x21)
wherein, said ef(i, j) represents the pixel point energy value in the zoomed image;
and (4.2.4) calculating the energy value of each pixel point of the zoomed image to obtain the zoomed image.
4. The method for multi-operator image redirection for preserving important content area aspect ratio according to claim 1 or 2, wherein the step (6) is specifically:
(6.1) calculating a GVF vector GVF (i, j) of each pixel point in the image, wherein the GVF (i, j) comprises the size and the direction;
(6.2) planning downwards line by taking pixel points (1, j) in the first line of the image as initial points to obtain n line seams, wherein n is the size of the image in the reorientation dimension;
(6.3) calculating the accumulated energy value of the n seams, wherein the energy is the size of the GVF vector;
and (6.4) deleting the seam with the minimum energy value, and repeating the steps (6.1) to (6.4) until the seam is scaled to the size of the target image, so as to finish image compression.
5. The method for multi-operator image redirection for preserving important content area aspect ratio according to claim 4, wherein said step (6.2) is specifically:
and planning to obtain a candidate cutting line according to the GVF vector direction of each pixel point in the image, and planning downwards if the current pixel point is (i, j), wherein the pixel points in the (i +1) th row are selected from the pixel points (i +1, j-1), (i +1, j) and (i +1, j +1), and the selection rule is as follows:
Figure FDA0002539729450000041
wherein θ [ GVF (i, j) ] represents the direction of the GVF vector of point (i, j).
6. The method for multi-operator image reorientation keeping the aspect ratio of the important content area according to claim 1 or 2, wherein the step (4) adopts a uniform magnification method to stretch the reorientation dimension direction to balance the size stretching of the important content area, and the stretched size is as follows:
Figure FDA0002539729450000051
wherein w represents an original size of an original image, and w represents a size of the original imageenlRepresenting the image size after stretching.
7. The method of multi-operator image redirection preserving important content area aspect ratio according to claim 1 or 2, wherein said energy map is calculated by:
E＝S×Gm
wherein S represents a saliency map of an original image, GmRepresenting a normalized gray scale map, wherein:
Figure FDA0002539729450000052
wherein G represents the gray value of the original image, the value range is (0,255), and G ismaxAnd GminRespectively the maximum and minimum of the image grey value.
8. The method for multi-operator image redirection for preserving important content area aspect ratio according to claim 1 or 2, wherein the step (2) is specifically:
(2.1) selecting an adaptive threshold, and performing binarization segmentation on the energy map according to the adaptive threshold to obtain a binarization energy map; the method comprises the following steps:
(2.1.1) choosing an adaptive threshold, the binarization threshold being T that maximizes the objective function0Value, energy greater than T0The point of (1) is called foreground point and has small energyAt T0The points of (2) are called background points, and the objective function is:
G(T0)＝ω0(U0-U)2+ω1(U1-U)2
wherein, the ω is0、ω1Respectively representing the proportion of the number of foreground points and background points to the total number of points of the whole atlas, wherein U is0、U1Respectively representing the energy average values of foreground points and background points, wherein U represents the average value of the energy of the whole spectrum;
(2.1.2) carrying out binarization segmentation according to the self-adaptive threshold, wherein the segmentation principle is as follows:
Figure FDA0002539729450000053
wherein E (x, y) and Em(x, y) respectively representing an input energy spectrum and an output binary energy spectrum;
(2.2) detecting the point with the median value of 1 in the binary energy spectrum, and selecting the uppermost (i)1，j1) The lowest part (i)2，j2) Leftmost (i)3，j3) And rightmost (i)4，j4) Four boundary points;
(2.3) a quadrilateral [ (j)1-j2)×(i4-i3)]I.e. important content area, the rest are background areas.
CN201710549826.7A 2017-07-07 2017-07-07 Multi-operator image redirection method for keeping aspect ratio of important content area Active CN107330885B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710549826.7A CN107330885B (en) 2017-07-07 2017-07-07 Multi-operator image redirection method for keeping aspect ratio of important content area

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710549826.7A CN107330885B (en) 2017-07-07 2017-07-07 Multi-operator image redirection method for keeping aspect ratio of important content area

Publications (2)

Publication Number Publication Date
CN107330885A CN107330885A (en) 2017-11-07
CN107330885B true CN107330885B (en) 2020-10-02

Family

ID=60197198

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710549826.7A Active CN107330885B (en) 2017-07-07 2017-07-07 Multi-operator image redirection method for keeping aspect ratio of important content area

Country Status (1)

Country Link
CN (1) CN107330885B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108665413A (en) * 2018-04-28 2018-10-16 中国科学院上海高等研究院 A kind of image adaptive amplification method, readable storage medium storing program for executing and terminal
CN109191456A (en) * 2018-09-19 2019-01-11 电子科技大学 Lung CT image processing method and system based on two-dimentional S-transformation
CN109447970B (en) * 2018-10-30 2020-12-29 河北工业大学 Image redirection method based on energy transfer and uniform scaling
CN109493275B (en) * 2018-11-16 2023-01-24 南通大学 Slot cutting redirection method fusing saliency map and depth map
CN109615670A (en) * 2018-12-05 2019-04-12 广西大学 A kind of multi-operator image reorientation method based on compression domain perception of content
CN109600667B (en) * 2018-12-05 2021-04-13 广西大学 Video redirection method based on grid and frame grouping
CN109712164A (en) * 2019-01-17 2019-05-03 上海携程国际旅行社有限公司 Image intelligent cut-out method, system, equipment and storage medium
CN109978858B (en) * 2019-03-27 2021-02-19 华南理工大学 Double-frame thumbnail image quality evaluation method based on foreground detection
CN112529765A (en) * 2019-09-02 2021-03-19 阿里巴巴集团控股有限公司 Image processing method, apparatus and storage medium
CN110580678B (en) * 2019-09-10 2023-06-20 北京百度网讯科技有限公司 Image processing method and device
CN111353382B (en) * 2020-01-10 2022-11-08 广西大学 Intelligent cutting video redirection method based on relative displacement constraint

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013104819A1 (en) * 2012-01-09 2013-07-18 Nokia Corporation Method and apparatus for image scaling in photography
CN103327261A (en) * 2013-06-04 2013-09-25 北京工业大学 Method for simply and fast redirecting video
CN105528591A (en) * 2016-01-14 2016-04-27 电子科技大学 Living fingerprint identification method based on multi-quadrant coding
US9530184B1 (en) * 2007-10-26 2016-12-27 Google Inc. N-patch image resizing

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101521004B (en) * 2008-02-29 2011-12-14 鹏智科技(深圳)有限公司 Electronic device with image processing function and image processing method thereof
US9823824B2 (en) * 2013-08-19 2017-11-21 Kodak Alaris Inc. Context sensitive adaptable user interface

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9530184B1 (en) * 2007-10-26 2016-12-27 Google Inc. N-patch image resizing
WO2013104819A1 (en) * 2012-01-09 2013-07-18 Nokia Corporation Method and apparatus for image scaling in photography
CN103327261A (en) * 2013-06-04 2013-09-25 北京工业大学 Method for simply and fast redirecting video
CN105528591A (en) * 2016-01-14 2016-04-27 电子科技大学 Living fingerprint identification method based on multi-quadrant coding

Also Published As

Publication number Publication date
CN107330885A (en) 2017-11-07

Similar Documents

Publication Publication Date Title
CN107330885B (en) Multi-operator image redirection method for keeping aspect ratio of important content area
Lin et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation
CN109741356B (en) Sub-pixel edge detection method and system
Gao et al. Sand-dust image restoration based on reversing the blue channel prior
CN110197153B (en) Automatic wall identification method in house type graph
JP2003274157A (en) Image processor, image processing method, image processing program, and computer readable recording medium with image processing program recorded
CN105335947A (en) Image de-noising method and image de-noising apparatus
CN106204441B (en) Image local amplification method and device
CN110782407B (en) Single image defogging method based on sky region probability segmentation
JP4772819B2 (en) Image search apparatus and image search method
CN109377450A (en) A kind of edge-protected denoising method
Pok et al. Efficient block matching for removing impulse noise
CN110060260B (en) Image processing method and system
CN111161222A (en) Printing roller defect detection method based on visual saliency
CN114418904A (en) Infrared image enhancement method based on improved histogram equalization and enhanced high-pass filtering
CN115331245A (en) Table structure identification method based on image instance segmentation
CN104268845A (en) Self-adaptive double local reinforcement method of extreme-value temperature difference short wave infrared image
CN109447970B (en) Image redirection method based on energy transfer and uniform scaling
CN111179173A (en) Image splicing method based on discrete wavelet transform and gradient fusion algorithm
WO2022046016A1 (en) Variable clipping level calculation method for clahe algorithm
EP2966613A1 (en) Method and apparatus for generating a super-resolved image from an input image
JP2017134596A (en) Image processing method and process simulation device
CN110705568B (en) Optimization method for image feature point extraction
Chen et al. Candidate region acquisition optimization algorithm based on multi-granularity data enhancement
CN110418195B (en) Real-time video abbreviating method based on optimal cutting

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20211117

Address after: 530004 room 901, floor 9, Qinghu center, No. 36, Zhuxi Avenue, Nanning, Guangxi Zhuang Autonomous Region

Patentee after: Guangxi Xinhao Zhiyun Technology Co.,Ltd.

Address before: 530004 100 East University Road, XiXiangTang District, Nanning, the Guangxi Zhuang Autonomous Region

Patentee before: GUANGXI University