CN114782253A - Image sub-pixel interpolation method and device and computer readable storage medium - Google Patents

Image sub-pixel interpolation method and device and computer readable storage medium Download PDF

Info

Publication number
CN114782253A
CN114782253A CN202210516225.7A CN202210516225A CN114782253A CN 114782253 A CN114782253 A CN 114782253A CN 202210516225 A CN202210516225 A CN 202210516225A CN 114782253 A CN114782253 A CN 114782253A
Authority
CN
China
Prior art keywords
pixel
sub
interpolation
pixels
coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210516225.7A
Other languages
Chinese (zh)
Other versions
CN114782253B (en
Inventor
浣石
廖恒一
陶为俊
戴淦锷
廖新焕
余满
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Zhongheng Housing Safety Appraisal Co ltd
Original Assignee
Guangzhou Zhongheng Housing Safety Appraisal Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Zhongheng Housing Safety Appraisal Co ltd filed Critical Guangzhou Zhongheng Housing Safety Appraisal Co ltd
Priority to CN202210516225.7A priority Critical patent/CN114782253B/en
Priority claimed from CN202210516225.7A external-priority patent/CN114782253B/en
Publication of CN114782253A publication Critical patent/CN114782253A/en
Application granted granted Critical
Publication of CN114782253B publication Critical patent/CN114782253B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)

Abstract

An image sub-pixel interpolation method and device and a computer readable storage medium relate to the technical field of image processing, the interpolation method is to adopt a newly-created interpolation algorithm to calculate the pixel value of a sub-pixel to be inserted, the interpolation algorithm is that the pixel value of the sub-pixel to be inserted is obtained by respectively multiplying the interpolation coefficients of 6 known pixels in the positive and negative directions by respective weights and accumulating, the weights are determined by weight functions, and the weights of the known pixels which are closer are larger. The interpolation algorithm is substituted into the known pixels to calculate and solve the interpolation coefficient of each known pixel, and then the pixel value of the sub-pixel to be inserted is calculated according to the interpolation algorithm. The interpolation method has higher precision.

Description

Image sub-pixel interpolation method and device and computer readable storage medium
Technical Field
The invention relates to the technical field of image processing.
Background
Digital image processing techniques are widely used in many fields, and photogrammetry is one of the application fields. Photogrammetry is a measurement method for shooting images of a measured object and obtaining measured parameters through analysis of image data. Due to the hardware limitation of the camera itself, the captured image is not of infinite pixel density, and therefore the position of the measured mark point in the image may be in the gap between whole pixels (the original pixel of the image, i.e. the pixel captured by the camera), rather than whole pixels, and at this time, a sub-pixel needs to be inserted to make the position of the measured mark point one pixel. Common sub-pixel interpolation algorithms include a bilinear interpolation algorithm, a cubic function interpolation algorithm and the like, and the accuracy of the algorithms is not high enough.
Disclosure of Invention
In view of this, the present invention provides an image sub-pixel interpolation method, which has higher precision.
In order to achieve the above object, the present invention provides the following technical solutions.
1. An image sub-pixel interpolation method, which defines a plane rectangular coordinate system in a plane of an image to be processed, wherein an X axis is parallel to one arrangement direction of whole pixels of the image, a Y axis is parallel to the other arrangement direction of the whole pixels of the image, the unit length of a space between two adjacent whole pixels arranged in the coordinate axis direction is 1,
the interpolation algorithm is as follows:
Figure BDA0003639597940000011
Figure BDA0003639597940000012
Figure BDA0003639597940000013
expressed as coordinates (x + Δ x, y)0) X + ax denotes a position at a distance ax from a known pixel with X coordinate X,
Figure BDA0003639597940000014
expressed in coordinates (x + Δ x, y)0) The pixel value of the interpolated sub-pixel,
Figure BDA0003639597940000021
representing coordinates of (x + k, y)0) The interpolation coefficient of the known pixel of (a),
Figure BDA0003639597940000022
as a function of the weight β5(m (in the coordinate of (k-. DELTA.x, y)0) A function value of the position of (a);
Figure BDA0003639597940000023
representing coordinates of (x)0Y + deltay), Y + deltay representing a position deltay away from a known pixel with Y coordinate,
Figure BDA0003639597940000024
expressed in coordinates of (x)0Y + Δ y) of the pixel values of the sub-pixels inserted at the position of y + Δ y),
Figure BDA0003639597940000025
representing coordinates of (x)0Y + k) of the known pixels,
Figure BDA0003639597940000026
as a function of the weight β5(m) in the coordinate (x)0K- Δ x);
Δx∈(0,1)，Δy∈(0,1)，k∈[-2,3]and k is an integer; x is a radical of a fluorine atom0、y0X and Y coordinates representing known pixels, respectively;
integer pixels refer to original pixels of the image, sub-pixels refer to pixels inserted at a later stage, and known pixels comprise integer pixels and inserted sub-pixels;
the Y coordinate being Y0When m is equal to k- Δ X, X is the coordinate X0When m is equal to k- Δ y, the weighting function β5(m) is:
Figure BDA0003639597940000027
substituting the coordinate values and the pixel values of the known pixels with unknown interpolation coefficients into the interpolation algorithm, and calculating to obtain the interpolation coefficients of the known pixels with unknown interpolation coefficients by making Δ x equal to 0 and Δ y equal to 0;
y coordinate is Y0Converting the X coordinate value of the position of the sub-pixel to be inserted into X + delta X, and calculating according to the interpolation algorithm to obtain the pixel value of the sub-pixel to be inserted;
x coordinate is X0And then converting the Y coordinate value of the position of the sub-pixel to be inserted into Y + delta Y, and calculating the pixel value of the sub-pixel to be inserted according to the interpolation algorithm.
The image sub-pixel interpolation method comprises the steps of firstly obtaining an interpolation coefficient of a known pixel through an interpolation algorithm according to a pixel value and a coordinate of the known pixel, then obtaining a pixel value of the sub-pixel to be inserted through the interpolation algorithm according to the coordinate of the position of the sub-pixel to be inserted, and adopting the same interpolation algorithm to calculate the interpolation coefficient and the pixel value of the sub-pixel, so that the whole pixel and the inserted sub-pixel are ensured to accord with the same rule under the framework of the interpolation algorithm. The image sub-pixel interpolation method of the invention considers the influence of each 3 nearest known pixels on an interpolation area in the positive and negative directions, and is obtained by respectively multiplying interpolation coefficients of each 3 nearest known pixels in the positive and negative directions by corresponding weights to accumulate, wherein the influence is larger for the nearer known pixels, the weight is larger, and the general rule of the pixels in the image is met, wherein the influence on the interpolation area is known by delta X epsilon (0,1), delta Y epsilon (0,1), k epsilon < -2,3 > and k is an integer. In conclusion, the sub-pixels inserted by the image sub-pixel interpolation method can reflect the shot object more accurately, and the precision is higher. The interpolation algorithm also has high-order conductibility and is wider in application range.
2. On the basis of the technical scheme 1, the calculation method of the interpolation coefficient comprises the following steps,
the convolution calculation is fast fourier transformed along the X-axis as follows:
Figure BDA0003639597940000031
Figure BDA0003639597940000032
Figure BDA0003639597940000033
Figure BDA0003639597940000034
for Fourier transform, ωxWhen y is equal to y0The frequency domain characteristic along the X-axis direction,
Figure BDA0003639597940000035
through Fourier transform
Figure BDA0003639597940000036
To obtain
Figure BDA0003639597940000037
Through Fourier transform
Figure BDA0003639597940000038
To obtain
Figure BDA0003639597940000039
Through Fourier transform
Figure BDA00036395979400000311
To obtain
Figure BDA00036395979400000310
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure BDA0003639597940000041
then it follows:
Figure BDA0003639597940000042
then to
Figure BDA0003639597940000043
The interpolation coefficient c (x +) can be obtained by performing inverse Fourier transform
Figure BDA0003639597940000044
The convolution calculation is fast fourier transformed along the Y-axis as follows:
Figure BDA0003639597940000045
Figure BDA0003639597940000046
Figure BDA0003639597940000047
Figure BDA00036395979400000420
for Fourier transform, ωyWhen x is equal to x0The frequency domain characteristic along the Y-axis direction,
Figure BDA0003639597940000048
through Fourier transform
Figure BDA0003639597940000049
To obtain
Figure BDA00036395979400000410
Through Fourier transform
Figure BDA00036395979400000411
To obtain
Figure BDA00036395979400000412
Through Fourier transform
Figure BDA00036395979400000413
To obtain
Figure BDA00036395979400000414
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure BDA00036395979400000415
then it follows:
Figure BDA00036395979400000416
then to
Figure BDA00036395979400000417
Obtaining interpolation coefficient by performing inverse Fourier transform
Figure BDA00036395979400000418
Figure BDA00036395979400000419
Before calculating the pixel value of the sub-pixel to be inserted, the interpolation coefficient of the known pixel with unknown interpolation coefficient is calculated according to the known pixel and the interpolation algorithm, and the interpolation algorithm is carried out on each known pixel with unknown interpolation coefficientColumn arithmetic relates to interpolation coefficients of six known pixels including the column arithmetic, the interpolation coefficients of the known pixels with unknown interpolation coefficients can be solved only by forming an equation set after all the known pixels with unknown interpolation coefficients are subjected to the column arithmetic of an interpolation algorithm, if the traditional method is adopted for calculation, n equations are established in the horizontal direction by taking n pixels in the horizontal direction as an example, the unknown numbers of the equations have n c (x, y), and the expression A of the linear equation set is obtainedn×nXn×1＝Bn×1Solving such a large matrix is very computationally expensive. According to the convolution characteristic of the interpolation algorithm, the frequency domain characteristic can be obtained by performing fast Fourier transform on the convolution of the matrix, and the frequency domain is converted into the time domain after the solution to obtain the interpolation coefficient, so that the calculation amount can be greatly reduced.
3. On the basis of the technical scheme 1, the Y coordinate is Y0If the position with X coordinate being X +2, X +3, X-1 or X-2 has no known pixel, filling the position with dummy pixel as known pixel; the X coordinate is X0If the position with Y coordinate of Y +2, Y +3, Y-1 or Y-2 has no known pixel, filling a dummy pixel as a known pixel at the position; the dummy pixel refers to a pixel of a preset pixel value. Because the image sub-pixel interpolation method of the invention needs to consider the left and right known pixels when calculating the pixel value of a sub-pixel to be inserted, the sub-pixel can not be inserted in the two interpolation areas at the edge of the image, therefore, if the position of the sub-pixel to be inserted is in the two interpolation areas at the edge of the image, the dummy pixel is filled at the corresponding position as the known pixel for calculating the pixel value of the sub-pixel to be inserted in the interpolation area, the sub-pixel can be interpolated and inserted in the whole area of the image, and the problem that the sub-pixel can not be inserted in the edge part of the image is avoided.
4. On the basis of the technical scheme 3, the pixel value of the dummy pixel is the pixel value of the known pixel closest to the dummy pixel.
5. A computer-readable storage medium storing a computer program, which when executed, can implement the image sub-pixel interpolation method according to any one of claims 1 to 4.
6. An image sub-pixel interpolation apparatus, comprising a computer readable storage medium and a processor, wherein the computer readable storage medium is the computer readable storage medium of claim 5, and the processor can execute a computer program stored therein.
Drawings
Fig. 1 is a schematic arrangement diagram of an image whole pixel.
FIG. 2 is a schematic diagram showing the arrangement of the positions of the whole pixel, the dummy pixel and the sub-pixel to be inserted in the first sub-pixel inserting process by using the image sub-pixel interpolation method of the present invention.
FIG. 3 is a schematic diagram showing the arrangement of the positions of whole pixels, sub-pixels, dummy pixels and sub-pixels to be inserted in the process of inserting sub-pixels for the second time by using the image sub-pixel interpolation method of the present invention.
FIG. 4 is a schematic diagram of arrangement of integer pixels and sub-pixels after two sub-pixel insertions are performed by the image sub-pixel interpolation method of the present invention.
FIG. 5 is a schematic diagram of the image sub-pixel interpolation method of the present invention inserting sub-pixels along the X-axis direction.
FIG. 6 is a schematic diagram of 6Y-axis to-be-inserted sub-pixel positions obtained from the X-axis direction by the image sub-pixel interpolation method of the present invention.
FIG. 7 is a schematic diagram of the image sub-pixel interpolation method of the present invention inserting sub-pixels along the Y-axis direction.
Detailed Description
The invention is described in detail below with reference to specific examples.
Description of the terms
Pixel value: the values given by the computer when digitizing an image are referred to, for example, R value, G value, and B value in RGB mode.
Interpolation area: in the X-axis direction or the Y-axis direction, the area between two adjacent known pixels is an interpolation area, such as areas a, b, and c in fig. 2, and area d in fig. 3, where the line segment represents an area.
And (3) integral pixel: refers to the original pixels of the image.
And (3) sub-pixel: refers to the pixels inserted later.
The known pixel: the pixels with known pixel values include integer pixels and interpolated sub-pixels.
A dummy pixel: the pixel is a pixel with a preset pixel value, which is neither a whole pixel nor a pixel with a pixel value calculated by an interpolation algorithm.
Fig. 1 shows arrangement of integer pixels of an image to be processed, the arrangement is arranged along two mutually orthogonal directions, a planar rectangular coordinate system is defined in a plane of the image to be processed, an X axis is parallel to one arrangement direction of the integer pixels of the image, a Y axis is parallel to the other arrangement direction of the integer pixels of the image, a distance between two adjacent integer pixels arranged in a coordinate axis direction is taken as a unit length 1, and in this embodiment, a position of the integer pixel at a lower left corner of the image is taken as a coordinate origin.
The interpolation algorithm in the image sub-pixel interpolation method of the embodiment is as follows:
Figure BDA0003639597940000071
Figure BDA0003639597940000072
in the formula (I), the compound is shown in the specification,
Figure BDA0003639597940000073
expressed as coordinates (x + Δ x, y)0) X + ax denotes a position at a distance ax from a known pixel with X coordinate X,
Figure BDA0003639597940000074
expressed in coordinates (x + Δ x, y)0) The pixel values of the interpolated sub-pixels,
Figure BDA0003639597940000075
representing coordinates of (x + k, y)0) The interpolation coefficient of the known pixel of (a),
Figure BDA0003639597940000076
is a weight ofFunction beta5(m) is in the coordinate (k- Δ x, y)0) A function value of the position of (a);
Figure BDA0003639597940000077
expressed as coordinates (x)0Y + deltay), Y + deltay representing a position deltay away from a known pixel with Y coordinate,
Figure BDA0003639597940000078
expressed in coordinates of (x)0Y + deltay) of the pixel value of the sub-pixel,
Figure BDA0003639597940000079
representing coordinates of (x)0Y + k) of the known pixels,
Figure BDA00036395979400000710
as a function of the weight β5(m) is in the coordinate (x)0K- Δ x).
Wherein, Δ x ∈ (0,1), Δ y ∈ (0,1), k ∈ [ -2,3]And k is an integer. x is a radical of a fluorine atom0、y0Respectively representing the X and Y coordinates of a known pixel.
The Y coordinate being Y0When m is equal to k- Δ X, X is the coordinate X0When m is equal to k- Δ y, the weighting function β5(m) is:
Figure BDA00036395979400000711
the weighting function is a function of a bell-shaped curve waveform, the function waveform is the same at any position, the influence of the known pixel which is farther away from the position of the sub-pixel to be inserted is smaller, and the influence of the known pixel which is closer to the position of the sub-pixel to be inserted is larger. And the weighting function is independent of x, y and only dependent on deltax, deltay, i.e. sub-pixels are inserted at different positions, whose influence of each 3 known pixels in the positive and negative directions depends only on deltax or deltay.
If a sub-pixel is to be inserted at any position of the image, two insertion operations are required, and the first insertion operation is to insert the sub-pixel in the X-axis direction and the Y-axis direction of the whole pixel, as shown in fig. 2. The second insertion operation is to insert sub-pixels in the X-axis direction and Y-axis direction with the inserted sub-pixels, as shown in FIG. 3. After these two insertions, the sub-pixel can be inserted at an arbitrary position, as shown in fig. 4. In this embodiment, for convenience of description, 2 sub-pixels are inserted into each interpolation region, and in actual operation, any number of sub-pixels can be inserted into each interpolation region, and the number of sub-pixels can be selected as needed. The second insertion operation is performed as needed.
In this embodiment, referring to fig. 2, when the sub-pixel is inserted for the first time, the known pixel is a whole pixel, x is an integer in x + Δ x, and Δ x is
Figure BDA0003639597940000081
Similarly, y is an integer in y + Δ y, and Δ y is
Figure BDA0003639597940000082
Referring to FIG. 3, when inserting a sub-pixel for the second time, x is an integer plus x in x + Δ x
Figure BDA0003639597940000083
Or
Figure BDA0003639597940000084
Δ x is
Figure BDA0003639597940000085
Similarly, y in y + Δ y is an integer plus
Figure BDA0003639597940000086
Or
Figure BDA0003639597940000087
Δ y is
Figure BDA0003639597940000088
The interpolation coefficients of the known pixels are solved. And setting Δ x to 0 and Δ y to 0, substituting the coordinate values and pixel values of the known pixels with unknown interpolation coefficients into the interpolation algorithm, and calculating the interpolation coefficients of the known pixels with unknown interpolation coefficients.
Then, the pixel value of the sub-pixel to be inserted is calculated, and the calculation is performed along the X-axis direction and the Y-axis direction respectively. In the direction of the X-axis, i.e. with the Y-coordinate being Y0And then, converting the X coordinate value of the position of the sub-pixel to be inserted into X + delta X, and calculating the pixel value of the sub-pixel to be inserted according to the interpolation algorithm. In the direction of the Y axis, i.e. with the X coordinate being X0And then converting the Y coordinate value of the sub-pixel position to be inserted into Y + delta Y, and calculating the pixel value of the sub-pixel to be inserted according to the interpolation algorithm.
Each sub-pixel insertion operation comprises the two steps of solving the interpolation coefficient and calculating the sub-pixel value.
With reference to fig. 2, a first sub-pixel interpolation operation is performed, an interpolation coefficient of a known pixel with an unknown interpolation coefficient is solved, the known pixels are integer pixels at this time, Δ X is equal to 0, and Δ Y is equal to 0, coordinate values and pixel values of the integer pixels are substituted into the interpolation algorithm to calculate an interpolation coefficient of each integer pixel, and then pixel values of sub-pixels to be interpolated are calculated along the X-axis direction and the Y-axis direction, respectively. In particular, along the X-axis, i.e. with the Y-coordinate being Y0The integer pixels arranged along the X-axis direction have a plurality of rows, and the Y coordinate of the first row integer pixel is 0 when Y is0Converting the X-coordinate value of the sub-pixel position to be inserted to X + Δ X, i.e. 0
Figure BDA0003639597940000091
(X coordinates of two to-be-interpolated sub-pixel positions of the third interpolation region from the left);
Figure BDA0003639597940000092
Figure BDA0003639597940000093
(from the left, the X coordinates of the two sub-pixel positions to be interpolated in the fourth interpolation region. cndot. cndot.) are calculated by substituting the coordinates of the sub-pixel positions to be interpolated into the interpolation algorithm, and then y is calculated to obtain the pixel value of each sub-pixel position to be interpolated0Respectively etc. ofRepeating the operations at 1,2 and 3. The same is true along the Y-axis direction. After the sub-pixel insertion operation is completed along both the X-axis direction and the Y-axis direction, the first sub-pixel insertion operation is completed.
With reference to fig. 3, performing a second sub-pixel interpolation operation, first solving an interpolation coefficient of a known pixel with an unknown interpolation coefficient, where the known pixel includes an integer pixel and a first interpolated sub-pixel, and because the interpolation coefficient of the integer pixel is already solved when the sub-pixel is interpolated for the first time, it is not necessary to solve the interpolation coefficient of the integer pixel, and in addition, the interpolation coefficient of the integer pixel is not actually used in the second sub-pixel interpolation process. And solving the interpolation coefficient of the sub-pixel to be inserted for the first time, enabling the delta X to be 0 and the delta Y to be 0, substituting the coordinate value and the pixel value of the sub-pixel to be inserted for the first time into the interpolation algorithm, calculating the interpolation coefficient of the sub-pixel to be inserted for the first time, and then calculating the pixel value of the sub-pixel to be inserted along the X-axis direction and the Y-axis direction respectively. In particular, along the X-axis, i.e. with the Y-coordinate Y0The Y coordinate of the first inserted subpixel in the first row is
Figure BDA0003639597940000094
At this time
Figure BDA0003639597940000095
Converting the X-coordinate value of the sub-pixel position to be inserted into X + Deltax, i.e.
Figure BDA0003639597940000096
(X coordinates of two to-be-interpolated sub-pixel positions of the third interpolation region from the left);
Figure BDA0003639597940000097
Figure BDA0003639597940000098
(the X coordinates of the two sub-pixel positions to be inserted of the fourth interpolation region from the left) are substituted into the interpolation algorithm to calculate the sub-pixel positions to be insertedSet pixel value, then y0Are respectively equal to
Figure BDA0003639597940000099
The above operations were repeated separately. The same is true along the Y-axis direction. After the sub-pixel inserting operation is completed along both the X-axis direction and the Y-axis direction, the sub-pixel inserting operation for the second time is completed.
Whether sub-pixels are inserted for the first time or for the second time, the two interpolation regions (regions b and c in fig. 2 and region d in fig. 3) of the image edge, and the number of known pixels in the positive direction or the negative direction is less than 3, that is: the Y coordinate being Y0When there is no known pixel at the location of the X coordinate X +2, X +3, X-1 or X-2, the X coordinate X0There are no known pixels at the location of the Y coordinate Y +2, Y +3, Y-1, or Y-2. Therefore, it is impossible to insert sub-pixels in two interpolation regions at the edge of an image, and in order to insert sub-pixels also in these regions, the Y coordinate is Y0If the position with X coordinate being X +2, X +3, X-1 or X-2 has no known pixel, filling the position with dummy pixel as known pixel; x coordinate is X0If there is no known pixel at the position with Y coordinate of Y +2, Y +3, Y-1 or Y-2, then filling dummy pixel at the position as known pixel, so that sub-pixel can be inserted into two interpolation regions at the edge of the image, that is, sub-pixel can be inserted into the whole image by interpolation calculation, thereby avoiding that sub-pixel can not be inserted into the edge of the image. The filling of the dummy pixels is performed before the calculation of the interpolation coefficients, so that the interpolation coefficients of two known pixels at the edge of the image can be calculated.
In this embodiment, the method for calculating and solving the interpolation coefficient includes:
the convolution calculation is fast fourier transformed along the X-axis as follows:
Figure BDA0003639597940000101
Figure BDA0003639597940000102
Figure BDA0003639597940000103
Figure BDA0003639597940000104
for Fourier transform, ωxWhen y is equal to y0The frequency domain characteristic along the X-axis direction,
Figure BDA0003639597940000105
through Fourier transform
Figure BDA0003639597940000106
To obtain
Figure BDA0003639597940000107
Through Fourier transform
Figure BDA0003639597940000108
To obtain
Figure BDA0003639597940000109
Through Fourier transform
Figure BDA00036395979400001010
To obtain
Figure BDA00036395979400001011
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure BDA00036395979400001012
then it follows:
Figure BDA0003639597940000111
then to
Figure BDA0003639597940000112
Obtaining an interpolation coefficient by performing inverse Fourier transform
Figure BDA0003639597940000113
Figure BDA0003639597940000114
The convolution calculation is fast fourier transformed along the Y-axis as follows:
Figure BDA0003639597940000115
Figure BDA0003639597940000116
Figure BDA0003639597940000117
Figure BDA0003639597940000118
for Fourier transform, ωyWhen x is equal to x0The frequency domain characteristic along the Y-axis direction,
Figure BDA0003639597940000119
through Fourier transform
Figure BDA00036395979400001110
To obtain
Figure BDA00036395979400001111
Through Fourier transform
Figure BDA00036395979400001112
To obtain
Figure BDA00036395979400001113
Through Fourier transform
Figure BDA00036395979400001114
To obtain
Figure BDA00036395979400001115
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure BDA00036395979400001116
then it follows:
Figure BDA00036395979400001117
then to
Figure BDA00036395979400001118
Obtaining an interpolation coefficient by performing inverse Fourier transform
Figure BDA00036395979400001119
Figure BDA00036395979400001120
According to the convolution characteristic of the interpolation algorithm, the convolution of the matrix can be subjected to fast Fourier transform to obtain frequency domain characteristics, and the frequency domain is converted into a time domain after solving to obtain an interpolation coefficient, so that the calculation amount can be greatly reduced. The concrete description is as follows:
discretizing the equation of continuity, let
Figure BDA00036395979400001121
Wherein M is 0,1,2, …, M-1, then
Figure BDA0003639597940000121
GmIs that
Figure BDA0003639597940000122
One period of discretization of
Figure BDA0003639597940000123
The term has symmetry
Figure BDA0003639597940000124
Periodicity of the period
Figure BDA0003639597940000125
Reducibility
Figure BDA0003639597940000126
Figure BDA0003639597940000127
Wherein
Figure BDA0003639597940000128
Figure BDA0003639597940000129
In the above formula, only the data results of the first half, i.e., 0,1,2, …, M/2-1, need to be calculated. The second half according to periodicity
Figure BDA00036395979400001210
Therefore, it is not only easy to use
Figure BDA00036395979400001211
Figure BDA00036395979400001212
The same reason is that
Figure BDA00036395979400001213
Figure BDA00036395979400001214
By analogy, the data of M/2 can be further regarded as odd and even parts, and is divided into two parts similarly, and only the data of the first half needs to be calculated. Thereby greatly reducing the amount of computation.
The following is a specific calculation, with reference to fig. 5-7:
when m is more than or equal to-3 and less than or equal to-2:
Figure BDA0003639597940000131
when the sub-pixel offset is generated, i.e., -3 ≦ -2+ Δ m ≦ -2, where Δ m represents the sub-pixel offset value
Figure BDA0003639597940000132
When m is more than or equal to-2 and less than or equal to-1:
Figure BDA0003639597940000133
when m is more than or equal to-1 and less than or equal to 0:
Figure BDA0003639597940000134
when m is more than or equal to 0 and less than or equal to 1:
Figure BDA0003639597940000135
when m is more than or equal to 1 and less than or equal to 2:
Figure BDA0003639597940000136
when m is more than or equal to 2 and less than or equal to 3:
Figure BDA0003639597940000137
g(xf+Δx，ys)＝[cf-2，s cf-1，s cf，s cf+1，s cf+2，s cf+3，s]MΔP
wherein
Figure BDA0003639597940000141
Figure BDA0003639597940000142
Then g (x)f+Δm，ys)＝[cf-2，s cf-1，s cf，s cf+1，s cf+2，s cf+3，s]MΔP
Subscripts "f" and "s" indicate positions in the X-axis direction and the Y-axis direction of the pixel, respectively.
The sub-pixel shift Δ m along the X-axis direction is calculated.
Sequentially moving the image by one pixel along the Y-axis direction, and calculating according to the above process to obtain p1，g(xf+Δx，ys+1)＝[cf-2，s+1 cf-1，s+1 cf，s cf+1，s+1 cf+2，s+1 cf+3，s+1]MΔP
By analogy, p is obtained-2、p-1、p2And p3As shown in fig. 5 and 6.
g(xf+Δx，ys-2)＝[cf-2，s-2 cf-1，s-2 cf，s-2 cf+1，s-2 cf+2，s-2 cf+3，s-2]MΔP
g(xf+Δx，ys-1)＝[cf-2，s-1 cf-1，s-1 cf，s-1 cf+1，s-1 cf+2，s-1 cf+3，s-1]MΔP
g(xf+Δx，ys+2)＝[cf-2，s+2 cf-1，s+2 cf，s+2 cf+1，s+2 cf+2，s+2 cf+3，s+2]MΔP
g(xf+Δx，ys+3)＝[cf-2，s+3 cf-1，s+3 cf，s+3 cf+1，s+3 cf+2，s+3 cf+3，s+3]MΔP
As shown in fig. 7, the same operation is then performed in the Y-axis direction, which specifically includes: the interpolation algorithm is compared with the calculated point p-2、p-1、p0、p1、p2And p3Performing interpolation operation by calculating coefficient [ c ] according to Fourier transform]f，sThen obtaining the result by an interpolation algorithm
g(xf+Δm，ys+Δn)＝ΔQTMT[C]f，sMΔP
ΔQ＝[1 Δn Δn2 Δn3 Δn4 Δn5]T
Where Δ n represents a sub-pixel offset value along the y-direction, f represents a pixel position in the x-direction, and s represents a pixel position in the y-direction.
Figure BDA0003639597940000151
Thus, an arbitrary integer pixel position (x) is obtainedf,ys) The interpolation point pixel value at the offset amount (Δ m, Δ n).
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the protection scope of the present invention, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims (6)

1. An image sub-pixel interpolation method is characterized in that a plane rectangular coordinate system is defined in a plane of an image to be processed, an X axis is parallel to one arrangement direction of whole pixels of the image, a Y axis is parallel to the other arrangement direction of the whole pixels of the image, the distance between two adjacent whole pixels arranged in the coordinate axis direction is taken as a unit length 1,
the interpolation algorithm is as follows:
Figure FDA0003639597930000011
Figure FDA0003639597930000012
Figure FDA0003639597930000013
expressed as coordinates (x + Δ x, y)0) X + ax denotes a position at a distance ax from a known pixel with X coordinate X,
Figure FDA0003639597930000014
expressed in coordinates of (x + Δ x, y)0) The pixel values of the interpolated sub-pixels,
Figure FDA0003639597930000015
representing coordinates of (x + k, y)0) The interpolation coefficient of the known pixel of (2),
Figure FDA0003639597930000016
as a function of the weight beta5(m) has the coordinate of (k-Deltax, y)0) A function value of the position of (a);
Figure FDA0003639597930000017
expressed as coordinates (x)0Y + Δ y) positionY + ay denotes a position at a distance ay from a known pixel with Y coordinate Y,
Figure FDA0003639597930000018
expressed in coordinates of (x)0Y + Δ y) of the pixel values of the sub-pixels inserted at the position of y + Δ y),
Figure FDA0003639597930000019
expressed as coordinates (x)0Y + k) interpolation coefficients of known pixels,
Figure FDA00036395979300000110
as a function of the weight beta5(m) in the coordinate (x)0K- Δ x);
Δx∈(0，1)，Δy∈(0，1)，k∈[-2，3]and k is an integer; x is a radical of a fluorine atom0、y0X and Y coordinates representing known pixels, respectively;
integer pixels refer to original pixels of the image, sub-pixels refer to pixels inserted at a later stage, and known pixels comprise integer pixels and inserted sub-pixels;
y coordinate is Y0When m is equal to k- Δ X, X is the coordinate X0When m is equal to k- Δ y, the weighting function β5(m) is:
Figure FDA0003639597930000021
substituting the coordinate values and the pixel values of the known pixels with unknown interpolation coefficients into the interpolation algorithm to calculate the interpolation coefficients of the known pixels with unknown interpolation coefficients, wherein Δ x is equal to 0 and Δ y is equal to 0;
y coordinate is Y0Converting the X coordinate value of the position of the sub-pixel to be inserted into X + delta X, and calculating according to the interpolation algorithm to obtain the pixel value of the sub-pixel to be inserted;
the X coordinate is X0And then converting the Y coordinate value of the sub-pixel position to be inserted into Y + delta Y, and calculating the pixel value of the sub-pixel to be inserted according to the interpolation algorithm.
2. The image sub-pixel interpolation method of claim 1, wherein the interpolation coefficient is calculated by,
the convolution calculation is fast fourier transformed along the X-axis as follows:
Figure FDA0003639597930000022
Figure FDA0003639597930000023
Figure FDA0003639597930000024
Figure FDA0003639597930000025
for Fourier transform, ωxIs when y ═ y0The frequency domain characteristic along the X-axis direction,
Figure FDA0003639597930000026
through Fourier transform
Figure FDA0003639597930000027
To obtain
Figure FDA0003639597930000028
Through Fourier transform
Figure FDA0003639597930000029
To obtain
Figure FDA00036395979300000210
Through Fourier transform
Figure FDA00036395979300000211
To obtain
Figure FDA00036395979300000212
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure FDA0003639597930000031
then it follows:
Figure FDA0003639597930000032
then to
Figure FDA0003639597930000033
Obtaining an interpolation coefficient by performing inverse Fourier transform
Figure FDA0003639597930000034
The convolution calculation is fast fourier transformed along the Y-axis as follows:
Figure FDA0003639597930000035
Figure FDA0003639597930000036
Figure FDA0003639597930000037
Figure FDA0003639597930000038
for Fourier transform, ωyWhen x is equal to x0The frequency domain characteristic along the Y-axis direction,
Figure FDA0003639597930000039
through Fourier transform
Figure FDA00036395979300000310
To obtain
Figure FDA00036395979300000311
Through Fourier transform
Figure FDA00036395979300000312
To obtain
Figure FDA00036395979300000313
Through Fourier transform
Figure FDA00036395979300000314
To obtain
Figure FDA00036395979300000315
Converting time domain calculation into frequency domain calculation, and performing Fourier transform to obtain:
Figure FDA00036395979300000316
then it follows:
Figure FDA00036395979300000317
then to
Figure FDA00036395979300000318
Obtaining an interpolation coefficient by performing inverse Fourier transform
Figure FDA00036395979300000319
3. The method of claim 1, wherein the Y coordinate is Y0If the position with X coordinate being X +2, X +3, X-1 or X-2 has no known pixel, filling the position with dummy pixel as known pixel; the X coordinate is X0If the position with Y coordinate being Y +2, Y +3, Y-1 or Y-2 has no known pixel, filling a dummy pixel in the position as a known pixel; the dummy pixel refers to a pixel of a preset pixel value.
4. The method of claim 3, wherein the pixel value of the dummy pixel is the pixel value of the known pixel closest to the dummy pixel.
5. Computer-readable storage medium, in which a computer program is stored which, when being executed, is able to carry out the image sub-pixel interpolation method according to any one of claims 1 to 4.
6. An image sub-pixel interpolation apparatus comprising a computer readable storage medium and a processor, wherein the computer readable storage medium is the computer readable storage medium of claim 5 and the processor is operable to execute a computer program stored therein.
CN202210516225.7A 2022-05-12 Image subpixel interpolation method and device, and computer readable storage medium Active CN114782253B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210516225.7A CN114782253B (en) 2022-05-12 Image subpixel interpolation method and device, and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210516225.7A CN114782253B (en) 2022-05-12 Image subpixel interpolation method and device, and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN114782253A true CN114782253A (en) 2022-07-22
CN114782253B CN114782253B (en) 2024-10-22

Family

ID=

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101212672A (en) * 2006-12-30 2008-07-02 安凯（广州）软件技术有限公司 Video content adaptive sub-pixel interpolation method and device
WO2019242528A1 (en) * 2018-06-20 2019-12-26 腾讯科技（深圳）有限公司 Video encoding and decoding method and device, storage medium, and computer device
CN110823358A (en) * 2019-11-08 2020-02-21 重庆邮电大学 Building vibration displacement measurement method based on visual processing
CN111968178A (en) * 2020-08-10 2020-11-20 吉林大学 Sub-pixel positioning method based on particle swarm algorithm

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101212672A (en) * 2006-12-30 2008-07-02 安凯（广州）软件技术有限公司 Video content adaptive sub-pixel interpolation method and device
WO2019242528A1 (en) * 2018-06-20 2019-12-26 腾讯科技（深圳）有限公司 Video encoding and decoding method and device, storage medium, and computer device
CN110823358A (en) * 2019-11-08 2020-02-21 重庆邮电大学 Building vibration displacement measurement method based on visual processing
CN111968178A (en) * 2020-08-10 2020-11-20 吉林大学 Sub-pixel positioning method based on particle swarm algorithm

Similar Documents

Publication Publication Date Title
US8717632B2 (en) Image processing method, image processing device and scanner
CN110738707A (en) Distortion correction method, device, equipment and storage medium for cameras
CN111263142A (en) Method, device, equipment and medium for testing optical anti-shake of camera module
JP2002517045A (en) How to determine the exact position of the template match point
CN110136114A (en) A kind of wave measurement method, terminal device and storage medium
CN106875328B (en) Image processing method and device
JP2011141710A (en) Device, method and program for estimating depth
JP2007538436A (en) Method and apparatus for interpolating and correcting images
CN114782253A (en) Image sub-pixel interpolation method and device and computer readable storage medium
CN114782253B (en) Image subpixel interpolation method and device, and computer readable storage medium
JP2013142948A5 (en)
CN113642397A (en) Object length measuring method based on mobile phone video
CN111340714B (en) Moire pattern processing method and device and electronic equipment
JP5252642B2 (en) Depth estimation apparatus, depth estimation method, and depth estimation program
TWI406187B (en) Fast and high quality image/video interpolation method and apparatus
CN116823639A (en) Image distortion correction method, device, equipment and storage medium
CN109035201B (en) Object deformation obtaining method based on digital image diffraction
CN107240064B (en) Image correction method and device and terminal
US20230169624A1 (en) Image scaling method and apparatus thereof
US8509568B2 (en) Image processing apparatus and image processing method
JP3394551B2 (en) Image conversion processing method and image conversion processing device
CN115131243A (en) Image processing method and device, electronic equipment and storage medium
CN111340724B (en) Image jitter removing method and device in LED screen correction process
CN112927276A (en) Image registration method and device, electronic equipment and storage medium
JP5387289B2 (en) Image processing apparatus, image processing method and program thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant