WO2021218404A1 - 图像处理方法、图像处理装置及显示装置 - Google Patents

图像处理方法、图像处理装置及显示装置 Download PDF

Info

Publication number
WO2021218404A1
WO2021218404A1 PCT/CN2021/079774 CN2021079774W WO2021218404A1 WO 2021218404 A1 WO2021218404 A1 WO 2021218404A1 CN 2021079774 W CN2021079774 W CN 2021079774W WO 2021218404 A1 WO2021218404 A1 WO 2021218404A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
pixel value
target
value
derivative
Prior art date
Application number
PCT/CN2021/079774
Other languages
English (en)
French (fr)
Inventor
刘小磊
孙建康
陈丽莉
张�浩
Original Assignee
京东方科技集团股份有限公司
北京京东方光电科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司, 北京京东方光电科技有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US17/642,455 priority Critical patent/US20220301101A1/en
Publication of WO2021218404A1 publication Critical patent/WO2021218404A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction

Definitions

  • This application relates to the field of image processing technology, and in particular to an image processing method, an image processing device, and a display device.
  • the resolution of the display is getting higher and higher, but due to the limitation of the image sensor in the display, the resolution of the image generated by the image sensor is low, and it is difficult to meet the resolution requirement of the display.
  • the application provides an image processing method, an image processing device, and a display device.
  • the technical solution is as follows:
  • an image processing method includes:
  • interpolation processing on the image to be processed to obtain an interpolated image, where the interpolated image includes a plurality of original pixels and a plurality of interpolated pixels, and the initial pixel value of each interpolated pixel is determined according to the pixel value of at least one of the original pixels;
  • the first candidate pixel value is greater than the initial pixel value
  • the second candidate pixel value is less than the initial pixel value
  • the target original pixel is an original pixel adjacent to the interpolation pixel among the plurality of original pixels.
  • the adjusting the initial pixel value of the interpolation pixel to obtain the first candidate pixel value and the second candidate pixel value includes:
  • the initial pixel value of the interpolation pixel is reduced by the pixel step value to obtain a second candidate pixel value.
  • the determining the first difference between the second derivative of the first candidate pixel value and the second derivative of the pixel value of the target original pixel among the plurality of original pixels includes:
  • the determining the second difference between the second derivative of the second candidate pixel value and the second derivative of the pixel value of the target original pixel includes:
  • the target original pixels include: two first target pixels and two second target pixels, wherein a connection line between each of the first target pixels and the interpolation pixel extends in a first direction , The connection line between each of the second target pixels and the interpolation pixels extends along a second direction, and the first direction is perpendicular to the second direction;
  • the determining the first and second derivative of the first candidate pixel value according to the first candidate pixel value and the pixel value of the target original pixel includes:
  • the determining the second-order derivative of the pixel value of the target original pixel according to the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel includes:
  • the connecting line between is the pixel value of the pixel in the first direction, and the second-order derivative of the pixel value of the target pixel in the first direction is determined;
  • the connecting line between is the pixel value of the pixel in the second direction, and the second-order derivative of the pixel value of the target pixel in the second direction is determined;
  • the determining the first difference between the first second-order derivative and the second-order derivative of the pixel value of the target original pixel includes:
  • the difference values of the four first directions and the difference values of the four second directions are summed to obtain a first difference value.
  • the two first target pixels corresponding to the interpolation pixel in the i-th row and the j-th column in the interpolation image are respectively the pixel in the i+1-th row and the j-1th column and the i-th-th column in the interpolated image.
  • the two second target pixels corresponding to the interpolated pixels in the i-th row and jth column are the pixels in the i-1th row and j-1th column and the i+1th row, respectively Column j+1 column pixel;
  • the second derivative I 11 + (i, j) of the first candidate pixel value of the interpolation pixel in the i-th row and j-th column in the first direction satisfies:
  • the I(i+1, j-1) is the pixel value of the pixel in the i+1th row and the j-1th column
  • the I(i-1, j+1) is the i-th column.
  • the I(i,j) is the initial pixel value of the pixel in the i-th row and the j-th column
  • the ⁇ is the pixel step value
  • i and j are both Is a positive integer greater than 1;
  • the second derivative I 22 + (i, j) of the first candidate pixel value of the interpolation pixel in the i-th row and j-th column in the second direction satisfies:
  • the I(i-1, j-1) is the pixel value of the pixel in the i-1th row and the j-1th column
  • the I(i+1, j+1) is the i+th column.
  • the target original pixels include: two first target pixels and two second target pixels, wherein a connection line between each of the first target pixels and the interpolation pixel extends in a first direction , The connection line between each of the second target pixels and the interpolation pixels extends along a second direction, and the first direction is perpendicular to the second direction;
  • the determining the second second derivative of the second candidate pixel value according to the second candidate pixel value and the pixel value of the target original pixel includes:
  • the determining the second-order derivative of the pixel value of the target original pixel according to the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel includes:
  • the connecting line between is the pixel value of the pixel in the first direction, and the second-order derivative of the pixel value of the target pixel in the first direction is determined;
  • the connecting line between is the pixel value of the pixel in the second direction, and the second-order derivative of the pixel value of the target pixel in the second direction is determined;
  • the determining the second difference between the second second derivative and the second derivative of the pixel value of the target original pixel includes:
  • the difference values of the four first directions and the difference values of the four second directions are summed to obtain a second difference value.
  • the two first target pixels corresponding to the interpolation pixel in the i-th row and the j-th column in the interpolation image are respectively the pixel in the i+1-th row and the j-1th column and the i-th-th column in the interpolated image.
  • 1 column of pixels in the j+1th column, and the two second target pixels corresponding to the interpolated pixels in the i-th row and jth column are the pixels in the i-1th row and j-1th column and the i+1th row, respectively column j + 1 column of pixels;
  • I 11 - (i, j) I (i + 1, j-1) + I (i-1, j + 1) -2 ⁇ [I (i, j) - ⁇ ];
  • the I(i+1, j-1) is the pixel value of the pixel in the i+1th row and the j-1th column
  • the I(i-1, j+1) is the i-th column.
  • the I(i,j) is the initial pixel value of the pixel in the i-th row and the j-th column
  • the ⁇ is the pixel step value
  • i and j are both Is a positive integer greater than 1;
  • the second pixel value of the interpolation pixel in the i-th row j-th column in the second direction, the second derivative I 22 - (i, j) is satisfied:
  • I 22 - (i, j) I (i-1, j-1) + I (i + 1, j + 1) -2 ⁇ [I (i, j) - ⁇ ];
  • the I(i-1, j-1) is the pixel value of the pixel in the i-1th row and the j-1th column
  • the I(i+1, j+1) is the i+th column.
  • the performing interpolation processing on the image to be processed to obtain an interpolated image includes:
  • first gradient is greater than the second gradient, use an interpolation algorithm to perform interpolation processing on the first area to be interpolated along the second direction to obtain an initial interpolated image;
  • the interpolation algorithm is used to perform interpolation processing on the second area to be interpolated along the third direction to obtain an interpolated image.
  • the pixel located in row 2i-1 and column 2i-1 in the interpolated image is the original pixel, and the pixel value of the pixel in row 2i-1 and column 2i-1 in the interpolated image is the same as
  • the pixel values of the pixels in the i-th row and the j-th column in the image to be processed are equal.
  • an image processing device including:
  • the interpolation processing module is used to perform interpolation processing on the image to be processed to obtain an interpolated image.
  • the interpolated image includes a plurality of original pixels and a plurality of interpolated pixels, and the initial pixel value of each interpolated pixel is based on at least one of the original pixels The pixel value is determined;
  • the adjustment module is configured to adjust the initial pixel value of the interpolation pixel for each interpolation pixel to obtain a first candidate pixel value and a second candidate pixel value, and the first candidate pixel value is greater than the initial pixel value. Pixel value, the second candidate pixel value is less than the initial pixel value;
  • a first determining module configured to determine the first difference between the second derivative of the first candidate pixel value and the second derivative of the pixel value of the target original pixel among the plurality of original pixels;
  • a second determining module configured to determine a second difference between the second derivative of the second candidate pixel value and the second derivative of the pixel value of the target original pixel;
  • An update module configured to update the initial pixel value of the interpolation pixel by using the candidate pixel value corresponding to the smaller difference value of the first difference value and the second difference value;
  • the target original pixel is an original pixel adjacent to the interpolation pixel among the plurality of original pixels.
  • the adjustment module is used to:
  • the initial pixel value of the interpolation pixel is reduced by the pixel step value to obtain a second candidate pixel value.
  • the first determining module includes:
  • a first determining submodule configured to determine a first and second derivative of the first candidate pixel value according to the first candidate pixel value and the pixel value of the target original pixel;
  • the second determining sub-module is configured to determine the second-order derivative of the pixel value of the target original pixel according to the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel;
  • a third determining sub-module configured to determine the first difference between the first second derivative and the second derivative of the pixel value of the target original pixel
  • the second determining module includes:
  • a fourth determining submodule configured to determine a second second derivative of the second candidate pixel value according to the second candidate pixel value and the pixel value of the target original pixel;
  • the fifth determining sub-module is configured to determine the second difference between the second second derivative and the second derivative of the pixel value of the target original pixel.
  • an image processing device includes a processor, a memory, and a computer program stored on the memory and running on the processor, and the processor executes the computer program When realizing the image processing method as described in the above aspect.
  • a computer-readable storage medium is provided, and instructions are stored in the computer-readable storage medium.
  • the computer-readable storage medium runs on a computer, the computer executes the image described in the above-mentioned aspect. Approach.
  • a display device comprising: a display panel and the image processing device as described in the foregoing aspect;
  • the display panel is used for displaying the image processed by the image processing device.
  • Fig. 1 is a flowchart of an image processing method provided by an embodiment of the present application
  • FIG. 2 is a flowchart of a method for generating an interpolated image provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a first area to be interpolated according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of a second area to be interpolated according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a relationship between a to-be-processed image and an interpolated image provided by an embodiment of the present application
  • Fig. 6 is a flowchart of a method for image processing an interpolated image provided by an embodiment of the present application
  • FIG. 7 is a flowchart of a method for determining the first and second derivative of the first candidate pixel value according to an embodiment of the present application
  • FIG. 8 is a flowchart of a method for determining the second-order derivative of the pixel value of a target original pixel according to an embodiment of the present application
  • FIG. 9 is a flowchart of a method for determining a first difference provided by an embodiment of the present application.
  • FIG. 10 is a flowchart of a method for determining a second second-order derivative of a second candidate pixel value according to an embodiment of the present application
  • FIG. 11 is a flowchart of a method for determining a second difference provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of an image processing effect provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a first determining module provided by an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of a second determining module provided by an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of an interpolation processing module provided by an embodiment of the present application.
  • FIG. 17 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the present application.
  • the image processed by the method in the related art usually produces artifacts, and the quality of the image is low.
  • Fig. 1 is a flowchart of an image processing method provided by an embodiment of the present application. This method can be applied to an image processing device. As can be seen with reference to Figure 1, the method may include:
  • Step 101 Perform interpolation processing on the image to be processed to obtain an interpolated image.
  • the image processing device may pre-store an interpolation algorithm, and after receiving the image to be processed, the image processing device may use the pre-stored interpolation algorithm to perform interpolation processing on the image to be processed to obtain an interpolated image.
  • the interpolated image may include a plurality of original pixels and a plurality of interpolated pixels, and the initial pixel value of each interpolated pixel may be determined according to the pixel value of at least one original pixel.
  • the initial pixel value of each interpolation pixel in the interpolation image may be determined according to the pixel value of the original pixel adjacent to the interpolation pixel among the multiple original pixels.
  • the interpolation algorithm pre-stored in the image processing device may be any one of: nearest neighbor interpolation algorithm, bilinear interpolation algorithm, and bicubic convolution interpolation algorithm.
  • Step 102 For each interpolated pixel, adjust the initial pixel value of the interpolated pixel to obtain the first candidate pixel value and the second candidate pixel value.
  • the pixel values of all pixels are known pixel values.
  • the image processing device may adjust the initial pixel value of the interpolation pixel for each interpolation pixel to obtain the first candidate pixel value and the second candidate pixel value.
  • the first candidate pixel value may be greater than the initial pixel value, and the second candidate pixel value may be less than the initial pixel value. That is, for each interpolated pixel, two candidate pixel values corresponding to the interpolated pixel can be obtained.
  • Step 103 Determine a first difference between the second derivative of the first candidate pixel value and the second derivative of the pixel value of the target original pixel among the multiple original pixels.
  • the image processing device may determine the second-order derivative of each original pixel among the multiple original pixels of the interpolated image. For each interpolation pixel, the image processing device may determine the second derivative of the first candidate pixel value after determining the first candidate pixel value of the interpolation pixel. And, for each interpolated pixel, the image processing device may determine the first difference between the second derivative of the first candidate pixel value of the interpolated pixel and the second derivative of the pixel value of the target original pixel among the plurality of original pixels.
  • the target original pixel may be an original pixel adjacent to the interpolation pixel among a plurality of original pixels.
  • the first difference is the difference between the second derivative of the first candidate pixel value of the interpolated pixel and the second derivative of the pixel value of the original pixel adjacent to the interpolated pixel among the plurality of original pixels.
  • the number of adjacent original pixels is usually multiple, that is, the target original pixel can be multiple, so the image processing device can determine the first candidate pixel value of the interpolated pixel The difference between the second derivative of the pixel value of each adjacent original pixel and the sum of all the differences is determined as the first difference.
  • the second-order derivative of the pixel value of each pixel in the interpolation pixel and the target original pixel can be calculated based on the pixel value of the pixel and the pixel values of other pixels adjacent to the pixel.
  • Step 104 Determine a second difference between the second derivative of the second candidate pixel value and the second derivative of the pixel value of the target original pixel.
  • the image processing apparatus may determine the second derivative of the second candidate pixel value after determining the second candidate pixel value of the interpolated pixel. Moreover, for each interpolated pixel, the image processing device may determine the second derivative of the second candidate pixel value of the interpolated pixel and the second derivative of each original pixel in the plurality of original pixels determined in step 103 above. The second difference of the second derivative of the pixel value of the target original pixel among the plurality of original pixels.
  • the image processing device can determine the second candidate pixel value of the interpolated pixel and the pixel value of each adjacent original pixel.
  • the difference value of the order derivative, and the sum of all the difference values is determined as the second difference value.
  • Step 105 Use the candidate pixel value corresponding to the smaller one of the first difference and the second difference to update the initial pixel value of the difference pixel.
  • the neighborhood of the pixel may be the L ⁇ L neighborhood of the pixel, and L is an odd number greater than 1, for example, L may be 5 or 7.
  • the L ⁇ L neighborhood of the pixel may include L rows and L columns of pixels, that is, it includes a total of L 2 pixels, and the pixel may be located in the (L+1)/2th row of the pixel L ⁇ L neighborhood. L+1)/2 columns.
  • the image processing device determines the first difference between the second derivative of the first candidate pixel value and the second derivative of the pixel value of the target original pixel according to step 103, and determines the second derivative according to step 104 After the second difference between the second derivative of the two candidate pixel values and the second derivative of the pixel value of the target original pixel, the image processing device may determine the magnitude of the first difference and the second difference.
  • the candidate pixel value corresponding to the smaller difference can make the image continuity in the neighborhood of the interpolation pixel in the interpolated image better, and the candidate pixel value corresponding to the larger difference can make the interpolated pixel in the interpolated image The image continuity in the neighborhood is poor.
  • the image processing device may use the smaller difference between the first difference and the second difference.
  • the candidate pixel value updates the initial pixel value of the interpolated pixel, so that the interpolated image can be updated.
  • the second candidate pixel value corresponding to the second difference can be used to update the initial pixel value of the interpolated pixel, that is, the pixel value of the interpolated pixel can be made equal to the The second candidate pixel value. If the second difference is less than or equal to the first difference, the first candidate pixel value corresponding to the first difference can be used to update the initial pixel value of the interpolated pixel, that is, the pixel value of the interpolated pixel can be made equal to the first backup pixel. Select the pixel value.
  • the embodiments of the present application provide an image processing method, which can first perform interpolation processing on the image to be processed to obtain an interpolated image, and update the initial pixel value of each interpolated pixel in the interpolated image, and update
  • the difference between the second-order derivative of the pixel value of the interpolated pixel and the second-order derivative of the pixel value of the target original pixel in the interpolated image is small, which reduces the artifacts generated in the interpolated image, and improves the neighborhood of the interpolated pixel in the interpolated image.
  • the image continuity within the domain, the quality of the image obtained by the final processing is higher.
  • Fig. 2 is a flowchart of a method for generating an interpolated image provided by an embodiment of the present application, and the method can be applied to an image processing device.
  • the method may include:
  • Step 201 Determine the first gradient of the first area to be interpolated in the first direction in the image to be processed.
  • the image processing device may pre-store a formula for calculating the gradient in the first direction of the image. After the image processing device receives the image to be processed, it may use the pre-stored formula for the gradient in the first direction. Determine the first gradient of the first area to be interpolated in the first direction in the image to be processed. Wherein, the first area to be interpolated may be an L1 ⁇ L1 neighborhood of a certain pixel in the image to be processed. L1 is an odd number greater than 1.
  • the first direction may be a direction with an included angle of 45° (degrees) with the pixel row direction.
  • the first gradient G1 of the first area to be interpolated in the first direction can satisfy:
  • m, n, and x are the preset gradient calculation coefficients, and the value of m can be 1, 3, and -1, and the value of n can also be 1, 3, and -1.
  • the value of x can be 2.
  • the values of m, n, and x may also be other values, for example, the value of x may also be 1, which is not limited in the embodiment of the present application.
  • i can be used to represent the number of pixel rows
  • j can be used to represent the number of pixel columns
  • i and j can both be positive integers greater than 1, and i is less than the total number of pixel rows of the image, and j is less than the total number of pixel columns of the image.
  • I(i+m,jn) is the pixel value of the pixel in the i+mth row and jnth column in the image to be processed
  • I(i+mx,j-n+x) is the jth row in the i+mxth row of the image to be processed -The pixel value of the pixel in the n+x column.
  • means to take the absolute value.
  • Step 202 Determine the second gradient of the first area to be interpolated in the second direction in the image to be processed.
  • the image processing device may also pre-store a formula for calculating the gradient in the second direction of the image, and the image processing device may use its pre-stored formula for the gradient in the second direction to determine whether the image to be processed is The second gradient in the second direction.
  • the second direction may be perpendicular to the first direction.
  • the second direction may be a direction with an angle of 135° with the pixel row direction.
  • the second gradient G2 of the first area to be interpolated in the second direction can satisfy:
  • m, n and x are the preset gradient calculation coefficients
  • the value of m can be 1, 3 and -1
  • the value of n can be 1, 3 and -1
  • x The value of can be 2.
  • the values of m, n, and x may also be other values, for example, the value of x may also be 1, which is not limited in the embodiment of the present application.
  • I(i+m, j+n) is the pixel value of the pixel in the i+mth row and j+n column in the image to be processed
  • I(i+mx, j-n+x) is the pixel value in the image to be processed The pixel value of the pixel in the j-n+x column of the i+mx row.
  • Step 203 Detect whether the first gradient is greater than the second gradient.
  • the image processing apparatus determines the first gradient of the first area to be interpolated in the first direction in the image to be processed according to step 201, and determines that the first area to be interpolated in the image to be processed is in step 202 After the second gradient in the second direction, it can be determined whether the first gradient is greater than the second gradient according to the first gradient and the second gradient.
  • the image processing device can perform the following step 204; if the second gradient is greater than the first gradient, the image processing device can perform the following step 205; if the first gradient is equal to the second gradient, the first
  • the image of the area to be interpolated may be a smooth texture image, and the image processing device may perform any of the following steps 204 and 205.
  • a first determination formula for determining whether the first gradient is greater than the second gradient may be pre-stored in the image processing device.
  • the first determination formula may satisfy:
  • the image processing device determines that T1 is greater than 1 according to the above formula (3), it means that the first gradient is greater than the second gradient; if the image processing device determines that T1 is less than 1 according to the above formula (3), it means that The second gradient is greater than the first gradient; if the image processing device determines that T1 is equal to 1, according to the above formula (3), it indicates that the first gradient is equal to the second gradient.
  • the first gradient may be greater than the second gradient
  • the second gradient may be greater than the first gradient. Therefore, for the first area to be interpolated where the first gradient is greater than the second gradient, the image processing apparatus may execute the following step 204 to perform interpolation processing on the first area to be interpolated. For the first area to be interpolated with the second gradient greater than the first gradient, the image processing apparatus may execute the following step 205 to perform interpolation processing on the first area to be interpolated. For the first area to be interpolated with the first gradient equal to the second gradient, the image processing apparatus may perform any of the following steps 204 and 205 to perform interpolation processing on the first area to be interpolated.
  • Step 204 Perform interpolation processing on the first area to be interpolated along the second direction by using an interpolation algorithm to obtain an initial interpolated image.
  • the image processing device determines a certain first area to be interpolated according to the above step 203 (for example, the first area to be interpolated is the 7 ⁇ 7 neighborhood of the pixel in the i-th row and the j-th column) corresponding to The first gradient is greater than the second gradient, indicating that the image of the first area to be interpolated may be a strong edge texture image in the second direction.
  • the image processing device can use an interpolation algorithm to perform the first area to be interpolated along the second direction.
  • the interpolation processing obtains the pixel value of the interpolated pixel in the first area to be interpolated, that is, the initial interpolated image is obtained.
  • the interpolation algorithm pre-stored in the image processing device may be any one of: nearest neighbor interpolation algorithm, bilinear interpolation algorithm, and bicubic convolution interpolation algorithm.
  • FIG. 3 is a schematic diagram of a first area to be interpolated according to an embodiment of the present application.
  • the pixel F1 represented by the black circle is the original pixel
  • the pixel F2 represented by the white square is the original pixel.
  • the pixels F3 represented by the circles are all interpolated pixels.
  • the image processing device uses an interpolation algorithm to perform interpolation processing on the first area to be interpolated in the second direction, the pixel value of the pixel represented by the white square can be obtained.
  • the pixel value of the pixel represented by the white square may be a known pixel value
  • the pixel value of the pixel represented by the white circle Can be an unknown pixel value.
  • the line between the pixel represented by the white square and the original pixel adjacent to the pixel represented by the white square in the original pixel is the first direction A or the second direction B. For example, it is 45° or 135°.
  • the image processing device may use an interpolation algorithm and the pixel values of the original pixels to perform interpolation processing on the first area to be interpolated in the second direction to obtain an initial interpolated image.
  • the initial pixel value of each interpolated pixel may be the image processing device according to the L1 of the interpolated pixel.
  • the line connecting the interpolation pixel is parallel to the pixel value of the original pixel in the second direction, and the weight of the pixel value of each original pixel is determined.
  • L1 can be equal to 7, or L1 can be equal to 5.
  • the embodiment of the present application takes L1 equal to 7 as an example for description.
  • L1 equal to 7
  • the embodiment of the present application takes L1 equal to 7 as an example for description.
  • the four original pixels are respectively: the i-3th row and the j-3th column Pixels, pixels in the i-1th column and j-1th column, pixels in the i+1th column and j+1th column, and pixels in the i+3th row and j+3th column.
  • the pixel value of the interpolated pixel in the i-th row and j-th column can be based on the pixel value of the pixel in the i-3th row and j-3th column, the pixel value of the pixel in the i-1th row and j-1th column, and the i+th row Determined by the pixel value of the pixel in the j+1th column of the 1 row and the pixel value of the pixel in the j+3th row of the i+3th row.
  • the distance between the pixel in the i-3th row and the j-3th column and the interpolated pixel in the i+3th row and j+3th column and the i-th row and jth column is relative to the i-1th row and j-th- 1 column of pixels, and the distance between the pixel in the i+1th row and the j+1th column and the i-th row and j-th column interpolated pixel is large, so the pixel value of the i-3th row and j-3th column pixel, and the The weight of the pixel value of the pixel in row i+3 and column j+3 is smaller. The weight is larger.
  • the weight of the pixel value of the pixel in the i-3th row and the j-3th column and the pixel value of the pixel in the i+3th row and j+3th column may both be 1/6, and the weight of the i-1th row
  • the weight of the pixel value of the pixel in the j-1th column and the pixel value of the pixel in the i+1th row and the j+1th column may both be 1/3.
  • the pixel value of the pixel in the i-3th row and the j-3th column is I1
  • the pixel value of the pixel in the i+3th column and j+3th column is I2
  • the pixel in the i-1th and j-1th column The value is I3
  • the pixel value of the pixel in the i+1th column and the j+1th column is I4
  • Step 205 Perform interpolation processing on the first area to be interpolated along the first direction by using an interpolation algorithm to obtain an initial interpolated image.
  • the image processing device determines a certain first area to be interpolated (for example, the first area to be interpolated is the 7 ⁇ 7 neighborhood of the pixel in the i-th row and the j-th column) according to the above step 203
  • the second gradient is greater than the first gradient, indicating that the image of the first area to be interpolated may be a strong edge texture image in the first direction.
  • the image processing device can use an interpolation algorithm to interpolate the first area to be interpolated in the first direction. Through processing, the pixel value of the interpolated pixel in the first area to be interpolated is obtained, that is, the initial interpolated image is obtained.
  • step 205 For the specific process of this step 205, reference may be made to the above step 204, which will not be repeated in this embodiment of the present application.
  • Step 206 Determine the third gradient of the second area to be interpolated in the third direction in the initial interpolated image.
  • the image processing device may pre-store a formula for calculating the gradient of the third direction of the image, and the image processing device may use the pre-stored formula of the third direction gradient to determine the first interpolation image in the initial interpolation image. Second, the third gradient of the area to be interpolated in the third direction.
  • the second area to be interpolated may be a certain L2 ⁇ L2 neighborhood in the initial interpolated image.
  • L2 is an odd number greater than 1.
  • the third direction C may have an angle of 45 degrees with the first direction A.
  • the third direction C may be a direction with an angle of 0° with respect to the pixel row direction.
  • r, s and k are the preset gradient calculation coefficients
  • the value of r can be 0 and 2
  • the value of s can be 1 and -1
  • the value of k can be -2, 0 and 2.
  • the values of r, s, and k may also be other values, which are not limited in the embodiment of the present application.
  • I(ir, j+s) is the pixel value of the pixel in the irth row and the j+s column in the image to be processed
  • I(i-r+2, j+s) is the i-r+th pixel in the image to be processed
  • the pixel value of the pixel in the j+s column of the 2nd row, I(i-1, j+k) is the pixel value of the pixel in the i-1th row and the j+k column of the image to be processed, I(i+1, j+ k) is the pixel value of the pixel in the i+1th row and the j+kth column in the image to be processed.
  • Step 207 Determine the fourth gradient of the second area to be interpolated in the fourth direction in the initial interpolated image.
  • the image processing device may pre-store a formula for calculating the gradient in the fourth direction of the image, and the image processing device may use its pre-stored formula for the gradient in the fourth direction to determine that the image to be processed is in the first The fourth gradient in four directions.
  • the fourth direction D may be perpendicular to the third direction C, for example, the fourth direction D may be a direction with an angle of 90° with the pixel row direction.
  • the fourth gradient G4 of the second area to be interpolated in the fourth direction D can be satisfy:
  • r, s, and k are the preset gradient calculation coefficients
  • the value of r can be 1 and -1
  • the value of s can be 0 and 2
  • the value of k can be -2, 0 and 2.
  • the values of r, s, and k may also be other values, which are not limited in the embodiment of the present application.
  • I(ir, js) is the pixel value of the pixel in the irth row and js column in the image to be processed
  • I(i+r, j-s+2) is the j-sth row in the i+rth row of the image to be processed +2 column pixel value
  • I(i+k, j-1) is the pixel value of the i+kth row and j-1th column of the image to be processed
  • I(i+k,j+1) is the Process the pixel value of the pixel in the i+kth row and j+1th column of the image.
  • Step 208 Detect whether the third gradient is greater than the fourth gradient.
  • the image processing device determines the third gradient of the second area to be interpolated in the initial interpolation image in the third direction according to step 206, and determines according to step 207 that the second area to be interpolated in the initial interpolation image is After the fourth gradient in the fourth direction, it may be determined whether the third gradient is greater than the fourth gradient according to the third gradient and the fourth gradient.
  • the image processing device can perform the following step 209; if the fourth gradient is greater than the third gradient, the image processing device can perform the following step 210; if the third gradient is equal to the fourth gradient, the second
  • the image of the area to be interpolated may be a smooth texture image, and the image processing device may perform any of the following steps 209 and 210.
  • a second determination formula for determining whether the third gradient is greater than the fourth gradient may be pre-stored in the image processing device.
  • the second determination formula may satisfy:
  • the image processing device determines that T2 is greater than 1 according to the above formula (6), it means that the third gradient is greater than the fourth gradient; if the image processing device determines that T2 is less than 1 according to the above formula (6), it means that the first The fourth gradient is greater than the third gradient; if the image processing device determines that T2 is equal to 1, according to the above formula (6), it indicates that the third gradient is equal to the fourth gradient.
  • the third gradient may be greater than the fourth gradient
  • the fourth gradient may be greater than the third gradient. Therefore, for the second area to be interpolated with the third gradient greater than the fourth gradient, the image processing device may perform the following step 204 to perform interpolation processing on the second area to be interpolated. For the second area to be interpolated with the fourth gradient greater than the third gradient, The image processing apparatus may execute the following step 205 to perform interpolation processing on the second area to be interpolated. For the second area to be interpolated with the third gradient equal to the fourth gradient, the image processing device may perform any of the following steps 209 and 210 to perform interpolation processing on the second area to be interpolated.
  • Step 209 Perform interpolation processing on the second area to be interpolated along the fourth direction by using an interpolation algorithm to obtain an interpolated image.
  • the image processing device determines according to the above step 208 that the third gradient of a certain second area to be interpolated (for example, the 5 ⁇ 5 neighborhood of the pixel in the i-th row and the j-th column) is greater than the fourth gradient, It is explained that the image of the second area to be interpolated may be a strong edge texture image in the fourth direction. In this case, the image processing device may use an interpolation algorithm to perform interpolation processing on the second area to be interpolated in the fourth direction to obtain an interpolated image.
  • the third gradient of a certain second area to be interpolated for example, the 5 ⁇ 5 neighborhood of the pixel in the i-th row and the j-th column
  • the pixel value of the original pixel (the pixel represented by the black circle) and the pixel value of the pixel represented by the white square in the initial interpolated image are known pixel values, and the image processing device can use an interpolation algorithm ,
  • the pixel value of the original pixel and the pixel value of the pixel represented by the white square perform interpolation processing on the second area to be interpolated to obtain an interpolated image.
  • the pixel value of the pixel represented by the white circle can be obtained. That is, in the interpolation image obtained by performing interpolation processing on the second area to be interpolated by using an interpolation algorithm, the pixel values of all pixels are known pixel values.
  • the line between the pixel represented by the white circle and the original pixel adjacent to the pixel represented by the white circle is the third direction or the fourth direction, for example, 0° or 90°.
  • the initial pixel value of each interpolated pixel (the pixel represented by the white circle) can be determined by the image processing device according to the L2 ⁇ L2 neighbor of the interpolated pixel.
  • the line with the interpolation pixel is parallel to the pixel value of all pixels with known pixel values in the fourth direction (including the original pixel and the pixel value obtained by the first interpolation process), and each known pixel value The weight is determined.
  • L2 can be equal to 7, or L2 can be equal to 5.
  • the embodiment of the present application takes L2 equal to 7 as an example for description.
  • L2 equal to 7
  • the pixels with known pixel values are: pixels in row i-3 and column j, pixels in row i-1 and column j, pixels in row i+1 and column j, and pixels in row i+3 and column j.
  • the interpolated pixel in the i-th row and j-th column can be based on the pixel value of the pixel in the i-3 row and the j-th column, the pixel value of the pixel in the i-1th row and the j-th column, and the pixel value of the i+1-th row and j-th column pixel.
  • the pixel value is determined by the pixel value of the pixel in the i+3 row and the j column.
  • the distance between the pixel in the i-3th row and the jth column and the interpolated pixel in the i+3th row and the jth column and the ith row and jth column is relative to the i-1th row and jth column pixel, and the The distance between the pixel in row i+1 and the pixel in the jth row and the jth column is large, so the pixel value of the pixel in the jth column of the i-3 row and the pixel in the jth column of the i+3 row
  • the weight of the value is smaller, and the weight of the pixel value of the pixel in the i-1th column and the pixel value of the pixel in the i+1th column is larger.
  • the weight of the pixel value of the pixel in the i-3th row and the jth column and the pixel value of the pixel in the i+3th row and the jth column may both be 1/6, and the weight of the pixel in the i-1th row and jth column is 1/6.
  • the weight of the pixel value of the pixel value and the pixel value of the pixel value of the i+1th row and the jth column may both be 1/3.
  • the pixel value of the pixel in the i-3th row and the jth column is I5
  • the pixel value of the i+3th row and jth column pixel is I5
  • the pixel value of the i-1th row and jth column pixel is I7
  • the pixel value of the pixel in row i+1 and column j is I8
  • the image processing module has already performed an interpolation processing and obtained the initial interpolation image.
  • the number of pixels with known pixel values in the initial interpolation image is greater than the number of pixels with known pixel values in the image to be interpolated. Therefore, when the image processing device performs interpolation processing on the second area to be interpolated in the initial interpolated image, the initial pixel value of each interpolated pixel can be determined by the pixel value of the pixel in the smaller neighborhood of the interpolated pixel. For example, L2 is equal to 5.
  • the image processing device may use the pixel value of the pixel in the 5 ⁇ 5 neighborhood a of the interpolated pixel to determine.
  • the pixel in the 2i-1th row and 2j-1th column in the interpolated image 302 is the original pixel, and the pixel in the 2i-1th row and 2j-1th column in the interpolated image 302
  • the pixel value I H (2i-1, 2j-1) of is equal to the pixel value I L (i, j) of the pixel in the i-th row and j-th column in the image 301 to be processed. That is, the image to be processed 301 and the interpolation image 302 satisfy the following mapping relationship:
  • Step 210 Perform interpolation processing on the second area to be interpolated along the third direction by using an interpolation algorithm to obtain an interpolated image.
  • the image processing device determines according to the above step 210 that the second area to be interpolated (for example, the 5 ⁇ 5 neighborhood of the pixel in the i-th row and the j-th column), the fourth gradient is greater than the third gradient, indicating that the second The image of the area to be interpolated may be a strong edge texture image in the third direction.
  • the image processing device may use an interpolation algorithm to perform interpolation processing on the second area to be interpolated along the third direction to obtain an interpolated image.
  • step 210 For the specific process of this step 210, reference may be made to the above step 209, which will not be repeated in this embodiment of the present application.
  • FIG. 6 is a flowchart of a method for image processing an interpolated image provided by an embodiment of the present application. The method can be applied to an image processing device. It can be seen with reference to FIG. 6 that the method may include:
  • Step 211 Increase the initial pixel value of the interpolated pixel by the pixel step value to obtain the first candidate pixel value.
  • the image processing device may pre-store the pixel step value. After the image processing device obtains the interpolated image, for each interpolated pixel, the image processing device may increase the initial pixel value of the interpolated pixel by pixel Step value to obtain the first candidate pixel value.
  • the pixel step value may range from 1 to 2, and the first candidate pixel value is greater than the initial pixel value.
  • Step 212 Determine the first and second derivative of the first candidate pixel value according to the first candidate pixel value and the pixel value of the target original pixel.
  • the image processing device may first base on the pixel value of the target original pixel and the foregoing
  • the first candidate pixel value determined in step 211 determines the first and second derivative of the first candidate pixel value.
  • the target original pixels may include: two first target pixels and two second target pixels.
  • the line between each first target pixel and the interpolation pixel may extend in the first direction
  • the line between each second target pixel and the interpolation pixel may extend in the second direction.
  • the image processing device can determine the second-order derivative of the pixel value of each target pixel among the four target pixels and the first candidate pixel value. The difference of the order derivatives.
  • this step 212 may include:
  • Step 2121 according to the first candidate pixel value and the pixel values of the two first target pixels, determine the second derivative of the first candidate pixel value in the first direction.
  • the image processing device may determine the sum of the pixel values of the two first target pixels corresponding to the interpolation pixel and the difference between the first candidate pixel value of the interpolation pixel Is the second derivative of the first candidate pixel value of the interpolated pixel in the first direction.
  • the two first target pixels corresponding to the interpolated pixel in the i-th row and j-th column in the interpolated image are respectively the pixel in the i+1-th row and the j-1th column and the i-1th in the interpolated image.
  • the image processing device may use the pixel value of the pixel in the i+1th row and the j-1th column, the pixel value of the pixel in the i-1th column and the j+1th column, the pixel value of the interpolated pixel in the ith row and the jth column, and the pixel value
  • the step value determines the second derivative in the first direction of the first candidate pixel value of the interpolated pixel in the i-th row and j-th column of the interpolated image.
  • the second derivative I 11 + (i, j) of the first candidate pixel value of the interpolation pixel in the i-th row and j-th column in the first direction satisfies:
  • I(i+1,j-1) is the pixel value of the pixel in the i+1th row and j-1th column
  • I(i-1,j+1) is the i-1th column pixel value.
  • the pixel value of the pixel in the row j+1 column, I(i,j) is the initial pixel value of the pixel in the ith row and the jth column, and ⁇ is the pixel step value.
  • I(i,j)+ ⁇ is the first candidate pixel value.
  • Step 2122 Determine the second derivative of the first candidate pixel value in the second direction according to the first candidate pixel value and the pixel values of the two second target pixels.
  • the image processing device may determine the sum of the pixel values of the two second target pixels corresponding to the interpolated pixel and the difference between the first candidate pixel value of the interpolated pixel Is the second derivative of the first candidate pixel value in the second direction.
  • the two second target pixels corresponding to the interpolation pixel in the i-th row and j-th column in the interpolated image are the pixel in the i-1th row and the j-1th column and the i+1th column and j+ 1 column of pixels.
  • the image processing device may use the pixel value of the pixel in the i-1th row and the j-1th column, the pixel value of the pixel in the i+1th column and the j+1th column, the pixel value of the interpolated pixel in the ith row and the jth column, and the pixel value
  • the step value determines the second derivative in the second direction of the first candidate pixel value of the interpolated pixel in the i-th row and j-th column of the interpolated image.
  • the second derivative I 22 + (i, j) of the first candidate pixel value of the interpolation pixel in the i-th row and j-th column in the second direction satisfies:
  • I(i-1, j-1) is the pixel value of the pixel in the i-1th row and j-1th column
  • I(i+1, j+1) is the i+1th column.
  • Step 213 Determine the second-order derivative of the pixel value of the target original pixel according to the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel.
  • the image processing device determines the first second derivative of the first candidate pixel value of the interpolated pixel according to the above step 212, in order to be able to determine the second derivative of the first candidate pixel value and the target
  • the difference of the second-order derivative of the pixel value of the original pixel also needs to determine the second-order derivative of the pixel value of the target original pixel based on the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel.
  • the image processing device can determine the second-order derivative of the pixel value of each target pixel in the four target pixels.
  • the image processing device needs to separately determine the second-order derivative of each target pixel in the first direction and the second direction, that is, the second-order derivative of each target pixel in the four target pixels in the first direction and the second direction can be determined separately. , That is, eight second-order derivatives need to be determined.
  • this step 213 may include:
  • Step 2131 For each target pixel of the two first target pixels and the two second target pixels, according to the pixel value of the target pixel and the connection line between the target pixel and the pixel adjacent to the target pixel Is the pixel value of the pixel in the first direction, and the second derivative of the pixel value of the target pixel in the first direction is determined.
  • the image processing device may set the connection with the target pixel in the first direction
  • the sum of the pixel values of the two pixels and the difference between the pixel value of the target pixel is determined as the second-order derivative of the pixel value of the target pixel in the first direction.
  • the second-order derivative I 11 (i+1, j-1) of the pixel value in the first direction of the pixel value of the first target pixel in the i+1-th row and j-1th column corresponding to the interpolated pixel in the i-th row and j-th column ) satisfy:
  • I 11 (i+1,j-1) I(i+2,j-2)+[I(i,j)+ ⁇ ]-2 ⁇ I(i+1,j-1), where I (i+2, j-2) is the pixel value of the pixel in the i+2th row and j-2th column.
  • the second-order derivative I 11 (i-1, j+1) in the first direction of the pixel value of the first target pixel in the i-1th row and j+1th column corresponding to the interpolated pixel in the i-th row and jth column satisfies :
  • I 11 (i-1,j+1) [I(i,j)+ ⁇ ]+I(i-2,j+2)-2 ⁇ I(i-1,j+1), where I (i-2, j+2) is the pixel value of the pixel in the i-2th row and j+2th column.
  • the second-order derivative I 11 (i-1, j-1) in the first direction of the pixel value of the second target pixel in the i-1th row and j-1th column corresponding to the interpolated pixel in the i-th row and j-th column satisfies :
  • I 11 (i-1,j-1) I(i,j-2)+I(i-2,j)-2 ⁇ I(i-1,j-1), where I(i,j -2) is the pixel value of the pixel in the i-th row and j-2th column, and I(i-2, j) is the pixel value of the pixel in the i-2th row and j-th column.
  • the second-order derivative I 11 (i+1, j+1) in the first direction of the pixel value of the second target pixel in the i+1-th row and j+1-th column corresponding to the interpolated pixel in the i-th row and j-th column satisfies :
  • I 11 (i+1,j+1) I(i+2,j)+I(i,j+2)-2 ⁇ I(i+1,j+1), where I(i+2 , J) is the pixel value of the pixel in the i+2th row and the jth column, and I(i, j+2) is the pixel value of the j+2th column in the i-th row.
  • Step 2132 For each of the two first target pixels and the two second target pixels, according to the pixel value of the target pixel and the connection line between the target pixel and the pixel adjacent to the target pixel Is the pixel value of the pixel in the second direction, and the second derivative of the pixel value of the target pixel in the second direction is determined.
  • the image processing device sets the connection with the target pixel in the second direction
  • the difference between the sum of the pixel values of the two pixels and the pixel value of the target pixel is determined as the second-order derivative of the pixel value of the target pixel in the second direction.
  • I 22 (i+1,j-1) I(i,j-2)+I(i+2,j)-2 ⁇ I(i+1,j-1), where I(i,j -2) is the pixel value of the pixel in the i-th row and j-2th column, and I(i+2, j) is the pixel value of the pixel in the i+2th row and j-th column.
  • the second-order derivative I 22 (i-1, j+1) in the second direction of the pixel value of the first target pixel in the i-1th row and j+1th column corresponding to the interpolated pixel in the i-th row and jth column satisfies :
  • I 22 (i-1,j+1) I(i-2,j)+I(i,j+2)-2 ⁇ I(i-1,j+1), where I(i-2 , J) is the pixel value of the pixel in the i-2th row and the jth column, and I(i, j+2) is the pixel value of the j+2th column in the i-th row.
  • the second-order derivative I 22 (i-1, j-1) of the pixel value of the second target pixel in the second direction of the pixel value of the second target pixel in the i-1th row and j-1th column corresponding to the interpolated pixel in the i-th row and j-th column satisfy:
  • I 22 (i-1,j-1) I(i-2,j-2)+[I(i,j)+ ⁇ ]-2 ⁇ I(i-1,j-1), where I (i-2, j-2) is the pixel value of the pixel in the i-2th row and j-2th column.
  • I 22 (i+1,j+1) [I(i,j)+ ⁇ ]+I(i+2,j+2)-2 ⁇ I(i+1,j+1), where I (i+2, j+2) is the pixel value of the pixel in the i+2th row and j+2th column.
  • Step 214 Determine a first difference between the first second-order derivative and the second-order derivative of the pixel value of the target original pixel.
  • the image processing device may use the The first second-order derivative and the second-order derivative of the pixel value of the target original pixel determine the first difference.
  • this step 214 may include:
  • Step 2141 for each of the two first target pixels and the two second target pixels, determine the second-order derivative of the first candidate pixel value in the first direction, and the pixel value of the target pixel is in the first direction.
  • the difference of the second derivative in one direction is the difference of the four first directions.
  • the image processing device determines the second derivative of the first candidate pixel value in the first direction according to step 2121, and determines two first target pixels and two second targets according to step 2131. After the second-order derivative of the pixel value of each target pixel in the pixel in the first direction, the second-order derivative of the first candidate pixel value in the first direction can be determined, and the pixel value of each target pixel is in the first direction. The difference of the second derivative in the directions, the difference of the four first directions is obtained.
  • Step 2142 for each of the two first target pixels and the two second target pixels, determine the second-order derivative of the first candidate pixel value in the second direction, and the pixel value of the target pixel is in the second direction. Interpolation of the second-order derivatives in the two directions obtains the four second-direction differences.
  • the image processing device determines the second derivative of the first candidate pixel value in the second direction according to step 2122, and determines two first target pixels and two second targets according to step 2132. After the second-order derivative of the pixel value of each target pixel in the pixel in the second direction, the second-order derivative of the first candidate pixel value in the second direction can be determined, and the pixel value of each target pixel is in the second direction. The difference of the second-order derivative in the direction, the difference of the four second directions is obtained.
  • Step 2143 Sum the difference values in the four first directions and the difference values in the four second directions to obtain the first difference value.
  • the image processing device may determine the difference of the four first directions.
  • the difference value and the difference value of the four second directions are summed to obtain the first difference value.
  • U + (i, j) U 11 +U 12 +U 13 +U 14 +U 15 +U 16 +U 17 +U 18 . That is, the first difference U + (i, j) satisfies:
  • Step 215 Reduce the initial pixel value of the interpolated pixel by the pixel step value to obtain a second candidate pixel value.
  • the image processing device may also reduce the initial pixel value of the interpolated pixel by the pixel step value to obtain a second candidate pixel value.
  • the second candidate pixel value is smaller than the initial pixel value.
  • Step 216 Determine a second second derivative of the second candidate pixel value according to the second candidate pixel value and the pixel value of the target original pixel.
  • the image processing device may determine the second difference between the second derivative of the target original pixel and the pixel value of the target original pixel and the above steps. 215 determines the second candidate pixel value, and determines the second second derivative of the second candidate pixel value.
  • the image processing device can determine the second-order derivative of the pixel value of each target pixel among the four target pixels and the second candidate pixel value. The difference of the second derivative.
  • this step 216 may include:
  • Step 2161 according to the second candidate pixel value and the pixel values of the two first target pixels, determine the second derivative of the second candidate pixel value in the first direction.
  • the image processing device may determine the sum of the pixel values of the two first target pixels corresponding to the interpolated pixel and the difference between the second candidate pixel value of the interpolated pixel Is the second derivative of the second candidate pixel value in the first direction.
  • the image processing device may use the pixel value of the pixel in the i+1th row and the j-1th column, the pixel value of the pixel in the i-1th column and the j+1th column, and the interpolation in the ith row and jth column.
  • the pixel value of the pixel and the pixel step value determine the second derivative in the first direction of the second candidate pixel value of the interpolated pixel in the i-th row and j-th column of the interpolated pixel.
  • I 11 - (i, j) I (i + 1, j-1) + I (i-1, j + 1) -2 ⁇ [I (i, j) - ⁇ ] Equation (11)
  • I(i+1,j-1) is the pixel value of the pixel in the i+1th row and j-1th column
  • I(i-1,j+1) is the i-1th column pixel value.
  • the pixel value of the pixel in the row j+1 column, I(i,j) is the initial pixel value of the pixel in the ith row and the jth column, and ⁇ is the pixel step value.
  • I(i,j)- ⁇ is the second candidate pixel value.
  • Step 2162 according to the second candidate pixel value and the pixel values of the two second target pixels, determine the second derivative of the second candidate pixel value in the second direction.
  • the image processing device may determine the sum of the pixel values of the two second target pixels and the difference between the second candidate pixel value of the interpolated pixel as the second backup pixel. Select the second derivative of the pixel value in the second direction.
  • the image processing device may use the pixel value of the pixel in the i-1th row and the j-1th column, the pixel value of the pixel in the i+1th column and the j+1th column, and the pixel value of the interpolated pixel in the i-th row and jth column. , And the pixel step value to determine the second derivative in the second direction of the second candidate pixel value of the interpolated pixel in the i-th row and j-th column of the interpolated image.
  • the second alternate value of the pixel i-th row j-th column of the interpolation pixel in the second direction of the second derivative I 22 - (i, j) is satisfied:
  • I 22 - (i, j) I (i-1, j-1) + I (i + 1, j + 1) -2 ⁇ [I (i, j) - ⁇ ] Equation (12)
  • I(i-1, j-1) is the pixel value of the pixel in the i-1th row and j-1th column
  • I(i+1, j+1) is the i+1th column.
  • Step 217 Determine the second difference between the second second derivative and the second derivative of the pixel value of the target original pixel.
  • the image processing device determines the second second derivative of the second candidate pixel value of the interpolated pixel according to the above step 216, and determines the second derivative of the pixel value of the target original pixel according to the above step 213. After that, the second difference value can be determined according to the second derivative of the second second derivative and the pixel value of the target original pixel.
  • this step 217 may include:
  • Step 2171 for each target pixel of the two first target pixels and the two second target pixels, determine the second derivative of the second candidate pixel value in the first direction, and the pixel value of the target pixel is in the first direction.
  • the difference of the second derivative in the directions, the difference of the four first directions is obtained.
  • the image processing device determines the second derivative of the second candidate pixel value in the first direction according to step 2161, and determines two first target pixels and two second targets according to step 2131. After the second-order derivative of the pixel value of each target pixel in the pixel in the first direction, the second-order derivative of the second candidate pixel value in the first direction can be determined, and the pixel value of each target pixel is in the first direction. The difference of the second derivative in the directions, the difference of the four first directions is obtained.
  • the second-order derivative of the second candidate pixel value of the interpolated pixel in the i-th row and the j-th column in the first direction and the pixel value of the first target pixel in the i+1-th row and j-1 column are in the first direction.
  • U 21
  • the second derivative of the second candidate pixel value of the interpolated pixel in the i-th row and the j-th column in the first direction is the second derivative of the pixel value of the first target pixel in the i-1th row and the j+1th column in the first direction.
  • the second derivative of the second candidate pixel value of the interpolated pixel in the i-th row and the j-th column in the first direction is the second derivative of the pixel value of the second target pixel in the i-1th row and j-1th column in the first direction.
  • Step 2172 for each target pixel of the two first target pixels and the two second target pixels, determine the second derivative of the second candidate pixel value in the second direction, and the pixel value of the target pixel is in the second direction.
  • the difference of the second-order derivative in the direction, the difference of the four second directions is obtained.
  • the image processing device determines the second derivative of the second candidate pixel value in the second direction according to step 2162, and determines two first target pixels and two second targets according to step 2132. After the second-order derivative of the pixel value of each target pixel in the pixel in the second direction, the second-order derivative of the second candidate pixel value in the second direction can be determined, and the pixel value of each target pixel is in the second direction. The difference of the second-order derivative in the direction, the difference of the four second directions is obtained.
  • the second-order derivative in the second direction of the second candidate pixel value of the interpolated pixel in the i-th row and the j-th column, and the i+1-th row and j-1th column corresponding to the interpolated pixel in the i-th row and j-th column the difference between the pixel value of the first target pixel in the second direction satisfies second derivative U 25:
  • U 25
  • Step 2173 Sum the difference values in the four first directions and the difference values in the four second directions to obtain a second difference value.
  • the image processing device may determine the difference values of the four first directions.
  • the difference value and the difference value of the four second directions are summed to obtain the second difference value.
  • U - (i, j) U 21 + U 22 + U 23 + U 24 + U 25 + U 26 + U 27 + U 28. That is, the second difference U - (i, j) is satisfied:
  • Step 218 Use the candidate pixel value corresponding to the smaller one of the first difference and the second difference to update the initial pixel value of the difference pixel.
  • the difference between the second derivative of the pixel value of a certain pixel in the image and the second derivative of the pixel values of other pixels adjacent to the pixel is smaller.
  • the image processing device determines the first difference U + (i, j) between the second derivative of the first candidate pixel value and the target original pixel according to step 214, and determines the second candidate pixel value according to step 217 the second derivative of the original target pixels of the second difference U - (i, j) then, the image processing means may determine the size of the first difference and the second difference.
  • the candidate pixel value corresponding to a smaller difference value can make the continuity in the neighborhood of the interpolation pixel in the interpolation image better, and the candidate pixel value corresponding to a larger difference value can make the interpolation pixel value in the interpolation image The image continuity in the neighborhood is poor.
  • the image processing device may use the device corresponding to the smaller of the first difference and the second difference.
  • the selected pixel value updates the initial pixel value of the interpolated pixel.
  • the second candidate pixel value corresponding to the second difference can be used to update the initial pixel value of the interpolated pixel, that is, the pixel value of the interpolated pixel can be made equal to the The second candidate pixel value. If the second difference is less than or equal to the first difference, the first candidate pixel value corresponding to the first difference can be used to update the initial pixel value of the interpolated pixel, that is, the pixel value of the interpolated pixel can be made equal to the first backup pixel. Select the pixel value.
  • the image processing apparatus may execute the above steps 211 to 218 cyclically, that is, the pixel value of each interpolated pixel in the interpolated image can be continuously updated, thereby improving the quality of the interpolated image.
  • the image processing device may stop when the cycle times of the above-mentioned step 211 to step 218 are cyclically executed is greater than or equal to the number threshold, and the image when the cycle is stopped is determined as the final processed image.
  • the threshold of the number of times may range from 20 times to 50 times.
  • the image processing device uses the first candidate pixel value and the target candidate pixel value in the second candidate pixel to update the initial pixel value of the interpolated pixel when performing step 218 for the first time, then
  • the target candidate pixel value can be used to replace the initial pixel value of the interpolation pixel, and the target candidate pixel value can be adjusted to obtain a new first candidate pixel value and a new second candidate pixel value.
  • the new first candidate pixel value is greater than the target candidate pixel value
  • the new second candidate pixel value is less than the target candidate pixel value.
  • the foregoing step 211 may be: increasing the target candidate pixel value of the interpolated pixel by the pixel step value to obtain the first candidate pixel value.
  • the foregoing step 215 may be: reducing the target candidate pixel value of the interpolated pixel by the pixel step value to obtain the second candidate pixel value.
  • FIG. 12 is a schematic diagram of an image processing effect provided by an embodiment of the present application.
  • 401 is a to-be-processed image
  • 402 is a partial image obtained by processing the to-be-processed image 401 using the image processing method provided in this embodiment of the application
  • 403 is a partial image obtained by processing the to-be-processed image 401 only using an interpolation algorithm.
  • FIG. 12 it can be seen that only the interpolation algorithm is used to process the image processed by the image 401 to be processed, and there are problems of artifacts and texture distortion.
  • the image 402 obtained by using the image processing method provided by the embodiment of the present application has clearer texture and better image quality.
  • the image processing device can obtain the interpolated pixels through the interpolation algorithm without adjusting the pixel values. Moreover, since the interpolation pixel is located at the edge of the image, it will not have a large impact on the quality of the image.
  • step 202 can be performed before step 201
  • step 207 can be performed before step 206
  • step 212 and steps 215 to 217 can be performed before step 211.
  • the embodiments of the present application provide an image processing method, which can first perform interpolation processing on the image to be processed to obtain an interpolated image, and update the initial pixel value of each interpolated pixel in the interpolated image, and update
  • the difference between the second-order derivative of the pixel value of the interpolated pixel and the second-order derivative of the pixel value of the target original pixel in the interpolated image is small, which reduces the artifacts generated in the interpolated image, and improves the neighborhood of the interpolated pixel in the interpolated image.
  • the image continuity within the domain, the quality of the image obtained by the final processing is higher.
  • FIG. 13 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. With reference to Figure 13, it can be seen that the device may include:
  • the interpolation processing module 501 is configured to perform interpolation processing on the image to be processed to obtain an interpolated image.
  • the interpolation image may include multiple original pixels and multiple interpolation pixels, and the initial pixel value of each interpolation pixel is determined according to the pixel value of at least one original pixel.
  • the adjustment module 502 is configured to adjust the initial pixel value of the interpolated pixel for each interpolation pixel to obtain the first candidate pixel value and the second candidate pixel value.
  • the first candidate pixel value is greater than the initial pixel value
  • the second candidate pixel value is less than the initial pixel value
  • the first determining module 503 is configured to determine the first difference between the second derivative of the first candidate pixel value and the second derivative of the pixel value of the target original pixel among the multiple original pixels.
  • the target original pixel is an original pixel adjacent to the interpolation pixel among the multiple original pixels.
  • the second determining module 504 is configured to determine the second difference between the second derivative of the second candidate pixel value and the second derivative of the pixel value of the target original pixel.
  • the update module 505 is configured to use the candidate pixel value corresponding to the smaller difference between the first difference and the second difference to update the initial pixel value of the interpolated pixel.
  • the embodiment of the present application provides an image processing device, which can first perform interpolation processing on an image to be processed to obtain an interpolated image, and update the initial pixel value of each interpolated pixel in the interpolated image, and update
  • the difference between the second-order derivative of the pixel value of the interpolated pixel and the second-order derivative of the pixel value of the target original pixel in the interpolated image is small, which reduces the artifacts generated in the interpolated image and improves the area where the interpolated pixel in the interpolated image is located
  • the continuity of the image, the quality of the final processed image is higher.
  • the adjustment module 502 can be used to:
  • the initial pixel value of the interpolated pixel is increased by the pixel step value to obtain the first candidate pixel value, and the initial pixel value of the interpolated pixel is decreased by the pixel step value to obtain the second candidate pixel value.
  • the first determining module 503 may include:
  • the first determining sub-module 5031 is configured to determine the first and second derivative of the first candidate pixel value according to the first candidate pixel value and the pixel value of the target original pixel.
  • the second determining sub-module 5032 is configured to determine the second derivative of the pixel value of the target original pixel according to the pixel value of the target original pixel and the pixel value of the pixel adjacent to the target original pixel.
  • the third determining sub-module 5033 is used to determine the first difference between the first second derivative and the second derivative of the pixel value of the target original pixel.
  • the second determining module 504 may include:
  • the fourth determining sub-module 5041 is configured to determine the second second derivative of the second candidate pixel value according to the second candidate pixel value and the pixel value of the target original pixel.
  • the fifth determining sub-module 5042 is configured to determine the second difference between the second second derivative and the second derivative of the pixel value of the target original pixel.
  • the target original pixels may include: two first target pixels and two second target pixels, wherein the connection line between each first target pixel and the interpolation pixel extends along the first direction, and each second target pixel The connecting line between the target pixel and the interpolation pixel extends along the second direction, and the first direction is perpendicular to the second direction.
  • the first determining sub-module 5031 may be used for:
  • the first candidate pixel value and the pixel values of the two first target pixels determine the second derivative of the first candidate pixel value in the first direction; according to the first candidate pixel value and the two second targets The pixel value of the pixel determines the second derivative of the first candidate pixel value in the second direction.
  • the second determining sub-module 5032 can be used for:
  • the pixel value of the pixel determines the second-order derivative of the pixel value of the target pixel in the first direction; for each of the two first target pixels and the two second target pixels, according to the pixel value of the target pixel, and The connection between the pixel adjacent to the target pixel and the target pixel is the pixel value of the pixel in the second direction, and the second-order derivative of the pixel value of the target pixel in the second direction is determined.
  • the third determining sub-module 5033 can be used for:
  • the difference of the four first directions and the difference of the four second directions are summed to obtain the first difference.
  • the two first target pixels corresponding to the interpolated pixel in the i-th row and j-th column in the interpolated image are respectively the pixel in the i+1-th row and the j-1th column and the i-1th column and j+1 in the interpolated image.
  • Columns of pixels, the two second target pixels corresponding to the interpolation pixels in the i-th row and the j-th column are the pixels in the i-1th row and the j-1th column and the i+1th column and j+1-th column pixels, respectively.
  • I 11 + (i, j) I(i+1) ,J-1)+I(i-1,j+1)-2 ⁇ [I(i,j)+ ⁇ ].
  • I(i+1,j-1) is the pixel value of the pixel in the i+1th row and j-1th column
  • I(i-1,j+1) is the pixel in the i-1th row and j+1th column
  • the pixel value of I(i,j) is the initial pixel value of the pixel in the i-th row and the j-th column
  • ⁇ is the pixel step value
  • both i and j are positive integers greater than 1.
  • I 22 + (i, j) I(i-1 ,J-1)+I(i+1,j+1)-2 ⁇ [I(i,j)+ ⁇ ];
  • I(i-1, j-1) is the pixel value of the pixel in row i-1 and column j-1
  • I(i+1, j+1) is the pixel in row i+1 and column j+1 The pixel value.
  • the fourth determining sub-module 5041 is configured to:
  • the second candidate pixel value and the pixel values of the two first target pixels determine the second derivative of the second candidate pixel value in the first direction; according to the second candidate pixel value, and the two second targets The pixel value of the pixel determines the second derivative of the second candidate pixel value in the second direction.
  • the fifth determining sub-module 5043 is used for:
  • the difference of the second derivative, the difference of the four first directions is obtained;
  • the difference of the four first directions and the difference of the four second directions are summed to obtain the second difference.
  • I(i+1,j-1) is the pixel value of the pixel in the i+1th row and j-1th column
  • I(i-1,j+1) is the pixel in the i-1th row and j+1th column
  • the pixel value of I(i,j) is the initial pixel value of the pixel in the i-th row and the j-th column
  • ⁇ is the pixel step value
  • both i and j are positive integers greater than 1.
  • I 22 - (i, j ) I (i-1 ,J-1)+I(i+1,j+1)-2 ⁇ [I(i,j)- ⁇ ];
  • I(i-1, j-1) is the pixel value of the pixel in row i-1 and column j-1
  • I(i+1, j+1) is the pixel in row i+1 and column j+1 The pixel value.
  • the interpolation processing module 501 may include:
  • the sixth determining sub-module 5011 is configured to determine the first gradient of the first area to be interpolated in the first direction in the image to be processed.
  • the seventh determining sub-module 5012 is configured to determine the second gradient of the first area to be interpolated in the second direction perpendicular to the first direction.
  • the first judgment sub-module 5013 is used to determine whether the first gradient is greater than the second gradient.
  • the first interpolation sub-module 5014 is configured to: if the first gradient is greater than the second gradient, use an interpolation algorithm to perform interpolation processing on the first area to be interpolated in the second direction to obtain an initial interpolated image; if the second gradient is greater than the first gradient, use The interpolation algorithm performs interpolation processing on the first area to be interpolated along the first direction to obtain an initial interpolated image.
  • the eighth determining sub-module 5015 is used to determine the third gradient of the second to-be-interpolated region of the initial interpolation image in the third direction.
  • the angle between the third direction and the first direction is 45 degrees.
  • the ninth determining sub-module 5016 is configured to determine the fourth gradient of the second area to be interpolated in the fourth direction perpendicular to the third direction.
  • the second judgment sub-module 5017 is used to determine whether the third gradient is greater than the fourth gradient.
  • the second interpolation sub-module 5018 is used for if the third gradient is greater than the fourth gradient, use an interpolation algorithm to perform interpolation processing on the second area to be interpolated along the fourth direction to obtain an interpolated image; if the fourth gradient is greater than the third gradient, use interpolation The algorithm performs interpolation processing on the second area to be interpolated along the third direction to obtain an interpolated image.
  • the pixel located in row 2i-1 and column 2i-1 in the interpolated image is the original pixel, and the pixel value of the pixel in row 2i-1 and column 2i-1 in the interpolated image is the same as in the image to be processed
  • the pixel values of the pixels in the i-th row and j-th column are equal.
  • the embodiment of the present application provides an image processing device, which can first perform interpolation processing on an image to be processed to obtain an interpolated image, and update the initial pixel value of each interpolated pixel in the interpolated image, and update
  • the difference between the second-order derivative of the pixel value of the interpolated pixel and the second-order derivative of the pixel value of the target original pixel in the interpolated image is small, which reduces the artifacts generated in the interpolated image and improves the area where the interpolated pixel in the interpolated image is located
  • the continuity of the image, the quality of the final processed image is higher.
  • FIG. 17 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the present application. It can be seen with reference to FIG. 17 that the device 60 may include a processor 601, a memory 602, and a computer program stored on the memory 602 and running on the processor. The processor 601 can implement the foregoing when the computer program is executed.
  • the image processing methods provided by the method embodiments are, for example, the methods shown in FIG. 1, FIG. 2, and FIG. 6 to FIG. 11.
  • the embodiments of the present application provide a computer-readable storage medium in which operations are stored, and when the computer-readable storage medium runs on a computer, the computer is caused to execute the image processing method provided by the above method embodiment For example, the methods shown in FIG. 1, FIG. 2, and FIG. 6 to FIG. 11 can be executed.
  • the embodiment of the present application also provides a computer program product containing instructions.
  • the computer program product runs on the computer, the computer executes the image processing method provided in the above method embodiment.
  • FIG. 1 and FIG. 2 can be executed.
  • the program can be stored in a computer-readable storage medium.
  • the storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.
  • An embodiment of the present application also provides a display device, which may include a display panel and the image processing device provided in the foregoing embodiments.
  • the display panel can be used to display the image processed by the image processing device.
  • the display device may be a liquid crystal display device, electronic paper, organic light-emitting diode (organic light-emitting diode, OLED) display device, active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED) display Devices, mobile phones, tablet computers, televisions, monitors, laptops, digital photo frames or navigators, and any other products or components with display functions.
  • organic light-emitting diode organic light-emitting diode, OLED
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode, AMOLED

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、图像处理装置及显示装置，涉及图像处理技术领域。该方法可以先对待处理图像进行插值处理，得到插值图像，并对插值图像中每个插值像素的初始像素值进行更新，且更新后的插值像素的像素值的二阶导与插值图像中目标原始像素的像素值的二阶导的差值较小，减少了插值图像中产生的伪影，提高了插值图像中插值像素的邻域内的图像连续性，最终处理得到的图像的质量较高。

Description

图像处理方法、图像处理装置及显示装置
本申请要求于2020年4月26日提交的申请号为202010337763.0、发明名称为“图像处理方法、图像处理装置及显示装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，特别涉及一种图像处理方法、图像处理装置及显示装置。
背景技术
随着技术的不断发展，显示器的分辨率越来越高，但由于显示器中图像传感器的局限性，采用该图像传感器生成的图像的分辨率较低，难以满足显示器的分辨率的要求。
发明内容
本申请提供了一种图像处理方法、图像处理装置及显示装置。所述技术方案如下：
一方面，提供了一种图像处理方法，所述方法包括：
对待处理图像进行插值处理，得到插值图像，所述插值图像包括：多个原始像素以及多个插值像素，每个所述插值像素的初始像素值根据至少一个所述原始像素的像素值确定；
对于每个所述插值像素，调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，所述第一备选像素值大于所述初始像素值，所述第二备选像素值小于所述初始像素值；
确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值；
确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值；
采用所述第一差值和所述第二差值中较小的差值对应的备选像素值更新所 述插值像素的初始像素值；
其中，所述目标原始像素为所述多个原始像素中与所述插值像素相邻的原始像素。
可选的，所述调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，包括：
将所述插值像素的初始像素值增加像素步进值，得到第一备选像素值；
将所述插值像素的初始像素值减小所述像素步进值，得到第二备选像素值。
可选的，所述确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值，包括：
根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导；
根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导；
确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值；
所述确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值，包括：
根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导；
确定所述第二二阶导与所述目标原始像素的像素值的二阶导的第二差值。
可选的，所述目标原始像素包括：两个第一目标像素和两个第二目标像素，其中，每个所述第一目标像素与所述插值像素之间的连线沿第一方向延伸，每个所述第二目标像素与所述插值像素之间的连线沿第二方向延伸，所述第一方向与所述第二方向垂直；
所述根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导，包括：
根据所述第一备选像素值，以及两个所述第一目标像素的像素值，确定所述第一备选像素值在所述第一方向上的二阶导；
根据所述第一备选像素值，以及两个所述第二目标像素的像素值，确定所述第一备选像素值在所述第二方向上的二阶导；
所述根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导，包括：
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第一方向的像素的像素值，确定所述目标像素的像素值在所述第一方向上的二阶导；
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第二方向的像素的像素值，确定所述目标像素的像素值在所述第二方向上的二阶导；
所述确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值，包括：
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第一备选像素值在所述第一方向上的二阶导，与所述目标像素的像素值在所述第一方向上的二阶导的差值，得到四个第一方向的差值；
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第一备选像素值在所述第二方向上的二阶导，与所述目标像素的像素值在所述第二方向上的二阶导的差值，得到四个第二方向的差值；
对四个所述第一方向的差值和四个所述第二方向的差值进行求和，得到第一差值。
可选的，所述插值图像中第i行第j列的插值像素对应的两个所述第一目标像素分别为所述插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素，所述第i行第j列的插值像素对应的两个所述第二目标像素分别为所述第i-1行第j-1列像素和第i+1列第j+1列像素；所述第i行第j列的插值像素的第一备选像素值在所述第一方向上的二阶导I 11 +(i，j)满足：
I 11 +(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)+Δ]；
其中，所述I(i+1，j-1)为所述第i+1行第j-1列像素的像素值，所述I(i-1，j+1)为所述第i-1行第j+1列像素的像素值，所述I(i，j)为所述第i行第j列像素的初始像素值，所述Δ为所述像素步进值，i和j均为大于1的正整数；
所述第i行第j列的插值像素的第一备选像素值在所述第二方向上的二阶导I 22 +(i，j)满足：
I 22 +(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)+Δ]；
其中，所述I(i-1，j-1)为所述第i-1行第j-1列像素的像素值，所述I(i+1，j+1)为所述第i+1行第j+1列像素的像素值。
可选的，所述目标原始像素包括：两个第一目标像素和两个第二目标像素，其中，每个所述第一目标像素与所述插值像素之间的连线沿第一方向延伸，每个所述第二目标像素与所述插值像素之间的连线沿第二方向延伸，所述第一方向与所述第二方向垂直；
所述根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导，包括：
根据所述第二备选像素值，以及两个所述第一目标像素的像素值，确定所述第二备选像素值在所述第一方向上的二阶导；
根据所述第二备选像素值，以及两个所述第二目标像素的像素值，确定所述第二备选像素值在所述第二方向上的二阶导；
所述根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导，包括：
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第一方向的像素的像素值，确定所述目标像素的像素值在所述第一方向上的二阶导；
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第二方向的像素的像素值，确定所述目标像素的像素值在所述第二方向上的二阶导；
所述确定所述第二二阶导与所述目标原始像素的像素值的二阶导的第二差值，包括：
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第二备选像素值在所述第一方向上的二阶导，与所述目标像素的像素值在所述第一方向上的二阶导的差值，得到四个第一方向的差值；
对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第二备选像素值在所述第二方向上的二阶导，与所述目标像素的像素值在所述第二方向上的二阶导的差值，得到四个第二方向的差值；
对四个所述第一方向的差值和四个所述第二方向的差值进行求和，得到第 二差值。
可选的，所述插值图像中第i行第j列的插值像素对应的两个所述第一目标像素分别为所述插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素，所述第i行第j列的插值像素对应的两个所述第二目标像素分别为所述第i-1行第j-1列像素和第i+1列第j+1列像素；所述第i行第j列的插值像素的第二备选像素值在所述第一方向上的二阶导I 11 -(i，j)满足：
I 11 -(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)-Δ]；
其中，所述I(i+1，j-1)为所述第i+1行第j-1列像素的像素值，所述I(i-1，j+1)为所述第i-1行第j+1列像素的像素值，所述I(i，j)为所述第i行第j列像素的初始像素值，所述Δ为所述像素步进值，i和j均为大于1的正整数；
所述第i行第j列的插值像素的第二备选像素值在所述第二方向上的二阶导I 22 -(i，j)满足：
I 22 -(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)-Δ]；
其中，所述I(i-1，j-1)为所述第i-1行第j-1列像素的像素值，所述I(i+1，j+1)为所述第i+1行第j+1列像素的像素值。
可选的，所述对待处理图像进行插值处理，得到插值图像，包括：
确定待处理图像中第一待插值区域在第一方向的第一梯度；
确定所述第一待插值区域在与所述第一方向垂直的第二方向的第二梯度；
若所述第一梯度大于所述第二梯度，采用插值算法沿所述第二方向对所述第一待插值区域进行插值处理，得到初始插值图像；
若所述第二梯度大于所述第一梯度，采用插值算法沿所述第一方向对所述第一待插值区域进行插值处理，得到初始插值图像；
确定所述初始插值图像中第二待插值区域在第三方向的第三梯度，所述第三方向与所述第一方向的夹角为45度；
确定所述第二待插值区域在与所述第三方向垂直的第四方向的第四梯度；
若所述第三梯度大于所述第四梯度，采用所述插值算法沿所述第四方向对所述第二待插值区域进行插值处理，得到插值图像；
若所述第四梯度大于所述第三梯度，采用所述插值算法沿所述第三方向对所述第二待插值区域进行插值处理，得到插值图像。
可选的，所述插值图像中位于第2i-1行第2i-1列的像素为所述原始像素， 且所述插值图像中第2i-1行第2i-1列的像素的像素值与所述待处理图像中第i行第j列的像素的像素值相等。
另一方面，提供了一种图像处理装置，所述装置包括：
插值处理模块，用于对待处理图像进行插值处理，得到插值图像，所述插值图像包括：多个原始像素以及多个插值像素，每个所述插值像素的初始像素值根据至少一个所述原始像素的像素值确定；
调整模块，用于对于每个所述插值像素，调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，所述第一备选像素值大于所述初始像素值，所述第二备选像素值小于所述初始像素值；
第一确定模块，用于确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值；
第二确定模块，用于确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值；
更新模块，用于采用所述第一差值和所述第二差值中较小的差值对应的备选像素值更新所述插值像素的初始像素值；
其中，所述目标原始像素为所述多个原始像素中与所述插值像素相邻的原始像素。
可选的，述调整模块，用于：
将所述插值像素的初始像素值增加像素步进值，得到第一备选像素值；
将所述插值像素的初始像素值减小所述像素步进值，得到第二备选像素值。
可选的，所述第一确定模块，包括：
第一确定子模块，用于根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导；
第二确定子模块，用于根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导；
第三确定子模块，用于确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值；
所述第二确定模块，包括：
第四确定子模块，用于根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导；
第五确定子模块，用于确定所述第二二阶导与所述目标原始像素的像素值 的二阶导的第二差值。
又一方面，提供了一种图像处理装置，所述装置包括：处理器、存储器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如上述方面所述的图像处理方法。
再一方面，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当所述计算机可读存储介质在计算机上运行时，使得计算机执行如上述方面所述的图像处理方法。
再一方面，提供了一种显示装置，所述显示装置包括：显示面板以及如上述方面所述的图像处理装置；
所述显示面板用于显示所述图像处理装置处理后的图像。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种图像处理方法的流程图；
图2是本申请实施例提供的一种生成插值图像的方法流程图；
图3是本申请实施例提供的一种第一待插值区域的示意图；
图4是本申请实施例提供的一种第二待插值区域的示意图；
图5是本申请实施例提供的一种待处理图像与插值图像的关系示意图；
图6是本申请实施例提供的一种对插值图像进行图像处理的方法流程图；
图7是本申请实施例提供的一种确定第一备选像素值的第一二阶导的方法流程图；
图8是本申请实施例提供的一种确定目标原始像素的像素值的二阶导的方法流程图；
图9是本申请实施例提供的一种确定第一差值的方法流程图；
图10是本申请实施例提供的一种确定第二备选像素值的第二二阶导的方法流程图；
图11是本申请实施例提供的一种确定第二差值的方法流程图；
图12是本申请实施例提供的一种图像处理效果示意图；
图13是本申请实施例提供个一种图像处理装置的结构示意图；
图14是本申请实施例提供的一种第一确定模块的结构示意图；
图15是本申请实施例提供的一种第二确定模块的结构示意图；
图16是本申请实施例提供的一种插值处理模块的结构示意图；
图17是本申请实施例提供的另一种图像处理装置的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
相关技术中，为了提高图像的分辨率，以便能够将该图像显示在显示器上，需要采用超分辨率算法对图像进行处理。
但是，采用相关技术中的方法处理后的图像中通常会产生伪影，图像的质量较低。
图1是本申请实施例提供的一种图像处理方法的流程图。该方法可以应用于图像处理装置中。参考图1可以看出，该方法可以包括：
步骤101、对待处理图像进行插值处理，得到插值图像。
在本申请实施例中，图像处理装置中可以预先存储有插值算法，图像处理装置在接收到待处理图像之后，可以采用预先存储的插值算法对该待处理图像进行插值处理，得到插值图像。
该插值图像可以包括：多个原始像素以及多个插值像素，每个插值像素的初始像素值可以根据至少一个该原始像素的像素值确定。例如，插值图像中每个插值像素的初始像素值可以根据多个原始像素中与该插值像素相邻的原始像素的像素值确定。
可选的，图像处理装置中预先存储的插值算法可以为：最近邻插值算法，双线性插值算法，以及双立方卷积插值算法中的任一种。
步骤102、对于每个插值像素，调整插值像素的初始像素值，得到第一备选像素值和第二备选像素值。
在本申请实施例中，图像处理装置在对待处理图像进行插值处理后得到的插值图像中，所有像素的像素值均为已知像素值。图像处理装置可以对于每个插值像素，调整该插值像素的初始像素值，得到第一备选像素值和第二备选像素值。
其中，该第一备选像素值可以大于初始像素值，第二备选像素值可以小于初始像素值。也即是，对于每个插值像素，可以得到与该插值像素对应的两个备选像素值。
步骤103、确定第一备选像素值的二阶导与多个原始像素中的目标原始像素的像素值的二阶导的第一差值。
在本申请实施例中，图像处理装置可以确定该插值图像的多个原始像素中每个原始像素的二阶导。对于每个插值像素，图像处理装置可以在确定出该插值像素的第一备选像素值之后，确定该第一备选像素值的二阶导。并且，对于每个插值像素，图像处理装置可以确定该插值像素的第一备选像素值的二阶导与多个原始像素中的目标原始像素的像素值的二阶导的第一差值。
其中，该目标原始像素可以为多个原始像素中与插值像素相邻的原始像素。该第一差值为插值像素的第一备选像素值的二阶导与多个原始像素中与该插值像素相邻的原始像素的像素值的二阶导的差值。
需要说明的是，由于对于每个插值像素，与其相邻的原始像素的数量通常为多个，即目标原始像素可以为多个，因此图像处理装置可以确定该插值像素的第一备选像素值与每个相邻的原始像素的像素值的二阶导的差值，并将所有差值之和确定为第一差值。
还需要说明的是，插值像素和目标原始像素中每个像素的像素值的二阶导可以根据该像素的像素值，以及与该像素相邻的其他像素的像素值计算得到。
步骤104、确定第二备选像素值的二阶导与目标原始像素的像素值的二阶导的第二差值。
在本申请实施例中，对于每个插值像素，图像处理装置可以在确定出该插值像素的第二备选像素值之后，确定该第二备选像素值的二阶导。并且，对于每个插值像素，图像处理装置可以根据上述步骤103中确定出的多个原始像素中每个原始像素的二阶导，确定该插值像素的第二备选像素值的二阶导与多个原始像素中的目标原始像素的像素值的二阶导的第二差值。
并且，对于每个插值像素，与其相邻的原始像素的数量通常为多个，因此图像处理装置可以确定该插值像素的第二备选像素值与每个相邻的原始像素的像素值的二阶导的差值，并将所有差值之和确定为第二差值。
步骤105、采用第一差值和第二差值中较小的差值对应的备选像素值更新差值像素的初始像素值。
由于对于待处理图像而言，该待处理图像中某个像素的像素值的二阶导与该像素相邻的其他像素的像素值的二阶导的差值越小，说明该图像中该像素的邻域内的图像连续性越好，图像的质量越高。例如，像素的邻域可以为该像素的L×L邻域，L为大于1的奇数，例如L可以为5或者7。其中，该像素的L×L邻域可以包括L行L列像素，即共包括L 2个像素，且该像素可以位于该像素L×L邻域中第(L+1)/2行第(L+1)/2列。
在本申请实施例中，图像处理装置在根据步骤103确定出第一备选像素值的二阶导与目标原始像素的像素值的二阶导的第一差值，并根据步骤104确定出第二备选像素值的二阶导与目标原始像素的像素值的二阶导的第二差值之后，图像处理装置可以确定出该第一差值和第二差值的大小。其中，较小的差值对应的备选像素值可以使得插值图像中该插值像素的邻域内的图像连续性较好，较大的差值对应的备选像素值可以使得插值图像中该插值像素的邻域内的图像连续性较差。因此，为了提高在插值像素的邻域内的图像连续性，以提高插值图像的质量，对于每个插值像素，图像处理装置可以采用第一差值和第二差值中较小的差值对应的备选像素值更新该插值像素的初始像素值，由此即可实现对插值图像的更新。
也即是，若第二差值大于第一差值，可以采用该第二差值对应的第二备选像素值更新该插值像素的初始像素值，即可以使得该插值像素的像素值等于该第二备选像素值。若第二差值小于或等于第一差值，可以采用第一差值对应的第一备选像素值更新该插值像素的初始像素值，即可以使得该插值像素的像素值等于该第一备选像素值。
综上所述，本申请实施例提供了一种图像处理方法，该方法可以先对待处理图像进行插值处理，得到插值图像，并对插值图像中每个插值像素的初始像素值进行更新，且更新后的插值像素的像素值的二阶导与插值图像中目标原始像素的像素值的二阶导的差值较小，减少了插值图像中产生的伪影，提高了插值图像中插值像素的邻域内的图像连续性，最终处理得到的图像的质量较高。
图2是本申请实施例提供的一种生成插值图像的方法流程图，该方法可以应用于图像处理装置中。参考图2，该方法可以包括：
步骤201、确定待处理图像中第一待插值区域在第一方向的第一梯度。
在本申请实施例中，图像处理装置中可以预先存储有计算图像的第一方向 的梯度的公式，在图像处理装置接收到待处理图像之后，可以采用其预先存储的第一方向的梯度的公式确定该待处理图像中第一待插值区域在第一方向的第一梯度。其中，该第一待插值区域可以为待处理图像中某个像素的L1×L1邻域。L1为大于1的奇数。
可选的，该第一方向可以为与像素行方向的夹角为45°(度)的方向。假设待处理图像中第一待插值区域为第i行第j列像素的7×7邻域(即L1＝7)，则该第一待插值区域在第一方向的第一梯度G1可以满足：
G1＝∑ m＝3,±1∑ n＝3,±1|I(i+m，j-n)-I(i+m-x，j-n+x)|      公式(1)
其中，上述公式(1)中，m，n以及x分别为预设的梯度计算系数，m的取值可以为1，3以及-1，n的取值也可以为1，3，以及-1，x的取值可以为2。当然，m，n以及x的取值还可以为其他值，例如x的取值还可以为1，本申请实施例对此不作限定。并且，i可以用于表示像素行数，j可以用于表示像素列数，i和j可以均为大于1的正整数，且i小于图像的像素总行数，j小于图像的像素总列数。I(i+m，j-n)为待处理图像中第i+m行第j-n列像素的像素值，I(i+m-x，j-n+x)为待处理图像中第i+m-x行第j-n+x列像素的像素值。||表示取绝对值。
步骤202、确定待处理图像中第一待插值区域在第二方向的第二梯度。
在本申请实施例中，图像处理装置中还可以预先存储有计算图像的第二方向的梯度的公式，该图像处理装置可以采用其预先存储的第二方向的梯度的公式确定该待处理图像在第二方向的第二梯度。
可选的，该第二方向可以与第一方向垂直，例如，该第一方向为45°，则该第二方向可以为与像素行方向的夹角为135°的方向。假设该第一待插值区域为第i行第j列像素的7×7邻域(即L1＝7)，则该第一待插值区域在第二方向的第二梯度G2可以满足：
G2＝∑ m＝3，±1∑ n＝3，±1|I(i+m，j+n)-I(i+m-x，j-n+x)|      公式(2)
其中，上述公式(2)中，m，n以及x分别为预设的梯度计算系数，m的取值可以为1，3以及-1，n的取值可以为1，3以及-1，x的取值可以为2。当然，m，n以及x的取值还可以为其他值，例如x的取值还可以为1，本申请实施例对此不作限定。并且，I(i+m，j+n)为待处理图像中第i+m行第j+n列像素的像素值，I(i+m-x，j-n+x)为待处理图像中第i+m-x行第j-n+x列像素的像素值。
步骤203、检测第一梯度是否大于第二梯度。
在本申请实施例中，图像处理装置在根据步骤201确定出待处理图像中第一待插值区域在第一方向的第一梯度，以及根据步骤202确定出待处理图像的第一待插值区域在第二方向的第二梯度之后，可以根据该第一梯度和第二梯度确定该第一梯度是否大于第二梯度。
若第一梯度大于第二梯度，图像处理装置可以执行下述步骤204；若第二梯度大于第一梯度，图像处理装置可以执行下述步骤205；若第一梯度等于第二梯度，该第一待插值区域的图像可能为平滑纹理图像，图像处理装置可以执行下述步骤204和步骤205中的任一步骤。
其中，图像处理装置中可以预先存储有确定第一梯度是否大于第二梯度的第一判定公式。可选的，该第一判定公式可以满足：
T1＝(G1+1)/(G2+1)             公式(3)
在本申请实施例中，若图像处理装置根据上述公式(3)确定出T1大于1，说明第一梯度大于第二梯度；若图像处理装置根据上述公式(3)确定出T1小于1，说明第二梯度大于第一梯度；若图像处理装置根据上述公式(3)确定出T1等于1，说明第一梯度等于第二梯度。
需要说明的是，由于对于待处理图像中的一些第一待插值区域，第一梯度可能大于第二梯度，而对于另一些第一待插值区域，第二梯度可能大于第一梯度，因此，对于第一梯度大于第二梯度的第一待插值区域，图像处理装置可以执行下述步骤204以对该第一待插值区域进行插值处理。对于第二梯度大于第一梯度的第一待插值区域，图像处理装置可以执行下述步骤205以对该第一待插值区域进行插值处理。对于第一梯度等于第二梯度的第一待插值区域，图像处理装置可以执行下述步骤204和步骤205中的任一步骤以对该第一待插值区域进行插值处理。
步骤204、采用插值算法沿第二方向对第一待插值区域进行插值处理，得到初始插值图像。
在本申请实施例中，若图像处理装置在根据上述步骤203确定出某个第一待插值区域(例如该第一待插值区域为第i行第j列像素的7×7邻域)对应的第一梯度大于第二梯度，说明该第一待插值区域的图像可能为第二方向的强边缘纹理图像，此时图像处理装置可以采用插值算法沿该第二方向对该第一待插值区域进行插值处理，得到该第一待插值区域的插值像素的像素值，即得到初 始插值图像。
可选的，图像处理装置中预先存储的插值算法可以为：最近邻插值算法，双线性插值算法，以及双立方卷积插值算法中的任一种。
在本申请实施例中，图3为本申请实施例提供的一种第一待插值区域的示意图，参考图3，黑色圆圈所表示的像素F1为原始像素，白色方块所表示的像素F2和白色圆圈所表示的像素F3均为插值像素。并且，图像处理装置采用插值算法沿第二方向对该第一待插值区域进行插值处理之后，可以得到白色方块所表示的像素的像素值。
也即是，采用插值算法对该第一待插值区域进行插值处理得到的初始插值图像中，白色方块所表示的像素的像素值可以为已知像素值，而白色圆圈所表示的像素的像素值可以为未知像素值。其中，白色方块所表示的像素与原始像素中和该白色方块所表示的像素相邻的原始像素之间的连线为第一方向A或第二方向B。例如，为45°或135°。
需要说明的是，在采用插值算法沿第二方向对该第一待插值区域进行插值处理得到初始插值图像之前，仅原始像素(黑色圆圈所表示的像素)的像素值为已知像素值，因此图像处理装置可以采用插值算法以及原始像素的像素值沿第二方向对该第一待插值区域进行插值处理，得到初始插值图像。
其中，图像处理装置沿第二方向对该第一待插值区域进行插值处理时，对于每个插值像素(白色方块所表示的像素)的初始像素值，可以是图像处理装置根据该插值像素的L1×L1邻域内，与该插值像素的连线平行于该第二方向上的原始像素的像素值，以及每个原始像素的像素值的权重确定的。并且该原始像素与该插值像素的距离越远，该原始像素的像素值的权重越小，该原始像素与该插值像素的距离越近，该原始像素的像素值的权重越大。其中，L1可以等于7，或者，L1可以等于5。
示例的，为了保证插值处理得到的插值像素的初始像素值的准确性，本申请实施例以L1等于7为例进行说明，参考图3，插值图像中第i行第j列的插值像素的7×7邻域内，有四个原始像素与该第i行第j列的插值像素的连线平行于该第二方向B，该四个原始像素分别为：第i-3行第j-3列像素，第i-1第j-1列像素，第i+1第j+1列像素，以及第i+3行第j+3列像素。则该第i行第j列的插值像素的像素值可以根据该第i-3行第j-3列像素的像素值，第i-1行第j-1列像素的像素值，第i+1行第j+1列像素的像素值，以及第i+3行第j+3行像素 的像素值确定的。
并且，由于第i-3行第j-3列像素，与第i+3行第j+3列与第i行第j列插值像素之间的距离，相对于第i-1行第j-1列像素，以及第i+1行第j+1列像素与第i行第j列插值像素之间的距离大，因此该第i-3行第j-3列像素的像素值，以及第i+3行第j+3列像素的像素值的权重较小，该第i-1行第j-1列像素的像素值，以及第i+1行第j+1列像素的像素值的权重较大。
可选的，该第i-3行第j-3列像素的像素值，以及第i+3行第j+3列像素的像素值的权重可以均为1/6，该第i-1行第j-1列像素的像素值，以及第i+1行第j+1列像素的像素值的权重可以均为1/3。由此假设第i-3行第j-3列像素的像素值为I1，第i+3行第j+3列像素的像素值为I2，该第i-1第j-1列像素的像素值为I3，第i+1第j+1列像素的像素值位I4，则图像处理装置确定出的第i行第j列插值像素的像素值I01可以为：I01＝1/6×I1+1/6×I2+1/3×I3+1/3×I4。
步骤205、采用插值算法沿第一方向对第一待插值区域进行插值处理，得到初始插值图像。
在本申请实施例中，若图像处理装置在根据上述步骤203确定出某个第一待插值区域(例如该第一待插值区域为第i行第j列像素的7×7邻域)的第二梯度大于第一梯度，说明该第一待插值区域的图像可能为第一方向的强边缘纹理图像，此时图像处理装置可以采用插值算法沿该第一方向对该第一待插值区域进行插值处理，得到该第一待插值区域的插值像素的像素值，即得到初始插值图像。
该步骤205的具体过程可以参考上述步骤204，本申请实施例在此不再赘述。
步骤206、确定初始插值图像中第二待插值区域在第三方向的第三梯度。
在本申请实施例中，图像处理装置中可以预先存储有计算图像的第三方向的梯度的公式，该图像处理装置可以采用其预先存储的第三方向的梯度的公式确定该初始插值图像中第二待插值区域在第三方向的第三梯度。
其中，该第二待插值区域可以为初始插值图像中某个L2×L2邻域。L2为大于1的奇数。并且，该第二待插值区域的大小可以与第一待插值区域的大小相同，例如，L1＝L2＝7。或者，该第二待插值区域的大小也可以与第一待插值区域的大小不同，例如，L1＝7，L2＝5。
可选的，结合图3和图4，该第三方向C可以与第一方向A的夹角为45度，例如该第三方向C可以为与像素行方向的夹角为0°的方向。假设初始插值图像 中的第二待插值区域为第i行第j列像素的5×5邻域(即L2＝5)，则该第二待插值区域在第三方向C的第三梯度G3可以满足：
Figure PCTCN2021079774-appb-000001
其中，上述公式(4)中，r，s以及k分别为预设的梯度计算系数，r的取值可以为0和2，s的取值可以为1和-1，k的取值可以为-2，0以及2。当然，r，s以及k的取值还可以为其他值，本申请实施例对此不作限定。并且，I(i-r，j+s)为待处理图像中第i-r行第j+s列像素的像素值，I(i-r+2，j+s)为待处理图像中第i-r+2行第j+s列像素的像素值，I(i-1，j+k)为待处理图像中第i-1行第j+k列像素的像素值，I(i+1，j+k)为待处理图像中第i+1行第j+k列像素的像素值。
步骤207、确定初始插值图像中第二待插值区域在第四方向的第四梯度。
在本申请实施例中，图像处理装置中可以预先存储有计算图像的第四方向的梯度的公式，该图像处理装置可以采用其预先存储的第四方向的梯度的公式确定该待处理图像在第四方向的第四梯度。
可选的，参考图4，该第四方向D可以与第三方向C垂直，例如该第四方向D可以为与像素行方向的夹角为90°的方向。假设该初始插值图像中第二待插值区域为第i行第j列像素的5×5邻域(即L2＝5)，则该第二待插值区域在第四方向D的第四梯度G4可以满足：
Figure PCTCN2021079774-appb-000002
其中，上述公式(5)中，r，s以及k分别为预设的梯度计算系数，r的取值可以为1和-1，s的取值可以为0和2，k的取值可以为-2，0以及2。当然，r，s以及k的取值还可以为其他值，本申请实施例对此不作限定。并且，I(i-r，j-s)为待处理图像中第i-r行第j-s列像素的像素值，I(i+r，j-s+2)为待处理图像中第i+r行第j-s+2列像素的像素值，I(i+k，j-1)为待处理图像中第i+k行第j-1列像素的像素值，I(i+k，j+1)为待处理图像中第i+k行第j+1列像素的像素值。
步骤208、检测第三梯度是否大于第四梯度。
在本申请实施例中，图像处理装置在根据步骤206确定出初始插值图像中第二待插值区域在第三方向的第三梯度，以及根据步骤207确定出初始插值图 像中第二待插值区域在第四方向的第四梯度之后，可以根据该第三梯度和第四梯度确定该第三梯度是否大于第四梯度。
若第三梯度大于第四梯度，图像处理装置可以执行下述步骤209；若第四梯度大于第三梯度，图像处理装置可以执行下述步骤210；若第三梯度等于第四梯度，该第二待插值区域的图像可能为平滑纹理图像，图像处理装置可以执行下述步骤209和步骤210中的任一步骤。
其中，图像处理装置中可以预先存储有确定第三梯度是否大于第四梯度的第二判定公式。可选的，该第二判定公式可以满足：
T2＝(G3+1)/(G4+1)               公式(6)
在本申请实施例中，若图像处理装置根据上述公式(6)确定出T2大于1，说明第三梯度大于第四梯度；若图像处理装置根据上述公式(6)确定出T2小于1，说明第四梯度大于第三梯度；若图像处理装置根据上述公式(6)确定出T2等于1，说明第三梯度等于第四梯度。
需要说明的是，由于对于待处理图像中的一些第二待插值区域，第三梯度可能大于第四梯度，而对于另一些第二待插值区域，第四梯度可能大于第三梯度，因此，对于第三梯度大于第四梯度的第二待插值区域，图像处理装置可以执行下述步骤204以对该第二待插值区域进行插值处理，对于第四梯度大于第三梯度的第二待插值区域，图像处理装置可以执行下述步骤205以对该第二待插值区域进行插值处理。对于第三梯度等于第四梯度的第二待插值区域，图像处理装置可以执行下述步骤209和步骤210中的任一步骤以对该第二待插值区域进行插值处理。
步骤209、采用插值算法沿第四方向对第二待插值区域进行插值处理，得到插值图像。
在本申请实施例中，若图像处理装置在根据上述步骤208确定出某个第二待插值区域(例如第i行第j列像素的5×5邻域)的第三梯度大于第四梯度，说明该第二待插值区域的图像可能为第四方向的强边缘纹理图像，此时图像处理装置可以采用插值算法沿该第四方向对第二待插值区域进行插值处理，得到插值图像。
需要说明的是，参考图4，该初始插值图像中原始像素(黑色圆圈所表示的像素)的像素值以及白色方块所表示的像素的像素值为已知像素值，图像处理装置可以采用插值算法，原始像素的像素值以及白色方块所表示的像素的像素 值对第二待插值区域进行插值处理，得到插值图像。
其中，参考图4，图像处理装置采用插值算法沿第四方向D对第二待插值区域进行插值处理之后，可以得到白色圆圈所表示的像素的像素值。也即是，采用插值算法对第二待插值区域进行插值处理得到的插值图像中，所有像素的像素值均为已知像素值。该白色圆圈所表示的像素与该白色圈圈所表示的像素相邻的原始像素之间的连线为第三方向或第四方向，例如为0°或90°。
该图像处理装置沿第四方向对第二待插值区域进行插值处理时，对于每个插值像素(白色圆圈所表示的像素)的初始像素值，可以是图像处理装置根据该插值像素L2×L2邻域内，与该插值像素的连线平行于第四方向上的所有已知像素值的像素(包括原始像素以及第一次插值处理得到像素值的像素)的像素值，以及每个已知像素值的权重确定的。并且该已知像素值的像素与该插值像素的距离越远，该像素的像素值的权重越小，该已知像素值的像素与该插值像素的距离越近，该像素的像素值的权重越大。其中，L2可以等于7，或者，L2可以等于5。
示例的，为了保证插值处理得到的插值像素的初始像素值的准确性，本申请实施例以L2等于7为例进行说明，参考图4，插值图像中经过第i行第j列的插值像素的7×7邻域内，有四个已知像素值的像素(均为白色方块所表示的像素)与该第i行第j列的插值像素的连线平行于第四方向D，该四个已知像素值的像素分别为：第i-3行第j列像素，第i-1行第j列像素，第i+1行第j列像素，以及第i+3行第j列像素。则该第i行第j列的插值像素可以根据该第i-3行第j列像素的像素值，第i-1行第j列像素的像素值，第i+1行第j列像素的像素值，以及第i+3行第j列像素的像素值确定的。并且，由于第i-3行第j列像素，与第i+3行第j列与第i行第j列插值像素之间的距离，相对于第i-1行第j列像素，以及第i+1行第j列像素与第i行第j列插值像素之间的距离大，因此该第i-3行第j列像素的像素值，以及第i+3行第j列像素的像素值的权重较小，该第i-1第j列像素的像素值，以及第i+1第j列像素的像素值的权重较大。
可选的，该第i-3行第j列像素的像素值，以及第i+3行第j列像素的像素值的权重可以均为1/6，该第i-1行第j列像素的像素值，以及第i+1行第j列像素的像素值的权重可以均为1/3。由此假设第i-3行第j列像素的像素值为I5，第i+3行第j列像素的像素值为I5，该第i-1行第j列像素的像素值为I7，第i+1行第j列像素的像素值位I8，则图像处理装置确定出的第i行第j列插值像素的 像素值I02可以满足I02＝1/6×I5+1/6×I6+1/3×I7+1/3×I8。
需要说明的是，由于在本次插值处理之前，图像处理模块已经进行了一次插值处理，并得到了初始插值图像。该初始插值图像中已知像素值的像素的数量，大于待插值图像中已知像素值的像素的数量。因此，图像处理装置在对初始插值图像中的第二待插值区域进行插值处理时，对于每个插值像素的初始像素值，可以采用该插值像素较小邻域内的像素的像素值确定。例如L2等于5，对于每个插值像素的初始像素值，图像处理装置可以采用该插值像素5×5邻域a内的像素的像素值确定。
在本申请实施例中，参考图5，该插值图像302中位于第2i-1行第2j-1列的像素为原始像素，且插值图像302中第2i-1行第2j-1列的像素的像素值I H(2i-1，2j-1)与待处理图像301中第i行第j列的像素的像素值I L(i，j)相等。也即是，该待处理图像301和插值图像302满足如下映射关系：
I H(2i-1，2j-1)＝I L(i，j)           公式(7)
步骤210、采用插值算法沿第三方向对第二待插值区域进行插值处理，得到插值图像。
在本申请实施例中，若图像处理装置在根据上述步骤210确定出第二待插值区域(例如第i行j列像素的5×5邻域)第四梯度大于第三梯度，说明该第二待插值区域的图像可能为第三方向的强边缘纹理图像，此时图像处理装置可以采用插值算法沿第三方向对第二待插值区域进行插值处理，得到插值图像。
该步骤210的具体过程可以参考上述步骤209，本申请实施例在此不再赘述。
图6是本申请实施例提供的一种对插值图像进行图像处理的方法流程图，该方法可以应用于图像处理装置中，参考图6可以看出，该方法可以包括：
步骤211、将插值像素的初始像素值增加像素步进值，得到第一备选像素值。
在本申请实施例中，图像处理装置中可以预先存储有像素步进值，在该图像处理装置得到插值图像之后，对于每个插值像素，图像处理装置可以将该插值像素的初始像素值增加像素步进值，得到第一备选像素值。其中，该像素步进值的范围可以为1至2，该第一备选像素值大于初始像素值。
示例的，假设某个插值像素的初始像素值为122，像素步进值为2，则图像处理装置得到该插值像素的第一备选像素值为122+2＝124。
步骤212、根据第一备选像素值，以及目标原始像素的像素值，确定第一备选像素值的第一二阶导。
在本申请实施例中，为了确定第一备选像素值的二阶导与目标原始像素的像素值的二阶导的第一差值，图像处理装置可以先根据目标原始像素的像素值以及上述步骤211确定出的第一备选像素值，确定该第一备选像素值的第一二阶导。
其中，目标原始像素可以包括：两个第一目标像素和两个第二目标像素。其中，每个第一目标像素可以与插值像素之间的连线沿第一方向延伸，每个第二目标像素可以与插值像素之间的连线沿第二方向延伸。
由于目标原始像素包括两个第一目标像素和两个第二目标像素，因此图像处理装置可以确定出该四个目标像素中每个目标像素的像素值二阶导与第一备选像素值二阶导的差值。
参考图7，该步骤212可以包括：
步骤2121、根据第一备选像素值，以及两个第一目标像素的像素值，确定第一备选像素值在第一方向上的二阶导。
在本申请实施例中，对于每个插值像素，图像处理装置可以将该插值像素对应的两个第一目标像素的像素值之和，与该插值像素的第一备选像素值的差值确定为该插值像素的第一备选像素值在第一方向上的二阶导。
示例的，参考图4，该插值图像中第i行第j列的插值像素对应的两个第一目标像素分别为该插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素。图像处理装置可以采用该第i+1行第j-1列像素的像素值，第i-1列第j+1列像素的像素值，第i行第j列插值像素的像素值，以及像素步进值，确定该插值图像中第i行第j列的插值像素的第一备选像素值在第一方向上的二阶导。
可选的，该第i行第j列的插值像素的第一备选像素值在第一方向上的二阶导I 11 +(i，j)满足：
I 11 +(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)+Δ]      公式(8)
其中，上述公式(8)中，I(i+1，j-1)为第i+1行第j-1列像素的像素值，I(i-1，j+1)为第i-1行第j+1列像素的像素值，I(i，j)为第i行第j列像素的初始像素值，Δ为像素步进值。I(i，j)+Δ为第一备选像素值。
步骤2122、根据第一备选像素值，以及两个第二目标像素的像素值，确定第一备选像素值在第二方向上的二阶导。
在本申请实施例中，对于每个插值像素，图像处理装置可以将该插值像素对应的两个第二目标像素的像素值之和，与该插值像素的第一备选像素值的差 值确定为该第一备选像素值在第二方向上的二阶导。
示例的，参考图4，插值图像中该第i行第j列的插值像素对应的两个第二目标像素分别为第i-1行第j-1列像素和第i+1列第j+1列像素。图像处理装置可以采用该第i-1行第j-1列像素的像素值，第i+1列第j+1列像素的像素值，第i行第j列插值像素的像素值，以及像素步进值，确定该插值图像中第i行第j列的插值像素的第一备选像素值在第二方向上的二阶导。
可选的，该第i行第j列的插值像素的第一备选像素值在第二方向上的二阶导I 22 +(i，j)满足：
I 22 +(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)+Δ]      公式(9)
其中，上述公式(9)中，I(i-1，j-1)为第i-1行第j-1列像素的像素值，I(i+1，j+1)为第i+1行第j+1列像素的像素值。
步骤213、根据目标原始像素的像素值，以及目标原始像素相邻的像素的像素值，确定目标原始像素的像素值的二阶导。
在本申请实施例中，图像处理装置在根据上述步骤212确定出插值像素的第一备选像素值的第一二阶导之后，为了能够确定该第一备选像素值的二阶导与目标原始像素的像素值的二阶导的差值，还需要根据目标原始像素的像素值，以及与该目标原始像素相邻的像素的像素值，确定目标原始像素的像素值的二阶导。
由于目标原始像素包括两个第一目标像素和两个第二目标像素，因此图像处理装置可以确定出四个目标像素中每个目标像素的像素值的二阶导。并且，图像处理装置需要分别确定每个目标像素在第一方向和第二方向的二阶导，即可以分别确定四个目标像素中每个目标像素在第一方向和第二方向的二阶导，即需确定八个二阶导。
参考图8，该步骤213可以包括：
步骤2131、对于两个第一目标像素和两个第二目标像素中的每个目标像素，根据目标像素的像素值，以及与该目标像素相邻的像素中与该目标像素之间的连线为第一方向的像素的像素值，确定目标像素的像素值在第一方向上的二阶导。
在本申请实施例中，对于插值像素对应的两个第一目标像素和两个第二目标像素中的每个目标像素，图像处理装置可以将与该目标像素之间的连线为第一方向的两个像素的像素值之和，与该目标像素的像素值的差值确定为该目标 像素的像素值在第一方向上的二阶导。
示例的，第i行第j列插值像素对应的第i+1行第j-1列的第一目标像素的像素值在第一方向上的二阶导I 11(i+1，j-1)满足：
I 11(i+1，j-1)＝I(i+2，j-2)+[I(i，j)+Δ]-2×I(i+1，j-1)，其中，I(i+2，j-2)为第i+2行第j-2列像素的像素值。
该第i行第j列插值像素对应的第i-1行第j+1列的第一目标像素的像素值在第一方向上的二阶导I 11(i-1，j+1)满足：
I 11(i-1，j+1)＝[I(i，j)+Δ]+I(i-2，j+2)-2×I(i-1，j+1)，其中，I(i-2，j+2)为第i-2行第j+2列像素的像素值。
该第i行第j列插值像素对应的第i-1行第j-1列的第二目标像素的像素值在第一方向上的二阶导I 11(i-1，j-1)满足：
I 11(i-1，j-1)＝I(i，j-2)+I(i-2，j)-2×I(i-1，j-1)，其中，I(i，j-2)为第i行第j-2列像素的像素值，I(i-2，j)为第i-2行第j列像素的像素值。
该第i行第j列插值像素对应的第i+1行第j+1列的第二目标像素的像素值在第一方向上的二阶导I 11(i+1，j+1)满足：
I 11(i+1，j+1)＝I(i+2，j)+I(i，j+2)-2×I(i+1，j+1)，其中，I(i+2，j)为第i+2行第j列像素的像素值，I(i，j+2)为第i行第j+2列像素的像素值。
步骤2132、对于两个第一目标像素和两个第二目标像素中的每个目标像素，根据目标像素的像素值，以及与该目标像素相邻的像素中与该目标像素之间的连线为第二方向的像素的像素值，确定目标像素的像素值在第二方向上的二阶导。
在本申请实施例中，对于插值像素对应的两个第一目标像素和两个第二目标像素中的每个目标像素，图像处理装置将与该目标像素之间的连线为第二方向的两个像素的像素值之和，与该目标像素的像素值的差值确定为该目标像素的像素值在第二方向的二阶导。
示例的，第i行第j列插值像素对应的第i+1行第j-1列的第一目标像素的像素值在第二方向上的二阶导I 22(i+1，j-1)满足：
I 22(i+1，j-1)＝I(i，j-2)+I(i+2，j)-2×I(i+1，j-1)，其中，I(i，j-2)为第i行第j-2列像素的像素值，I(i+2，j)为第i+2行第j列 像素的像素值。
该第i行第j列插值像素对应的第i-1行第j+1列的第一目标像素的像素值在第二方向上的二阶导I 22(i-1，j+1)满足：
I 22(i-1，j+1)＝I(i-2，j)+I(i，j+2)-2×I(i-1，j+1)，其中，I(i-2，j)为第i-2行第j列像素的像素值，I(i，j+2)为第i行第j+2列像素的像素值。
该第i行第j列插值像素对应的第i-1行第j-1列像素的第二目标像素的像素值在第二方向上的二阶导I 22(i-1，j-1)满足：
I 22(i-1，j-1)＝I(i-2，j-2)+[I(i，j)+Δ]-2×I(i-1，j-1)，其中，I(i-2，j-2)为第i-2行第j-2列像素的像素值。
该第i行第j列插值像素对应的该第i+1行第j+1列像素的第二目标像素的像素值在第二方向上的二阶导I 22(i+1，j+1)满足：
I 22(i+1，j+1)＝[I(i，j)+Δ]+I(i+2，j+2)-2×I(i+1，j+1)，其中，I(i+2，j+2)为第i+2行第j+2列像素的像素值。
步骤214、确定第一二阶导与目标原始像素的像素值的二阶导的第一差值。
在本申请实施例中，图像处理装置在根据步骤212确定出第一备选像素值的第一二阶导，以及根据步骤213确定出目标原始像素的像素值的二阶导之后，可以根据该第一二阶导与目标原始像素的像素值的二阶导确定第一差值。
参考图9，该步骤214可以包括：
步骤2141、对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第一备选像素值在第一方向上的二阶导，与该目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值。
在本申请实施例中，图像处理装置在根据步骤2121确定出第一备选像素值在第一方向上的二阶导，以及根据步骤2131确定出两个第一目标像素和两个第二目标像素中每个目标像素的像素值在第一方向上的二阶导之后，可以确定该第一备选像素值在第一方向上的二阶导，与每个目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值。
示例的，第i行第j列插值像素的第一备选像素值在第一方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j-1列的第一目标像素的像素值在第一方向上的二阶导的差值U 11满足：U 11＝|I 11 +(i，j)-I 11(i+1，j-1)|。
第i行第j列插值像素的第一备选像素值在第一方向上的二阶导，与该第i 行第j列插值像素对应的第i-1行第j+1列的第一目标像素的像素值在第一方向上的二阶导的差值U 12满足：U 12＝|I 11 +(i，j)-I 11(i-1，j+1)|。
第i行第j列插值像素的第一备选像素值在第一方向上的二阶导，与该第i行第j列插值像素对应的第i-1行第j-1列的第二目标像素的像素值在第一方向上的二阶导的差值U 13满足：U 13＝|I 11 +(i，j)-I 11(i-1，j-1)|。
第i行第j列插值像素的第一备选像素值在第一方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j+1列的第二目标像素的像素值在第一方向上的二阶导的差值U 14满足：U 14＝|I 11 +(i，j)-I 11(i+1，j+1)|。
步骤2142、对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第一备选像素值在第二方向上的二阶导，与该目标像素的像素值在第二方向上的二阶导的插值，得到四个第二方向的差值。
在本申请实施例中，图像处理装置在根据步骤2122确定出第一备选像素值在第二方向上的二阶导，以及根据步骤2132确定出两个第一目标像素和两个第二目标像素中每个目标像素的像素值在第二方向上的二阶导之后，可以确定该第一备选像素值在第二方向上的二阶导，与每个目标像素的像素值在第二方向上的二阶导的差值，得到四个第二方向的差值。
示例的，第i行第j列插值像素的第一备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j-1列的第一目标像素的像素值在第二方向上的二阶导的差值U 15满足：U 15＝|I 22 +(i，j)-I 22(i+1，j-1)|。
第i行第j列插值像素的第一备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i-1行第j+1列的第一目标像素的像素值在第二方向上的二阶导的差值U 16满足：U 16＝|I 22 +(i，j)-I 22(i-1，j+1)|。
第i行第j列插值像素的第一备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i-1行第j-1列的第二目标像素的像素值在第二方向上的二阶导的差值U 17满足：U 17＝|I 22 +(i，j)-I 22(i-1，j-1)|。
第i行第j列插值像素的第一备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j+1列的第二目标像素的像素值在第二方向上的二阶导的差值U 18满足：U 18＝|I 22 +(i，j)-I 22(i+1，j+1)|。
步骤2143、对四个第一方向的差值和四个第二方向的差值进行求和，得到第一差值。
在本申请实施例中，图像处理装置在根据步骤2141确定出四个第一方向的 差值，以及根据步骤2142确定出四个第二方向的差值之后，可以对该四个第一方向的差值和四个第二方向的差值进行求和，得到第一差值。
该第一差值U +(i，j)满足：
U +(i，j)＝U 11+U 12+U 13+U 14+U 15+U 16+U 17+U 18。也即是，该第一差值U +(i，j)满足：
Figure PCTCN2021079774-appb-000003
步骤215、将插值像素的初始像素值减小像素步进值，得到第二备选像素值。
在本申请实施例中，对于每个插值像素，图像处理装置还可以将该插值像素的初始像素值减小像素步进值，得到第二备选像素值，该第二备选像素值小于初始像素值。
示例的，假设某个插值像素的初始像素值为122，像素步进值为2，则图像处理装置得到该插值像素的第二备选像素值为122-2＝120。
步骤216、根据第二备选像素值，以及目标原始像素的像素值，确定第二备选像素值的第二二阶导。
在本申请实施例中，为了确定第二备选像素值的二阶导与目标原始像素的像素值的二阶导的第二差值，图像处理装置可以根据目标原始像素的像素值以及上述步骤215确定出的第二备选像素值，确定该第二备选像素值的第二二阶导。
由于目标原始像素包括两个第一目标像素和两个第二目标像素，因此图像处理装置可以确定出该四个目标像素中每个目标像素的像素值二阶导与第二备选像素值的二阶导的差值。
参考图10，该步骤216可以包括：
步骤2161、根据第二备选像素值，以及两个第一目标像素的像素值，确定第二备选像素值在第一方向上的二阶导。
在本申请实施例中，对于每个插值像素，图像处理装置可以将该插值像素对应的两个第一目标像素的像素值之和，与该插值像素的第二备选像素值的差值确定为该第二备选像素值在第一方向上的二阶导。
示例的，参考图4，图像处理装置可以采用该第i+1行第j-1列像素的像素值，第i-1列第j+1列像素的像素值，第i行第j列插值像素的像素值，以及像素步进值，确定该插值像素中第i行第j列的插值像素的第二备选像素值在第一方向上的二阶导。
可选的，第i行第j列的插值像素的第二备选像素值在第一方向上的二阶导I 11 -(i，j)满足：
I 11 -(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)-Δ]      公式(11)
其中，上述公式(11)中，I(i+1，j-1)为第i+1行第j-1列像素的像素值，I(i-1，j+1)为第i-1行第j+1列像素的像素值，I(i，j)为第i行第j列像素的初始像素值，Δ为像素步进值。I(i，j)-Δ为第二备选像素值。
步骤2162、根据第二备选像素值，以及两个第二目标像素的像素值，确定第二备选像素值在第二方向上的二阶导。
在本申请实施例中，对于每个插值像素，图像处理装置可以将两个第二目标像素的像素值之和，与该插值像素的第二备选像素值的差值确定为该第二备选像素值在第二方向上的二阶导。
示例的，图像处理装置可以采用该第i-1行第j-1列像素的像素值，第i+1列第j+1列像素的像素值，第i行第j列插值像素的像素值，以及像素步进值，确定该插值图像中第i行第j列插值像素的第二备选像素值在第二方向上的二阶导。
可选的，该第i行第j列的插值像素的第二备选像素值在第二方向上的二阶导I 22 -(i，j)满足：
I 22 -(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)-Δ]      公式(12)
其中，上述公式(12)中，I(i-1，j-1)为第i-1行第j-1列像素的像素值，I(i+1，j+1)为第i+1行第j+1列像素的像素值。
步骤217、确定第二二阶导与目标原始像素的像素值的二阶导的第二差值。
在本申请实施例中，图像处理装置在根据上述步骤216确定出插值像素的第二备选像素值的第二二阶导，以及根据上述步骤213确定出目标原始像素的像素值的二阶导之后，可以根据该第二二阶导与目标原始像素的像素值的二阶导确定第二差值。
参考图11，该步骤217可以包括：
步骤2171、对于两个第一目标像素和两个第二目标像素中的每个目标像素， 确定第二备选像素值在第一方向上的二阶导，与目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值。
在本申请实施例中，图像处理装置在根据步骤2161确定出第二备选像素值在第一方向上的二阶导，以及根据步骤2131确定出两个第一目标像素和两个第二目标像素中每个目标像素的像素值在第一方向上的二阶导之后，可以确定该第二备选像素值在第一方向上的二阶导，与每个目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值。
示例的，第i行第j列插值像素的第二备选像素值在第一方向上的二阶导与第i+1行第j-1列的第一目标像素的像素值在第一方向上的二阶导的差值U 21满足：U 21＝|I 11 -(i，j)-I 11(i+1，j-1)|。
第i行第j列插值像素的第二备选像素值在第一方向上的二阶导与第i-1行第j+1列的第一目标像素的像素值在第一方向上的二阶导的差值U 22满足：U 22＝|I 11 -(i，j)-I 11(i-1，j+1)|。
第i行第j列插值像素的第二备选像素值在第一方向上的二阶导与第i-1行第j-1列的第二目标像素的像素值在第一方向上的二阶导的差值U 23满足：U 23＝|I 11 -(i，j)-I 11(i-1，j-1)|。
第i行第j列插值像素的第二备选像素值在第一方向上的二阶导与第i+1行第j+1列的第二目标像素的像素值在第一方向上的二阶导的差值U 24满足：U 24＝|I 11 -(i，j)-I 11(i+1，j+1)|。
步骤2172、对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第二备选像素值在第二方向上的二阶导，与目标像素的像素值在第二方向上的二阶导的差值，得到四个第二方向的差值。
在本申请实施例中，图像处理装置在根据步骤2162确定出第二备选像素值在第二方向上的二阶导，以及根据步骤2132确定出两个第一目标像素和两个第二目标像素中每个目标像素的像素值在第二方向上的二阶导之后，可以确定该第二备选像素值在第二方向上的二阶导，与每个目标像素的像素值在第二方向上的二阶导的差值，得到四个第二方向的差值。
示例的，第i行第j列插值像素的第二备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j-1列的第一目标像素的像素值在第二方向上的二阶导的差值U 25满足：U 25＝|I 22 -(i，j)-I 22(i+1，j-1)|。
第i行第j列插值像素的第二备选像素值在第二方向上的二阶导，与该第i 行第j列插值像素对应的第i-1行第j+1列的第一目标像素的像素值在第二方向上的二阶导的差值U 26满足：U 26＝|I 22 -(i，j)-I 22(i-1，j+1)|。
第i行第j列插值像素的第二备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i-1行第j-1列的第二目标像素的像素值在第二方向上的二阶导的差值U 27满足：U 27＝|I 22 -(i，j)-I 22(i-1，j-1)|。
第i行第j列插值像素的第二备选像素值在第二方向上的二阶导，与该第i行第j列插值像素对应的第i+1行第j+1列的第二目标像素的像素值在第二方向上的二阶导的差值U 28满足：U 28＝|I 22 -(i，j)-I 22(i+1，j+1)|。
步骤2173、对四个第一方向的差值和四个第二方向的差值进行求和，得到第二差值。
在本申请实施例中，图像处理装置在根据步骤2171确定出四个第一方向的差值，以及根据步骤2172确定出四个第二方向的差值之后，可以对该四个第一方向的差值和四个第二方向的差值进行求和，得到第二差值。
该第二差值U -(i，j)满足：
U -(i，j)＝U 21+U 22+U 23+U 24+U 25+U 26+U 27+U 28。也即是，该第二差值U -(i，j)满足：
Figure PCTCN2021079774-appb-000004
步骤218、采用第一差值和第二差值中较小的差值对应的备选像素值更新差值像素的初始像素值。
在本申请实施例中，由于对于某个图像而言，该图像中某个像素的像素值的二阶导与该像素相邻的其他像素的像素值的二阶导的差值越小，说明该图像中该像素的邻域内的图像连续性越好，图像的质量越高。
因此，图像处理装置在根据步骤214确定出第一备选像素值的二阶导与目标原始像素的第一差值U +(i，j)，并根据步骤217确定出第二备选像素值的二阶导与目标原始像素的第二差值U -(i，j)之后，图像处理装置可以确定出该第一差值和第二差值的大小。其中，较小的差值对应的备选像素值可以使得插值图像中该插值像素的邻域内的连续性较好，较大的差值对应的备选像素值可以 使得插值图像中该插值像素的邻域内的图像连续性较差。因此，为了提高插值像素的邻域内的图像连续性，以提高插值图像的质量，对于每个插值像素，图像处理装置可以采用第一差值和第二差值中较小的差值对应的备选像素值更新该插值像素的初始像素值。
也即是，若第二差值大于第一差值，可以采用该第二差值对应的第二备选像素值更新该插值像素的初始像素值，即可以使得该插值像素的像素值等于该第二备选像素值。若第二差值小于或等于第一差值，可以采用第一差值对应的第一备选像素值更新该插值像素的初始像素值，即可以使得该插值像素的像素值等于该第一备选像素值。
需要说明的是，图像处理装置可以循环执行上述步骤211至步骤218，即可以不断更新插值图像中各个插值像素的像素值，从而改善插值图像的质量。其中，图像处理装置可以在循环执行上述步骤211至步骤218的循环次数大于或等于次数阈值时停止，并将停止循环时的图像确定为最终处理得到的图像。其中，次数阈值的范围可以为20次至50次。
若对于某个插值像素，图像处理装置在第一次执行步骤218时，采用第一备选像素值和第二备选像素中的目标备选像素值更新该插值像素的初始像素值，则第二次执行步骤211至步骤218时，可以采用该目标备选像素值替换该插值像素的初始像素值，并调整该目标备选像素值，得到新的第一备选像素值和新的第二备选像素值。该新的第一备选像素值大于目标备选像素值，该新的第二备选像素值小于目标备选像素值。
由此，上述步骤211可以为：将插值像素的目标备选像素值增加像素步进值，得到第一备选像素值。上述步骤215可以为：将插值像素的目标备选像素值减小像素步进值，得到第二备选像素值。
图12是本申请实施例提供的一种图像处理效果示意图。其中，401为待处理图像，402为采用本申请实施例提供的图像处理方法对待处理图像401处理得到的部分图像，403为仅采用插值算法对待处理图像401处理得到的部分图像。参考图12可以看出，仅采用插值算法对待处理图像401处理得到的图像存在伪影，纹理失真的问题。而采用本申请实施例提供的图像处理方法得到的图像402的纹理较清晰，图像质量较好。
需要说明的是，由于位于图像边缘的插值像素，无法计算该插值像素的二阶导，因此对于位于图像边缘的插值像素，无需执行上述步骤211至步骤218。 也即是，对于图像边缘的插值像素，图像处理装置可以通过插值算法得到，而无需对其像素值进行调整。并且，由于该插值像素位于图像边缘，不会对图像的质量造成较大的影响。
还需要说明的是，本申请实施例提供的图像处理方法的步骤的先后顺序可以进行适当调整，步骤也可以根据情况进行相应增减。例如，步骤202可以在步骤201之前执行，步骤207可以在步骤206之前执行，步骤212，以及步骤215至步骤217可以在步骤211之前执行，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化的方法，都应涵盖在本申请的保护范围之内，因此不再赘述。
综上所述，本申请实施例提供了一种图像处理方法，该方法可以先对待处理图像进行插值处理，得到插值图像，并对插值图像中每个插值像素的初始像素值进行更新，且更新后的插值像素的像素值的二阶导与插值图像中目标原始像素的像素值的二阶导的差值较小，减少了插值图像中产生的伪影，提高了插值图像中插值像素的邻域内的图像连续性，最终处理得到的图像的质量较高。
图13是本申请实施例提供的一种图像处理装置的结构示意图。参考图13可以看出，该装置可以包括：
插值处理模块501，用于对待处理图像进行插值处理，得到插值图像。
其中，该插值图像可以包括：多个原始像素以及多个插值像素，每个插值像素的初始像素值根据至少一个原始像素的像素值确定。
调整模块502，用于对于每个述插值像素，调整改插值像素的初始像素值，得到第一备选像素值和第二备选像素值。
其中，该第一备选像素值大于初始像素值，该第二备选像素值小于初始像素值。
第一确定模块503，用于确定第一备选像素值的二阶导与多个原始像素中的目标原始像素的像素值的二阶导的第一差值。
其中，目标原始像素为多个原始像素中与插值像素相邻的原始像素。
第二确定模块504，用于确定第二备选像素值的二阶导与目标原始像素的像素值的二阶导的第二差值。
更新模块505，用于采用第一差值和第二差值中较小的差值对应的备选像素值更新插值像素的初始像素值。
综上所述，本申请实施例提供了一种图像处理装置，该装置可以先对待处理图像进行插值处理，得到插值图像，并对插值图像中每个插值像素的初始像素值进行更新，且更新后的插值像素的像素值的二阶导与插值图像中目标原始像素的像素值的二阶导的差值较小，减少了插值图像中产生的伪影，提高了插值图像中插值像素所在区域的图像连续性，最终处理得到的图像的质量较高。
可选的，调整模块502可以用于：
将插值像素的初始像素值增加像素步进值，得到第一备选像素值，以及将插值像素的初始像素值减小像素步进值，得到第二备选像素值。
可选的，参考图14，该第一确定模块503，可以包括：
第一确定子模块5031，用于根据第一备选像素值，以及目标原始像素的像素值，确定第一备选像素值的第一二阶导。
第二确定子模块5032，用于根据目标原始像素的像素值，以及与目标原始像素相邻的像素的像素值，确定目标原始像素的像素值的二阶导。
第三确定子模块5033，用于确定第一二阶导与目标原始像素的像素值的二阶导的第一差值。
参考图15，该第二确定模块504，可以包括：
第四确定子模块5041，用于根据第二备选像素值，以及目标原始像素的像素值，确定第二备选像素值的第二二阶导。
第五确定子模块5042，用于确定第二二阶导与目标原始像素的像素值的二阶导的第二差值。
可选的，目标原始像素可以包括：两个第一目标像素和两个第二目标像素，其中，每个第一目标像素与插值像素之间的连线沿第一方向延伸，每个第二目标像素与插值像素之间的连线沿第二方向延伸，第一方向与第二方向垂直。
第一确定子模块5031，可以用于：
根据第一备选像素值，以及两个第一目标像素的像素值，确定第一备选像素值在第一方向上的二阶导；根据第一备选像素值，以及两个第二目标像素的像素值，确定第一备选像素值在第二方向上的二阶导。
第二确定子模块5032，可以用于：
对于两个第一目标像素和两个第二目标像素中的每个目标像素，根据目标像素的像素值，以及与目标像素相邻的像素中与目标像素之间的连线为第一方 向的像素的像素值，确定目标像素的像素值在第一方向上的二阶导；对于两个第一目标像素和两个第二目标像素中的每个目标像素，根据目标像素的像素值，以及与目标像素相邻的像素中与目标像素之间的连线为第二方向的像素的像素值，确定目标像素的像素值在第二方向上的二阶导。
第三确定子模块5033，可以用于：
对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第一备选像素值在第一方向上的二阶导，与目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值；
对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第一备选像素值在第二方向上的二阶导，与目标像素的像素值在第二方向上的二阶导的差值，得到四个第二方向的差值；
对四个第一方向的差值和四个第二方向的差值进行求和，得到第一差值。
可选的，插值图像中第i行第j列的插值像素对应的两个第一目标像素分别为插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素，第i行第j列的插值像素对应的两个第二目标像素分别为第i-1行第j-1列像素和第i+1列第j+1列像素。
该第i行第j列的插值像素的第一备选像素值在第一方向上的二阶导I 11 +(i，j)满足：I 11 +(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)+Δ]。
其中，I(i+1，j-1)为第i+1行第j-1列像素的像素值，I(i-1，j+1)为第i-1行第j+1列像素的像素值，I(i，j)为第i行第j列像素的初始像素值，Δ为像素步进值，i和j均为大于1的正整数。
该第i行第j列的插值像素的第一备选像素值在第二方向上的二阶导I 22 +(i，j)满足：I 22 +(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)+Δ]；
其中，I(i-1，j-1)为第i-1行第j-1列像素的像素值，I(i+1，j+1)为第i+1行第j+1列像素的像素值。
可选的，第四确定子模块5041，用于：
根据第二备选像素值，以及两个第一目标像素的像素值，确定第二备选像素值在第一方向上的二阶导；根据第二备选像素值，以及两个第二目标像素的像素值，确定第二备选像素值在第二方向上的二阶导。
第五确定子模块5043，用于：
对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第二 备选像素值在第一方向上的二阶导，与目标像素的像素值在第一方向上的二阶导的差值，得到四个第一方向的差值；
对于两个第一目标像素和两个第二目标像素中的每个目标像素，确定第二备选像素值在第二方向上的二阶导，与目标像素的像素值在第二方向上的二阶导的差值，得到四个第二方向的差值；
对四个第一方向的差值和四个第二方向的差值进行求和，得到第二差值。
可选的，第i行第j列的插值像素的第二备选像素值在第一方向上的二阶导I 11 -(i，j)满足：I 11 -(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)-Δ]。
其中，I(i+1，j-1)为第i+1行第j-1列像素的像素值，I(i-1，j+1)为第i-1行第j+1列像素的像素值，I(i，j)为第i行第j列像素的初始像素值，Δ为像素步进值，i和j均为大于1的正整数。
该第i行第j列的插值像素的第二备选像素值在第二方向上的二阶导I 22 -(i，j)满足：I 22 -(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)-Δ]；
其中，I(i-1，j-1)为第i-1行第j-1列像素的像素值，I(i+1，j+1)为第i+1行第j+1列像素的像素值。
可选的，参考图16，该插值处理模块501，可以包括：
第六确定子模块5011，用于确定待处理图像中第一待插值区域在第一方向的第一梯度。
第七确定子模块5012，用于确定第一待插值区域在与第一方向垂直的第二方向的第二梯度。
第一判断子模块5013，用于确定第一梯度是否大于第二梯度。
第一插值子模块5014，用于若第一梯度大于第二梯度，采用插值算法沿第二方向对第一待插值区域进行插值处理，得到初始插值图像；若第二梯度大于第一梯度，采用插值算法沿第一方向对第一待插值区域进行插值处理，得到初始插值图像。
第八确定子模块5015，用于确定初始插值图像的第二待插值区域在第三方向的第三梯度。该第三方向与第一方向的夹角为45度。
第九确定子模块5016，用于确定第二待插值区域在与第三方向垂直的第四方向的第四梯度。
第二判断子模块5017，用于确定第三梯度是否大于第四梯度。
第二插值子模块5018，用于若第三梯度大于第四梯度，采用插值算法沿第 四方向对第二待插值区域进行插值处理，得到插值图像；若第四梯度大于第三梯度，采用插值算法沿第三方向对第二待插值区域进行插值处理，得到插值图像。
可选的，该插值图像中位于第2i-1行第2i-1列的像素为原始像素，且该插值图像中第2i-1行第2i-1列的像素的像素值与待处理图像中第i行第j列的像素的像素值相等。
综上所述，本申请实施例提供了一种图像处理装置，该装置可以先对待处理图像进行插值处理，得到插值图像，并对插值图像中每个插值像素的初始像素值进行更新，且更新后的插值像素的像素值的二阶导与插值图像中目标原始像素的像素值的二阶导的差值较小，减少了插值图像中产生的伪影，提高了插值图像中插值像素所在区域的图像连续性，最终处理得到的图像的质量较高。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的装置、模块以及子模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
图17是本申请实施例提供的另一种图像处理装置的结构示意图。参考图17可以看出，该装置60可以包括：处理器601、存储器602以及存储在该存储器602上并可在该处理器上运行的计算机程序，该处理器601执行该计算机程序时可以实现上述方法实施例提供的图像处理方法，例如图1，图2，图6至图11所示的方法。
本申请实施例提供了一种计算机可读存储介质，该计算机可读存储介质中存储有操作，当该计算机可读存储介质在计算机上运行时，使得计算机执行上述方法实施例提供的图像处理方法，例如可以执行图1，图2，图6至图11所示的方法。
本申请实施例还提供了一种包含指令的计算机程序产品，当计算机程序产品在所述计算机上运行时，使得计算机执行上述方法实施例提供的图像处理方法，例如可以执行图1，图2，图6至图11所示的方法。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过 硬件来完成，也可以通过程序来操作相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
本申请实施例还提供了一种显示装置，该显示装置可以包括显示面板以及上述实施例提供的图像处理装置。其中该显示面板可以用于显示图像处理装置处理后的图像。
可选的，该显示装置可以为液晶显示装置、电子纸、有机发光二极管(organic light-emitting diode，OLED)显示装置、有源矩阵有机发光二极管(active-matrix organic light-emitting diode，AMOLED)显示装置、手机、平板电脑、电视机、显示器、笔记本电脑、数码相框或导航仪等任何具有显示功能的产品或部件。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (15)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    对待处理图像进行插值处理，得到插值图像，所述插值图像包括：多个原始像素以及多个插值像素，每个所述插值像素的初始像素值根据至少一个所述原始像素的像素值确定；
    对于每个所述插值像素，调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，所述第一备选像素值大于所述初始像素值，所述第二备选像素值小于所述初始像素值；
    确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值；
    确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值；
    采用所述第一差值和所述第二差值中较小的差值对应的备选像素值更新所述插值像素的初始像素值；
    其中，所述目标原始像素为所述多个原始像素中与所述插值像素相邻的原始像素。
  2. 根据权利要求1所述的方法，其特征在于，所述调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，包括：
    将所述插值像素的初始像素值增加像素步进值，得到第一备选像素值；
    将所述插值像素的初始像素值减小所述像素步进值，得到第二备选像素值。
  3. 根据权利要求1或2所述的方法，其特征在于，所述确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值，包括：
    根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导；
    根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导；
    确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值；
    所述确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值，包括：
    根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导；
    确定所述第二二阶导与所述目标原始像素的像素值的二阶导的第二差值。
  4. 根据权利要求3所述的方法，其特征在于，所述目标原始像素包括：两个第一目标像素和两个第二目标像素，其中，每个所述第一目标像素与所述插值像素之间的连线沿第一方向延伸，每个所述第二目标像素与所述插值像素之间的连线沿第二方向延伸，所述第一方向与所述第二方向垂直；
    所述根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导，包括：
    根据所述第一备选像素值，以及两个所述第一目标像素的像素值，确定所述第一备选像素值在所述第一方向上的二阶导；
    根据所述第一备选像素值，以及两个所述第二目标像素的像素值，确定所述第一备选像素值在所述第二方向上的二阶导；
    所述根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导，包括：
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第一方向的像素的像素值，确定所述目标像素的像素值在所述第一方向上的二阶导；
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第二方向的像素的像素值，确定所述目标像素的像素值在所述第二方向上的二阶导；
    所述确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值，包括：
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第一备选像素值在所述第一方向上的二阶导，与所述目标像素的像素 值在所述第一方向上的二阶导的差值，得到四个第一方向的差值；
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第一备选像素值在所述第二方向上的二阶导，与所述目标像素的像素值在所述第二方向上的二阶导的差值，得到四个第二方向的差值；
    对四个所述第一方向的差值和四个所述第二方向的差值进行求和，得到第一差值。
  5. 根据权利要求4所述的方法，其特征在于，所述插值图像中第i行第j列的插值像素对应的两个所述第一目标像素分别为所述插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素，所述第i行第j列的插值像素对应的两个所述第二目标像素分别为所述第i-1行第j-1列像素和第i+1列第j+1列像素；所述第i行第j列的插值像素的第一备选像素值在所述第一方向上的二阶导I 11 +(i，j)满足：
    I 11 +(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)+Δ]；
    其中，所述I(i+1，j-1)为所述第i+1行第j-1列像素的像素值，所述I(i-1，j+1)为所述第i-1行第j+1列像素的像素值，所述I(i，j)为所述第i行第j列像素的初始像素值，所述Δ为所述像素步进值，i和j均为大于1的正整数；
    所述第i行第j列的插值像素的第一备选像素值在所述第二方向上的二阶导I 22 +(i，j)满足：
    I 22 +(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)+Δ]；
    其中，所述I(i-1，j-1)为所述第i-1行第j-1列像素的像素值，所述I(i+1，j+1)为所述第i+1行第j+1列像素的像素值。
  6. 根据权利要求3至5任一所述的方法，其特征在于，所述目标原始像素包括：两个第一目标像素和两个第二目标像素，其中，每个所述第一目标像素与所述插值像素之间的连线沿第一方向延伸，每个所述第二目标像素与所述插值像素之间的连线沿第二方向延伸，所述第一方向与所述第二方向垂直；
    所述根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导，包括：
    根据所述第二备选像素值，以及两个所述第一目标像素的像素值，确定所述第二备选像素值在所述第一方向上的二阶导；
    根据所述第二备选像素值，以及两个所述第二目标像素的像素值，确定所述第二备选像素值在所述第二方向上的二阶导；
    所述根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导，包括：
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第一方向的像素的像素值，确定所述目标像素的像素值在所述第一方向上的二阶导；
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，根据所述目标像素的像素值，以及与所述目标像素相邻的像素中与所述目标像素之间的连线为所述第二方向的像素的像素值，确定所述目标像素的像素值在所述第二方向上的二阶导；
    所述确定所述第二二阶导与所述目标原始像素的像素值的二阶导的第二差值，包括：
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第二备选像素值在所述第一方向上的二阶导，与所述目标像素的像素值在所述第一方向上的二阶导的差值，得到四个第一方向的差值；
    对于两个所述第一目标像素和两个所述第二目标像素中的每个目标像素，确定所述第二备选像素值在所述第二方向上的二阶导，与所述目标像素的像素值在所述第二方向上的二阶导的差值，得到四个第二方向的差值；
    对四个所述第一方向的差值和四个所述第二方向的差值进行求和，得到第二差值。
  7. 根据权利要求6所述的方法，其特征在于，所述插值图像中第i行第j列的插值像素对应的两个所述第一目标像素分别为所述插值图像中第i+1行第j-1列像素和第i-1列第j+1列像素，所述第i行第j列的插值像素对应的两个所述第二目标像素分别为所述第i-1行第j-1列像素和第i+1列第j+1列像素；所述第i行第j列的插值像素的第二备选像素值在所述第一方向上的二阶导I 11 -(i，j)满 足：
    I 11 -(i，j)＝I(i+1，j-1)+I(i-1，j+1)-2×[I(i，j)-Δ]；
    其中，所述I(i+1，j-1)为所述第i+1行第j-1列像素的像素值，所述I(i-1，j+1)为所述第i-1行第j+1列像素的像素值，所述I(i，j)为所述第i行第j列像素的初始像素值，所述Δ为所述像素步进值，i和j均为大于1的正整数；
    所述第i行第j列的插值像素的第二备选像素值在所述第二方向上的二阶导I 22 -(i，j)满足：
    I 22 -(i，j)＝I(i-1，j-1)+I(i+1，j+1)-2×[I(i，j)-Δ]；
    其中，所述I(i-1，j-1)为所述第i-1行第j-1列像素的像素值，所述I(i+1，j+1)为所述第i+1行第j+1列像素的像素值。
  8. 根据权利要求1至7任一所述的方法，其特征在于，所述对待处理图像进行插值处理，得到插值图像，包括：
    确定待处理图像中第一待插值区域在第一方向的第一梯度；
    确定所述第一待插值区域在与所述第一方向垂直的第二方向的第二梯度；
    若所述第一梯度大于所述第二梯度，采用插值算法沿所述第二方向对所述第一待插值区域进行插值处理，得到初始插值图像；
    若所述第二梯度大于所述第一梯度，采用插值算法沿所述第一方向对所述第一待插值区域进行插值处理，得到初始插值图像；
    确定所述初始插值图像中第二待插值区域在第三方向的第三梯度，所述第三方向与所述第一方向的夹角为45度；
    确定所述第二待插值区域在与所述第三方向垂直的第四方向的第四梯度；
    若所述第三梯度大于所述第四梯度，采用所述插值算法沿所述第四方向对所述第二待插值区域进行插值处理，得到插值图像；
    若所述第四梯度大于所述第三梯度，采用所述插值算法沿所述第三方向对所述第二待插值区域进行插值处理，得到插值图像。
  9. 根据权利要求至1至8任一所述的方法，其特征在于，所述插值图像中位于第2i-1行第2i-1列的像素为所述原始像素，且所述插值图像中第2i-1行第2i-1 列的像素的像素值与所述待处理图像中第i行第j列的像素的像素值相等。
  10. 一种图像处理装置，其特征在于，所述装置包括：
    插值处理模块，用于对待处理图像进行插值处理，得到插值图像，所述插值图像包括：多个原始像素以及多个插值像素，每个所述插值像素的初始像素值根据至少一个所述原始像素的像素值确定；
    调整模块，用于对于每个所述插值像素，调整所述插值像素的初始像素值，得到第一备选像素值和第二备选像素值，所述第一备选像素值大于所述初始像素值，所述第二备选像素值小于所述初始像素值；
    第一确定模块，用于确定所述第一备选像素值的二阶导与所述多个原始像素中的目标原始像素的像素值的二阶导的第一差值；
    第二确定模块，用于确定所述第二备选像素值的二阶导与所述目标原始像素的像素值的二阶导的第二差值；
    更新模块，用于采用所述第一差值和所述第二差值中较小的差值对应的备选像素值更新所述插值像素的初始像素值；
    其中，所述目标原始像素为所述多个原始像素中与所述插值像素相邻的原始像素。
  11. 根据权利要求10所述的装置，其特征在于，所述调整模块，用于：
    将所述插值像素的初始像素值增加像素步进值，得到第一备选像素值；
    将所述插值像素的初始像素值减小所述像素步进值，得到第二备选像素值。
  12. 根据权利要求10或11所述的装置，其特征在于，所述第一确定模块，包括：
    第一确定子模块，用于根据所述第一备选像素值，以及所述目标原始像素的像素值，确定所述第一备选像素值的第一二阶导；
    第二确定子模块，用于根据所述目标原始像素的像素值，以及与所述目标原始像素相邻的像素的像素值，确定所述目标原始像素的像素值的二阶导；
    第三确定子模块，用于确定所述第一二阶导与所述目标原始像素的像素值的二阶导的第一差值；
    所述第二确定模块，包括：
    第四确定子模块，用于根据所述第二备选像素值，以及所述目标原始像素的像素值，确定所述第二备选像素值的第二二阶导；
    第五确定子模块，用于确定所述第二二阶导与所述目标原始像素的像素值的二阶导的第二差值。
  13. 一种图像处理装置，其特征在于，所述装置包括：处理器、存储器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1至9任一所述的图像处理方法。
  14. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有指令，当所述计算机可读存储介质在计算机上运行时，使得计算机执行如权利要求1至9任一所述的图像处理方法。
  15. 一种显示装置，其特征在于，所述显示装置包括：显示面板以及如权利要求10至13任一所述的图像处理装置；
    所述显示面板用于显示所述图像处理装置处理后的图像。
PCT/CN2021/079774 2020-04-26 2021-03-09 图像处理方法、图像处理装置及显示装置 WO2021218404A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/642,455 US20220301101A1 (en) 2020-04-26 2021-03-09 Method and device for processing image, display device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010337763.0 2020-04-26
CN202010337763.0A CN111553843B (zh) 2020-04-26 2020-04-26 图像处理方法、图像处理装置及显示装置

Publications (1)

Publication Number Publication Date
WO2021218404A1 true WO2021218404A1 (zh) 2021-11-04

Family

ID=72000297

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079774 WO2021218404A1 (zh) 2020-04-26 2021-03-09 图像处理方法、图像处理装置及显示装置

Country Status (3)

Country Link
US (1) US20220301101A1 (zh)
CN (1) CN111553843B (zh)
WO (1) WO2021218404A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114418846A (zh) * 2021-12-30 2022-04-29 深圳市安健科技股份有限公司 一种滤线栅摩尔纹的去除方法及终端

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111553843B (zh) * 2020-04-26 2023-08-18 京东方科技集团股份有限公司 图像处理方法、图像处理装置及显示装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002033964A (ja) * 2000-05-24 2002-01-31 Hewlett Packard Co <Hp> デジタル画像の欠陥画素補正方法
CN103856776A (zh) * 2013-12-26 2014-06-11 辽宁师范大学 边界和曲率驱动修复的图像编码和解码方法
CN104871206A (zh) * 2012-12-19 2015-08-26 马维尔国际贸易有限公司 用于数字图像的自适应缩放的系统和方法
CN106875340A (zh) * 2017-04-05 2017-06-20 上海矽奥微电子有限公司 基于方向差值的图像缩放方法
CN107610052A (zh) * 2017-09-22 2018-01-19 维沃移动通信有限公司 一种超分辨率重建方法及移动终端
CN111553843A (zh) * 2020-04-26 2020-08-18 京东方科技集团股份有限公司 图像处理方法、图像处理装置及显示装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002033964A (ja) * 2000-05-24 2002-01-31 Hewlett Packard Co <Hp> デジタル画像の欠陥画素補正方法
CN104871206A (zh) * 2012-12-19 2015-08-26 马维尔国际贸易有限公司 用于数字图像的自适应缩放的系统和方法
CN103856776A (zh) * 2013-12-26 2014-06-11 辽宁师范大学 边界和曲率驱动修复的图像编码和解码方法
CN106875340A (zh) * 2017-04-05 2017-06-20 上海矽奥微电子有限公司 基于方向差值的图像缩放方法
CN107610052A (zh) * 2017-09-22 2018-01-19 维沃移动通信有限公司 一种超分辨率重建方法及移动终端
CN111553843A (zh) * 2020-04-26 2020-08-18 京东方科技集团股份有限公司 图像处理方法、图像处理装置及显示装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114418846A (zh) * 2021-12-30 2022-04-29 深圳市安健科技股份有限公司 一种滤线栅摩尔纹的去除方法及终端

Also Published As

Publication number Publication date
CN111553843A (zh) 2020-08-18
CN111553843B (zh) 2023-08-18
US20220301101A1 (en) 2022-09-22

Similar Documents

Publication Publication Date Title
WO2021218404A1 (zh) 图像处理方法、图像处理装置及显示装置
US9240033B2 (en) Image super-resolution reconstruction system and method
TWI324329B (en) Image signal processing device
WO2016145776A1 (zh) 一种像素排列结构、显示装置及显示方法
CN105654891A (zh) 一种获取mura补偿值的方法、装置及显示面板
TWI404019B (zh) 資料轉換之裝置與方法、使用此裝置與方法以驅動影像顯示裝置之裝置與方法
KR101980025B1 (ko) 이미지 디스플레이 방법 및 디스플레이 시스템
KR102521949B1 (ko) 영상 보정부 및 표시 장치의 구동 방법
US10475394B2 (en) Driving method and driving device for liquid crystal display, and liquid crystal display
US20160247296A1 (en) Method of Setting Grayscale Value of Liquid Crystal Panel and Liquid Crystal Display
CN109741705B (zh) 一种显示面板的像素补偿方法及装置
TWI672669B (zh) 影像處理方法與裝置
WO2020020076A1 (zh) 灰阶设定方法、显示基板和显示装置
US10650491B2 (en) Image up-scale device and method
JP4011073B2 (ja) 階調補正装置
TWI792407B (zh) 影像處理方法與顯示裝置
KR20070063173A (ko) 액정 표시 장치, 영상 신호 보정 방법 및 그 구동 방법
US10930232B2 (en) Method for color shift compensation based on abnormal image detection
US11397551B2 (en) Method for determining offset distance of splicing screen and related apparatus
US20200143760A1 (en) Driving method and driving device for display panel, and display device
US20060222267A1 (en) Method and apparatus for pixel interpolation
JP2007086577A (ja) 画像処理装置、画像処理方法、画像処理プログラム及び画像表示装置
WO2022252555A1 (zh) 显示面板的灰阶补偿方法及其装置
TWI537873B (zh) 影像修正方法與影像修正裝置
TWI406187B (zh) 快速高畫質視頻影像內插方法及裝置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21796104

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21796104

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.06.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21796104

Country of ref document: EP

Kind code of ref document: A1