WO2024051657A1 - 图像处理方法、装置、计算机设备和存储介质 - Google Patents

图像处理方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2024051657A1
WO2024051657A1 PCT/CN2023/116829 CN2023116829W WO2024051657A1 WO 2024051657 A1 WO2024051657 A1 WO 2024051657A1 CN 2023116829 W CN2023116829 W CN 2023116829W WO 2024051657 A1 WO2024051657 A1 WO 2024051657A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixel
convolution kernel
target
processed
Prior art date
Application number
PCT/CN2023/116829
Other languages
English (en)
French (fr)
Inventor
曲超
Original Assignee
影石创新科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 影石创新科技股份有限公司 filed Critical 影石创新科技股份有限公司
Publication of WO2024051657A1 publication Critical patent/WO2024051657A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration

Definitions

  • the present application relates to the field of image processing technology, and in particular, to an image processing method, device, computer equipment, storage medium and computer program product.
  • this application provides an image processing method. Methods include:
  • each pixel on the downsampled image determines the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on each pixel on the downsampled image The gray value is obtained to obtain the downsampled image;
  • a reference image with the same size as the image to be processed is obtained.
  • the grayscale of the pixel is calculated based on the coordinates of the pixel and the second target convolution kernel.
  • the value is updated; based on the updated grayscale value of each pixel on the reference image, the blurred image corresponding to the image to be processed is obtained.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel, including:
  • the weights at each position in the downsampling convolution kernel are adjusted to obtain the first target convolution kernel.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel, including:
  • the size of the downsampling convolution kernel is adjusted to obtain the first target convolution kernel.
  • the filter radius at the target pixel is determined based on the coordinates of the target pixel and the projection density function of the image to be processed, including:
  • the filter radius at the target pixel is determined based on the projection density value at the target pixel and the original filter radius.
  • determining the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel includes:
  • calculating the grayscale value of the pixel point based on the first target convolution kernel and the pixel value of each sub-pixel point includes:
  • the grayscale value of the pixel is determined based on the accumulated sum of pixel values corresponding to the first convolution area and the normalization coefficient.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel, including:
  • M division directions are determined, and M is greater than 1;
  • the downsampling convolution kernel is adjusted to obtain M target convolution kernels
  • the grayscale value of the pixel is determined based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel, including:
  • the gray value among the M gray values that is closest to the gray value at the target pixel is used as the gray value of the pixel.
  • this application also provides an image processing device.
  • Devices include:
  • the pixel position acquisition module is used to determine the coordinates of each pixel point on the downsampled image based on the preset sampling coefficient and the image to be processed;
  • the first convolution kernel processing module is used to process the downsampling convolution kernel to obtain the first target convolution kernel
  • the downsampling module is used to determine the gray value of each pixel on the downsampled image based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on downsampling The gray value of each pixel on the image is obtained to obtain the downsampled image;
  • the second convolution kernel processing module is used to process the upsampling convolution kernel to obtain the second target convolution kernel
  • the upsampling module is used to obtain a reference image with the same size as the image to be processed based on the downsampled image and the preset sampling coefficient. For each pixel on the reference image, according to the coordinates of the pixel and the second target convolution kernel, The grayscale value of the pixel is updated; based on the updated grayscale value of each pixel on the reference image, a blurred image corresponding to the image to be processed is obtained.
  • this application also provides a computer device.
  • Computer equipment includes a memory and a processor.
  • the memory stores computer programs.
  • the processor executes the computer program, it implements the following steps:
  • each pixel on the downsampled image determines the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on each pixel on the downsampled image The gray value is obtained to obtain the downsampled image;
  • a reference image with the same size as the image to be processed is obtained.
  • the grayscale of the pixel is calculated based on the coordinates of the pixel and the second target convolution kernel.
  • the value is updated; based on the updated grayscale value of each pixel on the reference image, the blurred image corresponding to the image to be processed is obtained.
  • this application also provides a computer-readable storage medium.
  • each pixel on the downsampled image determines the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on each pixel on the downsampled image The gray value is obtained to obtain the downsampled image;
  • a reference image with the same size as the image to be processed is obtained.
  • the grayscale of the pixel is calculated based on the coordinates of the pixel and the second target convolution kernel.
  • the value is updated; based on the updated grayscale value of each pixel on the reference image, the blurred image corresponding to the image to be processed is obtained.
  • this application also provides a computer program product.
  • a computer program product includes a computer program that, when executed by a processor, performs the following steps:
  • each pixel on the downsampled image determines the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on each pixel on the downsampled image The gray value is obtained to obtain the downsampled image;
  • a reference image with the same size as the image to be processed is obtained.
  • the grayscale of the pixel is calculated based on the coordinates of the pixel and the second target convolution kernel.
  • the value is updated; based on the updated grayscale value of each pixel on the reference image, the blurred image corresponding to the image to be processed is obtained.
  • the above-mentioned image processing method, device, computer equipment, storage medium and computer program product perform downsampling processing on the image to be processed through the downsampling convolution kernel to obtain the downsampled image, and upsample the downsampled image through the upsampling convolution kernel to obtain the to-be-processed image.
  • the blurred image corresponding to the image can achieve the blurring of the image to be processed.
  • the size of the downsampled image is smaller than the size of the image to be processed.
  • Upsampling based on the downsampled image can reduce the amount of calculation.
  • the preset sampling coefficient can ensure that after downsampling and upsampling, the blurred image corresponding to the image to be processed is the same as the image to be processed.
  • the images are the same size.
  • the down-sampling convolution kernel and the up-sampling convolution kernel are processed separately, and each pixel on the down-sampling image and the blurred image corresponding to the image to be processed is calculated based on the obtained first target convolution kernel and the second target convolution kernel.
  • the gray value of the point because each pixel on the blurred image corresponding to the image to be processed can calculate its corresponding gray value through the processing of the convolution kernel, which can ensure that the image blur requirements in different scenarios are met.
  • Figure 1 is an application environment diagram of the image processing method in one embodiment
  • Figure 2 is a schematic flow chart of an image processing method in one embodiment
  • Figure 3 is a schematic sub-flow diagram of S202 in one embodiment
  • Figure 4 is a schematic sub-flow diagram of S202 in another embodiment
  • Figure 5 is a schematic sub-flow diagram of S402 in one embodiment
  • Figure 6 is a schematic sub-flow diagram of S203 in one embodiment
  • Figure 7 is a schematic sub-flow diagram of S606 in one embodiment
  • Figure 8 is a comparison diagram of the effect of image processing using a preset mask image in one embodiment
  • Figure 9 is a schematic diagram of the position of sub-pixel points of the target pixel point in one embodiment
  • Figure 10 is a schematic diagram of the position of sub-pixel points of pixels on the reference image in one embodiment
  • Figure 11 is a comparison chart of the effect of local contrast improvement in an embodiment
  • Figure 12 is a comparison chart of the effect of local contrast improvement in another embodiment
  • Figure 13 is a schematic diagram of the effect of the image processing method in one embodiment
  • Figure 14 is a structural block diagram of an image processing device in one embodiment
  • Figure 15 is an internal structure diagram of a computer device in one embodiment.
  • the image processing method provided by the embodiment of the present application can be applied in the application environment as shown in Figure 1.
  • the terminal 102 communicates with the server 104 through the network.
  • the data storage system may store data that server 104 needs to process.
  • the data storage system can be integrated on the server 104, or placed on the cloud or other network servers.
  • the image processing method provided by each embodiment of the present application can be executed individually by the terminal 102 or the server 104, or can also be executed by the terminal 102 and the server 104 working together.
  • the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices.
  • the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. .
  • Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.
  • the server 104 can be implemented as an independent server or a server cluster composed of multiple servers.
  • an image processing method is provided. This method is explained by taking the method applied to the terminal 102 in Figure 1 as an example, and includes the following steps:
  • S201 Determine the coordinates of each pixel on the downsampled image based on the preset sampling coefficient and the image to be processed.
  • the image to be processed is an image for image processing.
  • the types of images to be processed include images with uniform projection density or images with uneven projection density.
  • Images with uneven projection density can be images captured by fisheye cameras.
  • Images with uneven projection density include but are not limited to fisheye camera images, spherical A panoramic image or a plane unfolded image of a spherical panorama, an image with uniform projection density can be an image captured by a plane camera.
  • the preset sampling coefficient is the factor by which the image to be processed is reduced or enlarged.
  • the preset sampling coefficient is the factor by which the image to be processed is reduced in size. Downsampling is also called downsampling, which can be understood as reducing the image.
  • the downsampled image is an image obtained by downsampling the image to be processed.
  • the terminal determines the coordinates of each pixel on the downsampled image based on the preset sampling coefficient and the image to be processed. Specifically, the terminal performs downsampling processing on the image to be processed based on the preset sampling coefficient, and the size of the obtained downsampled image is smaller than the size of the image to be processed.
  • the size of the image to be processed is A*B
  • the preset sampling coefficient is S
  • the size of the downsampled image obtained after downsampling the image to be processed is (A/S)*(B/S).
  • the coordinates of each pixel on the downsampled image are the relative positions of each pixel on the downsampled image on the downsampled image. According to the size of the downsampled image, the coordinates of each pixel on the downsampled image can be obtained.
  • S202 Process the downsampling convolution kernel to obtain the first target convolution kernel.
  • the downsampling convolution kernel is a convolution kernel used when downsampling the image to be processed to generate a downsampled image. Specifically, when downsampling the image to be processed, the pixel values of the pixels in each small area in the image to be processed become the pixel values of the corresponding pixels in the downsampled image after a weighted average, wherein each pixel is weighted and averaged.
  • the weight is defined by a function, which is the downsampling convolution kernel.
  • the terminal processes the downsampling convolution kernel to obtain the first target convolution kernel.
  • the processing method includes any one of a preset mask method, a variable projection density method, or an edge preservation method.
  • the preset mask map method is a method of adjusting the weight values in the downsampling convolution kernel by introducing a preset mask map.
  • the terminal processes the downsampling convolution kernel through the preset mask map method to obtain the first target convolution kernel, which can effectively eliminate invalid pixels in the image to be processed when performing image processing on the image to be processed with irregular effective areas. interference.
  • the influence of surrounding areas on the processing area can be controlled by flexibly selecting a preset mask image.
  • the variable projection density method is a method that adjusts the radius of the downsampling convolution kernel for different projection densities.
  • the terminal processes the down-sampling convolution kernel through the variable projection density method to obtain the first target convolution kernel.
  • the down-sampling convolution kernel can be adjusted through the variable projection density. radius to achieve uniform blurring of the image to be processed.
  • the edge preservation method is to divide the sub-pixel points in the area to be processed in the image to be processed in multiple directions, perform image processing based on each of the multiple directions, and divide the sub-pixel points in the area to be processed into the image with the highest degree of similarity to the image to be processed. Image processing results as a method for downsampling images.
  • the terminal processes the downsampling convolution kernel through the edge preservation method to obtain the first target convolution kernel, which can ensure that the edges of the blurred image corresponding to the image to be processed are clearer.
  • the repeated areas in multiple division directions can be highly utilized. Avoid a large number of repeated calculations and improve image processing efficiency.
  • the target pixel point is a pixel point on the down-sampled image that is mapped to a corresponding pixel point in the image to be processed.
  • the coordinates of the target pixel point can be obtained according to the coordinates of the pixel point on the down-sampled image and the preset sampling coefficient.
  • the preset sampling coefficient is 2, the coordinates of the pixels on the downsampled image are (1, 2), and the coordinates of the target pixels are (2, 4).
  • the gray value of a pixel is the gray value of the pixel on the downsampled image.
  • the gray value represents the color depth of each pixel in the image, and can include any value between 0 and 255.
  • the terminal calculates the grayscale value of the corresponding pixel point on the down-sampled image according to the coordinates of the target pixel point obtained by mapping the pixel point to the image to be processed and the first target convolution kernel. Based on the grayscale value of each pixel on the downsampled image, the terminal can obtain the color depth of each pixel of the downsampled image, thereby obtaining the downsampled image.
  • RGB Red, Green, Blue, Red, Green, Blue
  • the upsampling convolution kernel is a convolution kernel used when upsampling a downsampled image to generate an upsampled image. Specifically, when upsampling the downsampled image, the pixel values of the pixels in each small area in the downsampled image are weighted and averaged to become the pixel values of the corresponding pixels in the upsampled image, where each pixel is weighted.
  • the average weight is defined by a function, which is the upsampling convolution kernel.
  • the terminal processes the upsampling convolution kernel to obtain the second target convolution kernel.
  • the processing method includes any one of a preset mask method, a variable projection density method, or an edge preservation method.
  • the preset mask map method is a method of adjusting the weight values in the upsampling convolution kernel by introducing a preset mask map.
  • the variable projection density method is a method that adjusts the radius of the upsampling convolution kernel for different projection densities.
  • the edge preservation method is to divide the sub-pixels in the area to be processed in the downsampled image in multiple directions, perform image processing based on each of the multiple directions, and divide the sub-pixels in the area that is most similar to the image to be processed. Image processing results as a method for upsampling images.
  • S205 based on the downsampled image and the preset sampling coefficient, obtain a reference image with the same size as the image to be processed. For each pixel on the reference image, according to the coordinates of the pixel and the second target convolution kernel, the pixel is The grayscale value is updated; based on the updated grayscale value of each pixel on the reference image, the blurred image corresponding to the image to be processed is obtained.
  • the preset sampling coefficient is a multiple of the size method for the downsampled image.
  • Upsampling is a method of inserting new pixels between the pixels of the downsampled image using an interpolation algorithm. Through upsampling, the downsampled image can be enlarged.
  • the upsampled image is an image obtained by upsampling the downsampled image.
  • the reference image is an image obtained by processing the downsampled image through an interpolation algorithm. Based on the downsampled image and the preset sampling coefficient, the terminal obtains a reference image with the same size as the image to be processed. Specifically, the terminal performs upsampling processing on the downsampled image based on the preset sampling coefficient, and the size of the obtained reference highlight is the same as the size of the image to be processed.
  • the size of the image to be processed is A*B
  • the preset sampling coefficient is S
  • the size of the downsampled image is (A/S)*(B/S)
  • the size of the reference image is (A/S*S)* (B/S*S), which is A*B.
  • the blurred image corresponding to the image to be processed is the result of image processing of the image to be processed.
  • the size of the blurred image corresponding to the image to be processed is equal to the size of the reference image, that is, equal to the size of the image to be processed.
  • the size of the image to be processed remains unchanged before and after image processing.
  • the gray value of a pixel refers to the pixel value of the pixel on the reference image.
  • the pixel value of the pixel on the reference image is obtained by performing an interpolation algorithm on the downsampled image. For each pixel on the reference image, the terminal recalculates the gray value of each pixel on the reference image based on the coordinates of the pixel, the gray value of the pixel, and the second target convolution kernel to obtain each pixel on the reference image.
  • the updated grayscale value of the point The terminal uses the updated grayscale value of the reference image as the blurred image corresponding to the image to be processed based on the updated grayscale value of each pixel point on the reference image.
  • the image to be processed is downsampled through the downsampling convolution kernel to obtain the downsampled image
  • the downsampled image is upsampled through the upsampling convolution kernel to obtain the blurred image corresponding to the image to be processed, which can realize the image to be processed of blurring.
  • the size of the downsampled image is smaller than the size of the image to be processed. Upsampling based on the downsampled image can reduce the amount of calculation.
  • the preset sampling coefficient can ensure that after downsampling and upsampling, the blurred image corresponding to the image to be processed is the same as the image to be processed.
  • the images are the same size.
  • the down-sampling convolution kernel and the up-sampling convolution kernel are processed separately, and each pixel on the down-sampling image and the blurred image corresponding to the image to be processed is calculated based on the obtained first target convolution kernel and the second target convolution kernel.
  • the gray value of the point because each pixel on the blurred image corresponding to the image to be processed can calculate its corresponding gray value through the processing of the convolution kernel, which can ensure that the image blur requirements in different scenarios are met.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel, including:
  • the down-sampling convolution kernel can be expressed in the form of a matrix, and the size of the down-sampling convolution kernel refers to the size of the matrix.
  • the default mask image is a mask image preset by the terminal, which can be flexibly selected according to actual needs.
  • the mask map is an image used for mask calculation.
  • the selected image is used to block the image to be processed to control the processing area and processing process of the image to be processed.
  • the selected image is the mask map.
  • Mask images are mainly used to extract areas of interest, shielding effects, structural feature extraction and special shape image production.
  • the terminal obtains a preset mask image with the same size as the downsampling convolution kernel, and the size of the preset mask image is equal to the size of the downsampling convolution kernel.
  • S304 Based on the preset mask map, adjust the weights at each position in the downsampling convolution kernel to obtain the first target convolution kernel.
  • the weight at each position in the down-sampling convolution kernel refers to the element value at each position in the matrix corresponding to the down-sampling convolution kernel.
  • the terminal adjusts the weights at each position in the downsampling convolution kernel to obtain the first target convolution kernel. Specifically, the terminal convolves the preset mask image with the downsampling convolution kernel. Calculate, and the obtained calculation result is used as the adjusted weight at each position in the downsampling convolution kernel, which is the first target convolution kernel.
  • the size of the first target convolution kernel is equal to the size of the downsampling convolution kernel. It should be noted that in this application, each pixel in the downsampled image corresponds to a preset mask image.
  • the size of the preset mask image is equal to the size of the downsampling convolution kernel.
  • Each pixel in the downsampled image corresponds to
  • the preset mask image can also be composed into a large mask image, and the size of the large mask image is equal to the size of the image to be processed.
  • the downsampling convolution kernel is represented by Dx
  • the preset mask map is represented by Ex
  • the first target convolution kernel is represented by D1.
  • the first target convolution kernel is obtained by adjusting the weights at each position in the down-sampling convolution kernel using a preset mask map with the same size as the down-sampling convolution kernel. It can be flexibly selected according to the actual situation. By selecting an appropriate preset mask image, the weights at each position in the downsampling convolution kernel can be adjusted. Compared with directly using the downsampling convolution kernel for downsampling, downsampling based on the obtained first target convolution kernel can effectively eliminate invalid pixels in the image to be processed when processing images with irregular effective areas. interference.
  • the influence of the surrounding area on the processing area can be flexibly selected by flexibly selecting a preset mask, which is beneficial to achieving blurring of the image to be processed in a scene with invalid pixel interference. need.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel, including:
  • S402 Determine the filter radius at the target pixel according to the coordinates of the target pixel and the projection density function of the image to be processed.
  • the projection density function is a function that reflects the projection density of the image to be processed at different pixel positions.
  • the projection density at each pixel position of the image to be processed may be different.
  • the filter radius is a parameter used to determine the size of the first target convolution kernel. The terminal brings the coordinates of the target pixel point into the projection density function of the image to be processed to obtain the filter radius at the target pixel point. Since the projection density at each pixel position of the image to be processed may be different, the obtained filter radius at each target pixel may be different.
  • S404 Based on the filter radius, adjust the size of the downsampling convolution kernel to obtain the first target convolution kernel.
  • the terminal adjusts the size of the downsampling convolution kernel based on the filter radius to obtain the first target convolution kernel.
  • the size of the first target convolution kernel is determined by the filter radius. For example, if the filter radius is r, then the size of the first target convolution kernel is (2r+1)*(2r+1).
  • the weights at each position in the first target convolution kernel are adjusted accordingly, and the value of the weight is the same as the value of the weight in the downsampling convolution kernel. For example, if the filter radius is 2, then the size of the first target convolution kernel is 5*5, the downsampling convolution kernel is represented by Dx, and the first target convolution kernel is represented by D1:
  • the filter radius at the target pixel is determined through the coordinates of the target pixel and the projection density function of the image to be processed. Since the projection density at each pixel position of the image to be processed may be different, each target pixel obtained The filter radius at may be different. Based on the filter radius, adjust the size of the downsampling convolution kernel to obtain the first target convolution kernel. The size of the first target convolution kernel is determined by the filter radius. The first target convolution kernel at each target pixel may be different. . When the terminal performs image processing on the image to be processed with uneven projection density through the first target convolution kernel, the size of the first target convolution kernel is adjusted through the projection density function, which is beneficial to realizing the image to be processed in the scene with uneven projection density. uniform blur.
  • the filter radius at the target pixel is determined based on the coordinates of the target pixel and the projection density function of the image to be processed, including:
  • the terminal determines the original filter radius based on the downsampling convolution kernel.
  • the original filter radius is a parameter that determines the size of the downsampling convolution kernel.
  • the original filter radius is r 0 and the size of the downsampling convolution kernel is (2r 0 +1)*(2r 0 +1).
  • the terminal obtains the coordinates of the center point of the image to be processed, where the coordinates of the center point of the image to be processed are the relative positions of the center point of the image to be processed in the image to be processed.
  • the coordinates of the center point of the image to be processed include the abscissa coordinate of the center point of the image to be processed and the ordinate coordinate of the center point of the image to be processed.
  • the coordinates of the target pixel point include the abscissa coordinate of the target pixel point and the ordinate coordinate of the target pixel point.
  • the terminal brings the coordinates of the target pixel point and the coordinates of the center point into the projection density function to obtain the projection density value at the target pixel point.
  • the projection density function is a function related to the coordinates of the target pixel point.
  • the coordinates of the target pixel point are represented by X.
  • the projection density function f(X) can be:
  • x and y are respectively the abscissa and ordinate coordinates of the target pixel point
  • c x and c y are respectively the abscissa and ordinate coordinates of the center point of the image to be processed.
  • S508 Determine the filter radius at the target pixel according to the projection density value at the target pixel and the original filter radius.
  • the terminal multiplies the projection density value at the target pixel point by the original filter radius, and uses the resulting product as the filter radius at the target pixel point.
  • the original filter radius is represented by r 0
  • the filter radius is represented by r X , then r value.
  • the projection density values at each target pixel are different, and the obtained filter radii at the target pixel are also different. For example, if the preset filter radius is 2, c x is 10, c y is 5, x is 6, and y is 3, then the filter radius r X at the first target position X is 1.2.
  • the coordinates of the target pixel point and the coordinates of the center point of the image to be processed are brought into the projection density function to obtain the projection density value at the target pixel point, and the projection density value at the target pixel point is compared with the original filter radius. Multiply to get the filter radius at the target pixel. Since the original filter radius is a fixed value, and for the image to be processed with uneven projection density, the projection density value at each target pixel point is different, and the obtained filter radius at the target pixel point is also different. For any image to be processed with uneven projection density, the filter radius of the target pixel can be calculated through the projection density function, and the first target convolution kernel is obtained based on the filter radius, which is beneficial to achieve the target pixel in scenes with uneven projection density. Processes uniform blurring of images.
  • determining the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel includes:
  • S602 Determine the first convolution area participating in the convolution operation according to the coordinates of the target pixel point and the first target convolution kernel.
  • the first convolution area is the image area covered by the first target convolution kernel at the target pixel point of the image to be processed.
  • the size of the first convolution area is equal to the size of the first target convolution kernel.
  • the terminal uses the target pixel as the center of the image to be processed and the corresponding area with the size of the first target convolution kernel as the first convolution area participating in the convolution operation.
  • sub-pixel points refer to smaller points subdivided between two adjacent pixels, which can be obtained by interpolating around the original pixels through an interpolation algorithm.
  • Common interpolation algorithms include bilinear interpolation.
  • the terminal searches for the sub-pixel point from the first convolution area. Specifically, the terminal uses bilinear interpolation to calculate the position of the sub-pixel point from the first convolution area.
  • S606 Calculate the gray value of the pixel according to the first target convolution kernel and the pixel value of each sub-pixel.
  • the pixel value of each sub-pixel point can be obtained through the interpolation algorithm.
  • the terminal calculates the grayscale value of the corresponding pixel on the downsampled image based on the first target convolution kernel and the pixel value of each sub-pixel.
  • the down-sampled image is calculated based on the first target convolution kernel and the pixel values of each sub-pixel point.
  • the gray value of each pixel Since each pixel on the downsampled image can be processed by the downsampling convolution kernel, its corresponding gray value can be calculated, which can ensure that the image blur requirements in different scenarios are met.
  • calculating the grayscale value of the pixel point based on the first target convolution kernel and the pixel value of each sub-pixel point includes:
  • S702 Calculate the product of the pixel value of each sub-pixel point and the corresponding weight value in the first target convolution kernel, and add the corresponding products of all sub-pixel points to obtain the cumulative sum of pixel values corresponding to the first convolution area.
  • the terminal multiplies the weight value at each position in the first target convolution kernel by the pixel value of the corresponding sub-pixel point to obtain the corresponding product of all sub-pixel points, and then adds the obtained products to obtain the third The cumulative sum of pixel values corresponding to a convolution area.
  • S704 Determine the normalization coefficient based on the downsampling convolution kernel and the first target convolution kernel.
  • the terminal determines the normalization coefficient based on the downsampling convolution kernel and the first target convolution kernel. Specifically, the terminal sums the weight values at each position in the downsampling convolution kernel to obtain the first summation result; and sums the weight values at each position in the first target convolution kernel to obtain the second summation result. ;Divide the first summation result by the second summation result to obtain the normalization coefficient.
  • the weight in the downsampling convolution kernel is expressed as dx
  • the pixel value in the preset mask image is expressed as mx
  • the normalization coefficient w is:
  • S706 Determine the gray value of the pixel according to the accumulated sum of pixel values corresponding to the first convolution area and the normalization coefficient.
  • the terminal multiplies the cumulative sum of pixel values corresponding to the first convolution area by the normalization coefficient to obtain the grayscale value of the corresponding pixel point on the downsampled image.
  • the pixel value of the sub-pixel point is expressed as I x
  • the weight in the downsampling convolution kernel is expressed as d x
  • the pixel value in the preset mask map is expressed as m x
  • the first convolution area is expressed as ⁇ .
  • the gray value of the corresponding pixel on the downsampled image is expressed as q
  • the gray value calculation formula is:
  • the coordinates of the target pixel point are ),(5.5,5.5)], the pixel value corresponding to the sub-pixel point
  • the default mask picture is Put I x , dx and m
  • the normalization coefficient is not used, the gray value of the corresponding pixel on the obtained down-sampled image is 10, and the brightness of the down-sampled image will be much lower than the brightness of the image to be processed.
  • the obtained down-sampled image will be much lower than the brightness of the image to be processed. If the grayscale value of the corresponding pixel in the sampled image is 20, the brightness of the downsampled image is closer to the brightness of the image to be processed, which is beneficial to obtaining better image processing effects.
  • the pixel corresponding to the first convolution area is obtained.
  • Value accumulation sum accumulate and multiply the normalization coefficient determined based on the downsampling convolution kernel and the first target convolution kernel and the pixel value corresponding to the first convolution area to determine the gray value of the pixel point.
  • the function of the normalization coefficient can keep the brightness of the image to be processed from changing drastically before and after downsampling. Since the gray value of each pixel on the downsampled image can be calculated by processing the downsampling convolution kernel, it can ensure that the image blur requirements in different scenarios are met.
  • processing the down-sampling convolution kernel to obtain the first target convolution kernel includes: determining the second convolution area participating in the convolution operation according to the coordinates of the target pixel point and the down-sampling convolution kernel; Based on the coordinates of each sub-pixel point in the second convolution area and the coordinates of the target pixel point, M division directions are determined, M is greater than 1; based on the M division directions, the downsampling convolution kernel is adjusted to obtain M targets Convolution kernel; correspondingly, determine the gray value of the pixel according to the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel, including: according to the coordinates of the target pixel and each target volume Accumulate the kernel and calculate M grayscale values; among the M grayscale values, the grayscale value closest to the grayscale value at the target pixel is used as the grayscale value of the pixel.
  • the second convolution area is the image area covered by the first target convolution kernel at the target pixel point of the image to be processed.
  • the size of the second convolution area is equal to the size of the first target convolution kernel.
  • the terminal uses the area corresponding to the target pixel as the center and the size of the first target convolution kernel as the second convolution area participating in the convolution operation on the image to be processed.
  • the terminal determines M division directions based on the coordinates of each sub-pixel point in the second convolution area and the coordinates of the target pixel point. Specifically, the terminal takes the coordinates of each target pixel point in the second convolution area as the center, selects a corresponding number of sub-pixel points in the second convolution area, and determines one of the M division directions.
  • the number of sub-pixel points corresponding to each division direction can be different, and M is greater than 1.
  • the terminal adjusts the down-sampling convolution kernel based on M division directions to obtain M target convolution kernels. Specifically, the terminal selects the weight corresponding to each division direction according to the M division directions for the down-sampling convolution kernel, and obtains M target convolution kernels. Target convolution kernel.
  • the terminal determines the grayscale value of the pixel according to the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel. Specifically, for each target pixel, the terminal brings the coordinates of the target pixel and each target convolution kernel into the gray value calculation formula, and calculates M gray values.
  • the terminal compares the M grayscale values with the grayscale value at the target pixel point respectively to obtain the comparison result. According to the comparison result, the grayscale value among the M grayscale values that is closest to the grayscale value at the target pixel point is selected. , as the gray value of the pixel.
  • M division directions are determined through the coordinates of each sub-pixel point in the second convolution area and the coordinates of the target pixel point, and the down-sampling convolution kernel is adjusted based on the M division directions, and the coordinates of the target pixel point are and each target convolution kernel are brought into the gray value calculation formula, and the gray value closest to the gray value at the target pixel is selected from the gray values obtained in the M division directions as the gray value of the pixel.
  • This division direction method of selecting the grayscale value closest to the grayscale value of the target pixel point as the grayscale value of the pixel point from M division directions can ensure that the edges of the downsampled image are clearer.
  • the preset mask map corresponding to the invalid pixels can be flexibly adjusted according to the actual situation.
  • the interference of invalid pixels in the image to be processed can be effectively eliminated, and Meet the image blur requirements in scenarios where the image to be processed contains invalid pixels.
  • the coordinates of each pixel on the downsampled image are determined.
  • the downsampling convolution kernel is processed to obtain the first target convolution kernel.
  • the first convolution area participating in the convolution operation is determined based on the coordinates of the target pixel point and the first target convolution kernel, and sub-pixel points are found from the first convolution area. Calculate the product of the pixel value of each sub-pixel point and the corresponding weight value in the first target convolution kernel, and add the corresponding products of all sub-pixel points to obtain the cumulative sum of pixel values corresponding to the first convolution area.
  • the convolution kernel and the first target convolution kernel determining the normalization coefficient, and determining the gray value of the pixel point based on the accumulated sum of pixel values corresponding to the first convolution area and the normalization coefficient.
  • the downsampled image is obtained. Since each pixel on the downsampled image can be processed by the downsampling convolution kernel, its corresponding gray value can be calculated, which can ensure that the image blur requirements in different scenarios are met.
  • the weight in the downsampling convolution kernel is expressed as dx
  • the pixel value in the preset mask image is expressed as mx
  • the normalization coefficient w is:
  • the pixel value of the sub-pixel point is represented by I x
  • the first convolution area is represented by ⁇
  • the gray value of the corresponding pixel point on the downsampled image is represented by q.
  • the gray value calculation formula is:
  • the coordinates of the target pixel point are ),(5.5,5.5)], the pixel value corresponding to the sub-pixel point
  • the default mask picture is Put I x , dx and m
  • the normalization coefficient is not used, the gray value of the corresponding pixel on the obtained down-sampled image is 10, and the brightness of the down-sampled image will be much lower than the brightness of the image to be processed.
  • the obtained down-sampled image will be much lower than the brightness of the image to be processed. If the grayscale value of the corresponding pixel in the sampled image is 20, the brightness of the downsampled image is closer to the brightness of the image to be processed, which is beneficial to obtaining better image processing effects.
  • the upsampling convolution kernel is processed to obtain the second target convolution kernel.
  • a reference image with the same size as the image to be processed is obtained.
  • the gray value of the pixel is updated according to the coordinates of the pixel and the second target convolution kernel.
  • the blurred image corresponding to the image to be processed is obtained.
  • the down-sampling convolution kernel is processed to obtain the first target convolution kernel. Specifically, a preset mask image with the same size as the down-sampling convolution kernel is obtained. Based on the preset mask image, the down-sampling convolution kernel is The weights at each position are adjusted to obtain the first target convolution kernel.
  • the downsampling convolution kernel is represented by Dx
  • the preset mask map is represented by Ex
  • the first target convolution kernel is represented by D1. in,
  • the upsampling convolution kernel is processed to obtain the second target convolution kernel. Specifically, a mask image with the same size as the upsampling convolution kernel is obtained. Based on the mask image, the images at each position in the upsampling convolution kernel are obtained. The weights are adjusted to obtain the second target convolution kernel.
  • the upsampling convolution kernel is represented by Ds
  • the mask map is represented by Es
  • the second target convolution kernel is represented by D2. in,
  • Figure 8 shows a comparison diagram of the image to be processed without and processed by the preset mask image, where (a) is the image to be processed, (b) is the preset mask image, ( c) is the image after image processing using traditional image processing methods on the image to be processed. Since it has not been processed by the preset mask image (b), a black halo infiltrates into the edge of the image in (c), and (d) is the image to be processed.
  • the blurred image corresponding to the processed image that is, the image processed by the preset mask map.
  • the edge of the image in (c) exists Black halo infiltrates, and no black halo infiltrates into the edge of the image in (d). Therefore, using the solution of this application, the preset mask map can be flexibly adjusted according to the actual situation, thereby flexibly controlling the area that needs to be blurred in the image to be processed, and also Ability to eliminate interfering pixels in blurred areas.
  • the weights at each position in the downsampling convolution kernel and the upsampling convolution kernel can be adjusted. Make adjustments. Downsampling based on the obtained first target convolution kernel is compared to directly using the downsampling convolution kernel for downsampling, and upsampling based on the obtained second target convolution kernel is compared to directly using the upsampling convolution kernel. Upsampling can effectively eliminate the interference of invalid pixels in the image to be processed when performing image processing on the image to be processed where the screen is blocked by objects and invalid pixels appear.
  • the image to be processed is a fisheye photo, a spherical panoramic image, or a flat expanded image of a spherical panorama:
  • the down-sampling convolution kernel to obtain the first target convolution kernel. Specifically, according to the down-sampling convolution kernel, determine the original filter radius, obtain the coordinates of the center point of the image to be processed, and combine the coordinates of the target pixel point and the center The coordinates of the point are brought into the projection density function to obtain the projection density value at the target pixel point. According to the projection density value at the target pixel point and the original filter radius, the filter radius at the target pixel point is determined. Based on the filter radius, downsampling is performed The size of the convolution kernel is adjusted to obtain the first target convolution kernel.
  • the upsampling convolution kernel is processed to obtain the second target convolution kernel.
  • the initial filter radius corresponding to the upsampling convolution kernel is determined, the coordinates of the center point of the reference image are obtained, and the reference The coordinates of each pixel point on the image and the coordinates of the center point are brought into the projection density function to obtain the projection density value at each pixel point on the reference image.
  • the projection density value at each pixel point on the reference image and the initial filter radius determine Referring to the filter radius at each pixel point on the image, based on the filter radius, adjust the size of the upsampling convolution kernel to obtain the second target convolution kernel.
  • the projection density at each pixel position of the image to be processed may be different, and the obtained filter radius at each target pixel may be different.
  • the original filter radius is r 0
  • the size of the downsampling convolution kernel is (2r 0 +1)*(2r 0 +1).
  • the projection density function is a function related to the coordinates of the target pixel point.
  • the coordinates of the target pixel point are represented by X
  • the original filter radius is represented by r 0
  • the filter radius is represented by r X.
  • the projection density function f(X) can be:
  • x and y are respectively the abscissa and ordinate coordinates of the target pixel point
  • c x and c y are respectively the abscissa and ordinate coordinates of the center point of the image to be processed.
  • F(X) is the projection function value obtained by bringing the coordinates of the target pixel point into the projection density function.
  • the filter radius r X at the first target position X is 1.2. Since the original filter radius is a fixed value, and for the image to be processed with uneven projection density, the projection density value at each target pixel point is different, and the obtained filter radius at the target pixel point is also different. For any image to be processed with uneven projection density, the filter radius of the target pixel and the filter radius of the pixel on the reference image can be calculated through the projection density function. Based on the two filter radii, the first target convolution kernel and the third target convolution kernel can be obtained respectively. The two-target convolution kernel is beneficial to achieve uniform blurring of the image to be processed in scenes with uneven projection density.
  • the down-sampling convolution kernel is processed to obtain the first target convolution kernel. Specifically, according to the coordinates of the target pixel point and the down-sampling convolution kernel, the second convolution area participating in the convolution operation is determined. Based on the second convolution The coordinates of each sub-pixel in the area and the coordinates of the target pixel determine M division directions, M is greater than 1. Based on the M division directions, the downsampling convolution kernel is adjusted to obtain M target convolution kernels. Correspondingly, the gray value of the pixel is determined according to the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel.
  • FIG. 9 is a schematic diagram of the position of the sub-pixel point of the target pixel point in the second convolution area, in which C, NW, NE, SW and SE in the figure respectively represent the position of the sub-pixel point of the target pixel point.
  • C, NW, NE, SW and SE in the figure respectively represent the position of the sub-pixel point of the target pixel point.
  • NW-C Fifth division direction: NW-C
  • the downsampling convolution kernel is adjusted to obtain 8 target convolution kernels.
  • the terminal brings the coordinates of the target pixel and each target convolution kernel into the gray value calculation formula, and calculates 8 gray values.
  • the repeating areas are first calculated: NW-C, NE-C, SW-C, SE-C, and then the corresponding values are added on the basis of the repeating areas, which is beneficial to improving calculation efficiency.
  • the terminal compares the eight grayscale values with the grayscale value at the target pixel point respectively to obtain the comparison result. Based on the comparison result, the terminal selects the grayscale value closest to the grayscale value at the target pixel point among the eight grayscale values. , as the gray value of the pixel.
  • the upsampling convolution kernel is processed to obtain the second target convolution kernel. Specifically, according to the coordinates of each pixel point on the reference image and the upsampling convolution kernel, the third convolution area participating in the convolution operation is determined, Based on the coordinates of each sub-pixel point in the third convolution area and the coordinates of the pixel points on the reference image, M division directions are determined, and M is greater than 1. Based on the M division directions, the upsampling convolution kernel is adjusted to obtain M target convolution kernel. Correspondingly, the gray value of the pixel is updated according to the coordinates of the pixel and the second target convolution kernel. Specifically, M gray values are calculated according to the coordinates of the pixel and each target convolution kernel. The gray value of the pixel is updated based on the gray value among the M gray values that is closest to the gray value of the target pixel.
  • Figure 10 shows a schematic diagram of the sub-pixel positions of pixels on the reference image in the third convolution area, where C, N, W, S, E, NW, NE, SW and SE in the figure represent the reference image respectively.
  • the position of the sub-pixel points around the upper pixel point is determined according to the coordinates of each sub-pixel point in the third convolution area and the coordinates of the pixel points on the reference image.
  • the eight division directions are divided into:
  • the upsampling convolution kernel is adjusted to obtain 8 target convolution kernels.
  • the terminal brings the coordinates of the pixel on the reference image and each target convolution kernel into the gray value calculation formula, and calculates 8 gray values.
  • the repeating areas are first calculated: NW-C, NE-C, SW-C, SE-C, and then the corresponding values are added on the basis of the repeating areas, which is beneficial to improving calculation efficiency.
  • the terminal compares the 8 grayscale values with the grayscale value at the corresponding pixel point on the reference image respectively to obtain the comparison result. Based on the comparison result, the terminal selects the 8 grayscale values that are closest to the grayscale value at the target pixel point. Gray value, as the gray value of the pixel.
  • This division direction method selects the grayscale value closest to the grayscale value of the target pixel point from M division directions as the grayscale value of the pixel point, which can ensure that the edges of the downsampled image and the blurred image of the image to be processed are more clear. clear. And for scenes where the image to be processed contains invalid pixels, the preset mask map corresponding to the invalid pixels can be flexibly adjusted according to the actual situation. When performing image processing on the image to be processed, the interference of invalid pixels in the image to be processed can be effectively eliminated, and Meet the image blur requirements in scenarios where the image to be processed contains invalid pixels.
  • the gray value of each pixel in the image to be processed is subtracted from the gray value of the corresponding pixel on the blurred image corresponding to the image to be processed to obtain the gray value difference corresponding to each pixel in the image to be processed.
  • the brightness value corresponding to the pixel point with a grayscale difference greater than 0 is increased, the brightness value corresponding to the pixel point in the image to be processed with a grayscale difference value less than 0 is reduced, and the grayscale difference value in the image to be processed is less than
  • the brightness value corresponding to the pixel point of 0 remains unchanged, and the image with the changed brightness value can be obtained, which can improve the local contrast between the image to be processed and the blurred image.
  • Figure 11 shows the before and after comparison of local contrast improvement of a spherical panoramic image, where (a) indicates that the image to be processed is a spherical panoramic image, and (b) indicates that the local contrast of the image to be processed indicated in (a) is improved. After the image, (b) the local contrast is more obvious than (a).
  • Figure 12 shows the before and after comparison of the local contrast improvement of the plane expanded image of the spherical panorama.
  • (a) in Figure 12 indicates that the image to be processed is the plane expanded image of the spherical panorama, and (b) in Figure 12 is After performing local contrast enhancement on the image to be processed indicated in (a) in Figure 12 , the local contrast in (b) in Figure 12 is more obvious than in (a) in Figure 12 .
  • edge preservation refers to the method of dividing M directions and selecting the dividing direction closest to the gray value at the target pixel point among the M dividing directions.
  • the above image processing method uses a down-sampling convolution kernel to down-sample the image to be processed to obtain a down-sampled image, and uses an up-sampling convolution kernel to up-sample the down-sampled image to obtain a blurred image corresponding to the image to be processed, which can realize the processing of the image to be processed. blur.
  • the size of the downsampled image is smaller than the size of the image to be processed. Upsampling based on the downsampled image can reduce the amount of calculation.
  • the preset sampling coefficient can ensure that after downsampling and upsampling, the blurred image corresponding to the image to be processed is the same as the image to be processed.
  • the images are the same size.
  • the down-sampling convolution kernel and the up-sampling convolution kernel are processed separately, and each pixel on the down-sampling image and the blurred image corresponding to the image to be processed is calculated based on the obtained first target convolution kernel and the second target convolution kernel.
  • the gray value of the point because each pixel on the blurred image corresponding to the image to be processed can calculate its corresponding gray value through the processing of the convolution kernel, which can ensure that the image blur requirements in different scenarios are met.
  • both the down-sampling step and the up-sampling step in the image processing method in this application can be iterated N times, where the number of iterations of the down-sampling step is equal to the number of iterations of the up-sampling step, and N is greater than or equal to 1.
  • embodiments of the present application also provide an image processing device for implementing the above-mentioned image processing method.
  • the solution to the problem provided by this device is similar to the solution described in the above method. Therefore, for the specific limitations in one or more image processing device embodiments provided below, please refer to the above limitations on the image processing method. I won’t go into details here.
  • an image processing device 100 including: a pixel position acquisition module 110, a first convolution kernel processing module 120, a downsampling module 130, and a second convolution kernel processing module. 140 and upsampling module 150, where:
  • the pixel position acquisition module 110 is used to determine the coordinates of each pixel point on the downsampled image based on the preset sampling coefficient and the image to be processed.
  • the first convolution kernel processing module 120 is used to process the downsampling convolution kernel to obtain the first target convolution kernel.
  • the downsampling module 130 is used to determine the grayscale value of each pixel on the downsampled image according to the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel; based on the following Sampling the gray value of each pixel on the image to obtain a downsampled image.
  • the second convolution kernel processing module 140 is used to process the upsampling convolution kernel to obtain a second target convolution kernel.
  • the upsampling module 150 is used to obtain a reference image with the same size as the image to be processed based on the downsampled image and the preset sampling coefficient. For each pixel on the reference image, according to the coordinates of the pixel and the second target convolution kernel , update the grayscale value of the pixel; based on the updated grayscale value of each pixel on the reference image, obtain the blurred image corresponding to the image to be processed.
  • the above image processing device performs downsampling processing on the image to be processed through a downsampling convolution kernel to obtain a downsampled image, and upsamples the downsampled image through an upsampling convolution kernel to obtain a blurred image corresponding to the image to be processed, which can realize the processing of the image to be processed. blur.
  • the size of the downsampled image is smaller than the size of the image to be processed. Upsampling based on the downsampled image can reduce the amount of calculation.
  • the preset sampling coefficient can ensure that after downsampling and upsampling, the blurred image corresponding to the image to be processed is the same as the image to be processed.
  • the images are the same size.
  • the down-sampling convolution kernel and the up-sampling convolution kernel are processed separately, and each pixel on the down-sampling image and the blurred image corresponding to the image to be processed is calculated based on the obtained first target convolution kernel and the second target convolution kernel.
  • the gray value of the point because each pixel on the blurred image corresponding to the image to be processed can calculate its corresponding gray value through the processing of the convolution kernel, which can ensure that the image blur requirements in different scenarios are met.
  • the first convolution kernel processing module 120 when processing the down-sampling convolution kernel to obtain the first target convolution kernel, is also used to: obtain a preset mask with the same size as the down-sampling convolution kernel. template map; based on the preset mask map, adjust the weights at each position in the downsampling convolution kernel to obtain the first target convolution kernel.
  • the first convolution kernel processing module 120 in processing the downsampling convolution kernel to obtain the first target convolution kernel, is also used to: according to the coordinates of the target pixel point and the projection density of the image to be processed function to determine the filter radius at the target pixel; based on the filter radius, adjust the size of the downsampling convolution kernel to obtain the first target convolution kernel.
  • the first convolution kernel processing module 120 is also used to: according to the downsampling convolution kernel , determine the original filter radius; obtain the coordinates of the center point of the image to be processed; bring the coordinates of the target pixel point and the coordinates of the center point into the projection density function to obtain the projection density value at the target pixel point; according to the The projected density value and the original filter radius determine the filter radius at the target pixel point.
  • the downsampling module 130 in determining the grayscale value of the pixel based on the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel, is also used to: based on the target pixel The coordinates of the point and the first target convolution kernel determine the first convolution area participating in the convolution operation; search for sub-pixel points from the first convolution area; according to the first target convolution kernel and the pixel value of each sub-pixel point , calculate the gray value of the pixel.
  • the downsampling module 130 is also used to: calculate the pixel value of each sub-pixel point and the pixel value of each sub-pixel point.
  • the product of the corresponding weight values in a target convolution kernel, and the corresponding products of all sub-pixel points are added to obtain the cumulative sum of pixel values corresponding to the first convolution area; according to the downsampling convolution kernel and the first target convolution Kernel, determine the normalization coefficient; determine the gray value of the pixel according to the accumulated sum of pixel values corresponding to the first convolution area and the normalization coefficient.
  • the first convolution kernel processing module 120 is also used to: according to the coordinates of the target pixel point and the down-sampling convolution kernel, Determine the second convolution area participating in the convolution operation; based on the coordinates of each sub-pixel point in the second convolution area and the coordinates of the target pixel point, determine M division directions, M is greater than 1; based on the M division directions,
  • the downsampling convolution kernel is adjusted to obtain M target convolution kernels; accordingly, the gray value of the pixel is determined according to the coordinates of the target pixel obtained by mapping the pixel to the image to be processed and the first target convolution kernel, It includes: calculating M grayscale values based on the coordinates of the target pixel and each target convolution kernel; taking the grayscale value closest to the grayscale value at the target pixel among the M grayscale values as the grayscale of the pixel. degree value.
  • Each module in the above image processing device can be implemented in whole or in part by software, hardware and combinations thereof.
  • Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure diagram may be as shown in Figure 15.
  • the computer device includes a processor, a memory, an input/output interface (Input/Output, referred to as I/O), and a communication interface.
  • the processor, memory and input/output interface are connected through the system bus, and the communication interface is connected to the system bus through the input/output interface.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes non-volatile storage media and internal memory.
  • the non-volatile storage medium stores operating systems, computer programs and databases. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media.
  • the database of the computer device is used to store preset sampling coefficients, images to be processed, downsampling convolution kernels, upsampling convolution kernels, first target convolution kernels, second target convolution kernels, and each pixel on the downsampling image.
  • the input/output interface of the computer device is used to exchange information between the processor and external devices.
  • the communication interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer program implements an image processing method when executed by the processor.
  • Figure 15 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied.
  • Specific computer equipment can May include more or fewer parts than shown, or combine certain parts, or have a different arrangement of parts.
  • a computer device including a memory and a processor.
  • a computer program is stored in the memory.
  • the processor executes the computer program, it implements the steps in the above method embodiments.
  • a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the steps in the above method embodiments are implemented.
  • a computer program product including a computer program that implements the steps in each of the above method embodiments when executed by a processor.
  • the user information including but not limited to user equipment information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the computer program can be stored in a non-volatile computer-readable storage medium. , when executed, the computer program may include the processes of the above method embodiments. Any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory.
  • Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory (MRAM), ferroelectric memory (Ferroelectric Random Access Memory (FRAM)), phase change memory (Phase Change Memory, PCM), graphene memory, etc.
  • Volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory.
  • RAM Random Access Memory
  • RAM Random Access Memory
  • RAM Random Access Memory
  • RAM Random Access Memory
  • RAM Random Access Memory
  • RAM Random Access Memory
  • RAM random access memory
  • RAM Random Access Memory
  • RAM can be in many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM) etc.
  • the databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database.
  • Non-relational databases may include blockchain-based distributed databases, etc., but are not limited thereto.
  • the processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to this.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Image Processing (AREA)

Abstract

本申请涉及一种图像处理方法、装置、计算机设备、存储介质和计算机程序产品。方法包括：基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；对下采样卷积核和上采样卷积核分别进行处理，分别得到第一目标卷积核和第二目标卷积核；对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，得到下采样图像；基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新，得到待处理图像对应的模糊图像。采用本方法能够保证满足不同场景下的图像模糊需求。

Description

图像处理方法、装置、计算机设备和存储介质 技术领域
本申请涉及图像处理技术领域，特别是涉及一种图像处理方法、装置、计算机设备、存储介质和计算机程序产品。
背景技术
随着科技的发展和用户需求的提高，出现了多种多样的图像处理算法，图像处理算法决定了图像呈现的最终效果。目前的图像处理方法，针对投影密度不均匀图像的场景，例如，鱼眼相机图像、球形全景图像以及球形全景的平面展开图像，投影密度不均匀的图像进行处理后存在图像模糊程度不均匀的问题；针对图像画面存在相机机身等物体遮挡的无效像素的场景，包含无效像素的图像进行图像处理后会存在一部分有效区域被损坏的问题，因此，针对投影密度不均匀图像的场景和图像包含无效像素的场景，现有的图像处理方法，存在不能同时满足不同场景下的图像模糊需求的问题。
发明内容
基于此，有必要针对投影密度不均匀图像的场景和图像包含无效像素的场景，传统的图像处理方法不能同时满足不同场景下的图像模糊需求的问题，提供一种图像处理方法、装置、计算机设备、计算机可读存储介质和计算机程序产品，针对投影密度不均匀图像的场景和图像包含无效像素的场景，能够同时满足不同场景下的图像模糊需求。
第一方面，本申请提供了一种图像处理方法。方法包括：
基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
对下采样卷积核进行处理，得到第一目标卷积核；
对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
对上采样卷积核进行处理，得到第二目标卷积核；
基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
在其中一个实施例中，对下采样卷积核进行处理，得到第一目标卷积核，包括：
获取与下采样卷积核尺寸相同的预设掩模图；
基于预设掩膜图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核。
在其中一个实施例中，对下采样卷积核进行处理，得到第一目标卷积核，包括：
根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径；
基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。
在其中一个实施例中，根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径，包括：
根据下采样卷积核，确定原始滤波半径；
获取待处理图像的中心点的坐标；
将目标像素点的坐标和中心点的坐标，带入投影密度函数，得到目标像素点处的投影密度值；
根据目标像素点处的投影密度值和原始滤波半径，确定目标像素点处的滤波半径。
在其中一个实施例中，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，包括：
根据目标像素点的坐标和第一目标卷积核，确定参与卷积运算的第一卷积区域；
从第一卷积区域中查找亚像素点；
根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值。
在其中一个实施例中，根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值，包括：
计算各个亚像素点的像素值和第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到第一卷积区域对应的像素值累加和；
根据下采样卷积核与第一目标卷积核，确定归一化系数；
根据第一卷积区域对应的像素值累加和与归一化系数，确定像素点的灰度值。
在其中一个实施例中，对下采样卷积核进行处理，得到第一目标卷积核，包括：
根据目标像素点的坐标以及下采样卷积核，确定参与卷积运算的第二卷积区域；
基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向，M大于1；
基于M个划分方向，对下采样卷积核进行调整，得到M个目标卷积核；
相应的，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，包括：
根据目标像素点的坐标和各个目标卷积核，计算得到M个灰度值；
将M个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
第二方面，本申请还提供了一种图像处理装置。装置包括：
像素位置获取模块，用于基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
第一卷积核处理模块，用于对下采样卷积核进行处理，得到第一目标卷积核；
下采样模块，用于对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
第二卷积核处理模块，用于对上采样卷积核进行处理，得到第二目标卷积核；
上采样模块，用于基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
第三方面，本申请还提供了一种计算机设备。计算机设备包括存储器和处理器，存储器存储有计算机程序，处理器执行计算机程序时实现以下步骤：
基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
对下采样卷积核进行处理，得到第一目标卷积核；
对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
对上采样卷积核进行处理，得到第二目标卷积核；
基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
第四方面，本申请还提供了一种计算机可读存储介质。计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：
基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
对下采样卷积核进行处理，得到第一目标卷积核；
对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
对上采样卷积核进行处理，得到第二目标卷积核；
基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
第五方面，本申请还提供了一种计算机程序产品。计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现以下步骤：
基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
对下采样卷积核进行处理，得到第一目标卷积核；
对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
对上采样卷积核进行处理，得到第二目标卷积核；
基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
上述图像处理方法、装置、计算机设备、存储介质和计算机程序产品，通过下采样卷积核对待处理图像进行下采样处理得到下采样图像，通过上采样卷积核对下采样图像进行上采样得到待处理图像对应的模糊图像，能够实现待处理图像的模糊化。同时，下采样图像的尺寸小于待处理图像的尺寸，基于下采样图像进行上采样能够减少计算量，预设采样系数能够保证经过下采样以及上采样后，待处理图像对应的模糊图像与待处理图像的尺寸相同。将下采样卷积核和上采样卷积核分别进行处理，基于得到的第一目标卷积核和第二目标卷积核分别计算出下采样图像以及待处理图像对应的模糊图像上每个像素点的灰度值，由于待处理图像对应的模糊图像上每个像素点都能够通过对卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。
附图说明
图1为一个实施例中图像处理方法的应用环境图；
图2为一个实施例中图像处理方法的流程示意图；
图3为一个实施例中S202的子流程示意图；
图4为另一个实施例中S202的子流程示意图；
图5为一个实施例中S402的子流程示意图；
图6为一个实施例中S203的子流程示意图；
图7为一个实施例中S606的子流程示意图；
图8为一个实施例中采用预设掩模图进行图像处理的效果对比图；
图9为一个实施例中目标像素点的亚像素点的位置示意图；
图10为一个实施例中参考图像上像素点的亚像素点的位置示意图；
图11为一个实施例中局部对比度提升的效果对比图；
图12为另一个实施例中局部对比度提升的效果对比图；
图13为一个实施例中图像处理方法的效果示意图；
图14为一个实施例中图像处理装置的结构框图；
图15为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例提供的图像处理方法，可以应用于如图1所示的应用环境中。其中，终端102通过网络与服务器104进行通信。数据存储系统可以存储服务器104需要处理的数据。数据存储系统可以集成在服务器104上，也可以放在云上或其他网络服务器上。本申请各实施例所提供的图像处理方法可通过终端102或服务器104单独执行，还可以通过终端102和服务器104共同协作执行。以终端102单独执行为例进行说明：基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；对下采样卷积核进行处理，得到第一目标卷积核；对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；对上采样卷积核进行处理，得到第二目标卷积核；基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。其中，终端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑、物联网设备和便携式可穿戴设备，物联网设备可为智能音箱、智能电视、智能空调、智能车载设备等。便携式可穿戴设备可为智能手表、智能手环、头戴设备等。服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一个实施例中，如图2所示，提供了一种图像处理方法，以该方法应用于图1中的终端102为例进行说明，包括以下步骤：
S201，基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标。
其中，待处理图像为进行图像处理的图像。待处理图像的类型包括投影密度均匀的图像或者投影密度不均匀的图像，投影密度不均匀的图像可以为鱼眼相机拍摄的图像，投影密度不均匀的图像包括但不限于鱼眼相机图像、球形全景图像或者球形全景的平面展开图像，投影密度均匀的图像可以为平面相机拍摄的图像。
预设采样系数为对待处理图像进行尺寸缩小或者放大的倍数，在对待处理图像进行下采样处理时，预设采样系数为对待处理图像进行尺寸缩小的倍数。下采样又称为降采样，可以理解为缩小图像。
下采样图像为对待处理图像进行下采样处理得到的图像。终端基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标。具体地，终端基于预设采样系数对待处理图像进行下采样处理，获得的下采样图像的尺寸小于待处理图像的尺寸。例如，待处理图像的尺寸为A*B，预设采样系数为S，对待处理图像进行下采样处理后得到的下采样图像的尺寸为(A/S)*(B/S)。
下采样图像上每个像素点的坐标为下采样图像上每个像素点在下采样图像上的相对位置。根据下采样图像的尺寸可以得出下采样图像上每个像素点的坐标。
S202，对下采样卷积核进行处理，得到第一目标卷积核。
其中，下采样卷积核是对待处理图像进行下采样处理生成下采样图像时采用的卷积核。具体地，在对待处理图像进行下采样处理时，待处理图像中各个小区域内的像素点的像素值经过加权平均后成为下采样图像中对应像素点的像素值，其中，各个像素点进行加权平均时的权值由一个函数定义，这个函数即为下采样卷积核。终端对下采样卷积核进行处理，得到第一目标卷积核。
其中，处理的方法包括预设掩模图法、可变投影密度法或者边缘保持方法中的任一种方法。预设掩模图法是通过引入预设掩模图对下采样卷积核中的权重值进行调整的方法。终端通过预设掩模图法对下采样卷积核进行处理，得到第一目标卷积核，在对于有效区域不规则的待处理图像进行图像处理时，能够有效消除待处理图像中无效像素的干扰。并且，在对待处理图像中任意形状的区域做图像处理时，可以通过灵活选择预设掩模图来控制周边区域对处理区域的影响。可变投影密度法是通过对不同投影密度调整下采样卷积核的半径的方法。终端通过可变投影密度法对下采样卷积核进行处理，得到第一目标卷积核，能够对投影密度不均匀的待处理图像进行图像处理时，通过可变投影密度调整下采样卷积核的半径，实现待处理图像均匀模糊。边缘保持方法是将待处理图像中的待处理区域中的亚像素点进行多个方向的划分，基于多个方向中的每个方向进行图像处理，将与待处理图像相似程度最高的方向对应的图像处理结果作为下采样图像的方法。终端通过边缘保持方法对下采样卷积核进行处理，得到第一目标卷积核，能够保证待处理图像对应的模糊图像的边缘更加清晰，同时，多个划分方向中的重复区域可高度利用，避免大量重复计算，提高图像处理效率。
S203，对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像。
其中，目标像素点是下采样图像上的像素点映射至待处理图像中对应的像素点，目标像素点的坐标可以根据下采样图像上的像素点的坐标和预设采样系数得到。例如预设采样系数为2，下采样图像上的像素点的坐标为(1，2)，则目标像素点的坐标为(2，4)。像素点的灰度值是下采样图像上的像素点的灰度值。其中，灰度值是表示图像各个像素点的颜色深度的值，可以包括0到255之间任意数值。终端对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，计算得到下采样图像上对应像素点的灰度值。终端基于下采样图像上每个像素点的灰度值，可以得到下采样图像各个像素点的颜色深度，从而得到下采样图像。采用单通道的灰度值相较于采用三通道的RGB(Red Green Blue，红绿蓝)值，简化了计算量，有利于提高图像处理的效率。
S204，对上采样卷积核进行处理，得到第二目标卷积核。
其中，上采样卷积核是对下采样图像进行上采样处理生成上采样图像时采用的卷积核。具体地，在对下采样图像进行上采样处理时，下采样图像中各个小区域内的像素点的像素值经过加权平均后成为上采样图像中对应像素点的像素值，其中，各个像素点进行加权平均时的权值由一个函数定义，这个函数即为上采样卷积核。终端对上采样卷积核进行处理，得到第二目标卷积核。
其中，处理的方法包括预设掩模图法、可变投影密度法或者边缘保持方法中的任一种方法。预设掩模图法是通过引入预设掩模图对上采样卷积核中的权重值进行调整的方法。可变投影密度法是通过对不同投影密度调整上采样卷积核的半径的方法。边缘保持方法是将下采样图像中的待处理区域中的亚像素点进行多个方向的划分，基于多个方向中的每个方向进行图像处理，将与待处理图像相似程度最高的方向对应的图像处理结果作为上采样图像的方法。
S205，基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
其中，在对下采样图像进行上采样处理时，预设采样系数为对下采样图像进行尺寸方法的倍数。上采样是在下采样图像的像素点之间采用插值算法插入新的像素点的方法，通过上采样，可以实现下采样图像的放大。
上采样图像为对下采样图像进行上采样处理得到的图像。参考图像是将下采样图像经过插值算法处理得到的图像。终端基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像。具体地，终端基于预设采样系数对下采样图像进行上采样处理，获得的参考凸显的尺寸与待处理图像尺寸相同。例如，待处理图像的尺寸为A*B，预设采样系数为S，下采样图像的尺寸为(A/S)*(B/S)，参考图像的尺寸为(A/S*S)*(B/S*S)，即为A*B。
待处理图像对应的模糊图像为对待处理图像进行图像处理得到的结果，待处理图像对应的模糊图像的尺寸等于参考图像的尺寸，也即等于待处理图像的尺寸。待处理图像在图像处理前后图像的尺寸不变。
像素点的灰度值指的是参考图像上的像素点的像素值，参考图像上的像素点的像素值是通过对下采样图像进行插值算法处理得到。对于参考图像上的每个像素点，终端根据像素点的坐标、像素点的灰度值以及第二目标卷积核，重新计算参考图像上各个像素点的灰度值，得到参考图像上各个像素点的更新后的灰度值。终端基于参考图像上各个像素点的更新后的灰度值，将更新灰度值后的参考图像作为待处理图像对应的模糊图像。
上述图像处理方法中，通过下采样卷积核对待处理图像进行下采样处理得到下采样图像，通过上采样卷积核对下采样图像进行上采样得到待处理图像对应的模糊图像，能够实现待处理图像的模糊化。同时，下采样图像的尺寸小于待处理图像的尺寸，基于下采样图像进行上采样能够减少计算量，预设采样系数能够保证经过下采样以及上采样后，待处理图像对应的模糊图像与待处理图像的尺寸相同。将下采样卷积核和上采样卷积核分别进行处理，基于得到的第一目标卷积核和第二目标卷积核分别计算出下采样图像以及待处理图像对应的模糊图像上每个像素点的灰度值，由于待处理图像对应的模糊图像上每个像素点都能够通过对卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。
在一个实施例中，如图3所示，对下采样卷积核进行处理，得到第一目标卷积核，包括：
S302，获取与下采样卷积核尺寸相同的预设掩模图。
其中，下采样卷积核可以为矩阵的形式表示，下采样卷积核尺寸指的是矩阵的大小。预设掩模图是终端预先设置的掩模图，可以根据实际需求灵活选取。掩模图是用于掩模计算的图像，采用选定的图像对待处理图像进行遮挡来控制对待处理图像的处理区域以及处理过程，其中，选定的图像即为掩模图。掩模图主要用于提取感兴趣区域、屏蔽作用、结构特征提取以及特殊形状图像制作。具体地，终端获取与下采样卷积核尺寸相同的预设掩模图，预设掩模图的尺寸等于下采样卷积核的尺寸。
S304，基于预设掩膜图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核。
其中，下采样卷积核中各个位置处的权重指的是下采样卷积核对应的矩阵中各个位置处的元素值。终端基于预设掩膜图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核，具体地，终端将预设掩膜图与下采样卷积核进行卷积计算，得到的计算结果作为下采样卷积核中各个位置处的调整后的权重，即为第一目标卷积核。第一目标卷积核的尺寸等于下采样卷积核的尺寸。需要说明的是，本申请中下采样图像中每个像素点对应一个预设掩膜图，预设掩膜图的尺寸等于下采样卷积核的尺寸，下采样图像中每个像素点对应的预设掩膜图还可以组成一张大的掩膜图，大的掩膜图的尺寸等于待处理图像的尺寸。例如，下采样卷积核表示为Dx，预设掩模图表示为Ex，第一目标卷积核表示为D1。其中，


本实施例中，通过与下采样卷积核尺寸相同的预设掩模图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核，由于预设掩膜图可以根据实际情况灵活选取，通过选取合适的预设掩膜图，可以对下采样卷积核中各个位置处的权重进行调整。基于得到的第一目标卷积核进行下采样相较于直接采用下采样卷积核进行下采样，在对于有效区域不规则的待处理图像进行图像处理时，能够有效消除待处理图像中无效像素的干扰。并且，在对待处理图像中任意形状的区域做图像处理时，可以通过灵活选择预设掩模图来控制周边区域对处理区域的影响，有利于在无效像素干扰的场景下实现对待处理图像的模糊需求。
在一个实施例中，如图4所示，对下采样卷积核进行处理，得到第一目标卷积核，包括：
S402，根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径。
其中，投影密度函数是反映待处理图像在不同像素点位置处的投影密度的函数。当待处理图像为投影密度不均匀的图像时，待处理图像各个像素点位置处的投影密度可能不同。滤波半径是用来确定第一目标卷积核的尺寸的参数。终端将目标像素点的坐标带入到待处理图像的投影密度函数中，得到目标像素点处的滤波半径。由于待处理图像各个像素点位置处的投影密度可能不同，得到的各个目标像素点处的滤波半径可能不同。
S404，基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。
其中，终端基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。其中，第一目标卷积核的尺寸由滤波半径确定。例如，滤波半径为r，则第一目标卷积核的尺寸为(2r+1)*(2r+1)。第一目标卷积核中各个位置处的权重相应调整，权重的数值与下采样卷积核中的权重的数值相同。例如，滤波半径为2，则第一目标卷积核的尺寸为5*5，下采样卷积核表示为Dx，第一目标卷积核表示为D1：

本实施例中，通过目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径，由于待处理图像各个像素点位置处的投影密度可能不同，得到的各个目标像素点处的滤波半径可能不同。基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核，第一目标卷积核的尺寸由滤波半径确定，各个目标像素点处的第一目标卷积核可能不同。终端通过第一目标卷积核对投影密度不均匀的待处理图像进行图像处理时，通过投影密度函数调整第一目标卷积核的尺寸，有利于在投影密度不均匀的场景下，实现待处理图像的均匀模糊。
在一个实施例中，如图5所示，根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径，包括：
S502，根据下采样卷积核，确定原始滤波半径。
其中，终端根据下采样卷积核，确定原始滤波半径。其中，原始滤波半径是确定下采样卷积核的尺寸的参数。例如，原始滤波半径为r0，下采样卷积核的尺寸为(2r0+1)*(2r0+1)。
S504，获取待处理图像的中心点的坐标。
其中，终端获取待处理图像的中心点的坐标，其中，待处理图像的中心点的坐标为待处理图像的中心点在待处理图像中的相对位置。待处理图像的中心点的坐标包括待处理图像的中心点的横坐标和待处理图像的中心点的纵坐标。
S506，将目标像素点的坐标和中心点的坐标，带入投影密度函数，得到目标像素点处的投影密度值。
其中，目标像素点的坐标包括目标像素点的横坐标和目标像素点的纵坐标。终端将目标像素点的坐标和中心点的坐标，带入投影密度函数，得到目标像素点处的投影密度值。投影密度函数为与目标像素点的坐标相关的函数，将目标像素点的坐标用X表示，投影密度函数f(X)可以为：
其中，x和y分别为目标像素点的横坐标和纵坐标，cx、cy分别为待处理图像的中心点的横坐标和纵坐标。
S508，根据目标像素点处的投影密度值和原始滤波半径，确定目标像素点处的滤波半径。
其中，终端将目标像素点处的投影密度值与原始滤波半径相乘，将得到的乘积作为目标像素点处的滤波半径。原始滤波半径用r0表示，滤波半径用rX表示，则rX＝F(X)·r0，其中，F(X)为将目标像素点的坐标带入到投影密度函数得到的投影函数值。对于投影密度不均匀的待处理图像，各个目标像素点处的投影密度值不同，得到的目标像素点处的滤波半径也不同。例如，预设滤波半径为2，cx为10、cy为5，x为6，y为3，则对于第一目标位置X处的滤波半径rX为1.2。
本实施例中，将目标像素点的坐标和待处理图像的中心点的坐标带入投影密度函数，得到目标像素点处的投影密度值，将目标像素点处的投影密度值和原始滤波半径相乘，得到目标像素点处的滤波半径。由于原始滤波半径是固定值，而对于投影密度不均匀的待处理图像，各个目标像素点处的投影密度值不同，得到的目标像素点处的滤波半径也不同。对于任何投影密度不均匀的待处理图像，都能够通过投影密度函数计算得到目标像素点的滤波半径，基于滤波半径得到第一目标卷积核，有利于在投影密度不均匀的场景下，实现待处理图像的均匀模糊。
在一个实施例中，如图6所示，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，包括：
S602，根据目标像素点的坐标和第一目标卷积核，确定参与卷积运算的第一卷积区域。
其中，第一卷积区域为第一目标卷积核在待处理图像的目标像素点处覆盖的图像区域。第一卷积区域的尺寸等于第一目标卷积核的尺寸。终端将待处理图像上以目标像素点为中心，以第一目标卷积核的尺寸为大小对应区域作为参与卷积运算的第一卷积区域。
S604，从第一卷积区域中查找亚像素点。
其中，亚像素点是指相邻两像素之间细分的较小的点，可以通过插值算法在原像素周围插值得到。常见的插值算法包括双线性插值。终端从第一卷积区域中查找亚像素点，具体地，终端从第一卷积区域中使用双线性插值计算得到的亚像素点的位置。
S606，根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值。
其中，各亚像素点的像素值可以通过插值算法得到。终端根据第一目标卷积核和各亚像素点的像素值，计算下采样图像上对应像素点的灰度值。
本实施例中，通过在待处理图像中第一目标卷积核对应的第一卷积区域中插值亚像素点，基于第一目标卷积核和各亚像素点的像素值计算下采样图像上每个像素点的灰度值，由于下采样图像上每个像素点都能够通过对下采样卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。
在一个实施例中，如图7所示，根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值，包括：
S702，计算各个亚像素点的像素值和第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到第一卷积区域对应的像素值累加和。
其中，终端将第一目标卷积核中的各个位置处的权重值分别与对应亚像素点的像素值相乘，得到所有亚像素点各自对应的乘积，再将得到的乘积相加，得到第一卷积区域对应的像素值累加和。
S704，根据下采样卷积核与第一目标卷积核，确定归一化系数。
其中，终端根据下采样卷积核与第一目标卷积核，确定归一化系数。具体地，终端将下采样卷积核中各个位置处的权重值求和，得到第一求和结果；将第一目标卷积核中各个位置处的权重值求和，得到第二求和结果；将第一求和结果与第二求和结果相除，得到归一化系数。下采样卷积核中的权重表示为dx，预设掩模图中的像素值表示为mx，则归一化系数w为：
S706，根据第一卷积区域对应的像素值累加和与归一化系数，确定像素点的灰度值。
其中，终端将第一卷积区域对应的像素值累计和与归一化系数相乘，得到下采样图像上对应像素点的灰度值。具体地，亚像素点的像素值表示为Ix，下采样卷积核中的权重表示为dx，预设掩模图中的像素值表示为mx，第一卷积区域表示为Ω，下采样图像上对应像素点的灰度值表示为q，则灰度值计算公式为：
例如，目标像素点的坐标为X＝(3.5，3.5)，目标像素点对应的亚像素点xi∈Ω＝[(1.5,1.5),(5.5,1.5),(3.5,3.5),(1.5,5.5),(5.5,5.5)]，亚像素点对应的像素值 预设掩模图为将Ix、dx和mX带入灰度值计算公式，得到的下采样图像上对应像素点的灰度值为20。若不采用归一化系数，得到的下采样图像上对应像素点的灰度值为10，则下采样图像的亮度将比待处理图像的亮度低很多，而采用归一化系数，得到的下采样图像对应像素点的灰度值为20，则下采样图像的亮度与待处理图像的亮度较为接近，有利于得到更有的图像处理效果。
本实施例中，通过计算各个亚像素点的像素值和第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到第一卷积区域对应的像素值累加和，将基于下采样卷积核与第一目标卷积核确定的归一化系数和第一卷积区域对应的像素值累加相乘，确定像素点的灰度值。其中，归一化系数的作用能够保持下采样前后，待处理图像的亮度不发生剧烈变化。由于下采样图像上每个像素点的灰度值都能够通过对下采样卷积核的处理计算得到，能够保证满足不同场景下的图像模糊需求。
在一个实施例中，对下采样卷积核进行处理，得到第一目标卷积核，包括：根据目标像素点的坐标以及下采样卷积核，确定参与卷积运算的第二卷积区域；基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向，M大于1；基于M个划分方向，对下采样卷积核进行调整，得到M个目标卷积核；相应的，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，包括：根据目标像素点的坐标和各个目标卷积核，计算得到M个灰度值；将M个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
其中，第二卷积区域为第一目标卷积核在待处理图像的目标像素点处覆盖的图像区域。第二卷积区域的尺寸等于第一目标卷积核的尺寸。终端将待处理图像上以目标像素点为中心，以第一目标卷积核的尺寸为大小对应区域作为参与卷积运算的第二卷积区域。终端基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向。具体地，终端以第二卷积区域内各个目标像素点的坐标为中心，在第二卷积区域中选取对应数量的亚像素点，确定M个划分方向中的一个划分方向。各个划分方向对应的亚像素点的数量可以不同，M大于1。终端基于M个划分方向，对下采样卷积核进行调整，得到M个目标卷积核，具体地，终端对下采样卷积核按照M个划分方向选取各个划分方向对应的权重，得到M个目标卷积核。终端根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值。具体地，针对每个目标像素点，终端将目标像素点的坐标和各个目标卷积核带入到灰度值计算公式，计算得到M个灰度值。终端将M个灰度值分别与目标像素点处的灰度值分别进行比较，获得比较结果，根据比较结果，将M个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
本实施例中，通过第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向，基于M个划分方向调整下采样卷积核，将目标像素点的坐标和各个目标卷积核带入到灰度值计算公式，从M个划分方向得到的灰度值中选取与目标像素点处灰度值最接近的灰度值作为像素点的灰度值。这种从M个划分方向中选取与目标像素点处灰度值最接近的灰度值作为像素点的灰度值的划分方向方法，能够保证下采样图像的边缘更加清晰。并且对于待处理图像包含无效像素的场景，在无效像素处对应的预设掩膜图能够根据实际情况灵活调整，对待处理图像进行图像处理时，能够有效消除待处理图像中无效像素的干扰，能够满足待处理图像包含无效像素的场景下的图像模糊需求。
为详细说明本方案中图像处理的方法及效果，下面以一个最详细实施例进行说明：
基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标。对下采样卷积核进行处理，得到第一目标卷积核。对于下采样图像上每个像素点，根据目标像素点的坐标和第一目标卷积核，确定参与卷积运算的第一卷积区域，从第一卷积区域中查找亚像素点。计算各个亚像素点的像素值和第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到第一卷积区域对应的像素值累加和，根据下采样卷积核与第一目标卷积核，确定归一化系数，根据第一卷积区域对应的像素值累加和与归一化系数，确定像素点的灰度值。基于下采样图像上每个像素点的灰度值，得到下采样图像。由于下采样图像上每个像素点都能够通过对下采样卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。下采样卷积核中的权重表示为dx，预设掩模图中的像素值表示为mx，则归一化系数w为：
亚像素点的像素值表示为Ix，第一卷积区域表示为Ω，下采样图像上对应像素点的灰度值表示为q，则灰度值计算公式为：
例如，目标像素点的坐标为X＝(3.5，3.5)，目标像素点对应的亚像素点xi∈Ω＝[(1.5,1.5),(5.5,1.5),(3.5,3.5),(1.5,5.5),(5.5,5.5)]，亚像素点对应的像素值 预设掩模图为将Ix、dx和mX带入灰度值计算公式，得到的下采样图像上对应像素点的灰度值为20。若不采用归一化系数，得到的下采样图像上对应像素点的灰度值为10，则下采样图像的亮度将比待处理图像的亮度低很多，而采用归一化系数，得到的下采样图像对应像素点的灰度值为20，则下采样图像的亮度与待处理图像的亮度较为接近，有利于得到更有的图像处理效果。
对上采样卷积核进行处理，得到第二目标卷积核。基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像。对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新。基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
针对待处理图像中画面被物体遮挡出现无效像素的场景：
对下采样卷积核进行处理，得到第一目标卷积核，具体地，获取与下采样卷积核尺寸相同的预设掩模图，基于预设掩膜图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核。例如，下采样卷积核表示为Dx，预设掩模图表示为Ex，第一目标卷积核表示为D1。其中，


对上采样卷积核进行处理，得到第二目标卷积核，具体地，获取与上采样卷积核尺寸相同的掩模图，基于掩膜图，对上采样卷积核中各个位置处的权重进行调整，得到第二目标卷积核。例如，上采样卷积核表示为Ds，掩模图表示为Es，第二目标卷积核表示为D2。其中，


如图8所示为待处理图像未经过预设掩模图处理和经过预设掩模图处理的对比示意图，其中，(a)为待处理图像，(b)为预设掩模图，(c)为采用传统图像处理方法对待处理图像进行图像处理后的图像，由于未经过预设掩模图(b)的处理，(c)中图像的边缘渗入有黑色光晕，(d)为待处理图像对应的模糊图像，即经过预设掩模图处理的图像，传统图像处理后的图像(c)和本申请中图像处理方法得到的图像(d)对比可知，(c)中图像边缘存在黑色光晕渗入，(d)中图像边缘无黑色光晕渗入，因此，采用本申请方案还能根据实际情况灵活调整预设掩模图，从而灵活控制待处理图像中需要模糊处理的区域，还能够消除模糊处理的区域中的干扰像素。由于预设掩膜图和掩膜图可以根据实际情况灵活选取，通过选取合适的预设掩膜图和掩膜图，可以对下采样卷积核以及上采样卷积核中各个位置处的权重进行调整。基于得到的第一目标卷积核进行下采样相较于直接采用下采样卷积核进行下采样，以及基于得到的第二目标卷积核进行上采样相较于直接采用上采样卷积核进行上采样，在对画面被物体遮挡出现无效像素的待处理图像进行图像处理时，能够有效消除待处理图像中无效像素的干扰。
针对投影密度不均匀的待处理图像进行图像处理的场景，例如，待处理图像为鱼眼相片、球形全景图像或者球形全景的平面展开图像：
对下采样卷积核进行处理，得到第一目标卷积核，具体地，根据下采样卷积核，确定原始滤波半径，获取待处理图像的中心点的坐标，将目标像素点的坐标和中心点的坐标，带入投影密度函数，得到目标像素点处的投影密度值，根据目标像素点处的投影密度值和原始滤波半径，确定目标像素点处的滤波半径，基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。
对上采样卷积核进行处理，得到第二目标卷积核，具体地，根据上采样卷积核，确定上采样卷积核对应的初始滤波半径，获取参考图像的中心点的坐标，将参考图像上各像素点的坐标和中心点的坐标，带入投影密度函数，得到参考图像上各像素点处的投影密度值，根据参考图像上各像素点处的投影密度值和初始滤波半径，确定参考图像上各像素点处的滤波半径，基于滤波半径，对上采样卷积核的大小进行调整，得到第二目标卷积核。
当待处理图像为投影密度不均匀的图像时，待处理图像各个像素点位置处的投影密度可能不同，得到的各个目标像素点处的滤波半径可能不同。例如， 原始滤波半径为r0，下采样卷积核的尺寸为(2r0+1)*(2r0+1)。投影密度函数为与目标像素点的坐标相关的函数，将目标像素点的坐标用X表示，原始滤波半径用r0表示，滤波半径用rX表示，投影密度函数f(X)可以为：
其中，x和y分别为目标像素点的横坐标和纵坐标，cx、cy分别为待处理图像的中心点的横坐标和纵坐标。
rX＝F(X)·r0
其中，F(X)为将目标像素点的坐标带入到投影密度函数得到的投影函数值。
例如，预设滤波半径为2，cx为10、cy为5，x为6，y为3，则对于第一目标位置X处的滤波半径rX为1.2。由于原始滤波半径是固定值，而对于投影密度不均匀的待处理图像，各个目标像素点处的投影密度值不同，得到的目标像素点处的滤波半径也不同。对于任何投影密度不均匀的待处理图像，都能够通过投影密度函数计算得到目标像素点的滤波半径以及参考图像上像素点的滤波半径，基于两种滤波半径分别得到第一目标卷积核和第二目标卷积核，有利于在投影密度不均匀的场景下，实现待处理图像的均匀模糊。
针对需要保持边缘清晰的场景：
对下采样卷积核进行处理，得到第一目标卷积核，具体地，根据目标像素点的坐标以及下采样卷积核，确定参与卷积运算的第二卷积区域，基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向，M大于1，基于M个划分方向，对下采样卷积核进行调整，得到M个目标卷积核。相应的，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，具体地，根据目标像素点的坐标和各个目标卷积核，计算得到M个灰度值，将M个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。如图9所示为第二卷积区域目标像素点的亚像素点的位置示意图，其中，图中的C、NW、NE、SW、SE分别代表目标像素点的亚像素点的位置，基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定8个划分方向，8个划分方向分为：
第一划分方向：NW-NE-C
第二划分方向：SW-SE-C
第三划分方向：NW-SW-C
第四划分方向：NE-SE-C
第五划分方向：NW-C
第六划分方向：NE-C
第七划分方向：SW-C
第八划分方向：SE-C
基于8个划分方向，对下采样卷积核进行调整，得到8个目标卷积核。针对每个目标像素点，终端将目标像素点的坐标和各个目标卷积核带入到灰度值计算公式，计算得到8个灰度值。计算过程中，在8个划分方向上，首先计算出重复区域：NW-C、NE-C、SW-C、SE-C，再在重复区域的基础上加入相应的值，有利于提高计算效率。终端将8个灰度值分别与目标像素点处的灰度值分别进行比较，获得比较结果，根据比较结果，将8个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
对上采样卷积核进行处理，得到第二目标卷积核，具体地，根据参考图像上的每个像素点的坐标以及上采样卷积核，确定参与卷积运算的第三卷积区域，基于第三卷积区域内每个亚像素点的坐标和参考图像上像素点的坐标，确定M个划分方向，M大于1，基于M个划分方向，对上采样卷积核进行调整，得到M个目标卷积核。相应的，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新，具体地，根据像素点的坐标以及和各个目标卷积核，计算得到M个灰度值，将M个灰度值中与目标像素点处灰度值最接近的灰度值，对像素点的灰度值进行更新。
如图10所示为第三卷积区域参考图像上像素点的亚像素点的位置示意图，其中，图中的C、N、W、S、E、NW、NE、SW、SE分别代表参考图像上像素点周围的亚像素点的位置，根据第三卷积区域内每个亚像素点的坐标和参考图像上像素点的坐标，确定8个划分方向，8个划分方向分为：
第一划分方向：W-NW-N-NE-E
第二划分方向：W-SW-S-SE-E
第三划分方向：N-NW-W-SW-S
第四划分方向：N-NE-E-SE-S
第五划分方向：W-NW-N
第六划分方向：N-NE-E
第七划分方向：W-SW-S
第八划分方向：S-SE-E
基于8个划分方向，对上采样卷积核进行调整，得到8个目标卷积核。针对每个目标像素点，终端将参考图像上像素点的坐标和各个目标卷积核带入到灰度值计算公式，计算得到8个灰度值。计算过程中，在8个划分方向上，首先计算出重复区域：NW-C、NE-C、SW-C、SE-C，再在重复区域的基础上加入相应的值，有利于提高计算效率。终端将8个灰度值分别与参考图像上对应像素点处的灰度值分别进行比较，获得比较结果，根据比较结果，将8个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
这种从M个划分方向中选取与目标像素点处灰度值最接近的灰度值作为像素点的灰度值的划分方向方法，能够保证下采样图像以及待处理图像的模糊图像的边缘更加清晰。并且对于待处理图像包含无效像素的场景，在无效像素处对应的预设掩膜图能够根据实际情况灵活调整，对待处理图像进行图像处理时，能够有效消除待处理图像中无效像素的干扰，能够满足待处理图像包含无效像素的场景下的图像模糊需求。
针对提升图像局部对比度场景：
将待处理图像中各个像素点的灰度值分别与待处理图像对应的模糊图像上对应像素点的灰度值相减，获得待处理图像中每个像素点对应的灰度差值，将待处理图像中灰度差值大于0的像素点对应的亮度值增大，将待处理图像中灰度差值小于0的像素点对应的亮度值减小，将待处理图像中灰度差值小于0的像素点对应的亮度值保持不变，得到亮度值变化后的图像，能够提升待处理图像与模糊图像的局部对比度。如图11所示为对球形全景图像进行局部对比度提升的前后对比图，其中，(a)表示待处理图像为球形全景图像，(b)为将(a)指示的待处理图像进行局部对比度提升后的图像，(b)比(a)局部对比度更明显。如图12所示为球形全景的平面展开图像进行局部对比度提升的前后对比图，其中，图12中的(a)表示待处理图像为球形全景的平面展开图像，图12 中的(b)为对图12中的(a)指示的待处理图像进行局部对比度提升后的图像，图12中的(b)比图12中的(a)局部对比度更明显。
针对待处理图像为鱼眼相机图像的场景，采用不同的滤波半径r、不同的迭代次数k以及是否进行边缘保持分别对鱼眼相机图像进行图像处理，其中，迭代次数表示对待处理图像进行下采样以及上采样的次数，边缘保持指的是划分M个方向，在M个划分方向中选取与目标像素点处的灰度值最接近的划分方向的方法。针对图11中的(a)指示的待处理图像进行了多组实验，其中图11中的(a)为鱼眼相机拍摄的图像。实验效果如图13所示，其中，(a)为采用r＝1、k＝2以及边缘保持时待处理图像对应的模糊图像；(b)为采用r＝1、k＝3以及边缘保持时待处理图像对应的模糊图像；(c)为采用r＝1、k＝4以及边缘保持时待处理图像对应的模糊图像；(d)为采用r＝1、k＝4以及未边缘保持时待处理图像对应的模糊图像；(e)为采用r＝5、k＝2以及边缘保持时待处理图像对应的模糊图像；(f)为采用r＝5、k＝3以及边缘保持时待处理图像对应的模糊图像；(g)为采用r＝5、k＝4以及边缘保持时待处理图像对应的模糊图像；(h)为采用r＝5、k＝4以及未边缘保持时待处理图像对应的模糊图像，8组实验可以看出，迭代次数越多，图像模糊程度越高，采用边缘保持比未采用边缘保持，图像的边缘更清晰。
上述图像处理方法，通过下采样卷积核对待处理图像进行下采样处理得到下采样图像，通过上采样卷积核对下采样图像进行上采样得到待处理图像对应的模糊图像，能够实现待处理图像的模糊化。同时，下采样图像的尺寸小于待处理图像的尺寸，基于下采样图像进行上采样能够减少计算量，预设采样系数能够保证经过下采样以及上采样后，待处理图像对应的模糊图像与待处理图像的尺寸相同。将下采样卷积核和上采样卷积核分别进行处理，基于得到的第一目标卷积核和第二目标卷积核分别计算出下采样图像以及待处理图像对应的模糊图像上每个像素点的灰度值，由于待处理图像对应的模糊图像上每个像素点都能够通过对卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。
应该理解的是，虽然如上的各实施例所涉及的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。 除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，如上的各实施例所涉及的流程图中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。需要说明的是，本申请中图像处理方法中的下采样步骤以及上采样步骤都可以迭代N次，其中，下采样步骤的迭代次数等于上采样步骤的迭代次数，N大于或者等于1。
基于同样的发明构思，本申请实施例还提供了一种用于实现上述所涉及的图像处理方法的图像处理装置。该装置所提供的解决问题的实现方案与上述方法中所记载的实现方案相似，故下面所提供的一个或多个图像处理装置实施例中的具体限定可以参见上文中对于图像处理方法的限定，在此不再赘述。
在一个实施例中，如图14所示，提供了一种图像处理装置100，包括：像素位置获取模块110、第一卷积核处理模块120、下采样模块130、第二卷积核处理模块140和上采样模块150，其中：
像素位置获取模块110，用于基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标。
第一卷积核处理模块120，用于对下采样卷积核进行处理，得到第一目标卷积核。
下采样模块130，用于对于下采样图像上每个像素点，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像。
第二卷积核处理模块140，用于对上采样卷积核进行处理，得到第二目标卷积核。
上采样模块150，用于基于下采样图像和预设采样系数，得到与待处理图像尺寸相同的参考图像，对于参考图像上的每个像素点，根据像素点的坐标以及第二目标卷积核，对像素点的灰度值进行更新；基于参考图像上各个像素点的更新后的灰度值，得到待处理图像对应的模糊图像。
上述图像处理装置，通过下采样卷积核对待处理图像进行下采样处理得到下采样图像，通过上采样卷积核对下采样图像进行上采样得到待处理图像对应的模糊图像，能够实现待处理图像的模糊化。同时，下采样图像的尺寸小于待处理图像的尺寸，基于下采样图像进行上采样能够减少计算量，预设采样系数能够保证经过下采样以及上采样后，待处理图像对应的模糊图像与待处理图像的尺寸相同。将下采样卷积核和上采样卷积核分别进行处理，基于得到的第一目标卷积核和第二目标卷积核分别计算出下采样图像以及待处理图像对应的模糊图像上每个像素点的灰度值，由于待处理图像对应的模糊图像上每个像素点都能够通过对卷积核的处理，计算得到各自对应的灰度值，能够保证满足不同场景下的图像模糊需求。
在一个实施例中，在对下采样卷积核进行处理，得到第一目标卷积核方面，第一卷积核处理模块120还用于：获取与下采样卷积核尺寸相同的预设掩模图；基于预设掩膜图，对下采样卷积核中各个位置处的权重进行调整，得到第一目标卷积核。
在一个实施例中，在对下采样卷积核进行处理，得到第一目标卷积核方面，第一卷积核处理模块120还用于：根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径；基于滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。
在一个实施例中，在根据目标像素点的坐标和待处理图像的投影密度函数，确定目标像素点处的滤波半径方面，第一卷积核处理模块120还用于：根据下采样卷积核，确定原始滤波半径；获取待处理图像的中心点的坐标；将目标像素点的坐标和中心点的坐标，带入投影密度函数，得到目标像素点处的投影密度值；根据目标像素点处的投影密度值和原始滤波半径，确定目标像素点处的滤波半径。
在一个实施例中，在根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值方面，下采样模块130还用于：根据目标像素点的坐标和第一目标卷积核，确定参与卷积运算的第一卷积区域；从第一卷积区域中查找亚像素点；根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值。
在一个实施例中，在根据第一目标卷积核和各亚像素点的像素值，计算像素点的灰度值方面，下采样模块130还用于：计算各个亚像素点的像素值和第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到第一卷积区域对应的像素值累加和；根据下采样卷积核与第一目标卷积核，确定归一化系数；根据第一卷积区域对应的像素值累加和与归一化系数，确定像素点的灰度值。
在一个实施例中，在对下采样卷积核进行处理，得到第一目标卷积核方面，第一卷积核处理模块120还用于：根据目标像素点的坐标以及下采样卷积核，确定参与卷积运算的第二卷积区域；基于第二卷积区域内每个亚像素点的坐标和目标像素点的坐标，确定M个划分方向，M大于1；基于M个划分方向，对下采样卷积核进行调整，得到M个目标卷积核；相应的，根据像素点映射至待处理图像得到的目标像素点的坐标以及第一目标卷积核，确定像素点的灰度值，包括：根据目标像素点的坐标和各个目标卷积核，计算得到M个灰度值；将M个灰度值中与目标像素点处灰度值最接近的灰度值，作为像素点的灰度值。
上述图像处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图15所示。该计算机设备包括处理器、存储器、输入/输出接口(Input/Output，简称I/O)和通信接口。其中，处理器、存储器和输入/输出接口通过系统总线连接，通信接口通过输入/输出接口连接到系统总线。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质和内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储预设采样系数、待处理图像、下采样卷积核、上采样卷积核、第一目标卷积核、第二目标卷积核、下采样图像上每个像素点的灰度值、下采样图像、参考图像上各个像素点的更新后的灰度值以及待处理图像对应的模糊图像。该计算机设备的输入/输出接口用于处理器与外部设备之间交换信息。该计算机设备的通信接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像处理方法。
本领域技术人员可以理解，图15中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现上述各方法实施例中的步骤。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述各方法实施例中的步骤。
在一个实施例中，提供了一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现上述各方法实施例中的步骤。
需要说明的是，本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存、光存储器、高密度嵌入式非易失性存储器、阻变存储器(ReRAM)、磁变存储器(Magnetoresistive Random Access Memory，MRAM)、铁电存储器(Ferroelectric Random Access Memory，FRAM)、相变存储器(Phase Change Memory，PCM)、石墨烯存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器等。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory， DRAM)等。本申请所提供的各实施例中所涉及的数据库可包括关系型数据库和非关系型数据库中至少一种。非关系型数据库可包括基于区块链的分布式数据库等，不限于此。本申请所提供的各实施例中所涉及的处理器可为通用处理器、中央处理器、图形处理器、数字信号处理器、可编程逻辑器、基于量子计算的数据处理逻辑器等，不限于此。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请的保护范围应以所附权利要求为准。

Claims (11)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
    对下采样卷积核进行处理，得到第一目标卷积核；
    对于所述下采样图像上每个像素点，根据所述像素点映射至所述待处理图像得到的目标像素点的坐标以及所述第一目标卷积核，确定所述像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
    对上采样卷积核进行处理，得到第二目标卷积核；
    基于所述下采样图像和所述预设采样系数，得到与所述待处理图像尺寸相同的参考图像，对于所述参考图像上的每个像素点，根据所述像素点的坐标以及所述第二目标卷积核，对所述像素点的灰度值进行更新；基于所述参考图像上各个像素点的更新后的灰度值，得到所述待处理图像对应的模糊图像。
  2. 根据权利要求1所述的方法，其特征在于，所述对下采样卷积核进行处理，得到第一目标卷积核，包括：
    获取与下采样卷积核尺寸相同的预设掩模图；
    基于所述预设掩膜图，对所述下采样卷积核中各个位置处的权重进行调整，得到所述第一目标卷积核。
  3. 根据权利要求1所述的方法，其特征在于，所述对下采样卷积核进行处理，得到第一目标卷积核，包括：
    根据所述目标像素点的坐标和所述待处理图像的投影密度函数，确定所述目标像素点处的滤波半径；
    基于所述滤波半径，对下采样卷积核的大小进行调整，得到第一目标卷积核。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述目标像素点的坐标和所述待处理图像的投影密度函数，确定所述目标像素点处的滤波半径，包括：
    根据所述下采样卷积核，确定原始滤波半径；
    获取所述待处理图像的中心点的坐标；
    将所述目标像素点的坐标和所述中心点的坐标，带入所述投影密度函数，得到所述目标像素点处的投影密度值；
    根据所述目标像素点处的投影密度值和所述原始滤波半径，确定所述目标像素点处的滤波半径。
  5. 根据权利要求1所述的方法，其特征在于，所述根据所述像素点映射至所述待处理图像得到的目标像素点的坐标以及所述第一目标卷积核，确定所述像素点的灰度值，包括：
    根据所述目标像素点的坐标和所述第一目标卷积核，确定参与卷积运算的第一卷积区域；
    从所述第一卷积区域中查找亚像素点；
    根据所述第一目标卷积核和各亚像素点的像素值，计算所述像素点的灰度值。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述第一目标卷积核和各亚像素点的像素值，计算所述像素点的灰度值，包括：
    计算各个亚像素点的像素值和所述第一目标卷积核中对应权重值的乘积，并将所有亚像素点各自对应的乘积相加，得到所述第一卷积区域对应的像素值累加和；
    根据所述下采样卷积核与所述第一目标卷积核，确定归一化系数；
    根据所述第一卷积区域对应的像素值累加和与所述归一化系数，确定所述像素点的灰度值。
  7. 根据权利要求1所述的方法，其特征在于，所述对下采样卷积核进行处理，得到第一目标卷积核，包括：
    根据所述目标像素点的坐标以及下采样卷积核，确定参与卷积运算的第二卷积区域；
    基于所述第二卷积区域内每个亚像素点的坐标和所述目标像素点的坐标，确定M个划分方向，所述M大于1；
    基于所述M个划分方向，对所述下采样卷积核进行调整，得到M个目标卷积核；
    相应的，所述根据所述像素点映射至所述待处理图像得到的目标像素点的坐标以及所述第一目标卷积核，确定所述像素点的灰度值，包括：
    根据所述目标像素点的坐标和各个目标卷积核，计算得到M个灰度值；
    将所述M个灰度值中与所述目标像素点处灰度值最接近的灰度值，作为所述像素点的灰度值。
  8. 一种图像处理装置，其特征在于，所述装置包括：
    像素位置获取模块，用于基于预设采样系数和待处理图像，确定下采样图像上每个像素点的坐标；
    第一卷积核处理模块，用于对下采样卷积核进行处理，得到第一目标卷积核；
    下采样模块，用于对于所述下采样图像上每个像素点，根据所述像素点映射至所述待处理图像得到的目标像素点的坐标以及所述第一目标卷积核，确定所述像素点的灰度值；基于下采样图像上每个像素点的灰度值，得到下采样图像；
    第二卷积核处理模块，用于对上采样卷积核进行处理，得到第二目标卷积核；
    上采样模块，用于基于所述下采样图像和所述预设采样系数，得到与所述待处理图像尺寸相同的参考图像，对于所述参考图像上的每个像素点，根据所述像素点的坐标以及所述第二目标卷积核，对所述像素点的灰度值进行更新；基于所述参考图像上各个像素点的更新后的灰度值，得到所述待处理图像对应的模糊图像。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至7中任一项所述的方法的步骤。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至7中任一项所述的方法的步骤。
  11. 一种计算机程序产品，包括计算机程序，其特征在于，该计算机程序被处理器执行时实现权利要求1至7中任一项所述的方法的步骤。
PCT/CN2023/116829 2022-09-06 2023-09-04 图像处理方法、装置、计算机设备和存储介质 WO2024051657A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211083809.6 2022-09-06
CN202211083809.6A CN117710189A (zh) 2022-09-06 2022-09-06 图像处理方法、装置、计算机设备和存储介质

Publications (1)

Publication Number Publication Date
WO2024051657A1 true WO2024051657A1 (zh) 2024-03-14

Family

ID=90146638

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/116829 WO2024051657A1 (zh) 2022-09-06 2023-09-04 图像处理方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN117710189A (zh)
WO (1) WO2024051657A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160063675A1 (en) * 2014-08-27 2016-03-03 Shenzhen China Star Optoelectronics Technology Co. Ltd. Downsampling based display method and apparatus
CN113096021A (zh) * 2019-12-23 2021-07-09 中国移动通信有限公司研究院 一种图像处理方法、装置、设备及存储介质
CN113936163A (zh) * 2020-07-14 2022-01-14 武汉Tcl集团工业研究院有限公司 一种图像处理方法、终端以及存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160063675A1 (en) * 2014-08-27 2016-03-03 Shenzhen China Star Optoelectronics Technology Co. Ltd. Downsampling based display method and apparatus
CN113096021A (zh) * 2019-12-23 2021-07-09 中国移动通信有限公司研究院 一种图像处理方法、装置、设备及存储介质
CN113936163A (zh) * 2020-07-14 2022-01-14 武汉Tcl集团工业研究院有限公司 一种图像处理方法、终端以及存储介质

Also Published As

Publication number Publication date
CN117710189A (zh) 2024-03-15

Similar Documents

Publication Publication Date Title
Wang et al. Real-esrgan: Training real-world blind super-resolution with pure synthetic data
WO2021088473A1 (en) Image super-resolution reconstruction method, image super-resolution reconstruction apparatus, and computer-readable storage medium
CN109389556B (zh) 一种多尺度空洞卷积神经网络超分辨率重构方法及装置
US7889949B2 (en) Joint bilateral upsampling
US8547389B2 (en) Capturing image structure detail from a first image and color from a second image
CN111563552B (zh) 图像融合方法以及相关设备、装置
WO2018082185A1 (zh) 图像处理方法和装置
WO2011126774A2 (en) Generation of multi-resolution image pyramids
US10810707B2 (en) Depth-of-field blur effects generating techniques
JP2003274157A (ja) 画像処理装置、画像処理方法、画像処理プログラムおよび画像処理プログラムを記録したコンピュータ読取可能な記録媒体
CN113222866B (zh) 灰度图像增强方法、计算机可读介质及计算机系统
CN111681165A (zh) 图像处理方法、装置、计算机设备和计算机可读存储介质
CN106846250B (zh) 一种基于多尺度滤波的超分辨率重建方法
CN113744142B (zh) 图像修复方法、电子设备及存储介质
WO2024041027A1 (zh) 全景图像处理方法、计算机设备和存储介质
WO2024051657A1 (zh) 图像处理方法、装置、计算机设备和存储介质
CN117152330A (zh) 一种基于深度学习的点云3d模型贴图方法和装置
CN111724292A (zh) 图像处理方法、装置、设备以及计算机可读介质
US20230098437A1 (en) Reference-Based Super-Resolution for Image and Video Enhancement
CN116071279A (zh) 图像处理方法、装置、计算机设备和存储介质
CN114118367B (zh) 增量式神经辐射场构建的方法及设备
US20220130012A1 (en) Demosaicing method and demosaicing device
CN115760658A (zh) 图像处理方法、装置、存储介质及电子设备
CN113570531A (zh) 图像处理方法、装置、电子设备和计算机可读存储介质
WO2020062899A1 (zh) 通过前景背景像素对和灰度信息获得透明度遮罩的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23862351

Country of ref document: EP

Kind code of ref document: A1