WO2022247630A1

WO2022247630A1 - Image processing method and apparatus, electronic device and storage medium

Info

Publication number: WO2022247630A1
Application number: PCT/CN2022/092072
Authority: WO
Inventors: 陈秋伯; 陈铭津
Original assignee: 北京字跳网络技术有限公司
Priority date: 2021-05-28
Filing date: 2022-05-10
Publication date: 2022-12-01
Also published as: CN115409696A

Abstract

An image processing method and apparatus, an electronic device and a storage medium. The method comprises: obtaining a plurality of initial image frames, the plurality of initial image frames being a plurality of different image frames photographed for a same scene, and the plurality of initial image frames comprising a reference image and a non-reference image; calculating an optical flow from the non-reference image to the reference image; determining, on the basis of the optical flow from the non-reference image to the reference image, a reference pixel point in the non-reference image corresponding to a reference pixel point in the reference image; and by interpolating the reference pixel point in the non-reference image and the reference pixel point in the reference image, performing pixel fusion on the non-reference image and the reference image to generate a target image. Original pixel information of each image frame in multiple image frames to be processed is fully utilized in the process of generating the target image, such that the loss of part image information is avoided, thereby improving the imaging effect and image details of the finally generated target image.

Description

Image processing method, device, electronic device and storage medium

This application claims the priority of the Chinese patent application with the application number 202110590685.X and the application name "image processing method, device, electronic equipment and storage medium" submitted to the China Patent Office on May 28, 2021, the entire content of which is passed References are incorporated in this application.

technical field

Embodiments of the present disclosure relate to the technical field of image processing, and in particular, to an image processing method, device, electronic equipment, storage medium, computer program product, and computer program.

Background technique

With the development of electronic imaging and image processing technology, the camera function based on smart terminals is becoming more and more powerful, but at the same time, users have higher and higher requirements for the quality of photos taken, which is limited by the size of photosensitive elements, lenses and modules, etc. Due to the limitations of hardware units in terms of performance, cost, and deployment space, the quality of photos taken by smartphones through the native camera system is poor, resulting in poor detail recognition and low signal-to-noise ratio in the photos taken.

At present, the solution in the prior art is to optimize the target photo through multiple photos to generate a high-resolution image after obtaining the target photo taken by the native camera, so as to improve the image quality. However, the solutions in the prior art still have the problems of poor image imaging effect and low detail recognition due to loss of image information.

Contents of the invention

Embodiments of the present disclosure provide an image processing method, device, electronic equipment, storage medium, computer program product, and computer program to overcome the problems of poor imaging effect and low image detail recognition when generating high-resolution images.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

Acquiring multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include a reference image and a non-reference image; calculating from the non-reference image to the The optical flow of the reference image; based on the optical flow from the non-reference image to the reference image, determine the reference pixel in the non-reference image corresponding to the reference pixel in the reference image; by adding the The reference pixel in the non-reference image is interpolated with the reference pixel in the reference image, and the non-reference image and the reference image are pixel fused to generate a target image.

In a second aspect, an embodiment of the present disclosure provides an image processing device, including:

An acquisition unit, configured to acquire multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include reference images and non-reference images;

a first determining unit, configured to calculate an optical flow from the non-reference image to the reference image;

A second determining unit, based on the optical flow from the non-reference image to the reference image, to determine a reference pixel in the non-reference image corresponding to a reference pixel in the reference image;

A generating unit, configured to perform interpolation calculation on the reference pixel in the non-reference image and the reference pixel in the reference image, perform pixel fusion on the non-reference image and the reference image, and generate a target image.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

the memory stores computer-executable instructions;

The at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the image processing method described in the above first aspect and various possible designs of the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the above first aspect and the first Aspects of various possible designs of the described image processing method.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the image processing method described in the above first aspect and various possible designs of the first aspect is implemented.

In a sixth aspect, an embodiment of the present disclosure provides a computer program. When the computer program is executed by a processor, the image processing method described in the above first aspect and various possible designs of the first aspect is implemented.

The image processing method, device, electronic equipment, storage medium, computer program product, and computer program provided in this embodiment obtain multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images shot for the same scene, The multi-frame initial image includes a reference image and a non-reference image; calculate an optical flow from the non-reference image to the reference image; determine the optical flow from the non-reference image to the reference image based on the optical flow from the non-reference image to the reference image the reference pixel in the non-reference image corresponding to the reference pixel in the reference image; by interpolating the reference pixel in the non-reference image with the reference pixel in the reference image, the The non-reference image is pixel fused with the reference image to generate the target image, because in the process of generating the target image, each interpolation pixel in the target image uses the corresponding reference pixel and non-reference pixel in the reference image The corresponding reference pixels in the benchmark image are obtained by pixel fusion. Therefore, the original pixel information of each frame image in the multi-frame image to be processed is fully utilized, and the problems of partial image information loss and error introduction are avoided, and the final image quality is improved. The imaging effect and image details of the generated target image.

Description of drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present disclosure, for those skilled in the art, other drawings can also be obtained according to these drawings on the premise of not paying creative efforts.

FIG. 1 is a diagram of an application scenario provided by an embodiment of the present disclosure;

Fig. 2 is a schematic diagram of a method for generating a high-resolution image in the prior art;

FIG. 3 is a first schematic flow diagram of an image processing method provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an interpolation calculation for a reference image provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of determining a reference pixel point provided by an embodiment of the present disclosure;

FIG. 6 is a second schematic flow diagram of an image processing method provided by an embodiment of the present disclosure;

Fig. 7 is a flow chart of the process of determining the reference point coordinates corresponding to the first interpolation pixel point;

FIG. 8 is a schematic diagram of a first interpolation pixel point and its corresponding edge pixel point provided by an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a first preset range provided by an embodiment of the present disclosure;

FIG. 10 is a schematic diagram of a second preset range provided by an embodiment of the present disclosure;

FIG. 11 is a structural block diagram of an image processing device provided by an embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments It is a part of the embodiments of the present disclosure, but not all of them. Based on the embodiments in the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present disclosure.

FIG. 1 is a diagram of an application scenario provided by an embodiment of the present disclosure. The image processing method provided by an embodiment of the present disclosure can be applied to a smart terminal, such as a smart phone. Referring to FIG. Use a smart phone to generate multiple frames of low-resolution images through one shooting operation. During this process, the smart phone prompts the user not to move the phone by displaying the prompt message "Resolution is being improved, please do not move" so as to facilitate the collection of more images. Frame continuously changing images of the same scene. Afterwards, the smartphone processes the low-resolution image through a built-in program or software to generate a frame of high-resolution image and display it, so that users can get photos with clearer images and better image details.

FIG. 2 is a schematic diagram of a method for generating a high-resolution image in the prior art. As shown in FIG. 2 , in the prior art, in the process of processing multiple frames of low-resolution images to generate a high-resolution image, usually The method used is to use one frame of images in the multi-frame images as a reference frame, calculate the optical flow from the reference frame to the non-reference frame, and then according to the optical flow calculation results, the images of other non-reference frames are transformed by pseudo-warp Align to the reference frame and perform interpolation fusion to generate an interpolated image. However, in the process of aligning other frame images to the reference frame, since the algorithm of pseudo-transformation itself will introduce certain errors, after each other frame image is aligned, each other frame image is no longer the original image, so there are original sub-images. The problem of pixel information loss leads to the use of aligned multi-frame images for interpolation and fusion, resulting in high-resolution interpolation images, which have poor imaging effects and low image detail recognition.

FIG. 3 is a first schematic flowchart of an image processing method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a smart terminal, and the image processing method includes:

Step S101 , acquiring multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images shot for the same scene, and the multiple frames of initial images include reference images and non-reference images.

Exemplarily, the smart terminal is, for example, a smart phone, which is provided with one or more cameras. Acquiring multiple frames of initial images may be continuously changing multiple frames of pictures in the same scene captured by one or more cameras of the smartphone. There are continuously changing offsets between multiple frames of pictures. More specifically, the action of taking multiple frames of pictures through the smartphone may be triggered by a user's shooting operation. For example, the user presses the "photograph" button, and the smartphone takes 10 pictures in a short period of time (for example, within 0.5 seconds). photos and save them to generate multiple frames of initial images. Since the mobile phone is shaken by the user, there is a certain degree of continuously changing offset between the pictures taken by the smart phone. Of course, the action of taking multiple frames of pictures through the smartphone may also be triggered by the user's multiple shooting operations within a short period of time. The process of generating multiple frames of initial images is similar and will not be repeated here.

Further, the multi-frame initial images include a reference image and a non-reference image, wherein, for example, the reference image may be an image with better imaging effect or a higher-resolution image among the multi-frame initial images, the reference image Images and non-reference images can be determined after processing multiple frames of initial images by preset algorithms, such as evaluating the clarity of each initial image, determining the picture to be processed with the highest image, determined to be a non-reference image. Of course, the reference image and the non-reference image in the multi-frame initial image may also be determined according to preset rules, for example, the first frame image or the last frame image in the multi-frame initial image is determined as the reference image, The process will not be repeated here.

Step S102, calculating the optical flow from the non-reference image to the reference image.

Optical flow is the projection of the motion of an object in three-dimensional space on a two-dimensional image plane. It is generated by the relative speed of the object and the camera, and reflects the motion direction and speed of the image pixel corresponding to the object in a very small time. In this embodiment, the optical flow from the non-reference image to the reference image is calculated, that is, the position mapping of each pixel in the non-reference image to the reference image is calculated through the preset optical flow algorithm, that is, for the non-reference image For each pixel of , the calculation is performed at the position of the pixel in the reference image that is most similar to the pixel in the non-reference image, and the obtained optical flow is the information that characterizes the position map. Wherein, the optical flow algorithm is an existing technology known to those skilled in the art, and the specific type of the optical flow algorithm is not limited this time, and details will not be described one by one.

Step S103, based on the optical flow from the non-reference image to the reference image, determine the reference pixel in the non-reference image corresponding to the reference pixel in the reference image.

Exemplarily, after determining the optical flow from the non-reference image to the reference image, according to the position of the reference pixel in the reference image, determine the pixels related to the reference pixel in other non-reference images, that is, the reference pixel. Among them, the reference pixel contains the sub-pixel information corresponding to the reference pixel due to the continuously changing offset between non-reference images. Therefore, the interpolation pixel determined according to the reference pixel can better express Image detail, making interpolated high-resolution images look more realistic.

Exemplarily, FIG. 4 is a schematic diagram of determining a reference pixel point provided by an embodiment of the present disclosure. As shown in FIG. 4 , the method for determining a reference pixel point corresponding to a reference pixel point can be performed by performing optical flow calculation on a reference image , get the optical flow from the non-reference image to the reference image, and then map the reference pixel to the non-reference image according to the optical flow calculation result, and determine the mapping point between the reference pixel and the non-reference image in the non-reference image The pixel points within the preset distance are used as the reference pixel points. Wherein, the optical flow calculation method is a prior art known to those skilled in the art, and will not be repeated here.

Step S104 , by interpolating the reference pixel points in the non-reference image and the reference pixel points in the reference image, performing pixel fusion on the non-reference image and the reference image to generate a target image.

Specifically, in order to generate a high-resolution target image, it is necessary to fill in pixel information for each interpolation pixel in the high-resolution target image generated by the interpolation of the reference image, wherein the reference pixel corresponding to each interpolation pixel , are one or more original pixels originally in other non-reference images, and the interpolation pixels correspond to the pixel fusion of the reference pixels and the corresponding reference pixels, and the generated pixels are the interpolation pixels. The interpolated pixels contain the original sub-pixel information in different non-reference images, so the high-resolution target image composed of interpolated pixels, compared with the prior art, after aligning the non-reference images through pseudo-transformation, and then performing In the scheme of obtaining the target image by pixel fusion, the interpolation image generated by the method provided in this embodiment better utilizes the image information in the initial image, so that the image details of the interpolation image are better.

Wherein, the method of performing pixel fusion on multiple pixel points can be implemented by performing weighted average of the pixel values of each pixel point, and the process will not be repeated here.

In this embodiment, by acquiring multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include reference images and non-reference images; calculating from non-reference images to reference The optical flow of the image; based on the optical flow from the non-reference image to the reference image, determine the reference pixels in the non-reference image corresponding to the reference pixels in the reference image; by combining the reference pixels in the non-reference image with the reference pixel The reference pixels in the target image are interpolated, and the non-reference image and the reference image are pixel-fused to generate the target image. Because in the process of generating the target image, each interpolation pixel in the target image uses the corresponding Therefore, the original pixel information of each frame image in the multi-frame image to be processed is fully utilized, and the loss of part of image information and the introduction of error are avoided. The problem of improving the imaging effect and image details of the final generated target image.

FIG. 5 is a second schematic flowchart of an image processing method provided by an embodiment of the present disclosure. Steps S102-S103 are refined in this embodiment, and the image processing method includes:

Step S201, acquiring continuously changing multiple frames of initial images, wherein the multiple frames of initial images include a reference image and a non-reference image.

Step S202, obtain the preset resolution information, and determine the reference pixel in the reference image according to the resolution information and the size information of the reference image, the reference pixel is obtained by interpolating the original pixel in the reference image, the reference pixel The number of points is greater than the number of original pixel points.

Exemplarily, on the basis of the reference image, interpolation calculation is performed on the reference image to generate a frame of high-resolution image, and the reference pixel points in the reference image are determined according to the high-resolution image. FIG. 6 is a schematic diagram of interpolation calculation for a reference image provided by an embodiment of the present disclosure. As shown in FIG. 6 , for example, the reference image is an image containing 4 pixels and a resolution of 2X2, and the reference image is After the interpolation calculation, the determined high-resolution image is an image including 16 pixels with a resolution of 4×4, and each pixel in the high-resolution image is a reference pixel. Afterwards, after filling the pixel information of each pixel in the high-resolution image, the target image can be generated, and after interpolation calculation is performed on the reference image, the position of each reference pixel in the reference image can be determined. The position of the reference pixel in the reference image is recorded in the interpolation image information. More specifically, the interpolated image information includes the coordinate values of each reference pixel in the interpolated image.

Among them, the interpolation calculation of the reference image can be determined according to the preset resolution information, or dynamically determined according to the frame number of the initial image. There is no limitation here, and the reference image is determined according to the resolution information and the size information of the reference image After the reference pixels, the number of reference pixels is greater than the number of original pixels, which means that after the reference image is interpolated, the resolution of the corresponding high-resolution image is greater than the resolution of the initial reference image, and the image interpolation The specific steps of calculation are the prior art known to those skilled in the art, and will not be repeated here.

Step S203, calculating the optical flow from the non-reference image to the reference image.

Step S204, based on the optical flow from the non-reference image to the reference image, determine the coordinates of the reference point corresponding to the image position where the reference pixel is located in the non-reference image.

Wherein, the reference image includes a plurality of reference pixels. For example, the optical flow calculation can be performed on each reference pixel in the manner of loop sequential processing or parallel processing. The following is to perform optical flow calculation on one of the reference pixels , the process of determining the coordinates of the reference point corresponding to the image position where the reference pixel is located will be introduced in detail.

Fig. 7 is the flowchart of the process of determining the coordinates of the reference point corresponding to the position of the image where the reference pixel is located. As shown in Fig. 7, the process includes:

S2041. Acquire the coordinates of the reference pixel.

S2042. In the reference image, determine an edge pixel point corresponding to the coordinates of the reference pixel point, where the edge pixel point is used to represent at least one pixel point adjacent to the reference pixel point.

For example, the coordinates of the reference pixel point can be obtained through the resolution information and the size information of the reference image, and the interpolation image information can be generated through the resolution information and the size information of the reference image, and the interpolation image information includes each reference pixel in the reference image The implementation of the coordinates of the points and the interpolation of image information has been introduced in detail in the foregoing embodiments, and will not be repeated here. Further, in the reference image, according to the coordinates of the reference pixel, determine several original pixel points closest to the coordinate, that is, edge pixel points. FIG. 8 is a schematic diagram of a reference pixel point and its corresponding edge pixel points provided by an embodiment of the present disclosure. As shown in FIG. 8 , according to the coordinates of the reference pixel point O, the four original pixel points closest to it are respectively calculated, That is, A, B, C, and D, so that the original pixels in the four reference images of A, B, C, and D are determined as edge pixels. Certainly, according to the change of the position of the first reference pixel O, there may be less than 4 adjacent pixel points, and the examples will not be repeated here.

More specifically, the specific calculation methods of A, B, C, and D can be shown in formula (1):

A=(floor(a),floor(b))

B=(floor(a)+1,floor(b))

C=(floor(a),floor(b)+1)

D＝(floor(a)+1,floor(b)+1) (1)

Wherein, a and b are the coordinate values of the coordinates (a, b) of the first reference pixel point O, and floor() is a function of rounding down.

S2043. Based on the optical flow from the non-reference image to the reference image, determine the offset vector of the edge pixel points in the non-reference image.

Exemplarily, one implementation of determining the offset vector of edge pixels in the non-reference image is, for example, to calculate the vector information between the reference image and the non-reference image through the optical flow algorithm, wherein the vector information is used to represent Pixels in the reference image and offset vectors between pixels at corresponding positions in the non-reference image; according to the motion vector information, determine the offset vector of the edge pixels in the non-reference image. Among them, the optical flow algorithm is realized by the optical flow function OF(), and the general expression of optical flow calculation is shown in formula (2):

F _t (x,y)=F _r ((x,y)+OF(x,y)),t∈[1,N] (2)

Among them, (x, y) is the coordinates of the pixel in the reference image, r is the index of the reference frame, t is the index of the non-reference frame, and N is the frame number of the non-reference image. OF() is the optical flow function, and OF(x,y) is the optical flow calculation result of the pixel. When the optical flow calculation result contains decimals, the decimal part represents the sub-pixel position to which it is mapped. The mapping of pixels in the reference image to non-reference images can be realized by formula (2).

Further, the optical flow calculation is performed on the edge pixels respectively, the optical flow calculation results of each edge pixel are determined, and the weighted average calculation is performed to obtain an average offset, that is, an offset vector. Exemplarily, the offset The calculation method of the vector is shown in formula (3):

offset＝medium(OF(A),OF(B),OF(C),OF(D)) (3)

Among them, offset is the offset vector, medium() is the weighted average calculation, OF(A), OF(B), OF(C), OF(D) are the optical flow of edge pixels A, B, C, and D respectively Calculation results.

S2044. Determine the coordinates of the reference point according to the coordinates of the reference pixel point and the offset vector.

Further, by performing offset processing on the coordinates of the first reference pixel point according to the offset vector, the reference coordinate point in the non-reference image can be determined. Exemplarily, the implementation method is shown in formula (4):

center1＝round((a,b)-offset) (4)

Wherein, a and b are the coordinate values of the coordinates (a, b) of the first reference pixel point O, round() is a rounding function, and the reference coordinate point is center1.

Step S205, determining all the pixels within the first preset range of the reference point coordinates in the non-reference image as reference pixels.

FIG. 9 is a schematic diagram of a first preset range provided by an embodiment of the present disclosure. With reference to FIG. 9 , further, according to the mapping of the reference pixel point 91 in the non-reference image and the offset vector, determine each location in the non-reference image After specifying the reference point coordinates 92, according to the reference point coordinates 92, respectively determine the pixel points within the first preset range of the reference point coordinates 92 in each non-reference image, and determine the one or more pixel points as reference points 93 pixels. Wherein, for example, the first preset range is determined according to the variance of the offset vector of the edge pixel points in the non-reference image and a preset weight coefficient.

Exemplarily, the implementation method of determining the pixel points within the first preset range of the coordinates of the reference point in the non-reference image can be realized by adding a window, wherein the first preset range is determined by the size of the windowing radius . More specifically, the method of determining the window radius is shown in formula (5) and formula (6):

r1＝(1-weight)*r1_th1+(weight)*r1_th2 (5)

weight=min(variance(OF(A),OF(B),OF(C),OF(D)),weight_max)/weight_max(6)

Among them, r1 is the window radius, weight is the window weight coefficient, variance() is the variance function, min() is the minimum value function, r1_th1, r1_th2 and weight_max are the preset thresholds, and r1_th2>r1_th1>1, weight_max>0 , r1_th1, r1_th2, and weight_max are determined by factors such as the image size, image content, and texture information of the initial image, and can be obtained through experiments, and the determination methods will not be described here.

In this embodiment, after the coordinates of the reference point are determined through optical flow calculation, the reference pixel points within the first preset range are obtained by determining the first preset range. Due to the determination process of the first preset range, through the parameters Setting, taking into account the influence of factors such as image size, image content, texture information, etc., the first preset range determined by this can better determine the reference pixel point with better correlation with the reference pixel point, and then improve the reference pixel point. The effectiveness of the points can improve the image details and image expressiveness of the target image for subsequent synthesis.

Optionally, after step S205, in order to further improve the correlation between the reference pixel and the reference pixel, and improve the accuracy of the subsequent pixel fusion of the non-reference image and the reference image, it also includes:

S206. Determine the mapping coordinates of the reference pixel point in the reference image.

S207. Determine the reference pixel points in the mapping coordinates that fall within the second preset range of the reference pixel points as valid reference pixel points.

Fig. 10 is a schematic diagram of a second preset range provided by an embodiment of the present disclosure. Referring to Fig. 10 , for example, after the first preset range with the coordinates of the reference point as the center point is determined, all original The pixel can be used as a reference pixel for subsequent pixel fusion to determine the pixel value of the corresponding interpolation pixel, wherein the target image is composed of interpolation pixels, and the positions of the reference pixel and the interpolation pixel coincide. However, after the reference pixel points 93 in the first preset range are mapped to the reference image, corresponding mapping coordinates 94 will be generated, and some reference pixel points 91 corresponding to the mapping coordinates 94 are relatively close and some are far away. For the reference pixel point 93 corresponding to the farther mapping coordinate 94, since it is far away from the interpolation pixel point, the image information represented by it is also quite different from the real image information at the position corresponding to the reference pixel point 91. Using such reference pixels for pixel fusion will result in distortion of the final interpolated pixels, affecting the accuracy of pixel fusion, and further affecting the image details and image effects of the target image.

Therefore, in order to further improve the correlation between the reference pixel and the reference pixel, and improve the accuracy of the subsequent calculation of the interpolation pixel, the reference pixel with a higher correlation with the reference pixel is determined as an effective reference pixel, And use effective reference pixel points to perform subsequent pixel fusion.

Wherein, for example, the method of determining the effective reference pixel point may be to determine the reference pixel point whose mapping coordinates fall within the second preset range of the reference pixel point as the effective reference pixel point, wherein, confirming that the second preset range Methods include:

Obtain image texture information and/or noise information corresponding to the reference pixel, wherein the image texture information is used to characterize the image texture structure near the reference pixel, and the noise information is used to characterize the noise level of the initial image; according to the Image texture information, and/or noise information, determine a second preset range.

Specifically, the method for determining the second preset range can be realized by formula (7), formula (8) and formula (9):

r2＝(1-weight)*r2_th1+(weight)*r2_th2 (7)

Vari＝variance(F(A),F(B),F(C),F(D)) (9)

Among them, r2_th1, r2_th2, scale are the preset thresholds, weight is the weight coefficient, r2_th2>r2_th1>=1, scale>1, the specific value is determined by the experimental debugging, variance() is the variance function, F(A), F (B), F(C), and F(D) are the coordinates of edge pixels A, B, C, and D in the reference image respectively, noise is the noise level, and is the system input value, and Vari is the second preset range.

In this embodiment, the second preset range is determined through the image texture information and/or noise information corresponding to the reference pixel, and effective reference information for subsequent fusion calculations is determined within the second preset range, further Improves the image detail and image effects of the resulting target image.

Step S208, performing weighted calculation on the pixel values of effective reference pixels among the reference pixels corresponding to the reference pixel, determining the pixel values of the interpolation pixels, and constructing the target image based on the pixel values of each interpolation pixel.

Exemplarily, after the pixel value of the effective reference pixel is determined, weighted calculation is performed on the pixel value of the effective reference pixel to determine the pixel value of the interpolation pixel corresponding to the corresponding reference pixel. Exemplarily, the method for determining the pixel value of the interpolated pixel point can be realized by formula (10):

Among them, O _i represents the i-th interpolation pixel in the target image, P _m is the pixel value of the m-th effective reference pixel, W _m is the weight of the m-th effective reference pixel, and n is the number of effective reference pixels Of course, it can be understood that if step S205 is not executed, that is, in the scheme of directly performing fusion calculation through reference pixel points to determine the interpolation pixel point corresponding to the reference pixel point, P _m and W _m here represent the reference pixel points respectively The pixel value of the point and the weight of the reference pixel, n is the number of reference pixels.

The target image can be generated by determining the pixel value of each interpolation pixel in the target image through parallel processing or polling and sequential processing.

In this embodiment, the implementation of steps S201 and S203 is the same as the implementation of steps S101 and S102 in the embodiment shown in FIG. 3 of the present application, and will not be repeated here.

Corresponding to the image processing method in the above embodiments, FIG. 11 is a structural block diagram of an image processing device provided in an embodiment of the present disclosure. For ease of description, only the parts related to the embodiments of the present disclosure are shown. Referring to Figure 11, the image processing device 3 includes:

The acquiring unit 31 is configured to acquire multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include reference images and non-reference images;

The first determination unit 32 is used to calculate the optical flow from the non-reference image to the reference image;

The second determination unit 33, based on the optical flow from the non-reference image to the reference image, determines the reference pixel in the non-reference image corresponding to the reference pixel in the reference image;

The generation unit 34 is used to interpolate the reference pixels in the non-reference image and the reference pixels in the reference image, perform pixel fusion on the non-reference image and the reference image, and generate the target image.

In one embodiment of the present disclosure, the second determining unit 33 is specifically configured to: determine the coordinates of the reference point in the non-reference image corresponding to the image position where the reference pixel is located based on the optical flow from the non-reference image to the reference image ; Determining all pixels within the first preset range of the reference point coordinates in the non-reference image as reference pixel points.

In one embodiment of the present disclosure, when the second determination unit 33 determines the coordinates of the reference point in the non-reference image corresponding to the image position where the reference pixel is located based on the optical flow from the non-reference image to the reference image, it specifically uses In: obtaining the coordinates of the reference pixel point; in the reference image, determining the edge pixel point corresponding to the coordinate of the reference pixel point, wherein the edge pixel point is used to represent at least one pixel point adjacent to the reference pixel point; based on non-reference pixel point The optical flow from the image to the reference image determines the offset vector of the edge pixels in the non-reference image; according to the coordinates and offset vector of the reference pixel, the coordinates of the reference point are determined.

In an embodiment of the present disclosure, the second determining unit 33 is further configured to: determine the first preset coordinates of the reference point according to the variance of the offset vector of the edge pixel in the non-reference image and the preset weight coefficient scope.

In one embodiment of the present disclosure, when the second determining unit 33 determines the offset vector of edge pixels in the non-reference image based on the optical flow from the non-reference image to the reference image, it is specifically configured to: through the optical flow Algorithm to calculate the vector information between the reference image and the non-reference image, wherein the vector information is used to characterize the pixels in the reference image, and the offset vector between the pixels of the corresponding positions in the non-reference image; according to the motion vector information to determine the offset vector of edge pixels in the non-reference image.

In an embodiment of the present disclosure, the second determination unit 33 is further configured to: determine the mapping coordinates of the reference pixel points in the reference image; , is determined to be an effective reference pixel point; when the generation unit 34 performs pixel fusion on the non-reference image and the reference image to generate the target image, it is specifically used to: the pixel value of the effective reference pixel point in the reference pixel point corresponding to the reference pixel point Carry out weighted calculation to determine the pixel value of the corresponding interpolation pixel point of the reference pixel point; according to the pixel value of the interpolation pixel point, the target image is formed.

In an embodiment of the present disclosure, the acquisition unit 31 is further configured to: acquire image texture information corresponding to the reference pixel point, and/or noise information, wherein the image texture information is used to characterize the image texture structure near the reference pixel point, The noise information is used to characterize the noise level of the initial image; the second determination unit 33 is further configured to: determine the second preset range according to the image texture information corresponding to the reference pixel and/or the noise information.

In an embodiment of the present disclosure, the acquisition unit 31 is further configured to: acquire preset resolution information; determine the reference pixel in the reference image according to the resolution information and the size information of the reference image, and the reference pixel is the reference pixel The original pixel points in the reference image are obtained by interpolation, and the number of reference pixel points is greater than the number of original pixel points.

The device provided in this embodiment can be used to implement the technical solutions of the above method embodiments, and its implementation principle and technical effect are similar, so this embodiment will not repeat them here.

FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 12 , the electronic device 4 includes at least one processor 401 and a memory 402;

memory 402 stores computer-executable instructions;

At least one processor 401 executes the computer-executed instructions stored in the memory 402, so that the at least one processor 401 executes the screen sharing method in the embodiment shown in FIGS. 3-10 .

Wherein, the processor 401 and the memory 402 are connected through a bus 403 .

Relevant descriptions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the embodiments corresponding to FIG. 3 to FIG. 10 , and details are not repeated here.

Referring to FIG. 13 , it shows a schematic structural diagram of an electronic device 900 suitable for implementing the embodiments of the present disclosure. The electronic device 900 may be a terminal device or a server. Among them, the terminal equipment may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA for short), tablet computers (Portable Android Device, PAD for short), portable multimedia players (Portable Media Player, referred to as PMP), mobile terminals such as vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 13 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 13, an electronic device 900 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 901, which may be stored in a program in a read-only memory (Read Only Memory, ROM for short) 902 or from a storage device. 908 loads the programs in the random access memory (Random Access Memory, RAM for short) 903 to execute various appropriate actions and processes. In the RAM 903, various programs and data necessary for the operation of the electronic device 900 are also stored. The processing device 901, ROM 902, and RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904 .

Generally, the following devices can be connected to the I/O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; ), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication means 909 may allow the electronic device 900 to perform wireless or wired communication with other devices to exchange data. While FIG. 13 shows electronic device 900 having various means, it is to be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 909, or from storage means 908, or from ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programming read-only memory (Erasable Programmable ROM, referred to as EPROM or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc ROM, referred to as CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . The program code contained on the computer readable medium can be transmitted by any appropriate medium, including but not limited to: electric wire, optical cable, radio frequency (Radio Frequency, RF for short), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is made to execute the methods shown in the above-mentioned embodiments.

Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external A computer (connected via the Internet, eg, using an Internet service provider).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short), Application Specific Integrated Circuit (ASIC for short), Application Specific Standard Products ( Application Specific Standard Product (ASSP for short), System on Chip (SOC for short), Complex Programmable Logic Device (CPLD for short), etc.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

In a first aspect, according to one or more embodiments of the present disclosure, an image processing method is provided, including:

According to one or more embodiments of the present disclosure, based on the optical flow from the non-reference image to the reference image, a reference pixel in the non-reference image corresponding to a reference pixel in the reference image is determined , comprising: based on the optical flow from the non-reference image to the reference image, determining the reference point coordinates corresponding to the image position where the reference pixel is located in the non-reference image; , all pixels located within the first preset range of coordinates of the reference point are determined as reference pixel points.

According to one or more embodiments of the present disclosure, based on the optical flow from the non-reference image to the reference image, determine the reference point coordinates corresponding to the image position where the reference pixel is located in the non-reference image , comprising: acquiring the coordinates of a reference pixel; in the reference image, determining an edge pixel corresponding to the coordinate of the reference pixel, wherein the edge pixel is used to represent a pixel adjacent to the reference pixel At least one pixel; based on the optical flow from the non-reference image to the reference image, determine the offset vector of the edge pixel in the non-reference image; according to the coordinates of the reference pixel and the The above offset vector is used to determine the coordinates of the reference point.

According to one or more embodiments of the present disclosure, the method further includes: determining the coordinates of the reference point according to the variance of the offset vector of the edge pixel in the non-reference image and a preset weight coefficient the first preset range of .

According to one or more embodiments of the present disclosure, based on the optical flow from the non-reference image to the reference image, determining the offset vector of the edge pixel in the non-reference image includes: passing light A stream algorithm, calculating vector information between a reference image and a non-reference image, wherein the vector information is used to characterize the pixels in the reference image, and the offset between pixels at corresponding positions in the non-reference image vector: determining an offset vector of the edge pixel in the non-reference image according to the vector information.

According to one or more embodiments of the present disclosure, the method further includes: determining the mapping coordinates of the reference pixel points in the reference image; placing the mapping coordinates within a second preset range of the reference pixel points The reference pixels within are determined as valid reference pixels; performing pixel fusion on the non-reference image and the reference image to generate a target image, including: valid reference pixels in the reference pixels corresponding to the reference pixels Weighted calculation is performed on the pixel values of the points to determine the pixel values of the corresponding interpolation pixel points of the reference pixel point; and the target image is formed according to the pixel values of the interpolation pixel points.

According to one or more embodiments of the present disclosure, the method further includes: acquiring image texture information corresponding to the reference pixel point, and/or noise information, wherein the image texture information is used to characterize the reference pixel point Nearby image texture structure, the noise information is used to characterize the noise level of the image to be processed; the second preset range is determined according to the image texture information and/or noise information corresponding to the reference pixel.

According to one or more embodiments of the present disclosure, before determining the reference pixel in the non-reference image corresponding to the reference pixel in the reference image, it further includes: acquiring preset resolution information; according to the resolution Ratio information and size information of the reference image to determine the reference pixels in the reference image, the reference pixels are obtained by interpolating the original pixels in the reference image, the number of the reference pixels is greater than The number of original pixels.

In a second aspect, according to one or more embodiments of the present disclosure, an image processing device is provided, including:

According to one or more embodiments of the present disclosure, the second determination unit is specifically configured to: determine, based on the optical flow from the non-reference image into the reference image, that in the non-reference image is related to the reference pixel The coordinates of the reference point corresponding to the position of the image where the point is located; determining all pixel points within the first preset range of the coordinates of the reference point in the non-reference image as reference pixel points.

According to one or more embodiments of the present disclosure, the second determining unit is based on the optical flow from the non-reference image to the reference image, and determines the image position of the reference pixel in the non-reference image For the corresponding reference point coordinates, it is specifically used to: obtain the coordinates of the reference pixel points; in the reference image, determine the edge pixel points corresponding to the coordinates of the reference pixel points, wherein the edge pixel points are used to represent At least one pixel point adjacent to the reference pixel point; based on the optical flow from the non-reference image to the reference image, determine the offset vector of the edge pixel point in the non-reference image; according to the The coordinates of the reference pixel point and the offset vector are used to determine the coordinates of the reference point.

According to one or more embodiments of the present disclosure, the second determination unit is further configured to: determine the first preset value of the coordinates of the reference point according to the variance of the offset vector of the edge pixel point in the non-reference image and the preset weight coefficient. set range.

According to one or more embodiments of the present disclosure, the second determination unit determines the offset vector of the edge pixel in the non-reference image based on the optical flow from the non-reference image to the reference image When, it is specifically used to: calculate the vector information between the reference image and the non-reference image through the optical flow algorithm, wherein the vector information is used to characterize the pixels in the reference image, and the pixel points at the corresponding positions in the non-reference image The offset vector between them; according to the motion vector information, determine the offset vector of the edge pixel in the non-reference image.

According to one or more embodiments of the present disclosure, the second determining unit is further configured to: determine the mapping coordinates of the reference pixel points in the reference image; point, determined as an effective reference pixel point; when the generation unit performs pixel fusion on the non-reference image and the reference image to generate the target image, it is specifically used to: performing weighted calculation on the pixel value of the reference pixel, and determining the pixel value of the interpolation pixel corresponding to the reference pixel; and constructing the target image according to the pixel value of the interpolation pixel.

According to one or more embodiments of the present disclosure, the acquisition unit is further configured to: acquire image texture information corresponding to the reference pixel point, and/or noise information, wherein the image texture information is used to characterize the image texture structure near the reference pixel point , the noise information is used to characterize the noise level of the initial image; the second determination unit is further configured to: determine the second preset range according to the image texture information corresponding to the reference pixel and/or the noise information.

According to one or more embodiments of the present disclosure, the obtaining unit is further configured to: obtain preset resolution information; determine reference pixels in the reference image according to the resolution information and size information of the reference image points, the reference pixel points are obtained by interpolating original pixel points in the reference image, and the number of the reference pixel points is greater than the number of the original pixel points.

In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one processor and a memory;

the memory stores computer-executable instructions;

In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, Realize the image processing method described in the above first aspect and various possible designs of the first aspect.

The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principles. Those skilled in the art should understand that the disclosure scope involved in this disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of equivalent features. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

In addition, while operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or performed in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while the above discussion contains several specific implementation details, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

An image processing method, characterized in that, comprising:

Acquiring multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include a reference image and a non-reference image;

calculating optical flow from said non-reference image to said reference image;

determining a reference pixel in the non-reference image corresponding to a reference pixel in the reference image based on an optical flow from the non-reference image to the reference image;

A target image is generated by performing interpolation calculation on the reference pixel in the non-reference image and the reference pixel in the reference image, and performing pixel fusion on the non-reference image and the reference image.
The method according to claim 1, characterized in that, based on the optical flow from the non-reference image to the reference image, determining the reference pixel corresponding to the reference pixel in the reference image in the non-reference image pixels, including:

Based on the optical flow from the non-reference image to the reference image, determine the reference point coordinates corresponding to the image position where the reference pixel is located in the non-reference image;

Determining all pixel points in the non-reference image within a first preset range of coordinates of the reference point as reference pixel points.
The method according to claim 2, characterized in that, based on the optical flow from the non-reference image to the reference image, the reference image corresponding to the image position where the reference pixel is located in the non-reference image is determined. Point coordinates, including:

Obtain the coordinates of the reference pixel point;

In the reference image, determine an edge pixel point corresponding to the coordinates of the reference pixel point, wherein the edge pixel point is used to represent at least one pixel point adjacent to the reference pixel point;

determining an offset vector of the edge pixel in the non-reference image based on the optical flow from the non-reference image to the reference image;

Determine the reference point coordinates according to the coordinates of the reference pixel point and the offset vector.
The method according to claim 3, further comprising:

A first preset range of coordinates of the reference point is determined according to a variance of an offset vector of the edge pixel point in the non-reference image and a preset weight coefficient.
The method according to claim 3, wherein, based on the optical flow from the non-reference image to the reference image, determining the offset vector of the edge pixel in the non-reference image comprises:

Through the optical flow algorithm, calculate the vector information between the reference image and the non-reference image, wherein the vector information is used to characterize the pixels in the reference image, and the pixels between the corresponding positions in the non-reference image offset vector;

According to the vector information, an offset vector of the edge pixel point in the non-reference image is determined.
The method according to claim 2, further comprising:

determining the mapping coordinates of the reference pixel point in the reference image;

Determining the reference pixel points falling within the second preset range of the reference pixel points in the mapping coordinates as valid reference pixel points;

Perform pixel fusion on the non-reference image and the reference image to generate a target image, including:

performing weighted calculation on the pixel values of effective reference pixels among the reference pixels corresponding to the reference pixel, and determining the pixel value of the corresponding interpolation pixel of the reference pixel;

A target image is formed according to the pixel values of the interpolated pixel points.
The method according to claim 6, further comprising:

Acquiring image texture information and/or noise information corresponding to the reference pixel, wherein the image texture information is used to characterize the image texture structure near the reference pixel, and the noise information is used to characterize the image to be processed noise level;

The second preset range is determined according to image texture information and/or noise information corresponding to the reference pixel.
The method according to any one of claims 1 to 7, wherein before determining the reference pixel in the non-reference image corresponding to the reference pixel in the reference image, further comprising:

Obtain preset resolution information;

Determine a reference pixel in the reference image according to the resolution information and the size information of the reference image, the reference pixel is obtained by interpolating the original pixel in the reference image, the reference pixel The number of dots is greater than the number of original pixel dots.
An image processing device, characterized in that it comprises:

An acquisition unit, configured to acquire multiple frames of initial images, wherein the multiple frames of initial images are multiple frames of different images taken for the same scene, and the multiple frames of initial images include reference images and non-reference images;

a first determining unit, configured to calculate an optical flow from the non-reference image to the reference image;

A second determining unit, based on the optical flow from the non-reference image to the reference image, to determine a reference pixel in the non-reference image corresponding to a reference pixel in the reference image;

A generating unit, configured to perform interpolation calculation on the reference pixel in the non-reference image and the reference pixel in the reference image, perform pixel fusion on the non-reference image and the reference image, and generate a target image.
An electronic device, characterized by comprising: at least one processor and a memory;

the memory stores computer-executable instructions;

The at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the image processing method according to any one of claims 1 to 8.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method described in any one of claims 1 to 8 is realized. image processing method.
A computer program product, characterized by comprising a computer program, which implements the image processing method according to any one of claims 1 to 8 when the computer program is executed by a processor.
A computer program, characterized in that, when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is realized.