WO2023207408A1

WO2023207408A1 - Data processing method and apparatus, and device and readable storage medium

Info

Publication number: WO2023207408A1
Application number: PCT/CN2023/082111
Authority: WO
Inventors: 夏思烽; 高欣玮
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2022-04-24
Filing date: 2023-03-17
Publication date: 2023-11-02
Also published as: CN114529490B; CN114529490A; WO2023207408A9

Abstract

Disclosed in the present application are a data processing method and apparatus, and a device and a readable storage medium. The method comprises: acquiring a filter size set for filtering processing, wherein the filter size set comprises N filter sizes, N being a positive integer greater than 1, and any two of the N filter sizes being different; respectively performing low-pass filtering processing on an original image on the basis of the filter sizes, so as to obtain N filtered images; respectively performing image conversion on the N filtered images according to the original image, so as to obtain N high-frequency images; and performing image fusion on the N high-frequency images, so as to obtain a fused image, and fusing the fused image and the original image, so as to obtain a sharpening enhanced image corresponding to the original image. By means of the present application, the quality of a sharpened image in image sharpening services can be improved. The present application can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation and assisted driving.

Description

A data processing method, device, equipment and readable storage medium

This application claims priority to the Chinese patent application submitted to the China Patent Office on April 24, 2022, with application number 202210432913.5 and the application title "A data processing method, device, equipment and readable storage medium", and its entire content is approved by This reference is incorporated into this application.

Technical field

This application relates to the field of computer technology, and in particular to data processing technology.

Background technique

With the advent of the digital age, images can be processed by computers, and people have higher requirements for image clarity. The image processing process of making blurry images clear is called image sharpening. There are many reasons for blurred images, such as camera shake when acquiring images, poor design of optical components of scanning equipment, or noise interference during image signal transmission. From the perspective of image spectrum analysis, image blur is caused by insufficient high-frequency components in the image, resulting in insufficient sharpness of the image. Therefore, when we perform image sharpening on blurred images, the essence is to reasonably increase the high-frequency components in the image.

At present, the image sharpening method simply enhances the high-frequency components in the image, which increases the brightness difference at the edge of the image, thereby achieving the sharpening effect.

However, this method will make the image information expressed by each detail in the image unreasonable, causing the image distortion after sharpening to be relatively large, and the image quality after sharpening is not high.

Contents of the invention

Embodiments of the present application provide a data processing method, device, equipment and readable storage medium, which can improve the quality of sharpened images in the image sharpening business.

On the one hand, embodiments of the present application provide a data processing method, which is executed by a computer device and includes:

Obtain the filter size set used for filtering processing; the filter size set includes N filter sizes, N is a positive integer greater than 1; any two filter sizes among the N filter sizes are different;

Perform low-pass filtering on the original image based on each filter size to obtain N filtered images;

Perform image conversion on N filtered images respectively based on the original image to obtain N high-frequency images;

Perform image fusion on N high-frequency images to obtain a fused image, and fuse the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image.

On the one hand, embodiments of the present application provide a data processing device, which is deployed on a computer device and includes:

The size acquisition module is used to obtain a filter size set used for filtering; the filter size set includes N filter sizes, and N is a positive integer greater than 1; any two filter sizes among the N filter sizes are different;

The filter module is used to perform low-pass filtering on the original image based on each filter size to obtain N filtered images;

The image conversion module is used to perform image conversion on N filtered images respectively based on the original image to obtain N high-frequency images;

The image fusion module is used to image fuse N high-frequency images to obtain a fused image;

The image sharpening module is used to fuse the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image.

On the one hand, embodiments of the present application provide a computer device, including: a processor and a memory;

The memory stores a computer program. When the computer program is executed by the processor, it causes the processor to execute the method in the embodiment of the present application.

On the one hand, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When executed by a processor, the program instructions execute the methods in the embodiments of the present application.

In one aspect of the present application, a computer program product is provided. The computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in one aspect of the embodiment of the present application.

In this embodiment of the present application, for a certain image (such as an original image), N different filter sizes can be used to perform low-pass filtering on the original image respectively, thereby obtaining N different filtered images; subsequently, Image conversion is performed on N different filtered images based on the original image to obtain N different high-frequency images; the N different high-frequency images can be used for sharpening enhancement. For example, N high-frequency images can be image fused. After obtaining the fused image, the fused image can be fused with the original image. That is, the fused high-frequency information containing multiple sizes can be added to the original image, that is, The high-frequency information at each size in the original image is enhanced, so the sharpened image corresponding to the original image can be obtained. It should be understood that after this application performs low-pass filtering on the original image through different filter sizes, low-frequency images (ie, N filtered images) under different filter sizes can be obtained, and then through the original image itself and each filtered image, the low-frequency images can be extracted The corresponding high-frequency information of each filtered image is obtained (that is, N high-frequency images are obtained). For multi-scale high-frequency images, this application can fuse them to obtain a fused image. The processed fused image can be compared with the original image. Fusion is performed again, so that the high-frequency intensity of the original image can be enhanced from different scales (filter sizes) to obtain a sharpened enhanced image. In addition, since this application uses different filter sizes to perform low-pass filtering on the original image at the same time, the high-frequency information obtained is also high-frequency information under different filter sizes, which can be used to filter different types of image details (such as smooth textures and complex sharp ones). Texture) has strong adaptive ability (for example, for complex and sharp textures, low-pass filtering based on low filter size can extract the corresponding high-frequency information and achieve corresponding enhancement; for smooth textures, low-pass filtering based on high filter size can The corresponding high-frequency information can be extracted through filtering and the corresponding enhancement can be achieved), that is, the detailed information of the original image can be enhanced from different scales, thereby improving the sharpening quality of the image and improving the clarity of the image. In summary, this application can improve the quality of sharpened images in the image sharpening business.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a network architecture diagram provided by an embodiment of the present application;

Figure 2 is a schematic diagram of a scene for sharpening an image provided by an embodiment of the present application;

Figure 3 is a schematic diagram of an image processing scenario provided by an embodiment of the present application;

Figure 4 is a schematic flowchart of a data processing method provided by an embodiment of the present application;

Figure 5 is a schematic diagram of pixel processing based on mean filtering provided by an embodiment of the present application;

Figure 6 is a schematic flowchart of fusing the original image and the fused image to obtain a sharpened and enhanced image provided by an embodiment of the present application;

Figure 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

This application relates to artificial intelligence and other related technologies, such as computer vision technology in artificial intelligence. The solution of this application specifically relates to image processing technology in computer vision technology, which can achieve sharpening and enhancement processing of images to obtain higher image quality. Sharpen the image. Another example can also involve machine learning in artificial intelligence. Through machine learning, a target image model can be trained, so that the target image model can be used to identify the area to which the object to be identified belongs.

Please refer to Figure 1. Figure 1 is a schematic structural diagram of a network architecture provided by an embodiment of the present application. As shown in Figure 1, the network architecture may include a service server 1000 and a terminal device cluster (ie, terminal device cluster). The terminal device cluster may include one or more terminal devices, and there will be no limit on the number of terminal devices here. As shown in Figure 1, multiple terminal devices may specifically include terminal devices 100a, terminal devices 100b, terminal devices 100c,..., terminal devices 100n. As shown in Figure 1, the terminal device 100a, the terminal device 100b, the terminal device 100c,..., the terminal device 100n can each have a network connection with the above-mentioned service server 1000, so that each terminal device can communicate with the service server 1000 through the network connection. Data interaction. The network connection here is not limited to a connection method. It can be connected directly or indirectly through wired communication, or directly or indirectly through wireless communication. It can also be connected through other methods. This application does not limit it here.

As shown in Figure 1, each terminal device can be integrated and installed with an application. When the application is run in each terminal device, the background server corresponding to each terminal device can store the business data in the application and coordinate it with the above Data exchange is performed between the business servers 1000 shown in Figure 1 . Among them, the application may include an application with the function of displaying data information such as text, images, audio, and video. For example, the application can be a multimedia application (such as a video application), which can be used by users to upload pictures or videos, or can be used by users to play and watch images or videos uploaded by others; the application can also be an entertainment application (such as a game application), which can be Used for users to play games. The application can also be other applications with data information processing functions, such as browser applications, social networking applications, image beautification applications, etc. Here we will not give examples of applications one by one. The applications integrated and installed on the terminal device can also be small programs, that is, independent programs that only need to be downloaded to the browser environment to run. Of course, the applications integrated and installed on the terminal device can be independent applications or embedded in the browser environment. A sub-application (such as an applet) in an application, which can be run or closed under user control. All in all, the applications integrated and installed on the terminal device can be any form of application, module or plug-in, and there is no limit to this.

In this embodiment of the present application, one terminal device can be selected as the target terminal device from multiple terminal devices. The terminal device may include: a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart TV, a smart speaker, a desktop computer, a smart phone Smart terminals carrying data processing functions (such as image data processing functions), such as watches, smart vehicle terminals, smart voice interaction devices, smart home appliances, aircraft, etc., but are not limited to these. For example, in this embodiment of the present application, the terminal device 100a shown in FIG. 1 can be used as the target terminal device, and the target terminal device can be integrated with the above-mentioned target application. At this time, the target terminal device can perform data exchange with the business server 1000. Interaction.

For example, when a user is using an application in a terminal device (such as an image beautification application), the business server 1000 can detect and collect through the terminal device that the user has uploaded an image containing an object to be identified (such as a user or someone else). animal object) (the image can be used as an unprocessed original image), the business server 1000 can perform image sharpening processing on the original image to enhance the image quality of the original image (such as enhancing the clarity of the original image) ). After performing image sharpening processing on the original image to obtain a sharpened enhanced image, the business server 1000 can also identify the area in the sharpened enhanced image to which the object to be identified belongs, and extract the area from the original image to obtain An image that only contains the object to be recognized but not the background (can be called a target area image). Subsequently, the business server 1000 can perform subsequent processing on the target area image that only contains the object to be recognized (such as adding special effects processing or beautification processing, etc. etc.), obtain a target area image with special effects or beautification effects (such as makeup effects, etc.); then, the business server 1000 can put the target area image with special effects or beautification effects back into the original image for the object to be identified In the corresponding area, a processed image with higher image quality and special effects or beautification effects can be obtained. Subsequently, the business server 1000 can return the processed image to the terminal device, and the user can view the processed image on the display page of the terminal device (viewing the image with higher image quality and special effects or with beautify the object to be identified).

Of course, after the business server 1000 performs sharpening and enhancement processing on the original image to obtain a sharpened and enhanced image, the business server 1000 may not perform special effects processing or beautification processing, but return the sharpened and enhanced image to the terminal device, then the user can The sharpened enhanced image is viewed on the display page of the terminal device (an image with higher image quality is viewed).

The specific process for the business server 1000 to sharpen and enhance the original image to obtain the sharpened and enhanced image may include: the business server 1000 may obtain a filter size set for filtering (the filter size set may include Different filter sizes, for example, may include N filter sizes, N is a positive integer greater than 1); based on each filter size, the business server 1000 can perform low-pass filtering processing on the original image respectively, so that different filters can be obtained image; then, the business server 100 can perform image conversion by reducing the N filtered images from the original image, thereby obtaining N high-frequency images; then, the business server 1000 can perform image fusion on the N high-frequency images, and obtain After the images are fused, the fused image is fused with the original image to obtain a sharpened enhanced image corresponding to the original image. Among them, the specific implementation method for the business server 1000 to sharpen and enhance the original image to obtain the sharpened enhanced image (for example, it may include a specific implementation method to perform low-pass filtering on the original image based on the filter size to obtain different filtered images; Perform image conversion on a certain filtered image to obtain a specific implementation method of a high-frequency image; perform image fusion on a high-frequency image to obtain a specific implementation method of a fused image; based on the fused image and the original image, obtain a specific implementation method of sharpening an enhanced image) , please refer to the subsequent description of the embodiment corresponding to FIG. 3 .

It should be understood that in image processing, sharpening processing is very important. Sharpening processing can improve the clarity of the image, and in sharpening processing, filtering processing is also very critical. In order to further improve the image quality (such as clarity) after sharpening, this application can configure different filter sizes for the image filtering process. These different filter sizes can form a filter size set, which will be processed for a certain original image. During filtering processing, the filter size set can be obtained, and then filtered through the filter size set. The sharpened enhanced image obtained after filtering with different filter sizes can process details in the image from different scales, which can greatly improve the image quality of the sharpened enhanced image.

In the above process, the specific method for the business server 1000 to identify the area to which the object to be recognized in an image (such as a sharpened enhanced image) belongs can be processed through an image model (such as an image recognition model). In order to improve the accuracy of image recognition, the image model can be trained so that the image model adjusted after training is optimal. Based on the trained image model, image recognition processing can be performed on the image (such as identifying the objects to be recognized in the image). identify the area to which the object belongs).

It can be understood that the methods provided by the embodiments of the present application can be executed by computer equipment, including but not limited to terminal equipment or business servers. Among them, the business server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, and cloud services. Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

Among them, the terminal equipment and the service server can be connected directly or indirectly through wired or wireless communication methods, which is not limited in this application.

It can be understood that the above-mentioned computer device (such as the above-mentioned business server 1000, terminal device 100a, terminal device 100b, etc.) can be a node in a distributed system, wherein the distributed system can be a blockchain system, and the The blockchain system can be a distributed system formed by connecting multiple nodes through network communication. Among them, nodes can form a point-to-point (P2P, Peer To Peer) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP, Transmission Control Protocol) protocol. In a distributed system, any form of computer equipment, such as business servers, terminal equipment and other electronic equipment, can become a node in the blockchain system by joining the peer-to-peer network. For ease of understanding, the concept of blockchain will be explained below: Blockchain is a new application model of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. It is mainly used to process data in chronological order. Organize it and encrypt it into a ledger so that it cannot be tampered with or forged, and data can be verified, stored and updated at the same time. When the computer device is a blockchain node, due to the non-tampering and anti-counterfeiting properties of the blockchain, the data in this application (such as uploaded image data such as target images, sharpened images, etc.) It has authenticity and security, which can make the results obtained after relevant data processing based on these data more reliable.

It should be noted that in the specific implementation of the present application, data related to user information, user data (such as uploaded images, videos, etc.) must be obtained with user authorization. In other words, when the above embodiments of this application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. For ease of understanding, please refer to Figure 2 , which is a schematic diagram of a scene for sharpening an image according to an embodiment of the present application. Among them, the service server 200 shown in Figure 2 can be the service server 1000 shown in Figure 1, and the terminal device 100a shown in Figure 2 can be selected from the terminal device cluster in the embodiment corresponding to Figure 1. Any terminal device, for example, the terminal device can be the above-mentioned terminal device 100b; the terminal device 100b shown in Figure 2 can be any terminal device selected from the terminal device cluster in the embodiment corresponding to Figure 1, For example, the terminal device may be the above-mentioned terminal device 100a.

As shown in Figure 2, user A can run an application (such as a short video application) through the terminal device 100a. User A uploads an image 20a in the short video application, where the image 20a includes an object to be recognized (such as an object B), this image 20a can be used as the original image. The service server 200 may receive the original image 20a through the backend server of the terminal device 100a. Subsequently, the service server 200 can perform low-pass filtering processing on the original image 20a based on different filter sizes (for example, it can include filter size 1, filter size 2, ..., filter size n, n can be a positive integer greater than 1), Thus, filtered image 1 corresponding to filter size 1, filtered image 2 corresponding to filter size 2, ..., and filtered image n corresponding to filter size n can be obtained. The filter size here may be a size used for low-pass filtering. The filter size may be a preset size. The filter size may include different sizes, including high and low. For example, the filter size may include 5× The size of 5, the size of 9×9, the size of 17×17, etc. will not be explained one by one here. It should be understood that low-pass filtering uses means such as mean filtering to suppress high-frequency information in videos or images, making the video or image look blurry. In fact, the image obtained after low-pass filtering is a blurred image ( Right now low-frequency image), then each filtered image (including filtered image, filtered image 2, ..., filtered image n) can also be called a low-frequency image.

After that, the business server 200 can determine the high-frequency information (also called high-frequency images) under different filter sizes based on the original image 20a and each filtered image. As shown in Figure 2, the business server 200 can determine the high-frequency information (also called high-frequency images) under different filter sizes based on the original image 20a and the filtered image. 1. Determine the high-frequency image 1 corresponding to the filtered image 1; determine the high-frequency image 2 corresponding to the filtered image 2 based on the original image 20a and the filtered image 2; ...; determine the high-frequency image 2 corresponding to the filtered image 2 based on the original image 20a and the filtered image n. The high-frequency image n corresponding to the filtered image n. Subsequently, after fusing the high-frequency images under different filter sizes, a fused image containing high-frequency information under each filter size can be obtained. The fused image is fused with the original image 20a, that is, the fused image under each filter size is fused. The high-frequency information is added to the original image 20a, and then after fusion, the sharpened enhanced image 20b corresponding to the original image 20a can be obtained. It should be understood that after the above-mentioned sharpening enhancement processing, the sharpening-enhanced image 20b can have higher definition (for example, the lines are clearer and the boundaries are more obvious).

The business server 200 can send the sharpened enhanced image 20b with higher definition to the terminal device 100b. Then when user C uses the application through the terminal device 100b and browses to the image uploaded by user A, the viewed image What is obtained is a sharpened enhanced image 20b with higher definition, rather than a distorted image. In the same way, the business server 200 can also return the sharpened image 20b with higher definition to the terminal device 100a, and user A can view the sharpened image 20b with higher definition on the display interface of the terminal device 100a. Image 20b.

It can be understood that after performing sharpening and enhancement processing on the original image 20a to obtain the sharpened and enhanced image 20b, subsequent image processing (such as adding special effects processing) can also be performed on the sharpened and enhanced image 20b, so that the final presentation can be Images to the display interface of the terminal device (such as the display interface of the above-mentioned terminal device 100a or the display interface of the terminal device 100b) are more interesting. For ease of understanding, please refer to Figure 3 as well. Figure 3 is a schematic diagram of an image processing scenario provided by an embodiment of the present application. As shown in FIG. 3 , the business server 200 can input the sharpened enhanced image 20b into an image model (such as an image recognition model), and the image recognition model can identify the area where the object B is located in the sharpened enhanced image 20b. As shown in Figure 3, the image recognition model recognizes that the area where object B is located in the sharpened enhanced image 20b is area P (that is, the area included in the boundary of object B). The image recognition model can extract the area P including object B. , then, the business server 200 may no longer consider other areas in the image 20a except the area P, and only perform special effects processing on the object B in the area P.

As shown in Figure 3, the business server 200 adds a "cat special effect" to the object B in the area P. Further, the business server 200 can put the object B with the "cat special effect" back into the sharpened enhanced image 20b. area P, from which a sharpened enhanced image 20c with "cat special effect" can be obtained. The sharpened and enhanced image 20c with "cat special effects" is shown in Figure 3. Subsequently, the service server 200 can return the sharpened and enhanced image 20c with "cat special effects" to the terminal device 100a. User A can log in to the terminal device 100a. The sharpened and enhanced image 20c with "cat special effect" is viewed on the display interface of the device 100a.

It should be noted that any object in an image in this application can be used as an object to be recognized. For example, if the above image 20a also contains other objects besides object B (such as a concession stand, an escalator, a basketball, etc.) , then these objects can also be used as objects to be recognized, and the image recognition model can also perform image recognition processing on other objects except object B at the same time. And the image recognition model in this application can be any model with image recognition function, and this application does not limit it.

For ease of understanding, please refer to Figure 4, which is a schematic flow chart of a data processing method provided by an embodiment of the present application. This method can be executed by a terminal device (for example, any terminal device in the terminal device cluster shown in Figure 1 above, such as the terminal device 100a), or can be executed by the terminal device and the service server (such as in the embodiment corresponding to Figure 1 above) The business server 1000) is jointly executed. For ease of understanding, this embodiment takes the method being executed by the above-mentioned terminal device as an example for description. As shown in Figure 4, the image processing method may at least include the following S101-S104:

S101, obtain a filter size set used for filtering processing; the filter size set includes N filter sizes, N is a positive integer greater than 1; any two filter sizes among the N filter sizes are different.

In this application, the filter size may refer to the template size for performing pixel calculation processing on a certain pixel point on the image. It can be understood that an image can contain one or more pixels. In the filtering process, a template can be given for a certain pixel (that is, the size of the template can be artificially specified), and the template includes the surrounding pixels. neighboring pixels and its own pixel, where the own pixel is used as the center, and the neighboring pixels around it are one or more neighborhood pixels with the pixel as the center. The pixel value of the own pixel can be determined based on the pixel values of all pixels in the template (that is, a final pixel value determined based on the pixel values of all pixels in the template can be used to replace the own pixel. the original pixel value of the point). For example, given that the template size is 5×5, the template contains a total of 25 pixels. For a certain pixel, it is necessary to select 24 adjacent pixels as its neighborhood pixels. The final pixel value can be determined jointly by the pixel values of 25 pixels within the template. For example, when the filtering process is mean filtering, the final pixel value of the pixel can be the average of the pixel values of 25 pixels in the template; when the filtering is median filtering, 25 pixels can be The pixel values of the points are sorted in order of size (such as from large to small), and then the median value is obtained from the sorted pixel value sequence, and the median value can be used as the final pixel value of the pixel point. Of course, different types of filtering processing methods have different template applications. Here, we only take the mean filtering processing and the median filtering processing as examples as an illustrative explanation.

For ease of understanding, please refer to Figure 5 as well. Figure 5 is a schematic diagram of pixel processing based on mean filtering provided by an embodiment of the present application. As shown in Figure 5, the image 50a may be an original image. For the original image 50a, the original image contains 49 pixels (including pixels a1, pixels a2, pixels a3, ..., pixels g7), it is assumed here that the original image 50a will be subjected to mean filtering, and it is assumed that the given template size (ie, filtering size) is 3×3. As shown in Figure 5, taking pixel b2 as an example, based on the template size 3×3, with pixel b2 as the center, the neighborhood pixels around it can be determined as pixel a1, pixel a2, pixel a3, Pixel point b1, pixel point b3, pixel point c1, pixel point c2 and pixel point c3. Among them, when determining the neighborhood pixel points of pixel point b2, a certain vertex of the original image 50a can be used as the coordinate origin, and the two image edges with the coordinate origin as the intersection point can each be used as a coordinate axis (which can be called x axis and y-axis), a coordinate system with the vertex as the coordinate origin can be constructed. Then each pixel point on the original image 50a can correspond to a coordinate. Then for the neighborhood pixel points of pixel point b2, it can be It is determined based on the coordinates of pixel point b2. For example, taking the coordinates of pixel b2 as (2, 6), you can add [-1,1] to the x-axis (that is, add -1, 0, 1), and add [-1, 1] to the y-axis ] (that is, increase -1, 0, 1), that is, increase (-1, -1), (-1, 0), (-1, 1), (0, -1) on coordinate (2, 6) , (0,1), (1,-1), (1,0) and (1,1), from which the coordinates of the neighborhood pixels can be obtained as (1,5), (1,6), (1,7), (2,5), (2,7), (3,5), (3,6) and (3,7), from which the above-mentioned pixel points a1, etc. can be obtained through coordinate correspondence. Each neighborhood pixel of the pixel.

Then, the pixel values of all pixels included in the template can be obtained, namely pixel a1, pixel a2, pixel a3, pixel b1, pixel b2, pixel b3, pixel c1, pixel c2 and The pixel values corresponding to pixel point c3 are the pixel values corresponding to pixel point a1, pixel point a2, pixel point a3, pixel point b1, pixel point b2, pixel point b3, pixel point c1, pixel point c2 and pixel point c3 respectively. Taking 11, 8, 11, 10, 9, 12, 10, 10, 9 as an example, it can be determined that the average value corresponding to these pixel values is 10 (that is, 11, 8, 11, 10, 9, 12, 10, 10 , 9 are added, the resulting sum is 90, the number of all pixels contained in the template is 9, then the average value is 10), then the average value of 10 can be used as the pixel point b2 the final pixel value. That is to say, through this mean filtering process, the final pixel value corresponding to each pixel point in the original image 50a can be obtained.

It should be understood that in image processing, sharpening processing is very important. Sharpening processing can improve the clarity of the image, and in sharpening processing, filtering processing is also very critical. In order to improve the image quality (such as clarity) after sharpening, this application can configure different filter sizes for the image filtering process. These different filter sizes can form a filter. Wave size set, when filtering a certain original image, the filter size set can be obtained, and then filtered through the filter size set.

S102: Perform low-pass filtering on the original image based on each filter size to obtain N filtered images.

In this application, it can be known from the above that each filter size can be used to filter the original image. The filtering process in this application may refer to low-pass filtering processing (such as mean filtering processing, median filtering processing, etc.). The method of performing low-pass filtering processing on the original image based on each filter size is similar. For the convenience of introduction, it can be Each of the N filter sizes is taken as a filter size _Si , and the filtered image corresponding to the filter size _Si among the N filtered images is taken as a filtered image _Ti (i is a positive integer). Based on a certain filter size, the original The image is subjected to low-pass filtering processing, and the specific implementation method of obtaining the filtered image corresponding to the filter size can be: obtaining the image pixel set corresponding to the original image, and obtaining the pixel coordinate set corresponding to the image pixel set; then, in the image pixel set, Obtain the pixels of the image to be processed, and obtain the pixel coordinates of the pixels to be processed corresponding to the pixels of the image to be processed in the pixel coordinate set; obtain the coordinate change amount indicated by the filter size _Si , and based on the pixel coordinates to be processed and the coordinate change amount, at the pixel coordinates The neighborhood pixel coordinates for the pixel coordinates to be processed are determined in the set; based on the pixel coordinates to be processed and the neighborhood pixel coordinates, the filtered image Ti corresponding to the filter size _Si can be determined.

It can be understood that the original image may contain one or more pixels (also called image pixels), and each pixel (each image pixel) may correspond to a coordinate, where the coordinates here may refer to The coordinates in the coordinate system established based on the original image, for example, can use an image vertex of the original image as the origin of the coordinates, and each of the two image edges with the origin of the coordinates as the intersection as a coordinate axis (which can be called (x-axis and y-axis), from which a coordinate system with the image vertex as the coordinate origin can be constructed, then each pixel point can correspond to a coordinate in the coordinate system. The coordinates corresponding to each pixel point can be called pixel coordinates. Then each image pixel (pixel point) can form an image pixel set (pixel point set), and the pixel coordinates corresponding to each image pixel can form a pixel coordinate set. When filtering the original image, the image pixel set of the original image and the pixel coordinate set corresponding to the image pixel set can be obtained.

It can be seen from the above embodiment corresponding to Figure 5 that in low-pass filtering processing (such as mean filtering processing), based on a filter size (such as the size of 3×3), a certain image pixel (such as the pixel of the image to be processed) can be determined ) of the neighborhood image pixels. Among them, the specific way to determine the pixels of the neighborhood image based on the filter size can be determined by the pixel coordinates. One filter size can correspond to a coordinate change amount. For example, the size of 3×3 can correspond to the coordinate change amount [-1, 1] ( That is, increase -1, 0 or 1 on the x-axis and y-axis at the same time); for example, the size of 3×3 can correspond to the coordinate change amount [-2, 2] (that is, increase -2 on the x-axis and y-axis at the same time, -1, 0, 1 or 2). Then after obtaining the pixel coordinates to be processed corresponding to the pixels of the image to be processed, the neighborhood pixel coordinates can be calculated based on the pixel coordinates to be processed and the coordinate change, and the neighborhood pixel coordinates can be determined based on the coordinates of the pixels to be processed. Get the neighborhood image pixels of the image pixel to be processed. The area composed of the neighborhood image pixels and the to-be-processed image pixels is the area covered by the filter size, and the area has the to-be-processed image pixel as the center position.

Then, based on the coordinates of the pixel to be processed and the coordinates of the neighborhood pixels, the filtered image corresponding to the filter size can be determined. Here, the low-pass filtering process is taken as the mean filtering process as an example. According to the pixel coordinates to be processed and the neighborhood pixel coordinates, the specific implementation method of determining the filtered image _Ti corresponding to the filter size _Si can be: obtaining the neighborhood in the image pixel set The neighborhood image pixel corresponding to the pixel coordinate; the neighborhood pixel value of the neighborhood image pixel can be obtained, and the pixel value of the image pixel to be processed can be obtained; subsequently, the neighborhood pixel value can be compared with the pixel value of the image pixel to be processed Add processing to obtain the pixel operation value; determine the ratio between the pixel operation value and the total number of pixels as the updated pixel value corresponding to the image pixel to be processed; where the total number of pixels is the number of neighborhood image pixels and the number of image pixels to be processed The sum of the numbers; when the updated pixel value corresponding to each image pixel in the image pixel set is determined, the image containing the updated pixel value corresponding to each image pixel can be determined as the filter corresponding to the filter size _Si Image _Ti .

That is to say, the embodiment of the present application can perform the pixel value (including the pixel value of the image pixel to be processed and the neighbor pixel value) of each pixel (including the pixel value of the image pixel to be processed and the neighbor image pixel) covered by the filter size. After the addition process, the obtained pixel operation value is then averaged (that is, the ratio between the pixel operation value and the total number of pixels is determined), and the average value can be used as the updated pixel value of the pixel of the image to be processed (that is, the average value replaces the pixel to be processed). The original pixel value of the processed image pixel, that is, the pixel value that replaces the pixel of the image to be processed). For a detailed description of the exemplary scenario, please refer to the description of the scenario example in the embodiment corresponding to Figure 5 above. It should be understood that for each image pixel in the original image, the updated pixel value of each image pixel can be determined by determining the updated pixel value of the image pixel to be processed. Then, after determining the updated pixel value of each image pixel, pixel value, it can be considered that the process of low-pass filtering of the original image based on the filter size is completed. At this time, the image containing each updated pixel value can be determined as the filtered image corresponding to the filter size (such as filter size Si ₎ . (Filtered image _Ti ).

The above only takes the filter size S _i as an example to describe the low-pass filtering process on the original image based on a certain filter size. For each of the N filter sizes, the same processing method can be used to perform the original image. Through low-pass filtering (such as mean filtering), filtered images corresponding to different filter sizes can be obtained, that is, N filtered images can be obtained.

For ease of understanding, please refer to formula (1), formula (2) and formula (3) together. Formula (1), formula (2) and formula (3) are based on N filter sizes including filter sizes 5×5, Taking 9×9 and 17×17 as examples, the specific implementation method of performing mean filtering on the original image.

Among them, in formula (1) It can represent the low-frequency pixel value of a certain pixel in the original image after performing mean filtering on the original image based on the filter size 5×5 (it can also be understood as the updated pixel value after filtering); Δx and Δy can be respectively Used to characterize the coordinate changes on the x-axis and the coordinate changes on the y-axis; Processing pixel coordinates), that is, the pixel coordinates to be processed are (I _{x, y} ). Specifically, for each pixel (image pixel) with coordinate position (x, y) in the original image, when performing mean filtering on it, its surroundings can be determined based on the filter size (template size 5×5) neighborhood image pixels, and then the average of its own pixel value and the pixel value of its neighboring image pixels can be calculated. This average value can be used as the pixel of the image pixel in the filtered image corresponding to the filter size of 5×5. value.

Among them, in formula (2) It can represent the low-frequency pixel value of a certain pixel in the original image after performing mean filtering on the original image based on the filter size 9×9 (it can also be understood as the updated pixel value after filtering); Δx and Δy can be respectively Used to characterize the coordinate changes on the x-axis and the coordinate changes on the y-axis; Processing pixel coordinates), that is, the pixel coordinates to be processed are (I _{x, y} ). Specifically, for each pixel (image pixel) with coordinate position (x, y) in the original image, when performing mean filtering on it, its surroundings can be determined based on the filter size (template size 9×9) Neighborhood image pixels, and then the average of its own pixel value and the pixel value of its neighboring image pixels can be calculated. This average value can be used as the pixel of the image pixel in the filtered image corresponding to the filter size of 9×9. value.

Among them, in formula (3) It can represent the low-frequency pixel value of a certain pixel in the original image after performing mean filtering on the original image based on the filter size 17×17 (it can also be understood as the updated pixel value after filtering); Δx and Δy can be respectively Used to characterize the coordinate changes on the x-axis and the coordinate changes on the y-axis; Processing pixel coordinates), that is, the pixel coordinates to be processed are (I _{x, y} ). Specifically, for each pixel (image pixel) with coordinate position (x, y) in the original image, when performing mean filtering on it, its surroundings can be determined based on the filter size (template size 17×17) Neighborhood image pixels, and then the average of its own pixel value and the pixel value of its neighboring image pixels can be calculated. This average value can be used as the pixel of the image pixel in the filtered image corresponding to the filter size of 17×17. value.

It should be understood that through the above formula (1), a filtered image corresponding to the original image under a low filter size can be obtained; through the above formula (2), a filtered image corresponding to the original image under a medium filter size can be obtained; through the above formula (3), a filtered image corresponding to the original image under high filter size can be obtained.

S103: Perform image conversion on N filtered images respectively according to the original image to obtain N high-frequency images.

In this application, it can be seen from the above that low-pass filtering can be performed on the original image based on each filter size. Specifically, the information obtained after low-pass filtering is low-frequency information (that is, each filtered image can be understood as a low-frequency image) , after obtaining the low-frequency information, the high-frequency information can be extracted based on the original image and low-frequency information (the high-frequency information can be understood as a high-frequency image). In a possible implementation, the method of extracting high-frequency information may be to perform a difference between the original image and the low-frequency information, and use the obtained result as high-frequency information. Let each of the N filter sizes be a filter size _Si , take the filtered image corresponding to the filter size _Si as the filtered image Ti, and take the high-frequency image corresponding to the filtered _image _Ti as the high-frequency image Z _i (i is a positive integer). For image conversion based on the original image filtered image T _i , the specific implementation method of obtaining the high-frequency image Z _i can be: obtaining the image pixel set corresponding to the original image, and obtaining the pixel coordinates corresponding to the image pixel set. set; then, obtain the filtered image pixel set corresponding to the filtered image T _i , and obtain the filtered pixel coordinate set corresponding to the filtered image pixel set; next, according to the pixel coordinate set and the filtered pixel coordinate set, the corresponding filtered image T _i can be determined The high-frequency image Z _i .

It can be understood from the above that the filtered image can be obtained by updating the pixel value of each pixel of the original image. In fact, compared with the original image, the coordinates of the pixels in the filtered image have not changed. However, the pixel value of each pixel may change, so the filtered image pixel set of the filtered image here can be the same pixel set as the image pixel set of the original image, and the filtered pixel coordinate set corresponding to the filtered image pixel set can also be The set of pixel coordinates corresponding to the set of image pixels can be the same set of pixels. That is to say, each filtered pixel coordinate in the filtered pixel coordinate set will correspond to a pixel coordinate in a pixel coordinate set (the two are the same coordinates). According to the pixel coordinate set and the filtered pixel coordinate set, the high-frequency image corresponding to a certain filtered image can be determined.

Taking the filtered image _Ti as an example, the specific implementation method for determining the high-frequency image Z _i corresponding to the filtered image _Ti according to the pixel coordinate set and the filtered pixel coordinate set can be: obtaining the filtered pixel coordinates to be processed in the filtered pixel coordinate set , and determine the pixel coordinates in the pixel coordinate set that have a mapping relationship with the filtered pixel coordinates to be processed as the mapped pixel coordinates; then, the mapped image pixels corresponding to the mapped pixel coordinates can be obtained in the image pixel set, and in the filtered image pixel set You can obtain the filtered pixel to be processed corresponding to the coordinates of the filtered pixel to be processed; obtain the mapped pixel value corresponding to the mapped image pixel, and obtain the filtered pixel value corresponding to the filtered pixel to be processed; obtain the difference pixel between the mapped pixel value and the filtered pixel value value, determined as the high-frequency pixel value corresponding to the filtered pixel to be processed; when the filtered image pixel set is determined When the high-frequency pixel value corresponding to each filtered image pixel in the filtered image is determined, the image containing the high-frequency pixel value corresponding to each filtered image pixel can be determined as the high-frequency image _{Z i} _{corresponding} to the filtered image Ti.

It can be understood that the pixel coordinates that have a mapping relationship with the filtered pixel coordinates to be processed can actually be understood as the pixel coordinates in the pixel coordinate set that are the same coordinates as the filtered pixel coordinates to be processed. From the above, it can be seen that the pixel coordinate set and the filtered pixel The coordinate set is actually the same coordinate set. Each pixel coordinate in the pixel coordinate set has the same coordinate in the filtered pixel coordinate set. These two same coordinates can be considered to have a mapping relationship and are actually the same pixel point. coordinate of. The mapped pixel value corresponding to the pixel of the mapped image can be understood as the original pixel value without filtering in the original image (such as the target pixel value corresponding to the above-mentioned image pixel to be processed), and the filtered pixel value corresponding to the filtered image pixel can be understood as the original pixel value. The pixel value of the image after filtering (for example, when the filtered image pixel is the above-mentioned image pixel to be processed, the filtered pixel value may refer to the updated pixel value corresponding to the image pixel to be processed).

It should be understood that for each pixel (such as the filtered pixel to be processed or the mapped image pixel), the updated pixel value (ie, the filtered pixel value) and the original pixel value (such as the mapped pixel value) can be used to make a difference, and the resulting difference The value result can be used as the high-frequency information corresponding to the pixel (i.e., the high-frequency pixel value). When the high-frequency pixel value corresponding to each pixel is determined, a high-frequency image containing each high-frequency pixel value can be obtained.

For ease of understanding, please refer to formula (4), formula (5) and formula (6) together. Formula (4), formula (5) and formula (6) are based on N filter sizes, including filter sizes 5×5, Taking 9×9 and 17×17 as examples, the specific implementation method of extracting high-frequency information.

Among them, I _{x, y} as shown in formula (4) can be used to characterize the original pixel value corresponding to the pixel point I _{x, y} at the position (x, y) in the original image; It can be used to characterize the filtered pixel value corresponding to the pixel point I _{x, y} determined based on the above formula (1); It can represent the high-frequency pixel value corresponding to the pixel point I _{x, y} . Specifically, for each pixel point (image pixel) with coordinate position (x, y) in the original image, or for each pixel point (image pixel) with coordinate position (x, y) in the filtered image, When extracting high-frequency information, the original pixel value and its filtered pixel value can be difference processed to obtain the high-frequency pixel value of the pixel. When the high-frequency pixel value corresponding to each pixel is obtained, a high-frequency image containing each high-frequency pixel value can be obtained. The high-frequency image as shown in equation (4) can correspond to the filter size 5×5.

Among them, I _{x, y} as shown in formula (5) can be used to characterize the original pixel value corresponding to the pixel point I _{x, y} at the position (x, y) in the original image; It can be used to characterize the filtered pixel value corresponding to the pixel point I _{x, y} determined based on the above formula (2); It can represent the high-frequency pixel value corresponding to the pixel point I _{x, y} . Specifically, for each pixel point (image pixel) with coordinate position (x, y) in the original image, or for each pixel point (image pixel) with coordinate position (x, y) in the filtered image, When extracting high-frequency information, the original pixel value and its filtered pixel value can be difference processed to obtain the high-frequency pixel value of the pixel. When the high-frequency pixel value corresponding to each pixel is obtained, a high-frequency image containing each high-frequency pixel value can be obtained. The high-frequency image as shown in equation (5) may correspond to the filter size 9×9.

Among them, I _{x, y} as shown in formula (6) can be used to characterize the original pixel value corresponding to the pixel point I _{x, y} at the position (x, y) in the original image; It can be used to characterize the filtered pixel value corresponding to the pixel point I _{x, y} determined based on the above formula (3); It can represent the high-frequency pixel value corresponding to the pixel point I _{x, y} . Specifically, for each pixel point (image pixel) with coordinate position (x, y) in the original image, or for each pixel point (image pixel) with coordinate position (x, y) in the filtered image, When extracting high-frequency information, the original pixel value and its filtered pixel value can be difference processed to obtain the high-frequency pixel value of the pixel. When the high-frequency pixel value corresponding to each pixel is obtained, a high-frequency image containing each high-frequency pixel value can be obtained. The high-frequency image as shown in equation (6) can correspond to the filter size 17×17.

It should be understood that the high-frequency information obtained through small-scale low-pass filtering (such as mean filtering with a smaller filter size) is relatively weak, while through large-scale low-pass filtering (such as mean filtering with a larger filter size) The high-frequency information obtained by processing) is relatively strong. For sharp textures in the original image (image information with complex content, dramatic changes, and complex content, such as the texture of grass and trees), small-scale low-pass filtering can extract the corresponding high-frequency information, while using large-scale low-pass filtering Filtering is likely to result in excessive enhancement of sharp textures. For gentle textures in the original image (image information with simple content and gentle changes, such as the texture of the sky), large-scale low-pass filtering is required to extract the corresponding high-frequency information. Then different filters from small to large are needed. size, after performing low-pass filtering on the original image, low-pass filtering of different scales can be used to accurately and targetedly obtain different types of high-frequency information in the original image, which can greatly improve the quality of the extracted high-frequency information. Accuracy and comprehensiveness can improve the pertinence of the sharpening process when subsequent image sharpening and enhancement processing is performed based on high-frequency information, thus improving the image quality after sharpening and enhancement.

S104, perform image fusion on N high-frequency images to obtain a fused image, and fuse the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image.

In this application, the extracted N high-frequency images under different filter sizes can be image fused. There are many ways of image fusion. In one possible implementation, this application mainly uses geometric mean image fusion. The N filter sizes include a first filter size and a second filter size, the N filter images include a first filter image corresponding to the first filter size and a second filter image corresponding to the second filter size, and the N high-frequency images include a Taking the first high-frequency image corresponding to a filtered image and the second high-frequency image corresponding to the second filtered image as an example, a specific implementation method for image fusion of the first high-frequency image and the second high-frequency image to obtain the fused image can be is: the first fusion weight corresponding to the first filter size can be obtained, and the second fusion weight corresponding to the second filter size can be obtained; then, the high-frequency image fusion function can be obtained, according to the first fusion weight, the second fusion weight and the high-frequency The image fusion function can fuse the first high-frequency image with the second high-frequency image, thereby obtaining the fused image.

Among them, the specific implementation method for image fusion of the first high-frequency image and the second high-frequency image according to the first fusion weight, the second fusion weight and the high-frequency image fusion function to obtain the fused image can be: according to the high-frequency The image fusion function adds the first fusion weight and the second fusion weight to obtain the operation weight; the first ratio between the first fusion weight and the operation weight can be determined, and the first high-frequency image is processed based on the first ratio. Exponential power operation can be used to obtain the first operation feature; the second ratio between the second fusion weight and the operation weight can be determined, and the second high-frequency image can be subjected to exponential power operation based on the second ratio to obtain the second operation feature; according to The high-frequency image fusion function can geometrically fuse the first operation feature and the second operation feature to obtain a fused image.

For ease of understanding, taking the filter sizes of 5×5, 9×9, and 17×17 as an example, the high-frequency images may include high-frequency images corresponding to the filter sizes of 5×5, 9×9, and 17×17 respectively. Please also refer to formula (7). Formula (7) is a method based on geometric mean fusion, which fuses N high-frequency images to obtain the specific implementation method of the fused image:

Among them, the formula (7) can be used to characterize the high-frequency image fusion function; α can be used to characterize the weight parameter corresponding to the filter size 5×5 (when the filter size is the first filter size, the weight parameter can be called is the first fusion weight); β can be used to characterize the weight parameter corresponding to the filter size 9×9 (when it is the second filter size, the weight parameter can be called the second fusion weight); γ can be used to characterize the filter size The weight parameter corresponding to 17×17 (when it is the second filter size, the weight parameter can be called the second fusion weight). The values of α, β and γ in this application can be 0.3, 0.4 and 0.3 respectively. Of course, the value of this parameter is not limited to this. Here is just an example to illustrate a set of reasonable parameter values. This application has It is not limiting. It can be used to characterize the high-frequency pixel value of a pixel at a certain position (x, y) in the high-frequency image corresponding to the filter size 5×5. When the corresponding high-frequency image is the first high-frequency image, and α is used When characterizing the first fusion weight, It can be used to characterize the first ratio; It can be used to characterize the high-frequency pixel value of a pixel at a certain position (x, y) in the high-frequency image corresponding to the filter size 9×9. When the corresponding high-frequency image is the second high-frequency image, and β is used When characterizing the second fusion weight, It can be used to characterize the second ratio; It can be used to characterize the high-frequency pixel value of a pixel at a certain position (x, y) in the high-frequency image corresponding to the filter size 17×17. When the corresponding high-frequency image is the second high-frequency image, and γ is used When characterizing the second fusion weight, It can be used to characterize the second ratio. It can be used to characterize the fused pixel value of the pixel at position (x, y) obtained after fusing the high-frequency pixel values in each high-frequency image. When the fused pixel value of the pixel point at each position is determined, a fused image containing each fused pixel value (that is, the fused high-frequency information) can be obtained. That is to say, the geometric fusion in this application may refer to performing the operation processing (such as multiplication operation processing) on the first operation feature and the first operation feature as shown in formula (7), and the result obtained after the operation processing is is the fusion result obtained after geometric fusion.

When the fused image is obtained, the fused high-frequency information can be added to the original image, that is, the fused image and the original image can be fused. Thus, the high-frequency information in the original image can be enhanced to obtain the corresponding original image. Sharpen the image. The specific implementation method of fusing the fused image with the original image can be shown in formula (8):

Among them, I _{x, y} can be used to characterize the original pixel value of the pixel point with position coordinates (x, y) in the original image; It can be used to represent the fused pixel value of the pixel whose position coordinates are (x, y); I′ _{x, y} can be used to represent the fused high-frequency pixel value. The sharpening enhancement pixel value (also called sharpening pixel value). When the sharpened pixel value of the pixel point at each position is determined, a sharpened image containing each sharpened pixel value can be obtained. That is to say, when a certain remapped pixel is obtained, the original image pixel (or image pixel) that has a mapping relationship with the remapped pixel can be obtained in the original image, where having a mapping relationship can mean With the relationship of the same pixel coordinates, and then adding the pixel values of the two pixels, the sharpened pixel value of the pixel at the coordinates can be obtained. That is: add the remapped pixel value at the same position coordinate to the original image pixel value (or image pixel value) to obtain the sharpened pixel value of the pixel at the position coordinate, and then obtain the sharpened pixel value at each position coordinate. When you sharpen pixel values, you get a sharpened image that contains each sharpened pixel value.

In the embodiment of the present application, for a certain original image, low-pass filtering of different scales can be used to extract high-frequency information of different types of textures (that is, low-pass filtering is performed on the original image based on different filter sizes to obtain different After filtering the images, high-frequency images corresponding to different filtered images are obtained based on the original image and each filtered image), and after fusing the high-frequency information, the fused high-frequency information (i.e., the fused image) can be obtained, After fusing the fused image with the original image, a sharpened enhanced image can be obtained. The sharpened enhanced image is obtained by reasonably enhancing both the gentle texture and the sharp texture in the original image based on different scales, so the sharpened enhanced image has high image quality. In summary, this application can improve the quality of sharpened images in the image sharpening business.

From the above, it can be seen that after image fusion of N high-frequency images, the fused image can be obtained (the fused pixel value corresponding to each pixel point is obtained), and then the fused image can be fused with the original image (that is, for each pixel point , add the fused pixel value to the original pixel value) to obtain a sharpened enhanced image. In a feasible embodiment, in order to further improve the accuracy and rationality of the fused high-frequency information (ie, fused pixel values), it can be linearly remapped and truncated to obtain the processed fused pixel values. Finally, the remaining original pixel values are fused. For ease of understanding, please refer to Figure 6 as well. Figure 6 is a schematic flowchart of fusing the original image and the fused image to obtain a sharpened enhanced image provided by an embodiment of the present application. This process may also correspond to the process of fusing the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image in the embodiment corresponding to FIG. 4 . As shown in Figure 6, the process may include at least the following S201-S202:

S201: Perform remapping processing on the fused image to obtain a remapped fused image.

The remapping fusion image may refer to an image containing a remapping pixel value, and the remapping pixel value may be determined by comparing the fusion pixel value corresponding to the fusion image with a pixel value threshold according to the remapping function.

In fact, every pixel in the fused image is the same as every pixel in the original image, but the pixel value of each pixel may be different. For a certain pixel in the original image, its unprocessed pixel value can be called the original pixel value, the pixel value in a certain filtered image can be called the filtered pixel value, and the pixel value in a certain high-frequency image can be called the filtered pixel value. The pixel values can be called high-frequency pixel values, and the pixel values in the fused image can be called fused pixel values. Based on this, a possible implementation method of performing remapping processing on the fused image to obtain the remapped fused image can be: obtaining the fused image pixels corresponding to the fused image, and obtaining the fused pixel values corresponding to the fused image pixels; then, you can obtain The remapping function is used to determine the remapping pixel value corresponding to the fusion image pixel based on the remapping function and the fusion pixel value; then, the image containing the remapping pixel value can be determined as the remapping fusion image.

Among them, a possible implementation method for determining the remapped pixel value corresponding to the fused image pixel according to the remapping function and the fused pixel value can be: according to the remapping function, compare the fused pixel value with the pixel value threshold; if the fusion If the pixel value is greater than or equal to the pixel value threshold, the preset pixel parameter can be determined as the remapped pixel value corresponding to the fusion image pixel; if the fusion pixel value is less than the pixel value threshold, the fusion pixel value can be compared with the preset fusion coefficient. Multiplication processing is performed to obtain the remapped pixel values corresponding to the pixels of the fused image.

From the above, it can be seen that the filtered pixel value corresponding to each pixel is determined based on the original pixel value, the high-frequency pixel value is determined based on the filtered pixel value, and the fused pixel value is determined based on the high-frequency pixel value. Then the fused image pixel of the fused image here can be the same pixel as a certain image pixel in the image pixel set of the original image. Pair each pixel After remapping and truncation of the corresponding fused pixel values (which can be referred to as remapping processing), the remapped pixel value corresponding to each pixel can be obtained. According to the remapped pixel value and the original pixel value of the pixel, it can be determined The sharpened pixel value corresponding to this pixel. When the sharpened pixel value of each pixel is determined, a sharpened enhanced image containing each sharpened pixel value can be obtained.

For ease of understanding, please also refer to formula (9). Formula (9) is a specific implementation method for remapping the fused image to obtain the remapping fused image.

Among them, the function shown in formula (9) can be used to characterize the remapping function; It can be used to characterize the fused pixel value of a pixel whose position coordinates are (x, y); It can be used to characterize the remapped pixel value of the pixel whose position coordinates are (x, y); 0.25 can be used to characterize the pixel value threshold, which can be an artificially specified value (0.25 is only used as an example here. In fact, The pixel value threshold can be any other reasonable value, which is not limited in this application). 0.8 can be a preset fusion coefficient, and the preset fusion coefficient can be an artificially prescribed value (0.8 is only used as an example here. In fact, the preset fusion coefficient can be any other reasonable value, which is not limited in this application) . When the fusion pixel value is lower than the pixel value threshold, the preset fusion coefficient can be multiplied by the fusion pixel value, and the result can be used as the sharpened pixel value; when the fusion pixel value is greater than or equal to the pixel value threshold, that is The default pixel parameter 0.2 can be used as the sharpening pixel value. Among them, the preset pixel parameters can also be other reasonable values, and 0.2 is only one of the reasonable values and is described as an example. Through the above formula (9), the remapped pixel value corresponding to each pixel can be obtained.

S202, fuse the remapping fusion image with the original image to obtain a sharpened enhanced image.

One possible implementation method of fusing the remapping fusion image with the original image to obtain a sharpened enhanced image may be: obtaining the remapping pixels corresponding to the remapping fusion image, and obtaining the remapping pixel values corresponding to the remapping pixels; and then , you can obtain the image pixels corresponding to the original image, and obtain the image pixel values corresponding to the image pixels; add the remapped pixel values to the image pixel values to obtain the sharpened pixel values; you can convert the image containing the sharpened pixel values, Confirm to sharpen the image.

It can be understood that after obtaining the remapped pixel value corresponding to each pixel point, the remapped high-frequency information (ie, the remapped pixel value) can be added to the original image, that is, the remapped fusion image and The original images are fused, whereby the high-frequency information in the original images can be enhanced to obtain a sharpened enhanced image corresponding to the original image. That is: the remapped pixel value corresponding to each pixel point (i.e., image pixel) can be added to the original pixel value, so that the sharpened pixel value corresponding to each image pixel can be obtained, and then the sharpened pixel value corresponding to each image pixel can be obtained. Sharpen the image by sharpening the pixel values.

For the specific implementation method of determining the sharpening enhanced image based on the remapped pixel value and the original pixel value, the specific implementation method can be as shown in formula (10):

Among them, I _{x, y} can be used to characterize the original pixel value of the pixel point with position coordinates (x, y) in the original image; It can be used to represent the remapped pixel value of the pixel whose position coordinates are (x, _y ); I′ ). When the sharpened pixel value of the pixel point at each position is determined, a sharpened image containing each sharpened pixel value can be obtained.

In the embodiment of the present application, after low-pass filtering is performed on the original image through different filter sizes, low-frequency images (ie, N filtered images) under different filter sizes can be obtained, and then the original image itself is combined with each filtered image, that is, The corresponding high-frequency information of each filtered image can be extracted (that is, N high-frequency images are obtained). For multi-scale high-frequency images, this application can fuse them to obtain a fused image. The processed fused image can be combined with The original images are fused again, so that the high-frequency intensity of the original images can be enhanced from different scales (filter sizes) to obtain a sharpened enhanced image. In addition, since this application uses different filter sizes to perform low-pass filtering on the original image at the same time, the high-frequency information obtained is also high-frequency information under different filter sizes, which can be used to filter different types of image details (such as smooth textures and complex sharp ones). Texture) has strong adaptive ability (for example, for complex and sharp textures, low-pass filtering based on low filter size can extract the corresponding high-frequency information and achieve corresponding enhancement; for smooth textures, low-pass filtering based on high filter size can The corresponding high-frequency information can be extracted through filtering and the corresponding enhancement can be achieved), that is, the detailed information of the original image can be enhanced from different scales, thereby improving the sharpening quality of the image and improving the clarity of the image. In summary, this application can improve the quality of sharpened images in the image sharpening business.

Please refer to FIG. 7 , which is a schematic structural diagram of a data processing device provided by an embodiment of the present application. The data processing device may be a computer program (including program instructions) running in a computer device, for example, the data processing device may be an application software; the data processing device may be used to execute the method shown in FIG. 4 . As shown in FIG. 7 , the data processing device 1 may include: a size acquisition module 11 , a filtering module 12 , an image conversion module 13 , an image fusion module 14 and an image sharpening module 15 .

The size acquisition module 11 is used to obtain a filter size set for filtering processing; the filter size set includes N filter sizes, and N is a positive integer greater than 1; any two filter sizes among the N filter sizes are different;

The filter module 12 is used to perform low-pass filtering on the original image based on each filter size to obtain N filtered images;

The image conversion module 13 is used to perform image conversion on N filtered images respectively according to the original image to obtain N high-frequency images;

The image fusion module 14 is used to image fuse N high-frequency images to obtain a fused image;

The image sharpening module 15 is used to fuse the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image.

Among them, for the specific implementation of the size acquisition module 11, the filtering module 12, the image conversion module 13, the image fusion module 14 and the image sharpening module 15, please refer to the description of S101-S104 in the embodiment corresponding to the above-mentioned Figure 4, which will not be mentioned here. Let’s go into details.

In one embodiment, each of the N filter sizes is taken as a filter size _Si , and the filtered image corresponding to the filter size _Si is taken as a filtered image _Ti , i is a positive integer;

The filtering module 12 may include: a set acquisition unit 121, a coordinate acquisition unit 122, a neighborhood coordinate determination unit 123, and a filtered image determination unit 124.

The set acquisition unit 121 is used to acquire the image pixel set corresponding to the original image, and to acquire the pixel coordinate set corresponding to the image pixel set;

The coordinate acquisition unit 122 is used to acquire the image pixels to be processed in the image pixel set, and acquire the pixel coordinates to be processed corresponding to the image pixels to be processed in the pixel coordinate set;

The neighborhood coordinate determination unit 123 is used to obtain the coordinate change amount indicated by the filter size _Si , and determine the neighborhood pixel coordinates for the pixel coordinates to be processed in the pixel coordinate set according to the pixel coordinates to be processed and the coordinate change amount;

_The filtered image determination unit 124 is used to determine the filtered image Ti corresponding to the filter size _Si according to the pixel coordinates to be processed and the neighbor pixel coordinates.

For the specific implementation of the set acquisition unit 121, the coordinate acquisition unit 122, the neighborhood coordinate determination unit 123, and the filtered image determination unit 124, please refer to the description of S102 in the embodiment corresponding to Figure 4 above, and will not be described again here.

In one embodiment, the filtered image determination unit 124 may include: a pixel operation sub-unit 1241 and a pixel update sub-unit 1242.

The pixel operation subunit 1241 is used to obtain the neighborhood image pixels corresponding to the neighborhood pixel coordinates in the image pixel set;

The pixel operation subunit 1241 is also used to obtain the neighborhood pixel value of the neighborhood image pixel, and obtain the pixel value of the image pixel to be processed;

The pixel operation subunit 1241 is also used to add the neighborhood pixel value and the pixel value of the image pixel to be processed to obtain a pixel operation value;

Pixel update subunit 1242 is used to determine the ratio between the pixel operation value and the total number of pixels as the updated pixel value corresponding to the image pixel to be processed; the total number of pixels is the number of neighborhood image pixels and the number of image pixels to be processed. Sum;

The pixel update subunit 1242 is also used to determine the updated pixel value corresponding to each image pixel in the image pixel set, and determine the image containing the updated pixel value corresponding to each image pixel to be corresponding to the filter size _Si filtered image _Ti .

For the specific implementation of the pixel operation sub-unit 1241 and the pixel update sub-unit 1242, please refer to the description of S102 in the embodiment corresponding to Figure 4 above, and will not be described again here.

In one embodiment, each of the N filter sizes is taken as a filter size _Si , the filtered image corresponding to the filter size _Si is taken as the filtered image Ti, _and the high-frequency image corresponding to the filtered image _Ti is taken as As the high-frequency image Z, i is a positive integer;

The image conversion module 13 may include: a pixel coordinate acquisition unit 131 and a high-frequency image determination unit 132.

The pixel coordinate acquisition unit 131 is used to acquire the image pixel set corresponding to the original image, and acquire the pixel coordinate set corresponding to the image pixel set;

The pixel coordinate acquisition unit 131 is also used to acquire the filtered image pixel set corresponding to the filtered image _Ti , and to acquire the filtered pixel coordinate set corresponding to the filtered image pixel set;

The high-frequency image determination unit 132 is used to determine the high-frequency image _Zi corresponding to the filtered image _Ti according to the pixel coordinate set and the filtered pixel coordinate set.

For the specific implementation of the pixel coordinate acquisition unit 131 and the high-frequency image determination unit 132, please refer to the relevant description of determining the high-frequency image in S103 in the embodiment corresponding to FIG. 4, and will not be described again here.

In one embodiment, the high-frequency image determination unit 132 may include: a high-frequency pixel value determination sub-unit 1321 and a high-frequency image determination sub-unit 1322.

The high-frequency pixel value determination subunit 1321 is used to obtain the filtered pixel coordinates to be processed in the filtered pixel coordinate set, and determine the pixel coordinates in the pixel coordinate set that have a mapping relationship with the filtered pixel coordinates to be processed as mapped pixel coordinates;

The high-frequency pixel value determination subunit 1321 is also used to obtain the mapped image pixels corresponding to the mapped pixel coordinates in the image pixel set, and to obtain the to-be-processed filtered pixels corresponding to the filtered pixel coordinates to be processed in the filtered image pixel set;

The high-frequency pixel value determination subunit 1321 is also used to obtain the mapped pixel value corresponding to the mapped image pixel, and to obtain the filtered pixel value corresponding to the filtered pixel to be processed;

The high-frequency pixel value determination subunit 1321 is also used to determine the difference pixel value between the mapped pixel value and the filtered pixel value as the high-frequency pixel value corresponding to the filtered pixel to be processed;

The high-frequency image determination subunit 1322 is used to determine the image containing the high-frequency pixel value corresponding to each filtered image pixel in the filtered image pixel set when the high-frequency pixel value corresponding to each filtered image pixel in the filtered image pixel set is determined. is the high-frequency image _Zi corresponding to the filtered image _Ti .

For the specific implementation of the high-frequency pixel value determination sub-unit 1321 and the high-frequency image determination sub-unit 1322, please refer to the description in S103 in the embodiment corresponding to Figure 4 above, and will not be described again here.

In one embodiment, the N filter sizes include a first filter size and a second filter size, and the N filter images include a first filter image corresponding to the first filter size and a second filter image corresponding to the second filter size. N The high-frequency image includes a first high-frequency image corresponding to the first filtered image and a second high-frequency image corresponding to the second filtered image;

The image fusion module 14 may include: a weight fusion unit 141 and a high-frequency image fusion unit 142.

The weight fusion unit 141 is used to obtain the first fusion weight corresponding to the first filter size, and obtain the second fusion weight corresponding to the second filter size;

High-frequency image fusion unit 142, used to obtain a high-frequency image fusion function;

The high-frequency image fusion unit 142 is also configured to image-fuse the first high-frequency image and the second high-frequency image according to the first fusion weight, the second fusion weight, and the high-frequency image fusion function to obtain a fused image.

For the specific implementation of the weight fusion unit 141 and the high-frequency image fusion unit 142, please refer to the relevant description of image fusion in S104 in the embodiment corresponding to Figure 4, and will not be described again here.

In one embodiment, the high-frequency image fusion unit 142 may include: a weight operation sub-unit 1421, an image operation sub-unit 1422, and a feature fusion sub-unit 1423.

The weight operation subunit 1421 is used to add the first image fusion weight and the second image fusion weight according to the high-frequency image fusion function to obtain the operation weight;

The image operation subunit 1422 is used to determine the first ratio between the first fusion weight and the operation weight, and perform an exponential power operation on the first high-frequency image based on the first ratio to obtain the first operation feature;

The image operation subunit 1422 is also used to determine the second ratio between the second fusion weight and the operation weight, and perform exponential power operation on the second high-frequency image based on the second ratio to obtain the second operation feature;

The feature fusion subunit 1423 is used to geometrically fuse the first operation feature and the second operation feature according to the high-frequency image fusion function to obtain a fused image.

For the specific implementation of the weight operation sub-unit 1421, the image operation sub-unit 1422 and the feature fusion sub-unit 1423, please refer to the description of S104 in the embodiment corresponding to Figure 4 above, and will not be described again here.

In one embodiment, the image sharpening module 15 may include: a remapping unit 151 and an image sharpening unit 152.

The remapping unit 151 is used to perform remapping processing on the fused image to obtain a remapped fused image;

The image sharpening unit 152 is used to fuse the remapped fusion image with the original image to obtain a sharpened enhanced image.

For the specific implementation of the remapping unit 151 and the image sharpening unit 152, please refer to the relevant description of S201-S202 in the embodiment corresponding to FIG. 7, and will not be described again here.

In one embodiment, a remapped fusion image refers to an image containing remapped pixel values. The remapped pixel values are determined by comparing the fused pixel values corresponding to the fused image with a pixel value threshold according to the remapping function.

In one embodiment, the remapping unit 151 may include: a remapping value determination subunit 1511 and a remapping image determination subunit 1512.

The remapping value determination subunit 1511 is used to obtain the fused image pixels corresponding to the fused image, and to obtain the fused pixel values corresponding to the fused image pixels;

The remapping value determination subunit 1511 is also used to obtain the remapping function;

The remapping value determination subunit 1511 is also used to determine the remapping pixel value corresponding to the fused image pixel according to the remapping function and the fused pixel value;

The remapped image determination subunit 1512 is used to determine an image containing remapped pixel values as a remapped fusion image.

For the specific implementation of the remapping value determination subunit 1511 and the remapping image determination subunit 1512, please refer to the description in S201 in the embodiment corresponding to Figure 7 above, and will not be described again here.

In one embodiment, the remapping value determination subunit 1511 is also specifically used to compare the fused pixel value with the pixel value threshold according to the remapping function;

The remapping value determination subunit 1511 is also specifically configured to determine the preset pixel parameter as the remapping pixel value corresponding to the fusion image pixel if the fusion pixel value is greater than or equal to the pixel value threshold;

The remapping value determination subunit 1511 is also specifically configured to multiply the fused pixel value and the preset fusion coefficient to obtain the remapped pixel value corresponding to the fused image pixel if the fused pixel value is less than the pixel value threshold.

In one embodiment, the image sharpening unit 152 may include: a sharpening value determination subunit 1521 and a sharpened image determination subunit 1522.

The sharpening value determination subunit 1521 is used to obtain the remapped pixels corresponding to the remapped fusion image, and to obtain the remapped pixel values corresponding to the remapped pixels;

The sharpening value determination subunit 1521 is also used to obtain the image pixels corresponding to the original image, and to obtain the image pixel values corresponding to the image pixels;

The sharpening value determination subunit 1521 is also used to add the remapped pixel value and the image pixel value to obtain the sharpened pixel value;

The sharpened image determination subunit 1522 is used to determine an image containing sharpened pixel values as a sharpened enhanced image.

For the specific implementation of the sharpening value determination sub-unit 1521 and the sharpened image determination sub-unit 1522, please refer to the description in S202 in the embodiment corresponding to Figure 7 above, and will not be described again here.

In the embodiment of the present application, after low-pass filtering is performed on the original image through different filter sizes, low-frequency images (ie, N filtered images) under different filter sizes can be obtained, and then the original image itself is combined with each filtered image, that is, The corresponding high-frequency information of each filtered image can be extracted (that is, N high-frequency images are obtained). For multi-scale High-frequency images, this application can fuse them to obtain a fused image. The processed fused image can be fused with the original image again, so that the high-frequency intensity of the original image can be enhanced from different scales (filter size) to obtain sharpening enhancement. image. In addition, since this application uses different filter sizes to perform low-pass filtering on the original image at the same time, the high-frequency information obtained is also high-frequency information under different filter sizes, which can be used to filter different types of image details (such as smooth textures and complex sharp ones). Texture) has strong adaptive ability (for example, for complex and sharp textures, low-pass filtering based on low filter size can extract the corresponding high-frequency information and achieve corresponding enhancement; for smooth textures, low-pass filtering based on high filter size can The corresponding high-frequency information can be extracted through filtering and the corresponding enhancement can be achieved), that is, the detailed information of the original image can be enhanced from different scales, thereby improving the sharpening quality of the image and improving the clarity of the image. In summary, this application can improve the quality of sharpened images in the image sharpening business.

Please refer to FIG. 8 , which is a schematic structural diagram of a computer device provided by an embodiment of the present application. As shown in Figure 8, the device 1 in the embodiment corresponding to Figure 7 can be applied to the above computer device 8000. The above computer device 8000 can include: a processor 8001, a network interface 8004 and a memory 8005. In addition, the above computer device 8000 also Including: user interface 8003, and at least one communication bus 8002. Among them, the communication bus 8002 is used to realize connection communication between these components. Among them, the user interface 8003 may include a display screen (Display) and a keyboard (Keyboard), and the optional user interface 8003 may also include a standard wired interface and a wireless interface. The network interface 8004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface). The memory 8005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 8005 may optionally be at least one storage device located remotely from the aforementioned processor 8001. As shown in Figure 8, memory 8005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

In the computer device 8000 shown in Figure 8, the network interface 8004 can provide network communication functions; the user interface 8003 is mainly used to provide an input interface for the user; and the processor 8001 can be used to call the device control application stored in the memory 8005 program to achieve:

It should be understood that the computer device 8000 described in the embodiment of the present application can perform the data processing method described in the embodiment corresponding to FIG. 4 to FIG. 7, and can also perform the data processing method in the embodiment corresponding to FIG. 7. The description of device 1 will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be described again.

In addition, it should be pointed out here that the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the aforementioned data processing computer device 8000, and The above-mentioned computer program includes program instructions. When the above-mentioned processor executes the above-mentioned program instructions, the above-mentioned data processing method described in the embodiment corresponding to FIG. 4 to FIG. 7 can be executed. Therefore, the details will not be described here. Other In addition, the description of the beneficial effects of using the same method will not be described again. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments in this application.

The above-mentioned computer-readable storage medium may be the data processing apparatus provided in any of the foregoing embodiments or the internal storage unit of the above-mentioned computer equipment, such as the hard disk or memory of the computer equipment. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card equipped on the computer device, Flash card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

The terms “first”, “second”, etc. in the description, claims, and drawings of the embodiments of this application are used to distinguish different objects, rather than describing a specific sequence. Furthermore, the term "includes" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally also includes unlisted steps or modules, or optionally also includes Other step units inherent to such processes, methods, apparatus, products or equipment.

Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of both. In order to clearly illustrate the relationship between hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described according to functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

The methods and related devices provided by the embodiments of the present application are described with reference to the method flowcharts and/or structural schematic diagrams provided by the embodiments of the present application. Specifically, each process and/or the method flowcharts and/or structural schematic diagrams can be implemented by computer program instructions. or blocks, and combinations of processes and/or blocks in flowcharts and/or block diagrams. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the structural diagram. These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in one process or multiple processes in the flowchart and/or in one block or multiple blocks in the structural diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart and/or a block or blocks of a structural representation.

What is disclosed above is only the preferred embodiment of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

A data processing method, the method is executed by a computer device, including:

Obtain a filter size set used for filtering processing; the filter size set includes N filter sizes, N is a positive integer greater than 1; any two filter sizes among the N filter sizes are different;

Perform low-pass filtering on the original image based on each filter size to obtain N filtered images;

Perform image conversion on the N filtered images respectively according to the original image to obtain N high-frequency images;

The N high-frequency images are image fused to obtain a fused image, and the fused image is fused with the original image to obtain a sharpened enhanced image corresponding to the original image.
According to the method of claim 1, each of the N filter sizes is used as a filter size Si , and the filtered image corresponding to the filter size Si is used as a filtered image Ti , i is a positive integer. ;

The original image is subjected to low-pass filtering processing based on each filtering size to obtain N filtered images, including:

Obtain a set of image pixels corresponding to the original image, and obtain a set of pixel coordinates corresponding to the set of image pixels;

Obtain the image pixels to be processed in the image pixel set, and obtain the pixel coordinates to be processed corresponding to the image pixels to be processed in the pixel coordinate set;

Obtain the coordinate change amount indicated by the filter size Si, and determine the neighborhood pixel coordinates for the pixel coordinates to be processed in the pixel coordinate set according to the pixel coordinates to be processed and the coordinate change amount;

According to the coordinates of the pixel to be processed and the coordinates of the neighborhood pixels, the filtered image Ti corresponding to the filter size Si is determined.
The method according to claim 2, wherein determining the filtered image Ti corresponding to the filter size Si according to the pixel coordinates to be processed and the neighborhood pixel coordinates includes:

Obtain the neighborhood image pixel corresponding to the neighborhood pixel coordinate in the image pixel set;

Obtain the neighborhood pixel value of the neighborhood image pixel, and obtain the pixel value of the image pixel to be processed;

Add the neighborhood pixel value to the pixel value of the image pixel to be processed to obtain a pixel operation value;

The ratio between the pixel operation value and the total number of pixels is determined as the updated pixel value corresponding to the pixel of the image to be processed; the total number of pixels is the number of pixels in the neighborhood image and the pixels of the image to be processed. The sum of the quantities;

When the updated pixel value corresponding to each image pixel in the image pixel set is determined, the image containing the updated pixel value corresponding to each image pixel is determined to be the corresponding updated pixel value of the filter size Si . Filter image Ti .
The method according to claim 1, using each of the N filter sizes as a filter size Si , using the filtered image corresponding to the filter size Si as a filtered image Ti , and using the filtered The high-frequency image corresponding to image T i is regarded as high-frequency image Z i , i is a positive integer;

The N filtered images are respectively subjected to image conversion according to the original image to obtain N high-frequency images, including:

Obtain a set of image pixels corresponding to the original image, and obtain a set of pixel coordinates corresponding to the set of image pixels;

Obtain a set of filtered image pixels corresponding to the filtered image Ti, and obtain a set of filtered pixel coordinates corresponding to the set of filtered image pixels;

The high-frequency image Zi corresponding to the filtered image Ti is determined according to the pixel coordinate set and the filtered pixel coordinate set.
The method according to claim 4, wherein determining the high-frequency image Zi corresponding to the filtered image Ti according to the pixel coordinate set and the filtered pixel coordinate set includes:

Obtain the filtered pixel coordinates to be processed from the filtered pixel coordinate set, and determine the pixel coordinates in the pixel coordinate set that have a mapping relationship with the filtered pixel coordinates to be processed as mapped pixel coordinates;

Obtain the mapped image pixels corresponding to the mapped pixel coordinates in the image pixel set, and obtain the to-be-processed filtered pixels corresponding to the to-be-processed filtered pixel coordinates in the filtered image pixel set;

Obtain the mapped pixel value corresponding to the mapped image pixel, and obtain the filtered pixel value corresponding to the filtered pixel to be processed;

Determine the difference pixel value between the mapped pixel value and the filtered pixel value as the high-frequency pixel value corresponding to the filtered pixel to be processed;

When the high-frequency pixel value corresponding to each filtered image pixel in the filtered image pixel set is determined, the image containing the high-frequency pixel value corresponding to each filtered image pixel is determined to be the filtered image Ti The corresponding high-frequency image Zi .
The method according to claim 1, the N filter sizes include a first filter size and a second filter size, and the N filter images include a first filter image corresponding to the first filter size and the second filter size. A second filtered image corresponding to the filter size, the N high-frequency images including a first high-frequency image corresponding to the first filtered image and a second high-frequency image corresponding to the second filtered image;

The image fusion of the N high-frequency images to obtain the fused image includes:

Obtain the first fusion weight corresponding to the first filter size, and obtain the second fusion weight corresponding to the second filter size;

Obtain a high-frequency image fusion function, and perform image fusion on the first high-frequency image and the second high-frequency image according to the first fusion weight, the second fusion weight, and the high-frequency image fusion function. , to obtain the fused image.
The method according to claim 6, wherein the first high-frequency image and the second high-frequency image are combined according to the first fusion weight, the second fusion weight and the high-frequency image fusion function. Perform image fusion to obtain the fused image, including:

According to the high-frequency image fusion function, the first fusion weight and the second fusion weight are added together to obtain an operation weight;

Determine a first ratio between the first fusion weight and the operation weight, and perform an exponential power operation on the first high-frequency image based on the first ratio to obtain a first operation feature;

Determine a second ratio between the second fusion weight and the operation weight, and perform an exponential power operation on the second high-frequency image based on the second ratio to obtain a second operation feature;

According to the high-frequency image fusion function, the first operation feature and the second operation feature are geometrically fused to obtain the fused image.
The method according to any one of claims 1 to 7, said fusing the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image includes:

Perform remapping processing on the fused image to obtain a remapped fused image;

The remapped fusion image is fused with the original image to obtain the sharpened enhanced image.
The method according to claim 8, the remapping fusion image refers to an image containing a remapping pixel value, and the remapping pixel value is a fusion pixel value corresponding to the fusion image and a pixel value according to a remapping function. The threshold is determined after comparison.
The method according to claim 9, performing remapping processing on the fused image to obtain a remapped fused image includes:

Obtain the fused image pixels corresponding to the fused image, and obtain the fused pixel values corresponding to the fused image pixels;

Obtain a remapping function, and determine the remapping pixel value corresponding to the fused image pixel based on the remapping function and the fused pixel value;

An image containing the remapped pixel values is determined as the remapped fusion image.
The method according to claim 10, wherein determining the remapping pixel value corresponding to the fused image pixel according to the remapping function and the fused pixel value includes:

Compare the fused pixel value with a pixel value threshold according to the remapping function;

If the fused pixel value is greater than or equal to the pixel value threshold, then determine the preset pixel parameter as the remapped pixel value corresponding to the fused image pixel;

If the fused pixel value is less than the pixel value threshold, the fused pixel value is multiplied by a preset fusion coefficient to obtain a remapped pixel value corresponding to the fused image pixel.
The method according to claim 8, said fusing the remapped fusion image with the original image to obtain the sharpened enhanced image includes:

Obtain the remapped pixel corresponding to the remapped fusion image, and obtain the remapped pixel value corresponding to the remapped pixel;

Obtain the image pixel corresponding to the original image, and obtain the image pixel value corresponding to the image pixel;

Add the remapped pixel value to the image pixel value to obtain a sharpened pixel value;

An image containing the sharpened pixel value is determined as the sharpened enhanced image.
A data processing device, the device is deployed on a computer device, including:

A size acquisition module, used to obtain a filter size set for filtering processing; the filter size set includes N filter sizes, N is a positive integer greater than 1; any two filter sizes among the N filter sizes different;

The filter module is used to perform low-pass filtering on the original image based on each filter size to obtain N filtered images;

An image conversion module, configured to perform image conversion on the N filtered images respectively according to the original image to obtain N high-frequency images;

An image fusion module, used to perform image fusion on the N high-frequency images to obtain a fused image;

An image sharpening module is used to fuse the fused image with the original image to obtain a sharpened enhanced image corresponding to the original image.
A computer device including: a processor, a memory and a network interface;

The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store computer programs, and the processor is used to call the computer program to enable The computer device performs the method of any one of claims 1-12.
A computer-readable storage medium, in which a computer program is stored, and the computer program is adapted to be loaded by a processor and execute the method described in any one of claims 1-12.
A computer program product includes a computer program that implements the method of any one of claims 1-12 when executed by a processor.