CN117635466A

CN117635466A - Image enhancement method, device, electronic equipment and readable storage medium

Info

Publication number: CN117635466A
Application number: CN202410108313.2A
Authority: CN
Inventors: 郑煜伟
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2024-01-26
Filing date: 2024-01-26
Publication date: 2024-03-01

Abstract

The application provides an image enhancement method, an image enhancement device, electronic equipment and a readable storage medium, and relates to the field of terminals, in particular to the field of image enhancement, wherein the method comprises the following steps: and acquiring a feature codebook of the target image, wherein the feature codebook is used for representing global features of the target image. And performing image blocking processing on the target images to obtain image blocks of a plurality of target images. And enhancing each image block according to the feature codebook to obtain enhanced image blocks. And according to the plurality of enhanced image blocks, merging to obtain an enhanced target image. By acquiring global features of the target image, generating a feature codebook, and carrying out block enhancement on the target image according to the feature codebook, the advantages of global enhancement and block enhancement can be simultaneously considered during image enhancement, the probability of blocking effect or artifact of the image after block enhancement is reduced, and the consistency and the integrity of the target image are ensured. The image enhancement effect is improved, and the user experience is improved.

Description

Image enhancement method, device, electronic equipment and readable storage medium

Technical Field

The present disclosure relates to the field of machine learning, and in particular, to an image enhancement method, an image enhancement device, an electronic device, and a readable storage medium.

Background

Image enhancement refers to the process of processing a clear, complete, high-resolution, high-quality image from an image that is affected by various degradation factors. The image enhancement comprises the functions of denoising, deblurring, filling, repairing, supernatural and the like, and has wide application scenes.

At present, image enhancement can be generally divided into full-image enhancement or block enhancement, and the full-image enhancement can consider the whole information, structure and priori of an image, so that the consistency and the integrity of the image are maintained, but the problems of large calculation amount, detail loss, easy excessive smoothing and the like exist. The block enhancement can keep the details and local characteristics of the image, and simultaneously reduce the calculation amount and the memory consumption, but has the problems of easy occurrence of blocking effect, artifact, degradation of enhancement capability and the like.

The existing image enhancement method has various defects, so that the existing image enhancement effect is poor and the user experience is poor.

Disclosure of Invention

The application provides an image enhancement method, an image enhancement device, electronic equipment and a readable storage medium, which can simultaneously consider the advantages of global enhancement and block enhancement during image enhancement, and solve the problems of poor effect and poor user experience of the current image enhancement due to various defects of the current image enhancement method.

In order to achieve the above purpose, the present application adopts the following technical scheme:

in a first aspect, an image enhancement method is provided, including: and acquiring a feature codebook of the target image, wherein the feature codebook is used for representing global features of the target image. And performing image blocking processing on the target images to obtain image blocks of a plurality of target images. And enhancing each image block according to the feature codebook to obtain enhanced image blocks. And according to the plurality of enhanced image blocks, merging to obtain an enhanced target image.

In the embodiment of the application, the image enhancement method can be applied to electronic equipment, such as a server, a cloud server, a mobile phone, a tablet computer, a notebook computer, a personal computer and the like.

In the first aspect, by acquiring global features of the target image and generating a feature codebook and carrying out block enhancement on the target image according to the feature codebook, the advantages of global enhancement and block enhancement can be simultaneously considered during image enhancement, the probability of blocking effect or artifact occurrence of the image after block enhancement is reduced, and the consistency and the integrity of the target image are ensured. The image enhancement effect is improved, and the user experience is improved.

In some possible embodiments, obtaining a feature codebook of a target image includes: the method comprises the steps of inputting a target image and/or a reference image of the target image into a pre-trained global guiding module, wherein the global guiding module comprises a first feature extraction module and a first coding module, and the reference image of the target image is obtained by matching the target image from a preset first database. And extracting global information of the target image and/or a reference image of the target image through a first feature extraction module, and encoding the global information into a feature codebook of the target image through a first encoding module.

In some possible implementations, the global boot module further includes a second encoding module.

Acquiring a feature codebook of a target image, comprising: and matching the target image from a preset first database to obtain corresponding priori information. The prior information is encoded into a feature codebook of the target image by a second encoding module.

In some possible implementations, the global boot module further includes a third encoding module.

Acquiring a feature codebook of a target image, comprising: and acquiring original shooting information of the target image. And encoding the original shooting information into a characteristic codebook of the target image through a third encoding module.

In some possible embodiments, each image partition is enhanced according to a feature codebook, so as to obtain an enhanced image partition, which includes: and acquiring a first enhancement characteristic of the image block through a preset first image enhancement algorithm. And processing the first enhancement features through the matching block according to the feature codebook to obtain second enhancement features. And inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks.

In some possible embodiments, according to the feature codebook, the processing, by the matching block, the first enhancement feature to obtain the second enhancement feature includes: and the matching block matches the feature blocks meeting the preset conditions in the first enhancement features through a preset similarity algorithm according to the global features in the feature codebook. And replacing or fusing the feature blocks according to the global feature coincidence to the preset condition to obtain a second enhancement feature.

In some possible implementations, the first image enhancement algorithm includes a first subnetwork and a second subnetwork.

Acquiring a first enhancement feature of the image block through a preset first image enhancement algorithm, wherein the first enhancement feature comprises the following steps: inputting the image block into a first sub-network to obtain a first sub-feature of the image block. And processing the first sub-feature through the matching block according to the feature codebook to obtain a second sub-feature. And inputting the second sub-feature into a second sub-network to obtain a first enhancement feature of the image block.

In some possible implementations, the second image enhancement algorithm includes a third sub-network and a fourth sub-network.

Inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks, wherein the method comprises the following steps: and inputting the first enhancement feature and the second enhancement feature into a third sub-network to obtain a third sub-feature of the image block. And processing the first enhancement feature through the matching block according to the feature codebook, and processing the third sub-feature to obtain a fourth sub-feature. And the fourth sub-network obtains the enhanced image segmentation according to the first enhancement feature, the second enhancement feature and the fourth sub-feature.

In a second aspect, there is provided an image enhancement apparatus comprising:

The acquisition module is used for acquiring a feature codebook of the target image, wherein the feature codebook at least comprises global information of the target image. And the processing module is used for carrying out image blocking processing on the target image to obtain a plurality of image blocks of the target image. And the enhancement module is used for enhancing each image block according to the characteristic codebook to obtain the enhanced image block. And the merging module is used for merging the enhanced target images according to the plurality of enhanced image blocks.

In some possible embodiments, the acquiring module is specifically configured to input the target image and/or a reference image of the target image into a pre-trained global guiding module, where the global guiding module includes a first feature extracting module and a first encoding module, and the reference image of the target image is obtained by matching the target image from a preset first database. And extracting global information of the target image and/or a reference image of the target image through a first feature extraction module, and encoding the global information into a feature codebook of the target image through a first encoding module.

The acquisition module is specifically configured to obtain corresponding prior information by matching from a preset first database according to the target image. The prior information is encoded into a feature codebook of the target image by a second encoding module.

The acquisition module is specifically used for acquiring the original shooting information of the target image. And encoding the original shooting information into a characteristic codebook of the target image through a third encoding module.

In some possible embodiments, the enhancement module is specifically configured to obtain, through a preset first image enhancement algorithm, a first enhancement feature of the image partition. And processing the first enhancement features through the matching block according to the feature codebook to obtain second enhancement features. And inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks.

In some possible embodiments, the enhancement module is specifically configured to match, according to the global feature in the feature codebook, a feature block meeting a preset condition in the first enhancement feature through a preset similarity algorithm. And replacing or fusing the preset condition feature blocks according to the global features to obtain second enhancement features.

The enhancement module is specifically configured to input the image block into the first sub-network to obtain a first sub-feature of the image block. And processing the first sub-feature through the matching block according to the feature codebook to obtain a second sub-feature. And inputting the second sub-feature into a second sub-network to obtain a first enhancement feature of the image block.

The enhancement module is specifically configured to input the first enhancement feature and the second enhancement feature into a third sub-network, so as to obtain a third sub-feature of the image partition. And processing the first enhancement feature through the matching block according to the feature codebook, and processing the third sub-feature to obtain a fourth sub-feature. And the fourth sub-network obtains the enhanced image segmentation according to the first enhancement feature, the second enhancement feature and the fourth sub-feature.

In a fourth aspect, there is provided a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, perform the method provided in the first aspect.

In a fifth aspect, a chip is provided, including: a processor for calling and running a computer program from a memory, so that the chip-mounted device performs the method provided in the first aspect.

In a sixth aspect, there is provided a computer program product comprising: computer program code which, when run by an electronic device, causes the electronic device to perform the method provided in the first aspect.

The advantages of the second aspect to the sixth aspect may refer to the first aspect, and are not described herein.

Drawings

Fig. 1 is an application scenario schematic diagram of an image enhancement method provided in an embodiment of the present application;

fig. 2 is a schematic application scenario diagram of another image enhancement method according to an embodiment of the present application;

fig. 3 is a block diagram of a hardware structure of an electronic device according to an embodiment of the present application;

fig. 4 is a system structural block diagram of an electronic device provided in an embodiment of the present application;

fig. 5 is a schematic software architecture of an electronic device according to an embodiment of the present application;

fig. 6 is a flowchart of an image enhancement method provided in an embodiment of the present application;

fig. 7 is a schematic flowchart of implementing S601 in the image enhancement method provided in the embodiment of the present application;

Fig. 8 is a schematic flowchart of implementing S603 and S604 in the image enhancement method provided in the embodiment of the present application;

fig. 9 shows a block diagram of an image enhancement apparatus provided in an embodiment of the present application;

fig. 10 is a schematic structural diagram of a chip according to an embodiment of the present application.

Detailed Description

The technical solutions in the present application will be described below with reference to the accompanying drawings.

In the description of the embodiments of the present application, unless otherwise indicated, "/" means or, for example, a/B may represent a or B; "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

Image enhancement refers to the process of processing a clear, complete, high-resolution, high-quality image from an image that is affected by various degradation factors. The method has the functions of denoising, deblurring, filling, repairing, supercution and the like, and has wide application scenes.

The image enhancement can be generally divided into full-image enhancement or block enhancement, and the full-image enhancement can consider the whole information, structure and priori of the image, so that the consistency and the integrity of the image are maintained, but the problems of large calculation amount, detail loss, easy excessive smoothing and the like exist.

The block enhancement can keep the details and local characteristics of the image, and simultaneously reduce the calculation amount and the memory consumption, but has the problems of easy occurrence of blocking effect, artifact, degradation of enhancement capability and the like.

At present, any image enhancement method has certain defects, and the single image enhancement method has the problems of poor effect and poor user experience.

In view of this, an embodiment of the present application provides an image enhancement method, including: and acquiring a feature codebook of the target image, wherein the feature codebook is used for representing global features of the target image. And performing image blocking processing on the target images to obtain image blocks of a plurality of target images. And enhancing each image block according to the feature codebook to obtain enhanced image blocks. And according to the plurality of enhanced image blocks, merging to obtain an enhanced target image.

By acquiring global features of the target image, generating a feature codebook, and carrying out block enhancement on the target image according to the feature codebook, the advantages of global enhancement and block enhancement can be simultaneously considered during image enhancement, the probability of blocking effect or artifact of the image after block enhancement is reduced, and the consistency and the integrity of the target image are ensured. The image enhancement effect is improved, and the user experience is improved.

First, application scenarios of the embodiments of the present application will be briefly described.

Fig. 1 is an application scenario schematic diagram of an image enhancement method provided in an embodiment of the present application, and fig. 2 is an application scenario schematic diagram of another image enhancement method provided in an embodiment of the present application.

In one possible implementation scenario, referring to fig. 1, an electronic device 100 may be included in an application scenario. In this scenario, the electronic device 100 is a mobile phone. As an example, the electronic device 100 may run a photographing application, and when the electronic device 100 instructs the photographing application to photograph in response to a user operation, a photographed target image may be obtained.

Then, the electronic equipment can locally enhance the shot target image through the image enhancement method.

Alternatively, referring to fig. 2, the application scenario may include the electronic device 100 and the user device 200. In this scenario, the electronic device 100 is a server, and the user device 200 is a mobile phone. As an example, the user device 200 may run a photographing application, and when the user device 200 instructs the photographing application to photograph in response to a user operation, a photographed target image may be obtained.

Then, the user device 200 may send the target image to the electronic device 100 through the network, and the electronic device 100 enhances the captured target image through the image enhancement method provided by the application, and then returns the enhanced target image to the user device 200.

Fig. 3 is a block diagram of a hardware structure of an electronic device according to an embodiment of the present application.

As examples, electronic devices may include cell phones, tablet computers, servers, cloud servers, palm game consoles, wearable devices, augmented Reality (AR)/Virtual Reality (VR) devices, notebook computers, ultra-mobile personal computer (UMPC), netbooks, personal digital assistants (personal digital assistant, PDA), and the like. Embodiments of the present application are not limited with respect to a particular type of electronic device.

Referring to fig. 3, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

For example, when the electronic device 100 is a mobile phone or a tablet computer, all the components in the illustration may be included, or only some of the components in the illustration may be included.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present invention is only illustrative, and is not meant to limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also use different interfacing manners, or a combination of multiple interfacing manners in the foregoing embodiments.

The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger.

The power management module 141 is used for connecting the battery 142, and the charge management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied to the electronic device 100.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field wireless communication technology (near field communication, NFC), infrared technology (IR), etc., as applied to the electronic device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

In some embodiments, antenna 1 and mobile communication module 150 of electronic device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that electronic device 100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques may include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a beidou satellite navigation system (beidou navigation satellite system, BDS), a quasi zenith satellite system (quasi-zenith satellite system, QZSS) and/or a satellite based augmentation system (satellite based augmentation systems, SBAS).

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, N being a positive integer greater than 1.

The electronic device 100 may implement photographing functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process data fed back by the camera 193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera 193.

The camera 193 is used to capture still images or video. In some embodiments, electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The NPU is a neural-network (NN) computing processor, and can rapidly process input information by referencing a biological neural network structure, for example, referencing a transmission mode between human brain neurons, and can also continuously perform self-learning. Applications such as intelligent awareness of the electronic device 100 may be implemented through the NPU, for example: image enhancement, image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to enable expansion of the memory capabilities of the electronic device 100. The external memory card communicates with the processor 110 through an external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 121 may be used to store computer executable program code including instructions. The internal memory 121 may include a storage program area and a storage data area. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like. The processor 110 performs various functional applications of the electronic device 100 and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or a portion of the functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also referred to as a "horn," is used to convert audio electrical signals into sound signals. The electronic device 100 may listen to music, or to hands-free conversations, through the speaker 170A.

A receiver 170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When electronic device 100 is answering a telephone call or voice message, voice may be received by placing receiver 170B in close proximity to the human ear.

Microphone 170C, also referred to as a "microphone" or "microphone", is used to convert sound signals into electrical signals.

The earphone interface 170D is used to connect a wired earphone. The headset interface 170D may be a USB interface 130 or a 3.5mm open mobile electronic device platform (open mobile terminal platform, OMTP) standard interface, a american cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used to sense a pressure signal, and may convert the pressure signal into an electrical signal.

The gyro sensor 180B may be used to determine a motion gesture of the electronic device 100.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude from barometric pressure values measured by barometric pressure sensor 180C, aiding in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity may be detected when the electronic device 100 is stationary.

A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it may be determined that there is an object in the vicinity of the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there is no object in the vicinity of the electronic device 100. The ambient light sensor 180L is used to sense ambient light level. The electronic device 100 may adaptively adjust the brightness of the display 194 based on the perceived ambient light level. The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 may utilize the collected fingerprint feature to unlock the fingerprint, access the application lock, photograph the fingerprint, answer the incoming call, etc. The temperature sensor 180J is for detecting temperature. The touch sensor 180K, also referred to as a "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is for detecting a touch operation acting thereon or thereabout. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys. Or may be a touch key. The electronic device 100 may receive key inputs, generating key signal inputs related to user settings and function controls of the electronic device 100. The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration alerting as well as for touch vibration feedback. The indicator 192 may be an indicator light, may be used to indicate a state of charge, a change in charge, a message indicating a missed call, a notification, etc. The SIM card interface 195 is used to connect a SIM card.

For the scenario in the above example, the operating system of electronic device 100 may include, but is not limited to, the operating systems of Seban (Symbian), android (android), windows (Windows), apples (MacOS, iOS), blackberry (Blackberry), hong (Harmony OS), lin Nasi (Linux), or You Nake (Unix), among others.

Fig. 4 is a system structural block diagram of an electronic device provided in an embodiment of the present application.

As an example, referring to the scenario shown in fig. 1, when the electronic device 100 is a mobile phone, the operating system of the electronic device 100 may be android, and the system structure thereof may refer to fig. 4.

The layered architecture divides the software into a plurality of layers, and each layer has clear roles and division. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Zhuoyun row (Android run) and system libraries, and a kernel layer, respectively.

The application layer may include a series of application packages.

As shown in fig. 4, the application package may include applications for cameras, gallery, calendar, phone calls, maps, navigation, WLAN, bluetooth, music, video, short messages, etc.

The application framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions.

As shown in fig. 4, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

The window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

The view system includes visual controls, such as controls to display characters, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text message notification icon may include a view displaying characters and a view displaying pictures.

The telephony manager is for providing communication functions of the electronic device. Such as the management of call status (including on, hung-up, etc.).

The resource manager provides various resources for the application program, such as localization strings, icons, pictures, layout files, video files, and the like.

The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

Android run time includes a core library and virtual machines. Android run time is responsible for scheduling and management of the Android system.

The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface manager (surface manager), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), etc.

The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

Fig. 5 is a schematic software architecture of an electronic device according to an embodiment of the present application.

Referring to fig. 5, a global boot module (global guidance module, GGM) and a block boot module (latch-based process module, PPM) are included. The global guiding module is used for extracting global features of the target image and generating a feature codebook. Wherein at least a first feature extraction module and a first encoding module are included. The first feature extraction module is used for extracting global information according to the target image. The first encoding module is used for encoding the global information into the feature codebook.

In other possible implementations, the global guidance module may further include a second encoding module and/or a third encoding module. The second encoding module is used for encoding the prior information of the target image into the feature codebook, and the third encoding module is used for encoding the original shooting information of the target image into the feature codebook.

The blocking processing module is used for enhancing the image blocks of the target image according to the feature codebook and synthesizing the enhanced target image. Including a matching block, a first image enhancement algorithm, and a second image enhancement algorithm.

Fig. 6 is a flowchart of an image enhancement method according to an embodiment of the present application.

By way of example, and not limitation, an image enhancement method is applied to the electronic device 100 shown in fig. 1.

Referring to fig. 6, the image enhancement method includes:

s601, acquiring a feature codebook of a target image.

In some possible implementations, the feature codebook may be a set containing feature information, where the features in the set are used to represent global features of the target image. The global features of the target image may include global information of the target image, prior information corresponding to the target image, and original photographing information corresponding to the target image.

Fig. 7 is a schematic flowchart of implementing S601 in the image enhancement method provided in the embodiment of the present application.

Referring to fig. 7, acquiring a feature codebook of a target image may include:

s6011, inputting the target image and/or a reference image of the target image into a pre-trained global guidance module.

In some possible implementations, referring to fig. 6, the global steering module includes a first feature extraction module and a first encoding module.

As an example, referring to the scene shown in fig. 1 or 2, the target image may be an image photographed by an electronic device (e.g., a cellular phone). Alternatively, the target image may be an image obtained by the electronic device from an instant messaging application, a network disk application, a browser application, a database, or the like.

In some possible implementations, the reference image of the target image is obtained by matching the target image from a preset first database. The reference map of the target image may contain a body, texture, similar to the target image.

As an example, the first database may include a plurality of images and features corresponding to each image, which may include color histograms, texture descriptors, shape features, local features, and so forth. When the reference image is matched from the first database, feature extraction can be performed on the target image to obtain matching features of the target image, wherein the matching features of the target image can be at least one of color histogram, texture descriptor, shape feature and local feature. And then calculating feature similarity according to the matching features and features in the database, sequencing the images in the first database according to the feature similarity, and taking the image with the highest feature similarity in the first database as a reference image of the target image.

S6012, extracting global information of the target image and/or a reference image of the target image by the first feature extraction module.

In some possible implementations, the first feature extraction module may be a pre-trained full-graph enhancement network, or may be a portion of a pre-trained full-graph enhancement network used for feature extraction.

As an example, the target image and/or the reference image of the target image may be input into a pre-trained global image enhancement network, global information of the target image and/or the reference image of the target image is extracted through the global image enhancement network, and the obtained global information is derived before the global image enhancement network enhances the target image according to the global information.

In some possible implementations, the first feature extraction module may be a Super-resolution generation resistive network (Super-Resolution Generative Adversarial Network, SRGAN), a U-Net (U-Net), or the like. When the networks are trained, parameters can be fixed after the characteristics can be accurately extracted from the networks after the networks are trained, so that robust and high-quality global information can be extracted.

And S6013, the global information is encoded into a feature codebook of the target image through the first encoding module.

In some possible implementations, the first encoding module may cluster features in the global information by a clustering algorithm to determine representative centroids in the feature space, and then write these to a feature codebook of the target image based on these centroids. The clustering algorithms may include, among others, K-means, hierarchical clustering (Hierarchical Clustering), density-based clustering with noise (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), and the like.

In some possible implementations, referring to fig. 5, assume that the target image and/or reference image of the target image isThe first feature extraction module is network A, denoted +.>The global information is expressed as->The first coding module is network B, denoted +.>Feature codebook of global information +.>The expression can be represented by the following formula:

in some possible implementations, referring to fig. 5, the global boot module further includes a second encoding module. Acquiring a feature codebook of a target image, comprising: and matching the target image from a preset first database to obtain corresponding priori information. The prior information is encoded into a feature codebook of the target image by a second encoding module.

In some possible implementations, the first database may further include a plurality of a priori information. The a priori information may be a set of learnable parameters of a predefined capacity or may be fixed a priori information obtained by other means, such as entering a common texture into a trained network. The a priori information may include a variety of sharp, fixed texture features. Clear, fixed texture prior information, which generally refers to some assumption or knowledge of texture features in an image, can be used to guide image processing algorithms to improve image quality.

As examples, the prior information may include natural image statistics prior, smoothing prior, edge preserving prior, texture consistency prior, structure prior, sparse representation prior, local consistency prior, and so forth.

The method for matching prior information from the preset first database according to the target image is similar to the method for matching the reference image from the preset first database according to the target image, and will not be described herein.

Wherein the second encoding module may be the same network as the first encoding moduleExpressed as->. The a priori information is denoted as P, the feature codebook of a priori information +.>The expression can be represented by the following formula:

When the global guidance module comprises a first encoding module and a second encoding module, the feature codebookFeature codebook comprising global information ++>And feature codebook of a priori information ++>I.e. +.>。

In some possible implementations, the global boot module further includes a third encoding module. Acquiring a feature codebook of a target image, comprising: and acquiring original shooting information of the target image. And encoding the original shooting information into a characteristic codebook of the target image through a third encoding module.

When the photographed target image is an image in an original (RAW) format, original photographing information such as original data of an image sensor, white balance information, exposure information, color space information, lens information, camera model and setting, metadata, histogram, and the like at the time of photographing is also included in the target image.

Since the RAW format image has the original shot information describing a wider dynamic range of the target image, i.e., more details of the dark or bright portions. More color information (e.g., 14bit or higher color depth) is also described, retaining more detail information. This raw shot information can also be used to guide image processing algorithms to improve image quality.

Wherein the third encoding module may be the same network as the first encoding module Expressed as->. The original shooting information is expressed as R, the feature codebook of the prior information is ++>The expression can be represented by the following formula:

/>

when the global guidance module comprises a first encoding module and a third encoding module, the feature codebookFeature codebook comprising global information ++>And the feature codebook of the original shot information +.>I.e. +.>。

When the global guidance module comprises a first encoding module, a second encoding module and a third encoding module, the feature codebookFeature codebook comprising global information ++>Feature codebook of a priori information->And the feature codebook of the original shot information +.>I.e.。

In this embodiment, the feature codebook dynamically changes according to the target image, so that the feature codebook can represent global features of different target images, and further, global features corresponding to the target image can be considered during block enhancement, the effect of image enhancement is improved, the probability of occurrence of blocking effect or artifacts is reduced, and the consistency and integrity of the target image are ensured.

In some possible implementations, the feature codebook may not include the feature codebook of global information when global information of the target image and/or a reference image of the target image cannot be acquired. For example, when the first feature extraction module cannot identify the target image and/or a reference image of the target image, the feature codebook Feature codebook which may only include a priori information +.>I.e.。

In this embodiment, when global information of the target image and/or a reference image of the target image cannot be acquired, the feature codebook may be generated only by using prior information, and feature supervision is performed on the segmentation enhancement, so that an application scenario of the image enhancement method is improved, and the probability of occurrence of blocking effect or artifact can be reduced to a certain extent.

S602, performing image blocking processing on the target images to obtain image blocks of a plurality of target images.

In some possible implementations, the size and number of tiles may be determined according to the size of the target image. The number of blocks is typically a power of 2 to N (N is a positive integer), such as 32 x 32, 64 x 64, etc. A blocking policy is then determined, such as whether overlap between blocks is required. And finally, performing blocking processing on the target image according to the size, the number and the blocking strategy of the blocking. The blocking process may be implemented by Java language code, python language code, or the like.

And S603, enhancing each image block according to the feature codebook to obtain enhanced image blocks.

S604, according to the plurality of enhanced image blocks, combining to obtain an enhanced target image.

Fig. 8 is a schematic flowchart of implementing S603 and S604 in the image enhancement method provided in the embodiment of the present application.

In some possible implementations, referring to fig. 8, each image partition is enhanced according to a feature codebook, to obtain an enhanced image partition, including:

s6031, acquiring first enhancement features of image blocks through a preset first image enhancement algorithm.

In some possible implementations, the first enhancement feature is a noise-separating feature, a histogram feature, an edge strength, texture information, and the like extracted by the first image enhancement algorithm according to the image partition.

Referring to fig. 5, the first image enhancement algorithm and the second image enhancement algorithm may be basic modules in a block enhancement algorithm, including at least one intermediate feature layer therebetween.

The blocking enhancement algorithm comprises a depth residual error network (ResNet), a U-Net, a visual transformer (Visual Transformer, VIT) and the like. The basic modules include network modules commonly used in deep learning, such as a convolution layer, a full connection layer, an attention layer, a downsampling layer, an upsampling layer and the like. The specific basic modules used to implement the first image enhancement algorithm or the second image enhancement algorithm may be determined according to the requirements in practical application, which is not limited in this application.

First enhancement feature) Can be represented by a matrix, i.e.)>I and j represent the ith row and jth column, respectively, of the matrix.

S6032, processing the first enhancement features through the matching block according to the feature codebook to obtain second enhancement features.

In some possible implementations, the matching block may match, according to the global feature in the feature codebook, a feature block meeting a preset condition in the first enhancement feature through a preset similarity algorithm. Replacing or fusing the preset condition feature blocks according to the global features to obtain second enhanced features #). Second enhancement feature (+)>) Can be represented by a matrix, i.e.)>I and j represent the ith row and jth column, respectively, of the matrix.

As an example, the matching block may be implemented by a nearest neighbor matching in Vector quantized-variable automatic encoding (VQ-VAE), a guard_softmax fusion, or a multi-head cross attention (cross attention), etc.

For example, when a matching block of the nearest neighbor matching method is usedThe second enhancement feature (++) can be obtained by the following formula>）：

The formula is used to determine the causeAnd->Cluster center with minimum Euclidean distance between them (++>). Wherein (1) >Representing the kth feature in the feature codebook, argmin represents the value of the expression (i.e.)>Euclidean norm) to a minimum value. In this case argmin is used to determine the cluster center (/ -A)>）。

In other examples, when a multi-headed cross-attention mode matching block is usedThe second enhancement feature (++) can be obtained by the following formula>）：

Wherein,、/>and->Is a transfer function from the cluster center to a Query (Query, q), key (Key, k), and Value (v) vector. d is the first enhancement feature->Is used in the manufacture of a printed circuit board,𝜎is a softmax function used to calculate the attention score. The query vector q is obtained by passing the determined nearest cluster center input to the function +.>Obtained.

S6033, inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks.

In some possible implementations, an example of the second image enhancement algorithm is shown in S6031, which is not described herein.

Referring to fig. 8, the steps S6031 to S6033 may be expressed by the following formula:

wherein,representing image tiles, tile processing module D is denoted +.>The first image enhancement algorithm D1 is represented asThe second image enhancement algorithm D2 is denoted +. >，/>Representing the enhanced target image. That is, the enhanced target image is the block processing module D blocks according to the image block +.>Matching block->Feature codebook ∈ ->Obtained. Specifically, the first enhancement feature +_for each image partition may be obtained by the first image enhancement algorithm D1>Then go through the matching block->Will be about the first enhancement feature>And the feature codebook C to obtain a second enhancement feature +.>. Finally, according to the first enhancement feature of each image partition +.>And second enhancement feature->The enhanced target image is obtained by processing with the second image enhancement algorithm D2>。

In some possible implementations, the first image enhancement algorithm includes a first subnetwork and a second subnetwork. Acquiring a first enhancement feature of the image block through a preset first image enhancement algorithm, wherein the first enhancement feature comprises the following steps: inputting the image block into a first sub-network to obtain a first sub-feature of the image block. And processing the first sub-feature through the matching block according to the feature codebook to obtain a second sub-feature. And inputting the second sub-feature into a second sub-network to obtain a first enhancement feature of the image block. The second image enhancement algorithm includes a third sub-network and a fourth sub-network. Inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks, wherein the method comprises the following steps: and inputting the first enhancement feature and the second enhancement feature into a third sub-network to obtain a third sub-feature of the image block. And processing the first enhancement feature through the matching block according to the feature codebook, and processing the third sub-feature to obtain a fourth sub-feature. And the fourth sub-network obtains the enhanced image segmentation according to the first enhancement feature, the second enhancement feature and the fourth sub-feature.

In this embodiment, the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network may be the same network, for example, may be implemented by a plurality of base modules in a network such as a res Net, a U-Net, or a VIT.

And matching the feature codebook with the extracted features for multiple times by using a matching block through the first sub-network and the second sub-network to obtain a first enhancement feature, wherein the obtained first enhancement feature can contain more global features. In this case, the global feature in the image partition can be better represented according to the second enhancement feature obtained from the first enhancement feature. And then the third sub-network and the fourth sub-network are used for continuously matching the global features in the process of generating the enhanced target image, so that the obtained enhanced target image is more in line with the global features, the probability of occurrence of blocking effect or artifacts is further reduced, and the consistency and the integrity of the target image are ensured. The image enhancement effect is improved, and the user experience is improved.

In some possible implementations, the feature codebook is used when global information of the target image and/or a reference image of the target image cannot be acquiredFeature codebook which may only include a priori information +. >. In this case, the target image may be directly input to the block processing module for image enhancement.

In still other possible implementations, the method may also not include a matching block, and the cross-attention module is converted into the self-attention module.

It should be appreciated that the above illustration is to aid one skilled in the art in understanding the embodiments of the application and is not intended to limit the embodiments of the application to the specific numerical values or the specific scenarios illustrated.

It will be apparent to those skilled in the art from the foregoing description that various equivalent modifications or variations can be made, and such modifications or variations are intended to be within the scope of the embodiments of the present application.

Corresponding to the image enhancement method provided in the above embodiment, fig. 9 shows a block diagram of the image enhancement apparatus provided in the embodiment of the present application, and for convenience of explanation, only the portions related to the embodiment of the present application are shown.

Referring to fig. 9, an embodiment of the present application provides an image enhancement apparatus, including:

the obtaining module 91 is configured to obtain a feature codebook of a target image, where the feature codebook includes at least global information of the target image.

And the processing module 92 is configured to perform image blocking processing on the target image to obtain image blocks of the plurality of target images.

And the enhancement module 93 is configured to enhance each image partition according to the feature codebook, so as to obtain the enhanced image partition.

A merging module 94, configured to merge the enhanced target images according to the plurality of enhanced image blocks.

In some possible embodiments, the obtaining module 91 is specifically configured to input the target image and/or a reference image of the target image into a pre-trained global guiding module, where the global guiding module includes a first feature extraction module and a first encoding module, and the reference image of the target image is obtained by matching the target image from a preset first database. And extracting global information of the target image and/or a reference image of the target image through a first feature extraction module, and encoding the global information into a feature codebook of the target image through a first encoding module.

The obtaining module 91 is specifically configured to obtain corresponding prior information by matching from a preset first database according to the target image. The prior information is encoded into a feature codebook of the target image by a second encoding module.

The acquiring module 91 is specifically configured to acquire original shooting information of the target image. And encoding the original shooting information into a characteristic codebook of the target image through a third encoding module.

In some possible embodiments, the enhancing module 93 is specifically configured to obtain the first enhancement characteristic of the image partition through a preset first image enhancement algorithm. And processing the first enhancement features through the matching block according to the feature codebook to obtain second enhancement features. And inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain enhanced image blocks.

In some possible embodiments, the enhancing module 93 is specifically configured to match, according to the global feature in the feature codebook, a feature block meeting a preset condition in the first enhanced feature through a preset similarity algorithm. And replacing or fusing the preset condition feature blocks according to the global features to obtain second enhancement features.

The enhancement module 93 is specifically configured to input the image partition into the first sub-network, so as to obtain a first sub-feature of the image partition. And processing the first sub-feature through the matching block according to the feature codebook to obtain a second sub-feature. And inputting the second sub-feature into a second sub-network to obtain a first enhancement feature of the image block.

The enhancement module 93 is specifically configured to input the first enhancement feature and the second enhancement feature into a third sub-network, so as to obtain a third sub-feature of the image partition. And processing the first enhancement feature through the matching block according to the feature codebook, and processing the third sub-feature to obtain a fourth sub-feature. And the fourth sub-network obtains the enhanced image segmentation according to the first enhancement feature, the second enhancement feature and the fourth sub-feature.

It should be understood that the hardware system and the chip in the embodiments of the present application may perform the various image enhancement methods in the embodiments of the present application, that is, the specific working processes of the following various products may refer to the corresponding processes in the embodiments of the foregoing methods.

The embodiment of the application also provides another electronic device which comprises a processor and a memory.

A memory for storing a computer program executable on the processor.

A processor for performing the steps of processing in the method of image enhancement as described above.

Embodiments of the present application also provide a computer-readable storage medium having computer instructions stored therein; the computer readable storage medium, when run on an electronic device, causes the electronic device to perform the method as previously described.

The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).

The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more servers, data centers, etc. that can be integrated with the medium.

The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

Embodiments of the present application also provide a computer program product comprising computer instructions which, when run on an electronic device, enable the electronic device to perform the technical solutions shown above.

Fig. 10 is a schematic structural diagram of a chip according to an embodiment of the present application. The chip shown in fig. 10 may be a general-purpose processor or a special-purpose processor. The chip includes a processor 1001. The processor 1001 is configured to support the electronic device to execute the foregoing technical solution.

Optionally, the chip further comprises a transceiver 1002, and the transceiver 1002 is configured to be controlled by the processor 1001 and to support the communication device to perform the foregoing technical solutions.

Optionally, the chip shown in fig. 10 may further include: a storage medium 1003.

It should be noted that the chip shown in fig. 10 may be implemented using the following circuits or devices: one or more field programmable gate arrays (field programmable gate array, FPGA), programmable logic devices (programmable logic device, PLD), controllers, state machines, gate logic, discrete hardware components, any other suitable circuit or combination of circuits capable of performing the various functions described throughout this application.

The electronic device, the computer storage medium, the computer program product, and the chip provided in the embodiments of the present application are used to execute the method provided above, so that the beneficial effects that can be achieved by the electronic device, the computer storage medium, the computer program product, and the chip can refer to the beneficial effects corresponding to the method provided above, and are not described herein again.

It should be understood that the foregoing is only intended to assist those skilled in the art in better understanding the embodiments of the present application and is not intended to limit the scope of the embodiments of the present application. Various equivalent modifications and variations will be apparent to those skilled in the art from the foregoing examples, given.

For example, some steps may not be necessary in the various embodiments of the methods described above, or some steps may be newly added, etc. Or a combination of any two or more of the above. Such modifications, variations, or combinations are also within the scope of embodiments of the present application.

It should also be understood that the foregoing description of embodiments of the present application focuses on highlighting differences between the various embodiments and that the same or similar elements not mentioned may be referred to each other and are not described in detail herein for brevity.

It should be further understood that the sequence numbers of the above processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and the internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

It should be further understood that, in the embodiments of the present application, the "preset" and "predefined" may be implemented by pre-storing corresponding codes, tables, or other manners that may be used to indicate relevant information in a device (including, for example, an electronic device), and the present application is not limited to a specific implementation manner thereof.

It should also be understood that the manner, condition, class and division of the embodiments in the embodiments of the present application are for convenience of description only and should not be construed as being particularly limited, and the various manners, classes, conditions and features of the embodiments may be combined without contradiction.

It is also to be understood that in the various embodiments of the application, terms and/or descriptions of the various embodiments are consistent and may be referenced to one another in the absence of a particular explanation or logic conflict, and that the features of the various embodiments may be combined to form new embodiments in accordance with their inherent logic relationships.

Finally, it should be noted that: the foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of image enhancement, the method comprising:

acquiring a feature codebook of a target image, wherein the feature codebook is used for representing global features of the target image;

Performing image blocking processing on the target image to obtain a plurality of image blocks of the target image;

enhancing each image block according to the feature codebook to obtain the enhanced image block;

and according to the plurality of enhanced image blocks, merging to obtain the enhanced target image.

2. The method of claim 1, wherein the acquiring the feature codebook of the target image comprises:

inputting the target image and/or a reference image of the target image into a pre-trained global guiding module, wherein the global guiding module comprises a first feature extraction module and a first coding module, and the reference image of the target image is obtained by matching the target image from a preset first database;

and extracting global information of the target image and/or a reference image of the target image through the first feature extraction module, and encoding the global information into a feature codebook of the target image through the first encoding module.

3. The method of claim 2, wherein the global steering module further comprises a second encoding module;

the feature codebook for acquiring the target image comprises the following steps:

Matching from a preset first database according to the target image to obtain corresponding priori information;

and encoding the prior information into a feature codebook of the target image through a second encoding module.

4. The method of claim 2, wherein the global steering module further comprises a third encoding module;

acquiring original shooting information of the target image;

and encoding the original shooting information into a feature codebook of the target image through a third encoding module.

5. The method according to any one of claims 1-4, wherein enhancing each of the image partitions according to the feature codebook to obtain the enhanced image partition comprises:

acquiring a first enhancement characteristic of the image block through a preset first image enhancement algorithm;

processing the first enhancement features through a matching block according to the feature codebook to obtain second enhancement features;

and inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain the enhanced image blocks.

6. The method of claim 5, wherein the processing the first enhancement feature by the matching block according to the feature codebook to obtain a second enhancement feature comprises:

The matching block matches the feature blocks meeting the preset conditions in the first enhancement features through a preset similarity algorithm according to the global features in the feature codebook;

and replacing or fusing the feature blocks meeting the preset conditions according to the global features to obtain the second enhancement features.

7. The method of claim 5, wherein the first image enhancement algorithm comprises a first subnetwork and a second subnetwork;

the obtaining, by a preset first image enhancement algorithm, a first enhancement feature of the image partition includes:

inputting the image blocks into the first sub-network to obtain first sub-features of the image blocks;

processing the first sub-feature through the matching block according to the feature codebook to obtain a second sub-feature;

and inputting the second sub-feature into the second sub-network to obtain the first enhancement feature of the image block.

8. The method of claim 5, wherein the second image enhancement algorithm comprises a third sub-network and a fourth sub-network;

inputting the first enhancement feature and the second enhancement feature into a preset second image enhancement algorithm to obtain the enhanced image blocks, wherein the method comprises the following steps:

Inputting the first enhancement feature and the second enhancement feature into the third sub-network to obtain a third sub-feature of the image block;

processing the first enhancement feature through a matching block according to the feature codebook, and processing the third sub-feature to obtain a fourth sub-feature;

the fourth sub-network is used for partitioning the image after being enhanced according to the first enhancement feature, the second enhancement feature and the fourth sub-feature.

9. An image enhancement device, the device comprising:

the acquisition module is used for acquiring a feature codebook of a target image, wherein the feature codebook at least comprises global information of the target image;

the processing module is used for carrying out image blocking processing on the target image to obtain image blocks of a plurality of target images;

the enhancement module is used for enhancing each image block according to the characteristic codebook to obtain the enhanced image block;

and the merging module is used for merging the enhanced target images according to the plurality of enhanced image blocks.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 8 when the computer program is executed.

11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, perform the method of any of claims 1 to 8.