CN118474553A

CN118474553A - Image processing method and electronic equipment

Info

Publication number: CN118474553A
Application number: CN202311655933.XA
Authority: CN
Inventors: 石坚
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-12-01
Filing date: 2023-12-01
Publication date: 2024-08-09

Abstract

An image processing method and electronic equipment relate to the field of image processing, solve the problems of texture detail loss and color deviation in the process of converting a high dynamic range image into a low dynamic range image, and improve the accuracy of the obtained low dynamic range image. In the method, a global mapping matrix and a local mapping matrix of a high dynamic range image are firstly obtained, and brightness correction is carried out on the high dynamic range image based on the global mapping matrix to obtain a second image; and carrying out color correction on the high dynamic range image based on the local mapping matrix and the second image to obtain a low dynamic range image with the final color and brightness restored.

Description

Image processing method and electronic equipment

Technical Field

The embodiment of the application relates to the field of image processing, in particular to an image processing method and electronic equipment.

Background

With the development of electronic technology, shooting functions of electronic devices such as mobile phones have also been rapidly developed. In order to improve the shooting quality, some mobile phones have integrated a High dynamic range (High-DYNAMIC RANGE, HDR) function. However, the dynamic range of luminance of an HDR image photographed by a mobile phone using the HDR function is relatively large, which means that the difference between the brightest and darkest portions in the image is very large, as typically at 0.001cd/m ²-100000cd/m². That is, there are cases where bright portions are overexposed and dark portions are excessively dark, resulting in that a truly usable image cannot be obtained. In order to enable an image presented by an electronic device to truly reflect a scene seen by a human eye, the electronic device generally converts an HDR image into a Low dynamic range (Low-DYNAMIC RANGE, LDR) image through a plurality of neural networks connected in series and presents the image to a user.

However, serious color cast and information loss exist in the finally obtained LDR image, so that the image can not truly reflect the scene seen by human eyes.

Disclosure of Invention

Based on the method, the image processing method and the electronic device are provided, the problems of texture detail loss and color deviation in the process of converting the high dynamic range image into the low dynamic range image are solved, and the accuracy of the obtained low dynamic range image is improved.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical scheme:

in a first aspect, the present application provides an image processing method. In the method, the electronic equipment acquires a first image with a high dynamic range, and further acquires a global mapping relation and a local mapping relation, namely a global mapping matrix and a local mapping matrix, from the first image to a corresponding low dynamic range image in brightness and color. After the global mapping matrix and the local mapping matrix are determined, carrying out brightness correction on the first image based on the global mapping matrix to obtain a second image; and carrying out color correction on the first image based on the local mapping matrix to obtain a third image with low dynamic range corresponding to the first image.

The image processing method is realized based on a neural network, that is to say, tone mapping can be realized by adopting a neural network, that is, the image with high dynamic range is converted into the image with low dynamic range, so that error accumulation is avoided, and the color accuracy of the finally output low dynamic image is ensured. In addition, the neural network includes two independent channels (for example, an upper channel for performing brightness correction on the first image based on the global mapping matrix and a lower channel for performing color correction on the first image based on the local mapping matrix), so as to respectively implement brightness and color restoration processing. It can be seen that partial decoupling of luminance and color is achieved by the two independent channels, avoiding loss of detail texture of the restored low dynamic range image. In conclusion, the accuracy of the obtained low dynamic range image is improved.

In addition, by performing color correction processing in the neural network, negative number truncation and high-order truncation of the image are avoided, and accurate restoration of texture details is further ensured.

In an implementation manner of the first aspect, after the first image is obtained, a downsampling process may be further performed on the first image. That is, the obtained first image is subjected to compression processing of the image size, thereby reducing the number of pixel values in the image, and reducing the amount of calculation.

In an implementation manner of the first aspect, acquiring the global mapping matrix and the local mapping matrix of the first image may include: and acquiring image features of the first image, determining global features and local features of the first image based on the image features, and determining a corresponding global mapping matrix and local mapping matrix according to the obtained global features and local features and different weight coefficients. If the global mapping matrix is determined based on the global feature, the local feature and the first weight coefficient, the local mapping matrix is determined based on the global feature, the local feature and the second weight coefficient.

The brightness and the color are restored respectively through the global mapping matrix and the local mapping matrix of the first image. The global mapping matrix and the layout mapping matrix not only comprise global features, but also comprise local features, that is, when brightness and color are restored, the global features of the image are considered, the local features of the image are considered, that is, through a parameter sharing mechanism, the interaction of brightness and color is realized, and the loss of texture details of the restored low dynamic range image is further avoided. In addition, as the global feature is more focused on the brightness effect, the main processing of brightness is ensured by the fact that the weight value corresponding to the global feature in the first weight coefficient is larger than the weight value corresponding to the local feature; similarly, for the local features, the color effect is reflected more heavily, so that the main processing of the color is ensured by the fact that the weight value corresponding to the local feature in the second weight coefficient is larger than the weight value corresponding to the global feature. The probability of losing the texture details and color deviation of the image is further reduced, so that the detail and the color of the third image which is finally output from the whole to the part are clearer and more accurate.

In an implementation manner of the first aspect, correcting the brightness of the first image may include: firstly, texture information of a first image is obtained, global mapping matrix of the first image is subjected to interpolation processing according to the texture information, a processed mapping relation diagram is obtained, and brightness correction is carried out on the first image based on the mapping relation diagram so as to obtain a second image. For example, the second image may be obtained by the following formula.

Contrast(R，G，B)(x，y)＝input(R，G，B)(x，y)*Gain_map(x，y)

Wherein, contrast (R, G, B) is the second image, input (R, G, B) (x, y) is the first image, gain_map (x, y) is the map.

The first image is subjected to brightness correction based on the mapping relation diagram, so that a corrected second image is obtained, and the effect of image brightness alignment is ensured.

In an implementation manner of the first aspect, correcting the color of the first image may include: the color correction matrix is determined based on the second image and the local mapping matrix, for example, the local correction matrix may be interpolated by the second image to obtain the color correction matrix, and then the color of the first image is corrected based on the color correction matrix, for example, the RGB channels of the second image may be corrected based on the color correction matrix to determine the corrected image, i.e., the third image. That is, the third image is an image corrected for both brightness and color, i.e., an image of low dynamic range. For example, the third image may be obtained by the following formula.

R’(x，y)＝a₀(x,y)*R(x,y)+a₁(x,y)*G(x,y)+a₂(x，y)*B(x，y)

G’(x，y)＝b₀(x,y)*R(x,y)+b₁(x,y)*G(x,y)+b₂(x，y)*B(x，y)

B’(x，y)＝c₀(x,y)*R(x,y)+c₁(x,y)*G(x,y)+c₂(x，y)*B(x，y)

Wherein a ₀、a₁、a₂、b₀、b₁、b₂、c₀、c₁、c₂ is a coefficient in the color correction matrix; r (x, y), G (x, y), B (x, y) are second images of R channel, G channel, B channel; r ' (x, y), G ' (x, y) and B ' (x, y) are images of R channel, G channel and B channel after color correction matrix processing.

The partial decoupling of the colors and the brightness is ensured through the separate correction processing of the brightness and the colors, and the interaction of the colors and the brightness is ensured through the parameter sharing processing of the colors and the colors, so that the color accuracy of the finally output third image is effectively ensured, the detail is clear, and the experience of a user is improved.

In an implementation manner of the first aspect, for the above neural network, the neural network may be trained by using the brightness constraint and the color constraint as constraints of the image output, so as to ensure that the image output by the upper channel meets the constraints of the brightness and the image output by the lower channel meets the constraints of the color, which are obtained by processing the neural network and meet the corresponding constraints. Thereby ensuring the degree of completion of converting the high dynamic range image into the low dynamic range image.

In a second aspect, the present application provides an image processing apparatus having a function of implementing the behavior of an electronic device in the method of the first aspect. The functions may be realized by hardware, or may be realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above, for example, an input unit or module, a display unit or module, a processing unit or module.

In a third aspect, an electronic device is provided, the electronic device comprising: a processor; a memory; a camera module; and a computer program, wherein the computer program is stored on the memory, which when executed by the processor causes the electronic device to perform the image processing method as in the first aspect and any implementation thereof described above.

In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium comprising a computer program, which, when run on an electronic device, causes the electronic device to perform the method of any one of the first aspects above.

In a fifth aspect, a computer program product is provided comprising instructions which, when run on an electronic device, enable the electronic device to perform the image processing method of the first aspect and any implementation thereof.

In a sixth aspect, an embodiment of the present application provides a chip including a processor for invoking a computer program in a memory to perform an image processing method as in the first aspect and any implementation thereof.

It will be appreciated that the advantages achieved by the apparatus according to the second aspect, the electronic device according to the third aspect, the computer readable storage medium according to the fourth aspect, the computer program product according to the fifth aspect, and the chip according to the sixth aspect provided above may refer to the advantages in any one of the possible implementations of the first aspect and the advantages that are not described herein.

Drawings

FIG. 1A is a schematic diagram of an image processing method according to the related art;

FIG. 1B is a color-biased image provided by the related art;

FIG. 1C is a representation of a texture detail loss image provided by the related art;

fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a process flow of an image method according to an embodiment of the present application;

FIG. 4 is a flowchart of another image processing method according to an embodiment of the present application;

Fig. 5 is a schematic structural diagram of a chip system according to an embodiment of the present application.

Detailed Description

In the description of the present application, unless otherwise indicated, "and/or" in the present application is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural.

In the description of the present application, unless otherwise indicated, "a plurality" means two or more than two. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

In order to clearly describe the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and effect. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.

In embodiments of the application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g." in an embodiment should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion that may be readily understood.

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

Scenes under natural light, viewed at the angle of the human eye, tend to be more attractive than scenes seen through electronic devices. This is because the luminance relationship in the scene seen by the human eye is nonlinear, while the luminance relationship in the scene recorded by the electronic device is linear, so the dynamic range of the scene seen by the human eye is larger than the dynamic range that the electronic device can record. The so-called dynamic range (DYNAMIC RANGE, DR) is one of the most important parameters of the image sensor in the electronic device, and determines the range of intensity distribution from the darkest part (or called shadow part) to the brightest part (or called highlight part) that the image sensor can receive, i.e. determines the detail and level of the image captured by the electronic device. Where an HDR image is an image with a very high dynamic range, this means that the difference between the brightest and darkest parts in an HDR image is very large. That is, the purpose of high dynamic range imaging is to correctly represent a wide range of brightness in the real world from direct sunlight to darkest shadows.

However, the current image shot by the electronic device is often affected by natural light and the reason of the electronic device, and the brightness difference of the obtained HDR image is very uneven, i.e. the bright part is too exposed and the dark part is too dark. In addition, the dynamic range of most of the output devices such as displays and printers is much smaller than the ordinary high dynamic range, so that the output image does not reflect the real scene well. To solve this problem, tone mapping techniques have been developed. Tone mapping is the process of converting an HDR image to an LDR image, enabling the image presented to the user to reflect the real scene seen by the human eye. That is, tone mapping is a key step in reconstructing natural images.

Taking an electronic device as a mobile phone, a real scene shown as 11 in fig. 1A is taken as an example as a normal human eye. When a user uses an imaging module (such as the rear camera 10 in fig. 1A) in the mobile phone to capture a picture shown in the real scene 11, the brightness interval of the captured image may be larger due to the influence of factors such as light. As the image taken by the mobile phone through the rear camera 10 is shown as an image 12 in fig. 1A, it can be seen that some pictures lose details due to overexposure, i.e. the final output image cannot reflect the picture of the real scene. The captured image, such as image 12 in fig. 1A, may be converted into an LDR image by tone mapping and then presented to the user, so that the output image may better reflect the real scene.

In the related art, tone mapping techniques mainly achieve tone mapping through two or more neural networks connected in series. Among such techniques are typically included Dynamic Range Compression (DRC) networks and local contrast enhancement (Local Contrast Enhancement, LCE) networks. The DRC network is mainly responsible for realizing the processing of the image brightness. LCE networks are mainly responsible for the processing of image colors. For example, the HDR image may be first enhanced in brightness by the DRC network, and then adjusted in color details by the LCE network, so as to finally output the LDR image.

However, the disadvantages of this technique are mainly manifested in the following aspects:

On the other hand, in the processing mode of connecting two or more neural networks in series, errors of a previous stage network are transmitted to a next stage network, and error accumulation is easy to form. For example, in the above technique, the output of the DRC network is the input of the LCE network, and in this architecture, if an error occurs in the DRC network during processing, the error is transferred to the LCE network of the next stage, and an error accumulation is formed, so that the effect of the LDR image output last, such as color deviation, is affected. As shown in fig. 1B, fig. 1B is an image of a color deviation provided by the related art.

On the other hand, LCE networks mainly employ color correction matrices (Color correction matrix, CCM) for color detail processing. However, because the LCE network is a separate network, negative truncation and high order truncation of the image is required before color processing of the full image using the color correction matrix. Such an operation may result in loss of texture details in the highly saturated region of the image, resulting in loss of texture details in the LDR image that is ultimately output. As shown in fig. 1C, fig. 1C is an image with a texture detail loss provided in the related art. The pixels of the image generally have a value ranging from 0 to 255, but the actual value of the pixels of the image may be outside the range, so as to ensure the brightness of the image (for example, an image with a value ranging from 0 to 625). Negative truncation refers to the image input to the LCE network requiring removal of pixels whose pixel value is before 0. High order clipping refers to the image input to the LCE network requiring the removal of pixels with pixel values after 255.

On the other hand, the technology adopts two independent neural networks to separately process the color and the brightness of the image, but the color and the brightness cannot be completely decoupled in actual processing. That is, when the color is processed, the brightness is often changed, so that the color of the image is often deviated; when brightness is handled, color is also affected, resulting in loss of information. So that it cannot be guaranteed that the image can truly restore the actual scene picture.

In summary, the second type of technique described above causes problems of loss of texture details and color cast.

In order to solve the above problems, an embodiment of the present application provides an image processing method, which processes an image color and brightness through two independent paths, so as to realize partial decoupling of brightness and color. In addition, through a parameter sharing mechanism, interaction of brightness and color is realized. The problem of color deviation and texture detail loss in the process of converting an HDR image into an LDR image is solved, the wonderful property of an output image is ensured, and the experience of a user is improved.

The device for performing image processing in the embodiment of the present application may be an electronic device, such as a mobile phone, a tablet computer, a smart watch, a desktop, a laptop, a handheld computer, a notebook, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cellular phone, a personal digital assistant (personal digitalassistant, PDA), an augmented reality (augmented reality, AR) \virtual reality (VR) device, or the like, and the embodiment of the present application is not limited to a specific form of the electronic device.

In addition, it should be noted that the image processing method provided in this embodiment may be applied to a process of performing tone mapping processing on a captured HDR image to obtain an LDR image. The method can also be applied to preprocessing of target (e.g. object) detection, preprocessing of human image recognition and other scenes needing image enhancement, and the method is not limited only to the application scene of the scheme.

For example, taking the mobile phone 200 as an example of the electronic device, fig. 2 shows a schematic structural diagram of the mobile phone 200. As shown in fig. 2, the mobile phone 200 may include a processor 210, an external memory interface 220, an internal memory 221, a mobile communication module 230, a wireless communication module 240, a charge management module 250, a power management module 260, a battery 270, an antenna 1, an antenna 2, an audio module 280, a speaker 280A, a receiver 280B, a microphone 280C, an earphone interface 280D, a camera 290, a display 291, and a sensor module 292, etc. Where the sensor module 292 may include a distance sensor 292A, an ambient light sensor 292B, and the like.

It should be understood that the structure illustrated in the embodiment of the present application is not limited to the specific embodiment of the mobile phone 200. In other embodiments of the application, the handset 200 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. For example, the mobile phone 200 may further include: subscriber identity module (subscriber identification module, SIM) card interface, etc. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 210 may include one or more processing units such as, for example: processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural-Network Processor (NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may hold instructions or data that the processor 210 has just used or recycled. If the processor 210 needs to reuse the instruction or data, it may be called directly from the memory. Repeated accesses are avoided and the latency of the processor 210 is reduced, thereby improving the efficiency of the system. In some embodiments, processor 210 may include one or more interfaces.

The charge management module 250 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. The battery 270 is charged by the charging management module 250, and the mobile phone 200 can be powered by the power management module 250.

The power management module 260 is used to connect the battery 270, and the charge management module 260 and the processor 210. The power management module 260 receives input from the battery 270 and/or the charge management module 250 to power the processor 210, the internal memory 221, the display 291, the camera 290, the wireless communication module 240, and the like. The power management module 260 may also be configured to monitor battery capacity, battery cycle times, battery health (leakage, impedance) and other parameters. In other embodiments, the power management module 260 may also be disposed in the processor 210. In other embodiments, the power management module 260 and the charge management module 250 may be disposed in the same device.

The wireless communication function of the mobile phone 200 may be implemented by the antenna 1, the antenna 2, the mobile communication module 230, the wireless communication module 240, the modem processor, the baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the handset 200 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 230 may provide a solution for wireless communication including 2G/3G/4G/5G, etc. applied to the handset 200. The mobile communication module 230 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 230 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation.

The wireless communication module 240 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (WIRELESS FIDELITY, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), near field communication (NEAR FIELD communication, NFC), infrared (IR), etc. applied to the mobile phone 200.

In some embodiments, antenna 1 and mobile communication module 230 of handset 200 are coupled, and antenna 2 and wireless communication module 240 are coupled, such that handset 200 may communicate with a network and other devices via wireless communication technology.

The mobile phone 200 implements display functions through a GPU, a display 291, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 291 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

The display 291 is for displaying images, videos, and the like. The display 291 includes a display panel. For example, the display 291 may be a touch screen. In some embodiments of the present application, after the user opens the camera application, the display 291 may be used to display the preview stream captured by the camera 290.

Camera 290 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the cell phone 200 may include 1 or N cameras 290, N being a positive integer greater than 1. In some embodiments of the present application, after the mobile phone 200 receives the operation of opening the camera application by the user, the mobile phone 200 may control the camera 290 to be turned on. After camera 290 is turned on, camera 290 may be used to capture a preview image of the current scene. After receiving the operation of clicking the photographing by the user, the mobile phone 200 may photograph an image using the camera 290. In case the image is an HDR image, the inventive solution may be employed for processing to obtain a corresponding LDR image. The cell phone 200 may save the obtained LDR image to a gallery.

Internal memory 221 may be used to store computer executable program code that includes instructions. The internal memory 221 may include a storage program area and a storage data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data (e.g., audio data, phonebook, etc.) created during use of the handset 200, etc. In addition, the internal memory 221 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like. The processor 210 performs various functions of the mobile phone 200 and data processing by executing instructions stored in the internal memory 221 and/or instructions stored in a memory provided in the processor 210. The audio module 280 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 280 may also be used to encode and decode audio signals. In some embodiments, the audio module 280 may be disposed in the processor 210, or some functional modules of the audio module 280 may be disposed in the processor 210.

Speaker 280A, also known as a "horn," is used to convert audio electrical signals into sound signals. The handset 200 may listen to music, or to hands-free calls, through the speaker 280A.

A receiver 280B, also known as a "earpiece", is used to convert the audio electrical signal into a sound signal. When the phone 200 is answering a phone call or voice message, the phone may be answering voice by placing the receiver 280B close to the human ear.

Microphone 280C, also known as a "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can sound near the microphone 280C through the mouth, inputting a sound signal to the microphone 280C. The handset 200 may be provided with at least one microphone 280C. In other embodiments, the mobile phone 200 may be provided with two microphones 280C, and may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the mobile phone 200 may further be provided with three, four or more microphones 280C to collect sound signals, reduce noise, identify the source of sound, implement directional recording, etc.

The earphone interface 280D is used to connect a wired earphone. The earphone interface 280D may be a USB interface 210 or a 3.5mm open mobile TERMINAL P atform (OMTP) standard interface, a american cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

Distance sensor 292A for measuring distance. The cell phone 200 may measure the distance by infrared or laser. In some embodiments, the cell phone 200 may range using the distance sensor 292A to achieve quick focus when shooting a scene.

The ambient light sensor 292B is used to sense ambient light level. The cell phone 200 may adaptively adjust the brightness of the display 291 according to the perceived ambient light level. The ambient light sensor 292B may also be used to automatically adjust white balance when taking a photograph. The ambient light sensor 292B may also cooperate with the proximity light sensor 292B to detect if the phone 200 is in a pocket to prevent false touches.

The specific flow of implementing tone mapping by an electronic device is described below in conjunction with fig. 3 and 4. As shown in fig. 4, fig. 4 is a flowchart of an image processing method according to an embodiment of the present application, where the method includes S401 to S406.

S401, acquiring a first image; the first image is a high dynamic range image.

The first image may be a high dynamic range image that needs to be tone mapped. In other words, the first image may refer to an image in which a difference in brightness is relatively large due to a large dynamic range, or in which there is a bright portion overexposure or a dark portion overexposure. I.e. there is an unclear condition of the content presented by the first image.

In some examples, the difference in brightness of the first image is relatively large, possibly due to the angle of the natural light at the time of photographing. For example, a first image obtained by backlight shooting is dark in main body and bright in background, so that details of the main body in the first image are not clearly seen, and only shadows and color deviations exist. In other examples, the dynamic range of the electronic device itself may be narrow, which is the case when the electronic device is said to have a reduced image quality. For example, the daily scene seen by naked eyes is shocked, but the daily scene recorded by the electronic equipment is not shocked by naked eyes.

In some embodiments, the acquiring the first image may specifically be an image captured by a camera of the electronic device. For example, the electronic device may open the camera application after receiving an operation of opening the camera application by the user. Then, in response to shooting operation of a user, the electronic device can acquire an image through a camera of the electronic device, and then a first image can be obtained. In another example, in a scene requiring image recognition, the electronic device may respond to the operation of the user, and acquire the image of the user by using the camera, so as to obtain the first image. In other embodiments, the first image may also be received from other devices, or downloaded from a network, or read from a memory of the electronic device.

Considering that it is impossible to completely decouple the color and brightness of an image, there is a case where the processing of the two affects each other. That is, when the brightness of an image is processed, it may be possible to ensure restoration of the brightness of the image, but affect the color; similarly, when processing the color of an image, it may be possible to ensure that the color is restored, but the brightness of the image is changed accordingly. Therefore, in the embodiment of the present application, the color and the luminance are reduced by two paths, such as an upper path and a lower path, respectively. Parameters for performing the color and luminance restoration process, such as a global mapping matrix and a local mapping matrix, may be obtained before performing the color and luminance restoration process through the two paths, respectively. That is, after the first image is acquired, the global mapping relationship and the local mapping relationship of the first image to the corresponding low dynamic range image in terms of brightness and color may be acquired, that is, the global mapping matrix and the local mapping matrix of the first image are acquired for subsequent processing of the color and brightness. The process of obtaining the global mapping matrix and the local mapping matrix of the first image may specifically include: S402-S406.

S402, downsampling processing is carried out on the first image.

Where downsampling refers to sampling a sequence of samples once at intervals of a few samples, so that a new sequence is obtained as a result of downsampling the original sequence. Downsampling the image can be understood simply by scaling the image down. In the embodiment of the application, the first image may be subjected to downsampling processing to obtain a downsampled first image. For example, the pixel values in the first image may be decimated to obtain a downsampled first image. For example, in connection with fig. 3, where the first image is represented as an input image (fullinput), full input may be downsampled to obtain a downsampled first image, as shown in fig. 3, in a low input preview image (low input_ CCM PREVIEW). By performing downsampling processing on the image, the calculation amount in the subsequent image processing process can be reduced, so that the processing efficiency is improved.

For example, assuming that the first image is a 1024-Pixel (px) x 1024 px-size image, the first image obtained after the downsampling process may be a 512px x 512 px-size image. It should be noted that the first image after the downsampling process in the above embodiment is exemplified by a 512px x 512px size, and may also be a 64px x 64px size, which is not limited in particular.

In some embodiments, S402 is an optional step.

S403, acquiring image features of the first image, wherein the image features comprise one or more of color features, texture features, shape features and spatial relationship features.

In some embodiments of the present application, the processing unit in the neural network may be utilized to extract image features of the first image. The neural network is a nonlinear and adaptive information processing system formed by interconnecting a large number of processing units. As an example, the neural network may include a feature processing unit. For example, with continued reference to fig. 3, after the first image (e.g., full input) is acquired and the first image is downsampled, low input_ CCM PREVIEW, the downsampled first image may be input to the feature processing unit of the neural network, so that it completes feature extraction of the downsampled first image, and an image feature of the first image is output, (low level feature) represents the feature. The image feature of the first image is two-dimensional data, which can be simply understood as data of a plane, and in the embodiment of the present application, the image feature can be understood as a feature map of the first image. The image features may include one or more of color features, texture features, shape features, spatial relationship features.

Wherein the color features are used to describe the surface properties of the scene to which the image or image area corresponds. The texture features are used to describe the surface properties of the scene to which the image or image region corresponds. Unlike color features, texture features are not pixel value based features, which require statistical calculations in areas containing multiple pixel values. Shape features are represented in two classes, one is outline features and the other is region features. The contour features of the image are mainly directed to the outer boundary of the object, whereas the region features of the image are related to the whole shape region. The spatial relationship features are used to describe the mutual spatial position or relative direction relationship between the multiple objects segmented in the image, and these relationships may also be classified into a connected/adjacent relationship, an overlapping/overlapping relationship, an inclusion/containment relationship, and the like.

In the processing of the first image by the feature processing unit of the neural network, feature extraction is performed by multi-layer convolution. In this process, the resulting image features are also subjected to compression processing. In other words, the image feature of the obtained first image is an image feature map of a smaller size than the first image (or the first image after the downsampling process). In the above example, the first image is an image with size of 1024px x 1024px, the first image obtained after the processing is an image with size of 512px x 512px, and the image features of the first image obtained after the processing by the feature processing unit may be images with sizes of 64px x 64 px.

S404, determining global features and local features of the first image based on the image features.

Specifically, after the image features of the first image are acquired, the electronic device may determine global features and local features corresponding to the first image based on the image features.

The global features are used for representing the global mapping relation between the high dynamic range image and the low dynamic range image in terms of color and brightness. The global mapping relationship is that the brightness and color values in the global area of the high dynamic range image and the global area of the low dynamic range image are in one-to-one correspondence. That is, the number of pixel values of the global area of the high dynamic range image and the number of pixel values of the global area of the low dynamic range image are equal. For example, the typical value range of the pixel value of the high dynamic range image is [0-255], and the typical value range of the pixel value of the low dynamic range image is also [0-255]. The local features are used to characterize the mapping of the high dynamic range image to the local of the low dynamic range image in terms of color and brightness. The local mapping relationship means that the pixel values of the luminance and the color in the partial region of the high dynamic range image and the partial region of the low dynamic range image are in one-to-one correspondence. That is, the number of pixel values of the high dynamic range image and the number of pixel values of the low dynamic range image are equivalent in the local area. If the high dynamic range image is [0-10], the low dynamic range image is also [0-10], or the high dynamic range image is [0-10], the low dynamic range image is also [5-15]. In short, the local features can reflect features of local areas of the first image, such as features of particularly bright or particularly dark areas in the first image, so as to facilitate targeted processing, such as adaptive dimming for dark areas in the image, and adaptive dimming for bright areas in the image. In this embodiment, the local feature of the first image may include a plurality of. As can be understood simply as a map of the area of the image features of the first image divided into a plurality of small areas, for example 4 3*3.

In some embodiments of the present application, global features and local features of the first image may be extracted using a processing unit in the neural network. As an example, the neural network may further include a global feature processing unit and a local feature processing unit. For example, with continued reference to fig. 3, after the image features of the first image are acquired, or the feature processing unit outputs low level feature, the low level feature may be input to a global feature processing unit and a local feature processing unit. Thereafter, the Global Feature processing unit may perform extraction of the Global Feature of the first image based on the image Feature of the first image to obtain the Global Feature of the first image, as represented by Global Feature (global_feature) in fig. 3. The Local Feature processing unit may perform extraction of the Local Feature of the first image based on the image Feature of the first image to obtain the Local Feature of the first image, as represented by Local Feature (local_feature) in fig. 3.

The extraction of the global features of the first image may be achieved by a fully connected layer, among other things, as an example. The extraction of the local features of the first image may be achieved by convolution and downsampling.

It will be appreciated that both the global feature processing unit and the local feature processing unit are pre-trained, with their processing unit cores being fixed. For example, the output obtained by inputting the image features of the first image to the global feature processing unit may be a global feature of 64px x 64px size. This is because the kernel of the global feature processing unit is 64px x 64px, so the global feature output last is an image of 64px x 64px size. For the local feature, an output obtained by inputting the image feature of the first image to the local feature processing unit is a local feature of a plurality of local region sizes. The multiple local area sizes refer to splitting the image feature of the first image into n×m local areas, for example, N and M are set to 3, and the local area sizes are 3px×3px. Assuming that the convolution kernel of the local feature processing unit is 3px×3px, the image features of the first image are divided into a plurality of images with the size of 3px×3px by the processing of the local feature processing unit, for example, the final output local feature is an image with the size of 0-10 of four 3px×3px.

It should be noted that, the number of cores of the global feature processing unit or the local feature processing unit is set according to actual needs, and is not limited to the size of the cores in the above example, but may be cores of other sizes, for example, the cores of the global feature processing unit are 512px x 512px, and the cores of the local feature processing unit are 24px x 24px.

Considering the difference in the brightness and the color in the image characteristics, the brightness focuses on the global characteristics and the color focuses on the local characteristics. After the global and local features of the first image are acquired, in some embodiments, the brightness of the first image may be reduced based on the global feature and the color may be reduced based on the local feature. In other embodiments, it is considered that the brightness and the color cannot be completely decoupled or are mutually influenced, so that not only the global feature but also the local feature of the image can be considered when the brightness is reduced, and similarly, when the color is reduced, not only the local feature of the image but also the local feature can be considered. Specifically, a global mapping matrix corresponding to the global feature and a local mapping matrix corresponding to the local feature can be obtained and used for subsequent restoration of brightness and color respectively. I.e. S405 and S406 described below are performed.

S405, determining a global mapping matrix based on the global features, the local features and the first weight coefficients.

Wherein the global features and the local features may be multiplied by different weight coefficients to obtain a global mapping matrix.

In some embodiments of the present application, the global mapping matrix may be obtained through a gating mechanism (GATE FIRES MECHANISM, GFM). For example, in connection with fig. 4, after the Global Feature (global_feature) and the Local Feature (local_feature) of the first image are acquired, the Global Feature (global_feature) and the Local Feature (local_feature) may be input to the GFM. Thereafter, the GFM may output a global mapping matrix corresponding to the global feature, as represented by coeff1 in fig. 4, based on the input global feature, the local feature, and the first weight coefficient. That is, the global mapping matrix may be determined based on the global features and the local features and the first weight coefficients in the GFM. For example, the global and local features are multiplied by different weight values, respectively, to determine a global mapping matrix. The global mapping matrix is mainly used for the subsequent reduction processing of brightness, that is, the global feature processing is more focused, so that the first weight coefficient should focus on the global feature.

As one example, the first weight coefficient may include a plurality of weight values for the global feature and the local feature. In addition, since there may be a plurality of local features, there are also a plurality of weight values for the local features, and there are one-to-one correspondence with the plurality of local features. Wherein the weight value for the global feature is different from the weight value for the local feature among the plurality of weight values. For example, the weight value for a global feature is greater than the weight value for a local feature. The weight values for different local features may be the same or different.

For example, with global feature denoted G, there are two local features denoted by L1 and L2, respectively, as examples, the first weight coefficient may comprise three weight values, respectively for global feature G, such as a, and for two local features L1 and L2, such as B and C, respectively. After the global feature G, the two local features L1 and L2 are input to the GFM, the global mapping matrix coeff1 output by the GFM can be expressed simply as: coeff1=g a+l 1x b+l2 x C. Wherein A > B, A > C.

S406, determining a local mapping matrix based on the global feature, the local feature and the second weight coefficient.

Similar to S405, the global features and the local features may be multiplied by different weight coefficients to obtain a local mapping matrix.

In some embodiments, the local mapping matrix may be obtained by GFM. For example, in connection with fig. 3, global features (global_features) and Local features (local_features) may be input to GFM. Thereafter, the GFM may output a local mapping matrix corresponding to the local feature based on the input global feature, the local feature, and the second weight coefficient, as represented by coeff2 in fig. 3. That is, the local mapping matrix may be determined based on the global features and the local features and the second weight coefficients in the GFM. The local mapping matrix is mainly used for the subsequent color restoration processing, that is, the local feature processing is more focused, so that the second weight coefficient should focus on the local feature.

Similarly, the second weight coefficient may include a plurality of weight values for the global feature and the local feature. Among the plurality of weight values, the weight value for the global feature is different from the weight value for the local feature. For example, the weight value for a local feature is greater than the weight value for a global feature.

For example, continuing with the global feature denoted G, there are two local features denoted by L1 and L2, respectively, as examples, and the second weight coefficient may comprise three weight values, respectively for the global feature G, such as X, and for the two local features L1 and L2, such as Y and Z, respectively. After the global feature G, the two local features L1 and L2 are input to the GFM, they may also output a local mapping matrix coeff2. The local mapping matrix coeff2 can be expressed simply as: coeff2=g x+l1Y and G X +L2.z. Wherein Y > X, Z > X.

Note that, the present embodiment does not limit the execution sequence of S405 and S406. S405 may be performed first, S406 may be performed first, S405 may be performed second, or S405 and S406 may be performed simultaneously.

Based on the above, it can be understood that, in this embodiment, the global mapping matrix and the global feature are both used to indicate the global mapping relationship from the first image to the corresponding low dynamic range image in brightness and color, where the difference between the two is that, in the global mapping matrix, not only the global feature is considered, but also the local feature is added. The local mapping matrix and the local feature are used for indicating the local mapping relation from the first image to the corresponding low dynamic range image in brightness and color, and the difference between the two is that the local mapping matrix not only considers the local feature, but also adds the global feature.

After the global mapping matrix and the local mapping matrix are acquired, color and brightness reduction processing can be performed through different channels (such as an upper channel and a lower channel) based on the global mapping matrix and the local mapping matrix, respectively. For example, the brightness of the first image may be corrected based on the global mapping matrix to obtain the second image. Thereafter, the color of the first image may be corrected based on the local mapping matrix and the second image to obtain a third image. Finally, a third image may be output, the third image being a low dynamic range image corresponding to the first image. Specifically, the method can comprise the following steps: S407-S411.

S407, acquiring a texture image of the first image.

In some embodiments of the present application, a processing unit in the neural network may be utilized to extract a texture image of the first image. As an example, the neural network may further comprise a guidance processing unit. For example, with continued reference to fig. 3, the first image may be input to the Guidance processing unit of the neural network so that it completes acquisition of the texture image of the first image, i.e., so that it outputs the texture image (may also be referred to as a guide_map) of the first image. In particular, the acquisition of the texture image of the first image avoids to some extent the loss of details of the first image.

S408, determining a mapping relation diagram of the first image based on the texture image and the global mapping matrix.

Specifically, after acquiring the texture image of the first image, the electronic device may determine a mapping relationship diagram of the first image through the upper channel based on the texture image and the global mapping matrix.

In some embodiments of the present application, a processing unit in the neural network may be utilized to determine a map of the first image from the texture image and the global mapping matrix. As an example, the neural network may further comprise a decoding processing unit 1. For example, with continued reference to fig. 3, after the texture image of the first image is acquired, or after the guide processing unit outputs the guide_map, the guide_map and coeff1 may be input to the decoding processing unit 1 (may also be referred to as a decoder_drc). After that, the decoding processing unit 1 performs decoding processing based on the guide_map and coeff1 to obtain a map of the first image (which may be referred to as a gain_map, for example).

Specifically, the decoding processing unit 1 performs guided interpolation processing on coeff1 through the guide_map to obtain a map of the first image. For example, the texture image and the global mapping matrix are input to a decoder_drc, resulting in an output gain_map.

The interpolation, sometimes referred to as "reset sample", is a method of increasing the pixel size of an image without generating pixels, calculating the color of a missing pixel using a mathematical formula based on the color of surrounding pixels, and artificially increasing the resolution of the image.

It should be understood that the map obtained in step S408 is determined mainly by interpolating the global mapping matrix, and the interpolation increases the size of the image. In an embodiment of the present application, the size of the specific interpolation may be determined based on the size of the first image. That is, this step may be understood as performing size reduction on the first image, thereby obtaining a map of the first image. In the above embodiment, the image size is compressed (for example, downsampling, feature extraction, etc. are performed) and then the image is restored, so that the calculation amount in the neural network processing can be reduced, and the probability of error occurrence can be reduced. For example, the decoding processing unit 1 outputs a global full resolution map of 1024px x 1024px, i.e. the map described above.

Based on the above, it can be understood that, in this embodiment, the mapping relationship diagram and the global mapping matrix are both used to indicate the global mapping relationship from the brightness and the color of the first image to the corresponding low dynamic range image, and the difference between the two is that the mapping relationship diagram is the image after the global mapping matrix performs up-sampling, that is, the image after performing the image size reduction.

In one example, in the step S401-S407, the size of the first image is 1024px x 1024px, the first image is subjected to downsampling processing to obtain a first image with a size of 512px x 512px, then image features of the first image are obtained, feature maps of 64px x 64px are obtained, and then the size of the global mapping matrix determined by the global features is 64px x 64px, and based on the texture image and the global mapping matrix, the size of the mapping relation map of the first image is determined to be 1024px x 1024px.

S409, correcting the brightness of the first image based on the mapping relation graph to obtain a second image.

Specifically, after the mapping relation diagram is determined, the electronic device may perform luminance restoration processing on the first image through the mapping relation diagram, to obtain a second image after the luminance restoration processing.

In some embodiments of the present application, the intermediate image may be obtained after performing luminance reduction processing on three channels (R channel, G channel, B channel) of the first image based on the map obtained in the upper channel. As an example, continuing with fig. 3, an intermediate image may be determined from the map and the first image, e.g., a product of the map and the first image may be calculated to obtain an intermediate image, as represented by image1 (img 1) in fig. 3. The intermediate image may be the second image in the present application.

The second image may be determined, for example, by the following formula:

Contrast(R,G,B)(x,y)＝input(R,G,B)(x,y)*Gain_map(x,y)

wherein, contrast (R, G, B) is the second image, i.e. img1; input (R, G, B) (x, y) is the first image, fullinput; gain_map (x, y) is a map.

The above-described S407 to S409 are reduction processing performed on the luminance in the upper channel. The following describes the procedure of the color reduction processing in the lower channel, including S410 to S411.

S410, determining a color correction matrix of the second image based on the second image and the local mapping matrix.

Specifically, after determining the second image, the electronic device may determine a color correction matrix of the second image through the down channel based on the second image and the local mapping matrix. The color correction matrix may be a 3*3 matrix, that is to say the color correction matrix may comprise 9 coefficients. The second image can be color corrected using the 9 coefficients of the color correction matrix.

In some embodiments of the application, a processing unit in the neural network may be utilized to determine a color correction matrix for the second image. As an example, the neural network may further comprise a decoding processing unit 2. For example, with continued reference to fig. 3, after img1 is acquired, img1 and coeff2 may be subjected to a decoding process to obtain a color correction matrix for the second image, which may be referred to as CC (x, y).

Similarly, the decoding processing unit 2 performs interpolation processing on coeff2 through img1 to obtain a color correction matrix of the first image. For example, the second image and the local mapping matrix are input to a decoder_cc, resulting in an output CC (x, y). That is, the decoding neural network processing unit 2 outputs a global full resolution map image of 1024px×1024px.

In one example, in the steps of S401 to S407, the first image is 1024px x 1024px, the first image is subjected to downsampling to obtain a first image with a size of 512px x 512px, then image features of the first image are obtained, a plurality of feature maps of 64px x 64px are obtained, and then the size of the local mapping matrix determined by the local features is 64px x 64px, and based on the second image and the local mapping matrix, the size of the color correction matrix of the first image is determined to be 1024px x 1024px.

S411, correcting the color of the second image based on the color correction matrix to obtain a third image.

Specifically, after the color correction matrix is determined, the electronic device may perform color reduction processing on the second image through the color correction matrix, to obtain a third image after the color reduction processing.

In some embodiments of the present application, the RGB channels of the second image may be color corrected based on the color correction matrix obtained in the lower channel to obtain a corrected third image, as represented by img2 in fig. 3. For example, color correction is performed based on the product of the RGB channels of CC (x, y) and img2, and a third image is output.

The third image may be determined, for example, by the following formula:

R’(x，y)＝a₀(x,y)*R(x,y)+a₁(x,y)*G(x,y)+a₂(x，y)*B(x，y)

G’(x，y)＝b₀(x,y)*R(x,y)+b₁(x,y)*G(x,y)+b₂(x，y)*B(x，y)

B’(x，y)＝c₀(x,y)*R(x,y)+c₁(x,y)*G(x,y)+c₂(x，y)*B(x，y)

It can be understood from the above that, in the embodiment of the present application, tone mapping can be implemented by using a neural network, that is, the conversion of the image with a high dynamic range into the image with a low dynamic range is implemented, so that error accumulation is avoided, and the color accuracy of the finally output low dynamic image is ensured. In addition, the neural network comprises two independent channels (such as an upper channel and a lower channel) to respectively realize the brightness and color restoration treatment. It can be seen that the luminance and color partial decoupling is achieved by the two independent channels. In addition, when brightness and color reduction is carried out, not only global characteristics of the image but also local characteristics of the image are considered, namely, interaction between brightness and color is realized through a parameter sharing mechanism. And the loss of texture details of the restored low dynamic range image is avoided. In addition, by performing color correction processing in the neural network, negative number truncation and high-order truncation of the image are avoided, and accurate restoration of texture details is further ensured.

It should be noted that, before tone mapping is performed by applying the neural network provided by the present application, the neural network needs to be trained. In the process of training the neural network, after the images output by the upper channel, such as img1, and the images output by the lower channel, such as img2, meet the constraint conditions of the neural network, the iterative training of the neural network is stopped, and the neural network which can be finally used for tone application is obtained.

For example, the constraint of brightness in the neural network needs to be complied with for img1 output from the upper channel; the constraints of the colors in the neural network need to be complied with for img2 output from the lower channel.

As an example, the constraint function of luminance may be determined based on an average of differences between luminance values of img1 and luminance values of full input. For example, the constraint function of luminance may be the following formula:

Light_LOSS＝Avg(F(Img1)-F(Label))。

Wherein, light_loss is used for representing the constraint function of brightness, avg is the mean value, label is the first image, and F is the formula for calculating brightness.

As another example, the constraint function of the color is determined according to the difference between the RGB image and the LAB image, e.g., the constraint function of the color may be the following formula:

O_L，O_A，O_B＝rgb2lab(Img2)

L_L，L_A，L_B＝rgb2lab(Label)

SC＝1+0.0225*(C₁+C₂)

SH＝1+0.0075*(C₁+C₂)

delta_a＝O_A-L_A

delta_b＝O_B-L_B

delta_C＝C1-C2

delta_L＝O_L-L_L

delta_H＝delta_a**2+delta_b**2-delta_C**2

Wherein color_LOSS is used for representing a constraint function of Color, the Label image is a standard image, O_L is the brightness of the img2 image, O_A is the chroma of the img2 image, O_B is the hue of the img2 image, L_L is the brightness of the Label image, O_A is the chroma of the Label image, O_B is the hue of the Label image, C ₁ is the hue and chroma total value of the img2 image, C ₂ is the hue and chroma total value of the Label image, SC is the weight coefficients of the img2 image and the Label image under the coefficient of 0.0225, and SH is the weight coefficients of the img2 image and the Label image under the coefficient of 0.0075; delta_A is the color difference of red/green of the img2 image and the Label image, delta_B is the color difference of yellow/blue of the img2 image and the Label image, delta_C is the color difference of the img2 image and the Label image, delta_L is the brightness difference of the img2 image and the Label image, and delta_H is the color loss value of the img2 image and the Label image. It should be noted that the 0.0225 coefficient and the 0.0075 coefficient are preferable under the current constraint condition, and can be 0.045 and 0.015, or other values, and are not limited only herein.

Other embodiments of the present application provide an electronic device that may include: the display screen, the camera, the memory and the one or more processors. The display, camera, memory and processor are coupled. The memory is for storing computer program code, the computer program code comprising computer instructions. When the processor executes the computer instructions, the electronic device may perform the functions or steps performed by the mobile phone in the above-described method embodiments. The structure of the electronic device may refer to the structure of the mobile phone shown in fig. 2.

The present application also provides a chip system, as shown in fig. 5, the chip system 500 includes at least one processor 501 and at least one interface circuit 502. The processor 501 and the interface circuit 502 may be interconnected by wires. For example, interface circuitry 502 may be used to receive signals from other devices (e.g., a memory of an electronic apparatus). For another example, interface circuitry 502 may be used to send signals to other devices (e.g., processor 501). Illustratively, the interface circuitry 502 may read instructions stored in the memory and send the instructions to the processor 501. The instructions, when executed by the processor 501, may cause the electronic device to perform the various steps of the embodiments described above. Of course, the system-on-chip may also include other discrete devices, which are not particularly limited in accordance with embodiments of the present application.

The embodiment of the application also provides a computer storage medium, which comprises computer instructions, when the computer instructions run on the electronic equipment, the electronic equipment is caused to execute the functions or steps executed by the mobile phone in the embodiment of the method.

The embodiment of the application also provides a computer program product which, when run on a computer, causes the computer to execute the functions or steps executed by the mobile phone in the above method embodiment.

It will be apparent to those skilled in the art from this description that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

acquiring a first image; the first image is a high dynamic range image;

Acquiring a global mapping matrix and a local mapping matrix of the first image; the global mapping matrix is used for indicating the global mapping relation between the brightness and the color of the first image and the corresponding low dynamic range image, and the local mapping matrix is used for indicating the local mapping relation between the brightness and the color of the first image and the corresponding low dynamic range image;

Correcting the brightness of the first image based on the global mapping matrix to obtain a second image;

And correcting the color of the first image based on the local mapping matrix and the second image to obtain a third image, and outputting the third image, wherein the third image is a low dynamic range image corresponding to the first image.

2. The method of claim 1, wherein the acquiring the global mapping matrix and the local mapping matrix of the first image comprises:

Acquiring image features of the first image, wherein the image features comprise one or more of color features, texture features, shape features and spatial relationship features;

Determining global features and local features of the first image based on the image features;

determining the global mapping matrix based on the global feature, the local feature and a first weight coefficient;

The local mapping matrix is determined based on the global feature, the local feature, and a second weight coefficient.

3. The method of claim 2, wherein prior to the acquiring the image features of the first image, the method further comprises:

and carrying out downsampling processing on the first image.

4. A method according to any of claims 1-3, wherein correcting the brightness of the first image based on the global mapping matrix to obtain a second image comprises:

acquiring a texture image of the first image;

determining a mapping relation diagram of the first image according to the texture image and the global mapping matrix;

And correcting the brightness of the first image according to the mapping relation diagram so as to obtain the second image.

5. The method of claim 4, wherein determining the map of the first image from the texture image and the global mapping matrix comprises:

And interpolating the global mapping matrix based on the texture image to obtain the mapping relation diagram.

6. The method of any of claims 1-5, wherein correcting the color of the first image based on the local mapping matrix and the second image to obtain a third image comprises:

Determining a color correction matrix for the first image based on the second image and the local mapping matrix;

And correcting the color of the first image based on the color correction matrix to obtain the third image.

7. The method of claim 6, wherein the determining a color correction matrix for the first image based on the second image and the local mapping matrix comprises:

The local mapping matrix is interpolated based on the second image to obtain the color correction matrix.

8. The method according to claim 6 or 7, wherein correcting the color of the first image based on the color correction matrix to obtain the third image comprises:

And performing color correction on RGB channels of the second image based on the color correction matrix to obtain the third image.

9. An electronic device, the electronic device comprising: a memory and one or more processors; the memory is coupled with the processor;

Wherein the memory is for storing computer program code, the computer program code comprising computer instructions; the computer instructions, when executed by the processor, cause the electronic device to perform the method of any of claims 1-8.

10. A computer-readable storage medium comprising computer instructions;

The computer instructions, when run on an electronic device, cause the electronic device to perform the method of any one of claims 1-8.