WO2021143284A1 - 一种图像处理的方法、装置、终端以及存储介质 - Google Patents

一种图像处理的方法、装置、终端以及存储介质 Download PDF

Info

Publication number
WO2021143284A1
WO2021143284A1 PCT/CN2020/125611 CN2020125611W WO2021143284A1 WO 2021143284 A1 WO2021143284 A1 WO 2021143284A1 CN 2020125611 W CN2020125611 W CN 2020125611W WO 2021143284 A1 WO2021143284 A1 WO 2021143284A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
gain compensation
compensation array
ryb
grid
Prior art date
Application number
PCT/CN2020/125611
Other languages
English (en)
French (fr)
Inventor
赵琳
姚添宇
卢曰万
胡宏伟
郜文美
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to US17/793,211 priority Critical patent/US20230079582A1/en
Priority to EP20913573.0A priority patent/EP4072131A4/en
Publication of WO2021143284A1 publication Critical patent/WO2021143284A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/85—Camera processing pipelines; Components thereof for processing colour signals for matrixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • H04N25/611—Correction of chromatic aberration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/67—Circuits for processing colour signals for matrixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/77—Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • This application belongs to the field of data processing technology, and in particular relates to an image processing method, device, terminal, and storage medium.
  • the red, yellow and blue RYB image sensor has been widely used because its imaging effect is better than that of the red, green and blue RGB sensor.
  • a filter can be added to the RYYB image sensor.
  • the transmittance of the filter at different incident angles is different, while reducing the color distortion of the RYB image, it will cause the overall image brightness to be uneven, and there will be color shading.
  • the embodiments of the present application provide an image processing method, device, terminal, and storage medium, which can solve the problems of the existing image processing technology based on the RYB format that cannot solve the color distortion and color shading at the same time.
  • an image processing method including:
  • the second brightness layer of the RYB image is adjusted by the target gain compensation array to generate a corrected image.
  • the method before the obtaining the second gain compensation array based on the RYB format associated with the first gain compensation array based on the preset compensation array correspondence relationship, the method further includes:
  • each of the training control groups includes at least one first training image based on the RYB format and a second training image based on the RGB format;
  • the first training compensation array is used as the output sample of the back propagation algorithm network
  • the second training compensation array is used as the input sample of the back propagation algorithm network, and the back propagation algorithm network is trained to obtain Compensation conversion network
  • Each reference gain compensation array based on the RGB format is input to the compensation conversion network, the target gain compensation array based on the RYB format corresponding to each reference gain compensation array is determined, and the corresponding relationship of the compensation array is generated.
  • the converting the red, yellow, and blue RYB image into a grid image based on the red, green, and blue RGB format includes:
  • the down-sampled image is converted into the grid image based on the RGB format through an RGB conversion algorithm.
  • the converting the down-sampled image into the grid image based on the RGB format by using an RGB conversion algorithm includes:
  • the down-sampled image is converted into the grid image based on the RGB format through an RGB conversion algorithm.
  • the adjusting the second brightness layer of the RYB image through the target gain compensation array to generate a corrected image includes:
  • the correction image is obtained by adjusting each brightness value in the second brightness layer through each compensation coefficient in the extended gain compensation array.
  • the generating the first brightness layer of the grid image and determining the reference gain compensation array for adjusting the first brightness layer includes:
  • the R/G layer and the B/G layer are recognized as the first brightness layer.
  • the generating the first brightness layer of the grid image and determining the reference gain compensation array for adjusting the first brightness layer includes:
  • the candidate gain compensation array with the largest smoothing coefficient is selected as the reference gain compensation array.
  • an image processing apparatus including:
  • the grid image conversion unit is used to convert the red, yellow and blue RYB image into a grid image based on the red, green and blue RGB format;
  • a reference gain compensation array determining unit configured to generate a first brightness layer of the grid image, and determine a reference gain compensation array for adjusting the first brightness layer;
  • a target gain compensation array obtaining unit configured to obtain a target gain compensation array based on the RYB format associated with the reference gain compensation array based on a preset corresponding relationship of the compensation array;
  • the image calibration unit is configured to adjust the second brightness layer of the RYB image through the target gain compensation array to generate a corrected image.
  • embodiments of the present application provide a terminal device, a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the The computer program implements the image processing method described in any one of the above-mentioned first aspects.
  • an embodiment of the present application provides a computer-readable storage medium that stores a computer program, and is characterized in that, when the computer program is executed by a processor, any of the above-mentioned aspects of the first aspect is implemented.
  • the embodiments of the present application provide a computer program product, which when the computer program product runs on a terminal device, causes the terminal device to execute the image processing method described in any one of the above-mentioned first aspects.
  • the RYB image is converted to a grid image in RGB format that is better for the discrete performance of different light source environments, and a first gain compensation array corresponding to the grid image is generated, and then a preset compensation array correspondence relationship is adopted , Generated a second gain compensation array based on the RYB format, and adjusted the RYB image through the second gain compensation array to generate a corrected image. While avoiding color distortion, the brightness layer of the RYB image was adjusted by the second gain compensation array. Make adjustments to eliminate color shadows and improve imaging effects.
  • FIG. 1 is a block diagram of a part of the structure of a mobile phone provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of the software structure of a mobile phone according to an embodiment of the present application.
  • Fig. 3 is an implementation flowchart of an image processing method provided by the first embodiment of the present application.
  • FIG. 4 is an imaging principle diagram based on RYYB image sensor provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an RYB image taken by a camera module equipped with an infrared filter according to an embodiment of the present application
  • FIG. 6 is a schematic diagram of a curved surface of a first brightness image provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the comparison of light source coordinates in RGB images and light source coordinates in RYB images for different light source types according to an embodiment of the present application;
  • FIG. 8 is a schematic diagram of brightness adjustment provided by an embodiment of the present application.
  • FIG. 9 is a specific implementation flowchart of an image processing method provided by the second embodiment of the present application.
  • FIG. 10 is a schematic diagram of training a BP network provided by an embodiment of the present application.
  • FIG. 11 is a specific implementation flowchart of an image processing method S301 provided by the third embodiment of the present application.
  • FIG. 12 is a schematic diagram of generating a down-sampled image provided by an embodiment of the present application.
  • FIG. 13 is a specific implementation flowchart of an image processing method S3013 provided by the fourth embodiment of the present application.
  • FIG. 14 is a specific implementation flowchart of an image processing method S304 provided by the fifth embodiment of the present application.
  • FIG. 16 is a schematic diagram of generating R/G layers and B/G layers provided by an embodiment of the present application.
  • FIG. 17 is a specific implementation flowchart of an image processing method S302 provided by the seventh embodiment of the present application.
  • FIG. 18 is a structural block diagram of an image processing device provided by an embodiment of the present application.
  • FIG. 19 is a schematic diagram of image correction provided by an embodiment of the present application.
  • FIG. 20 is a schematic diagram of a terminal device provided by another embodiment of the present application.
  • the term “if” can be construed as “when” or “once” or “in response to determination” or “in response to detecting “.
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • the image processing method provided in the embodiments of this application can be applied to mobile phones, smart cameras, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) devices, notebook computers, Ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (personal digital assistants, PDAs) and other terminal devices can also be applied to databases, servers, and service response systems based on terminal artificial intelligence.
  • AR augmented reality
  • VR virtual reality
  • UMPC Ultra-mobile personal computers
  • PDAs personal digital assistants
  • the embodiments of the present application do not impose any restrictions on the specific types of terminal devices.
  • the terminal device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a wireless local loop (Wireless Local Loop, WLL) station, Personal Digital Assistant (PDA) devices, handheld devices with wireless communication functions, computing devices or other processing devices connected to wireless modems, computers, laptops, handheld communication devices, handheld computing devices, and /Or other devices used to communicate on the wireless system and next-generation communication systems, for example, mobile terminals in 5G networks or mobile terminals in the future evolved Public Land Mobile Network (PLMN) network, etc.
  • STAION, ST station
  • WLAN Wireless Local Loop
  • PDA Personal Digital Assistant
  • the wearable device can also be a general term for the application of wearable technology to intelligently design daily wear and develop wearable devices, such as those with shooting functions. Glasses, gloves, watches, clothing and shoes, etc.
  • a wearable device is a portable device that is directly worn on the body or integrated into the user’s clothes or accessories. It is attached to the user and used to record the image of the user during the journey or collect environmental images according to the shooting instructions initiated by the user. Wait.
  • Wearable devices are not only a kind of hardware device, but also realize powerful functions through software support, data interaction, and cloud interaction.
  • wearable smart devices include full-featured, large-sized, complete or partial functions that can be implemented without relying on smart phones, such as smart watches or smart glasses, and only focus on a certain type of application function, and need to be used in conjunction with other devices such as smart phones. , Such as all kinds of smart watches and smart glasses for image collection.
  • Fig. 1 shows a block diagram of a part of the structure of a mobile phone provided in an embodiment of the present application.
  • the mobile phone includes: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a near field communication module 170, a processor 180, and a power supply 190.
  • RF radio frequency
  • FIG. 1 does not constitute a limitation on the mobile phone, and may include more or fewer components than those shown in the figure, or a combination of some components, or different component arrangements.
  • the RF circuit 110 can be used for receiving and sending signals during information transmission or communication. In particular, after receiving the downlink information of the base station, it is processed by the processor 180; in addition, the designed uplink data is sent to the base station.
  • the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like.
  • the RF circuit 110 can also communicate with the network and other devices through wireless communication.
  • the above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division) Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE)), Email, Short Messaging Service (SMS), etc., through RF circuits 110 receives images acquired by other terminals, processes the acquired images, and outputs corresponding corrected images.
  • GSM Global System of Mobile Communication
  • GPRS General Packet Radio Service
  • CDMA Code Division Multiple Access
  • WCDMA Wideband Code Division Multiple Access
  • LTE Long Term Evolution
  • Email Short Messaging Service
  • SMS Short Messaging Service
  • the memory 120 may be used to store software programs and modules.
  • the processor 180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 120, for example, stores the corresponding relationship of the pre-configured compensation array in the memory 120 Inside.
  • the memory 120 may mainly include a program storage area and a data storage area.
  • the program storage area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of mobile phones (such as audio data, phone book, etc.), etc.
  • the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
  • the input unit 130 can be used to receive inputted numeric or character information, and generate key signal input related to user settings and function control of the mobile phone 100.
  • the input unit 130 may include a touch panel 131 and other input devices 132.
  • the touch panel 131 also known as a touch screen, can collect user touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on the touch panel 131 or near the touch panel 131. Operation), and drive the corresponding connection device according to the preset program.
  • the display unit 140 may be used to display information input by the user or information provided to the user and various menus of the mobile phone, such as outputting an adjusted corrected image.
  • the display unit 140 may include a display panel 141.
  • the display panel 141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.
  • the touch panel 131 can cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it transmits it to the processor 180 to determine the type of the touch event, and then the processor 180 responds to the touch event. The type provides corresponding visual output on the display panel 141.
  • the touch panel 131 and the display panel 141 are used as two independent components to realize the input and input functions of the mobile phone, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated. Realize the input and output functions of the mobile phone.
  • the mobile phone 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor.
  • the ambient light sensor can adjust the brightness of the display panel 141 according to the brightness of the ambient light.
  • the proximity sensor can close the display panel 141 and/or when the mobile phone is moved to the ear. Or backlight.
  • the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary.
  • the mobile phone 100 may also include a camera 160.
  • the position of the camera on the mobile phone 100 may be front-mounted or rear-mounted, which is not limited in the embodiment of the present application.
  • the mobile phone 100 may include a single camera, a dual camera, or a triple camera, etc., which is not limited in the embodiment of the present application.
  • the mobile phone 100 may include three cameras, of which one is a main camera, one is a wide-angle camera, and one is a telephoto camera.
  • the multiple cameras may be all front-mounted, or all rear-mounted, or partly front-mounted and some rear-mounted, which is not limited in the embodiment of the present application.
  • the terminal device can receive images to be processed sent by other devices through the near field communication module 170.
  • the near field communication module 170 is integrated with a Bluetooth communication module, establishes a communication connection with the smart camera through the Bluetooth communication module, and receives the waiting information from the smart camera. Process the image.
  • FIG. 1 shows the near field communication module 170, it can be understood that it is not a necessary component of the mobile phone 100, and can be omitted as needed without changing the essence of the application.
  • the processor 180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes by running or executing software programs and/or modules stored in the memory 120 and calling data stored in the memory 120. Various functions and processing data of the mobile phone can be used to monitor the mobile phone as a whole.
  • the processor 180 may include one or more processing units; preferably, the processor 180 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application programs, etc. , The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 180.
  • the mobile phone 100 also includes a power source 190 (such as a battery) for supplying power to various components.
  • a power source 190 such as a battery
  • the power source can be logically connected to the processor 180 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.
  • the mobile phone 100 also includes an audio circuit and a speaker, and the microphone can provide an audio interface between the user and the mobile phone.
  • the audio circuit can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is converted into audio after being received by the audio circuit
  • the data is processed by the audio data output processor 180, and then sent to, for example, another mobile phone via the RF circuit 110, or the audio data is output to the memory 120 for further processing.
  • the user can collect the user's voice signal through an audio circuit, control the camera 160 to perform an image collection operation based on the voice signal, and process the collected image to obtain a corrected image.
  • FIG. 2 is a schematic diagram of the software structure of the mobile phone 100 according to an embodiment of the present application.
  • the Android system is divided into four layers, namely the application layer, the application framework layer (framework, FWK), the system layer, and the hardware abstraction layer. Communication between the layers through software interface.
  • the application layer can be a series of application packages, which can include applications such as short message, calendar, camera, video, navigation, gallery, and call.
  • the speech recognition algorithm can be embedded in the application program, the image processing flow is started through the relevant controls in the application program, and the obtained RYB image is processed to obtain the corrected image after color shading is eliminated.
  • the application framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for applications in the application layer.
  • the application framework layer may include some predefined functions, such as functions for receiving events sent by the application framework layer.
  • the application framework layer can include a window manager, a resource manager, and a notification manager.
  • the window manager is used to manage window programs.
  • the window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, take a screenshot, etc.
  • the content provider is used to store and retrieve data and make these data accessible to applications.
  • the data may include videos, images, audios, phone calls made and received, browsing history and bookmarks, phone book, etc.
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and it can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, and so on.
  • the notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or a scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, text messages are prompted in the status bar, prompt sounds, electronic devices vibrate, and indicator lights flash.
  • the application framework layer can also include:
  • a view system which includes visual controls, such as controls that display text, controls that display pictures, and so on.
  • the view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface that includes a short message notification icon may include a view that displays text and a view that displays pictures.
  • the phone manager is used to provide the communication function of the mobile phone 100. For example, the management of the call status (including connecting, hanging up, etc.).
  • the system layer can include multiple functional modules. For example: sensor service module, physical state recognition module, 3D graphics processing library (for example: OpenGL ES), etc.
  • the sensor service module is used to monitor the sensor data uploaded by various sensors at the hardware layer and determine the physical state of the mobile phone 100;
  • Physical state recognition module used to analyze and recognize user gestures, faces, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.
  • the system layer can also include:
  • the surface manager is used to manage the display subsystem and provides a combination of 2D and 3D layers for multiple applications.
  • the media library supports a variety of commonly used static image files, video format playback and recording, and audio.
  • the media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the hardware abstraction layer is the layer between hardware and software.
  • the hardware abstraction layer may include display drivers, camera drivers, sensor drivers, microphone drivers, etc., used to drive related hardware at the hardware layer, such as display screens, cameras, sensors, and microphones.
  • the camera module is driven by the camera head, and the camera module is specifically a camera module based on the RYYB image sensor.
  • the RYYB image sensor collects the light information corresponding to the trigger time of the shooting instruction to generate the RYB image.
  • image processing method provided in the embodiment of the present application can be executed in any of the above-mentioned levels, which is not limited herein.
  • the execution subject of the process is a device installed with an image processing program.
  • the device of the image processing program may specifically be a terminal device.
  • the terminal device may be a smart phone, smart camera, tablet computer, etc. used by the user to process the acquired RYB image and generate an adjusted Correct the image to eliminate the color shading introduced in the process of collecting RYB images and improve the effectiveness.
  • FIG. 3 shows an implementation flowchart of the image processing method provided by the first embodiment of the present application, which is described in detail as follows:
  • the red, yellow, and blue RYB image is converted into a grid image based on the red, green, and blue RGB format.
  • the terminal device can acquire RYB images through a built-in camera module based on the RYB imaging principle.
  • the user can activate the camera module by launching a specific application in the terminal device, such as camera applications, real-time video Call applications, etc.; the user can also click some controls in the current application to activate the camera module, for example, click the send camera control in a social application, and send the collected RYB image as interactive information to the communication peer.
  • the terminal device The environment image at the moment of the user's click operation will be collected through the camera module as the above-mentioned RYB image that needs to be adjusted.
  • the terminal device can also collect the RYB image to be adjusted through an external camera module.
  • the terminal device can establish a communication connection with the external camera module through a wireless communication module or a serial interface.
  • the user can click on the camera Open the door on the module, control the camera module to collect images, and generate the corresponding RYB image through the built-in RYB image sensor, and transmit the RYB image to the terminal device through the communication connection established above.
  • the terminal device receives the RYB image fed back by the camera module , You can perform subsequent image processing procedures.
  • the terminal device may also acquire the RYB image through a communication peer sending method.
  • the terminal device can establish a communication connection with the communication peer through the communication module, and receive the RYB image sent by the communication peer through the communication connection.
  • the method for the communication peer to obtain the RYB image can be referred to the above process, which will not be repeated here.
  • the terminal device After receiving the RYB image fed back by the communication peer, the terminal device can perform color shadow removal processing on the target RYB image.
  • the terminal device may be a cloud server, and each communication peer may be installed with a client program corresponding to the cloud server, or locally generated on the communication peer through the API interface corresponding to the cloud server
  • the locally acquired RYB image is sent to the cloud server through the client program or API interface, and the cloud server feeds back the processed corrected image to the communication peer.
  • the communication peer After the communication peer receives the processed RYB image, it will be After the above-mentioned corrected image, the above-mentioned corrected image can be output on the display module of the communication opposite end, and the corrected image can be stored after receiving the shooting completion instruction.
  • the camera module used when the terminal device collects the RYB image is generated based on the RYYB image sensor.
  • the RYYB image sensor contains four color channels, one for collecting red light.
  • the terminal device can convert the RYYB image to the RYB image, and the process can be realized by calculating the average value of the corresponding pixel value of each pixel in the RYYB image in two different Y channels, as the pixel of the Y channel of the merged RYB image Value, the pixel value of the red channel and the blue channel remain unchanged, so as to realize the conversion of RYYB image to RYB image.
  • FIG. 4 shows an imaging principle diagram based on an RYYB image sensor provided by an embodiment of the present application.
  • the RYYB image sensor includes four-channel sensors, one R-channel sensor, one B-channel sensor, and two Y-channel sensors.
  • the photosensitive element in the RYYB image sensor is based on a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensor imaging, and the CMOS sensor is more sensitive to infrared radiation, so it is prone to color distortion.
  • CMOS complementary Metal Oxide Semiconductor
  • an infrared filter can be installed in the incident light path.
  • the different incident angles between different incident light and the infrared filter will affect the transmittance of light at different incident angles, resulting in the red light transmittance in the center area of the image.
  • FIG. 5 shows a schematic diagram of an RYB image captured by a camera module with an infrared filter provided in an embodiment of the present application.
  • the pixel distribution will be different in different areas of the image.
  • the corresponding brightness layer generated from the captured RYB image it can be determined that the brightness value of the central area of the brightness layer is higher, while the brightness value of the boundary area is lower, and color shadows appear, which reduces the imaging effect.
  • the above-mentioned color shading is particularly obvious when shooting scenes with a large number of areas of the same color, such as shooting indoor walls, outdoor bridge decks, incandescent lamps, purple flower fields, or images with the above scenes as the background. Therefore, in order to eliminate the above-mentioned distortion caused by the infrared filter in the captured RYB image, adjustments can be made through S301 to S304 to obtain an adjusted corrected image.
  • the pixel value of each pixel in the R channel and the pixel value of each pixel in the Y channel can be used. Determine the corresponding pixel value of each pixel in the G channel, and realize the conversion of the RYB image into the corresponding RGB image.
  • the terminal device may be configured with a conversion algorithm corresponding to the RYB image sensor.
  • the terminal device can obtain the device model of the RYB image sensor, download the corresponding conversion algorithm based on the aforementioned device model, and convert the RYB image into an RGB image based on the obtained conversion algorithm.
  • the adjustment coefficient has a strong correlation with the location area.
  • the converted RGB image can be down-sampled, the RGB image can be divided into multiple grid areas, and the characteristic pixel points corresponding to each grid area can be obtained to generate the aforementioned RGB format-based Grid image.
  • the image size of a certain RYB image is 3100*2100, that is, each row contains 3100 pixels and each column contains 2100 pixels
  • the image size of the converted RGB image is consistent with the image size of the RYB image
  • it can be down-sampled to a grid of 31*21, that is, each grid contains 100*100 pixels, and the above-mentioned pixels are represented by a characteristic pixel value, thereby down-sampling It is a 31*21 grid image.
  • a first brightness layer of the grid image is generated, and a reference gain compensation array for adjusting the first brightness layer is determined.
  • the terminal device may generate the first brightness layer corresponding to the grid image.
  • the first brightness layer is specifically used to represent the brightness value of each grid area in the grid image, and the brightness value is independent of the chromaticity of the characteristic pixel value corresponding to the grid area.
  • the above-mentioned first brightness layer may be a V layer based on hue, saturation, and brightness (Hue, Saturation, Value, HSV) format, or an illuminance (L) layer in Lab color format.
  • the R/G layer and the B/G layer after normalization of the grid image can also be normalized.
  • the expression mode of the first brightness layer is not limited here.
  • the method of generating the first brightness layer can be: because the grid image is an image generated based on the RGB format, that is, the characteristic pixel value corresponding to each grid area in the grid image contains three RGB The pixel value of the channel. Based on this, the terminal device can map the pixel values of the three RGB channels of each area grid in the grid image to the HSV channel through the conversion algorithm between RGB and HSV, which is represented by the three HSV channels The characteristic pixel value of each grid area is combined according to the pixel value of each grid area in the V channel to generate the above-mentioned first brightness layer.
  • the method of generating the first brightness layer can be: because the grid image is an image generated based on the RGB format, that is, the characteristic pixel value corresponding to each grid area in the grid image contains three RGB
  • the pixel value of the channel based on this, the terminal device can pass the conversion algorithm between RGB and Lab (among which, the Lab format image contains L channel, a channel and b channel; L channel refers to the pixel point illuminance Channel; a channel refers to the channel used to represent red to green; b channel refers to the channel used to represent blue to yellow), grid each area in the grid image in the pixels corresponding to the three channels of RGB
  • the value is mapped to the Lab channel, the characteristic pixel value of each grid area is represented by the three Lab channels, and the pixel value of each grid area in the L channel is combined to generate the above-mentioned first brightness layer.
  • the terminal device can adjust each brightness value in the first brightness layer through a preset reference gain compensation array, so that each brightness value tends to be the same, so as to eliminate color shadows.
  • FIG. 6 shows a schematic diagram of a curved surface of a first brightness image provided by an embodiment of the present application.
  • (a) in Figure 6 is the first brightness image before adjustment by the reference gain compensation array. Due to the presence of color shading, the central area of the image has a higher brightness value, while the surrounding brightness values Lower. In the absence of color shadows, the three-dimensional curved surface corresponding to the first brightness layer approaches a plane. Therefore, the terminal device can adjust the brightness values in the first brightness image through the reference gain compensation array, so that The three-dimensional curved surface of the adjusted first brightness layer is a plane, and (b) in FIG. 6 is the first brightness image adjusted by the reference gain compensation array.
  • the terminal device may be configured with a plurality of preset candidate gain compensation arrays, and adjust the first brightness layer through each candidate gain array, and identify each of the adjusted first brightness layers.
  • the standard deviation between the brightness values of the grid area is selected, and a candidate gain compensation array with the smallest value of the standard deviation is selected as the above-mentioned reference gain compensation arrangement. Since the purpose of adjustment is to make the three-dimensional surface corresponding to the first brightness layer approach a plane, the standard deviation between the brightness values of each grid area after the adjustment should be small, ideally the standard deviation is 0 Therefore, the terminal device can calculate the standard deviation between the brightness values in the first brightness layer adjusted by each candidate gain compensation array, and select the one with the smallest standard deviation as the reference gain compensation array.
  • the terminal device may import the first brightness layer into a preset gain array selection network, and output a reference compensation gain array matching the first brightness layer.
  • the gain array selection network can be built based on shallow neural networks, or based on models such as vector machines, random forests, and reinforcement learning.
  • the terminal device can collect training images under different light source types and light intensity through the RGB image sensor.
  • the light source types include but are not limited to: A, D50, D65, D75, H, TL83, TL84, U30, and the light intensity includes but Not limited to: 1000lux, 700lux, 300lux, 100lux, 20lux.
  • M*N shooting scenes can be obtained, M is the number of light source types, and N is the number of light intensity.
  • the number of reference gain compensation arrays obtained above matches the number of layers, for example .
  • the first brightness layer of the grid image contains the R/G layer and the B/G layer, you can configure the corresponding reference gain compensation array for the R/G layer and the corresponding reference for the B/G layer
  • the number of target compensation arrays obtained subsequently may be multiple, and the R/Y layer and the B/Y layer are adjusted respectively through the obtained target gain compensation array.
  • a target gain compensation array based on the RYB format associated with the reference gain compensation array is obtained based on the preset corresponding relationship of the compensation array.
  • the above-mentioned reference gain compensation array is based on the RGB format gain compensation array, that is, the three channels in the RGB image can be adjusted, and the image obtained by the original shooting is obtained based on the RYB image sensor. Therefore, it is necessary to map the reference gain compensation array to the target gain compensation array based on the RYB format in order to adjust the brightness values of the three RYB channels in the original RYB image.
  • the reason why the grid image in RGB format is used to obtain the corresponding reference gain compensation array is that it is equivalent to the RYB image, and the RGB image has a greater degree of dispersion on the brightness layer for different light sources, as shown in Figure 7
  • An embodiment of the present application provides a schematic diagram of the comparison of the light source coordinates of different light source types in the RGB image and the light source coordinates in the RYB image.
  • the above-mentioned light source coordinates specifically refer to the brightness value in the central area of the brightness layer of the RGB image, and for the RGB image, the brightness layer can be normalized by the R/G layer and the normalized The latter B/G layer is represented.
  • the brightness value of the center coordinate point of the above two layers (for the R/G layer, that is, the R channel pixel value and the G channel normalized by the center coordinate point
  • the first ratio of pixel values is the R index in the figure; for the B/G layer, it is the second ratio of the B channel pixel value and the G channel pixel value after the center coordinate point is normalized, which is the second ratio of the pixel value in the G channel.
  • a coordinate point can be determined in the preset coordinate system, that is, the light source coordinates of the RGB image corresponding to the light source type), relatively, for the RYB image, you can use the R/Y layer And the B/Y layer is used as the brightness layer of the RYB image, thereby determining the R index and B index of the RYB image.
  • the light source coordinates of the 8 light sources, A, D50, D65, D75, H, TL83, TL84, and U30 are used as examples for illustration.
  • Figure 7(a) is the light source coordinate distribution diagram of the RGB image
  • Figure 7(b) is the light source coordinate distribution diagram of the RYB image.
  • the coordinate distance between the light source coordinates of different light sources is relatively large.
  • the degree of dispersion is higher, that is, it is easier to distinguish different light source types.
  • Different light source types are more different from each other in the brightness layer of the RGB format, so that the subsequent recognition accuracy is higher; while in the RYB image, the difference
  • the distance between the light source coordinates of the light source is small, and the degree of dispersion is low (the R index values of the above five light sources of H, A, D65, D75, D50 are almost the same), that is, it is more difficult to distinguish different light source types.
  • the brightness layers in RYB format have relatively small differences between each other, so the subsequent recognition accuracy is low.
  • the RYB image can be first converted to an RGB image with large brightness difference, and the corresponding relationship between the reference gain compensation array and the compensation array of different format images can be obtained to obtain the RYB-based image.
  • Format of the target gain compensation array can improve the accuracy of the target gain compensation array acquisition, thereby improving the effect of image correction.
  • the terminal device pre-stores the corresponding relationship of the compensation array, and the corresponding relationship of the compensation array specifically records the corresponding relationship between the reference gain compensation array in each RGB format and the target gain compensation array in the RYB format. After obtaining the reference gain compensation array based on the RGB format, the terminal device may search in the corresponding relationship of the compensation array to obtain the target gain compensation array corresponding to the reference gain compensation array.
  • the number of reference gain compensation arrays is fixed, and they are used to adjust the RGB images shot under different lighting scenarios.
  • the number of target gain compensation arrays can also be fixed, respectively.
  • the terminal device can establish a compensation array correspondence relationship between the reference gain compensation array and the target gain compensation array according to the association relationship between the illumination scenes. For example, for lighting scene A, the reference gain compensation array A in the RGB format, and the target gain compensation array B in the RYB format, the correlation between the reference gain compensation array A and the target gain compensation array B can be established, and Mark the corresponding lighting scene.
  • the terminal device can identify the illumination type of the RGB image according to the reference gain compensation array, and search for a matching target gain compensation array in the corresponding relationship of the compensation array based on the aforementioned illumination type.
  • the terminal device can construct the above-mentioned compensation array correspondence relationship through a machine learning algorithm.
  • the terminal device can create multiple training samples, and each training sample corresponds to a sample image and belongs to The sample image is based on the first training image in the RGB format and the second training image in the RYB format, and the corresponding first training compensation array and the second training compensation array are generated for the two training images respectively, and the pre-processing is performed according to the multiple training samples
  • the set learning algorithm is trained to determine the target gain compensation array corresponding to each different reference gain compensation array.
  • the above-mentioned learning algorithm can be a shallow neural network, or a model based on vector machines, random forests, and reinforcement learning.
  • the second brightness layer of the RYB image is adjusted through the target gain compensation array to generate a corrected image.
  • the terminal device can obtain the second brightness layer of the RYB image, where the second brightness layer is specifically a layer that has nothing to do with chroma and is only related to brightness. .
  • the second brightness layer is specifically a layer that has nothing to do with chroma and is only related to brightness.
  • the above-mentioned second brightness layer may be a normalized R/Y layer and a normalized B/Y layer.
  • the method of generating the normalized R/Y layer and the normalized B/Y layer may be: the terminal device obtains the pixel value of the center coordinate point of the RYB image, and based on the pixel value of the center pixel point Value as the reference value, normalize each other pixel, and calculate the first ratio between the normalized R channel pixel value and the Y channel pixel value of each pixel, and all the first ratios form the above In the same way, the second ratio between the normalized B-channel pixel value and the Y-channel pixel value of each pixel can be calculated, and all the second ratios form the above-mentioned B/Y layer .
  • the terminal device can adjust the pixel value of each pixel in the RYB image through the target gain compensation array, thereby being able to generate a corrected image with color shading eliminated.
  • the manner of generating the above-mentioned corrected image may be: since the reference compensation array is generated based on a grid image, that is, the number of rows and columns of the reference compensation array is the same as the number of grids corresponding to the grid image, The number of grids is smaller than the number of pixels in the RYB image. Therefore, the number of rows and columns of the target gain compensation array converted based on the reference gain compensation array is also the same as that of the grid image.
  • the terminal device can magnify the target grid image in equal proportions, determine the pixel points corresponding to each grid area in the RYB image, and compare the brightness values of all pixels contained in the grid area , Through the gain compensation coefficient of the grid area to adjust.
  • the RGB image is converted into a grid image by downsampling, which can reduce the amount of calculation to determine the reference gain compensation coefficient in the subsequent process according to the strong relationship between the gain compensation coefficient and the position of the pixel in the image, thereby improving the overall The efficiency of image processing.
  • FIG. 8 shows a schematic diagram of brightness adjustment provided by an embodiment of the present application, in which (a) of FIG. 8 is a target gain compensation array; (b) of FIG. 8 is based on target integral gain compensation
  • the target gain compensation array is a 3*3 grid matrix, each grid area corresponds to an adjustment coefficient, and the RYB image contains multiple pixels, and the terminal device can compensate the target gain of the array
  • the size is adjusted to be consistent with the RYB image, and each grid area can be associated with a corresponding pixel on the RYB image.
  • the brightness value of the associated pixel can be adjusted according to the adjustment coefficient corresponding to the grid area, and based on The adjusted second brightness layer generates a corrected image to eliminate the color shadows contained in the image.
  • the terminal device can calculate the coordinates of the central pixel of the RYB image to ensure that the central pixel falls on a preset channel, and the coordinates of the central pixel of the image are centered. Divide the image to obtain the divided target area, calculate the average value of each channel in the target area, and use it as the gain adjustment target of the corresponding channel, and use the target value and the position of the pixel to adaptively perform pixel compensation.
  • the above method mainly uses the average value of each channel in the target area as the gain adjustment target. It is not suitable for images with rich colors, and for images with more pixels, it is necessary to calculate the gain compensation value of each pixel, which will increase the amount of calculation and decrease. The calculation speed.
  • the terminal device can obtain the brightness value from each photosensitive point of the sensor array and set it to compensate for each photosensitive point associated with the current setting of the camera based on the light field imaging principle.
  • a set of weighted values for the dots the brightness value of each photosensitive dot is changed through a set of weighted values.
  • the above-mentioned camera modules are based on the principle of light field imaging. Most of the existing camera modules are based on the principle of RYB image sensor for image acquisition, thereby reducing the scope of application, and only use the brightness value of the photosensitive point to perform shadow correction. Color shading correction effectively compensates and reduces the effect of correction.
  • the embodiment of the application reduces the amount of calculation and increases the calculation speed while ensuring the adjustment effect.
  • the RYB image is down-sampled and converted to generate a grid image based on the RGB format, so that there is no need to calculate each time.
  • the gain compensation coefficient corresponding to each pixel thereby reducing the amount of calculation to generate the target gain compensation array, and adopting the advantages of better dispersion under different light sources in the RGB format, which improves the accuracy of the target gain compensation array. Improved the calibration effect.
  • the image processing method converts the RYB image to a grid image in RGB format that is better for the discrete performance of different light source environments, and generates the first grid image corresponding to the grid image.
  • the gain compensation array then generates a second gain compensation array based on the RYB format through the preset corresponding relationship of the compensation array, and adjusts the RYB image through the second gain compensation array to generate a corrected image. While avoiding color distortion, The brightness layer of the RYB image is adjusted through the second gain compensation array, which eliminates color shadows and improves the imaging effect.
  • FIG. 9 shows a specific implementation flowchart of an image processing method provided by the second embodiment of the present application.
  • the RYB-based data associated with the first gain compensation array is obtained.
  • the second gain compensation array of the format it also includes: S901 ⁇ S904, the details are as follows:
  • the method further includes:
  • each of the training control groups includes at least one first training image based on the RYB format and a second training image based on the RGB format.
  • the terminal device can train and learn the preset back propagation algorithm through a large amount of sample data, so as to establish the conversion relationship between the reference gain compensation array in RGB format and the target gain compensation array in RYB format, thereby The conversion of compensation array can be realized. Therefore, in S901, a large number of training control groups can be collected, where the above training control group is included in environments covering multiple different brightness levels, so as to improve the accuracy and scope of application of subsequent correspondences.
  • Table 1 is a schematic diagram of a training control group under different brightness environments provided by an embodiment of the application.
  • the above-mentioned brightness environment is determined by two parameters of light source type and illuminance intensity.
  • Table 1 shows 8 different light source types, namely A, D50, D65, D75, H, TL83, TL84, U30, and illuminance
  • There are five kinds of intensity: 1000lux, 700lux, 300lux, 100lux, and 20lux. Therefore, the number of brightness environments constructed is 8*5 40, and the corresponding number of training control groups are configured for the 40 different brightness environments mentioned above.
  • the number of training control groups for each brightness environment is 10, so that the total number of training samples is 400 groups.
  • each training control group includes at least one first training image based on the RYB format and a second training image based on the RGB format.
  • the first training image and the second training image belonging to the same training control group are both Images obtained by shooting the same object at the same angle under the same brightness environment, where the first training image can be the RYB image collected by the RYB image sensor, or it can be obtained after format conversion based on the existing RGB image RYB image; similarly, the second training image may be an RGB image collected by an RGB image sensor, or an RGB image obtained after format conversion based on an existing RYB image.
  • a first training compensation array of the first training image is generated, and a second training compensation array of the second training image is generated.
  • the terminal device may configure the first training compensation array for the first training image.
  • the method of generating the first training compensation array may be manually configured by the user or calculating the compensation coefficient of each pixel through a preset compensation algorithm, so as to generate the first training compensation array corresponding to the first training image.
  • the method of configuring the first training compensation array may be specifically as follows: the terminal device generates the first training brightness layer of the first training compensation array, and the terminal device may be configured with multiple preset candidate gain compensations Array, adjust the first training brightness layer through each candidate gain array, identify the standard deviation between the brightness values of each grid area in the adjusted first training brightness layer, and select the one with the smallest standard deviation A candidate gain compensation array is used as the first training compensation array mentioned above.
  • the manner of generating the second training compensation array for the second training image is the same as the manner of generating the first training compensation array, which can be referred to the above description, which will not be repeated here.
  • the first training compensation array is used as the output sample of the back propagation algorithm network
  • the second training compensation array is used as the input sample of the back propagation algorithm network. Perform training to get the compensation conversion network.
  • the above-mentioned compensation conversion network may be constructed based on a back propagation algorithm (BP) network.
  • BP back propagation algorithm
  • the terminal device can train and learn the back propagation algorithm network through multiple training control groups.
  • FIG. 10 shows a schematic diagram of training a BP network provided by an embodiment of the present application.
  • the BP network contains five levels.
  • the first and fifth levels contain 1271 network nodes
  • the second and fourth levels contain 2000 network nodes
  • the third level contains 3000 networks. Nodes, different network nodes can be configured with corresponding learning parameters.
  • the BP network can be adapted to establish the corresponding relationship between the reference gain compensation array and the target gain compensation array, where the BP network
  • the input parameter is the second training compensation array based on the RGB format
  • the output parameter is the first training compensation array based on the RYB format
  • the first training compensation array and the second training compensation array are both Belonging to the same training control group, by calculating the corresponding loss rate of the BP network, determine whether the BP network algorithm has been adjusted. If the loss rate of the BP network is less than the preset loss threshold, it is recognized that the adjustment of the BP network has been completed, and the adjusted BP network is recognized as the compensation conversion network described above.
  • each reference gain compensation array based on the RGB format is input to the compensation conversion network, the target gain compensation array based on the RYB format corresponding to each reference gain compensation array is determined, and the corresponding relationship of the compensation array is generated.
  • the corresponding relationship between the reference gain compensation array and the target gain compensation array can be established through the compensation conversion network.
  • the number of reference gain compensation arrays based on the RGB format and the number of target gain compensation arrays based on the RYB format are fixed. Therefore, each pre-configured reference gain compensation array based on the RGB format can be imported into the above In the compensation conversion network, determine and select the associated target gain compensation array from the existing RYB format target gain compensation array, and associate the reference gain compensation array with the associated target gain compensation array, and generate according to all the established association relationships Correspondence of the above compensation array.
  • a compensation conversion network is generated; the generated compensation conversion network is used to determine the target associated with each reference gain compensation array
  • the gain compensation array can improve the accuracy of the subsequent establishment of the corresponding relationship of the compensation array, thereby improving the accuracy of brightness adjustment.
  • FIG. 11 shows a specific implementation flowchart of an image processing method S301 provided by the third embodiment of the present application.
  • S301 in an image processing method provided in this embodiment includes: S3011 to S3013, which are detailed as follows:
  • the conversion of the red, yellow, and blue RYB image into a grid image based on the red, green, and blue RGB format includes:
  • the RYB image is divided into a plurality of grid areas, and the characteristic pixel value of each grid area is determined according to the pixel points in each grid area.
  • the terminal device after the terminal device divides the RYB image into multiple grid areas, it can identify the pixel values of the pixels contained in each grid area, and based on all the above pixel values, determine the features corresponding to the grid area Pixel value, so that multiple pixels can be represented by a grid area, which achieves the purpose of downsampling and reduces the number of pixels.
  • the method for determining the characteristic pixel value may be: the terminal device uses the average value of the pixel values of the pixels in the grid area as the characteristic pixel value of the grid area; the terminal device may also use the grid area to determine the characteristic pixel value. The pixel value of the center coordinate point of the grid area is used as the characteristic pixel value of the grid area.
  • the terminal device can determine the weighted weight of each pixel according to the distance between each pixel in the grid area and the center coordinate point corresponding to the grid area, and calculate the weight of each pixel. For the weighted average of pixel values, the above-mentioned weighted average is used as the characteristic pixel value.
  • a down-sampled image of the RYB image is generated based on the characteristic pixel value of each of the grid regions.
  • the terminal device combines multiple grid areas according to the characteristic pixel value of each grid area and the location area where the grid area is located to generate a down-sampled image corresponding to the RYB image.
  • Fig. 12 shows a schematic diagram of generating a down-sampled image provided by an embodiment of the present application.
  • Fig. 12(a) is the RYB image before down-sampling
  • Fig. 12(b) is the down-sampled image of the RYB image.
  • the terminal device can downsample an image containing multiple pixels into a grid image, thereby reducing the image size and increasing the processing rate.
  • the down-sampled image is converted into the grid image based on the RGB format through an RGB conversion algorithm.
  • the terminal device since the down-sampled image is generated based on the RYB format, the three channels included in the down-sampled image are the R channel, the Y channel, and the B channel, respectively.
  • the terminal device needs to convert the down-sampled image in the RYB format into a grid image based on the RGB format. Therefore, the terminal device can generate a grid image in RGB format corresponding to the down-sampled image through a preset RGB conversion algorithm.
  • the foregoing RGB conversion algorithm may specifically be: the terminal device may use the following algorithm to calculate the pixel value of each pixel in the RYB image converted to the G channel in the RGB image:
  • G (2Y-R), where G represents the pixel value converted to the G channel; Y represents the pixel value of the Y channel before the conversion; R represents the pixel value of the R channel before the conversion.
  • the down-sampled image is obtained by first down-sampling the RYB image, and the RGB format conversion is performed to generate a grid image, which can reduce the calculation amount of the format conversion, thereby improving the calculation efficiency.
  • FIG. 13 shows a specific implementation flowchart of an image processing method S3013 provided by the fourth embodiment of the present application.
  • S3013 in an image processing method provided in this embodiment includes: S1301 to S1303, which are detailed as follows:
  • converting the down-sampled image into the grid image based on the RGB format includes:
  • the terminal device can generate the brightness layer of the RYB image according to the pixel value of each pixel in the RYB image, and identify the light source when the RYB image is taken according to the brightness value of each pixel in the brightness layer Types of.
  • the terminal device may match different candidate light sources according to the brightness value of the center coordinate point of the RYB image, and determine the light source type of the RYB image based on the matching result.
  • the light source types include but are not limited to: A, D50, D65, D75, H, TL83, TL84, U30 and other eight light source types.
  • the above-mentioned light source type may also include light intensity, that is, the above-mentioned light source type may be represented by (A, 200 lux).
  • the terminal device can configure corresponding RGB algorithms for different light source types. After the terminal device determines the corresponding light source type when shooting RYB images, it can select the RGB corresponding to the light source type from the RGB conversion algorithm library. Conversion algorithm. Wherein, the aforementioned RGB conversion algorithm is specifically a conversion algorithm from RYB format to RGB format.
  • the aforementioned RGB conversion algorithm may be a conversion matrix.
  • Table 2 shows the index table of the RGB conversion algorithm provided by an embodiment of the present application. Refer to Table 2.
  • the index table shows the conversion matrices corresponding to eight different light source types, namely A, D50, D65, D75, H, TL83, CWF, and U30. Since the RYB image needs to be converted into an RGB image, the above two images are both images containing three channels, so the corresponding conversion matrix is also a 3*3 matrix. For example, for the light source type of U30, the corresponding conversion matrix is:
  • the down-sampled image is converted into the grid image based on the RGB format through an RGB conversion algorithm.
  • the terminal device can convert the down-sampled image into a grid image based on the RGB format through the above-mentioned RGB conversion algorithm.
  • the above RGB algorithm is specifically a conversion matrix
  • the image matrix corresponding to the down-sampled image can be multiplied by the conversion matrix to obtain the image matrix based on the grid image in the RGB format.
  • the identified light source type is U30
  • the selected RGB conversion matrix is as described above
  • the image array corresponding to the down-sampled image in the RYB format is multiplied with the conversion matrix corresponding to U30 to obtain the RGB format-based Grid image
  • the calculation method is as follows:
  • [R, Y, B] is the image array corresponding to the down-sampled image
  • [R, G, B] is the image array corresponding to the grid image.
  • the RGB conversion algorithm corresponding to the light source type is selected by identifying the type of light source when the RYB image is taken, thereby converting the down-sampled image into a grid image, which improves the accuracy of the grid image conversion and improves The correction effect of subsequent operations.
  • FIG. 14 shows a specific implementation flowchart of an image processing method S304 provided by the fifth embodiment of the present application.
  • S304 in an image processing method provided in this embodiment includes: S3041 to S3043, which are detailed as follows:
  • the adjusting the second brightness layer of the RYB image through the target gain compensation array to generate a corrected image includes:
  • the target gain compensation array is generated using a grid image based on the down-sampled RGB format, the array size of the target gain compensation array is inconsistent with the original RYB size, so the target gain needs to be compensated
  • the array is expanded to the size of the original image, and based on this, the terminal device can obtain the image size of the RYB image.
  • the image size can be determined by the number of pixels, and can also be expressed in terms of image resolution and image length.
  • the target gain compensation array is expanded to an expanded gain compensation array having the same size as the image by a bilinear interpolation method.
  • the terminal device can determine the scaling ratio, and adjust the bilinear interpolation algorithm based on the scaling ratio, and pass the adjusted bilinear interpolation algorithm And the gain value of each element of the target gain compensation array to generate an expanded gain compensation array that matches the image size.
  • the bilinear interpolation method may specifically be:
  • f(Q ij ) is the gain compensation coefficient corresponding to the coordinates (x i , y j ) in the target gain compensation array ; x 1 and x 2 are the two closest to any coordinates (x, y) in the extended gain compensation array Abscissa; y 1 and y 2 are the two ordinates closest to any coordinate (x, y) in the expansion gain compensation array, f(x, y) is the gain compensation of any coordinate (x, y) in the expansion gain compensation array Numerical value.
  • each brightness value in the second brightness layer is adjusted by each compensation coefficient in the extended gain compensation array to obtain the corrected image.
  • the brightness values in the second brightness layer of the RYB image can be adjusted by each gain compensation value in the expanded gain compensation array. , And generate a corrected image according to the adjusted second brightness layer.
  • the method for generating the above-mentioned corrected image may specifically be: the second brightness layer corresponding to the RYB image specifically includes: a normalized R/Y layer and a normalized B/Y Therefore, the expanded gain compensation array obtained above also includes a first gain compensation array for adjusting the R/Y layer and a second gain compensation array for adjusting the B/Y layer. Adjust each pixel value in the R/Y layer, and perform the inverse normalization operation on the adjusted R/Y layer to obtain the adjusted R layer and the first Y layer. Similarly, pass The above method can obtain the adjusted B layer and the second Y layer, calculate the average value between the two Y layers, and merge the adjusted Y layer to combine the adjusted R and Y layers And layer B generates the above-mentioned corrected image.
  • FIG. 15 shows a specific implementation flowchart of an image processing method S302 provided by the sixth embodiment of the present application.
  • an image processing method S303 provided in this embodiment includes: S1501 to S1505, which are detailed as follows:
  • the generating the first brightness layer of the grid image and determining the reference gain compensation array for adjusting the first brightness layer includes:
  • the terminal device may use the normalized R/G layer and the normalized B/G layer as the first brightness layer of the grid image.
  • the reference value needs to be determined before the normalized R/G layer and the normalized B/G layer are generated. Therefore, the pixel value of the central grid area of the grid image needs to be determined, and the central grid area The pixel value is used as the reference value. Since the transmittance of the central area is high, that is, the distortion rate is low, it matches the actual shooting light source, so the pixel value in the central area can be used as a reference value to normalize the overall image.
  • the terminal device may use the pixel value of the central grid area as a reference value, and perform normalization processing on each characteristic pixel value in the grid area, thereby determining the normalized pixel value of each grid area.
  • the pixel value of the central grid area is (R 0 , G 0 , B 0 )
  • the characteristic pixel value corresponding to any grid area is (R, G, B)
  • the normalized pixel value is
  • the R/G layer corresponding to the grid image is generated according to the ratio of the R channel value and the G channel value of the normalized pixel value of all the grid regions.
  • the terminal device can calculate the ratio of the R channel value to the G channel value of the normalized pixel value corresponding to each grid area to obtain the brightness value of the grid area, namely And according to the brightness value corresponding to each grid area, an R/G layer is generated.
  • a B/G layer corresponding to the grid image is generated according to the ratio of the B channel value to the G channel value of the normalized pixel value of all the grid regions.
  • the terminal device can calculate the ratio of the B channel value to the G channel value of the normalized pixel value corresponding to each grid area to obtain the brightness value of the grid area, namely And according to the brightness value corresponding to each grid area, a B/G layer is generated.
  • FIG. 16 shows a schematic diagram of generating R/G layers and B/G layers provided by an embodiment of the present application.
  • the three-dimensional surfaces corresponding to the three RGB channels of a grid image in RGB format are as follows. After normalization processing and normalized pixel division, the R/G layer and B/ G layer.
  • the R/G layer and the B/G layer are identified as the first brightness layer.
  • the above two layers are collectively referred to as the first brightness layer.
  • the normalized R/G layer and the normalized B/G layer are generated by normalizing the grid image, and the above two layers are used as the grid image
  • the first brightness layer can facilitate subsequent adjustments to the pixel values of different channels, which improves the efficiency and accuracy of adjustment.
  • FIG. 17 shows a specific implementation flowchart of an image processing method S302 provided by the seventh embodiment of the present application.
  • the image processing method provided in this embodiment includes in S302: S1701 to S1703, and the details are as follows :
  • the generating the first brightness layer of the grid image and determining the reference gain compensation array for adjusting the first brightness layer includes:
  • each of the candidate gain compensation arrays in the gain compensation set is used to adjust the first brightness layer to obtain a brightness calibration layer corresponding to each of the candidate gain compensation arrays.
  • the number of gain compensation arrays for brightness adjustment may be fixed.
  • all candidate gain compensation arrays may be stored in a gain compensation set, and the terminal device may, during each adjustment process, Each candidate gain compensation array can be extracted from the gain compensation set to adjust the brightness layer to obtain the above-mentioned brightness calibration layer.
  • the first brightness layer adopts the method of the fifth embodiment, that is, includes a normalized R/G layer and a normalized B/G layer, the above two layers can be generated separately Corresponding brightness calibration layers, and configure corresponding reference gain compensation arrays for the above two layers respectively.
  • the terminal device can calculate the smoothing coefficient corresponding to the candidate gain compensation array according to the brightness value of each pixel in the brightness calibration layer.
  • the smoothing coefficient can be determined based on the standard deviation or the mean square error of each brightness value, and
  • the three-dimensional surface corresponding to the brightness calibration layer can be generated, and the dispersion coefficient of the three-dimensional surface can be obtained, and the smoothing coefficient is determined based on the dispersion coefficient. The higher the dispersion, the smaller the value of the smoothing coefficient.
  • the terminal device adjusts the first brightness layer through a candidate gain compensation array to generate a brightness calibration layer, and then calculates the smoothing coefficient of the brightness calibration layer, and detects If the smoothing coefficient is greater than the preset smoothing threshold, the candidate gain compensation array is identified as the reference gain compensation array, and there is no need to continue to adjust the first brightness layer through other candidate gain compensation arrays; on the contrary, if the smoothing coefficient is less than or equal to the preset Then continue to extract other candidate gain compensation arrays from the gain compensation set to adjust the first brightness layer until it is detected that the smoothing coefficient is greater than the preset smoothing threshold or the smoothing coefficients corresponding to all candidate gain compensation arrays are calculated.
  • the candidate gain compensation array with the largest smoothing coefficient is selected as the reference gain compensation array.
  • the terminal device can select the candidate gain compensation array with the largest smoothing coefficient value as the reference gain compensation array.
  • the larger the smoothing coefficient the smaller the difference between the brightness values of the adjusted brightness calibration layer.
  • the corresponding three-dimensional curved surface approaches a plane, and the degree of color shading is low, so the corresponding candidate gain compensation array can be used as the reference gain compensation array.
  • the reference compensation array with the best smoothing effect can be selected, which improves the accuracy of the selection of the gain compensation array.
  • FIG. 17 shows a structural block diagram of an image processing apparatus provided in an embodiment of the present application. For ease of description, only the parts related to the embodiment of the present application are shown.
  • the image processing apparatus includes:
  • the grid image conversion unit 181 is configured to convert the red, yellow, and blue RYB image into a grid image based on the red, green, and blue RGB format;
  • the reference gain compensation array determining unit 183 is configured to generate the first brightness layer of the grid image and determine the reference gain compensation array used to adjust the first brightness layer;
  • the target gain compensation array obtaining unit 183 is configured to obtain a target gain compensation array based on the RYB format associated with the reference gain compensation array based on the preset corresponding relationship of the compensation array;
  • the image calibration unit 184 is configured to adjust the second brightness layer of the RYB image through the target gain compensation array to generate a corrected image.
  • the image processing apparatus further includes:
  • the training control group acquisition unit is used to acquire a plurality of training control groups under different brightness environments; each of the training control groups includes at least one first training image based on the RYB format and a second training image based on the RGB format;
  • a training compensation array generating unit configured to generate a first training compensation array of the first training image, and a second training compensation array of the second training image;
  • the training and learning unit is configured to use the first training compensation array as the output sample of the back propagation algorithm network, and the second training compensation array as the input sample of the back propagation algorithm network, to perform The algorithm network is trained to obtain the compensation conversion network;
  • the compensation array correspondence relationship establishment unit is used to input each reference gain compensation array based on the RGB format to the compensation conversion network, determine the target gain compensation array based on the RYB format corresponding to each of the reference gain compensation arrays, and generate the compensation Array correspondence.
  • the grid image conversion unit 181 includes:
  • the characteristic pixel value determining unit is configured to divide the RYB image into a plurality of grid areas, and respectively determine the characteristic pixel value of each of the grid areas according to the pixel points in each of the grid areas;
  • a down-sampled image generating unit configured to generate a down-sampled image of the RYB image based on the characteristic pixel value of each of the grid regions;
  • the down-sampled image conversion unit is used to convert the down-sampled image into the grid image based on the RGB format through an RGB conversion algorithm.
  • the down-sampled image conversion unit includes:
  • a light source type determining unit configured to determine the type of light source when the RYB image is collected based on the pixel value of each pixel in the RYB image;
  • An RGB conversion algorithm selection unit configured to select the RGB conversion algorithm matching the type of the light source
  • the RGB conversion algorithm adjustment unit is configured to convert the down-sampled image into the grid image based on the RGB format through the RGB conversion algorithm.
  • the image calibration unit 183 includes:
  • An image size acquiring unit for acquiring the image size of the RYB image
  • a gain compensation array expansion unit configured to expand the target gain compensation array to an expansion gain compensation array with the same size as the image by a bilinear interpolation method
  • the corrected image generating unit is configured to adjust each brightness value in the second brightness layer through each compensation coefficient in the extended gain compensation array to obtain the corrected image.
  • the reference gain compensation array determining unit 182 includes:
  • a central pixel value acquiring unit configured to acquire the pixel value of the central grid area of the grid image
  • the normalized pixel value determining unit is configured to perform normalization processing on each characteristic pixel value in the grid image according to the pixel value of the central grid area to obtain the normalized pixel value of each grid area ;
  • the R/G layer generating unit is configured to generate the R/G layer corresponding to the grid image according to the ratio of the R channel value to the G channel value of the normalized pixel value of all the grid areas;
  • the B/G layer generating unit is configured to generate the B/G layer corresponding to the grid image according to the ratio of the B channel value to the G channel value of the normalized pixel value of all the grid areas;
  • the first brightness layer generating unit is configured to recognize the R/G layer and the B/G layer as the first brightness layer.
  • the reference gain compensation array determining unit 182 includes:
  • a brightness calibration layer generating unit configured to adjust the first brightness layer through each candidate gain compensation array in the gain compensation set to obtain the brightness calibration layer corresponding to each candidate gain compensation array;
  • a smoothing coefficient calculation unit configured to determine the smoothing coefficient of each candidate gain compensation array according to the brightness value of each pixel in the brightness calibration layer;
  • the candidate gain compensation array selection unit is configured to select the candidate gain compensation array with the largest smoothing coefficient as the reference gain compensation array.
  • the image processing device can also convert the RYB image to a grid image in RGB format that is better for the discrete performance of different light source environments, and generate a first gain compensation array corresponding to the grid image. Then, through the preset compensation array correspondence relationship, a second gain compensation array based on the RYB format is generated, and the RYB image is adjusted through the second gain compensation array to generate a corrected image. While avoiding color distortion, through the second gain compensation array The gain compensation array adjusts the brightness layer of the RYB image to eliminate color shadows and improve the imaging effect.
  • FIG. 19 shows a schematic diagram of image correction provided by an embodiment of the present application, in which (a) of FIG. 19 is the brightness layer of the RYB image before adjustment, and (b) of FIG. 19 is The pixel distribution map of the R/G layer corresponding to the brightness layer of the RYB image before adjustment, where the abscissa is the pixel point column coordinates, and the ordinate is the normalized R/G value; Figure 19(c) It is the brightness layer of the RYB image adjusted according to the target gain compensation array, and Figure 19(d) is the pixel distribution map of the R/G layer corresponding to the brightness layer of the adjusted RYB image, where the abscissa is Pixel column coordinates, the ordinate is the normalized R/G value.
  • FIG. 20 is a schematic structural diagram of a terminal device provided by an embodiment of this application.
  • the terminal device 20 of this embodiment includes: at least one processor 200 (only one is shown in FIG. 20), a processor, a memory 201, and a processor stored in the memory 201 and capable of being processed in the at least one processor.
  • the computer program 202 running on the processor 200 implements the steps in any of the foregoing image processing method embodiments when the processor 200 executes the computer program 202.
  • the terminal device 20 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the terminal device may include, but is not limited to, a processor 200 and a memory 201.
  • FIG. 20 is only an example of the terminal device 20, and does not constitute a limitation on the terminal device 20. It may include more or less components than shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices, network access devices, and so on.
  • the so-called processor 200 may be a central processing unit (Central Processing Unit, CPU), and the processor 200 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 201 may be an internal storage unit of the terminal device 20, such as a hard disk or a memory of the terminal device 20.
  • the memory 201 may also be an external storage device of the terminal device 20, for example, a plug-in hard disk equipped on the terminal device 20, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, Flash Card, etc.
  • the memory 201 may also include both an internal storage unit of the terminal device 20 and an external storage device.
  • the memory 201 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program.
  • the memory 201 can also be used to temporarily store data that has been output or will be output.
  • An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and running on the at least one processor, and the processor executes The computer program implements the steps in any of the foregoing method embodiments.
  • the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the foregoing method embodiments can be realized.
  • the embodiments of the present application provide a computer program product.
  • the steps in the foregoing method embodiments can be realized when the mobile terminal is executed.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), and random access memory (RAM, Random Access Memory), electric carrier signal, telecommunications signal and software distribution medium.
  • ROM read-only memory
  • RAM random access memory
  • electric carrier signal telecommunications signal and software distribution medium.
  • U disk mobile hard disk, floppy disk or CD-ROM, etc.
  • computer-readable media cannot be electrical carrier signals and telecommunication signals.
  • the disclosed apparatus/network equipment and method may be implemented in other ways.
  • the device/network device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units.
  • components can be combined or integrated into another system, or some features can be omitted or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

本申请适用于数据处理技术领域，提供了一种图像处理的方法、装置、终端以及存储介质，该方法包括：将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列；基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。本申请提供的技术方案能够在避免颜色失真的同时，通过第二增益补偿阵列对RYB图像的亮度图层进行调整，消除了颜色阴影，提高了成像效果。

Description

一种图像处理的方法、装置、终端以及存储介质
本申请要求于2020年01月15日提交国家知识产权局、申请号为202010044246.4、申请名称为“一种图像处理的方法、装置、终端以及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请属于数据处理技术领域，尤其涉及一种图像处理的方法、装置、终端以及存储介质。
背景技术
红黄蓝RYB图像传感器，由于成像效果优于红绿蓝RGB传感器，已被广泛应用。为了降低RYYB图像传感器在拍摄过程中的颜色失真，可以在RYYB图像传感器上加装滤光片。但由于滤光片在不同的入射角度的透射率不同，减少RYB图像的颜色失真的同时，会导致了整体图像亮度不均匀，存在颜色阴影(Color Shading)的情况。
发明内容
本申请实施例提供了一种图像处理的方法、装置、终端以及存储介质，可以解决现有的基于RYB格式的图像处理技术，无法同时解颜色失真以及颜色阴影的问题。
第一方面，本申请实施例提供了一种图像处理的方法，包括：
将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；
生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列；
基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；
通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
在第一方面的一种可能的实现方式中，在所述基于预设的补偿阵列对应关系，获得与所述第一增益补偿阵列关联的基于RYB格式的第二增益补偿阵列之前，还包括：
获取多个不同亮度环境下的训练对照组；每个所述训练对照组包括至少一个基于RYB格式的第一训练图像以及基于RGB格式的第二训练图像；
生成所述第一训练图像的第一训练补偿阵列，以及生成所述第二训练图像的第二训练补偿阵列；
将所述第一训练补偿阵列作为反向传播算法网络的输出样本、将所述第二训练补偿阵列作为所述反向传播算法网络的输入样本，对所述反向传播算法网络进行训练，得到补偿转换网络；
将基于RGB格式的各个基准增益补偿阵列输入至所述补偿转换网络，确定各个所述基准增益补偿阵列对应的基于RYB格式的目标增益补偿阵列，生成所述补偿阵列对应关系。
在第一方面的一种可能的实现方式中，所述将红黄蓝RYB图像转换为基于红绿蓝 RGB格式的网格图像，包括：
将RYB图像划分为多个网格区域，并根据各个所述网格区域内的像素点，分别确定各个所述网格区域的特征像素值；
基于各个所述网格区域的特征像素值，生成所述RYB图像的降采样图像；
通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
在第一方面的一种可能的实现方式中，所述通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像，包括：
基于所述RYB图像内各个像素点的像素值，确定采集所述RYB图像时的光源类型；
选取与所述光源类型匹配的所述RGB转换算法；
通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
在第一方面的一种可能的实现方式中，所述通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像，包括：
获取所述RYB图像的图像尺寸；
通过双线性插值法将所述目标增益补偿阵列扩展至与所述图像尺寸相同的扩展增益补偿阵列；
通过所述扩展增益补偿阵列内各个补偿系数，对所述第二亮度图层内各个亮度值进行调整，得到所述校正图像。
在第一方面的一种可能的实现方式中，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
获取所述网格图像的中心网格区域的像素值；
根据所述中心网格区域的像素值，对所述网格图像内的各个特征像素值进行归一化处理，得到各个所述网格区域的归一像素值；
根据所有所述网格区域的所述归一像素值的R通道数值与G通道数值之比，生成所述网格图像对应的R/G图层；
根据所有所述网格区域的所述归一像素值的B通道数值与G通道数值之比，生成所述网格图像对应的B/G图层；
将所述R/G图层以及所述B/G图层识别为所述第一亮度图层。
在第一方面的一种可能的实现方式中，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
分别通过增益补偿集合内各个候选增益补偿阵列，对所述第一亮度图层进行调整，获得各个所述候选增益补偿阵列对应的亮度校准图层；
根据所述亮度校准图层内各个像素点的亮度值，分别确定各个所述候选增益补偿阵列的平滑系数；
选取所述平滑系数最大的候选增益补偿阵列作为所述基准增益补偿阵列。
第二方面，本申请实施例提供了一种图像处理的装置，包括：
网格图像转换单元，用于将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；
基准增益补偿阵列确定单元，用于生成所述网格图像的第一亮度图层，并确定用 于调整所述第一亮度图层的基准增益补偿阵列；
目标增益补偿阵列获取单元，用于基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；
图像校准单元，用于通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
第三方面，本申请实施例提供了一种终端设备，存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述第一方面中任一项所述图像处理的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现上述第一方面中任一项所述图像处理的方法。
第五方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备执行上述第一方面中任一项所述图像处理的方法。
可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
本申请实施例与现有技术相比存在的有益效果是：
本申请实施例通过将RYB图像转换到对不同光源环境的离散表现更好的RGB格式的网格图像，并生成与网格图像对应的第一增益补偿阵列，继而通过预设的补偿阵列对应关系，生成了基于RYB格式的第二增益补偿阵列，并通过第二增益补偿阵列对RYB图像进行调整，生成校正图像，在避免颜色失真的同时，通过第二增益补偿阵列对RYB图像的亮度图层进行调整，消除了颜色阴影，提高了成像效果。
附图说明
图1是本申请实施例提供的手机的部分结构的框图；
图2是本申请实施例的手机的软件结构示意图；
图3是本申请第一实施例提供的一种图像处理的方法的实现流程图；
图4是本申请一实施例提供的基于RYYB图像传感器的成像原理图；
图5是本申请一实施例提供的通过加装红外滤光片的摄像模块拍摄的RYB图像的示意图；
图6是本申请一实施例提供的第一亮度图像的曲面示意图；
图7是本申请一实施例提供的不同光源类型在RGB图像的光源坐标以及在RYB图像的光源坐标的对比示意图；
图8是本申请一实施例提供的亮度调整的示意图；
图9是本申请第二实施例提供的一种图像处理的方法具体实现流程图；
图10是本申请一实施例提供的BP网络的训练示意图；
图11是本申请第三实施例提供的一种图像处理的方法S301具体实现流程图；
图12是本申请一实施例提供的降采样图像的生成示意图；
图13是本申请第四实施例提供的一种图像处理的方法S3013具体实现流程图；
图14是本申请第五实施例提供的一种图像处理的方法S304具体实现流程图；
图15是本申请第六实施例提供的一种图像处理的方法S302的具体实现流程图；
图16是本申请一实施例提供的R/G图层与B/G图层的生成示意图；
图17是本申请第七实施例提供的一种图像处理的方法S302具体实现流程图；
图18是本申请一实施例提供的一种图像处理的设备的结构框图；
图19是本申请一实施例提供的图像校正的示意图；
图20是本申请另一实施例提供的一种终端设备的示意图。
具体实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
应当理解，当在本申请说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本申请说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外，在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
本申请实施例提供的图像处理的方法可以应用于手机、智能照相机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等终端设备上，还可以应用于数据库、服务器以及基于终端人工智能的服务响应系统，用于响应其他终端反馈的图像处理请求，本申请实施例对终端设备的具体类型不作任何限制。
例如，所述终端设备可以是WLAN中的站点(STAION，ST)，可以是蜂窝电话、无绳电话、会话启动协议(Session InitiationProtocol，SIP)电话、无线本地环路(Wireless Local Loop，WLL)站、个人数字处理(Personal Digital Assistant，PDA)设备、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、电脑、膝 上型计算机、手持式通信设备、手持式计算设备、和/或用于在无线系统上进行通信的其它设备以及下一代通信系统，例如，5G网络中的移动终端或者未来演进的公共陆地移动网络(Public Land Mobile Network，PLMN)网络中的移动终端等。
作为示例而非限定，当所述终端设备为可穿戴设备时，该可穿戴设备还可以是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如具有拍摄功能的眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备，通过附着与用户身上，用于记录用户行进过程中的图像或根据用户发起的拍摄指令，采集环境图像等。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，如智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行图像采集的智能手表、智能眼镜等。
以所述终端设备为手机为例。图1示出的是与本申请实施例提供的手机的部分结构的框图。参考图1，手机包括：射频(Radio Frequency，RF)电路110、存储器120、输入单元130、显示单元140、传感器150、音频电路160、近场通信模块170、处理器180、以及电源190等部件。本领域技术人员可以理解，图1中示出的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图1对手机的各个构成部件进行具体的介绍：
RF电路110可用于收发信息或通话过程中，信号的接收和发送，特别地，将基站的下行信息接收后，给处理器180处理；另外，将设计上行的数据发送给基站。通常，RF电路包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器(Low Noise Amplifier，LNA)、双工器等。此外，RF电路110还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议，包括但不限于全球移动通讯系统(Global System of Mobile communication，GSM)、通用分组无线服务(General Packet Radio Service，GPRS)、码分多址(Code Division Multiple Access，CDMA)、宽带码分多址(Wideband Code Division Multiple Access,WCDMA)、长期演进(Long Term Evolution,LTE))、电子邮件、短消息服务(Short Messaging Service，SMS)等，通过RF电路110接收其他终端获取的图像，并对获取的图像进行处理，输出对应的校正图像。
存储器120可用于存储软件程序以及模块，处理器180通过运行存储在存储器120的软件程序以及模块，从而执行手机的各种功能应用以及数据处理，例如将预先配置的补偿阵列对应关系存储于存储器120内。存储器120可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器120可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
输入单元130可用于接收输入的数字或字符信息，以及产生与手机100的用户设 置以及功能控制有关的键信号输入。具体地，输入单元130可包括触控面板131以及其他输入设备132。触控面板131，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板131上或在触控面板131附近的操作)，并根据预先设定的程式驱动相应的连接装置。
显示单元140可用于显示由用户输入的信息或提供给用户的信息以及手机的各种菜单，例如输出调整后的校正图像。显示单元140可包括显示面板141，可选的，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板141。进一步的，触控面板131可覆盖显示面板141，当触控面板131检测到在其上或附近的触摸操作后，传送给处理器180以确定触摸事件的类型，随后处理器180根据触摸事件的类型在显示面板141上提供相应的视觉输出。虽然在图1中，触控面板131与显示面板141是作为两个独立的部件来实现手机的输入和输入功能，但是在某些实施例中，可以将触控面板131与显示面板141集成而实现手机的输入和输出功能。
手机100还可包括至少一种传感器150，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板141的亮度，接近传感器可在手机移动到耳边时，关闭显示面板141和/或背光。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；至于手机还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
手机100还可以包括摄像头160。可选地，摄像头在手机100的上的位置可以为前置的，也可以为后置的，本申请实施例对此不作限定。
可选地，手机100可以包括单摄像头、双摄像头或三摄像头等，本申请实施例对此不作限定。
例如，手机100可以包括三摄像头，其中，一个为主摄像头、一个为广角摄像头、一个为长焦摄像头。
可选地，当手机100包括多个摄像头时，这多个摄像头可以全部前置，或者全部后置，或者一部分前置、另一部分后置，本申请实施例对此不作限定。
终端设备可以通过近场通信模块170可以接收其他设备发送的待处理图像，例如该近场通信模块170集成有蓝牙通信模块，通过蓝牙通信模块与智能相机建立通信连接，并接收智能相机反馈的待处理图像。虽然图1示出了近场通信模块170，但是可以理解的是，其并不属于手机100的必须构成，完全可以根据需要在不改变申请的本质的范围内而省略。
处理器180是手机的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器120内的软件程序和/或模块，以及调用存储在存储器120内的数据，执行手机的各种功能和处理数据，从而对手机进行整体监控。可选的，处理器180可包括一个或多个处理单元；优选的，处理器180可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制 解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器180中。
手机100还包括给各个部件供电的电源190(比如电池)，优选的，电源可以通过电源管理系统与处理器180逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。
手机100还包括音频电路、扬声器，传声器可提供用户与手机之间的音频接口。音频电路可将接收到的音频数据转换后的电信号，传输到扬声器，由扬声器转换为声音信号输出；另一方面，传声器将收集的声音信号转换为电信号，由音频电路接收后转换为音频数据，再将音频数据输出处理器180处理后，经RF电路110以发送给比如另一手机，或者将音频数据输出至存储器120以便进一步处理。例如，用户可以通过音频电路采集用户的语音信号，基于语音信号控制摄像头160执行图像采集的操作，并对采集后的图像进行处理，得到校正图像。
图2是本申请实施例的手机100的软件结构示意图。以手机100操作系统为Android系统为例，在一些实施例中，将Android系统分为四层，分别为应用程序层、应用程序框架层(framework，FWK)、系统层以及硬件抽象层，层与层之间通过软件接口通信。
如图2所示，所述应用程序层可以一系列应用程序包，应用程序包可以包括短信息，日历，相机，视频，导航，图库，通话等应用程序。特别地，语音识别算法可以嵌入至应用程序内，通过应用程序内的相关控件启动图像处理流程，并处理获取得到的RYB图像，得到消除颜色阴影后的校正图像。
应用程序框架层为应用程序层的应用程序提供应用编程接口(applicationprogramming interface，API)和编程框架。应用程序框架层可以包括一些预先定义的函数，例如用于接收应用程序框架层所发送的事件的函数。
如图2所示，应用程序框架层可以包括窗口管理器、资源管理器以及通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
应用程序框架层还可以包括：
视图系统，所述视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如， 包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供手机100的通信功能。例如通话状态的管理(包括接通，挂断等)。
系统层可以包括多个功能模块。例如：传感器服务模块，物理状态识别模块，三维图形处理库(例如：OpenGL ES)等。
传感器服务模块，用于对硬件层各类传感器上传的传感器数据进行监测，确定手机100的物理状态；
物理状态识别模块，用于对用户手势、人脸等进行分析和识别；
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
系统层还可以包括：
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的静态图像文件，视频格式回放和录制，以及音频等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
硬件抽象层是硬件和软件之间的层。硬件抽象层可以包括显示驱动、摄像头驱动、传感器驱动、麦克风驱动等，用于驱动硬件层的相关硬件，如显示屏、摄像头、传感器以及麦克风等。特别地，通过摄像头驱动启动摄像模块，该摄像模块具体为基于RYYB图像传感器的摄像模块，通过RYYB图像传感器采集拍摄指令触发时刻对应的光信息，生成RYB图像。
需要说明的是，本申请实施例提供的图像处理的方法可以在上述任一层级中执行，在此不做限定。
在本申请实施例中，流程的执行主体为安装有图像处理的程序的设备。作为示例而非限定，图像处理的程序的设备具体可以为终端设备，该终端设备可以为用户使用的智能手机、智能相机、平板电脑等，对获取得到的RYB图像进行处理，并生成调整后的校正图像，以消除采集RYB图像过程中引入了颜色阴影，提高成效效果。图3示出了本申请第一实施例提供的图像处理的方法的实现流程图，详述如下：
在S301中，将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像。
在本实施例中，终端设备可以通过内置有基于RYB成像原理的摄像模块获取得到RYB图像，在该情况下，用户可以通过启动终端设备内的特定应用以激活摄像模块，例如相机应用、实时视频通话应用等；用户还可以通过点击当前应用中的部分控件，以激活摄像模块，例如在社交应用中点击发送拍照控件，将采集到的RYB图像作为交互信息发送给通信对端，此时终端设备会通过摄像模块采集用户在点击操作时刻的环境图像，作为上述的需要调整的RYB图像。终端设备还可以通过外置的摄像模块采集待调整的RYB图像，在该情况下，终端设备可以通过无线通信模块或串行接口等方式与外置的摄像模块建立通信连接，用户可以通过点击摄像模块上的开门，控制摄像模块采集图像，并通过内置的RYB图像传感器生成对应的RYB图像，并将RYB图像通过上述建立的通信连接传输给终端设备，终端设备接收到摄像模块反馈的RYB图像后， 可以执行后续的图像处理流程。
在一种可能的实现方式中，终端设备除了可以通过内置或外置的摄像模块获取待处理的RYB图像后外，还可以通过通信对端发送的方式进行获取。终端设备可以通过通信模块与通信对端建立通信连接，通过通信连接接收通信对端发送的RYB图像，其中，通信对端获取RYB图像的方式可以参见上述过程，在此不再赘述。终端设备在接收到通信对端反馈的RYB图像后，可以对该目RYB图像进行颜色阴影的消除处理。在一种可能的实现方式中，该终端设备可以为一云端服务器，各个通信对端可以安装有与云端服务器对应的客户端程序，或通过与API接口在通信对端本地生成与云端服务器对应的应用界面，将本地获取得到的RYB图像通过客户端程序或API接口发送给云端服务器，云端服务器将处理后的校正图像反馈给通信对端，通信对端在接收到处理后的RYB图像后，即上述的校正图像后，可以在通信对端的显示模块上输出上述校正图像，并在接收到拍摄完成指令后，对校正图像进行存储。
在一种可能的实现方式中，终端设备采集RYB图像的时所使用的摄像模块，是基于RYYB图像传感器生成的，该RYYB图像传感器包含有四个颜色通道，分别为一个用于采集红光的R通道传感器、一个用于采集蓝光的B通道传感器以及两个用于采集黄光的Y通道传感器，原始生成的图像具有四个不同的颜色通道，即生成的为RYYB图像。终端设备可以将RYYB图像转换为RYB图像，实现的过程可以为通过计算RYYB图像中每个像素点在两个不同的Y通道对应的像素值的均值，作为合并后的RYB图像的Y通道的像素值，红色通道以及蓝色通道的像素值保持不变，从而实现将RYYB图像转换为RYB图像。
示例性地，图4示出了本申请一实施例提供的基于RYYB图像传感器的成像原理图。参见图4所示，该RYYB图像传感器包含有四个通道的传感器，分别为一个R通道传感器、一个B通道传感器以及两个Y通道的传感器，在入射光与RYYB图像传感器之间的光路中，配置有一红外滤光片，由于RYYB图像传感器是内的感光元件是基于互补金属氧化物半导体(Complementary Metal Oxide Semiconductor，CMOS)传感器成像，而CMOS传感器对于红外线照射较为敏感，因此容易出现颜色失真，为了减少因对红光敏感而导致图像产生色偏，可以在入射光路中加装红外滤光片。而拍摄图像时存在多个方向的入射光，不同的入射光与红外滤光片之间的入射角度不同，会影响不同入射角度的光线的透射率，从而导致了图像中心区域的红光透射率高，而图像边界区域的红果透射率低，造成图像中心区域偏红，色彩空间分布不均匀的情况。
示例性地，图5示出了本申请一实施例提供的通过加装红外滤光片的摄像模块拍摄的RYB图像的示意图。参见图5可以确定，在加装了红外滤光片后的摄像模块拍摄同一类型物体时，在图像不同区域下，其像素分布会存在差异。根据拍摄得到的RYB图像生成对应的亮度图层，可以确定，该亮度图层的中心区域的亮度值较高，而边界区域的亮度值较低，出现了颜色阴影的情况，降低了成像效果。上述颜色阴影的情况，在拍摄大量区域颜色相同的场景下尤为明显，例如在拍摄室内墙壁、室外桥面、白炽灯、紫色花田或以上述场景为背景的图像。因此，为了消除拍摄的RYB图像时由于红外滤光片引入的上述失真，可以通过S301至S304的方式进行调整，得到调整后的校正图像。
在本实施例中，由于黄光是由红光以及绿光叠加后得到的，在获取得到RYB图像后，可以根据R通道内各个像素点的像素值以及Y通道内各个像素点的像素值，确定各个像素点在G通道对应的像素值，实现了将RYB图像转换为对应的RGB图像。
在一种可能的实现方式中，终端设备可以采用以下算法计算得到RYB图像内各个像素点在转换为RGB图像中G通道的像素值：G＝(2Y-R)，其中，G表示转换为G通道的像素值；Y表示转换前Y通道的像素值；R表示转换前R通道的像素值。
在一种可能的实现方式中，终端设备可以配置有与RYB图像传感器对应的转换算法。终端设备可以获取RYB图像传感器的设备型号，并基于上述的设备型号下载对应的转换算法，基于获取得到的转换算法将RYB图像转换为RGB图像。
在本实施例中，由于颜色阴影与像素所在的位置区域相关，即调整系数与位置区域具有强相关性。为了减少后续操作的计算量，可以将转换得到的RGB图像进行降采样，将RGB图像划分为多个网格区域，并获取每个网格区域对应的特征像素点，生成上述的基于RGB格式的网格图像。
举例性地，若某一RYB图像的图像尺寸为3100*2100，即每行包含3100个像素点，每列包含2100个像素点，而转换得到的RGB图像的图像尺寸与RYB图像的图像尺寸一致，可以同样为3100*2100，在该情况下，可以降采样为31*21的网格，即每个网格包含100*100个像素点，上述像素点通过一个特征像素值表示，从而降采样为一31*21的网格图像。
在S302中，生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列。
在本实施例中，终端设备在获取得到RYB图像对应的基于RGB格式的网格图像后，可以生成网格图像对应的第一亮度图层。其中，该第一亮度图层具体用于表示网格图像中各个网格区域的亮度值，该亮度值与该网格区域对应的特征像素值的色度无关。在一种可能的实现方式中，上述第一亮度图层可以是基于色调饱和度亮度(Hue,Saturation,Value，HSV)格式的V图层，还可以为Lab色彩格式的照度(L)图层，还可以对网格图像进行归一化后的R/G图层以及B/G图层，在此不限定第一亮度图层的表示方式。
在一种可能的实现方式中，生成第一亮度图层的方式可以为：由于网格图像是基于RGB格式生成的图像，即网格图像内各个网格区域对应的特征像素值包含RGB三个通道的像素值，基于此，终端设备可以通过RGB与HSV之间的转换算法，将网格图像内各个区域网格在RGB三个通道对应的像素值映射到HSV通道，以HSV三个通道表征每个网格区域的特征像素值，并根据各个网格区域在V通道的像素值进行组合，生成上述的第一亮度图层。
在一种可能的实现方式中，生成第一亮度图层的方式可以为：由于网格图像是基于RGB格式生成的图像，即网格图像内各个网格区域对应的特征像素值包含RGB三个通道的像素值，基于此，终端设备可以通过RGB与Lab之间的转换算法(其中，Lab格式的图像中，包含L通道、a通道以及b通道；L通道指的是用于表示像素点照度的通道；a通道指的是用于表示红色至绿色的通道；b通道指的是用于表示蓝色至黄色的通道)，将网格图像内各个区域网格在RGB三个通道对应的像素值映射到Lab 通道，以Lab三个通道表征每个网格区域的特征像素值，并根据各个网格区域在L通道的像素值进行组合，生成上述的第一亮度图层。
在本实施例中，由于摄像模块加装有红外滤光片，不同角度的入射光线的透光率不同，会导致中间区域的亮度高于四周边界的亮度，因此，第一亮度图层内各个网格区域的亮度值会存在差异，而在无颜色阴影的情况下，各个网格区域对应的亮度值应该相同。基于此，终端设备可以通过预设的基准增益补偿阵列，对第一亮度图层内各个亮度值进行调整，从而使得各个亮度值趋于相同，以消除颜色阴影。
示例性地，图6示出了本申请一实施例提供的第一亮度图像的曲面示意图。参见图6所示，图6中的(a)为通过基准增益补偿阵列调整前的第一亮度图像，由于存在颜色阴影的情况，该图像的中心区域的亮度值较高，而四周的亮度值较低。而由于在没有颜色阴影的情况下，第一亮度图层对应的三维曲面是趋近一个平面，因此，终端设备可以通过基准增益补偿阵列对第一亮度图像内的各个亮度值进行调整，从而使得调整后的第一亮度图层的三维曲面是一个平面，图6中的(b)为通过基准增益补偿阵列调整后的第一亮度图像。
在一种可能的实现方式中，终端设备可以配置有多个预设的候选增益补偿阵列，分别通过各个候选增益阵列对第一亮度图层进行调整，识别调整后的第一亮度图层内各个网格区域的亮度值之间的标准差，并选取标准差的数值最小的一个候选增益补偿阵列作为上述的基准增益补偿整理。由于调整的目的是使第一亮度图层对应的三维曲面是趋近一个平面，因此该调整后的各个网格区域的亮度值之间的标准差应该较小，理想情况下该标准差为0，因此，终端设备可以计算通过各个候选增益补偿阵列调整后的第一亮度图层内各个亮度值之间的标准差，选取标准差最小的一个作为基准增益补偿阵列。
在一种可能的实现方式中，终端设备可以将第一亮度图层导入到预设的增益阵列选取网络，输出与第一亮度图层匹配的基准补偿增益阵列。其中，该增益阵列选取网络可以基于浅层神经网络搭建，还可以基于向量机、随机森林、强化学习等模型进行搭建。可选地，终端设备可以通过RGB图像传感器在不同光源类型以及光照强度下采集训练图像，光源类型包括但不限于：A，D50，D65，D75，H,TL83，TL84，U30，光照强度包括但不限于：1000lux，700lux，300lux，100lux，20lux，因此，通过将不同光源类型以及不同光照强度进行组合，可以得到M*N种拍摄场景，M为光源类型的个数，N为光照强度的个数，在不同的拍摄场景下采集多个不同的训练图像，并为不同的训练图像的训练亮度图层配置对应的训练增益补偿阵列，将训练亮度图层作为增益阵列选取网络的输入、将训练增益补偿阵列作为增益阵列选取网络的输出，对增益阵列选取网络进行训练学习，从而生成上述的增益阵列选取网络。
需要说明的是，若第一亮度图层通过两个或以上的图层表示RGB的网格图像的亮度信息，则上述获得的的基准增益补偿阵列的个数与图层个数相匹配，例如，网格图像的第一亮度图层包含了R/G图层以及B/G图层，则可以为R/G图层配置对应的基准增益补偿阵列以及为B/G图层配置对应的基准增益补偿阵列，在后续获取的目标补偿阵列的个数也可以为多个，通过获取得到的目标增益补偿阵列分别对R/Y图层以及B/Y图层进行调整。
在S303中，基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列。
在本实施例中，由于上述的基准增益补偿阵列是基于RGB格式的增益补偿阵列，即可以对RGB图像内的三个通道进行调整，而原始拍摄得到的图像是基于RYB图像传感器拍摄得到的，因此需要将基准增益补偿阵列映射到基于RYB格式的目标增益补偿阵列中，以便对原始的RYB图像内的RYB三个通道的亮度值进行调整。
在本实施例中，采用RGB格式的网格图像获取对应的基准增益补偿阵列的原因在于，相当于RYB图像，RGB图像对于不同光源在亮度图层上的离散程度较大，图7示出了本申请一实施例提供的不同光源类型在RGB图像的光源坐标以及在RYB图像的光源坐标的对比示意图。其中，上述光源坐标具体指的是，在RGB图像的亮度图层的中心区域的亮度值，而对于RGB图像而言，亮度图层可以通过归一化后的R/G图层以及归一化后的B/G图层进行表示，因此，上述两个图层的中心坐标点的亮度值(对于R/G图层而言，即中心坐标点归一化后的R通道像素值与G通道像素值的第一比值，即图中的R索引；对于B/G图层而言，即中心坐标点归一化后的B通道像素值与G通道像素值的第二比值，即图中的B索引；通过上述两个比值，可以在预设的坐标系为一确定一个坐标点，即该RGB图像对应光源类型的光源坐标)，相对地，对于RYB图像，则可以通过R/Y图层以及B/Y图层来作为RYB图像的亮度图层，从而确定了RYB图像的R索引以及B索引。参见图7，分别以A，D50，D65，D75，H,TL83，TL84，U30这8中光源的光源坐标进行示例说明。图7的(a)为RGB图像的光源坐标分布图，图7(b)为RYB图像的光源坐标分布图，可以看出，在RGB图像中，不同光源的光源坐标之间坐标距离较大，离散程度较高，即较容易区分不同的光源类型，不同光源类型在RGB格式的亮度图层内相互之间的差异性较大，从而后续的识别准确率较高；而在RYB图像中，不同光源的光源坐标之间的距离较小，离散程度较低(H、A、D65、D75、D50上述五个光源的R索引值几乎相同)，即较难区分不同的光源类型，不同光源类型在RYB格式的亮度图层内相互之间的差异性较小，从而后续的识别准确率较低。并且，为了提高获取的基准增益补偿阵列的准确性，可以首先将RYB图像转换到亮度差异性较大的RGB图像，并且根据基准增益补偿阵列以及不同格式图像的补偿阵列对应关系，从而得到基于RYB格式的目标增益补偿阵列，从而能够提高目标增益补偿阵列获取的准确性，从而提高了图像校正的效果。
在本实施例中，终端设备预先存储有补偿阵列对应关系，该补偿阵列对应关系具体记录有各个RGB格式的基准增益补偿阵列与RYB格式的目标增益补偿阵列之间的对应关系。终端设备在获取得到基于RGB格式的基准增益补偿阵列后，可以在上述补偿阵列对应关系中进行查找，获取得到与上述基准增益补偿阵列相对应的目标增益补偿阵列。
在一种可能的实现方式中，基准增益补偿阵列的个数是固定的，分别用于调整不同光照场景下拍摄的RGB图像，同样地，目标增益补偿阵列的个数也可以是固定的，分别用于调整不同光照场景下拍摄得到的RYB图像。因此，终端设备可以根据光照场景之间的关联关系，建立基准增益补偿阵列与目标增益补偿阵列之间的补偿阵列对应关系。例如，对于光照场景A，在RGB格式下为基准增益补偿阵列A，在RYB格式 下为目标增益补偿阵列B，则可以建立基准增益补偿阵列A与目标增益补偿阵列B之间的关联关系，并标记出对应的光照场景。在该情况下，终端设备可以根据基准增益补偿阵列识别RGB图像的光照类型，并基于上述光照类型在补偿阵列对应关系内查找匹配的目标增益补偿阵列。
在一种可能的实现方式中，终端设备可以通过机器学习算法构建上述的补偿阵列对应关系，在该情况下，终端设备可以创建多个训练样本，每个训练样本对应一个样本图像，并分别属于样本图像基于RGB格式的第一训练图像以及基于RYB格式的第二训练图像，并分别为上述两个训练图像生成对应的第一训练补偿阵列以及第二训练补偿阵列，根据多个训练样本对预设的学习算法进行训练，从而确定了各个不同基准增益补偿阵列对应的目标增益补偿阵列。上述学习算法可以为浅层神经网络，还可以是基于向量机、随机森林、强化学习等模型。
在S304中，通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
在本实施例中，终端设备在确定了目标增益补偿阵列后，可以获取RYB图像的第二亮度图层，其中，该第二亮度图层具体为与色度无关且只与亮度相关的图层。生成亮度图层的方式可以参见S302的相关描述，在此不再赘述。在一种可能的实现方式中，上述第二亮度图层可以为归一化后的R/Y图层以及归一化后的B/Y图层。其中，生成上述归一化后的R/Y图层以及归一化后的B/Y图层的方式可以为：终端设备获取RYB图像的中心坐标点的像素值，并基于中心像素点的像素值作为基准值，对其他各个像素点进行归一化处理，并计算各个像素点在归一化后的R通道像素值与Y通道像素值之间的第一比值，将所有第一比值构成上述的R/Y图层；同理，可以计算各个像素点在归一化后的B通道像素值与Y通道像素值之间的第二比值，将所有第二比值构成上述的B/Y图层。
在本实施例中，终端设备可以通过目标增益补偿阵列对RYB图像内各个像素点的像素值进行调整，从而能够生成消除颜色阴影后的校正图像。在一种可能的实现方式中，生成上述校正图像的方式可以为：由于基准补偿阵列是基于网格图像生成的，即基准补偿阵列的行列数与网格图像所对应的网格个数相同，其网格个数小于RYB图像的像素点个数，因此，基于基准增益补偿阵列转换得到的目标增益补偿阵列的行列数也与网格图像相同。在后续调整亮度值的过程中，终端设备可以将目标网格图像进行等比例放大，确定各个网格区域在RYB图像中对应的像素点，并将网格区域内包含的所有像素点的亮度值，通过该网格区域的增益补偿系数进行调整。采用降采样的方式将RGB图像转换为网格图像，能够根据增益补偿系数与像素点在图像中位置存在强关系性的特点，从而能够减少后续确定基准增益补偿系数的计算量，从而提高了整体图像处理的效率。
作为示例而非限定，图8示出了本申请一实施例提供的亮度调整的示意图，其中，图8的(a)为一目标增益补偿阵列；图8的(b)为基于目标整增益补偿阵列内各个增益补偿系数对RYB图像的第二亮度图层内各个像素点的亮度值进行调整的示意图。参见图8可以确定，该目标增益补偿阵列为一3*3的网格矩阵，每个网格区域对应一个调整系数，而RYB图像则包含多个像素点，终端设备可以将目标增益补偿阵列的尺 寸调整到与RYB图像相一致，并每个网格区域可以在RYB图像上关联有对应的像素点，可以根据网格区域对应的调整系数，对关联的像素点的亮度值进行调整，并基于调整后的第二亮度图层生成校正图像，消除图像包含的颜色阴影。
在一种现有的可能实现方式中，终端设备可以通过计算RYB图像中心像素点的坐标，确保该中心像素点落在一预设的通道上，将所述图像中心像素点的坐标为中心，对图像进行划分，获得划分后的目标区域，计算目标区域内每个通道的平均值，并将其作为对应通道的增益调整目标，利用目标值和像素点所在的位置自适应的进行像素补偿。然而上述方式主要是利用目标区域每个通道的均值作为增益调整目标不适用于颜色丰富的图像，并且对于像素点较多的图像均需要计算每个像素点的增益补偿值会增加计算量，降低了运算速度。
在一种现有的可能实现方式中，终端设备可以通过从传感阵列的每一个感光点获得亮度值，并设置用于补偿与基于光场成像原理的相机的当前设置相关联的每个感光点的一组加权值，通过一组加权值改变每个感光点的亮度值。然而上述方式基于光场成像原理的摄像模块，现有大部分的摄像模块均是基于RYB图像传感器原理进行图像采集，从而降低了适用范围，并且只利用感光点的亮度值进行阴影校正，无法对颜色阴影校正进行有效的补偿，降低了校正的效果。
本申请实施例为了解决上述问题，在保证调整效果的同时，减少计算量以及提高运算速度，采用了将RYB图像进行降采样以及转换的操作，生成基于RGB格式的网格图像，从而无需计算每个像素点对应的增益补偿系数，从而降低了生成目标增益补偿阵列的计算量，并且并且采用RGB格式在不同光源下的离散程度较好的优点，提高了目标增益补偿阵列的准确性，从而太提高了校准效果。
以上可以看出，本申请实施例提供的一种图像处理的方法通过将RYB图像转换到对不同光源环境的离散表现更好的RGB格式的网格图像，并生成与网格图像对应的第一增益补偿阵列，继而通过预设的补偿阵列对应关系，生成了基于RYB格式的第二增益补偿阵列，并通过第二增益补偿阵列对RYB图像进行调整，生成校正图像，在避免颜色失真的同时，通过第二增益补偿阵列对RYB图像的亮度图层进行调整，消除了颜色阴影，提高了成像效果。
图9示出了本申请第二实施例提供的一种图像处理的方法的具体实现流程图。参见图9，相对于图3所述实施例，本实施例提供的一种图像处理的方法中在所述基于预设的补偿阵列对应关系，获得与所述第一增益补偿阵列关联的基于RYB格式的第二增益补偿阵列之前，还包括：S901～S904，具体详述如下：
进一步地，在所述基于预设的补偿阵列对应关系，获得与所述第一增益补偿阵列关联的基于RYB格式的第二增益补偿阵列之前，还包括：
在S901中，获取多个不同亮度环境下的训练对照组；每个所述训练对照组包括至少一个基于RYB格式的第一训练图像以及基于RGB格式的第二训练图像。
在本实施例中，终端设备可以通过大量样本数据对预设的反向传播算法进行训练学习，从而能够建立RGB格式的基准增益补偿阵列与RYB格式的目标增益补偿阵列之间的转换关系，从而能够实现补偿阵列的转换。因此，在S901中，可以采集大量的 训练对照组，其中，上述训练对照组包含在覆盖了多个不同的亮度环境，从而能够提高后续对应关系的准确性以及适用范围。
示例性地，表1为本申请一实施例提供的不同亮度环境下训练对照组的示意图。参见表1，上述亮度环境由光源类型以及照度强度两个参量确定，表1示出了8种不同的光源类型，分别为A，D50，D65，D75，H,TL83，TL84，U30，而照度强度则包含有1000lux，700lux，300lux，100lux，20lux五种，因此，构建得到的亮度环境个数则为8*5＝40，并为上述不同的40种亮度环境配置对应数量的训练对照组，在本实施例中，每个亮度环境的训练对照组的个数为10个，从而得到训练样本总数为400组。
  1000lux 700lux 300lux 100lux 20lux
H 10 10 10 10 10
A 10 10 10 10 10
U30 10 10 10 10 10
TL83 10 10 10 10 10
TL84 10 10 10 10 10
D50 10 10 10 10 10
D65 10 10 10 10 10
D75 10 10 10 10 10
表1
在本实施例中，每个训练对照组包括至少一个基于RYB格式的第一训练图像以及基于RGB格式的第二训练图像，属于同一训练对照组内的第一训练图像以及第二训练图像均是基于相同的亮度环境下对同一物体同一角度进行拍摄得到的图像，其中，第一训练图像可以是通过RYB图像传感器采集得到的RYB图像，还可以是基于已有的RGB图像进行格式转换后得到的RYB图像；同样地，第二训练图像可以是通过RGB图像传感器采集得到的RGB图像，还可以是基于已有的RYB图像进行格式转换后得到的RGB图像。
在S902中，生成所述第一训练图像的第一训练补偿阵列，以及生成所述第二训练图像的第二训练补偿阵列。
在本实施例中，终端设备在获取了基于RYB格式的第一训练图像后，可以为第一训练图像配置第一训练补偿阵列。其中，生成第一训练补偿阵列的方式可以为用户手动配置或者通过预设的补偿算法计算各个像素点的补偿系数，从而生成与第一训练图像相对应的第一训练补偿阵列。
在一种可能的实现方式中，配置第一训练补偿阵列方式可以具体为：终端设备生成第一训练补偿阵列的第一训练亮度图层，并且终端设备可以配置有多个预设的候选增益补偿阵列，分别通过各个候选增益阵列对第一训练亮度图层进行调整，识别调整后的第一训练亮度图层内各个网格区域的亮度值之间的标准差，并选取标准差的数值最小的一个候选增益补偿阵列作为上述的第一训练补偿阵列。
同样地，对于第二训练图像生成第二训练补偿阵列的方式与第一训练补偿阵列的生成方式相同，可以参见上述描述，在此不再赘述。
在S903中，将所述第一训练补偿阵列作为反向传播算法网络的输出样本、将所述 第二训练补偿阵列作为所述反向传播算法网络的输入样本，对所述反向传播算法网络进行训练，得到补偿转换网络。
在本实施例中，上述补偿转换网络可以是基于反向传播算法(BP)网络进行搭建的。在该情况下，终端设备在获取了多个训练对照组后，可以通过多个训练对照组对反向传播算法网络进行训练学习。示例性地，图10示出了本申请一实施例提供的BP网络的训练示意图。参见图10，该BP网络包含有五个层级，第一层级以及第五层级包含有1271个网络节点，第二层级以及第四层级则包含有2000个网络节点，第三层级包含有3000个网络节点，不同的网络节点可以配置有对应的学习参数，通过调整上述的学习参数，以使BP网络能够适用于建立基准增益补偿阵列与目标增益补偿阵列的对应关系的场景下，其中，该BP网络的输入参量为基于RGB格式的第二训练补偿阵列，而输出参量为基于RYB格式的第一训练补偿阵列，并且在一次输入以及输出过程中，上述第一训练补偿阵列与第二训练补偿阵列均属于同一训练对照组，通过计算BP网络对应的损失率，确定BP网络算法是否已经调整完毕。若上述BP网络的损失率小于预设的损失阈值，则识别上述BP网络已经调整完毕，将调整后的BP网络识别为上述的补偿转换网络。
在S904中，将基于RGB格式的各个基准增益补偿阵列输入至所述补偿转换网络，确定各个所述基准增益补偿阵列对应的基于RYB格式的目标增益补偿阵列，生成所述补偿阵列对应关系。
在本实施例中，终端设备在生成了补偿转换网络后，可以通过补偿转换网络建立建立基准增益补偿阵列与目标增益补偿阵列之间的对应关系。其中，基于RGB格式的基准增益补偿阵列的个数以及基于RYB格式的目标增益补偿阵列的个数均是固定的，因此可以将各个预先配置好的基于RGB格式的基准增益补偿阵列导入到上述的补偿转换网络内，从已有的RYB格式的目标增益补偿阵列中，确定选取出关联的目标增益补偿阵列，并关联基准增益补偿阵列与关联的目标增益补偿阵列，根据所有建立的关联关系，生成上述的补偿阵列对应关系。
在本申请实施例中，通过创建多个训练对照组，通过多个训练对照组对预设的BP网络进行训练，生成补偿转换网络；通过生成的补偿转换网络确定各个基准增益补偿阵列关联的目标增益补偿阵列，从而能够提高后续补偿阵列对应关系建立的准确性，从而提高亮度调整的准确性。
图11示出了本申请第三实施例提供的一种图像处理的方法S301的具体实现流程图。参见图11，相对于图3所述实施例，本实施例提供的一种图像处理的方法中S301包括：S3011～S3013，具体详述如下：
进一步地，所述将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像，包括：
在S3011中，将RYB图像划分为多个网格区域，并根据各个所述网格区域内的像素点，分别确定各个所述网格区域的特征像素值。
在本实施例中，终端设备可以先对RYB图像进行降采样，即将包含多个像素点的RYB图像通过网格化后的降采样图像进行表示。具体地，终端设备可以配置有网格尺 寸，根据网格尺寸以及RYB图像的图像尺寸，确定每个网格区域对应的区域面积，举例性地，降采样前的RYB图像是2100*1800的图像，而预设的网格尺寸为3*3，则每个网格区域对应的区域尺寸为700*600，包含有420000个像素点；若降采样前的RYB图像是2100*2100的图像，而预设的网格尺寸为3*3，则每个网格区域对应的区域尺寸为700*700，包含有490000个像素点，因此，每个网格区域包含大小，与RYB图像的图像尺寸以及预设的网格尺寸相关。
在本实施例中，终端设备在将RYB图像划分为多个网格区域后，可以识别各个网格区域内包含的像素点的像素值，并基于上述所有像素值，确定网格区域对应的特征像素值，从而能够将多个像素点通过一个网格区域进行表示，实现了降采样的目的，减少了像素点的个数。在一种可能的实现方式中，确定特征像素值的方式可以为：终端设备将该网格区域的像素点的像素值的均值作为该网格区域的特征像素值；终端设备还可以将该网格区域的中心坐标点的像素值作为该网格区域的特征像素值。
在一种可能的实现方式中，终端设备可以根据网格区域内各个像素点与该网格区域对应的中心坐标点之间的距离值，确定各个像素点的加权权重，并计算各个像素点的像素值的加权平均值，将上述的加权平均值作为特征像素值。
在S3012中，基于各个所述网格区域的特征像素值，生成所述RYB图像的降采样图像。
在本实施例中，终端设备根据各个网格区域的特征像素值以及该网格区域所在的位置区域，将多个网格区域进行合并，生成RYB图像对应的降采样图像。图12示出了本申请一实施例提供的降采样图像的生成示意图，图12的(a)为降采样前的RYB图像，而图12的(b)为RYB图像的降采样图像。参见图12，终端设备可以将包含多个像素点的图像，降采样为一网格图像，从而能够减少图像尺寸，提高处理速率。
在S3013中，通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
在本实施例中，由于降采样图像是基于RYB格式生成的，即将降采样图像包含的三个通道分别为R通道、Y通道以及B通道。终端设备需要将RYB格式的降采样图像转换为基于RGB格式的网格图像。因此，终端设备可以通过预设的RGB转换算法，生成与降采样图像对应的RGB格式的网格图像。
在一种可能的实现方式中，上述RGB转换算法具体可以为：终端设备可以采用以下算法计算得到RYB图像内各个像素点在转换为RGB图像中G通道的像素值：
G＝(2Y-R)，其中，G表示转换为G通道的像素值；Y表示转换前Y通道的像素值；R表示转换前R通道的像素值。
在本申请实施例中，通过先对RYB图像进行降采样，得到降采样图像，在进行RGB格式转换，生成网格图像，能够减少格式转换的计算量，从而提高了运算效率。
图13示出了本申请第四实施例提供的一种图像处理的方法S3013的具体实现流程图。参见图13，相对于图11所述实施例，本实施例提供的一种图像处理的方法中S3013包括：S1301～S1303，具体详述如下：
进一步地，将所述降采样图像转换为基于RGB格式的所述网格图像，包括：
在S1301中，基于所述RYB图像内各个像素点的像素值，确定采集所述RYB图像时的光源类型。
在本实施例中，终端设备可以根据RYB图像内各个像素点的像素值，生成RYB图像的亮度图层，并根据该亮度图层内各个像素点的亮度值，识别得到拍摄RYB图像时的光源类型。在一种可能的实现方式中，终端设备可以根据RYB图像的中心坐标点的亮度值，与不同的候选光源进行匹配，基于匹配结果确定该RYB图像的光源类型。其中，光源类型包括但不限于：A，D50，D65，D75，H,TL83，TL84，U30等八种光源类型。可选地，上述光源类型还可以包含光照强度，即上述的光源类型可以通过(A，200lux)的方式表示。
在S1302中，选取与所述光源类型匹配的所述RGB转换算法。
在本实施例中，终端设备可以为不同的光源类型配置对应的RGB算法，终端设备在确定了拍摄RYB图像时对应的光源类型后，可以从RGB转换算法库内选取与该光源类型对应的RGB转换算法。其中，上述的RGB转换算法具体为由RYB格式转换为RGB格式的转换算法。
在一种可能的实现方式中，上述RGB转换算法可以为一转换矩阵。表2示出了本申请一实施例提供的RGB转换算法的索引表。参见表2，该索引表给出了八种不同光源类型对应的转换矩阵，分别为A，D50，D65，D75，H,TL83，CWF，U30。由于需要将RYB图像转换为RGB图像，上述两个图像均为包含三个通道的图像，因此对应的转换矩阵也为一3*3的矩阵。例如，对于U30的光源类型，则对应的转换矩阵为：
Figure PCTCN2020125611-appb-000001
D75 984 11 29 -300 1261 63 -29 43 1010
D65 984 11 29 -300 1261 63 -29 43 1010
D50 984 11 29 -300 1261 63 -29 43 1010
CWF 991 -18 51 -271 1217 78 -26 6 1044
TL84 978 11 35 -370 1343 51 -49 42 1031
U30 1049 46 -71 -461 1399 86 -136 57 1103
A 1092 7 -75 -609 1525 108 -240 19 1245
H 1092 7 -75 -609 1525 108 -240 19 1245
表2
在S1303中，通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
在本实施例中，终端设备在获取了与RYB图像对应光源类型匹配的RGB转换算法后，可以通过上述RGB转换算法，将降采样图像转换为基于RGB格式的网格图像。在一种可能的实现方式中，若上述RGB算法具体为一转换矩阵，则可以将降采样图像对应的图像矩阵通过右乘转换矩阵的方式，得到基于RGB格式的网格图像的图像矩阵。示例性地，上述识别的的光源类型为U30，因此选取得到的RGB转换矩阵如上所述，将RYB格式的降采样图像对应的图像阵列与U30对应的转换矩阵进行相乘，得到基于RGB格式的网格图像，计算方式如下：
Figure PCTCN2020125611-appb-000002
其中，[R,Y,B]为降采样图像对应的图像阵列；[R,G,B]为网格图像对应的图像阵列。
在本申请实施例中，通过识别拍摄RYB图像时的光源类型，选取与光源类型对应的RGB转换算法，从而将降采样图像转换为网格图像，提高了网格图像的转换准确性，提高了后续操作的校正效果。
图14示出了本申请第五实施例提供的一种图像处理的方法S304的具体实现流程图。参见图14，相对于图3所述实施例，本实施例提供的一种图像处理的方法中S304包括：S3041～S3043，具体详述如下：
进一步地，所述通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像，包括：
在S3041中，获取所述RYB图像的图像尺寸。
在本实施例中，由于生成目标增益补偿阵列是采用的是基于降采样后的RGB格式的网格图像，因此目标增益补偿阵列的阵列尺寸与原始的RYB的尺寸不一致，因此需要将目标增益补偿阵列扩展至原始图像的大小，基于此，终端设备可以获取RYB图像的图像尺寸。该图像尺寸可以像素点的个数确定，还可以以图像分辨率以及图像长度的方式进行表示。
在S3042中，通过双线性插值法将所述目标增益补偿阵列扩展至与所述图像尺寸相同的扩展增益补偿阵列。
在本实施例中，终端设备在确定了图像尺寸以及目标增益补偿阵列的阵列大小后，可以确定缩放比例，并基于该缩放比例调整双线性插值算法，并通过调整后的双线性插值算法以及目标增益补偿阵列各个元素的增益值，生成与图像尺寸相匹配的扩展增益补偿阵列。其中，该双线性插值法具体可以为：
Figure PCTCN2020125611-appb-000003
其中，f(Q ij)为目标增益补偿阵列中坐标(x i,y j)对应的增益补偿系数；x 1以及x 2为扩展增益补偿阵列中最接近任意坐标(x，y)的两个横坐标；y 1以及y 2为扩展增益补偿阵列中最接近任意坐标(x，y)的两个纵坐标，f(x,y)为扩展增益补偿阵列中任意坐标(x，y)增益补偿数值。
在S3043中，通过所述扩展增益补偿阵列内各个补偿系数，对所述第二亮度图层内各个亮度值进行调整，得到所述校正图像。
在本实施例中，由于扩展增益补偿阵列的阵列尺寸与RYB图像的图像尺寸一致，因此可以通过扩展增益补偿阵列内各个增益补偿数值，对RYB图像的第二亮度图层内各个亮度值进行调整，并根据调整后的第二亮度图层生成校正图像。
在一种可能的实现方式中，生成上述校正图像的方式具体可以为：RYB图像对应的第二亮度图层具体包括：归一化后的R/Y图层以及归一化后的B/Y图层，因此上述 获取得到的扩展增益补偿阵列也包括有用于调整R/Y图层的第一增益补偿阵列以及用于调整B/Y图层的第二增益补偿阵列，通过第一增益补偿阵列对R/Y图层内的各个像素值进行调整，并将调整后的R/Y图层进行逆归一化操作，从而得到调整后的R图层以及第一Y图层，同样地，通过上述方式可以得到调整后的B图层以及第二Y图层，计算两个Y图层之间的均值，合并后一个调整后的Y图层，从而将调整后的R图层、Y图层以及B图层生成上述的校正图像。
在本申请实施例中，通过将目标增益补偿阵列扩展为与图像尺寸匹配的扩展增益补偿阵列，从而实现对RYB图像内的各个像素点进行调整，提高了校正准确性。
图15示出了本申请第六实施例提供的一种图像处理的方法S302的具体实现流程图。参见图15，相对于图3、图9、图11、图13以及图14任一所述实施例，本实施例提供的一种图像处理的方法S303包括：S1501～S1505，具体详述如下：
进一步地，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
在S1501中，获取所述网格图像的中心网格区域的像素值。
在本实施例中，终端设备可以将归一化后的R/G图层以及归一化后的B/G图层作为网格图像的第一亮度图层。其中，生成归一化的R/G图层以及归一化的B/G图层之前需要确定基准值，因此需要确定网格图像的中心网格区域的像素值，并将中心网格区域的像素值作为基准值。由于中心区域的透射率较高，即失真率较低，与实际拍摄光源的情况相匹配，因此可以将中心区域的像素值作为基准值对整体图像进行归一化操作。
在S1502中，根据所述中心网格区域的像素值，对所述网格图像内的各个特征像素值进行归一化处理，得到各个所述网格区域的归一像素值。
在本实施例中，终端设备可以将中心网格区的像素值作为基准值，对网格区域内各个特征像素值进行归一化处理，从而确定了各个网格区域的归一像素值。例如，中心网格区域的像素值为(R 0,G 0,B 0)，而任一网格区域对应的特征像素值为(R,G,B)，则归一像素值为
Figure PCTCN2020125611-appb-000004
在S1503中，根据所有所述网格区域的所述归一像素值的R通道数值与G通道数值之比，生成所述网格图像对应的R/G图层。
在本实施例中，终端设备可以计算各个网格区域所对应的归一像素值的R通道数值与G通道数值之比，从而得到关于该网格区域的亮度值，即
Figure PCTCN2020125611-appb-000005
并根据各个网格区域对应的亮度值，生成R/G图层。
在S1504中，根据所有所述网格区域的所述归一像素值的B通道数值与G通道数值之比，生成所述网格图像对应的B/G图层。
在本实施例中，终端设备可以计算各个网格区域所对应的归一像素值的B通道数值与G通道数值之比，从而得到关于该网格区域的亮度值，即
Figure PCTCN2020125611-appb-000006
并根据各个网格区域对应的亮度值，生成B/G图层。
示例性地，图16示出了本申请一实施例提供的R/G图层与B/G图层的生成示意 图。参见图16所示，一RGB格式的网格图像的在RGB三个通道对应的三维曲面分别如下，经过归一化处理并进行归一化像素相除后，得到R/G图层与B/G图层。
在S1505中，将所述R/G图层以及所述B/G图层识别为所述第一亮度图层。
在本实施例中，将上述两个图层统称为第一亮度图层。
在本申请实施例中，通过对网格图像归一化，生成归一化后的R/G图层以及归一化后的B/G图层，并将上述两个图层作为网格图像的第一亮度图层，能够方便后续对不同通道的像素值进行调整，提高了调整的效率以及准确性。
图17示出了本申请第七实施例提供的一种图像处理的方法S302的具体实现流程图。参见图17，相对于图3、图9、图11、图13以及图14任一所述实施例，本实施例提供的一种图像处理的方法在S302包括：S1701～S1703，具体详述如下：
进一步地，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
在S1701中，分别通过增益补偿集合内各个候选增益补偿阵列，对所述第一亮度图层进行调整，获得各个所述候选增益补偿阵列对应的亮度校准图层。
在本实施例中，对亮度调整的增益补偿阵列的个数可以是固定的，在该情况下，可以将所有候选增益补偿阵列存储于一增益补偿集合内，终端设备在每次调整过程中，可以从增益补偿集合中提取各个候选增益补偿阵列对亮度图层进行调整，得到上述的亮度校准图层。其中，若第一亮度图层采用第五实施例的方式，即包含有归一化后的R/G图层以及归一化后的B/G图层，则可以分别生成上述两个图层对应的亮度校准图层，并分别为上述两个图层配置对应的基准增益补偿阵列。
在S1702中，根据所述亮度校准图层内各个像素点的亮度值，分别确定各个所述候选增益补偿阵列的平滑系数。
在本实施例中，终端设备可以根据亮度校准图层内各个像素点的亮度值，计算该候选增益补偿阵列对应的平滑系数，该平滑系数可以基于各个亮度值的标准差或者均方差确定，还可以生成亮度校准图层对应的三维曲面，并获取三维曲面的离散系数，基于离散系数确定平滑系数，其中，离散程度越高，则平滑系数的数值越小。
在一种可能的实现方式中，终端设备在每通过一个候选增益补偿阵列对第一亮度图层进行调整，生成亮度校准图层后，可以计算该亮度校准图层的平滑系数，并在检测到该平滑系数大于预设的平滑阈值，则识别该候选增益补偿阵列为基准增益补偿阵列，无需继续通过其他候选增益补偿阵列对第一亮度图层进行调整；反之，若平滑系数小于或等于预设的平滑阈值，则继续从增益补偿集合中提取其他候选增益补偿阵列对第一亮度图层进行调整，直到检测到平滑系数大于预设的平滑阈值或计算得到所有候选增益补偿阵列对应的平滑系数。
在S1703中，选取所述平滑系数最大的候选增益补偿阵列作为所述基准增益补偿阵列。
在本实施例中，终端设备可以选取平滑系数的数值最大的候选增益补偿阵列作为基准增益补偿阵列，由于平滑系数越大，则表示调整后的亮度校准图层的亮度值之间的差异越小，对应的三维曲面趋近一个平面，颜色阴影程度较低，因此可以将对应的候选增益补偿阵列作为基准增益补偿阵列。
在本申请实施例中，通过计算各个候选增益补偿阵列对应的平滑系数，能够选取出平滑效果最好的基准补偿阵列，提高了增益补偿阵列的选取准确性。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
对应于上文实施例所述的图像处理的方法，图17示出了本申请实施例提供的图像处理的装置的结构框图，为了便于说明，仅示出了与本申请实施例相关的部分。
参照图18，该图像处理的装置包括：
网格图像转换单元181，用于将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；
基准增益补偿阵列确定单元183，用于生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列；
目标增益补偿阵列获取单元183，用于基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；
图像校准单元184，用于通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
可选地，所述图像处理的装置还包括：
训练对照组获取单元，用于获取多个不同亮度环境下的训练对照组；每个所述训练对照组包括至少一个基于RYB格式的第一训练图像以及基于RGB格式的第二训练图像；
训练补偿阵列生成单元，用于生成所述第一训练图像的第一训练补偿阵列，以及生成所述第二训练图像的第二训练补偿阵列；
训练学习单元，用于将所述第一训练补偿阵列作为反向传播算法网络的输出样本、将所述第二训练补偿阵列作为所述反向传播算法网络的输入样本，对所述反向传播算法网络进行训练，得到补偿转换网络；
补偿阵列对应关系建立单元，用于将基于RGB格式的各个基准增益补偿阵列输入至所述补偿转换网络，确定各个所述基准增益补偿阵列对应的基于RYB格式的目标增益补偿阵列，生成所述补偿阵列对应关系。
可选地，所述网格图像转换单元181包括：
特征像素值确定单元，用于将RYB图像划分为多个网格区域，并根据各个所述网格区域内的像素点，分别确定各个所述网格区域的特征像素值；
降采样图像生成单元，用于基于各个所述网格区域的特征像素值，生成所述RYB图像的降采样图像；
降采样图像转换单元，用于通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
可选地，所述降采样图像转换单元包括：
光源类型确定单元，用于基于所述RYB图像内各个像素点的像素值，确定采集所述RYB图像时的光源类型；
RGB转换算法选取单元，用于选取与所述光源类型匹配的所述RGB转换算法；
RGB转换算法调整单元，用于通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
可选地，所述图像校准单元183包括：
图像尺寸获取单元，用于获取所述RYB图像的图像尺寸；
增益补偿阵列扩展单元，用于通过双线性插值法将所述目标增益补偿阵列扩展至与所述图像尺寸相同的扩展增益补偿阵列；
校正图像生成单元，用于通过所述扩展增益补偿阵列内各个补偿系数，对所述第二亮度图层内各个亮度值进行调整，得到所述校正图像。
可选地，所述基准增益补偿阵列确定单元182包括：
中心像素值获取单元，用于获取所述网格图像的中心网格区域的像素值；
归一像素值确定单元，用于根据所述中心网格区域的像素值，对所述网格图像内的各个特征像素值进行归一化处理，得到各个所述网格区域的归一像素值；
R/G图层生成单元，用于根据所有所述网格区域的所述归一像素值的R通道数值与G通道数值之比，生成所述网格图像对应的R/G图层；
B/G图层生成单元，用于根据所有所述网格区域的所述归一像素值的B通道数值与G通道数值之比，生成所述网格图像对应的B/G图层；
第一亮度图层生成单元，用于将所述R/G图层以及所述B/G图层识别为所述第一亮度图层。
可选地，所述基准增益补偿阵列确定单元182包括：
亮度校准图层生成单元，用于分别通过增益补偿集合内各个候选增益补偿阵列，对所述第一亮度图层进行调整，获得各个所述候选增益补偿阵列对应的亮度校准图层；
平滑系数计算单元，用于根据所述亮度校准图层内各个像素点的亮度值，分别确定各个所述候选增益补偿阵列的平滑系数；
候选增益补偿阵列选取单元，用于选取所述平滑系数最大的候选增益补偿阵列作为所述基准增益补偿阵列。
因此，本申请实施例提供的图像处理的装置同样可以通过将RYB图像转换到对不同光源环境的离散表现更好的RGB格式的网格图像，并生成与网格图像对应的第一增益补偿阵列，继而通过预设的补偿阵列对应关系，生成了基于RYB格式的第二增益补偿阵列，并通过第二增益补偿阵列对RYB图像进行调整，生成校正图像，在避免颜色失真的同时，通过第二增益补偿阵列对RYB图像的亮度图层进行调整，消除了颜色阴影，提高了成像效果。
作为示例而非限定，图19示出了本申请一实施例提供的图像校正的示意图，其中，图19的(a)为调整前的RYB图像的亮度图层，而图19的(b)为调整前的RYB图像的亮度图层对应的R/G图层的像素分布图，其中，横坐标为像素点列坐标，纵坐标为归一化后的R/G值；图19的(c)为根据目标增益补偿阵列调整后的RYB图像的亮度图层，而图19的(d)为调整后的RYB图像的亮度图层对应的R/G图层的像素分布图，其中，横坐标为像素点列坐标，纵坐标为归一化后的R/G值。通过图19可以确定，通过本申请提供的实施例对RYB图像的亮度图层进行调整后，能够有效地降低颜色阴影，从而提高了画像的质量，同时解决了图像的颜色偏移以及颜色阴影的问 题。
图20为本申请一实施例提供的终端设备的结构示意图。如图20所示，该实施例的终端设备20包括：至少一个处理器200(图20中仅示出一个)处理器、存储器201以及存储在所述存储器201中并可在所述至少一个处理器200上运行的计算机程序202，所述处理器200执行所述计算机程序202时实现上述任意各个图像处理的方法实施例中的步骤。
所述终端设备20可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。该终端设备可包括，但不仅限于，处理器200、存储器201。本领域技术人员可以理解，图20仅仅是终端设备20的举例，并不构成对终端设备20的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如还可以包括输入输出设备、网络接入设备等。
所称处理器200可以是中央处理单元(Central Processing Unit，CPU)，该处理器200还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器201在一些实施例中可以是所述终端设备20的内部存储单元，例如终端设备20的硬盘或内存。所述存储器201在另一些实施例中也可以是所述终端设备20的外部存储设备，例如所述终端设备20上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器201还可以既包括所述终端设备20的内部存储单元也包括外部存储设备。所述存储器201用于存储操作系统、应用程序、引导装载程序(BootLoader)、数据以及其他程序等，例如所述计算机程序的程序代码等。所述存储器201还可以用于暂时地存储已经输出或者将要输出的数据。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
本申请实施例还提供了一种网络设备，该网络设备包括：至少一个处理器、存储器以及存储在所述存储器中并可在所述至少一个处理器上运行的计算机程序，所述处 理器执行所述计算机程序时实现上述任意各个方法实施例中的步骤。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现可实现上述各个方法实施例中的步骤。
本申请实施例提供了一种计算机程序产品，当计算机程序产品在移动终端上运行时，使得移动终端执行时实现可实现上述各个方法实施例中的步骤。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质至少可以包括：能够将计算机程序代码携带到拍照装置/终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区，根据立法和专利实践，计算机可读介质不可以是电载波信号和电信信号。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/网络设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/网络设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (10)

  1. 一种图像处理的方法，其特征在于，包括：
    将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；
    生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列；
    基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；
    通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
  2. 根据权利要求1所述的方法，其特征在于，在所述基于预设的补偿阵列对应关系，获得与所述第一增益补偿阵列关联的基于RYB格式的第二增益补偿阵列之前，还包括：
    获取多个不同亮度环境下的训练对照组；每个所述训练对照组包括至少一个基于RYB格式的第一训练图像以及基于RGB格式的第二训练图像；
    生成所述第一训练图像的第一训练补偿阵列，以及生成所述第二训练图像的第二训练补偿阵列；
    将所述第一训练补偿阵列作为反向传播算法网络的输出样本、将所述第二训练补偿阵列作为所述反向传播算法网络的输入样本，对所述反向传播算法网络进行训练，得到补偿转换网络；
    将基于RGB格式的各个基准增益补偿阵列输入至所述补偿转换网络，确定各个所述基准增益补偿阵列对应的基于RYB格式的目标增益补偿阵列，生成所述补偿阵列对应关系。
  3. 根据权利要求1所述的方法，其特征在于，所述将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像，包括：
    将RYB图像划分为多个网格区域，并根据各个所述网格区域内的像素点，分别确定各个所述网格区域的特征像素值；
    基于各个所述网格区域的特征像素值，生成所述RYB图像的降采样图像；
    通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
  4. 根据权利要求3所述的方法，所述通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像，包括：
    基于所述RYB图像内各个像素点的像素值，确定采集所述RYB图像时的光源类型；
    选取与所述光源类型匹配的所述RGB转换算法；
    通过RGB转换算法，将所述降采样图像转换为基于RGB格式的所述网格图像。
  5. 根据权利要求1所述的方法，其特征在于，所述通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像，包括：
    获取所述RYB图像的图像尺寸；
    通过双线性插值法将所述目标增益补偿阵列扩展至与所述图像尺寸相同的扩展增益补偿阵列；
    通过所述扩展增益补偿阵列内各个补偿系数，对所述第二亮度图层内各个亮度值进行调整，得到所述校正图像。
  6. 根据权利要求1-5任一所述的方法，其特征在于，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
    获取所述网格图像的中心网格区域的像素值；
    根据所述中心网格区域的像素值，对所述网格图像内的各个特征像素值进行归一化处理，得到各个所述网格区域的归一像素值；
    根据所有所述网格区域的所述归一像素值的R通道数值与G通道数值之比，生成所述网格图像对应的R/G图层；
    根据所有所述网格区域的所述归一像素值的B通道数值与G通道数值之比，生成所述网格图像对应的B/G图层；
    将所述R/G图层以及所述B/G图层识别为所述第一亮度图层。
  7. 根据权利要求1-5任一所述的方法，其特征在于，所述生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列，包括：
    分别通过增益补偿集合内各个候选增益补偿阵列，对所述第一亮度图层进行调整，获得各个所述候选增益补偿阵列对应的亮度校准图层；
    根据所述亮度校准图层内各个像素点的亮度值，分别确定各个所述候选增益补偿阵列的平滑系数；
    选取所述平滑系数最大的候选增益补偿阵列作为所述基准增益补偿阵列。
  8. 一种图像处理的装置，其特征在于，包括：
    网格图像转换单元，用于将红黄蓝RYB图像转换为基于红绿蓝RGB格式的网格图像；
    基准增益补偿阵列确定单元，用于生成所述网格图像的第一亮度图层，并确定用于调整所述第一亮度图层的基准增益补偿阵列；
    目标增益补偿阵列获取单元，用于基于预设的补偿阵列对应关系，获得与所述基准增益补偿阵列关联的基于RYB格式的目标增益补偿阵列；
    图像校准单元，用于通过所述目标增益补偿阵列对所述RYB图像的第二亮度图层进行调整，生成校正图像。
  9. 一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述的方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的方法。
PCT/CN2020/125611 2020-01-15 2020-10-31 一种图像处理的方法、装置、终端以及存储介质 WO2021143284A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/793,211 US20230079582A1 (en) 2020-01-15 2020-10-31 Image processing method and apparatus, terminal, and storage medium
EP20913573.0A EP4072131A4 (en) 2020-01-15 2020-10-31 IMAGE PROCESSING METHOD AND DEVICE, TERMINAL DEVICE AND STORAGE MEDIUM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010044246.4A CN113132704B (zh) 2020-01-15 2020-01-15 一种图像处理的方法、装置、终端以及存储介质
CN202010044246.4 2020-01-15

Publications (1)

Publication Number Publication Date
WO2021143284A1 true WO2021143284A1 (zh) 2021-07-22

Family

ID=76771970

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/125611 WO2021143284A1 (zh) 2020-01-15 2020-10-31 一种图像处理的方法、装置、终端以及存储介质

Country Status (4)

Country Link
US (1) US20230079582A1 (zh)
EP (1) EP4072131A4 (zh)
CN (1) CN113132704B (zh)
WO (1) WO2021143284A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128759A (zh) * 2023-02-08 2023-05-16 爱芯元智半导体(上海)有限公司 图像的光照补偿方法及其装置
CN117516726A (zh) * 2023-12-29 2024-02-06 深圳市英博伟业科技有限公司 一种基于红外技术的温度测量方法及终端设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11869395B2 (en) * 2020-11-04 2024-01-09 POSTECH Research and Business Development Foundation Color calibration display apparatus, color calibration display method, and switchable display system for providing virtual reality or augmented reality using color calibration display apparatus
CN116152361B (zh) * 2023-04-20 2023-07-07 高视科技(苏州)股份有限公司 用于估算色度的方法、电子设备及计算机可读存储介质
CN116664550B (zh) * 2023-07-10 2024-04-12 广州医科大学附属第一医院（广州呼吸中心） 肺癌组织免疫组化pd-l1病理切片的智能识别方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103578084A (zh) * 2013-12-09 2014-02-12 西安电子科技大学 基于亮通道滤波的彩色图像增强方法
US20180122103A1 (en) * 2016-10-27 2018-05-03 Adobe Systems Incorporated Providing predictable and customizable electronic color mixing
CN108921172A (zh) * 2018-05-31 2018-11-30 清华大学 基于支持向量机的图像处理装置及方法
CN109166176A (zh) * 2018-08-23 2019-01-08 百度在线网络技术（北京）有限公司 三维人脸图像的生成方法与装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014153098A1 (en) * 2013-03-14 2014-09-25 Pelican Imaging Corporation Photmetric normalization in array cameras
US9367916B1 (en) * 2014-12-10 2016-06-14 Intel Corporation Method and system of run-time self-calibrating lens shading correction
GB201516173D0 (en) * 2015-09-14 2015-10-28 Apical Ltd Adaptive shading correction
US10542243B2 (en) * 2018-04-10 2020-01-21 Intel Corporation Method and system of light source estimation for image processing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103578084A (zh) * 2013-12-09 2014-02-12 西安电子科技大学 基于亮通道滤波的彩色图像增强方法
US20180122103A1 (en) * 2016-10-27 2018-05-03 Adobe Systems Incorporated Providing predictable and customizable electronic color mixing
CN108921172A (zh) * 2018-05-31 2018-11-30 清华大学 基于支持向量机的图像处理装置及方法
CN109166176A (zh) * 2018-08-23 2019-01-08 百度在线网络技术（北京）有限公司 三维人脸图像的生成方法与装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4072131A4

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128759A (zh) * 2023-02-08 2023-05-16 爱芯元智半导体(上海)有限公司 图像的光照补偿方法及其装置
CN116128759B (zh) * 2023-02-08 2024-01-09 爱芯元智半导体(上海)有限公司 图像的光照补偿方法及其装置
CN117516726A (zh) * 2023-12-29 2024-02-06 深圳市英博伟业科技有限公司 一种基于红外技术的温度测量方法及终端设备
CN117516726B (zh) * 2023-12-29 2024-03-15 深圳市英博伟业科技有限公司 一种基于红外技术的温度测量方法及终端设备

Also Published As

Publication number Publication date
EP4072131A4 (en) 2023-02-01
EP4072131A1 (en) 2022-10-12
US20230079582A1 (en) 2023-03-16
CN113132704B (zh) 2022-08-19
CN113132704A (zh) 2021-07-16

Similar Documents

Publication Publication Date Title
WO2021143284A1 (zh) 一种图像处理的方法、装置、终端以及存储介质
JP6967160B2 (ja) 画像処理方法および関連デバイス
CN107172364B (zh) 一种图像曝光补偿方法、装置和计算机可读存储介质
CN109191410B (zh) 一种人脸图像融合方法、装置及存储介质
CN111932664A (zh) 图像渲染方法、装置、电子设备及存储介质
WO2022042290A1 (zh) 一种虚拟模型处理方法、装置、电子设备和存储介质
WO2021143281A1 (zh) 颜色阴影校正方法、终端设备及计算机可读存储介质
WO2020259412A1 (zh) 展示资源的方法、装置、设备及存储介质
WO2021098609A1 (zh) 图像检测方法、装置及电子设备
CN108200352A (zh) 一种调解图片亮度的方法、终端及存储介质
US11032529B2 (en) Selectively applying color to an image
WO2023005870A1 (zh) 一种图像处理方法及相关设备
CN114449243B (zh) 一种白平衡方法和终端设备
CN109348137A (zh) 移动终端拍照控制方法、装置、移动终端以及存储介质
CN113157357A (zh) 一种页面显示的方法、装置、终端以及存储介质
WO2022095640A1 (zh) 对图像中的树状组织进行重建的方法、设备及存储介质
EP3249999B1 (en) Intelligent matching method for filter and terminal
WO2021103919A1 (zh) 构图推荐方法和电子设备
CN110097570B (zh) 一种图像处理方法和装置
WO2023108444A1 (zh) 图像处理方法、智能终端及存储介质
CN109685802A (zh) 一种低延迟的视频分割实时预览方法
CN110807411B (zh) 一种月亮识别方法及电子设备
CN114333001A (zh) 图像处理方法、智能终端及存储介质
CN107194363B (zh) 图像饱和度处理方法、装置、存储介质及计算机设备
CN110503618A (zh) 图像处理方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20913573

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020913573

Country of ref document: EP

Effective date: 20220707

NENP Non-entry into the national phase

Ref country code: DE