WO2023098251A1 - 图像处理方法、设备及可读存储介质 - Google Patents

图像处理方法、设备及可读存储介质 Download PDF

Info

Publication number
WO2023098251A1
WO2023098251A1 PCT/CN2022/120792 CN2022120792W WO2023098251A1 WO 2023098251 A1 WO2023098251 A1 WO 2023098251A1 CN 2022120792 W CN2022120792 W CN 2022120792W WO 2023098251 A1 WO2023098251 A1 WO 2023098251A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
brightness
processed
data
brightness correction
Prior art date
Application number
PCT/CN2022/120792
Other languages
English (en)
French (fr)
Inventor
洪国伟
董治
姜涛
丁嘉文
Original Assignee
腾讯音乐娱乐科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯音乐娱乐科技（深圳）有限公司 filed Critical 腾讯音乐娱乐科技（深圳）有限公司
Publication of WO2023098251A1 publication Critical patent/WO2023098251A1/zh

Links

Images

Classifications

    • G06T5/92—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • the present application relates to the technical field of image processing, in particular to an image processing method, electronic equipment, and a computer-readable storage medium.
  • Webcasting is a method of setting up an independent signal acquisition device on the spot to collect signals (audio and/or video) and import them into the broadcasting terminal (directing equipment or platform), and the broadcasting terminal uploads the signal to the server through the network, and then publishes it to a designated website for Technology that people watch.
  • the collected video streams are usually processed to increase their brightness.
  • the uniform and fixed brightness enhancement process is directly performed on the entire interface of the video stream, that is, the same brightness is increased on all parts of all pictures generated during the live broadcast.
  • the dimness of the scene may change, that is, from dim to bright, and a uniform and fixed brightness enhancement process will cause the image after brightness enhancement to be too bright, affecting the user's visual experience.
  • the purpose of the present application is to provide an image processing method, an electronic device, and a computer-readable storage medium, so that users can have a better visual experience.
  • the present application provides an image processing method, including:
  • the image to be processed is in RGB format
  • the processed image is determined as the brightness-optimized image corresponding to the image to be processed.
  • the obtaining the image to be processed from the video stream includes:
  • using the Y channel data to generate brightness correction parameters corresponding to each pixel of the first image includes:
  • using the Y channel data to generate brightness correction parameters corresponding to each pixel of the first image includes:
  • the brightness correction parameters to perform gamma brightness correction on the Y channel data corresponding to each pixel to obtain corrected data, including:
  • the normalized gamma is calculated to obtain the corrected data, including:
  • the power value is inversely proportional to the brightness difference, and the power value is greater than zero;
  • the evaluation parameter is within the target interval, it includes:
  • the new processed image is used to obtain new evaluation parameters, until the new evaluation parameters are not in the target interval, and the brightness optimized image is obtained.
  • the updating the brightness correction parameters includes:
  • Acquire user feedback information for responding to the processed image generate training data according to the user feedback information, and use the training data to perform additional training on the evaluation model.
  • the present application also provides an electronic device, including a memory and a processor, wherein:
  • the memory is used to store computer programs
  • the processor is configured to execute the computer program to implement the above image processing method.
  • the present application also provides a computer-readable storage medium for storing a computer program, wherein the above-mentioned image processing method is implemented when the computer program is executed by a processor.
  • the image processing method provided by the application obtains the image to be processed from the video stream; the image to be processed is in RGB format; the image to be processed is converted to the YUV format to obtain the first image, and extract the Y channel data corresponding to the first image; Y channel data, generating brightness correction parameters corresponding to each pixel of the first image; using the brightness correction parameters to perform gamma brightness correction on the Y channel data corresponding to each pixel respectively, to obtain corrected data; using the corrected data to replace the Y channel data, Obtain the second image, and convert the second image to RGB format to obtain the processed image; input the processed image and the optimized image of the adjacent historical frame into the evaluation model to obtain the evaluation parameters; determine that the evaluation parameters are not in the target interval, then process The post-image is determined as a brightness-optimized image corresponding to the image to be processed.
  • the image to be processed is acquired by this method, it is converted into YUV format, where the Y channel data is used to represent the brightness of the image.
  • the influence of brightness correction on the image color can be reduced as much as possible.
  • the Y-channel data corresponding to each pixel in the image to be processed the overall brightness of the image to be processed can be determined, and then the basis for brightness adjustment, that is, brightness correction parameters, can be determined.
  • the brightness correction parameters based on the specific situation of the Y channel data, the gamma brightness correction can be performed on the Y channel data, and the dark part of the image to be processed can be improved to a large extent, while the bright part can be slightly improved.
  • the enhancement or non-enhancement makes the image clear and natural.
  • the obtained correction data is used to generate the second image and then restored to the RGB format to obtain the processed image.
  • the evaluation model is trained based on the user's needs and preferences for image brightness, and it has good spatial and temporal adaptive capabilities. The evaluation model can evaluate whether the brightness change response of the processed image and the optimized image of the adjacent frames in the time series is good or not.
  • the target interval is used to indicate that the processed image has an abnormal brightness change based on the optimized image of the adjacent frame in history, so if it is not in the target interval, it means that the brightness optimization of the image to be processed is reasonable, so set
  • the processed image is determined to be a brightness optimized image after brightness optimization, and it is determined that the optimization process of the image to be processed ends.
  • FIG. 1 is a schematic diagram of a hardware composition framework applicable to an image processing method provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a hardware composition framework applicable to another image processing method provided by the embodiment of the present application;
  • FIG. 3 is a schematic flow diagram of an image processing method provided in an embodiment of the present application.
  • FIG. 4 is an image to be processed provided by the embodiment of the present application.
  • FIG. 5 is a processed image obtained by processing according to a related processing method provided by the embodiment of the present application.
  • FIG. 6 is a schematic flow chart of converting RGB format to YUV format according to the embodiment of the present application.
  • FIG. 7 is a schematic flow chart of a specific image processing method provided in the embodiment of the present application.
  • Fig. 8 is a specific comparison diagram of before and after image processing provided by the embodiment of the present application.
  • FIG. 9 is another specific comparison diagram of before and after image processing provided by the embodiment of the present application.
  • FIG. 10 is a schematic diagram of an optimized video stream effect provided by an embodiment of the present application.
  • FIG. 1 is a schematic diagram of a hardware composition framework applicable to an image processing method provided in an embodiment of the present application.
  • the electronic device 100 may include a processor 101 and a memory 102 , and may further include one or more of a multimedia component 103 , an information input/information output (I/O) interface 104 and a communication component 105 .
  • I/O information input/information output
  • the processor 101 is used to control the overall operation of the electronic device 100, so as to complete the image processing method, and/or, all or part of the steps in the audio processing method;
  • the memory 102 is used to store various types of data to support electronic devices. 100 , these data may include, for example, instructions for any application or method operating on the electronic device 100 , as well as application-related data.
  • the memory 102 can be realized by any type of volatile or non-volatile storage device or their combination, such as Static Random Access Memory (Static Random Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read-Only Memory, One or more of Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • Static Random Access Memory Static Random Access Memory
  • SRAM Static Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • Read-Only Memory One or more of Only Memory, ROM
  • ROM Read-Only Memory
  • the image to be processed is in RGB format
  • the processed image is determined as the brightness optimized image corresponding to the image to be processed.
  • Multimedia components 103 may include screen and audio components.
  • the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals.
  • an audio component may include a microphone for receiving external audio signals.
  • the received audio signal may be further stored in the memory 102 or sent via the communication component 105 .
  • the audio component also includes at least one speaker for outputting audio signals.
  • the I/O interface 104 provides an interface between the processor 101 and other interface modules, which may be a keyboard, a mouse, buttons, and the like. These buttons can be virtual buttons or physical buttons.
  • the communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices.
  • Wireless communication such as Wi-Fi, Bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 105 may include: Wi-Fi parts, Bluetooth parts, NFC parts.
  • the electronic device 100 may be implemented by one or more Application Specific Integrated Circuit (ASIC for short), Digital Signal Processor (DSP for short), Digital Signal Processing Device (DSPD for short), Programmable Logic Device (PLD for short), Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short), controller, microcontroller, microprocessor or other electronic component implementation for performing image processing methods .
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • controller microcontroller, microprocessor or other electronic component implementation for performing image processing methods .
  • the structure of the electronic device 100 shown in FIG. 1 does not constitute a limitation on the electronic device in the embodiment of the present application.
  • the electronic device 100 may include more or fewer components than those shown in FIG. 1 , or combine certain parts.
  • FIG. 2 is a schematic diagram of a hardware composition framework applicable to another image processing method provided in the embodiment of the present application.
  • the hardware composition framework may include: a first electronic device 11 and a second electronic device 12 connected through a network 13 .
  • the hardware structure of the first electronic device 11 and the second electronic device 12 may refer to the electronic device 100 in FIG. 1 . That is, it can be understood that there are two electronic devices 100 in this embodiment, and the two perform data interaction.
  • the form of the network 13 is not limited in the embodiment of the present application, that is, the network 13 may be a wireless network (such as WIFI, Bluetooth, etc.), or a wired network.
  • the first electronic device 11 and the second electronic device 12 may be the same electronic device, for example, both the first electronic device 11 and the second electronic device 12 are servers; they may also be different types of electronic devices, for example, the first electronic device
  • the device 11 may be a smart phone or other smart terminals
  • the second electronic device 12 may be a server.
  • a server with strong computing capability may be used as the second electronic device 12 to improve data processing efficiency and reliability, and further improve image processing efficiency.
  • a smart phone with low cost and wide application range is used as the first electronic device 11 to realize the interaction between the second electronic device 12 and the user.
  • the interaction process may be as follows: the smart phone obtains each image forming the video stream, and sends the video stream or the image to the server, and the server performs brightness correction.
  • the server sends the resulting brightness-optimized image to the smartphone.
  • FIG. 3 is a schematic flowchart of an image processing method provided in an embodiment of the present application.
  • the method in this example includes:
  • S101 Obtain an image to be processed from a video stream.
  • a video stream refers to a video that is subjected to brightness correction processing, including multiple video frames, and each video frame can be processed as an image to be processed. For example, each video frame is sequentially selected as an image to be processed in chronological order deal with.
  • the image to be processed is in RGB format.
  • the RGB format is also called RGB color, which is often referred to as the optical three primary colors, R stands for Red (red), G stands for Green (green), and B stands for Blue (blue). Any color that can be seen by the naked eye in nature can be formed by mixing and superimposing these three colors, so the mode of synthesizing colors in the RGB format is also called the additive color mode.
  • the non-first video frame in the video stream can be used as the image to be processed for brightness correction processing.
  • the image to be processed this time may be determined according to the last processed video frame. Therefore, for the first video frame in the video stream, it does not have corresponding historical adjacent frame images, and has no corresponding basis for judgment, so no processing can be done on the first video frame, or the first video frame can be processed Brightness (for example, average brightness) is adjusted to a preset brightness.
  • Brightness for example, average brightness
  • a series of preprocessing can be performed on the original video frame to obtain an object that is more convenient for brightness correction, that is, an image to be processed.
  • the contrast of the video frame can be increased to make bright places brighter and dark places darker, so as to accurately determine the correction intensity during subsequent gamma brightness correction.
  • the process of obtaining images to be processed from the video stream may include:
  • Step 11 Obtain an initial image from the video stream according to the historical adjacent frame images, and perform normalization processing on the initial image to obtain a normalized image.
  • Step 12 Map the normalized image to a linear color space to obtain a first intermediate image.
  • Step 13 increasing the contrast of each color channel in the first intermediate image to obtain a second intermediate image.
  • Step 14 Map the second intermediate image to the gamma color space to obtain a third intermediate image.
  • Step 15 Denormalize the third intermediate image to obtain the image to be processed.
  • the initial image refers to the video frame at a moment after the historical adjacent frame image in the video stream
  • the historical adjacent frame image refers to the image to be processed in the last image processing process, that is, the previous image to be processed in the video stream
  • Normalization processing refers to the processing of mapping the pixel values of each color channel of the initial image to the interval from 0 to 1.
  • the pixel values obtained after normalization processing can be mapped to the linear color space in a certain way. After the mapping is obtained,
  • the image of is the first intermediate image. By increasing the contrast of the first intermediate image, the image contrast is enhanced.
  • the contrast enhancement in the linear space is better than the contrast enhancement in the non-linear space such as gamma color space.
  • denormalization processing that is, map the pixel value from 0 to 1 back to the value range of the original color channel , to get the image to be processed.
  • the specific manner of color space mapping is not limited in this embodiment, and may be set as required.
  • S102 Convert the image to be processed into a YUV format to obtain a first image, and extract Y channel data corresponding to the first image.
  • the YUV format is a color coding method, which has three color channels of Y, U, and V.
  • Y represents the brightness (Luminance or Luma), that is, the grayscale value
  • U and V represent the color Degree (Chrominance or Chroma)
  • the role is to describe the image color and saturation, used to specify the color of the pixel.
  • the Y channel data used to represent the brightness can be obtained.
  • each pixel in it has a set of YUV data, so the number of Y channel data corresponding to the extracted first image is multiple, and is the same as the number of pixels of the first image same.
  • the long side of the first image (that is, the image to be processed) has H pixels and the wide side has W pixels, the number of pixels is W*H, and the number of Y channel data is also W*H.
  • This embodiment does not limit the specific manner of image format conversion.
  • a single Y channel data can represent the brightness level of a single pixel, and all Y channel data corresponding to the first image can represent the overall brightness level of the first image. According to the Y channel data, it can be determined whether the first image is darker, and the darker degree, and determine the pixel-to-one brightness correction parameter according to the brightness level.
  • the brightness correction parameters refer to the basic parameters used to generate the gamma parameters used in the subsequent gamma brightness correction. According to the different generation methods of the gamma parameters, the number of brightness correction parameters and the specific determination method are not limited. Specifically, when the gamma parameter is less than 1, the area with lower brightness in the image will be stretched, and the part with higher brightness will be compressed at the same time.
  • the brightness of the darker part of the image will increase greatly, and the brightness of the brighter part will increase.
  • the improvement is small; when the gamma parameter is greater than 1, the area with higher brightness in the image will be stretched, and the part with lower brightness will be compressed at the same time, which is manifested as an increase in the contrast of the image.
  • the brightness correction parameter is in the range of 0 to 1.
  • the Y channel corresponding to all pixels of the first image can be used The data determines the brightness correction parameters, and realizes the effect of determining the brightness correction parameters according to the overall brightness level of the first image, and then determining the gamma brightness correction range according to the overall brightness level of the first image.
  • the brightness correction parameter is a global parameter, that is, all pixels of the first image correspond to the same brightness correction parameter.
  • the brightness correction parameter of the global parameter can be quickly determined, and uniform brightness correction can be performed on the first image.
  • the process of generating brightness correction parameters corresponding to each pixel of the first image may include:
  • Step 21 Sorting the Y-channel data corresponding to each pixel according to size or frequency of occurrence to obtain a data sequence.
  • Step 22 Determine the target data at the preset ratio position in the data sequence as the brightness correction parameter.
  • the preset ratio position can be 95% position, or 80% position, that is, after the number of Y channel data is 95% or 80%, the Y channel data whose sequence number is this value in the data sequence is determined to be in the preset ratio Target data at the position, and determine the target data as brightness correction parameters.
  • the brightness correction parameter of the global parameter only needs to be determined once, so its determination speed is relatively fast. However, global parameters cannot perform separate brightness corrections for different parts of the same image.
  • the brightness correction parameter is a local parameter, that is, the brightness correction parameter corresponding to each pixel may be different, and the brightness correction parameter of the local parameter has local adaptability.
  • the process of generating brightness correction parameters corresponding to each pixel of the first image may include:
  • Step 31 Determine the adjacent range corresponding to each pixel.
  • Step 32 Sorting the Y-channel data corresponding to the pixels and adjacent pixels in the adjacent range according to size or frequency of occurrence to obtain a data sequence.
  • Step 33 Determine the target data at a preset ratio position in the data sequence as the brightness correction parameter corresponding to the pixel.
  • the adjacent range refers to the range closer to the specified pixel, and the pixels in the adjacent range can be regarded as a part of the image, wherein the Y channel data of each pixel can be integrated to represent the brightness level of the part.
  • the specific size and determination method of the adjacent range are not limited. For example, a range that is two pixels away from a certain pixel may be selected as the adjacent range. That is, for a specified pixel, other pixels whose distance to it is less than or equal to the distance of two pixels are all within the vicinity of the pixel, and these other pixels are adjacent pixels.
  • the brightness correction parameters corresponding to a certain part of the image can be determined according to the brightness of a certain part of the image, so that the subsequent gamma brightness correction process has a local Ability to adapt.
  • S104 Perform gamma brightness correction on the Y-channel data corresponding to each pixel by using the brightness correction parameter to obtain corrected data.
  • the gamma transformation formula is:
  • c is a coefficient, and in the application scenario of this application, it is the maximum brightness value, namely 255, r is the normalized Y channel data, ⁇ is the gamma parameter, and s is the corrected data.
  • the specific manner of generating the gamma parameter by using the brightness correction parameter can be set as required, and the specific manner is also related to the specific selection method of the brightness correction parameter.
  • the gamma brightness correction is performed on the Y channel data corresponding to each pixel by using the brightness correction parameters, and the process of obtaining the corrected data may include:
  • Step 41 Use the brightness difference between the maximum brightness value and the brightness correction parameter to generate a power value that is inversely proportional to the brightness difference, and use the power value and the Y channel data to perform normalized gamma calculations to obtain the corrected data .
  • the brightness difference can be determined by using the maximum brightness value and the brightness correction parameter.
  • the maximum brightness value refers to the maximum value within the optional range of the Y channel parameters.
  • the brightness difference can represent the brightness level of the first image or a certain part of the first image. It can be understood that the smaller the difference, the higher the brightness, and vice versa, the lower the brightness. Since the gamma parameter is less than 1, the smaller the gamma parameter, the greater the brightness improvement of the dark area, and the larger the gamma parameter, the smaller the brightness improvement of the dark area.
  • the gamma transformation formula can be regarded as a power function, and the gamma parameter is the power value.
  • the normalized value of the Y channel data is determined as the r value, and the gamma calculation is completed to obtain the corrected data.
  • step 41 may be further refined to include the following steps:
  • Step 51 Using the maximum brightness value and the brightness correction parameter to generate a brightness difference value greater than zero.
  • Step 52 Generate a power value by using the brightness difference and a preset threshold; the power value is inversely proportional to the brightness difference, and the power value is greater than zero.
  • Step 53 Use the ratio of the Y channel data and the maximum brightness value as the base number, and use the base number and the power value to obtain a power function value.
  • Step 54 Multiply the power function value with the maximum brightness value to obtain corrected data.
  • the gamma parameter Since the gamma parameter must be greater than zero, when generating the brightness difference, it is necessary to subtract the brightness correction parameter from the maximum brightness value to obtain a brightness difference greater than zero. After obtaining the brightness difference, compare it with the preset threshold according to the preset Set the formula to calculate and get the power value.
  • the specific content of the preset formula is not limited, for example, it can be:
  • gamma is a gamma parameter
  • ⁇ is a preset threshold, and its size is not limited, for example, it can be 0.45
  • y_max is a brightness correction parameter
  • 255-y_max is a brightness difference.
  • S105 Replace the Y channel data with the corrected data to obtain a second image, and convert the second image into an RGB format to obtain a processed image.
  • the corrected data After the corrected data is obtained, use it as the new Y channel data, replace the original Y channel data, form the second image with U channel data and V channel data, and convert it back to RGB format to obtain the processed image .
  • the processed image has undergone a brightness correction, it may still not meet the preset requirements (eg, image quality requirements), or may not meet the needs of the user. In order to ensure the effect of brightness correction, after the processed image is obtained, it can also be evaluated.
  • S106 Input the processed image and the optimized image of the adjacent historical frame into the evaluation model, and obtain an evaluation parameter used to characterize the brightness difference between the processed image and the optimized image of the adjacent historical frame.
  • the evaluation model refers to a model that evaluates the brightness level of the processed image, and the structure and type of the model are not limited.
  • the evaluation model can be obtained by training a convolutional neural network model.
  • the convolutional neural network model can be trained using training data to obtain an evaluation model, wherein the training data is multiple groups, and each group of training data includes two training images, which can be referred to as the first training image and the second training image , the first training image and the second training image are respectively images corresponding to two adjacent moments in time sequence, and the brightness of the first training image and the second training image may be the same or different.
  • the label according to the difference in image brightness for example, you can calculate the first average brightness of the first training image and the second average brightness of the second training image, and subtract the first average brightness from the second average brightness to obtain the brightness difference , and determine the brightness difference as the label value of this group of training images; or, the brightness calculation method based on the weighted average of pixel positions can be used, and the brightness difference can be obtained by subtracting the first average brightness and the second average brightness after calculation . Labeling can be done manually or automatically. Multiple sets of training data are used to train the convolutional neural network model as the initial model. During the training process, the initial model can learn the ability to accurately identify the brightness difference between two images according to the labels, and obtain an evaluation model. Specifically, the evaluation model expresses the brightness difference between two images in the form of output evaluation parameters. For example, when the evaluation parameter is larger, it indicates that the brightness difference between the two images is smaller; Small.
  • the historical adjacent frame optimized image refers to the brightness optimized image corresponding to the historical adjacent frame image, and the historical adjacent frame image is the historical adjacent frame image mentioned in the above step S101.
  • the processed image and the optimized image of the adjacent frames in the history can be input into the evaluation model, and the evaluation model can be used to output evaluation parameters.
  • the form characterizes the brightness difference between the processed image and the optimized image of the historical adjacent frames, so that the evaluation parameters can be used to evaluate whether the brightness level of the processed image is consistent with the optimized image of the historical adjacent frames.
  • the form of the evaluation parameter is not limited, for example, it may be a percentage.
  • step S108 If it is in the target interval, execute step S108, otherwise execute step S109.
  • S108 Update the brightness correction parameters, so as to use the updated brightness correction parameters to re-perform gamma brightness correction on the Y channel data corresponding to each pixel to obtain new corrected data, and then use the new corrected data to obtain new processed image, using the new processed image to obtain a new evaluation parameter, until the new evaluation parameter is not in the target interval, and obtain the brightness optimized image.
  • the target interval refers to the interval that indicates that the brightness level of the processed image is inconsistent with the optimized image of the adjacent frame in history, and the brightness difference is large.
  • the specific range and upper and lower limits are not limited.
  • the evaluation parameter is in the target range, it means that the brightness difference between the optimized image and the processed image of adjacent frames in history is large, and the brightness optimization effect is not good, and it is necessary to perform brightness correction on the image to be processed again, so update the brightness correction parameters to re-perform Gamma brightness correction, that is, re-execute step S104 after updating the brightness correction parameters to obtain new corrected data, and then use the new corrected data to obtain a new processed image, and use the new processed image to obtain new evaluation parameters, until The new evaluation parameter is not in the target interval, and the brightness-optimized image is obtained.
  • the process of updating the brightness correction parameters may include:
  • Step 61 If the processed image is darker than the optimized image of the historical adjacent frame, reduce the brightness correction parameter.
  • Step 62 If the processed image is brighter than the optimized image of historical adjacent frames, increase the brightness correction parameter.
  • the brightness correction parameter needs to be reduced to increase the brightness difference, which in turn leads to a decrease in the gamma parameter, making the brightness increase more big.
  • the evaluation model can determine the benchmark brightness for brightness evaluation according to the user's preference, that is, when the evaluation model ensures that the brightness levels of the optimized image of the historical adjacent frames are consistent with the brightness level of the processed image, it also needs to ensure that it is consistent with Baseline brightness matches.
  • the evaluation model can be determined for the first video frame in the video stream, which does not have the optimized image of the historical adjacent frame, after generating the corresponding processed image, it can be compared with the reference brightness to determine whether the brightness needs to be adjusted.
  • the update of the correction parameters further determines the tone of the subsequent brightness correction.
  • the user's feedback on the processed image can also be obtained, and then the evaluation model can be trained so that the evaluation model can meet the user's preference. Specifically, the following steps may also be included:
  • Step 71 Visualize and output the processed image.
  • Step 72 Obtain user feedback information for responding to the processed image, generate training data according to the user feedback information, and use the training data to perform additional training on the evaluation model.
  • the specific form of the user feedback information is not limited. Using user feedback information and processed images, training data can be composed, and the evaluation model can be additionally trained by using it, so that the evaluation model can more accurately understand the user's preference for image brightness.
  • the processed images corresponding to all the video frames in the video stream can be used to form an optimized video stream, and the optimized video stream can be visually output so that users can view videos with appropriate brightness.
  • S109 Determine that the evaluation parameter is not in the target interval, then determine the processed image as the brightness-optimized image corresponding to the image to be processed.
  • the evaluation parameter is not in the target range, it means that the brightness difference between the optimized image of the historical adjacent frame and the processed image is small, the brightness optimization effect is better, and there is no need to perform brightness correction on the image to be processed again. Therefore, the processed image is determined as the brightness-optimized image of the image to be processed.
  • the gamma brightness correction can be performed on the Y channel data, and the dark part of the image to be processed can be improved to a large extent, while the bright part can be slightly improved.
  • the enhancement or non-enhancement makes the image clear and natural.
  • the obtained correction data is used to generate the second image and then restored to the RGB format to obtain the processed image.
  • the evaluation model is trained based on the user's needs and preferences for image brightness, and it has good spatial and temporal adaptive capabilities. The evaluation model can evaluate whether the brightness change response of the processed image and the optimized image of the adjacent frames in the time series is good or not.
  • the target interval is used to indicate that the processed image has an abnormal brightness change based on the optimized image of the adjacent frame in history, so if it is not in the target interval, it means that the brightness optimization of the image to be processed is reasonable, so set
  • the processed image is determined to be a brightness optimized image after brightness optimization, and it is determined that the optimization process of the image to be processed ends.
  • FIG. 4 is an image to be processed provided by the embodiment of the present application
  • FIG. 5 is a processed image obtained by processing according to a related processing method provided by the embodiment of the present application. It can be seen that the relevant processing method improves the brightness of the entire image to be processed, and still improves the already bright part, resulting in poor image quality and loss of bright part information. Moreover, there is no correlation between image frames in the video stream, and when the brightness of the external scene changes, the brightness of the processed video stream will also change accordingly.
  • FIG. 6 is a schematic flowchart of a specific conversion from RGB format to YUV format provided by an embodiment of the present application.
  • set the preset threshold ⁇ and the preset ratio position (for generating y_max) input the RGB image as the image to be processed, and perform preprocessing on the image to be processed.
  • the RGB information of each pixel is obtained and normalized to [0,1] respectively to obtain R' data, G' data and B' data, or R1 data, G1 data and B1 data.
  • the conversion method is: when R' data, G' data or B' data is less than or equal to 0.04045, divide by 12.92; when RGB When the value is greater than 0.04045, return pow((RGB+0.055)/1.055,2.4), namely:
  • color1 is R' data, G' data and B' data, that is, the data corresponding to the normalized image
  • color2 is the data corresponding to the first intermediate image, which can be specifically divided into R" data, G" data and B" data, or R2 data, G2 data and B2 data.
  • contrast enhancement is performed to enhance the contrast of the image.
  • the formula is as follows:
  • hdrmode and hdrGamma are two fixed parameters, which are used to adjust the brightness of the image, and their values are greater than zero, which can be set according to needs.
  • color3 is the data corresponding to the second intermediate image, which can be specifically divided into R"' data, G"' data and B"' data, or R3 data, G3 data and B3 data.
  • the value of color2 is 1.5857
  • the output image is the same as The original image is the same.
  • RGB ⁇ 1.5857 the brightness of the image decreases
  • RGB>1.5857 the brightness of the image increases.
  • color4 is the data corresponding to the third intermediate image, which can be specifically divided into R"" data, G"" data and B"" data, or R4 data, G4 data and B4 data.
  • color5 is the data corresponding to the image to be processed, which can be specifically divided into R””’ data, G””’ data and B””’ data, or R5 data, G5 data and B5 data.
  • the R data, G data and B data in formula (7) specifically refer to R5 data, G5 data and B5 data.
  • Obtain the brightness Y channel information of the image that is, the Y channel data
  • analyze and sort the distribution of the brightness values of the image take the top95% value of the brightness sequence from small to large, and record it as the brightness correction parameter, where top95% is the preset ratio Location.
  • top95% is the preset ratio Location.
  • the width and height of the image are w pixels and h pixels respectively, the brightness values of w ⁇ h pixels are recorded, and the frequency of occurrence of each brightness value is counted.
  • Y' is the corrected data. Since the value of ⁇ is greater than 0, the value is between 0 and 1. When the entire picture is dark, y_max will be smaller, so the gamma value will be larger, and the video brightness will be increased more; when the image is a normal brightness picture, y_max will be close to 255, and the gamma value will tend to 1 at this time , with little change to the image. In actual use, the default value of ⁇ is 0.45.
  • the UV channel remains unchanged, the Y’UV is synthesized, and then converted into an RGB image to obtain the rendered image, that is, the processed image, and the method of converting YUV to RGB is as follows:
  • FIG. 7 is a schematic flowchart of a specific image processing method provided by an embodiment of the present application.
  • the input video is processed by an adaptive algorithm, such as formula 11:
  • APA represents the processing of the image by the adaptive algorithm, that is, the process of brightness correction
  • H in represents the input image
  • the evaluation network MASK Input the evaluation network MASK, and the evaluation network outputs a score, according to which it is judged whether the processed image is of optimal quality and/or whether it is the user's favorite brightness. If not, for example, in the target range, the system continues to send the original video to the adaptive algorithm for optimization, and continuously optimizes the video by adjusting ⁇ and y_max until the output result is the user's favorite and the best quality. And the video with the highest score will be output.
  • formula 12 Such as formula 12:
  • Figure 8 is a specific comparison diagram of before and after image processing provided by the embodiment of this application
  • Figure 9 is another specific comparison diagram of before and after image processing provided by this embodiment of the application
  • the a image and the b image may specifically be video frames in the video stream acquired by the anchor in a live video scene
  • the a1 image and b1 image may be video frames in the optimized video stream.
  • image b1 in Figure 9 adaptively enhances the brightness of the darker scene behind the face, making the image look clear and natural.
  • FIG. 10 is a schematic diagram of an optimized video stream effect provided by an embodiment of the present application.
  • the graph is the response curve of each pixel in the small image. It can be seen from the curve that the brightness correction method provided by this application has a good time-domain response characteristic, and it has continuity and adaptability in the space domain.
  • the computer-readable storage medium provided by the embodiment of the present application is introduced below, and the computer-readable storage medium described below and the image processing method described above may be referred to in correspondence.
  • the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.
  • the computer-readable storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. medium.
  • each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other.
  • the description is relatively simple, and for relevant details, please refer to the description of the method part.
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically programmable ROM
  • EEPROM electrically erasable programmable ROM
  • registers hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

图像处理方法、设备及可读存储介质，该方法包括：从视频流中获取待处理图像；将待处理图像转换为YUV格式，得到第一图像，并提取第一图像对应的Y通道数据；利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数；利用亮度校正参数对各个像素分别对应的Y通道数据进行伽马亮度校正，得到校正后数据；利用校正后数据替换Y通道数据，得到第二图像，并将第二图像转换为RGB格式，得到处理后图像；将处理后图像和历史相邻帧优化图像输入评价模型，得到用于表征处理后图像与历史相邻帧优化图像之间亮度差距的评价参数；确定评价参数不处于目标区间，则将处理后图像确定为待处理图像对应的亮度优化图像；使得视频流亮度保持总体平稳。

Description

图像处理方法、设备及可读存储介质
本申请要求于2021年12月3日提交至中国专利局、申请号为202111469660.0、发明名称为“图像处理方法、设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，特别涉及图像处理方法、电子设备及计算机可读存储介质。
背景技术
网络直播是一种在现场架设独立的信号采集设备采集信号(音频和/或视频)并导入导播端(导播设备或平台)，导播端通过网络将信号上传至服务器，进而发布至指定的网址供人观看的技术。在某些较昏暗的场景下，为了使用户能够看清楚直播的图像，通常会对采集到的视频流进行处理，提高其亮度。当前，是直接对视频流的整个界面进行统一固定的亮度提升处理，即对直播过程中生成的所有图片的各个部分提升相同的亮度。然而，场景的昏暗程度可能发生变化，即由昏暗转变为明亮，统一固定的亮度提升处理会使得导致亮度提升后的图像太亮，影响用户的视觉体验。
发明内容
有鉴于此，本申请的目的在于提供一种图像处理方法、电子设备及计算机可读存储介质，使得用户能够具有较好的视觉体验。
为解决上述技术问题，第一方面，本申请提供了一种图像处理方法，包括：
从视频流中获取待处理图像；所述待处理图像为RGB格式；
将所述待处理图像转换为YUV格式，得到第一图像，并提取所述第一图像对应的Y通道数据；
利用所述Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数；
利用所述亮度校正参数对各个像素分别对应的所述Y通道数据进行伽 马亮度校正，得到校正后数据；
利用所述校正后数据替换所述Y通道数据，得到第二图像，并将所述第二图像转换为RGB格式，得到处理后图像；
将所述处理后图像和历史相邻帧优化图像输入评价模型，得到用于表征所述处理后图像与所述历史相邻帧优化图像之间亮度差距的评价参数；
确定所述评价参数不处于目标区间，则将所述处理后图像确定为所述待处理图像对应的亮度优化图像。
可选地，所述从视频流中获取待处理图像，包括：
根据历史相邻帧图像从所述视频流中获取初始图像，并对所述初始图像进行归一化处理，得到归一化图像；
将所述归一化图像映射到线性色彩空间，得到第一中间图像；
提升所述第一中间图像中各个色彩通道的对比度，得到第二中间图像；
将所述第二中间图像映射到伽马色彩空间，得到第三中间图像；
将所述第三中间图像进行反归一化处理，得到所述待处理图像。
可选地，所述利用所述Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数，包括：
对各个所述像素对应的所述Y通道数据按照大小或出现频次排序，得到数据序列；
将所述数据序列中处于预设比例位置的目标数据确定为所述亮度校正参数。
可选地，所述利用所述Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数，包括：
确定各个所述像素对应的临近范围；
将所述像素和所述临近范围内的临近像素对应的所述Y通道数据按照大小或出现频次排序，得到数据序列；
将所述数据序列中处于预设比例位置的目标数据确定为所述像素对应的所述亮度校正参数。
可选地，所述利用所述亮度校正参数对各个像素分别对应的所述Y通道数据进行伽马亮度校正，得到校正后数据，包括：
利用最大亮度值与所述亮度校正参数之间的亮度差值，生成与所述亮 度差值成反比的幂值，并利用所述幂值和所述Y通道数据进行归一化的伽马计算，得到所述校正后数据。
可选地，所述利用最大亮度值与所述亮度校正参数之间的亮度差值，生成与所述亮度差值成反比的幂值，并利用所述幂值和所述Y通道数据进行归一化的伽马计算，得到所述校正后数据，包括：
利用所述最大亮度值与所述亮度校正参数生成大于零的所述亮度差值；
利用所述亮度差值和预设阈值生成幂值；所述幂值与所述亮度差值成反比，所述幂值大于零；
利用所述Y通道数据和所述最大亮度值的比值作为底数，并利用所述底数和所述幂值得到幂函数值；
将所述幂函数值与所述最大亮度值相乘，得到所述校正后数据。
可选地，若所述评价参数处于目标区间，包括：
更新所述亮度校正参数，以便利用更新后的所述亮度校正参数重新对各个像素分别对应的所述Y通道数据进行伽马亮度校正，得到新的校正后数据，进而利用新的校正后数据得到新的处理后图像，利用新的处理后图像得到新的评价参数，直至新的评价参数不处于目标区间，得到所述亮度优化图像。
可选地，所述更新所述亮度校正参数，包括：
若所述处理后图像比所述历史相邻帧优化图像更暗，则降低所述亮度校正参数；
若所述处理后图像比所述历史相邻帧优化图像更亮，则提高所述亮度校正参数。
可选地，还包括：
可视化输出所述处理后图像；
获取用于响应所述处理后图像的用户反馈信息，并根据所述用户反馈信息生成训练数据，并利用所述训练数据对所述评价模型进行追加训练。
可选地，还包括：
利用所述视频流中的所有视频帧对应的处理后图像构成优化视频流，并可视化输出所述优化视频流。
第二方面，本申请还提供了一种电子设备，包括存储器和处理器，其中：
所述存储器，用于保存计算机程序；
所述处理器，用于执行所述计算机程序，以实现上述的图像处理方法。
第三方面，本申请还提供了一种计算机可读存储介质，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现上述的图像处理方法。
本申请提供的图像处理方法，从视频流中获取待处理图像；待处理图像为RGB格式；将待处理图像转换为YUV格式，得到第一图像，并提取第一图像对应的Y通道数据；利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数；利用亮度校正参数对各个像素分别对应的Y通道数据进行伽马亮度校正，得到校正后数据；利用校正后数据替换Y通道数据，得到第二图像，并将第二图像转换为RGB格式，得到处理后图像；将处理后图像和历史相邻帧优化图像输入评价模型，得到评价参数；确定评价参数不处于目标区间，则将处理后图像确定为待处理图像对应的亮度优化图像。
可见，该方法在获取到待处理图像后，将其转换为YUV格式，其中Y通道数据用于表示图像的亮度，通过颜色格式的转换，可以尽可能降低亮度校正对图像颜色的影响。根据待处理图像中各个像素对应的Y通道数据，可以确定待处理图像的总体亮度情况，进而确定对其进行亮度调节的基础，即亮度校正参数。利用亮度校正参数，可以基于Y通道数据的具体情况，对Y通道数据进行伽马亮度校正，能够根据需要对待处理图像中昏暗的部分进行较大程度的提升，而对明亮的部分进行较小程度的提升或不提升，使得图像清晰自然。利用得到的校正数据生成第二图像后将其恢复为RGB格式，得到处理后图像。评价模型基于用户对图像亮度的需要和喜好进行训练得到，其具有良好的空间自适应和时间自适应能力。评价模型能够评价处理后图像与历史相邻帧优化图像在时序上的亮度变化响应是否良好。若评价参数处于目标区间，所述目标区间用于指示处理后图像在历史相邻帧优化图像的基础上亮度变化异常，因此若不处于目标区间，则说明对待处理图像的亮度优化合理，因此将处理后图像确定为亮度优化后的亮度优 化图像，确定对待处理图像的优化流程结束。利用评价参数可以在视频流获取场景的明亮程度发生变化时相应的改变对不同待处理图像的伽马亮度校正方式，使得各个处理后图像的亮度在时域上连续。视频流的亮度在各个时刻上保持总体平稳，且各个处理后图像均经过伽马亮度校正，使得用户能够具有较好的视觉体验。
此外，本申请还提供了一种电子设备及计算机可读存储介质，同样具有上述有益效果。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请实施例提供的一种图像处理方法所适用的硬件组成框架示意图；
图2为本申请实施例提供的另一种图像处理方法所适用的硬件组成框架示意图；
图3为本申请实施例提供的一种图像处理方法的流程示意图；
图4为本申请实施例提供的一种待处理图像；
图5为本申请实施例提供的一种按照相关处理方式处理得到处理后图像；
图6为本申请实施例提供的一种具体的RGB格式转YUV格式的流程示意图；
图7为本申请实施例提供的一种具体的图像处理方法的流程示意图；
图8为本申请实施例提供的一种具体的图像处理前后效果比对图；
图9为本申请实施例提供的另一种具体的图像处理前后效果比对图；
图10为本申请实施例提供的一种优化视频流效果示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描 述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了便于理解，先对本申请实施例提供的图像处理方法，和/或，音频处理方法对应的方案所使用的硬件组成框架进行介绍。请参考图1，图1为本申请实施例提供的一种图像处理方法所适用的硬件组成框架示意图。其中电子设备100可以包括处理器101和存储器102，还可以进一步包括多媒体组件103、信息输入/信息输出(I/O)接口104以及通信组件105中的一种或多种。
其中，处理器101用于控制电子设备100的整体操作，以完成图像处理方法，和/或，音频处理方法中的全部或部分步骤；存储器102用于存储各种类型的数据以支持在电子设备100的操作，这些数据例如可以包括用于在该电子设备100上操作的任何应用程序或方法的指令，以及应用程序相关的数据。该存储器102可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，例如静态随机存取存储器(Static Random Access Memory，SRAM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、只读存储器(Read-Only Memory，ROM)、磁存储器、快闪存储器、磁盘或光盘中的一种或多种。在本实施例中，存储器102中至少存储有用于实现以下功能的程序和/或数据：
从视频流中获取待处理图像；待处理图像为RGB格式；
将待处理图像转换为YUV格式，得到第一图像，并提取第一图像对应的Y通道数据；
利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数；
利用亮度校正参数对各个像素分别对应的Y通道数据进行伽马亮度校正，得到校正后数据；
利用校正后数据替换Y通道数据，得到第二图像，并将第二图像转换为RGB格式，得到处理后图像；
将处理后图像和历史相邻帧优化图像输入评价模型，得到评价参数；
确定评价参数不处于目标区间，则将处理后图像确定为待处理图像对应的亮度优化图像。
多媒体组件103可以包括屏幕和音频组件。其中屏幕例如可以是触摸屏，音频组件用于输出和/或输入音频信号。例如，音频组件可以包括一个麦克风，麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器102或通过通信组件105发送。音频组件还包括至少一个扬声器，用于输出音频信号。I/O接口104为处理器101和其他接口模块之间提供接口，上述其他接口模块可以是键盘，鼠标，按钮等。这些按钮可以是虚拟按钮或者实体按钮。通信组件105用于电子设备100与其他设备之间进行有线或无线通信。无线通信，例如Wi-Fi，蓝牙，近场通信(Near Field Communication，简称NFC)，2G、3G或4G，或它们中的一种或几种的组合，因此相应的该通信组件105可以包括：Wi-Fi部件，蓝牙部件，NFC部件。
电子设备100可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit，简称ASIC)、数字信号处理器(Digital Signal Processor，简称DSP)、数字信号处理设备(Digital Signal Processing Device，简称DSPD)、可编程逻辑器件(Programmable Logic Device，简称PLD)、现场可编程门阵列(Field Programmable Gate Array，简称FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行图像处理方法。
当然，图1所示的电子设备100的结构并不构成对本申请实施例中电子设备的限定，在实际应用中电子设备100可以包括比图1所示的更多或更少的部件，或者组合某些部件。
可以理解的是，本申请实施例中并不对电子设备的数量进行限定，其可以是多个电子设备共同协作完成图像处理方法，和/或，音频处理方法。在一种可能的实施方式中，请参考图2，图2为本申请实施例提供的另一种图像处理方法所适用的硬件组成框架示意图。由图2可知，该硬件组成框架可以包括：第一电子设备11和第二电子设备12，二者之间通过网络13连接。
在本申请实施例中，第一电子设备11与第二电子设备12的硬件结构可以参考图1中电子设备100。即可以理解为本实施例中具有两个电子设备100，两者进行数据交互。进一步，本申请实施例中并不对网络13的形式进行限定，即，网络13可以是无线网络(如WIFI、蓝牙等)，也可以是有线网络。
其中，第一电子设备11和第二电子设备12可以是同一种电子设备，如第一电子设备11和第二电子设备12均为服务器；也可以是不同类型的电子设备，例如，第一电子设备11可以是智能手机或其它智能终端，第二电子设备12可以是服务器。在一种可能的实施方式中，可以利用计算能力强的服务器作为第二电子设备12来提高数据处理效率及可靠性，进而提高图像处理的处理效率。同时利用成本低，应用范围广的智能手机作为第一电子设备11，用于实现第二电子设备12与用户之间的交互。可以理解的是，该交互过程可以为：智能手机获取组成视频流的各个图像，并将视频流或图像发送至服务器，由服务器进行亮度校正。服务器将得到的亮度优化图像至智能手机。
基于上述说明，请参考图3，图3为本申请实施例提供的一种图像处理方法的一种流程示意图。该实施例中的方法包括：
S101：从视频流中获取待处理图像。
视频流，是指被进行亮度校正处理的视频，其中包括多个视频帧，各个视频帧均可以作为待处理图像进行处理，示例性的，各个视频帧按照时间顺序依次被选为待处理图像进行处理。其中，待处理图像为RGB格式。RGB格式也被称为RGB色彩，即常说的光学三原色，R代表Red(红色)，G代表Green(绿色)，B代表Blue(蓝色)。自然界中肉眼所能看到的任何色彩都可以由这三种色彩混合叠加而成，因此RGB格式下合成色彩的模式也称为加色模式。
为了保证视频流各个时刻的图像亮度均符合要求，可以将视频流中的非首个视频帧作为待处理图像进行亮度校正处理。具体的，可以根据上一次被处理的视频帧确定本次处理的待处理图像。因此对于视频流中的首个视频帧来说，其不具有对应的历史相邻帧图像，不具有相应的判断依据， 故此可以不对首个视频帧做任何处理，或者可以对首个视频帧的亮度(例如平均亮度)调节为预设亮度。此外，为了更好的进行亮度校正，可以对原本的视频帧进行一系列预处理，得到更方便进行亮度校正的对象，即待处理图像。优选的，在一种实施方式中，可以提高视频帧的对比度，使亮处更亮，暗处更暗，以便在后续伽马亮度校正时跟准确地确定校正强度。具体的，从视频流中获取待处理图像的过程可以包括：
步骤11：根据历史相邻帧图像从视频流中获取初始图像，并对初始图像进行归一化处理，得到归一化图像。
步骤12：将归一化图像映射到线性色彩空间，得到第一中间图像。
步骤13：提升第一中间图像中各个色彩通道的对比度，得到第二中间图像。
步骤14：将第二中间图像映射到伽马色彩空间，得到第三中间图像。
步骤15：将第三中间图像进行反归一化处理，得到待处理图像。
其中，初始图像，是指视频流中处于历史相邻帧图像之后一个时刻的视频帧，历史相邻帧图像，是指上一次图像处理过程的待处理图像，即视频流中待处理图像的上一帧图像。归一化处理，是指将初始图像的各个颜色通道的像素值映射至0到1的区间的处理，经过归一化处理得到的像素值能够按照一定的方式映射到线性色彩空间，得到映射后的图像即第一中间图像。通过提升第一中间图像的对比度，完成图像对比度的提升。在线性空间内进行对比度的提升，相比在伽马色彩空间等非线性空间内进行对比度提升，处理后的图像鲁棒性更好，外在表现为图像的视觉观感更好。将第二中间图像映射回伽马色彩空间后得到第三中间图像，再经过反归一化的处理完成像素值的恢复，即将像素值从0到1的区间映射回原本色彩通道的取值区间，得到待处理图像。对于色彩空间映射的具体方式本实施例不做限定，可以根据需要进行设置。
S102：将待处理图像转换为YUV格式，得到第一图像，并提取第一图像对应的Y通道数据。
其中，YUV格式，是一种颜色编码方法，其具有Y、U、V三个色彩通道，其中，Y表示明亮度(Luminance或Luma)，也就是灰阶值；U和V表示的则是色度(Chrominance或Chroma)，作用是描述影像色彩及饱 和度，用于指定像素的颜色。通过将待处理图像由RGB格式转换为YUV格式，可以得到用于表示亮度的Y通道数据。需要说明的是，对于第一图像来说，其中的每个像素均具有一组YUV数据，因此提取得到的第一图像对应的Y通道数据的数量为多个，且与第一图像的像素数量相同。示例性的，若第一图像(即待处理图像)的长边有H个像素，宽边有W个像素，则像素数量为W*H，Y通道数据的数量也为W*H。本实施例并不限定图像格式转换的具体方式。
S103：利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数。
单个Y通道数据可以表征单个像素的亮度水平，而第一图像对应的所有Y通道数据能够表征第一图像的整体亮度水平，根据Y通道数据，可以确定第一图像是否较暗，以及较暗的程度，并根据亮度水平确定像素对一个的呢亮度校正参数。亮度校正参数，是指用于生成后续进行伽马亮度校正时采用的伽马参数的基础参数，根据伽马参数的生成方式的不同，亮度校正参数的数量和具体确定方式不做限定。具体的，当伽马参数小于1时，会拉伸图像中亮度较低的区域，同时会压缩亮度较高的部分，对外表现为图像中较暗的部分亮度提升较大，较亮的部分亮度提升较小；伽马参数大于1时，会拉伸图像中亮度较高的区域，同时会压缩亮度较低的部分，表现为图像的对比度提高。在本实施例中，为了进行亮度调节，应当保证亮度校正参数处于0至1的范围，由于其大小影响了伽马亮度校正对亮度的提升程度，因此可以利用第一图像所有像素对应的Y通道数据确定亮度校正参数，实现根据第一图像的整体亮度水平确定亮度校正参数，进而根据第一图像的整体亮度水平确定伽马亮度校正幅度的效果。
在一种实施方式中，亮度校正参数为全局参数，即第一图像的所有像素对应于相同的亮度校正参数。全局参数的亮度校正参数可以快速确定，能够对第一图像进行统一的亮度矫正。具体的，利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数的过程可以包括：
步骤21：对各个像素对应的Y通道数据按照大小或出现频次排序，得到数据序列。
步骤22：将数据序列中处于预设比例位置的目标数据确定为亮度校正 参数。
对Y通道数据排序时，可以按照从大到小的顺序排序，或者可以按照从小到大的顺序排序，或者可以统计其出现频次，并按照频次从大到小或频次从小到大的顺序排序。其中，预设比例位置可以为95％位置，或者为80％位置，即Y通道数据的数量取95％或80％后，将数据序列中序号为该值的Y通道数据确定为处于预设比例位置的目标数据，并将目标数据确定为亮度校正参数。全局参数的亮度校正参数仅需确定一次，因此其确定速度较快。但是全局参数无法针对同一图像不同部分的情况进行单独的亮度校正。为了解决该问题，在另一种实施方式中，亮度校正参数为局部参数，即每个像素对应的亮度校正参数可以不同，局部参数的亮度校正参数具有局部自适应性。具体的，利用Y通道数据，生成第一图像各个像素分别对应的亮度校正参数的过程可以包括：
步骤31：确定各个像素对应的临近范围。
步骤32：将像素和临近范围内的临近像素对应的Y通道数据按照大小或出现频次排序，得到数据序列。
步骤33：将数据序列中处于预设比例位置的目标数据确定为像素对应的亮度校正参数。
其中，临近范围，是指与指定的像素距离较近的范围，临近范围内的像素可以视为图像上的一个部分，其中各个像素的Y通道数据能够综合起来表征该部分的亮度水平。临近范围的具体大小和确定方式不做限定，示例性的，可以选择距离某一像素两个像素距离的范围为临近范围。即，对于某一指定的像素，与其的距离小于或等于两个像素距离的其他像素均处于该像素的临近范围内，这些其他像素即为临近像素。
通过在对临近范围内的像素对应的Y通道数据进行排序并确定亮度校正参数，可以根据图像中某部分的亮度确定该部分像素对应的亮度校正参数，使得后续的伽马亮度校正过程具备了局部自适应的能力。
S104：利用亮度校正参数对各个像素分别对应的Y通道数据进行伽马亮度校正，得到校正后数据。
在得到亮度校正参数后，利用其确定伽马亮度校正所需的伽马参数，进而利用伽马参数对各个像素对应的Y通道数据进行伽马亮度校正。伽马 亮度校正，即按照伽马变换公式对Y通道数据进行校正，伽马变换公式为：
s＝c*r γ
其中c为系数，在本申请的应用场景下，其为最大亮度值，即255，r为经过归一化的Y通道数据，γ即为伽马参数，s为校正后数据。对于利用亮度校正参数生成伽马参数的具体方式，可以根据需要进行设定，该具体方式同样与亮度校正参数的具体选取方式相关。具体的，在一种实施方式中，利用亮度校正参数对各个像素分别对应的Y通道数据进行伽马亮度校正，得到校正后数据的过程可以包括：
步骤41：利用最大亮度值与亮度校正参数之间的亮度差值，生成与亮度差值成反比的幂值，并利用幂值和Y通道数据进行归一化的伽马计算，得到校正后数据。
在本实施例中，可以利用最大亮度值与亮度校正参数确定亮度差值，最大亮度值，是指Y通道参数可选范围内的最大值。亮度差值能够表示第一图像或第一图像的某个部分的亮度水平，可以理解的是，该差值越小，说明亮度越高，反之说明亮度越低。由于伽马参数在小于1的情况下，伽马参数越小，对暗部区域的亮度提升越大，伽马参数越大，对暗部区域的亮度提升越小，因此在确定伽马参数时，需要其与亮度差值成反比，即亮度差值越小，伽马参数越大，亮度参数越大，伽马参数越小。
伽马变换公式可以被视为一种幂函数，伽马参数即为其中的幂值。将Y通道数据进行归一化后的值确定为r值，完成伽马计算，即可得到校正后数据。
进一步的，上述步骤41可以进一步细化包括如下步骤：
步骤51：利用最大亮度值与亮度校正参数生成大于零的亮度差值。
步骤52：利用亮度差值和预设阈值生成幂值；幂值与亮度差值成反比，幂值大于零。
步骤53：利用Y通道数据和最大亮度值的比值作为底数，并利用底数和幂值得到幂函数值。
步骤54：将幂函数值与最大亮度值相乘，得到校正后数据。
由于伽马参数必须大于零，因此在生成亮度差值时，需要利用最大亮度值减去亮度校正参数，得到大于零的亮度差值，在得到亮度差值后，将 其与预设阈值按照预设公式进行计算，得到幂值。预设公式的具体内容不做限定，示例性的，其可以为：
gamma＝1-β×(255-y_max)/255
其中，gamma即为伽马参数，β为预设阈值，其大小不做限定，例如可以为0.45，y_max为亮度校正参数，255-y_max为亮度差值。
S105：利用校正后数据替换Y通道数据，得到第二图像，并将第二图像转换为RGB格式，得到处理后图像。
在得到校正后数据后，将其作为新的Y通道数据，对原本的Y通道数据进行替换，与U通道数据和V通道数据组成第二图像，并将其转回RGB格式，得到处理后图像。处理后图像虽然经过了一次亮度校正，但是其仍可能不满足预设的要求(例如画质要求)，或者不满足用户的需要。为了保证亮度校正的效果，在得到处理后图像后，还可以对其进行评价。
S106：将处理后图像和历史相邻帧优化图像输入评价模型，得到用于表征处理后图像与历史相邻帧优化图像之间亮度差距的评价参数。
评价模型，是指对处理后图像的亮度水平进行评价的模型，模型的结构、类型等不做限定，例如，可以利用卷积神经网络模型训练得到评价模型。示例性的，可以利用训练数据对卷积神经网络模型进行训练得到评价模型，其中，训练数据为多组，每组训练数据包括两张训练图像，可以称为第一训练图像和第二训练图像，第一训练图像和第二训练图像分别为在时序上两个相邻时刻分别对应的图像，第一训练图像和第二训练图像的亮度可以相同或不同。根据图像亮度的差距大小标注标签，例如，可以计算第一训练图像的第一平均亮度和第二训练图像的第二平均亮度，并将第一平均亮度与第二平均亮度相减，得到亮度差距，并将该亮度差距确定为本组训练图像的标签值；或者，可以采用基于像素位置的加权平均的亮度计算方式，并在计算得到第一平均亮度和第二平均亮度后相减得到亮度差距。标注可以采用人工标注或自动标注。利用多组训练数据对作为初始模型的卷积神经网络模型进行训练，在训练过程中，初始模型能够根据标签学习到准确识别两张图像的亮度差距的能力，得到评价模型。具体的，评价模型以输出评价参数的方式表示两张图像的亮度差距的大小，例如当评价参 数越大，表明二者亮度差距越小；或者可以为评价参数越小，表明二者亮度差距越小。
由于视频流中不同视频帧生成时所处的环境亮度可能发生变化，因此采用相同的亮度矫正参数进行的相同的亮度校正过程对不同的待处理图像进行亮度矫正处理后得到的最终效果不同。历史相邻帧优化图像，是指历史相邻帧图像对应的亮度优化图像，历史相邻帧图像即为上述S101步骤中所提到的历史相邻帧图像。为了使得视频流中每个视频帧对应的处理后图像在时序上保持亮度的相对平稳，可以将处理后图像和历史相邻帧优化图像输入评价模型，利用评价模型输出评价参数，通过评价参数的形式对处理后图像与历史相邻帧优化图像之间的亮度差距进行表征，以便利用评价参数对处理后图像与历史相邻帧优化图像亮度水平是否一致进行评价。评价参数的形式不做限定，示例性的，可以为百分制分数。
S107：判断评价参数是否处于目标区间。
若处于目标区间，则执行S108步骤，否则执行S109步骤。
S108：更新亮度校正参数，以便利用更新后的亮度校正参数重新对各个像素分别对应的Y通道数据进行伽马亮度校正，得到新的校正后数据，进而利用新的校正后数据得到新的处理后图像，利用新的处理后图像得到新的评价参数，直至新的评价参数不处于目标区间，得到所述亮度优化图像。
目标区间，是指表明处理后图像与历史相邻帧优化图像亮度水平不一致，亮度差距较大的区间，具体范围和上下限值不做限定。评价参数处于目标区间时，说明历史相邻帧优化图像和处理后图像之间的亮度差异较大，亮度优化效果不佳，需要重新对待处理图像进行亮度校正，因此更新亮度校正参数，以便重新进行伽马亮度校正，即更新亮度校正参数后重新执行S104步骤，得到新的校正后数据，进而利用新的校正后数据得到新的处理后图像，利用新的处理后图像得到新的评价参数，直至新的评价参数不处于目标区间，得到所述亮度优化图像。
在一种实施方式中，若采用上述的方式确定亮度校正参数以及进行伽马亮度校正，更新亮度校正参数的过程可以包括：
步骤61：若处理后图像比历史相邻帧优化图像更暗，则降低亮度校正 参数。
步骤62：若处理后图像比历史相邻帧优化图像更亮，则提高亮度校正参数。
若处理后图像比历史相邻帧优化图像更暗，在评价参数处于目标区间的情况下，需要降低亮度校正参数，以便增加亮度差值，进而导致伽马参数减小，使得亮度提升的程度更大。反之，需要提高亮度校正参数，以便减少亮度差值，提高伽马参数，使得亮度提升的程度更小。
进一步的，在一种实施方式中，评价模型可以按照用户的偏好确定亮度评价的基准亮度，即评价模型在保证历史相邻帧优化图像和处理后图像的亮度水平一致时，还需要保证其与基准亮度相匹配。示例性的，对于视频流中的第一张视频帧，其不具有历史相邻帧优化图像，在生成其对应的处理后图像后，可以利用其与基准亮度进行比对，判断是否需要进行亮度校正参数的更新，进而确定后续亮度校正的基调。此外，还可以获取用户对处理后图像的反馈，进而对评价模型进行训练，以便评价模型能够满足用户的偏好。具体还可以包括如下步骤：
步骤71：可视化输出处理后图像。
步骤72：获取用于响应处理后图像的用户反馈信息，并根据用户反馈信息生成训练数据，并利用训练数据对评价模型进行追加训练。
其中，用户反馈信息的具体形式不做限定。利用用户反馈信息和处理后图像，可以组成训练数据，并利用其对评价模型进行追加训练，使得评价模型能够更加准确地了解用户对图像亮度的偏好。
由于待处理图像源自视频流，因此可以利用视频流中的所有视频帧对应的处理后图像构成优化视频流，并可视化输出优化视频流，以便用户查看亮度合适的视频。
S109：确定评价参数不处于目标区间，则将处理后图像确定为待处理图像对应的亮度优化图像。
评价参数不处于目标区间，则说明历史相邻帧优化图像和处理后图像之间的亮度差异较小，亮度优化效果较好，不需要重新对待处理图像进行亮度校正。因此将处理后图像确定为待处理图像的亮度优化图像。
应用本申请实施例提供的图像处理方法，在获取到待处理图像后，将 其转换为YUV格式，其中Y通道数据用于表示图像的亮度，通过颜色格式的转换，可以尽可能降低亮度校正对图像颜色的影响。根据待处理图像中各个像素对应的Y通道数据，可以确定待处理图像的总体亮度情况，进而确定对其进行亮度调节的基础，即亮度校正参数。利用亮度校正参数，可以基于Y通道数据的具体情况，对Y通道数据进行伽马亮度校正，能够根据需要对待处理图像中昏暗的部分进行较大程度的提升，而对明亮的部分进行较小程度的提升或不提升，使得图像清晰自然。利用得到的校正数据生成第二图像后将其恢复为RGB格式，得到处理后图像。评价模型基于用户对图像亮度的需要和喜好进行训练得到，其具有良好的空间自适应和时间自适应能力。评价模型能够评价处理后图像与历史相邻帧优化图像在时序上的亮度变化响应是否良好。若评价参数处于目标区间，所述目标区间用于指示处理后图像在历史相邻帧优化图像的基础上亮度变化异常，因此若不处于目标区间，则说明对待处理图像的亮度优化合理，因此将处理后图像确定为亮度优化后的亮度优化图像，确定对待处理图像的优化流程结束。利用评价参数可以在视频流获取场景的明亮程度发生变化时相应的改变对不同待处理图像的伽马亮度校正方式，使得各个处理后图像的亮度在时域上连续。视频流的亮度在各个时刻上保持总体平稳，且各个处理后图像均经过伽马亮度校正，使得用户能够具有较好的视觉体验。
基于上述实施例，本实施例将说明一种具体的实施例。首先，请参考图4和图5，图4为本申请实施例提供的一种待处理图像，图5为本申请实施例提供的一种按照相关处理方式处理得到处理后图像。可以看出，相关处理方式对整个待处理图像进行亮度提升，在原本已经较亮的部分上仍然进行提升，使得图像质量较差，丢失亮部信息。且视频流中的各个图像帧之间没有相关性，在外部场景亮度发生变化时，处理后的视频流的亮度也会随之变化。
为了解决上述问题，本申请提供了一种图像处理方法，请参考图6，图6为本申请实施例提供的一种具体的RGB格式转YUV格式的流程示意图。首先，设定好预设阈值β和预设比例位置(用于生成y_max)，输入RGB图像作为待处理图像，对待处理图像进行预处理。具体的，获取其中 每个像素的RGB信息，并分别将其归一化至[0,1]，得到R’数据、G’数据和B’数据，或R1数据、G1数据和B1数据。对R’数据、G’数据和B’数据分别进行从gamma空间到线性空间的转换，转换方式是：当R’数据、G’数据或B’数据小于等于0.04045时，除以12.92；当RGB值大于0.04045时，返回pow((RGB+0.055)/1.055,2.4)，即:
Figure PCTCN2022120792-appb-000001
Figure PCTCN2022120792-appb-000002
其中，color1为R’数据、G’数据和B’数据，即归一化图像对应的数据，color2为第一中间图像对应的数据，其可以具体分为R”数据、G”数据和B”数据，或R2数据、G2数据和B2数据。
在转换完毕后，进行对比度提升，作用是提升图像的对比度，公式如下：
color3＝hdrmode×(color2/hdrmode) hdrGamma    (3)
其中，hdrmode和hdrGamma是两个固定参数，用来调整图像的亮度，其值大于零，具体可以根据需要进行设置。color3为第二中间图像对应的数据，其可以具体分为R”’数据、G”’数据和B”’数据，或R3数据、G3数据和B3数据。当color2的值为1.5857时输出图像与原图像相同。很显然，当RGB<1.5857时图像亮度降低，当RGB>1.5857时图像亮度增加。
在处理完毕后，再将其转换为伽马色彩空间：
color4＝12.92×color3，color3≤0.0031308   (4)
Figure PCTCN2022120792-appb-000003
color4为第三中间图像对应的数据，其可以具体分为R””数据、G””数据和B””数据，或R4数据、G4数据和B4数据。
对color4乘以1.2，并映射到[0,255]范围内，生成待处理图像即：
color5＝1.2×color4×255   (6)
color5为待处理图像对应的数据其可以具体分为R””’数据、G””’数据和B””’数据，或R5数据、G5数据和B5数据。得到待处理图像后，即带渲染的RGB图后，将其转换为YUV格式，具体的：
Y＝0.299R+0.587G+0.114B
U＝-0.147R-0.289G+0.436B    (7)
V＝0.615R-0.515G-0.100B
公式(7)中的R数据、G数据和B数据具体是指R5数据、G5数据和B5数据。获取图像的亮度Y通道信息(即Y通道数据)，分析图像的亮度值的分布并排序，取从小到大亮度序列的top95％的值，记为亮度校正参数，其中top95％即为预设比例位置。假设图像宽和高分别为w像素和h像素，则记录w×h个像素的亮度值，统计每个亮度值出现的频次。将0-255的亮度值以及出现频次按亮度值从小到大进行排序，取从小到大的第w×h×0.95的亮度值作为整个图像最大的亮度值记为y_max。之所以不取最大亮度值，是因为大部分图像可能有部分像素的亮度值为255，会导致无法进行亮度校正。进行伽马亮度校正：
Figure PCTCN2022120792-appb-000004
gamma＝1-β×(255-y_max)/255     (9)
Y’为矫正后数据。由于β值大于0，取值在0到1之间。当整个图片都比较暗时，y_max会比较小，从而gamma值较大，视频亮度提高的会比较多；而当图像是正常亮度图片时，y_max接近于255，此时gamma的值会趋于1，对图像基本没有变化。实际使用时，β预设值是0.45。UV通道 不变，合成Y’UV，再转换成RGB图像，得到渲染的图像，即处理后图像，YUV转为RGB的方式如下：
R＝Y+1.140V
G＝Y-0.394U-0.581V      (10)
B＝Y+2.032U
请参考图7，图7为本申请实施例提供的一种具体的图像处理方法的流程示意图。将输入的视频经过自适应算法去处理，如公式11：
Figure PCTCN2022120792-appb-000005
APA表示自适应算法对图像的处理，即亮度校正的过程，H in表示输入图像，
Figure PCTCN2022120792-appb-000006
表示经过自适应算法得到的在迭代的图像，即处理后图像，随后连同视频流中待处理图像上一帧的处理后图像，即历史相邻帧优化图像
Figure PCTCN2022120792-appb-000007
输入评价网络MASK，评价网络输出一个分数，根据该分数判断处理后图像是否为最优画质和/或是否为用户最喜欢的亮度。如果为否，例如处于目标区间，该系统继续把原视频送入自适应算法去优化，通过去调节β和y_max不断去优化视频，直到输出的结果为用户最喜欢的，画质最优的,且分数最高的视频，并将该视频输出。如公式12：
Figure PCTCN2022120792-appb-000008
Figure PCTCN2022120792-appb-000009
为经过迭代之后最终输出结果，即优化视频流。
请参考图8和图9，图8为本申请实施例提供的一种具体的图像处理前后效果比对图，图9为本申请实施例提供的另一种具体的图像处理前后效果比对图。其中，a图像和b图像具体可以为在视频直播场景下，主播端获取到的视频流中的视频帧，a1图像和b1图像可以为优化视频流中的视频帧。可以看出，图8中的a1图像相比a图像，较暗部分的人脸进行了提亮，使画面看着清晰自然，而对于背光原图中的明亮部分，自适应的做稍微调整或者不调整。图9中的b1图像相比b图像，对人脸后面较暗的场景自适应的做亮度增强，使得画面看起来清晰自然。
请参考图10，图10为本申请实施例提供的一种优化视频流效果示意图。其中曲线图是小图部分各个像素点的响应曲线，从曲线可以看出，本申请提供的亮度矫正方法的时域响应特性表现良好，此外其在空间域上有连续性和自适应性。
下面对本申请实施例提供的计算机可读存储介质进行介绍，下文描述的计算机可读存储介质与上文描述的图像处理方法可相互对应参照。
本申请还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现上述的图像处理方法的步骤。
该计算机可读存储介质可以包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
本领域技术人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是 软件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应该认为超出本申请的范围。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系属于仅仅用来将一个实体或者操作与另一个实体或者操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语包括、包含或者其他任何变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。
本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (12)

  1. 一种图像处理方法，其特征在于，包括：
    从视频流中获取待处理图像；所述待处理图像为RGB格式；
    将所述待处理图像转换为YUV格式，得到第一图像，并提取所述第一图像对应的Y通道数据；
    利用所述Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数；
    利用所述亮度校正参数对各个像素分别对应的所述Y通道数据进行伽马亮度校正，得到校正后数据；
    利用所述校正后数据替换所述Y通道数据，得到第二图像，并将所述第二图像转换为RGB格式，得到处理后图像；
    将所述处理后图像和历史相邻帧优化图像输入评价模型，得到用于表征所述处理后图像与所述历史相邻帧优化图像之间亮度差距的评价参数；
    确定所述评价参数不处于目标区间，则将所述处理后图像确定为所述待处理图像对应的亮度优化图像。
  2. 根据权利要求1所述的图像处理方法，其特征在于，所述从视频流中获取待处理图像，包括：
    根据历史相邻帧图像从所述视频流中获取初始图像，并对所述初始图像进行归一化处理，得到归一化图像；
    将所述归一化图像映射到线性色彩空间，得到第一中间图像；
    提升所述第一中间图像中各个色彩通道的对比度，得到第二中间图像；
    将所述第二中间图像映射到伽马色彩空间，得到第三中间图像；
    将所述第三中间图像进行反归一化处理，得到所述待处理图像。
  3. 根据权利要求1所述的图像处理方法，其特征在于，所述利用所述Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数，包括：
    对各个所述像素对应的所述Y通道数据按照大小或出现频次排序，得到数据序列；
    将所述数据序列中处于预设比例位置的目标数据确定为所述亮度校正参数。
  4. 根据权利要求1所述的图像处理方法，其特征在于，所述利用所述 Y通道数据，生成所述第一图像各个像素分别对应的亮度校正参数，包括：
    确定各个所述像素对应的临近范围；
    将所述像素和所述临近范围内的临近像素对应的所述Y通道数据按照大小或出现频次排序，得到数据序列；
    将所述数据序列中处于预设比例位置的目标数据确定为所述像素对应的所述亮度校正参数。
  5. 根据权利要求1所述的图像处理方法，其特征在于，所述利用所述亮度校正参数对各个像素分别对应的所述Y通道数据进行伽马亮度校正，得到校正后数据，包括：
    利用最大亮度值与所述亮度校正参数之间的亮度差值，生成与所述亮度差值成反比的幂值，并利用所述幂值和所述Y通道数据进行归一化的伽马计算，得到所述校正后数据。
  6. 根据权利要求5所述的图像处理方法，其特征在于，所述利用最大亮度值与所述亮度校正参数之间的亮度差值，生成与所述亮度差值成反比的幂值，并利用所述幂值和所述Y通道数据进行归一化的伽马计算，得到所述校正后数据，包括：
    利用所述最大亮度值与所述亮度校正参数生成大于零的所述亮度差值；
    利用所述亮度差值和预设阈值生成幂值；所述幂值与所述亮度差值成反比，所述幂值大于零；
    利用所述Y通道数据和所述最大亮度值的比值作为底数，并利用所述底数和所述幂值得到幂函数值；
    将所述幂函数值与所述最大亮度值相乘，得到所述校正后数据。
  7. 根据权利要求1所述的图像处理方法，其特征在于，若所述评价参数处于目标区间，包括：
    更新所述亮度校正参数，以便利用更新后的所述亮度校正参数重新对各个像素分别对应的所述Y通道数据进行伽马亮度校正，得到新的校正后数据，进而利用新的校正后数据得到新的处理后图像，利用新的处理后图像得到新的评价参数，直至新的评价参数不处于目标区间，得到所述亮度优化图像。
  8. 根据权利要求7所述的图像处理方法，其特征在于，所述更新所述亮度校正参数，包括：
    若所述处理后图像比所述历史相邻帧优化图像更暗，则降低所述亮度校正参数；
    若所述处理后图像比所述历史相邻帧优化图像更亮，则提高所述亮度校正参数。
  9. 根据权利要求1所述的图像处理方法，其特征在于，还包括：
    可视化输出所述处理后图像；
    获取用于响应所述处理后图像的用户反馈信息，并根据所述用户反馈信息生成训练数据，并利用所述训练数据对所述评价模型进行追加训练。
  10. 根据权利要求1所述的图像处理方法，其特征在于，还包括：
    利用所述视频流中的所有视频帧对应的处理后图像构成优化视频流，并可视化输出所述优化视频流。
  11. 一种电子设备，其特征在于，包括存储器和处理器，其中：
    所述存储器，用于保存计算机程序；
    所述处理器，用于执行所述计算机程序，以实现如权利要求1至10任一项所述的图像处理方法。
  12. 一种计算机可读存储介质，其特征在于，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至10任一项所述的图像处理方法。
PCT/CN2022/120792 2021-12-03 2022-09-23 图像处理方法、设备及可读存储介质 WO2023098251A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111469660.0 2021-12-03
CN202111469660.0A CN114092360A (zh) 2021-12-03 2021-12-03 图像处理方法、设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2023098251A1 true WO2023098251A1 (zh) 2023-06-08

Family

ID=80306465

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/120792 WO2023098251A1 (zh) 2021-12-03 2022-09-23 图像处理方法、设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN114092360A (zh)
WO (1) WO2023098251A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977214A (zh) * 2023-07-21 2023-10-31 萱闱(北京)生物科技有限公司 图像优化方法、装置、介质和计算设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092360A (zh) * 2021-12-03 2022-02-25 腾讯音乐娱乐科技（深圳）有限公司 图像处理方法、设备及可读存储介质
CN114827723B (zh) * 2022-04-25 2024-04-09 阿里巴巴（中国）有限公司 视频处理方法、装置、电子设备及存储介质
CN117133252A (zh) * 2023-02-27 2023-11-28 荣耀终端有限公司 图像处理方法和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070279500A1 (en) * 2006-06-05 2007-12-06 Stmicroelectronics S.R.L. Method for correcting a digital image
CN105427257A (zh) * 2015-11-18 2016-03-23 四川汇源光通信有限公司 图像增强方法及装置
CN107851421A (zh) * 2015-07-31 2018-03-27 索尼公司 视频信号处理装置、视频信号处理方法和显示装置
CN110111288A (zh) * 2019-04-15 2019-08-09 电子科技大学 基于深度辅助学习的图像增强及盲图质量评价网络
CN114092360A (zh) * 2021-12-03 2022-02-25 腾讯音乐娱乐科技（深圳）有限公司 图像处理方法、设备及可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070279500A1 (en) * 2006-06-05 2007-12-06 Stmicroelectronics S.R.L. Method for correcting a digital image
CN107851421A (zh) * 2015-07-31 2018-03-27 索尼公司 视频信号处理装置、视频信号处理方法和显示装置
CN105427257A (zh) * 2015-11-18 2016-03-23 四川汇源光通信有限公司 图像增强方法及装置
CN110111288A (zh) * 2019-04-15 2019-08-09 电子科技大学 基于深度辅助学习的图像增强及盲图质量评价网络
CN114092360A (zh) * 2021-12-03 2022-02-25 腾讯音乐娱乐科技（深圳）有限公司 图像处理方法、设备及可读存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977214A (zh) * 2023-07-21 2023-10-31 萱闱(北京)生物科技有限公司 图像优化方法、装置、介质和计算设备

Also Published As

Publication number Publication date
CN114092360A (zh) 2022-02-25

Similar Documents

Publication Publication Date Title
WO2023098251A1 (zh) 图像处理方法、设备及可读存储介质
US11158032B2 (en) Perceptually preserving scene-referred contrasts and chromaticities
US8965120B2 (en) Image processing apparatus and method of controlling the same
KR101429352B1 (ko) 장면 화이트 포인트들을 산출하기 위한 노이즈-최적화된 선택 기준의 사용
TWI473039B (zh) 影像的動態範圍壓縮與局部對比增強方法及影像處理裝置
CN105407296B (zh) 实时视频增强方法和装置
US8159616B2 (en) Histogram and chrominance processing
TWI497320B (zh) 色調對應方法及影像處理裝置
US20080123952A1 (en) Contrast optimization of images
CN107154059A (zh) 一种高动态范围视频处理方法
US9307213B2 (en) Robust selection and weighting for gray patch automatic white balancing
CN108769804A (zh) 一种高动态范围视频的格式转换方法
WO2018231968A1 (en) Efficient end-to-end single layer inverse display management coding
CN103380451B (zh) 视频显示装置
CN111669560A (zh) 一种基于fpga的实时自动白平衡校正方法、系统及存储介质
CN107592517B (zh) 一种肤色处理的方法及装置
EP3639238A1 (en) Efficient end-to-end single layer inverse display management coding
JPWO2007069328A1 (ja) 画像処理プログラム、画像処理方法、画像処理装置
TWI462575B (zh) 影像處理裝置及影像處理方法
WO2023241339A1 (zh) 色偏校正方法、装置、设备、存储介质及程序产品
WO2023000868A1 (zh) 图像处理方法及装置、设备、存储介质
CN116368513A (zh) 基于创作意图元数据和环境光的hdr色调映射
KR101903428B1 (ko) 연관 이미지간 컬러 보정 시스템 및 보정방법
Lee et al. Piecewise tone reproduction for high dynamic range imaging
Jung et al. Detail-preserving tone mapping for low dynamic range displays with adaptive gamma correction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22900059

Country of ref document: EP

Kind code of ref document: A1