WO2022052820A1 - 一种数据处理方法、系统及装置 - Google Patents

一种数据处理方法、系统及装置 Download PDF

Info

Publication number
WO2022052820A1
WO2022052820A1 PCT/CN2021/115085 CN2021115085W WO2022052820A1 WO 2022052820 A1 WO2022052820 A1 WO 2022052820A1 CN 2021115085 W CN2021115085 W CN 2021115085W WO 2022052820 A1 WO2022052820 A1 WO 2022052820A1
Authority
WO
WIPO (PCT)
Prior art keywords
parameters
data
image
raw data
rgb image
Prior art date
Application number
PCT/CN2021/115085
Other languages
English (en)
French (fr)
Inventor
张新雨
遇冰
钟钊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022052820A1 publication Critical patent/WO2022052820A1/zh
Priority to US18/182,655 priority Critical patent/US20230222639A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • the present application relates to the technical field of artificial intelligence (AI), and in particular, to a data processing method, system and device.
  • AI artificial intelligence
  • Computational photography is a technique that integrates computer software methods (computer vision, digital signal processing, graphics, etc.) with photographic equipment and related applications. Computational photography combines hardware design with software computing power, which greatly simplifies the photography process, improves the photography experience, and allows more people to enjoy photography.
  • the RAW data all grayscale data of the image recorded by the photosensitive element
  • the RGB data generated by the SLR camera can be used.
  • the acquisition of the RAW-RGB dataset takes a long time, and the gap between the acquired RAW-RGB dataset and the real data is large.
  • the related art proposes to construct a RAW-RGB data set by degrading RGB data into RAW data.
  • this method requires manual calculation of the values of multiple parameters required for degrading RGB data into RAW data. This method is time-consuming and labor-intensive.
  • the present application provides a data processing method, system and device to improve data processing efficiency.
  • the present application provides a data processing method.
  • the data processing method can be executed by a server or a device with a data processing function, which is not specifically limited herein.
  • reference data needs to be obtained first; wherein, the reference data includes: RGB image data, device parameters of the image device, then multiple conversion parameters required for converting the RGB image data into RAW data are determined, and finally, according to multiple Conversion parameters process RGB image data into RAW data; the RAW data matches the device parameters of the imaging device.
  • the above-mentioned image device can be understood as a device with image processing function, which can be a terminal device such as a mobile phone, a tablet computer, a mobile robot, or other devices, which are not specifically limited in this application.
  • the present application converts RGB image data into RAW data using conversion parameters, and converts RGB image data into RAW data based on the conversion parameters.
  • the RAW data obtained in this way can improve the processing efficiency of the data, and the RAW data matches the device parameters of the image device, which is more suitable for the device requirements of the image device.
  • AutoML automated machine learning
  • the present application utilizes AutoML to determine a plurality of conversion parameters required for converting RGB image data into RAW data, and converts RGB image data into RAW data based on the conversion parameters.
  • the RAW data obtained in this way can improve the processing efficiency of the data, and the RAW data matches the device parameters of the image device, which is more suitable for the device requirements of the image device.
  • the search space corresponding to the image device may be determined according to the device parameters; and then a plurality of transformation parameters are determined from the search space.
  • the device parameters of different image devices are different, and the present application can adapt the search space that meets the requirements of the image device according to the device parameters of different image devices, and determine the adaptation to the requirements of the image device within the determined search space.
  • the conversion parameters determined in this way can improve the acquisition efficiency of the conversion parameters, and the conversion parameters are adapted to the needs of the image device, which is conducive to better conversion of RGB image data into RAW data.
  • an image pair of RGB image data and RAW data can be constructed; then the image pair is input into a task processing unit for training to determine a feedback signal; the task processing unit is used to process video or image data; the feedback signal Used to indicate the construction quality of the image pair; finally, according to the feedback signal, update the multiple conversion parameters required to convert the RGB image data into RAW data.
  • the above construction quality reflects the degree of matching between the RAW data and the device parameters of the image device when the RGB image data is converted into RAW data.
  • the conversion parameter can convert the accuracy of the parameter, so that the RAW data can better meet the requirements of the device parameters of the image device.
  • the search space corresponding to the image device includes multiple image processing modules; wherein, the image processing modules include one or more of the following: a noise adding module, a mosaicking module, and a brightness adjustment module .
  • the image processing module in this application is not limited to a noise adding module, a mosaic adding module, and a brightness adjustment module, but may also include a level adjustment module, a white balance adjustment module, and the like.
  • other image processing modules required for converting RGB image data into RAW data are applicable to the present application, and the present application does not specifically limit the number and types of specifically included image processing modules.
  • the conversion parameters include one or more of the following: noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters, and white balance adjustment parameters.
  • the conversion parameters in this application are not limited to noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters and white balance adjustment parameters, and may also include dead pixel correction parameters and the like.
  • the conversion parameters required for converting other RGB image data into RAW data are all applicable to the present application, and the present application does not specifically limit the number and types of conversion parameters specifically included.
  • the present application provides a data processing system, comprising: an image degradation unit and a strategy unit; the image degradation unit is configured to convert RGB image data into a format compatible with an image device according to a plurality of conversion parameters output by the strategy unit RAW data matching device parameters; the strategy unit is used to determine a plurality of conversion parameters required to convert RGB image data into RAW data.
  • the above-mentioned image degradation unit and strategy unit may be understood as an image degradation device and a strategy device, and may also be understood as an image degradation model and a strategy model, which are not specifically limited in this application.
  • the image degeneration unit can convert the RGB image data into RAW data according to the conversion parameters output by the strategy unit, and the image degeneration unit can convert the RGB image data into RAW data based on the conversion parameters.
  • the RAW data obtained through the data processing system can improve the processing efficiency of the data, and the RAW data matches the device parameters of the image device, which is more suitable for the device requirements of the image device.
  • the strategy unit is used to determine a plurality of conversion parameters required for the conversion of RGB image data into RAW data using automated machine learning AutoML.
  • the image degradation unit is further configured to output an image pair of the RGB image data and the RAW data.
  • the method further includes: a task processing unit; the task processing unit is configured to perform training according to the image pair of the RGB image data and the RAW data output by the image degradation unit, and determine a feedback signal , and input the feedback signal to the strategy unit; the feedback signal is used to indicate the construction quality of the image pair.
  • the strategy unit is further configured to receive the feedback signal output by the task processing unit, and adjust the network parameters of the strategy unit according to the feedback signal; update all network parameters based on the adjusted network parameters. multiple conversion parameters.
  • the feedback signal is used to indicate the construction quality of the generated RGB image data and the image pair of the RAW data
  • the network parameters are adjusted based on the feedback signal
  • the RGB image data conversion is updated based on the adjusted network parameters.
  • Multiple conversion parameters required for RAW data can make the RAW data more suitable for the device parameters of the image device.
  • the strategy unit is further configured to determine a search space corresponding to the image device according to the device parameter; and determine the plurality of conversion parameters from the search space.
  • the device parameters of different image devices are different, and the present application can adapt the search space that meets the requirements of the image device according to the device parameters of different image devices, and determine the adaptation to the requirements of the image device within the determined search space.
  • the conversion parameters determined in this way can improve the acquisition efficiency of the conversion parameters, and the conversion parameters are adapted to the needs of the image device, which is conducive to better conversion of RGB image data into RAW data.
  • the image degradation unit includes multiple image processing modules; wherein, the image processing modules include one or more of the following: a noise adding module, a mosaic adding module, and a brightness adjustment module.
  • the image processing module in this application is not limited to a noise adding module, a mosaic adding module, and a brightness adjustment module, but may also include a level adjustment module, a white balance adjustment module, and the like.
  • other image processing modules required for converting RGB image data into RAW data are applicable to the present application, and the present application does not specifically limit the number and types of specifically included image processing modules.
  • the conversion parameters include one or more of the following: noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters, and white balance adjustment parameters.
  • the conversion parameters in this application are not limited to noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters and white balance adjustment parameters, and may also include dead pixel correction parameters and the like.
  • the conversion parameters required for converting other RGB image data into RAW data are all applicable to the present application, and the present application does not specifically limit the number and types of conversion parameters specifically included.
  • the present application provides a data processing apparatus, including a processor and a memory, the memory storing a computer program; the processor being configured to execute the computer program stored in the memory, so that the above-mentioned first The solution described in any implementation of the aspect is performed.
  • the present application provides a computer-readable storage medium, where computer-readable instructions are stored in the computer storage medium, and when the computer reads and executes the computer-readable instructions, the computer is made to execute any of the implementations of the first aspect above. the scheme described.
  • the present application provides a computer program product, which, when the computer reads and executes the computer program product, causes the computer to execute the solution described in any implementation manner of the foregoing first aspect.
  • FIG. 1 shows a schematic structural diagram of a data processing network provided by an embodiment of the present application
  • FIG. 2 shows a schematic structural diagram of an image degradation unit provided by an embodiment of the present application
  • FIG. 3 shows a schematic structural diagram of another data processing network provided by an embodiment of the present application.
  • FIG. 4 shows a schematic flowchart of a data processing method provided by an embodiment of the present application
  • FIG. 5 shows a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application.
  • references in this specification to "one embodiment” or “some embodiments” and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily References to the same embodiment are meant to mean “one or more, but not all, embodiments, but rather a particular feature, structure, or characteristic described in connection with that embodiment.
  • AutoML It is the process of applying machine learning to real-world problems to automate data processing. In practical applications, it can provide users with parameters of different needs according to different tasks. For example: the task is to find the prime numbers in 1000 natural numbers, and the 1000 natural numbers and the task of finding prime numbers are used as the input data of AutoML, then all the prime numbers in the 1000 natural numbers can be output; And the human landscape is classified, and 10,000 images are used as the input data of AutoML, and images can be output according to different categories.
  • RGB image An RGB image consists of a three-dimensional array in the format of M ⁇ N ⁇ 3, where “3” can be understood as three M ⁇ N two-dimensional images (gray value images). These three images represent R, G, and B components respectively, and the pixel value range of each component is [0, 255]. Among them, R stands for Red (red), G stands for Green (green), and B stands for Blue (blue). When the computer defines the color, the value range of the three components of R, G and B is 0-255, 0 means no stimulation, 255 means the stimulation reaches the maximum value. Among them, when R, G, and B are all 255, white light is synthesized, and when R, G, and B are all 0, black light is formed.
  • RAW data It can be understood as a file that records the original information of the sensor of the imaging device, and records some metadata (such as sensitivity settings, shutter speed, aperture value, white balance, etc.) generated by the imaging device.
  • FIG. 1 is a schematic diagram of a data processing system provided by this application, including an image degradation unit and a strategy unit.
  • the data processing system can be integrated in one device or in multiple devices. If integrated in one device, the data processing system
  • the image degradation unit may be an image degradation model, and the strategy unit may be a model for outputting strategies; if integrated in multiple devices, the image degradation unit may be a processing device for processing image degradation data, and the strategy unit is It can be a processing device for outputting an image processing strategy, and this application does not specifically limit whether the unit is a model or a device.
  • the strategy unit can determine multiple conversion parameters required for converting RGB image data into RAW data, for example: use AutoML to determine multiple conversion parameters required for converting RGB image data into RAW data; the image degradation unit can receive RGB Image data and device parameters of the image device, and convert the RGB image data into RAW data matching the device parameters of the image device according to the plurality of conversion parameters output by the strategy unit.
  • the image device can be understood as a device with image processing functions, which can be a terminal device such as a mobile phone, a tablet computer, a mobile robot, or other devices, which are not specifically limited in this application.
  • RGB image data can be images directly downloaded from the network by the data processing system, or images captured by a SLR camera and then imported into the data processing system, or images downloaded from the network and transmitted to the data processing system by an image device.
  • the application does not specifically limit the data source of the RGB image data here.
  • the present application utilizes AutoML to determine multiple conversion parameters required for converting RGB image data into RAW data, and converts RGB image data into RAW data based on the conversion parameters.
  • the RAW data obtained in this way can improve the processing efficiency of the data, and the RAW data determined in the present application matches the device parameters of the image device, which is more suitable for the device requirements of the image device.
  • the strategy unit in this application can also determine the search space corresponding to the image device according to the device parameters; and determine a plurality of transformation parameters from the search space.
  • the conversion parameters determined based on the strategy unit can adaptively adapt to the requirements of the device parameters of the image device. For example, in the device parameters of image device 1, the sensitivity is parameter 1 and the aperture value is parameter 2.
  • the server can determine according to parameter 1 and parameter 2.
  • the search space 1 of the image device 1 is adapted, and a plurality of conversion parameters adapted to the requirements of the image device 1 are determined in the search space 1; among the device parameters of the image device 2, the sensitivity is parameter 2, the aperture value is parameter 4, and the shutter speed is For parameter 3, the server may determine a search space 2 adapted to the image device 2 according to the parameter 1, parameter 2, and parameter 3, and determine a plurality of conversion parameters adapted to the needs of the image device 2 in the search space 2.
  • the image degradation unit may include a plurality of image processing modules; wherein, the image processing modules may include one or more of the following: a noise addition module, a mosaic addition module, and a brightness adjustment module.
  • the conversion parameters may include one or more of the following: noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters, and white balance adjustment parameters.
  • the image degradation unit of the present application converts RGB image data and RAW data, that is, a high-definition image into a low-definition image, it is necessary to perform image processing procedures such as noise addition, mosaic addition, and brightness adjustment. , but in practical application, it may not be limited to the above image processing modules and may also include other image processing modules.
  • the image processing modules that may be involved in the process of combining RGB image data with RAW data are applicable to this application. This application is here Not one by one indicated.
  • the conversion parameters matching the image processing module may include noise addition parameters, mosaic addition parameters, brightness adjustment (such as: increase brightness, reduce brightness) parameters, gamma parameters, level adjustment (such as: add black level, Added white level) parameters and white balance adjustment parameters.
  • the noise may include: Gaussian white noise, additive noise and multiplicative noise; gamma parameters can be used for grayscale adjustment and transparency adjustment; white balance can be used to eliminate the influence of light source on the image, such as increasing the light source by adjusting the white balance adjustment parameter effect on the image.
  • the conversion parameters that may be involved in the process of converting the RGB image data to the RAW data are all applicable to this application, which are not illustrated here in this application.
  • the search space is related to the image processing module in the image degradation unit and the parameter corresponding to each image processing module in the image degradation unit (the parameter is matched with the device parameter of the image device).
  • the conventional image processing process is to remove the noise of the RAW data through the denoising module, remove the mosaic of the RAW data through the demosaic module, and convert the RAW data into RGB image data.
  • the image degradation processing of the present application is to add noise to the RGB image data through a noise adding module, add mosaic to the RGB image data through a mosaic addition module, and convert the RGB image data into RAW data.
  • Image processing such as adding noise and mosaicking is performed on the RGB image data (that is, the image processing operation opposite to the conversion of RAW data into RGB image data).
  • the image degradation unit may further include other image processing modules (illustrated by A module in FIG. 2 ), which are not illustrated one by one here.
  • the search space mentioned in this application is related to each image processing module in the image degradation unit.
  • the image degradation unit for the adapted image device 1 includes 3 image processing modules, such as: a noise adding module, a mosaic adding module and a brightness adjustment module , the search space of the adaptive image device 1 can be determined through the strategy unit, and the search space includes parameters corresponding to the noise adding module, the mosaic adding module and the brightness adjustment module.
  • the conversion parameters corresponding to the noise adding module, the mosaic adding module and the brightness adjusting module may include multiple ones, and the value range of the conversion parameters corresponding to the noise adding module, the mosaic adding module and the brightness adjusting module can be determined according to the device parameters of the image device.
  • the aperture value of the image device 1 is A
  • the strategy unit can know that the value range of the conversion parameter corresponding to the brightness adjustment module is 50-500 according to the aperture value A, not 0 to infinity, so the parameter search range of the strategy unit can be narrowed.
  • the relationship between the parameters of other image processing modules and the parameters of the image device will not be described one by one in this application.
  • the data processing system may further include a task processing unit, as shown in FIG. 3 , the task processing unit may perform training according to the image pair of RGB image data and RAW data output by the image degradation unit, determine a feedback signal, and use the feedback signal to perform training.
  • Input to the strategy unit so that the strategy unit adjusts the network parameters of the strategy unit according to the feedback signal, and updates a plurality of conversion parameters based on the adjusted network parameters.
  • the strategy unit can be understood as a neural network model, and a large number of network parameters related to the model structure are required when the network model is constructed. Multiple conversion parameters.
  • the feedback signal is used to indicate the construction quality of the generated image pair, and the feedback signal can be indicated by the loss value of the test data set or the training data set; the construction quality reflects that when the RGB image data is converted to the RAW data, How well the RAW data matches the device parameters of the imaging device.
  • the task processing unit can be used to process video or image data, such as converting low-resolution images into ultra-high-definition videos, converting low-resolution images into high-resolution images, or converting low-brightness images into high-brightness images etc., which are not specifically limited in this application.
  • the image pair of RGB image data and RAW data is input to the task processing unit for training, the feedback signal obtained is input into the strategy unit, and the strategy unit adaptively adjusts the transformation to be output by the strategy unit according to the feedback signal. parameters, and input the conversion parameters into the image degradation unit to degrade the RGB image data into RAW data. After continuous loop iteration, the construction quality of the image pair indicated by the feedback signal meets the preset requirements.
  • the number of iterations can also be a preset value based on user requirements, such as 500 times.
  • the conversion parameters determined by the strategy unit at the 500th iteration are used as Conversion parameters required by the image degradation unit, and based on the conversion parameters, the RGB image data obtained by the data processing network is converted into RAW data, that is, the conversion parameters determined in the last iteration are used as the final conversion parameters.
  • Different conversion parameters updated based on the feedback signal during the search process of the strategy unit can also be used as the final parameters. For example, when the loss value of the validation data set is used as the feedback signal, the conversion parameter corresponding to the minimum loss value of the validation data set can be selected as the final parameter. final parameter.
  • the strategy unit in this application can determine the conversion parameters through networks such as long short-term memory (LSTM), recurrent neural network (RNN), etc.
  • networks such as long short-term memory (LSTM), recurrent neural network (RNN), etc.
  • LSTM long short-term memory
  • RNN recurrent neural network
  • any network that can determine conversion parameters based on AutoML is applicable to this application.
  • the data processing method may be executed in the above-mentioned data processing network, or may be executed in a server, or a data processing device with data processing function, which is not specifically limited in this application.
  • the execution body of the data processing method can determine the conversion parameters of RGB image data into RAW data based on other methods in practical application, such as: machine learning, deep learning, AutoML, etc.
  • Figure 4 only takes AutoML as an example to illustrate.
  • FIG. 4 takes the execution subject as the server as an example for description, and the server can refer to FIG. 4 to execute:
  • Step 401 Obtain reference data; wherein, the reference data includes: RGB image data and device parameters of the image device.
  • Step 402 using AutoML to determine multiple conversion parameters required for converting the RGB image data into RAW data.
  • Step 403 Process the RGB image data into the RAW data according to the multiple conversion parameters; the RAW data matches the device parameters of the image device.
  • the present application utilizes AutoML to determine multiple conversion parameters required for converting RGB image data into RAW data, and converts RGB image data into RAW data based on the conversion parameters.
  • the RAW data obtained in this way can improve the processing efficiency of the data, and the RAW data matches the device parameters of the image device, which is more suitable for the device requirements of the image device.
  • the server may also determine a search space corresponding to the image device according to the device parameters; and then determine a plurality of conversion parameters from the search space.
  • the device parameters of different image devices are different, and the present application can adapt the search space that meets the requirements of the image device according to the device parameters of different image devices, and determine the adaptation to the requirements of the image device within the determined search space.
  • the conversion parameters determined in this way can improve the acquisition efficiency of the conversion parameters, and the conversion parameters are adapted to the needs of the image device, which is conducive to better converting RGB image data into RAW data, such as implementing the above data processing network.
  • the function of the strategy unit in which is not specifically described in this application.
  • the server can also construct an image pair of RGB image data and RAW data; then, the image pair is input into the task processing unit for training to determine a feedback signal; finally, according to the feedback signal, the required number of RGB image data to be converted into RAW data is updated. conversion parameters; wherein, the task processing unit is used to process video or image data; the feedback signal is used to indicate the construction quality of the image pair.
  • the feedback signal is used to indicate the construction quality of the generated RGB image data and the image pair of the RAW data. Based on the feedback signal, the multiple conversion parameters required to convert the RGB image data into the RAW data can be adjusted. The accuracy of the conversion parameters makes the RAW data more suitable for the device parameters of the image device.
  • the search space corresponding to the image device includes a plurality of image processing modules.
  • the image processing module includes one or more of the following: a noise adding module, a mosaic adding module. The module and the brightness adjustment module will not be repeated in this application.
  • the conversion parameters include one or more of the following: noise addition parameters, mosaic addition parameters, brightness adjustment parameters, gamma parameters, level adjustment parameters, and white balance adjustment parameters, as described in the above data processing network, This application will not repeat them here.
  • a data processing apparatus 500 is provided for this application.
  • the data processing apparatus 500 may be a chip or a system of chips.
  • the chip system may be composed of chips, or may include chips and other discrete devices.
  • the data processing apparatus 500 may include at least one processor 510, and the data processing apparatus 500 may also include at least one memory 520 for storing computer programs, program instructions and/or data.
  • Memory 520 is coupled to processor 510 .
  • the coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which may be in electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.
  • Processor 510 may cooperate with memory 520 .
  • the processor 510 may execute computer programs stored in the memory 520 .
  • at least one of the at least one memory 520 may be included in the processor 510 .
  • the data processing apparatus 500 may further include a transceiver 530, and the data processing apparatus 500 may exchange information with other devices through the transceiver 530.
  • the transceiver 530 can be a circuit, a bus, a transceiver, or any other device that can be used for information exchange.
  • the data processing apparatus 500 may be applied to the foregoing network equipment, and the specific fault analysis apparatus 500 may be the foregoing network equipment, or may be an apparatus capable of supporting the foregoing network equipment to implement any of the foregoing embodiments.
  • the memory 520 holds the necessary computer programs, program instructions and/or data to implement the functions of the network device in any of the above-described embodiments.
  • the processor 510 can execute the computer program stored in the memory 520 to complete the method in any of the foregoing embodiments.
  • connection medium between the transceiver 530 , the processor 510 , and the memory 520 is not limited in the embodiments of the present application.
  • the memory 520, the processor 510, and the transceiver 530 are connected by a bus in FIG. 5.
  • the bus is represented by a thick line in FIG. 5.
  • the connection mode between other components is only for schematic illustration. It is not limited.
  • the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 5, but it does not mean that there is only one bus or one type of bus.
  • the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which may implement or The methods, steps and logic block diagrams disclosed in the embodiments of this application are executed.
  • a general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.
  • the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or may also be a volatile memory (volatile memory), for example Random-access memory (RAM).
  • the memory may also be, but is not limited to, any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and that can be accessed by a computer.
  • the memory in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, for storing computer programs, program instructions and/or data.
  • the embodiments of the present application further provide a readable storage medium, where the readable storage medium stores instructions, and when the instructions are executed, the method for executing the security detection device in any of the above embodiments is implemented .
  • the readable storage medium may include: a USB flash drive, a removable hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk and other media that can store program codes.
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising the instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)

Abstract

本申请提供一种数据处理方法、系统及装置，涉及人工智能AI领域，该数据处理方法可通过服务器来执行，也可通过具有数据处理功能的设备来执行，在执行时，先获取参考数据；其中，参考数据包括：RGB图像数据、图像设备的设备参数，之后确定将RGB图像数据转换成RAW数据所需的多个转换参数，最后根据多个转换参数将RGB图像数据处理成RAW数据；其中，RAW数据与图像设备的设备参数相匹配。由于本申请根据多个转换参数将RGB图像数据转换成RAW数据，并非基于人工经验来确定，该方式提高了数据处理的效率。

Description

一种数据处理方法、系统及装置
相关申请的交叉引用
本申请要求在2020年09月14日提交中国专利局、申请号为202010959989.4、申请名称为“一种数据处理方法、系统及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能(artificial intelligence，AI)技术领域，尤其涉及一种数据处理方法、系统及装置。
背景技术
计算摄影是一种将计算机软件方法(计算机视觉、数字信号处理、图形学等)与摄影设备以及相关应用进行综合的技术。计算摄影将硬件设计与软件计算能力有机结合，极大地简化了摄影流程，提高了摄影体验，让更多的人可以享受到摄影的乐趣。
为了通过终端设备(手机、平板电脑等)得到良好的图像效果，可利用终端设备在拍摄过程产生的RAW数据(感光元件记录的图像的所有灰度数据)和单反相机产生的RGB数据组成的RAW-RGB数据集，用于神经网络学习，来提高终端设备的拍摄能力。但是RAW-RGB数据集的采集耗时较长，且采集的RAW-RGB数据集与真实数据的差距较大。
考虑到数据采集时出现的问题，相关技术提出通过RGB数据退化成RAW数据，来构建RAW-RGB数据集，但是该方式需要人工计算RGB数据退化成RAW数据所需的多个参数的取值，该方式费时费力。
发明内容
基于此，本申请提供一种数据处理方法、系统及装置，以提高数据的处理效率。
第一方面，本申请提供一种数据处理方法，该数据处理方法可通过服务器来执行，也可通过具有数据处理功能的设备来执行，在此不作具体限定。该方法在执行时，需要先获取参考数据；其中，参考数据包括：RGB图像数据、图像设备的设备参数，之后确定将RGB图像数据转换成RAW数据所需的多个转换参数，最后根据多个转换参数将RGB图像数据处理成RAW数据；RAW数据与图像设备的设备参数相匹配。
上述的图像设备可以理解为具有图像处理功能的设备，可以为终端设备如：手机、平板电脑、移动机器人，也可以为其他设备，本申请在此不做具体限定。本申请利用转换参数将RGB图像数据转换成RAW数据，并基于该转换参数将RGB图像数据转换成RAW数据。通过该方式获取的RAW数据可以提高数据的处理效率，且RAW数据与图像设备的设备参数是相匹配的，更加适配图像设备的设备需求。
在一种可能的实现方式中，利用自动化机器学习(auto machine learning，AutoML)确定RGB图像数据转换成RAW数据所需的多个转换参数。本申请利用AutoML确定RGB图像数据转换成RAW数据所需的多个转换参数，并基于该转换参数将RGB图像数据转换 成RAW数据。通过该方式获取的RAW数据可以提高数据的处理效率，且RAW数据与图像设备的设备参数是相匹配的，更加适配图像设备的设备需求。
在一种可能的实现方式中，可根据设备参数确定与图像设备对应的搜索空间；之后从搜索空间确定多个转换参数。
需要说明的是，不同的图像设备的设备参数是不同的，本申请可根据不同的图像设备的设备参数适配符合图像设备需求的搜索空间，并在确定的搜索空间内确定适配图像设备需求的转换参数，通过该方式确定的转换参数可以提高转换参数的获取效率，且该转换参数适配图像设备的需求，有利于更好地将RGB图像数据转换成RAW数据。
在一种可能的实现方式中，可构建RGB图像数据以及RAW数据的图像对；之后将图像对输入任务处理单元中进行训练，确定反馈信号；任务处理单元用于处理视频或图像数据；反馈信号用于指示图像对的构建质量；最后根据反馈信号更新RGB图像数据转换成RAW数据所需的多个转换参数。
需要说明的是，上述构建质量反映的是RGB图像数据转换到RAW数据时，RAW数据与图像设备的设备参数匹配程度，本申请基于反馈信号来调整RGB图像数据转换成RAW数据所需的多个转换参数可以转换参数的准确度，使得RAW数据的更加适配图像设备的设备参数的需求。
在一种可能的实现方式中，图像设备对应的搜索空间包括多个图像处理模块；其中，所述图像处理模块包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块。
需要说明的是，本申请中的图像处理模块并不限于加噪声模块、加马赛克模块以及亮度调整模块，还可包括电平调整模块、白平衡调整模块等。此外，其他RGB图像数据转换成RAW数据所需的图像处理模块均适用于本申请，本申请在此不具体限定具体包括的图像处理模块的数量和类型。
在一种可能的实现方式中，转换参数包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数。
需要说明的是，本申请中的转换参数并不限于加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数，还可包括坏点校正参数等。此外，其他RGB图像数据转换成RAW数据所需的转换参数均适用于本申请，本申请在此不具体限定具体包括的转换参数的数量和类型。
第二方面，本申请提供一种数据处理系统，包括：图像退化单元以及策略单元；所述图像退化单元用于根据所述策略单元输出的多个转换参数将RGB图像数据转换成与图像设备的设备参数相匹配的RAW数据；所述策略单元用于确定将RGB图像数据转换成RAW数据所需的多个转换参数。
需要说明的是，在实际应用时，上述的图像退化单元以及策略单元，可以理解为图像退化装置以及策略装置，还可以理解为图像退化模型以及策略模型，本申请在此不具体限定。本申请中图像退化单元可根据策略单元输出的转换参数将RGB图像数据转换成RAW数据所需的多个转换参数，图像退换单元可基于该转换参数将RGB图像数据转换成RAW数据。通过该数据处理系统获取的RAW数据可以提高数据的处理效率，且RAW数据与图像设备的设备参数是相匹配的，更加适配图像设备的设备需求。
在一种可能的实现方式中，策略单元用于利用自动化机器学习AutoML确定RGB图 像数据转换成RAW数据所需的多个转换参数。
在一种可能的实现方式中，所述图像退化单元还用于输出所述RGB图像数据与所述RAW数据的图像对。
在一种可能的实现方式中，还包括：任务处理单元；所述任务处理单元用于根据所述图像退化单元输出的所述RGB图像数据与所述RAW数据的图像对进行训练，确定反馈信号，并将所述反馈信号输入至所述策略单元；所述反馈信号用于指示所述图像对的构建质量。
在一种可能的实现方式中，所述策略单元还用于接收所述任务处理单元输出的所述反馈信号，并根据反馈信号调整所述策略单元的网络参数；基于调整后的网络参数更新所述多个转换参数。
需要说明的是，本申请中反馈信号是用来指示生成的RGB图像数据以及RAW数据的图像对的构建质量的，基于反馈信号来调整网络参数，并基于调整后的网络参数更新RGB图像数据转换成RAW数据所需的多个转换参数，可以使得RAW数据的更加适配图像设备的设备参数的需求。
在一种可能的实现方式中，所述策略单元还用于根据所述设备参数确定与所述图像设备对应的搜索空间；从所述搜索空间确定所述多个转换参数。
需要说明的是，不同的图像设备的设备参数是不同的，本申请可根据不同的图像设备的设备参数适配符合图像设备需求的搜索空间，并在确定的搜索空间内确定适配图像设备需求的转换参数，通过该方式确定的转换参数可以提高转换参数的获取效率，且该转换参数适配图像设备的需求，有利于更好地将RGB图像数据转换成RAW数据。
在一种可能的实现方式中，所述图像退化单元包括多个图像处理模块；其中，所述图像处理模块包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块。
需要说明的是，本申请中的图像处理模块并不限于加噪声模块、加马赛克模块以及亮度调整模块，还可包括电平调整模块、白平衡调整模块等。此外，其他RGB图像数据转换成RAW数据所需的图像处理模块均适用于本申请，本申请在此不具体限定具体包括的图像处理模块的数量和类型。
在一种可能的实现方式中，所述转换参数包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数。
需要说明的是，本申请中的转换参数并不限于加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数，还可包括坏点校正参数等。此外，其他RGB图像数据转换成RAW数据所需的转换参数均适用于本申请，本申请在此不具体限定具体包括的转换参数的数量和类型。
第三方面，本申请提供一种数据处理装置，包括处理器和存储器，所述存储器，存储有计算机程序；所述处理器，用于执行所述存储器中存储的计算机程序，以使得上述第一方面任一实现方式所述的方案被执行。
第四方面，本申请提供一种计算机可读存储介质，计算机存储介质中存储有计算机可读指令，当计算机读取并执行计算机可读指令时，使得计算机执行上述第一方面任一实现方式所述的方案。
第五方面，本申请提供一种计算机程序产品，当计算机读取并执行计算机程序产品时，使得计算机执行如执行上述第一方面任一实现方式所述的方案。
上述第二方面至第五方面可以达到的技术效果，请参照上述第一方面中相应可能设计方案可以达到的技术效果说明，本申请这里不再重复赘述。
附图说明
图1示出本申请实施例提供的数据处理网络的结构示意图；
图2示出本申请实施例提供的图像退化单元的结构示意图；
图3示出本申请实施例提供的另一数据处理网络的结构示意图；
图4示出了本申请实施例提供的数据处理方法的流程示意图；
图5示出了本申请实施例提供的数据处理装置的结构示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行详尽描述。
如背景技术所述，相关技术虽然提出通过RGB数据退化成RAW数据，来构建RAW-RGB数据集，但是该方式需要人工计算RGB数据退化成RAW数据所需的多个参数的取值，如：RGB数据退化成RAW数据需要经过加噪声、加马赛克的图像处理步骤，假定与加噪声相关的参数有100个，与加马赛克相关的参数有200个，那么RGB数据退化成RAW数最多则需要尝试20000次(100*200)来确定RGB数据退化成RAW数据所需要的参数值，该方式费时费力，且该方式生成的RAW数据也不一定适配不同图像设备的设备需求，故而亟需一种数据处理方法来解决上述的问题。
需要说明的是，本申请中，“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。以及，除非有相反的说明，本申请实施例提及“第一”、“第二”等序数词是用于对多个对象进行区分，不用于限定多个对象的顺序、时序、优先级或者重要程度。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例，而是意味着结合该实施例描述的特定特征、结构或特点。
为了更好地说明本申请的方案，先对本申请涉及的某些专业名词进行简单介绍，具体如下：
AutoML：是将机器学习应用于现实问题自动化数据处理的过程，在实际应用时可以根据不同的任务为用户提供不同需求的参数。如：任务为寻找1000个自然数中的质数，将1000个自然数以及寻找质数的任务作为AutoML的输入数据，则可输出1000个自然数中所有的质数；任务为将1万张图像按照人像、自然风景以及人文景观进行分类，将1万张图像作为AutoML的输入数据，则可按照不同的类别输出图像。
RGB图像：RGB图像由格式为M×N×3的三维数组组成，其中的“3”可以理解为三幅M×N的二维图像(灰度值图像)。这三幅图像分别代表R、G、B分量，每个分量的像素 点取值范围是[0，255]。其中，R代表Red(红色)，G代表Green(绿色)，B代表Blue(蓝色)。计算机定义颜色时R、G、B三种成分的取值范围是0-255，0表示没有刺激量，255表示刺激量达最大值。其中，R、G、B均为255时就合成了白光，R、G、B均为0时就形成了黑色。
RAW数据：可以理解为记录了图像设备传感器的原始信息，同时记录了由图像设备拍摄所产生的一些元数据(如感光度设置、快门速度、光圈值、白平衡等)的文件。
如图1为本申请提供的数据处理系统的示意图，包括图像退化单元以及策略单元，该数据处理系统可以集成在一个设备中，也可以集成在多个设备中，若集成在一个设备中，该图像退化单元可以为图像退化模型，该策略单元可以为用于输出策略的模型；若集成在多个设备中，该图像退化单元则可为用于处理图像退化数据的处理设备，该策略单元则可为用于输出图像处理策略的处理设备，本申请在此不具体限定单元到底为模型还是设备。
需要说明的是，策略单元可确定RGB图像数据转换成RAW数据所需的多个转换参数，如：利用AutoML确定RGB图像数据转换成RAW数据所需的多个转换参数；图像退化单元可接收RGB图像数据以及图像设备的设备参数，并根据策略单元输出的多个转换参数将RGB图像数据转换成与图像设备的设备参数相匹配的RAW数据。其中，图像设备可以理解为具有图像处理功能的设备，可以为终端设备如：手机、平板电脑、移动机器人，也可以为其他设备，本申请在此不做具体限定。RGB图像数据可以为数据处理系统从网络直接下载的图像，也可以为单反相机拍摄的图像后导入到数据处理系统中的，还可以为图像设备从网络下载后传输给数据处理系统的图像，本申请在此不具体限定该RGB图像数据的数据来源。
本申请利用AutoML确定RGB图像数据转换成RAW数据所需的多个转换参数，并基于该转换参数将RGB图像数据转换成RAW数据。通过该方式获取的RAW数据可以提高数据的处理效率，且本申请确定的RAW数据与图像设备的设备参数是相匹配的，更加适配图像设备的设备需求。
另外，本申请中的策略单元还可根据设备参数确定与图像设备对应的搜索空间；从搜索空间确定多个转换参数。基于策略单元确定的转换参数可自适应地适配图像设备的设备参数的需求，如图像设备1的设备参数中感光度为参数1、光圈值为参数2，服务器可根据参数1以及参数2确定适配图像设备1的搜索空间1，并在搜索空间1中确定适配图像设备1需求的多个转换参数；图像设备2的设备参数中感光度为参数2、光圈值为参数4、快门速度为参数3，服务器可根据参数1、参数2以及参数3确定适配图像设备2的搜索空间2，并在搜索空间2中确定适配图像设备2需求的多个转换参数。
此外，图像退化单元可包括多个图像处理模块；其中，图像处理模块可包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块。转换参数可包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数。
需要说明的是，由于本申请的图像退化单元是将RGB图像数据与RAW数据，也即将高清晰度的图像变成低清晰度的图像，需要进行加噪声、加马赛克、亮度调整等图像处理流程，但是在实际应用时可能并不限于上述几种图像处理模块还可能包括其他图像处理模块，在将RGB图像数据与RAW数据过程中可能涉及的图像处理模块均适用于本申请，本申请在此不一一示意。
另外，与图像处理模块相匹配的转换参数，可能包括加噪参数、加马赛克参数、亮度调整(如：提高亮度、降低亮度)参数、伽马参数、电平调整(如：添加黑电平、添加白电平)参数、白平衡调整参数。其中，噪声可能包括：高斯白噪声、加性噪声以及乘性噪声；伽马参数可用于灰度调整、透明度调整；白平衡可用于消除光源对图像的影响，如通过调整白平衡调整参数增加光源对图像的影响。在将RGB图像数据与RAW数据过程中可能涉及的转换参数均适用于本申请，本申请在此不一一示意。
需要说明的是，搜索空间与图像退化单元中图像处理模块，以及图像退化单元中各个图像处理模块对应的参数(该参数是与图像设备的设备参数相匹配的)相关。如图2所示，常规的图像处理流程是通过去噪声模块去除RAW数据的噪声、通过去马赛克模块去除RAW数据的马赛克，将RAW数据转换成RGB图像数据。本申请的图像退化处理是通过加噪声模块对RGB图像数据添加噪声、通过加马赛克模块对RGB图像数据添加马赛克，将RGB图像数据转换成RAW数据。对RGB图像数据进行加噪声、加马赛克等图像处理(也即与RAW数据转换成RGB图像数据相反的图像处理操作)。在实际应用时图像退化单元中可能还包括其他图像处理模块(图2中以A模块来示意)在此不一一示意。本申请提及的搜索空间有图像退化单元中的各个图像处理模块相关，例如：针对适配图像设备1的图像退化单元包括3个图像处理模块如：加噪声模块、加马赛克模块以及亮度调整模块，则可通过策略单元确定适配图像设备1搜索空间，该搜索空间包括与加噪声模块、加马赛克模块以及亮度调整模块对应的参数。此外，加噪声模块、加马赛克模块以及亮度调整模块对应的转换参数可能会包括多个，可根据图像设备的设备参数确定加噪声模块、加马赛克模块以及亮度调整模块对应的转换参数的取值范围，如：图像设备1的光圈值为A，策略单元根据光圈值A可知亮度调整模块对应的转换参数取值范围为50-500，并非0到无穷大，故而可以缩小策略单元的参数搜索范围。针对其他图像处理模块的参数与图像设备参数之间的关系本申请在此不再一一说明。
作为示例，数据处理系统还可包括任务处理单元，如图3所示，该任务处理单元可根据图像退化单元输出的RGB图像数据与RAW数据的图像对进行训练，确定反馈信号，并将反馈信号输入至策略单元，以使策略单元根据反馈信号调整策略单元的网络参数，并基于调整后的网络参数更新多个转换参数。需要说明的是，策略单元可理解为一个神经网络模型，在该网络模型构建时需要大量的与模型结构相关的网络参数，通过调整网络参数可以调整策略单元的输出策略，进而调整策略单元输出的多个转换参数。其中，所述反馈信号用于指示生成的图像对的构建质量，该反馈信号可通过测试数据集或训练数据集的损失值进行指示；该构建质量反映的是RGB图像数据转换到RAW数据时，RAW数据与图像设备的设备参数匹配程度。该任务处理单元可用于处理视频或图像数据，如将低分辨率的变成超高清视频，将低分辨率的图像变成高分辨率的图像，或者将亮度低的图像变成亮度高的图像等，本申请在此不作具体限定。
例如，将RGB图像数据与RAW数据的图像对输入至任务处理单元进行训练，获取的反馈信号，将反馈信号输入到策略单元中，策略单元根据该反馈信号自适应地调整策略单元要输出的转换参数，并将转换参数输入到图像退化单元中将RGB图像数据退化成RAW数据，经过不断的循环迭代，直到反馈信号指示的图像对的构建质量，满足预设的需求为止。迭代的次数也可为根据用户需求的预设值，如为500次，500次迭代之后无论反馈信号的取值为多少则不再进行迭代，将第500次迭代时策略单元确定的转换参数作为图像退 化单元所需的转换参数，并基于该转换参数将数据处理网络获取的RGB图像数据转换成RAW数据，也即将最后一次迭代确定的转换参数作为最终的转换参数。也可以根据策略单元搜索过程中，基于反馈信号更新的不同转换参数作为最终的参数，例如当验证数据集的损失值作为反馈信号时，可以挑选验证数据集的损失值最小时对应的转换参数作为最终参数。
此外，还要说明的是，本申请中策略单元可通过长短期记忆网络(long short-term memory，LSTM)、循环神经网络(recurrent neural network，RNN)等网络来确定转换参数，本申请在此不做具体限定，凡是可基于AutoML确定转换参数的网络均适用于本申请。
接下来介绍使用于本申请的数据处理方法，该数据处理方法可在上述的数据处理网络中执行，也可在服务器，或具有数据处理功能的数据处理设备中执行，本申请在此不具体限定数据处理方法的执行主体，在实际应用时可基于其他方式确定RGB图像数据转换成RAW数据的转换参数，如：机器学习、深度学习、AutoML等，图4仅以AutoML为例来说明。图4以执行主体为服务器为例进行说明，服务器可参照图4执行：
步骤401，获取参考数据；其中，所述参考数据包括：RGB图像数据、图像设备的设备参数。
步骤402，利用AutoML确定RGB图像数据转换成RAW数据所需的多个转换参数。
步骤403，根据多个转换参数将RGB图像数据处理成所述RAW数据；RAW数据与图像设备的设备参数相匹配。
本申请利用AutoML确定RGB图像数据转换成RAW数据所需的多个转换参数，并基于该转换参数将RGB图像数据转换成RAW数据。通过该方式获取的RAW数据可以提高数据的处理效率，且RAW数据与图像设备的设备参数是相匹配的，更加适配图像设备的设备需求。
示例性地，服务器还可根据设备参数确定与图像设备对应的搜索空间；之后从搜索空间确定多个转换参数。
需要说明的是，不同的图像设备的设备参数是不同的，本申请可根据不同的图像设备的设备参数适配符合图像设备需求的搜索空间，并在确定的搜索空间内确定适配图像设备需求的转换参数，通过该方式确定的转换参数可以提高转换参数的获取效率，且该转换参数适配图像设备的需求，有利于更好地将RGB图像数据转换成RAW数据，如实现上述数据处理网络中的策略单元的功能，本申请在此不具体说明。
示例性地，服务器还可构建RGB图像数据以及RAW数据的图像对；之后将图像对输入任务处理单元中进行训练，确定反馈信号；最后根据反馈信号更新RGB图像数据转换成RAW数据所需的多个转换参数；其中，任务处理单元用于处理视频或图像数据；反馈信号用于指示图像对的构建质量。
需要说明的是，本申请中反馈信号是用来指示生成的RGB图像数据以及RAW数据的图像对的构建质量的，基于反馈信号来调整RGB图像数据转换成RAW数据所需的多个转换参数可以转换参数的准确度，使得RAW数据的更加适配图像设备的设备参数的需求。
示例性地，图像设备对应的搜索空间包括多个图像处理模块如上述数据处理网络中的图像退化单元所述，所述图像处理模块包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块，本申请在此不再赘述。
示例性地，转换参数包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整 参数、伽马参数、电平调整参数、白平衡调整参数，如上述数据处理网络所述，本申请在此不再赘述。
基于相同的构思，如图5所示，为本申请提供的一种数据处理装置500。示例性地，数据处理装置500可以是芯片或芯片系统。可选的，在本申请实施例中芯片系统可以由芯片构成，也可以包含芯片和其他分立器件。
数据处理装置500可以包括至少一个处理器510，数据处理装置500还可以包括至少一个存储器520，用于存储计算机程序、程序指令和/或数据。存储器520和处理器510耦合。本申请实施例中的耦合是装置、单元或模块之间的间接耦合或通信连接，可以是电性，机械或其它的形式，用于装置、单元或模块之间的信息交互。处理器510可能和存储器520协同操作。处理器510可能执行存储器520中存储的计算机程序。可选的，所述至少一个存储器520中的至少一个可以包括于处理器510中。
数据处理装置500中还可以包括收发器530，数据处理装置500可以通过收发器530和其它设备进行信息交互。收发器530可以是电路、总线、收发器或者其它任意可以用于进行信息交互的装置。
在一种可能的实施方式中，该数据处理装置500可以应用于前述网络设备，具体故障分析装置500可以是前述网络设备，也可以是能够支持前述网络设备实施上述任一实施例的装置。存储器520保存实施上述任一实施例中的网络设备的功能的必要计算机程序、程序指令和/或数据。所述处理器510可执行所述存储器520存储的计算机程序，完成上述任一实施例中的方法。
本申请实施例中不限定上述收发器530、处理器510以及存储器520之间的具体连接介质。本申请实施例在图5中以存储器520、处理器510以及收发器530之间通过总线连接，总线在图5中以粗线表示，其它部件之间的连接方式，仅是进行示意性说明，并不引以为限。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图5中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
在本申请实施例中，处理器可以是通用处理器、数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件，可以实施或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。
在本申请实施例中，存储器可以是非易失性存储器，比如硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)等，还可以是易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)。存储器还可以是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。本申请实施例中的存储器还可以是电路或者其它任意能够实施存储功能的装置，用于存储计算机程序、程序指令和/或数据。
基于以上实施例，本申请实施例还提供一种可读存储介质，该可读存储介质存储有指令，当所述指令被执行时，使上述任一实施例中安全检测设备执行的方法被实施。该可读存储介质可以包括：U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产 品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请的方法、装置(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理装置的处理器以产生一个机器，使得通过计算机或其他可编程数据处理装置的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理装置以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理装置上，使得在计算机或其他可编程装置上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程装置上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。

Claims (16)

  1. 一种数据处理方法，其特征在于，包括：
    获取参考数据；其中，所述参考数据包括：RGB图像数据、图像设备的设备参数；
    确定将所述RGB图像数据转换成RAW数据所需的多个转换参数；根据所述多个转换参数将所述RGB图像数据处理成所述RAW数据；所述RAW数据与所述图像设备的设备参数相匹配。
  2. 根据权利要求1所述的方法，其特征在于，所述确定将所述RGB图像数据转换成RAW数据所需的多个转换参数，包括：
    利用自动化机器学习AutoML确定所述RGB图像数据转换成RAW数据所需的多个转换参数。
  3. 根据权利要求2所述的方法，其特征在于，所述利用AutoML确定所述RGB图像数据转换成RAW数据所需的多个转换参数，包括：
    根据所述设备参数确定与所述图像设备对应的搜索空间；
    从所述搜索空间确定所述多个转换参数。
  4. 根据权利要求1-3中任一所述的方法，其特征在于，还包括：
    构建所述RGB图像数据以及所述RAW数据的图像对；
    将所述图像对输入任务处理单元中进行训练，确定反馈信号；所述任务处理单元用于处理视频或图像数据；所述反馈信号用于指示所述图像对的构建质量；
    根据所述反馈信号更新所述RGB图像数据转换成所述RAW数据所需的多个转换参数。
  5. 根据权利要求1-4中任一所述的方法，其特征在于，所述图像设备对应的搜索空间包括多个图像处理模块；
    其中，所述图像处理模块包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块。
  6. 根据权利要求1-5中任一所述的方法，其特征在于，所述转换参数包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数。
  7. 一种数据处理系统，其特征在于，包括：图像退化单元以及策略单元；
    所述图像退化单元用于根据所述策略单元输出的多个转换参数将RGB图像数据转换成与图像设备的设备参数相匹配的RAW数据；
    所述策略单元用于确定将所述RGB图像数据转换成RAW数据所需的多个所述转换参数。
  8. 根据权利要求7所述的系统，其特征在于，所述策略单元用于利用自动化机器学习AutoML确定所述RGB图像数据转换成RAW数据所需的多个转换参数。
  9. 根据权利要求7或8所述的系统，其特征在于，所述图像退化单元还用于：
    输出所述RGB图像数据与所述RAW数据的图像对。
  10. 根据权利要求7-9中任一所述的系统，其特征在于，还包括：任务处理单元；
    所述任务处理单元用于根据所述图像退化单元输出的所述RGB图像数据与所述RAW数据的图像对进行训练，确定反馈信号，并将所述反馈信号输入至所述策略单元；所述反馈信号用于指示所述图像对的构建质量。
  11. 根据权利要求7-10中任一所述的系统，其特征在于，所述策略单元还用于：
    接收所述任务处理单元输出的所述反馈信号，并根据所述反馈信号调整所述策略单元的网络参数；
    基于调整后的所述网络参数更新所述多个转换参数。
  12. 根据权利要求7-11中任一所述的系统，其特征在于，所述策略单元还用于：
    根据所述设备参数确定与所述图像设备对应的搜索空间；
    从所述搜索空间确定所述多个转换参数。
  13. 根据权利要求7-12中任一所述的系统，其特征在于，所述图像退化单元包括多个图像处理模块；
    其中，所述图像处理模块包括以下中的一种或多种：加噪声模块、加马赛克模块以及亮度调整模块。
  14. 根据权利要求7-13中任一所述的系统，其特征在于，所述转换参数包括以下中的一种或多种：加噪参数、加马赛克参数、亮度调整参数、伽马参数、电平调整参数以及白平衡调整参数。
  15. 一种数据处理装置，其特征在于，包括：处理器和存储器；
    所述存储器，存储有计算机程序；
    所述处理器，用于执行所述存储器中存储的计算机程序，以使得如权利要求1-6任意一项所述的方法被执行。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机指令，当所述指令在计算机上运行时，使得计算机执行如权利要求1-6任一项所述的方法。
PCT/CN2021/115085 2020-09-14 2021-08-27 一种数据处理方法、系统及装置 WO2022052820A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/182,655 US20230222639A1 (en) 2020-09-14 2023-03-13 Data processing method, system, and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010959989.4A CN114187185A (zh) 2020-09-14 2020-09-14 一种数据处理方法、系统及装置
CN202010959989.4 2020-09-14

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/182,655 Continuation US20230222639A1 (en) 2020-09-14 2023-03-13 Data processing method, system, and apparatus

Publications (1)

Publication Number Publication Date
WO2022052820A1 true WO2022052820A1 (zh) 2022-03-17

Family

ID=80538954

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/115085 WO2022052820A1 (zh) 2020-09-14 2021-08-27 一种数据处理方法、系统及装置

Country Status (3)

Country Link
US (1) US20230222639A1 (zh)
CN (1) CN114187185A (zh)
WO (1) WO2022052820A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115589539A (zh) * 2022-11-29 2023-01-10 荣耀终端有限公司 一种图像调节的方法、设备及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104735354A (zh) * 2015-03-13 2015-06-24 广东欧珀移动通信有限公司 一种拍摄图像的方法及装置
US20190005356A1 (en) * 2017-06-30 2019-01-03 Canon Kabushiki Kaisha Image recognition apparatus, learning apparatus, image recognition method, learning method, and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104735354A (zh) * 2015-03-13 2015-06-24 广东欧珀移动通信有限公司 一种拍摄图像的方法及装置
US20190005356A1 (en) * 2017-06-30 2019-01-03 Canon Kabushiki Kaisha Image recognition apparatus, learning apparatus, image recognition method, learning method, and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KOSKINEN SAMU; YANG DAN; KAMARAINEN JONI-KRISTIAN: "Reverse Imaging Pipeline for Raw RGB Image Augmentation", 2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 22 September 2019 (2019-09-22), pages 2896 - 2900, XP033642290, DOI: 10.1109/ICIP.2019.8804406 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115589539A (zh) * 2022-11-29 2023-01-10 荣耀终端有限公司 一种图像调节的方法、设备及存储介质
CN115589539B (zh) * 2022-11-29 2023-05-09 荣耀终端有限公司 一种图像调节的方法、设备及存储介质

Also Published As

Publication number Publication date
CN114187185A (zh) 2022-03-15
US20230222639A1 (en) 2023-07-13

Similar Documents

Publication Publication Date Title
US10237527B2 (en) Convolutional color correction in digital images
US11995800B2 (en) Artificial intelligence techniques for image enhancement
US10579908B2 (en) Machine-learning based technique for fast image enhancement
Jiang et al. Learning the image processing pipeline
US10949958B2 (en) Fast fourier color constancy
WO2016168145A1 (en) Convolutional color correction
CN113454680A (zh) 图像处理器
CN109191403A (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
CN108616700B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
US20230209029A1 (en) Apparatus and method for white balance editing
US20210390658A1 (en) Image processing apparatus and method
US10600170B2 (en) Method and device for producing a digital image
CN107613216A (zh) 对焦方法、装置、计算机可读存储介质和电子设备
CN116485979B (zh) 映射关系计算方法、颜色校准方法及电子设备
WO2022052820A1 (zh) 一种数据处理方法、系统及装置
CN110807735A (zh) 图像处理方法、装置、终端设备及计算机可读存储介质
CN113658091A (zh) 一种图像评价方法、存储介质及终端设备
WO2020133331A1 (en) Systems and methods for exposure control
CN113034412A (zh) 视频处理方法及装置
CN117156289A (zh) 色彩风格校正方法、系统、电子设备、存储介质及芯片
US9256959B1 (en) Systems and methods for color lens shading correction
Buzzelli et al. Consensus-driven illuminant estimation with GANs
CN111435986B (zh) 源图像数据库的获取方法、训练装置及电子设备
Korus et al. Neural imaging pipelines-the scourge or hope of forensics?
CN113194267B (zh) 图像处理方法及装置、拍照方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21865876

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21865876

Country of ref document: EP

Kind code of ref document: A1