WO2022141476A1 - 一种图像处理方法、数据的获取方法及设备 - Google Patents

一种图像处理方法、数据的获取方法及设备 Download PDF

Info

Publication number
WO2022141476A1
WO2022141476A1 PCT/CN2020/142327 CN2020142327W WO2022141476A1 WO 2022141476 A1 WO2022141476 A1 WO 2022141476A1 CN 2020142327 W CN2020142327 W CN 2020142327W WO 2022141476 A1 WO2022141476 A1 WO 2022141476A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
weight
pixel
data
values
Prior art date
Application number
PCT/CN2020/142327
Other languages
English (en)
French (fr)
Inventor
杨晖
徐海松
鲁洋
叶正男
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202080039169.1A priority Critical patent/CN115643811A/zh
Priority to PCT/CN2020/142327 priority patent/WO2022141476A1/zh
Publication of WO2022141476A1 publication Critical patent/WO2022141476A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction

Definitions

  • the present application relates to the field of computer software, and in particular, to an image processing method, a data acquisition method and related equipment.
  • color imaging devices In the field of digital imaging technology, color imaging devices generally use three spectral channels to capture an image of a subject, for example, red (red, R), green (green, G), and blue (blue, B).
  • red red
  • green green
  • blue blue
  • the spectral information of the illumination light source needs to be used, and currently, the spectral information of the illumination light source is collected by a high-cost special measuring instrument, which is troublesome and costly.
  • the embodiments of the present application provide an image processing method, and provide a solution for directly obtaining spectral information of a light source from image data, which no longer needs to be measured by a high-cost special measuring instrument, and no longer needs to be executed. Additional operations save labor costs.
  • an embodiment of the present application provides an image processing method, which is applied to the field of image processing, and includes: an electronic device obtains first image data of a subject under illumination by an illumination light source, and the first image data includes Image data of the subject in n spectral channels, the first image data includes m pixels, n is an integer greater than 3, and m is an integer greater than or equal to 1.
  • the electronic device acquires first weight data corresponding to the first image data, wherein the first weight data includes m weight values corresponding to the m pixel points one-to-one, and the first weight value corresponding to the first pixel point in the m pixel points.
  • the weight value is higher than the second weight value corresponding to the second pixel among the m pixels, and the saturation of the color corresponding to the first pixel is lower than the saturation of the color corresponding to the second pixel.
  • the first pixel point and the second pixel point are any two different pixel points among the m pixel points included in the first image data, and the concept of the first pixel point and the second pixel point is introduced here to compare the A pixel point and a second pixel point to express the concept of "the weight value of the pixel point corresponding to the color area with lower saturation is higher, and the weight value of the pixel point corresponding to the color area with higher saturation is lower". , but does not mean that the above m pixels are divided into two categories.
  • a solution for directly obtaining the spectral information of the light source from the image data is provided, which no longer needs to be measured by a high-cost special measuring instrument, no longer needs to perform additional operations, and saves labor costs; in addition, Assign a higher weight to the first pixel corresponding to the color with low saturation, assign a lower weight to the second pixel corresponding to the color of high saturation, and adjust the first pixel according to the weight value of each pixel.
  • the size of the pixel values of the first pixel point and the second pixel point in the image data is obtained to obtain the second image data.
  • the pixel points corresponding to the low-saturation colors in the second image data will be more obvious, that is, the second image data.
  • the pixels corresponding to the highly saturated colors in the data cause less interference, and then the spectral information of the illumination light source is generated according to the second image data, because the highly saturated color area of the subject will absorb most of the spectrum of the illumination light source Therefore, it is not easy to extract the spectral information of the lighting source in all spectral bands from the pixels corresponding to the high-saturation color, and because the low-saturation color area of the subject can reflect the lighting source in each band more comprehensively Therefore, it is not easy to extract the spectral information of the lighting source in all spectral bands from the pixels corresponding to the low-saturation colors. Therefore, the second image data can be obtained from the second image data. The more accurate information of the lighting source in n spectral bands spectral information.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, including: the electronic device reduces the first image value according to the first weight data For the pixel values of the first pixel point and the second pixel point in the data, the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point.
  • the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point, so that the pixel point corresponding to the low-saturation color in the second image data can be more
  • the obvious purpose is to reduce the pixel value of the first pixel point and the second pixel point in the first image data to avoid excessive pixel values in the second image data, which is beneficial to reduce the complexity of the subsequent processing process.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, including: the electronic device converts the first image data Multiply the pixel value of the first pixel in the second image data by the first weight value to obtain the pixel value of the first pixel in the second image data; compare the pixel value of the second pixel in the first image data with the second weight value. Multiply to obtain the pixel value of the second pixel in the second image data.
  • a specific implementation manner of adjusting the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data is provided. The multiplication method is implemented, the operation is simple, and it is easy to implement.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain the second image data, including: electronic The device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first weight data to obtain second image data.
  • the image data of each spectral channel can specifically be represented as a p-by-q matrix, and the first image data is represented as a three-dimensional tensor, including n p-by-q matrices;
  • the first weight data may specifically be represented as a matrix of p times q.
  • the electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first weight data, including: the electronic device acquires each spectrum in the first image data one by one For the image data of the channel, a Hadamard product operation is performed on the image data of a single spectral channel and the first weight data.
  • the electronic device generates a second tensor according to the first weight data
  • the second tensor is a three-dimensional tensor, which includes n pieces of first weight data
  • the electronic device directly performs Hadamard on the first image data with the second tensor.
  • the method further includes: the electronic device performs matrix multiplication on the first image data and the first vector to obtain second weight data; wherein the first image data is an m-by-n matrix, The first vector includes n elements, the element values of the n elements included in the first vector are obtained based on pre-training operations and configured in the electronic device, and the second weight data includes m pixels corresponding to m pixels one-to-one. Weights.
  • the electronic device normalizes the m weight values included in the second weight data to generate the first weight data.
  • an implementation manner of generating the first weight data is provided, and since the pixel values in different multispectral images or hyperspectral images are different, the first weighting data corresponding to different first image data is caused.
  • the value ranges of the second weight data are different, and normalizing the second weight data is beneficial to reduce the complexity of the subsequent processing process, thereby improving the accuracy of the spectral information of the finally generated illumination light source.
  • the first image data includes m rows of values, and each row has n values.
  • the electronic device performs matrix multiplication of the first image data and the first vector to obtain the second weight data, including: the electronic device arranges the values of each row in the m rows of values in descending order to obtain a first matrix, The first matrix is multiplied by the first vector to obtain second weight data, and the n values in the first vector are arranged in ascending order.
  • the characteristics of the pixels corresponding to the highly saturated colors are that the pixel values of several spectral channels in the n spectral channels are high, and the pixel values of the remaining spectral channels are very low, which is the same as the high saturation
  • the distribution of the n pixel values of the pixels corresponding to the color of the degree is very uneven.
  • the values of each row in the m rows of values are arranged in descending order, and the n values in the first vector are Arranged in ascending order, it is convenient to distinguish pixels corresponding to colors with high saturation and pixels corresponding to colors with low saturation.
  • a process of matrix multiplication is performed on the first image data and the first vector.
  • the first image data includes m rows of values, each row has n values, and the n values in the first vector are arranged in descending order.
  • the electronic device arranges the values of each row in the m rows of values in ascending order to obtain a first matrix, and multiplies the first matrix and the first vector to obtain second weight data.
  • the method further includes: the electronic device increases the value of the third weight value among the m weight values, and decreases the value of the fourth weight value among the m weight values, To obtain the updated first weight data, the third weight value is a weight value greater than or equal to a preset threshold, and the fourth weight value is a weight value less than the preset threshold.
  • the electronic device may be preconfigured with a segment mapping function, the segment mapping function may include a first function and a second function, and a segment mapping is performed on each of the m weight values included in the first weight data.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, including: the electronic device adjusts the first pixel value in the first image data according to the updated first weight data.
  • the weight value of the pixel corresponding to the color with high saturation is further enlarged, and the weight value of the pixel corresponding to the color with low saturation is reduced.
  • Reflective spectral bands are more susceptible to noise interference, so further reducing the weight of pixels corresponding to highly saturated colors is not only conducive to further reducing redundant pixels (that is, pixels corresponding to highly saturated colors)
  • the interference to the generation of the spectral information of the illumination light source is beneficial to reduce the interference of the noise to the generation of the spectral information of the illumination light source.
  • the first image data is obtained by using an image sensor
  • the electronic device generates spectral information of the illumination light source according to the second image data, including: the electronic device performs feature extraction on the second image data,
  • the first feature information is obtained, and the first feature information may specifically be represented as a vector including n numerical values, where the n numerical values are respectively estimated values of the response values of the image sensor for the illumination light source in the n spectral channels.
  • the electronic device generates spectral information of the illumination light source by solving equations according to the first characteristic information and the spectral sensitivity of the image sensor in n spectral bands, where the n spectral bands correspond to the n spectral channels one-to-one.
  • a specific implementation scheme for generating the spectral information of the illumination light source is provided. First, the response values of the illumination light source in n spectral channels are generated, and then the spectral information of the illumination light source is solved, and a large step is divided into two parts. This small step is beneficial to improve the accuracy of the generation process of the spectral information of the illumination light source.
  • the method further includes: the electronic device generates, according to the first weight data, first label information corresponding to the m pixels, wherein the first label information includes a value corresponding to the m pixels.
  • a corresponding m label values are used to indicate the category of each pixel in the m pixels. If the classification result of the third pixel is the first category, the label value of the third pixel is 0.
  • the label value of the third pixel point is 1, the third pixel point is any pixel point among the m pixel points, and the first category and the second category are different categories; further , the first label information may be expressed as a matrix of p times q, which is the same size as each of the n layers (that is, the image data of the n spectral channels) included in the first image data.
  • the electronic device sets the pixel values of the pixel points of the first category among the m pixel points to zero to obtain the updated first image data.
  • the electronic device enhances the pixel value of the first pixel point in the first image data, and weakens the pixel value of the second pixel point in the first image data, including: the electronic device enhances the pixel value of the first pixel point in the updated first image data.
  • the pixel value of one pixel is enhanced, and the pixel value of the second pixel in the updated first image data is weakened.
  • the first weight data corresponding to the first image data m pixels in the first image data are classified, and then the pixel values of the first class in the first image data are set to zero, The pixel value of the pixel corresponding to the high saturation color in the first image data is set to zero, that is, the redundant information in the first image data is further eliminated, so that more reliable pixel information can be used to generate spectral information of the illumination light source to improve the accuracy of the generated spectral information of the illumination light source.
  • the electronic device sets, according to the first label information, a pixel value of a pixel point of the first category among the m pixel points to zero, including: the electronic device sets the first image data to include: The image data of each spectral channel in the image data of the n spectral channels is subjected to a Hadamard product operation with the first label information.
  • a specific implementation is provided for setting the pixel value of the pixel of the first category among the m pixels to zero according to the first label information, which is implemented by performing a Hadamard product operation, Simple operation and easy implementation.
  • the method further includes: the electronic device uses a third algorithm to generate the spectral reflectance of the object in n spectral bands according to the first image data and the spectral information of the illumination light source.
  • the third algorithm includes but is not limited to least squares algorithm, particle swarm algorithm, Wiener estimation algorithm, pseudo-inverse algorithm, etc.
  • the spectral reflectance of the subject in n spectral bands is used to perform any one of the following operations: Classify the subject, identify the subject, generate a visual image of the subject, analyze the physical characteristics of the subject, analyze the chemical characteristics of the subject, and quantitatively measure the color of the subject.
  • the spectral reflectance of the photographed object in n spectral bands is also generated according to the first image data and the spectral information of the illumination light source, and then the spectral reflectance of the photographed object in the n spectral bands is listed.
  • This application scenario expands the application scenario of this solution and improves the flexibility of this solution.
  • an embodiment of the present application provides a method for acquiring data, which is applied to the field of image processing, including: training a device to acquire training image data of a subject and third weight data corresponding to the training image data, wherein , the training image data includes the image data of the subject in n spectral channels, the first image data includes m pixels, the size and specific expression of each training image data are similar to the first image data; the third weight data includes The labeling weight of each pixel in the m pixels, the weight value of the first pixel in the m pixels is higher than the weight value of the second pixel in the m pixels, and the saturation of the color corresponding to the first pixel. The degree is lower than the saturation of the color corresponding to the second pixel point.
  • the training device performs matrix multiplication of the training image data and the first vector to obtain second weight data, and normalizes the m weight values included in the second weight data to generate the first weight data, wherein the training image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding to m pixels one-to-one.
  • the training device performs iterative training on the element values of the n elements in the first vector according to the objective function, until the preset condition is met, wherein the objective function indicates the similarity between the first weight data and the third weight data, and the first target
  • the function may be the angle difference between the first weight data and the third weight data, or the first objective function may also be an L2 norm function or other types of objective functions;
  • the preset condition may be that the number of training times reaches a preset value times, or the value of the first objective function is less than the preset value.
  • the embodiments of the present application provide an image processing device, which can be used in the field of image processing.
  • the device includes: an acquisition module, configured to acquire first image data of a subject under the illumination of an illumination light source, the first image
  • the data includes the image data of the subject in n spectral channels, the first image data includes m pixels, n is an integer greater than 3, and m is an integer greater than or equal to 1;
  • the adjustment module is used for according to the first weight data , adjust the size of the pixel values of the first pixel point and the second pixel point in the first image data to obtain the second image data, and obtain the second image data, wherein the first weight data includes and the m
  • the m weight values corresponding to the pixel points one-to-one, the first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point
  • the degree of saturation is lower than the saturation degree of the color corresponding to the second
  • the image processing apparatus of the third aspect may also perform other steps performed by the electronic device in the first aspect.
  • steps of the third aspect and various possible implementation manners of the third aspect in the embodiments of the present application and each possible implementation manner For the beneficial effects brought about, reference may be made to the descriptions in the various possible implementation manners in the first aspect, which will not be repeated here.
  • an embodiment of the present application provides a data acquisition device, which can be used in the field of image processing.
  • the device includes: an acquisition module for acquiring training image data of a subject and a third weight corresponding to the training image data data, wherein the training image data includes the image data of the subject in n spectral channels, the training image data includes m pixels, the third weight data includes the labeling weight of each pixel in the m pixels, and m pixels
  • the weight value of the first pixel point in the point is higher than the weight value of the second pixel point in the m pixel points, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point;
  • the module is used to perform matrix multiplication between the training image data and the first vector to obtain the second weight data, and normalize the m weight values included in the second weight data to generate the first weight data, wherein the training The image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes
  • the apparatus for acquiring data in the fourth aspect may also perform other steps performed by the training device in the second aspect.
  • steps of the fourth aspect and various possible implementations of the fourth aspect in the embodiments of the present application and each possible implementation
  • for the beneficial effects brought about by the manner reference may be made to the descriptions in the various possible implementation manners in the second aspect, which will not be repeated here.
  • an embodiment of the present application provides an electronic device, which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and the image of the first aspect is implemented when the program instructions stored in the memory are executed by the processor.
  • Approach For the steps of the processor executing the electronic device in each possible implementation manner of the first aspect, reference may be made to the first aspect for details, and details are not repeated here.
  • an embodiment of the present application provides a training device, which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the image of the second aspect above is realized Approach.
  • a training device which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the image of the second aspect above is realized Approach.
  • an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when it runs on a computer, causes the computer to execute the image processing method of the first aspect, or , so that the computer executes the data acquisition method of the second aspect.
  • an embodiment of the present application provides a circuit system, the circuit system includes a processing circuit, and the processing circuit is configured to execute the image processing method of the first aspect above, or the processing circuit is configured to execute the acquisition of the data of the second aspect. method.
  • an embodiment of the present application provides a computer program that, when running on a computer, causes the computer to execute the image processing method of the first aspect, or causes the computer to execute the data acquisition method of the second aspect.
  • embodiments of the present application provide a chip system, where the chip system includes a processor for implementing the functions involved in the above aspects, for example, sending or processing the data and/or information involved in the above methods.
  • the chip system further includes a memory for storing necessary program instructions and data of the server or the communication device.
  • the chip system may be composed of chips, or may include chips and other discrete devices.
  • Fig. 1 is a kind of structural schematic diagram of artificial intelligence main frame
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of updated first weight data in an image processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of first label information in an image processing method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of first image data and updated first image data in an image processing method provided by an embodiment of the present application
  • FIG. 6 is two schematic diagrams of second image data in the image processing method provided by the embodiment of the present application.
  • FIG. 7 is a schematic diagram of updated first image data and second image data in an image processing method provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of spectral information of an illumination light source in an image processing method provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of first feature information in an image processing method provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a method for acquiring data according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a comparison between the estimated value of the response value of the illumination light source in n spectral channels generated by the image processing method provided by the embodiment of the present application and the real response value of the illumination light source in the n spectral channels;
  • FIG. 12 is a schematic diagram of a comparison between the spectral information of the illumination light source generated by the image processing method provided by the embodiment of the present application and the actual spectral information of the illumination light source;
  • FIG. 13a is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 13b is another schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of an apparatus for acquiring data according to an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • 16 is a schematic structural diagram of a training device provided by an embodiment of the application.
  • FIG. 17 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • Figure 1 shows a schematic structural diagram of the main frame of artificial intelligence.
  • IT value chain
  • vertical axis two dimensions to illustrate the above-mentioned artificial intelligence theme framework.
  • the "intelligent information chain” reflects a series of processes from data acquisition to processing. For example, it can be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data has gone through the process of "data-information-knowledge-wisdom".
  • the "IT value chain” reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecological process of the system.
  • the infrastructure provides computing power support for artificial intelligence systems, realizes communication with the outside world, and supports through the basic platform. Communicate with the outside through sensors; computing power is provided by a smart chip, as an example, the smart chip includes a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processor (graphics unit) processing unit, GPU), application specific integrated circuit (ASIC), field programmable gate array (FPGA) and other hardware acceleration chips; the basic platform includes distributed computing framework and network-related platforms Guarantee and support can include cloud storage and computing, interconnection networks, etc. For example, sensors communicate with external parties to obtain data, and these data are provided to the intelligent chips in the distributed computing system provided by the basic platform for calculation.
  • CPU central processing unit
  • NPU neural-network processing unit
  • graphics processor graphics processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • Guarantee and support can include cloud storage and computing, interconnection networks, etc.
  • sensors communicate with external parties to obtain data, and these data are provided to the intelligent chips in the distributed
  • the data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence.
  • the data involves graphics, images, voice, and text, as well as IoT data from traditional devices, including business data from existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.
  • Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.
  • machine learning and deep learning can perform symbolic and formalized intelligent information modeling, extraction, preprocessing, training, etc. on data.
  • Reasoning refers to the process of simulating human's intelligent reasoning method in a computer or intelligent system, using formalized information to carry out machine thinking and solving problems according to the reasoning control strategy, and the typical function is search and matching.
  • Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.
  • some general capabilities can be formed based on the results of data processing, such as algorithms or a general system, such as translation, text analysis, computer vision processing, speech recognition, image identification, etc.
  • Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, the productization of intelligent information decision-making, and the realization of landing applications. Its application areas mainly include: intelligent terminals, intelligent manufacturing, Smart transportation, smart home, smart healthcare, smart security, autonomous driving, safe city, etc.
  • the embodiments of the present application can be applied to application scenarios in various fields where multispectral images or hyperspectral images need to be processed.
  • the embodiments of the present application can also be applied to intelligent monitoring in the field of intelligent security.
  • the monitoring is a multispectral camera or a hyperspectral camera.
  • the first image data of the subject under the illumination of the lighting source is collected through intelligent monitoring.
  • the first image data is a multispectral image or a hyperspectral image, and the lighting source can be used to From the spectral information of the n spectral bands and the first image data, the spectral reflectance of the object in the n spectral bands is obtained, and then the object can be identified according to the spectral reflectance of the object in the n spectral bands.
  • n is an integer greater than 3. Since the multispectral image or hyperspectral image carries the image data of the subject in multiple spectral channels, it is beneficial to improve the accuracy of the image recognition process.
  • a multi-spectral camera or a hyper-spectral camera may be configured in the smart terminal to collect the first image data of the subject under the illumination of the illumination light source, and use the spectral information of the illumination light source in n spectral bands and The first image data is to obtain the spectral reflectance of the object in n spectral bands, and then generate a visual image of the object (that is, the image in the gallery of the smart terminal for users to view) according to the spectral reflectance of the n spectral bands. image), which is beneficial to improve the reproduction accuracy of the color of the image, and improve the resolution and fidelity of the color of the image.
  • a multi-spectral camera or a hyper-spectral camera may also be configured in the electronic device to collect the first image data of the subject under the illumination of the illumination light source, and use the spectral information of the illumination light source in n spectral bands and the first image data to obtain the spectral reflectance of the object in n spectral bands, and then classify or identify the object according to the spectral reflectance of the object in the n spectral bands, so as to improve the image
  • the application scenarios of the embodiments of the present application are exhaustively enumerated here.
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • the image processing method provided by an embodiment of the present application may include:
  • the electronic device acquires first image data of a subject under illumination by an illumination light source.
  • the electronic device acquires first image data of the subject under the illumination of the illumination light source, where the first image data is multispectral images (MSI) or hyperspectral images, including the subject
  • the first image data is m pixels, n is an integer greater than 3, and m is a positive integer.
  • the electronic device may specifically be represented as a complete device, or as a chip in the complete device.
  • the image data of n spectral channels included in the first image data can be regarded as n layers, the size of the aforementioned n layers is the same, and each image includes m pixel values corresponding to m pixel points one-to-one, Then, the first image data includes n pixel values of each pixel point in the m pixel points.
  • the first image data may be represented by n matrices, each of which has m pixel values, and each of the foregoing matrices may be represented by a matrix of p times q.
  • the electronic device is embodied as a device configured with a multispectral camera or a hyperspectral camera, and the electronic device can use the configured multispectral camera or hyperspectral camera to capture the subject under the illumination of the lighting source. of the first image data.
  • the electronic device is embodied as a chip, and the chip and the multispectral camera/hyperspectral camera are integrated into a complete device, then the chip can pass the multispectral camera or hyperspectral camera in the complete device. Collect first image data of the subject under the illumination of the illumination light source.
  • the electronic device is embodied as a device configured with a memory, and the electronic device may also pre-store first image data, where the first image data is collected by the subject under the illumination of the illumination light source.
  • the electronic device is embodied as a chip, the electronic device is integrated into a complete device, and the complete device is configured with a memory, and the electronic device can acquire the first image data from the memory.
  • the electronic device can also download the first image data through a browser, or the electronic device receives the first image data sent by other electronic devices, etc.
  • the acquisition methods of the first image data are not exhaustive here. .
  • the electronic device acquires first weight data corresponding to the first image data.
  • the electronic device after acquiring the first image data, the electronic device needs to acquire first weight data corresponding to the first image data.
  • the first weight data includes m weight values corresponding to m pixels one-to-one, and the first weight value corresponding to the first pixel among the m pixels is higher than that corresponding to the second pixel among the m pixels.
  • the second weight value of , the saturation of the color corresponding to the first pixel is lower than the saturation of the color corresponding to the second pixel; it should be noted that the first pixel and the second pixel are the first image data.
  • the concept of the first pixel point and the second pixel point is introduced here in order to express "the higher the saturation and the higher the saturation," by comparing the first and second pixel points.
  • This concept does not mean that the above m pixels are divided into two categories. .
  • step 202 may include: the electronic device may convert the first image data from 3-dimensional data to 2-dimensional data, and obtain an m-by-n matrix, that is, a matrix with m rows and n columns, and the first image in the form of 2-dimensional data is obtained.
  • the n pixel values included in each row of an image data represent the pixel values of the same pixel in n layers.
  • the electronic device performs matrix multiplication on the first image data and the first vector to obtain the second weight data, wherein the first vector includes n elements, and the element values of the n elements included in the first vector are obtained based on a pre-training operation and are combined.
  • the training process of the first vector will be described in subsequent embodiments, which will not be repeated here.
  • the second weight data includes m weight values corresponding to m pixels one-to-one, and the second weight data can also be expressed as a matrix of p times q.
  • the electronic device performs normalization processing on each of the m weight values included in the second weight data to generate the first weight data.
  • an implementation manner of generating the first weight data is provided, and since the values of the pixel values in different multispectral images or hyperspectral images are different, resulting in different first image data corresponding to The value ranges of the second weight data are different, and normalizing the second weight data is beneficial to reduce the complexity of the subsequent processing process, thereby improving the accuracy of the spectral information of the finally generated illumination light source.
  • the first image data includes m rows of values, each row has n values, and the n values in the first vector are arranged in ascending order.
  • the electronic device arranges the values of each row in the m rows of values in descending order to obtain a first matrix, and multiplies the first matrix and the first vector to obtain second weight data.
  • the characteristics of the pixels corresponding to the colors with high saturation are that the pixel values of several spectral channels in the n spectral channels are high, and the pixel values of the remaining spectral channels are very low, that is, the pixel values of the remaining spectral channels are very low, that is, the pixel values of the remaining spectral channels are very low.
  • the distribution of the n pixel values of the pixels corresponding to the saturation color is very uneven.
  • the values in each row of the m rows of values are arranged in descending order, and the n values in the first vector In order to arrange in descending order, it is convenient to distinguish the pixel points corresponding to the color of high saturation and the pixel points corresponding to the color of low saturation.
  • the electronic device converts the first image data from 3-dimensional data to 2-dimensional matrix data, it can take the following form:
  • a 11 to a 1n represent the n pixel values corresponding to the first pixel point in the n layers (that is, the image data of the n spectral channels) included in the first image data, and a m1 to a mn represent the first pixel values.
  • n pixel values corresponding to the mth pixel point in the n layers (that is, the image data of n spectral channels) included in an image data
  • a 11 to a 1n can also be expressed as r 1
  • a 21 to a 2n can also be expressed as r 2
  • a m1 to a mn can also be expressed as rm , that is, r 1 to rm can respectively correspond to each row of data in m rows of data included in the 2-dimensional matrix data.
  • the second weight data can be calculated by the following formula:
  • w represents the second weight data
  • M represents the first image data in the form of 2-dimensional data
  • Sort(M) represents that each row of data in the m rows of data included in the first image data is arranged in descending order
  • C represents the first vector
  • Sort(M)*C represents the matrix multiplication of the first image data and the first vector.
  • the first image data includes m rows of values, each row has n values, and the n values in the first vector are arranged in descending order.
  • the electronic device arranges the values of each row in the m rows of values in ascending order to obtain a first matrix, and multiplies the first matrix and the first vector to obtain second weight data.
  • the electronic device may acquire a fifth weight value with the largest value among the m weight values included in the second weight data, and associate each weight value among the m weight values included in the second weight data with the The fifth weight value is divided, that is, normalization processing is performed on each of the m weight values, so as to obtain the second weight data.
  • the formula for normalizing the m weight values included in the second weight data is shown as a formula below:
  • W N represents the first weight data
  • w represents the second weight data
  • Max(w) represents the fifth weight value with the largest value among the m weight values
  • w/Max(w) represents the second weight data including the second weight data.
  • Each weight value in the m weight values is divided by the fifth weight value.
  • the electronic device may further acquire the sum of m weight values included in the second weight data, and compare each weight value of the m weight values included in the second weight data with the sum of the foregoing m weight values and division, that is, performing normalization processing on each of the m weight values, so as to obtain second weight data, etc., the manner of performing normalization on the electronic device is not exhaustive here.
  • the electronic device may also increase the value of the third weight value among the m weight values, and increase the value of the fourth weight value among the m weight values.
  • the value of is reduced to obtain the updated first weight data.
  • the third weight value is a weight value that is greater than or equal to the preset threshold value among the multiple weight values included in the second weight data
  • the fourth weight value is the weight value that is smaller than the preset threshold value among the multiple weight values included in the second weight data.
  • the value of the preset threshold can be between 0.3 and 0.5, as an example, for example, the value of the preset threshold can be 0.35, 0.4, 0.45 or other values, etc.
  • the value of the specific preset threshold can be combined with practical applications It is set according to the scene, which is not limited here.
  • the electronic device may be preconfigured with a segmented mapping function
  • the segmented mapping function may include a first function and a second function, and each of the m weight values included in the first weight data is segmented once mapping to obtain the updated first weight data; that is, when one of the m weight values included in the first weight data is smaller than a preset threshold, the first function is executed, and the m included in the first weight data is When one of the weight values is greater than or equal to the preset threshold, the second function is executed.
  • the implementation is described below by formula:
  • mapping(W N ) refers to performing segmentation mapping on each of the m weight values included in the first weight data. Further, the following formula can be used in the process of segmentation mapping:
  • u represents any weight value in W N (that is, the first weight data)
  • k represents the preset threshold
  • exp represents the exponential function with the base e
  • a and b are two hyperparameters, as an example, For example, the value of a may be 50, the value of b may be 0.6, and the specific values of a and b may be determined in combination with actual application scenarios.
  • formula (5) is only an example to facilitate understanding of this solution, and the pre-stored in the electronic device may also be other types of segment mapping functions.
  • m weights included in the first weight data In the case where one of the weighted values is smaller than the preset threshold, the electronic device divides the aforementioned one weighted value by the first numerical value, and the first numerical value is greater than 1.
  • the value of the first numerical value may be 1.1, 1.2 , 1.3, 2, 5 or other values, etc., are not limited here.
  • the electronic device multiplies the aforementioned one weight value by a second value, and the second value is greater than 1.
  • the value of the second numerical value may be 1.1, 1.2, 1.3, 1.4, 1.5, etc., which is not exhaustive here.
  • the electronic device adds the aforementioned one weight value to a third numerical value, and the third numerical value is a positive number, as
  • the third value may be 0.3, 0.4, 0.5, 0.6, etc., which is not exhaustive here.
  • FIG. 3 includes two sub-schematic diagrams (a) and (b), and sub-schematic diagram (a) in FIG. 3 represents the first image data.
  • the sub-schematic diagram (b) of FIG. 3 represents the updated first weight map of the first image data corresponding to the sub-schematic diagram (a) of FIG.
  • the reflective point is white and the color saturation is the lowest, so the assigned weight value is the highest.
  • the electronic device generates m tag values corresponding to the m pixels one-to-one according to the first weight data.
  • the electronic device may further classify the m pixels according to the first weight data or the updated first weight data, so as to generate the first label information corresponding to the m pixels.
  • the first label information includes m label values corresponding to m pixels one-to-one, and is used to indicate the category of each pixel in the m pixels. If the classification result of the third pixel is the first category, then The label value of the third pixel point is 0. If the classification result of the third pixel point is the second category, the label value of the third pixel point is 1, and the third pixel point is any pixel point among the m pixels.
  • the first category and the second category are different categories, and the pixel with a higher weight value has a greater probability of being classified into the first category; further, the first label information can be expressed as a matrix of p times q, and the first image data includes Each of the n layers (that is, the image data of n spectral channels) has the same size.
  • the electronic device may use a clustering algorithm to classify the m weight values included in the first weight data (or the updated first weight data), that is, classify the m pixels corresponding to the m weight values one-to-one, Get the first classification result.
  • the aforementioned clustering algorithm may be a K-means clustering algorithm, a support vector machine (support vector machines, SVM), a logistic regression algorithm, or other clustering algorithms, etc., which are not exhaustive here.
  • the first classification result may specifically be expressed as a matrix of p times q, which includes m classification values corresponding to m pixels one-to-one.
  • the classification value corresponding to the reliable pixel point (that is, the second category) is 1, and the classification value corresponding to the unreliable pixel point (that is, the first category) is 1. If it is 0, the electronic device can directly determine the generated first classification result as the first label information. In other implementations, the value included in the first classification result cannot be directly determined as the first label information, and the electronic device can convert the first classification result to obtain the first label information.
  • the clustering algorithm adopts the K-means clustering algorithm, and in the obtained first classification result, the classification value of reliable pixels is 1, and the classification value of unreliable pixels is 2.
  • the first label information corresponding to the m pixels is generated by using a binary method. It should be understood that the examples here are only for the convenience of understanding this solution, and are not used to limit this solution.
  • FIG. 4 is a kind of first label information in the image processing method provided by the embodiment of the present application.
  • FIG. 4 includes two sub-schematic diagrams (a) and (b), the sub-schematic diagram (a) of FIG. 4 represents the first image data, and the sub-schematic diagram (b) of FIG. 4 represents the first image data corresponding to the sub-schematic diagram (a) of FIG. 4 .
  • the label map (label map, LM) corresponding to each pixel in the image data since the label value in the first label information has only two values of 1 or 0, the schematic diagram of the first label information only has black and white, and the figure In the sub-schematic diagram (b) of Figure 4, the white area represents the part with the label value of 1, that is, the pixel corresponding to the label value of the white area is a reliable pixel point, and the black area in the sub-schematic (b) of Figure 4 represents the label value.
  • the part that is 0, that is, the pixel corresponding to the label value of the black area is an unreliable pixel, and it should be understood that the example in FIG.
  • the electronic device sets the pixel values of the pixel points whose category is the first category among the m pixel points to zero to obtain updated first image data.
  • step 204 may include: the electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first label information, so as to combine the m pixel points with the image data of each spectral channel and the first label information.
  • the pixel values of the unreliable pixel points are set to zero to obtain the updated first image data.
  • the image data of each spectral channel can be expressed as a p-by-q matrix
  • the first image data can be expressed as a three-dimensional tensor, which includes n p-by-q matrices
  • the first label information can be expressed as A matrix of p by q.
  • the electronic device generates a first tensor according to the first label information, the first tensor is a three-dimensional tensor, which includes n pieces of first label information, and the electronic device directly associates the first image data with the first tensor.
  • a number is subjected to the Hadamard product operation. In order to understand the scheme more intuitively, the formula used for the Hadamard product operation is disclosed as follows:
  • MSI d represents the updated first image data
  • LM 1 represents the first tensor, which is generated based on the first label information
  • MSI represents the first image data
  • It represents that the Hadamard product operation is performed on the first tensor and the first image data.
  • FIG. 5 includes two sub-schematic diagrams (a) and (b), the sub-schematic diagram (a) of FIG. 5 represents the first image data, and the sub-schematic diagram (b) of FIG. 5 represents the updated first image data. It should be noted that, Both the first image data and the updated first image data are invisible, and the first image data and the updated first image data are displayed after being visualized for the convenience of understanding the solution. Comparing the sub-schematic diagram (a) of FIG. 5 and the sub-schematic diagram of FIG.
  • the electronic device can also perform a decompression process on the updated first image data.
  • noise processing to generate denoised first image data.
  • the electronic device may input the updated first image data (that is, the n layers after zeroing the pixel values of the pixel points of the first category) into the low-pass filter respectively, so as to pass the low-pass filtering
  • the denoising operation and the dead pixel removal operation are performed by the processor.
  • the low-pass filter includes, but is not limited to, a Gaussian filter, an average filter, a median filter, or other low-pass filters, etc., which are not limited here.
  • the Gaussian filter used in the low-pass filter is taken as an example, and the
  • MSI e GF5 ⁇ MSI d ⁇ ; (7)
  • MSI e represents the first image data after denoising
  • GF5 represents a Gaussian filter with a window size of 5
  • MSI d represents the updated first image data. It should be noted that in formula (7) The example is only for the convenience of understanding the solution, and other types of low-pass filters may also be used in actual situations, and other values may also be used for the window size, which are not limited here.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain the second image data.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain the first weight data.
  • step 205 It may include: the electronic device reduces the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, and the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point Proportion.
  • the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point, so that the pixel point corresponding to the low-saturation color in the second image data can be more
  • the obvious purpose is to reduce the pixel values of the first pixel point and the second pixel point in the first image data, so as to avoid excessive pixel values in the second image data.
  • step 205 may include: the electronic device multiplies the pixel value of the first pixel in the first image data by the first weight value to obtain the pixel value of the first pixel in the second image data; The pixel value of the second pixel in the image data is multiplied by the second weight value to obtain the pixel value of the second pixel in the second image data.
  • a specific implementation manner of adjusting the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data is provided.
  • the multiplication method is implemented, the operation is simple, and it is easy to implement.
  • the electronic device may perform a Hadamard product operation on the image data of each spectral channel and the first weight data in the image data of the n spectral channels included in the first image data to adjust the first image data.
  • the size of the pixel values of the first pixel point and the second pixel point in the image data is used to obtain the second image data.
  • the image data of each spectral channel can be expressed as a p-by-q matrix
  • the first image data can be expressed as a three-dimensional tensor, which includes n p-by-q matrices
  • the first weight data can be expressed as A matrix of p by q.
  • the electronic device generates a second tensor according to the first weight data
  • the second tensor is a three-dimensional tensor, which includes n pieces of first weight data
  • the electronic device directly associates the first image data with the first image data.
  • Two tensors are subjected to the Hadamard product operation.
  • the electronic device can also directly obtain the target weight value corresponding to the target pixel point from the first weight data, and directly multiply the target pixel point by the target weight value, and the target pixel point is m pixels For any pixel point in the point, the electronic device performs the foregoing operation on each pixel point of the m pixels to adjust the size of the pixel values of the first pixel point and the second pixel point in the first image data.
  • the electronic device adjusts the size of the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data. Specifically, the electronic device reduces the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data, and the reduction ratio of the pixel value of the second pixel point is greater than that of the first pixel point. The rate of decrease in value.
  • step 205 may include: the electronic device multiplies the pixel value of the first pixel in the first image data by the updated first weight value to obtain the pixel value of the first pixel in the second image data; The pixel value of the second pixel in the first image data is multiplied by the updated second weight value to obtain the pixel value of the second pixel in the second image data.
  • the electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the updated first weight data, to perform a Hadamard product operation on the first image data in the first image data.
  • the pixel value of one pixel is enhanced, and the pixel value of the second pixel in the first image data is weakened to obtain second image data.
  • the weight value of the pixel point corresponding to the color with high saturation is further enlarged, and the weight value of the pixel point corresponding to the color with low saturation is reduced.
  • the low-reflection spectral band is more susceptible to noise interference, so further reducing the weight of pixels corresponding to high-saturation colors is not only conducive to further reducing redundant pixels (that is, pixels corresponding to high-saturation colors) ) interferes with generating the spectral information of the illumination light source, and is beneficial to reduce the interference of noise on the spectral information of the generating illumination light source.
  • steps 203 and 204 are optional steps. If steps 203 and 204 are executed, step 205 may include: the electronic device adjusts the adjustment step according to the first weight data (or the updated first weight data) generated in step 202 The size of the pixel values of the first pixel point and the second pixel point in the updated first image data (or the denoised first image data) generated in 204 is used to obtain the second image data.
  • the specific implementation manner of adjusting the pixel value of the pixel point by the electronic device can refer to the above description, and details are not repeated here.
  • the electronic device compares the image data of each spectral channel in the updated first image data (or the denoised first image data) generated in step 204 with the first weight data (or updated first image data) generated in step 202 After the first weight data), a Hadamard product operation is performed to adjust the difference between the first pixel point and the second pixel point in the updated first image data (or the denoised first image data) generated in step 204. The size of the pixel value to obtain the second image data.
  • the first weight data corresponding to the first image data m pixels in the first image data are classified, and then the pixels in the first image data whose class is the first class of pixels are classified.
  • the value is set to zero, so that the pixel value of the pixel corresponding to the high-saturation color in the first image data is set to zero, that is, the redundant information in the first image data is further eliminated, so that a more reliable pixel value can be used.
  • information to generate spectral information of the illumination light source so as to improve the accuracy of the generated spectral information of the illumination light source.
  • step 205 may include: the electronic device adjusts, according to the first weight data (or the updated first weight data) generated in step 202 , in the first image data acquired in step 201 The size of the pixel values of the first pixel point and the second pixel point is used to obtain the second image data.
  • the electronic device performs a Hadamard product operation on the image data of each spectral channel in the first image data obtained in step 201 and the first weight data (or the updated first weight data) generated in step 202,
  • the second image data is obtained by adjusting the size of the pixel values of the first pixel point and the second pixel point in the first image data obtained in step 201 .
  • the sixth represents that according to the first image data and the updated first weight data (that is, increasing the value of the third weight value among the m weight values, the fourth weight value among the m weight values is increased).
  • the second image data generated after the numerical value of the value is reduced) is obtained by performing visualization processing on the aforementioned second image data. Comparing the sub-schematic diagram (a) of FIG. 6 with the sub-schematic diagram (b) of FIG. 6 , the redundant information in the sub-schematic diagram (b) of FIG. 6 is less, and the spectral information of the illumination light source can be more accurately obtained.
  • FIG. 7 takes steps 203 and 204 as an example, and visualizes the updated first image data and the second image data generated in step 204.
  • FIG. 7 which is an embodiment of the present application.
  • FIG. 7 is an example in conjunction with FIG. 3 , that is, the first image data corresponding to FIG. 7 is the sub-schematic diagram (a) of FIG. 3 , FIG. 7 includes two sub-schematic diagrams (a) and (b), and (a) of FIG. 7
  • the sub-schematic diagram represents the updated first image data, and the sub-schematic diagram in (b) of FIG. 7 represents the second image data.
  • the electronic device may determine the spectral information of the illumination light source according to the second image data.
  • the spectral information of the illumination light source is used to indicate the spectral power distribution (spectral power distribution, SPD) of the illumination light source in multiple spectral bands, and the spectral information of the illumination light source may include a one-to-one correspondence with a variety of color lights of different wavelengths. Multiple groups of intensity values, each group of intensity values represents the value of the radiant energy of each color light in the illumination light source.
  • FIG. 8 is a schematic diagram of spectral information of an illumination light source in the image processing method provided by the embodiment of the present application. In Fig.
  • the partial spectral information of the illumination light source is shown in the form of a coordinate diagram as an example. Since the illumination light source is generally a compound light formed by mixing color lights of different wavelengths, Fig. 8 is the spectral power distribution diagram of the illumination light source. Fig. 8 The abscissa represents the wavelength value of each color light in a variety of color lights, and the ordinate in Figure 8 represents the value of the radiant energy of each color light in the illumination light source. It should be noted that the example in Figure 8 is only for the convenience of understanding this scheme. The spectral information of the illumination light source can also be expressed in other ways, which will not be exhaustive here.
  • the first image data is obtained by using an image sensor, and the electronic device performs feature extraction on the second image data to obtain the first feature information.
  • the first feature information may specifically be represented as a vector including n numerical values, where the aforementioned n numerical values are respectively estimated values of the response values of the image sensor for the illumination light source in the n spectral channels.
  • the electronic device generates spectral information of the illumination light source according to the first characteristic information and the spectral sensitivity of the image sensor in the n spectral bands.
  • a specific implementation scheme for generating spectral information of an illumination light source is provided.
  • the response values of the illumination light source in n spectral channels are generated, and then the spectral information of the illumination light source is solved, and a large step is divided into Two small steps are beneficial to improve the accuracy of the generation process of the spectral information of the illumination light source.
  • the electronic device may convert the second image data into data in the form of two-dimensional data, that is, from a three-dimensional tensor (including n matrices, each of which is a p-by-q matrix) into a two-dimensional matrix with m rows and n columns .
  • the electronic device may perform feature extraction on the second image data in the form of two-dimensional data to directly obtain the first feature information.
  • the electronic device may also input the second image data into the feature extraction network, so as to extract the first feature information and the like output by the feature extraction network.
  • Coff represents the second feature information
  • F 2D represents the second image data in the form of two-dimensional data
  • PCA(F 2D ) represents the feature extraction of the second image data in the form of two-dimensional data using the PCF algorithm.
  • the electronic device After obtaining the second feature information through the PCA algorithm, the electronic device uses the first principal component information in the second feature information as the first feature information, that is:
  • FIG. 9 is a schematic diagram of the first feature information in the image processing method provided by the embodiment of the present application.
  • the normalized first feature information is displayed in the form of a coordinate diagram as an example.
  • the abscissa in FIG. 9 represents n spectral channels corresponding to the first image data, and the value of n is used in FIG. 9 .
  • B1, B2, B3, B4, B5, B6, B7, and B8 in Fig. 9 represent the estimated values of the response values of the image sensor for the illumination light source in n spectral channels, respectively.
  • Fig. 9 The examples in are only to facilitate understanding of this solution, and the spectral information of the illumination light source can also be expressed in other ways, which will not be exhaustive here.
  • a process for generating spectral information for an illumination light source After the electronic device obtains the first characteristic information, it can generate the spectral information of the illumination light source by solving the equation according to the first characteristic information and the spectral sensitivity of the image sensor in the n spectral bands, the n spectral bands and the n spectral bands. One-to-one correspondence between channels.
  • MSI represents the first image data
  • R represents the spectral reflectance of the subject to n spectral bands
  • S represents the spectral sensitivity of the image sensor in the n spectral bands
  • L represents the spectral information of the illumination light source.
  • Equation (11) can be equivalent to the following equation (12):
  • ⁇ represents the regularization constraint matrix
  • ⁇ is the hyperparameter, which represents the regularization constraint coefficient
  • I represents the identity matrix
  • D represents the identity matrix
  • diag() represents the generation of the diagonal matrix
  • L CH and S refer to The descriptions in formula (10) and formula (11) are not repeated here.
  • ⁇ can be set as follows:
  • the process of generating the spectral information of the illumination light source by the electronic device according to the first characteristic information and the spectral sensitivity of the image sensor in the n spectral bands can be converted into the spectral information of the image sensor in the n spectral bands according to the first characteristic information and the image sensor.
  • the spectral sensitivity is the process of using the second algorithm to solve the spectral information of the illumination light source.
  • the second algorithm includes but is not limited to least squares algorithm, particle swarm algorithm, genetic algorithm or other algorithms.
  • the spectral information of the illumination light source can be generated by solving the following formula:
  • the electronic device may also use a third algorithm to generate the spectral reflectance of the object in n spectral bands according to the first image data and the spectral information of the illumination light source.
  • the third algorithm includes but is not limited to least squares algorithm, particle swarm algorithm, Wiener estimation algorithm, pseudo-inverse algorithm, etc., which will not be exhaustive here.
  • the spectral reflectance of the subject in the n spectral bands is used to perform any one of the following operations: classify the subject, identify the subject, generate a visual image of the subject, analyze the subject Physical properties of objects, analysis of chemical properties of objects, and quantitative measurement of object color.
  • the electronic device can input the spectral reflectance of the object in the n spectral bands into the neural network for image classification, so that the neural network can identify the target object. Objects are classified.
  • the electronic device may also use other algorithms other than the neural network to classify the photographed object according to the spectral reflectance of the photographed object in n spectral bands.
  • the electronic device can also input the spectral reflectance of the object in n spectral bands into the neural network for image recognition, so as to recognize the object through the neural network.
  • the electronic device may also use other algorithms other than the neural network to identify the subject according to the spectral reflectance of the subject in n spectral bands.
  • the electronic device may also input the spectral reflectance of the object in the n spectral bands into the neural network for analyzing the physical properties, so as to analyze the physical properties of the object through the neural network.
  • the electronic device may also use other algorithms other than the neural network to analyze the physical characteristics of the object according to the spectral reflectance of the object in n spectral bands.
  • the electronic device can also input the spectral reflectance of the object in the n spectral bands into the neural network for chemical property analysis, so as to perform chemical property analysis on the object through the neural network.
  • the electronic device may also use other algorithms other than the neural network to analyze the chemical characteristics of the object according to the spectral reflectance of the object in n spectral bands.
  • the electronic device can use other algorithms besides the neural network to quantitatively measure the color of the object according to the spectral reflectance of the object in n spectral bands.
  • the electronic device can also generate a visual image of the object according to the spectral reflectance of the object in the n spectral bands and the spectral information of the illumination light source.
  • the spectral reflectance of the object in n spectral bands is also generated according to the first image data and the spectral information of the illumination light source, and the spectral reflectance of the object in the n spectral bands is further listed.
  • a variety of application scenarios expand the application scenarios of this solution and improve the flexibility of this solution.
  • a solution for directly acquiring spectral information of a light source from image data, which no longer needs to be measured by a high-cost special measuring instrument, no longer needs to perform additional operations, and saves labor costs; , assign a higher weight to the first pixel corresponding to the color with low saturation, assign a lower weight to the second pixel corresponding to the color of high saturation, and adjust the weight value of each pixel according to the The size of the pixel values of the first pixel point and the second pixel point in the first image data, to obtain the second image data, the pixel points corresponding to the low-saturation color in the second image data will be more obvious, that is, the second image data The pixels corresponding to the highly saturated colors in the image data cause less interference, and then the spectral information of the illumination light source is generated according to the second image data.
  • the highly saturated color area of the subject will absorb most of the illumination light source Therefore, it is not easy to extract the spectral information of the lighting source in all spectral bands from the pixels corresponding to the high-saturation color, and because the low-saturation color area of the subject can reflect the lighting source in each Therefore, it is not easy to extract the spectral information of the lighting source in all spectral bands from the pixels corresponding to the low-saturation colors. Therefore, from the second image data, the relative comparison of the lighting source in the n spectral bands can be obtained. Accurate spectral information.
  • FIG. 10 is a schematic flowchart of a method for acquiring data provided by an embodiment of the present application.
  • the method for acquiring data provided by an embodiment of the present application may include: :
  • the training device acquires training image data of a subject and third weight data corresponding to the training image data.
  • the training device may be pre-configured with a plurality of training image data and third weight data corresponding to each training image data.
  • the training image data includes the image data of the subject in n spectral channels, the first image data includes m pixels, and the size and specific expression of each training image data can be referred to in the corresponding embodiment of FIG.
  • the third weight data includes the labeling weight of each pixel in the m pixels. The weight of the first pixel in the m pixels is higher than the weight of the second pixel in the m pixels, and the weight of the first pixel is higher than that of the first pixel. The saturation of the corresponding color is lower than the saturation of the color corresponding to the second pixel;
  • the training device performs matrix multiplication on the training image data and the first vector to obtain second weight data.
  • the training device may perform matrix multiplication between the training image data and the first vector obtained by initialization, or perform matrix multiplication between the training image data and the first vector generated in the last training process to obtain the second weight data .
  • the training image data is an m-by-n matrix
  • the first vector includes n elements
  • the second weight data includes m weight values corresponding to m pixels one-to-one.
  • the training device performs normalization processing on m weight values included in the second weight data to generate first weight data.
  • step 1002 for a specific implementation manner of step 1002, reference may be made to the description in step 202 in the corresponding embodiment of FIG. 2 , which is not repeated here.
  • the training device after generating the first weight data, the training device generates the function value of the first objective function according to the first weight data generated in step 1003 and the third weight data obtained in step 1001, and according to the first objective
  • the function value of the function is derived by gradient, so as to update the element value of the n elements in the first vector in reverse, so as to complete one training of the element value of the n elements in the first vector.
  • the training device re-enters step 1001 after performing step 1004 to perform the next training on the first vector, and the training device repeatedly performs steps 1001 to 1004 to iteratively train the element values of n elements in the first vector until the According to the preset conditions, the first vector after training is obtained, and the first vector in the embodiment corresponding to FIG. 2 is the first vector after training.
  • the first objective function indicates the similarity between the first weighted data and the third weighted data.
  • the first objective function may be the angle difference between the first weighted data and the third weighted data, or the An objective function can also be an L2 norm function or other types of objective functions.
  • the preset condition may be that the number of times of training reaches the preset number of times, or the value of the first objective function is smaller than the preset value.
  • the electronic device in the embodiment corresponding to FIG. 2 and the training device in the embodiment corresponding to FIG. 10 may be the same device or different devices.
  • a training step of the first vector is also provided, which improves the integrity of the solution.
  • FIG. 11 is a schematic diagram of a comparison between the estimated value of the response value of the illumination light source in n spectral channels generated by the image processing method provided by the embodiment of the present application and the actual response value of the illumination light source in the n spectral channels, FIG. 11 .
  • the abscissa in Figure 11 represents the eight spectral channels, the ordinate in Figure 11 represents the response value in each spectral channel, and the broken line pointed to by C1 represents the lighting source at n n real response values of each spectral channel, the broken line pointed to by C2 represents the estimated value of the n response values of the illumination light source in the n spectral channels, as can be seen from FIG. 11 , the illumination light source generated by the image processing method provided by the embodiment of the present application The accuracy of the estimation of the response values in the n spectral channels is high.
  • FIG. 12 is a schematic diagram of a comparison between the spectral information of the illumination light source generated by the image processing method provided by the embodiment of the present application and the actual spectral information of the illumination light source, and the abscissa of FIG. 12 represents a variety of color lights
  • the wavelength value of each color light in Fig. 8 represents the value of the radiant energy of each color light in the illumination light source
  • the broken line pointed to by D1 represents the real spectral information of the lighting light source
  • the broken line pointed to by D2 represents the use of the
  • the spectral information of the illumination light source generated by the image processing method can be seen from FIG. 12 , the accuracy of the spectral information of the illumination light source generated by the image processing method provided by the embodiment of the present application is relatively high.
  • FIG. 13a is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • the image processing apparatus 1300 may include an acquisition module 1301, an adjustment module 1302, and a determination module 1303, wherein the acquisition module 1301 is used to acquire first image data of the subject under the illumination of the illumination light source, and the first image data includes the subject In the image data of n spectral channels, the first image data includes m pixels, n is an integer greater than 3, and m is an integer greater than or equal to 1; the adjustment module 1302 is configured to adjust the The size of the pixel values of the first pixel point and the second pixel point in the first image data to obtain the second image data, wherein the first weight data includes m weight values corresponding to the m pixel points one-to-one , the first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point is lower than that of the second pixel point The saturation of the corresponding color; the determining module 1303 is configured to determine the spectral information of the lighting source according to the second image data
  • the adjustment module 1302 is specifically configured to reduce the pixel value of the first pixel point and the second pixel point in the first image data according to the first weight data, and the reduction ratio of the pixel value of the second pixel point The drop ratio of pixel values greater than the first pixel point.
  • the adjustment module 1302 is specifically configured to: multiply the pixel value of the first pixel in the first image data by the first weight value to obtain the pixel of the first pixel in the second image data value; multiply the pixel value of the second pixel in the first image data by the second weight value to obtain the pixel value of the second pixel in the second image data.
  • FIG. 13b is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • the apparatus 1300 further includes: a generating module 1304, configured to combine the first image data with the first image data. Perform matrix multiplication on the vector to obtain second weight data, where the first image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding to m pixels one-to-one ;
  • the generating module 1304 is further configured to perform normalization processing on m weight values included in the second weight data to generate the first weight data.
  • the adjustment module 1302 is further configured to increase the value of the third weight value among the m weight values, and decrease the value of the fourth weight value among the m weight values, so as to obtain an update
  • the updated first weight data, the third weight value is a weight value greater than or equal to the preset threshold, and the fourth weight value is a weight value less than the preset threshold value; the adjustment module 1302 is specifically used for according to the updated first weight data , the pixel value of the first pixel point in the first image data is enhanced, and the pixel value of the second pixel point in the first image data is weakened.
  • the first image data is obtained by using an image sensor
  • the determination module 1303 is specifically configured to: perform feature extraction on the second image data to obtain first feature information, where the first feature information is that the image sensor is used for lighting The estimated value of the response value of the light source in the n spectral channels; the spectral information of the illumination light source is generated according to the first feature information and the spectral sensitivity of the image sensor in the n spectral bands, and the n spectral bands correspond to the n spectral channels one-to-one.
  • the apparatus 1300 further includes: a classification module 1305, configured to classify the m pixels according to the first weight data, so as to obtain m corresponding to the m pixels one-to-one A label value is used to indicate the category of the m pixel points; the updating module 1306 is used to set the pixel value of the pixel point whose category is the first category among the m pixel points to zero to obtain the updated first image data; adjust The module 1302 is specifically configured to adjust the size of the pixel values of the first pixel point and the second pixel point in the updated first image data.
  • the apparatus 1300 further includes: a generating module 1304, configured to generate the spectral reflectance of the object in n spectral bands according to the first image data and the spectral information of the illumination light source,
  • the spectral reflectance of the subject in n spectral bands is used to perform any of the following operations: classify the subject, identify the subject, generate a visual image of the subject, analyze the physical properties, analyze the chemical properties of the subject and make quantitative measurements of the subject's color.
  • FIG. 14 is a schematic structural diagram of the apparatus for acquiring data provided by an embodiment of the present application.
  • the data acquisition apparatus 1400 may include an acquisition module 1401 , a generation module 1402 and a training module 1403 .
  • the acquisition module 1401 is used to acquire training image data of the subject and third weight data corresponding to the training image data, wherein the training image data includes the image data of the subject in n spectral channels, and the training image data includes m
  • the third weight data includes the labeling weight of each pixel point in the m pixel points, and the weight value of the first pixel point in the m pixel points is higher than the weight value of the second pixel point in the m pixel points.
  • the saturation of the color corresponding to one pixel is lower than the saturation of the color corresponding to the second pixel;
  • the generating module 1402 is configured to perform matrix multiplication between the training image data and the first vector to obtain the second weight data, and perform a matrix multiplication on the first vector.
  • the m weight values included in the two-weight data are normalized to generate the first weight data, wherein the training image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes and m
  • the m weight values corresponding to the pixel points one-to-one; the training module 1403 is used to iteratively train the element values of the n elements in the first vector according to the objective function, until the preset conditions are met, and the objective function indicates that the first weight data and The similarity between the third weight data.
  • the generating module 1402 is specifically configured to arrange the values of each row in the m rows of values in descending order to obtain a first matrix, and multiply the first matrix by the first vector, The second weight data is obtained, and the n values in the first vector are arranged in descending order.
  • FIG. 15 is a schematic structural diagram of the electronic device provided by the embodiment of the present application.
  • the electronic device 1500 is used to implement the corresponding embodiments of FIGS. 2 to 9 . function of electronic equipment.
  • the electronic device 1500 includes: a receiver 1501, a transmitter 1502, a processor 1503 and a memory 1504 (wherein the number of processors 1503 in the electronic device 1500 may be one or more, and one processor is taken as an example in FIG. 15 ) , wherein the processor 1503 may include an application processor 15031 and a communication processor 15032.
  • the receiver 1501, the transmitter 1502, the processor 1503, and the memory 1504 may be connected by a bus or otherwise.
  • Memory 1504 may include read-only memory and random access memory, and provides instructions and data to processor 1503 .
  • a portion of memory 1504 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1504 stores processors and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations.
  • the processor 1503 controls the operation of the electronic device.
  • various components of an electronic device are coupled together through a bus system, where the bus system may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus.
  • the various buses are referred to as bus systems in the figures.
  • the methods disclosed in the above embodiments of the present application may be applied to the processor 1503 or implemented by the processor 1503 .
  • the processor 1503 may be an integrated circuit chip, which has signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 1503 or an instruction in the form of software.
  • the above-mentioned processor 1503 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and may further include an application specific integrated circuit (ASIC), a field programmable Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • FPGA field programmable Field-programmable gate array
  • the processor 1503 may implement or execute the methods, steps, and logical block diagrams disclosed in the embodiments of this application.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory 1504, and the processor 1503 reads the information in the memory 1504, and completes the steps of the above method in combination with its hardware.
  • the receiver 1501 can be used to receive input digital or character information, and generate signal input related to related settings and function control of the electronic device.
  • the transmitter 1502 can be used to output digital or character information through the first interface; the transmitter 1502 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1502 can also include a display device such as a display screen .
  • FIG. 16 is a schematic structural diagram of a training device provided by an embodiment of the present application.
  • the training device 1600 is used to implement the functions of the training device in the embodiment corresponding to FIG. 10 .
  • the training device 1600 is implemented by one or more servers, and the training device 1600 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPU) 1622 (for example, one or more processors) and memory 1632, one or more storage media 1630 (eg, one or more mass storage devices) that store applications 1642 or data 1644.
  • the memory 1632 and the storage medium 1630 may be short-term storage or persistent storage.
  • the program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instructions to operate on the training device. Further, the central processing unit 1622 may be configured to communicate with the storage medium 1630 to execute a series of instruction operations in the storage medium 1630 on the training device 1600 .
  • Training device 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1658, and/or, one or more operating systems 1641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and many more.
  • operating systems 1641 such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and many more.
  • Embodiments of the present application also provide a computer-readable storage medium, where a program for generating a vehicle speed is stored in the computer-readable storage medium, and when the computer-readable storage medium runs on a computer, the computer executes the programs shown in FIGS. 2 to 9 above.
  • the steps performed by the electronic device in the method described in the illustrated embodiment, or the computer is caused to perform the steps performed by the training device in the method described in the aforementioned embodiment shown in FIG. 10 .
  • the embodiments of the present application also provide a computer program product, which, when running on a computer, causes the computer to perform the steps performed by the electronic device in the methods described in the foregoing embodiments shown in FIG. 2 to FIG.
  • the computer performs the steps performed by the training device in the method described in the aforementioned embodiment shown in FIG. 10 .
  • An embodiment of the present application further provides a circuit system, where the circuit system includes a processing circuit, and the processing circuit is configured to perform the steps performed by the electronic device in the methods described in the foregoing embodiments shown in FIG. 2 to FIG. 9 , or , the processing circuit is configured to perform the steps performed by the electronic device in the method described in the foregoing embodiment shown in FIG. 10 .
  • the image processing apparatus or electronic device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or circuit etc.
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chip executes the image processing methods described in the embodiments shown in FIG. 2 to FIG. get method.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), etc.
  • ROM Read-only memory
  • RAM random access memory
  • FIG. 17 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • the chip may be represented as a neural network processor NPU 170, and the NPU 170 is mounted as a co-processor to the main CPU (Host CPU), tasks are allocated by the Host CPU.
  • the core part of the NPU is the arithmetic circuit 1703, which is controlled by the controller 1704 to extract the matrix data in the memory and perform multiplication operations.
  • the arithmetic circuit 1703 includes multiple processing units (Process Engine, PE). In some implementations, the arithmetic circuit 1703 is a two-dimensional systolic array. The arithmetic circuit 1703 may also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, arithmetic circuit 1703 is a general-purpose matrix processor.
  • the arithmetic circuit fetches the data corresponding to the matrix B from the weight memory 1702 and buffers it on each PE in the arithmetic circuit.
  • the arithmetic circuit fetches the data of matrix A and matrix B from the input memory 1701 to perform matrix operation, and stores the partial result or final result of the matrix in the accumulator 1708 .
  • Unified memory 1706 is used to store input data and output data.
  • the weight data is directly passed through the storage unit access controller (Direct Memory Access Controller, DMAC) 1705, and the DMAC is transferred to the weight memory 1702.
  • Input data is also moved into unified memory 1706 via the DMAC.
  • DMAC Direct Memory Access Controller
  • the BIU is the Bus Interface Unit, that is, the bus interface unit 1710, which is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (Instruction Fetch Buffer, IFB) 1709.
  • IFB Instruction Fetch Buffer
  • the bus interface unit 1710 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1709 to obtain instructions from the external memory, and also for the storage unit access controller 1705 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • the DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 1706 , the weight data to the weight memory 1702 , or the input data to the input memory 1701 .
  • the vector calculation unit 1707 includes a plurality of operation processing units, and further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc., if necessary. It is mainly used for non-convolutional/fully connected layer network computation in neural networks, such as Batch Normalization, pixel-level summation, and upsampling of feature planes.
  • the vector computation unit 1707 can store the vector of processed outputs to the unified memory 1706 .
  • the vector calculation unit 1707 may apply a linear function and/or a non-linear function to the output of the operation circuit 1703, such as linear interpolation of the feature plane extracted by the convolution layer, such as a vector of accumulated values, to generate activation values.
  • the vector computation unit 1707 generates normalized values, pixel-level summed values, or both.
  • the vector of processed outputs can be used as an activation input to the arithmetic circuit 1703, such as for use in subsequent layers in a neural network.
  • the instruction fetch buffer (instruction fetch buffer) 1709 connected to the controller 1704 is used to store the instructions used by the controller 1704;
  • the unified memory 1706, the input memory 1701, the weight memory 1702 and the instruction fetch memory 1709 are all On-Chip memories. External memory is private to the NPU hardware architecture.
  • each layer in the RNN can be performed by the operation circuit 1703 or the vector calculation unit 1707 .
  • the processor mentioned in any one of the above may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the method in the first aspect.
  • the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be A physical unit, which can be located in one place or distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • U disk mobile hard disk
  • ROM read-only memory
  • RAM magnetic disk or optical disk
  • a computer device which may be a personal computer, server, or network device, etc.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • wire eg, coaxial cable, fiber optic, digital subscriber line (DSL)
  • wireless eg, infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server, data center, etc., which includes one or more available media integrated.
  • the usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、数据的获取方法及设备，该方法可应用于图像处理领域，方法包括：获取被摄对象在照明光源的照射下的第一图像数据，第一图像数据包括m个像素点；根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，第一权重数据包括与m个像素点一一对应的m个权重值，第一像素点对应的第一权重值高于第二像素点对应的第二权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度；根据第二图像数据，确定照明光源的光谱信息。提供了从图像数据中直接获取光源的光谱信息的方案，不再需要通过高成本的专用测量仪器进行测量，节省了人工成本。

Description

一种图像处理方法、数据的获取方法及设备 技术领域
本申请涉及计算机软件领域，尤其涉及一种图像处理方法、数据的获取方法及相关设备。
背景技术
在数字成像技术领域，彩色成像设备普遍采用三个光谱通道来采集被摄对象的图像，例如，红(red，R)、绿(green，G)以及蓝(blue，B)。但由于传统的RGB相机的光谱分辨率低，RGB相机成像技术的颜色复现精度还有很大的提升空间。
为了改善RGB相机的成像弊端，目前出现了多光谱相机或高光谱相机，多光谱相机或高光谱相机能够采集到被摄对象在多个光谱通道的图像数据，从而可以提高成像系统的光谱分辨率，也能够实现更高的颜色复现精度。
但由于在对多光谱相机采集的图像数据进行处理的过程中，需要利用照明光源的光谱信息，而目前均为通过高成本的专用测量仪器来采集照明光源的光谱信息，操作麻烦且成本高。
发明内容
有鉴于此，本申请实施例提供了一种图像处理方法，提供了一种从图像数据中直接获取光源的光谱信息的方案，不再需要通过高成本的专用测量仪器进行测量，不再需要执行额外操作，节省了人工成本。
第一方面，本申请实施例提供了一种图像处理方法，该方法应用于图像处理领域，包括：电子设备获取被摄对象在照明光源的照射下的第一图像数据，第一图像数据包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，n为大于3的整数，m为大于或等于1的整数。电子设备获取与第一图像数据对应的第一权重数据，其中，第一权重数据包括与m个像素点一一对应的m个权重值，与m个像素点中第一像素点对应的第一权重值高于与m个像素点中第二像素点对应的第二权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度，需要说明的是，第一像素点和第二像素点为第一图像数据包括的m个像素点中的任意两个不同的像素点，此处引出第一像素点和第二像素点的概念是为了通过对比第一像素点和第二像素点来表达“与饱和度越低的颜色区域对应的像素点的权重值越高，与饱和度越高的颜色区域对应的像素点的权重值越低”这一概念，而不代表将上述m个像素点分为两种类别。电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，根据第二图像数据，确定照明光源的光谱信息，照明光源的光谱信息用于指示照明光源光谱功率分布，照明光源的光谱信息中可以包括与多种不同波长的色光一一对应的多组强度值，每组强度值代表照明光源中每种色光的辐射能的数值。
本实现方式中，提供了一种从图像数据中直接获取光源的光谱信息的方案，不再需要通过高成本的专用测量仪器进行测量，不再需要执行额外操作，且节省了人工成本；此外， 为与低饱和度的颜色对应的第一像素点分配更高的权重，为与高饱和度的颜色对应的第二像素点分配更低的权重，并根据每个像素点的权重值，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，第二图像数据中与低饱和度的颜色对应的像素点会更为明显，也即第二图像数据中与高饱和度颜色对应的像素点所带来的干扰更少，进而根据第二图像数据，生成照明光源的光谱信息，由于被摄对象的高饱和度颜色区域会吸收照明光源大部分光谱波段的光，从而与高饱和度颜色对应的像素点中不容易提取出照明光源在所有光谱波段的光谱信息，而由于被摄对象的低饱和度颜色区域能够较为全面的反射照明光源在各个波段的光，从而从与低饱和度颜色对应的像素点中不容易提取出照明光源在所有光谱波段的光谱信息，因此，从第二图像数据中能够获取到照明光源在n个光谱波段的较为准确的光谱信息。
在第一方面的一种可实现方式中，若第一权重数据为执行过归一化处理的，也即第一图像数据包括的m个像素点中每个像素点所对应的权重值均在0至1之间，则电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，包括：电子设备根据第一权重数据，降低第一图像数据中第一像素点和第二像素点的像素值，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例。本申请实施例中，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例，从而实现了使第二图像数据中与低饱和度的颜色对应的像素点会更为明显的目的，且采用降低第一图像数据中第一像素点和第二像素点的像素值的方式，避免第二图像数据中出现过高的像素值，有利于降低后续处理过程的复杂度。
在第一方面的一种可实现方式中，电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，包括：电子设备将第一图像数据中第一像素点的像素值与第一权重值相乘，以得到第二图像数据中第一像素点的像素值；将第一图像数据中第二像素点的像素值与第二权重值相乘，以得到第二图像数据中第二像素点的像素值。本申请实施例中，提供了根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小的一种具体实现方式，通过将像素点与对应的权重值相乘的方式来实现，操作简单，易于实现。
在第一方面的一种可实现方式中，电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，包括：电子设备将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与第一权重数据进行哈达玛积运算，以得到第二图像数据。
在第一方面的一种可实现方式中，每个光谱通道的图像数据具体可以表现为p乘q的矩阵，第一图像数据表现为三维的张量，其中包括n个p乘q的矩阵；第一权重数据具体可以表现为p乘q的矩阵。电子设备将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与第一权重数据进行哈达玛积运算，包括：电子设备逐个获取第一图像数据中的每个光谱通道的图像数据，将单个光谱通道的图像数据与第一权重数据进行哈达玛积运算。或者，电子设备根据第一权重数据生成第二张量，第二张量为三维的张量，其中包括n个第一权重数据，电子设备将第一图像数据直接与第二张量进行哈达玛积运算。
在第一方面的一种可实现方式中，方法还包括：电子设备将第一图像数据与第一向量 进行矩阵乘法，得到第二权重数据；其中，第一图像数据为m乘n的矩阵，第一向量包括n个元素，第一向量包括的n个元素的元素值为基于预先的训练操作得到并配置于电子设备中的，第二权重数据包括与m个像素点一一对应的m个权重值。电子设备对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据。
本实现方式中，提供了生成第一权重数据的一种实现方式，且由于不同的多光谱图像或高光谱图像中的像素值的取值不同，从而导致与不同的第一图像数据对应的第二权重数据的取值范围不同，将第二权重数据进行归一化处理，有利于降低后续处理过程的复杂度，进而提高最后生成的照明光源的光谱信息的准确度。
在第一方面的一种可实现方式中，第一图像数据包括m行数值，每行有n个数值。电子设备将第一图像数据与第一向量进行矩阵乘法，得到第二权重数据，包括：电子设备将m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据，第一向量中的n个数值为按照从小到大的顺序排列的。
本实现方式中，由于与高饱和度的颜色对应的像素点的特性为n个光谱通道中几个光谱通道的像素值很高，剩余几个光谱通道的像素值很低，也即与高饱和度的颜色对应的像素点的n个像素值分布很不平均，基于前述特性，将m行数值中的每行数值按照从大到小的顺序进行排列，并且第一向量中的n个数值为按照从小到大的顺序排列，便于区分与高饱和度的颜色对应的像素点和与低饱和度的颜色对应的像素点。
在第一方面的一种可实现方式中，针对第一图像数据与第一向量进行矩阵乘法的过程。第一图像数据包括m行数值，每行有n个数值，第一向量中的n个数值为按照从大到小的顺序排列的。电子设备将m行数值中的每行数值按照从小到大的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据。
在第一方面的一种可实现方式中，方法还包括：电子设备将m个权重值中的第三权重值的数值增大，将m个权重值中的第四权重值的数值减小，以得到更新后的第一权重数据，第三权重值为大于或等于预设阈值的权重值，第四权重值为小于预设阈值的权重值。具体的，电子设备可以预先配置有分段映射函数，分段映射函数可以包括第一函数和第二函数，将第一权重数据包括的m个权重值中的每个权重值进行一次分段映射，得到更新后的第一权重数据；也即在第一权重数据包括的m个权重值中的一个权重值小于预设阈值的情况下，执行第一函数，以将m个权重值中的第四权重值的数值减小；在第一权重数据包括的m个权重值中的一个权重值大于或等于预设阈值的情况下，执行第二函数，以将m个权重值中的第三权重值的数值增大。电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，包括：电子设备根据更新后的第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小。
本实现方式中，进一步放大与高饱和度的颜色对应的像素点的权重值，并缩小与低饱和度的颜色对应的像素点的权重值，由于与高饱和度的颜色对应的像素点的低反射光谱波段，更容易受到噪声的干扰，所以进一步降低与高饱和度颜色对应的像素点的权重之后，不仅有利于进一步降低冗余像素点(也即与高饱和度的颜色对应的像素点)对生成照明光源的光谱信息的干扰，且有利于降低噪声对生成照明光源的光谱信息的干扰。
在第一方面的一种可实现方式中，第一图像数据为利用图像传感器得到，电子设备根 据第二图像数据，生成照明光源的光谱信息，包括：电子设备对第二图像数据进行特征提取，得到第一特征信息，第一特征信息具体可以表现为包括n个数值的向量，前述n个数值分别为图像传感器针对照明光源在n个光谱通道的响应值的估计值。电子设备根据第一特征信息和图像传感器在n个光谱波段的光谱灵敏度，通过求解方程式的方式，生成照明光源的光谱信息，n个光谱波段与n个光谱通道一一对应。
本实现方式中，提供了生成照明光源的光谱信息的一种具体实现方案，先生成照明光源在n个光谱通道的响应值，再求解照明光源的光谱信息，将一个大的步骤拆分为两个小的步骤，有利于提高照明光源的光谱信息的生成过程的精度。
在第一方面的一种可实现方式中，方法还包括：电子设备根据第一权重数据，生成与m个像素点对应的第一标签信息，其中，第一标签信息包括与m个像素点一一对应的m个标签值，用于指示m个像素点中每个像素点的类别，若第三像素点的分类结果为第一类别，则第三像素点的标签值为0，若第三像素点的分类结果为第二类别，则第三像素点的标签值为1，第三像素点为m个像素点中任一个像素点，第一类别和第二类别为不同的类别；进一步地，第一标签信息可以表现为p乘q的矩阵，与第一图像数据包括的n个图层(也即n个光谱通道的图像数据)中的每个图层的尺寸相同。电子设备将m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据。电子设备对第一图像数据中的第一像素点的像素值进行增强，对第一图像数据中第二像素点的像素值进行削弱，包括：电子设备对更新后的第一图像数据中的第一像素点的像素值进行增强，对更新后的第一图像数据中的第二像素点的像素值进行削弱。
本实现方式中，根据与第一图像数据对应的第一权重数据，将第一图像数据中的m个像素点进行分类，继而将第一图像数据中类别为第一类别的像素值置零，以将第一图像数据中与高饱和度颜色对应的像素点的像素值置零，也即进一步消除第一图像数据中的冗余信息，从而可以利用更为可靠的像素点的信息，来生成照明光源的光谱信息，以提高生成的照明光源的光谱信息的准确度。
在第一方面的一种可实现方式中，电子设备根据第一标签信息，将m个像素点中类别为第一类别的像素点的像素值置零，包括：电子设备将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与第一标签信息进行哈达玛积运算。
本实现方式中，提供了根据第一标签信息，将m个像素点中类别为第一类别的像素点的像素值置零的一种具体实现方式，通过执行哈达玛积运算的方式来实现，操作简单，易于实现。
在第一方面的一种可实现方式中，方法还包括：电子设备根据第一图像数据和照明光源的光谱信息，采用第三算法，生成被摄对象在n个光谱波段的光谱反射比。其中，第三算法包括但不限于最小二乘算法、粒子群算法、维纳估计算法、伪逆算法等，被摄对象在n个光谱波段的光谱反射比用于执行以下中任一项操作：对被摄对象进行分类、对被摄对象进行识别、生成被摄对象的可视化图像、对分析被摄对象的物理特性、分析被摄对象的化学特性和对被摄对象进行颜色的定量测量。
本实现方式中，还会根据第一图像数据和照明光源的光谱信息，生成被摄对象在n个光谱波段的光谱反射比，进而列举了被摄对象在n个光谱波段的光谱反射比的多种应用场 景，扩展了本方案的应用场景，提高了本方案的灵活性。
第二方面，本申请实施例提供了一种数据的获取方法，该方法应用于图像处理领域，包括：训练设备获取被摄对象的训练图像数据和与训练图像数据对应的第三权重数据，其中，训练图像数据包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，每个训练图像数据的尺寸以及具体表现形式与第一图像数据类似；第三权重数据包括m个像素点中每个像素点的标注权重，m个像素点中第一像素点的权重值高于m个像素点中第二像素点的权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度。训练设备将训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，并对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据，其中，训练图像数据为m乘n的矩阵，第一向量包括n个元素，第二权重数据包括与m个像素点一一对应的m个权重值。训练设备根据目标函数，对第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，其中，目标函数指示第一权重数据与第三权重数据之间的相似度，第一目标函数可以为第一权重数据与第三权重数据之间的角度差，或者，第一目标函数还可以为L2范数函数、或其他类型的目标函数；预设条件可以为训练的次数达到预设次数，或者第一目标函数的数值小于预设数值。
在第二方面的一种可实现方式中，训练图像数据包括m行数值，每行有n个数值。训练设备将训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，包括：训练设备将m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据，第一向量中的n个数值为按照从大到小的顺序排列的。
本申请实施例第二方面以及第二方面的各种可能实现方式的具体实现步骤，以及每种可能实现方式所带来的有益效果，均可以参考第一方面中各种可能的实现方式中的描述，此处不再一一赘述。
第三方面，本申请实施例提供了一种图像处理装置，可用于图像处理领域中，装置包括：获取模块，用于获取被摄对象在照明光源的照射下的第一图像数据，第一图像数据包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，n为大于3的整数，m为大于或等于1的整数；调整模块，用于根据第一权重数据，调整所述第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，得到第二图像数据，其中，所述第一权重数据包括与所述m个像素点一一对应的m个权重值，所述第一像素点对应的第一权重值高于所述第二像素点对应的第二权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；确定模块，用于根据第二图像数据，确定照明光源的光谱信息，照明光源的光谱信息指示照明光源光谱功率分布。
第三方面的图像处理装置还可以执行第一方面中电子设备执行的其他步骤，对于本申请实施例第三方面以及第三方面的各种可能实现方式的具体实现步骤，以及每种可能实现方式所带来的有益效果，均可以参考第一方面中各种可能的实现方式中的描述，此处不再一一赘述。
第四方面，本申请实施例提供了一种数据的获取装置，可用于图像处理领域中，装置包括：获取模块，用于获取被摄对象的训练图像数据和与训练图像数据对应的第三权重数据，其中，训练图像数据包括被摄对象在n个光谱通道的图像数据，训练图像数据包括m 个像素点，第三权重数据包括m个像素点中每个像素点的标注权重，m个像素点中第一像素点的权重值高于m个像素点中第二像素点的权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度；生成模块，用于将训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，并对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据，其中，训练图像数据为m乘n的矩阵，第一向量包括n个元素，第二权重数据包括与m个像素点一一对应的m个权重值；训练模块，用于根据目标函数，对第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，目标函数指示第一权重数据与第三权重数据之间的相似度。
第四方面的数据的获取装置还可以执行第二方面中训练设备执行的其他步骤，对于本申请实施例第四方面以及第四方面的各种可能实现方式的具体实现步骤，以及每种可能实现方式所带来的有益效果，均可以参考第二方面中各种可能的实现方式中的描述，此处不再一一赘述。
第五方面，本申请实施例提供了一种电子设备，可以包括处理器，处理器和存储器耦合，存储器存储有程序指令，当存储器存储的程序指令被处理器执行时实现上述第一方面的图像处理方法。对于处理器执行第一方面的各个可能实现方式中电子设备的步骤，具体均可以参阅第一方面，此处不再赘述。
第六方面，本申请实施例提供了一种训练设备，可以包括处理器，处理器和存储器耦合，存储器存储有程序指令，当存储器存储的程序指令被处理器执行时实现上述第二方面的图像处理方法。对于处理器执行第二方面的各个可能实现方式中训练设备的步骤，具体均可以参阅第二方面，此处不再赘述。
第七方面，本申请实施例提供了一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行上述第一方面的图像处理方法，或者，使得计算机执行上述第二方面的数据的获取方法。
第八方面，本申请实施例提供了一种电路系统，电路系统包括处理电路，处理电路配置为执行上述第一方面的图像处理方法，或者，处理电路配置为执行上述第二方面的数据的获取方法。
第九方面，本申请实施例提供了一种计算机程序，当其在计算机上运行时，使得计算机执行上述第一方面的图像处理方法，或者，使得计算机执行上述第二方面的数据的获取方法。
第十方面，本申请实施例提供了一种芯片系统，该芯片系统包括处理器，用于实现上述方面中所涉及的功能，例如，发送或处理上述方法中所涉及的数据和/或信息。在一种可能的设计中，芯片系统还包括存储器，存储器，用于保存服务器或通信设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。
附图说明
图1为人工智能主体框架的一种结构示意图；
图2为本申请实施例提供的图像处理方法的一种流程示意图；
图3为本申请实施例提供的图像处理方法中更新后的第一权重数据的一种示意图；
图4为本申请实施例提供的图像处理方法中第一标签信息的一种示意图；
图5为本申请实施例提供的图像处理方法中第一图像数据和更新后的第一图像数据的一种示意图；
图6为本申请实施例提供的图像处理方法中第二图像数据的两种示意图；
图7为本申请实施例提供的图像处理方法中更新后的第一图像数据和第二图像数据的一种示意图；
图8为本申请实施例提供的图像处理方法中照明光源的光谱信息的一种示意图；
图9为本申请实施例提供的图像处理方法中第一特征信息的一种示意图；
图10为本申请实施例提供的数据的获取方法的一种流程示意图；
图11为采用本申请实施例提供的图像处理方法生成的照明光源在n个光谱通道的响应值的估计值与照明光源在n个光谱通道的真实响应值之间的一种对比示意图；
图12为采用本申请实施例提供的图像处理方法生成的照明光源的光谱信息与照明光源的实际的光谱信息之间的一种对比示意图；
图13a为本申请实施例提供的图像处理装置的一种结构示意图；
图13b为本申请实施例提供的图像处理装置的另一种结构示意图；
图14为本申请实施例提供的数据的获取装置的一种结构示意图；
图15为本申请实施例提供的电子设备的一种结构示意图；
图16为本申请实施例提供的训练设备一种结构示意图；
图17为本申请实施例提供的芯片的一种结构示意图。
具体实施方式
下面结合附图，对本申请的实施例进行描述。本领域普通技术人员可知，随着技术的发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
首先对人工智能系统总体工作流程进行描述，请参见图1，图1示出的为人工智能主体框架的一种结构示意图，下面从“智能信息链”(水平轴)和“信息技术(information technology，IT)价值链”(垂直轴)两个维度对上述人工智能主题框架进行阐述。其中，“智能信息链”反映从数据的获取到处理的一列过程。举例来说，可以是智能信息感知、智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在这个过程中，数据经历了“数据—信息—知识—智慧”的凝练过程。“IT价值链”从人智能的底层基础设施、信息(提供和处理技术实现)到系统的产业生态过程，反映人工智能为信息技术产业带来的价值。
(1)基础设施
基础设施为人工智能系统提供计算能力支持，实现与外部世界的沟通，并通过基础平台实现支撑。通过传感器与外部沟通；计算能力由智能芯片提供，作为示例，该智能芯片包括中央处理器(central processing unit，CPU)、神经网络处理器(neural-network processing unit，NPU)、图形处理器(graphics processing unit，GPU)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程逻辑门阵列(field programmable gate array，FPGA)等硬件加速芯片；基础平台包括分布式计算框架及网络等相关的平台保障和支持，可以包括云存储和计算、互联互通网络等。举例来说，传感器和外部沟通获取数据，这些数据提供给基础平台提供的分布式计算系统中的智能芯片进行计算。
(2)数据
基础设施的上一层的数据用于表示人工智能领域的数据来源。数据涉及到图形、图像、语音、文本，还涉及到传统设备的物联网数据，包括已有系统的业务数据以及力、位移、液位、温度、湿度等感知数据。
(3)数据处理
数据处理通常包括数据训练，机器学习，深度学习，搜索，推理，决策等方式。
其中，机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理、训练等。
推理是指在计算机或智能系统中，模拟人类的智能推理方式，依据推理控制策略，利用形式化的信息进行机器思维和求解问题的过程，典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程，通常提供分类、排序、预测等功能。
(4)通用能力
对数据经过上面提到的数据处理后，进一步基于数据处理的结果可以形成一些通用的能力，比如可以是算法或者一个通用系统，例如，翻译，文本的分析，计算机视觉的处理，语音识别，图像的识别等等。
(5)智能产品及行业应用
智能产品及行业应用指人工智能系统在各领域的产品和应用，是对人工智能整体解决方案的封装，将智能信息决策产品化、实现落地应用，其应用领域主要包括：智能终端、智能制造、智能交通、智能家居、智能医疗、智能安防、自动驾驶、平安城市等。
本申请实施例可以应用于各种领域中需要对多光谱图像或高光谱图像进行处理的应用场景中，作为示例，例如本申请实施例也可以应用于智能安防领域中的智能监控中，该智能监控为多光谱相机或高光谱相机，通过智能监控采集到被摄对象在照明光源的照射下的第一图像数据，该第一图像数据为多光谱图像或高光谱图像，则可以利用照明光源在n个光谱波段的光谱信息和第一图像数据，得到被摄对象的在n个光谱波段的光谱反射比，进而可以根据拍摄对象在n个光谱波段的光谱反射比，对被摄对象进行识别操作，n为大于3的整数，由于多光谱图像或高光谱图像中携带有被摄对象在多个光谱通道的图像数据，从而有利于提高图像识别过程的准确率。
作为另一示例，智能终端中可以配置有多光谱相机或高光谱相机，用于采集被摄对象在照明光源的照射下的第一图像数据，并利用照明光源在n个光谱波段的光谱信息和第一图像数据，得到被摄对象的在n个光谱波段的光谱反射比，进而根据在n个光谱波段的光 谱反射比，生成被摄对象的可视化图像(也即智能终端图库中供用户查看的图像)，有利于提高图像的颜色的复现精度，提高图像的颜色的分辨率和保真度。
作为另一示例，电子设备中也可以配置有多光谱相机或高光谱相机，用于采集被摄对象在照明光源的照射下的第一图像数据，并利用照明光源在n个光谱波段的光谱信息和第一图像数据，得到被摄对象的在n个光谱波段的光谱反射比，进而根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行分类或识别等操作，以提高图像识别过程的准确率，进而提高电子设备在行驶过程的安全度等等，应当理解，此处对本申请实施例的应用场景进行穷举。
在上述种种应用场景中，均需要获取照明光源的光谱信息，而目前通过高成本的专用测量仪采集照明光源的光谱信息的方式不仅成本高且操作麻烦。为了解决前述问题，本申请实施例提供了一种图像处理方法，请参阅图2。图2为本申请实施例提供的图像处理方法的一种流程示意图，本申请实施例提供的图像处理方法可以包括：
201、电子设备获取被摄对象在照明光源的照射下的第一图像数据。
本申请实施例中，电子设备获取被摄对象在照明光源的照射下的第一图像数据，其中，第一图像数据为多光谱图像(multispectral images，MSI)或高光谱图像，其中包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，n为大于3的整数，m为正整数。进一步地，电子设备具体可以表现为一个完整的设备，也可以表现为完整的设备中的一个芯片。第一图像数据包括的n个光谱通道的图像数据可以视为n个图层，前述n个图层的尺寸相同，每个图像中均包括与m个像素点一一对应的m个像素值，则第一图像数据中包括m个像素点中每个像素点的n个像素值。第一图像数据具体可以表现为n个矩阵，每个矩阵中有m个像素值，前述每个矩阵可以表现为p乘q的矩阵。
具体的，在一些应用场景中，电子设备具体表现为配置有多光谱相机或高光谱相机的设备，则电子设备可以通过配置的多光谱相机或高光谱相机采集被摄对象在照明光源的照射下的第一图像数据。在另一些应用场景中，电子设备具体表现为芯片，该芯片与多光谱相机/高光谱相机集成于一个完整的设备中，则该芯片可以通过该完整的设备中的多光谱相机或高光谱相机采集被摄对象在照明光源的照射下的第一图像数据。在另一些应用场景中，电子设备具体表现为配置有存储器的设备，电子设备中也可以预先存储有第一图像数据，该第一图像数据为被摄对象在照明光源的照射下采集到的。在另一些场景中，电子设备具体表现为芯片，电子设备集成于一个完整的设备中，该该完整的设备中配置有存储器，则电子设备可以从存储器中获取到第一图像数据。在另一些应用场景中，电子设备也可以通过浏览器下载第一图像数据，或者，电子设备接收其他电子设备发送的第一图像数据等等，此处不对第一图像数据的获取方式进行穷举。
202、电子设备获取与第一图像数据对应的第一权重数据。
本申请实施例中，电子设备在获取到第一图像数据之后，需要获取与第一图像数据对应的第一权重数据。其中，第一权重数据包括与m个像素点一一对应的m个权重值，与m个像素点中第一像素点对应的第一权重值高于与m个像素点中第二像素点对应的第二权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度；需要说明的是，第一像素点和第二像素点为第一图像数据包括的m个像素点中的任意两个不同的像 素点，此处引出第一像素点和第二像素点的概念是为了通过对比第一像素点和第二像素点来表达“与饱和度越低的颜色区域对应的像素点的权重值越高，与饱和度越高的颜色区域对应的像素点的权重值越低”这一概念，而不代表将上述m个像素点分为两种类别。
具体的，步骤202可以包括：电子设备可以将第一图像数据由3维数据转换为2维数据，得到一个m乘n的矩阵，也即一个m行n列的矩阵，2维数据形式的第一图像数据中每行包括的n个像素值代表同一个像素点在n个图层中的像素值。电子设备将第一图像数据与第一向量进行矩阵乘法，得到第二权重数据，其中，第一向量包括n个元素，第一向量包括的n个元素的元素值为基于预先的训练操作得到并配置于电子设备中的，第一向量的训练过程将通过后续实施例进行描述，此处不做赘述。第二权重数据包括与m个像素点一一对应的m个权重值，第二权重数据也可以表现为p乘q的矩阵。电子设备对第二权重数据包括的m个权重值中每个权重值进行归一化处理，以生成第一权重数据。
本申请实施例中，提供了生成第一权重数据的一种实现方式，且由于不同的多光谱图像或高光谱图像中的像素值的取值不同，从而导致与不同的第一图像数据对应的第二权重数据的取值范围不同，将第二权重数据进行归一化处理，有利于降低后续处理过程的复杂度，进而提高最后生成的照明光源的光谱信息的准确度。
更具体的，针对第一图像数据与第一向量进行矩阵乘法的过程。在一种实现方式中，第一图像数据包括m行数值，每行有n个数值，第一向量中的n个数值为按照从小到大的顺序排列的。电子设备将m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据。
本申请实施例中，由于与高饱和度的颜色对应的像素点的特性为n个光谱通道中几个光谱通道的像素值很高，剩余几个光谱通道的像素值很低，也即与高饱和度的颜色对应的像素点的n个像素值分布很不平均，基于前述特性，将m行数值中的每行数值按照从大到小的顺序进行排列，并且第一向量中的n个数值为按照从大到小的顺序排列，便于区分与高饱和度的颜色对应的像素点和与低饱和度的颜色对应的像素点。
为更直观地理解本方案，以下通过公式的方式来描述本实现方式。电子设备将第一图像数据从3维数据转换为2维的矩阵数据后可以表现为如下形态：
Figure PCTCN2020142327-appb-000001
其中，a 11至a 1n代表第一图像数据包括的n个图层(也即n个光谱通道的图像数据)中与第一个像素点对应的n个像素值，a m1至a mn代表第一图像数据包括的n个图层(也即n个光谱通道的图像数据)中与第m个像素点对应的n个像素值，a 11至a 1n也可以表示为r 1，a 21至a 2n也可以表示为r 2，a m1至a mn也可以表示为r m，也即r 1至r m可以分别与2维的矩阵数据包括的m行数据中的每行数据对应。
第二权重数据可以通过如下公式计算：
w＝Sort(M)*C＝[Sort(r 1)；Sort(r 2)；...；Sort(r m)]*C；  (2)
其中，w代表第二权重数据，M代表2维数据形式的第一图像数据，Sort(M)代表将 第一图像数据包括的m行数据中的每行数据按照从大到小的顺序进行排列，C代表第一向量，Sort(M)*C代表第一图像数据与第一向量进行矩阵乘法。
在另一种实现方式中，第一图像数据包括m行数值，每行有n个数值，第一向量中的n个数值为按照从大到小的顺序排列的。电子设备将m行数值中的每行数值按照从小到大的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据。
针对对第二权重数据中的每个权重值进行归一化处理的过程。在一种实现方式中，电子设备可以获取第二权重数据包括的m个权重值中取值最大的第五权重值，并将第二权重数据包括的m个权重值中每个权重值与该第五权重值相除，也即对m个权重值中的每个权重值进行归一化处理，从而得到了第二权重数据。为更直观地理解本方案，以下通过公式的方式展示了对第二权重数据包括的m个权重值进行归一化处理的公式：
W N＝w/Max(w)；  (3)
其中，W N代表第一权重数据，w代表第二权重数据，Max(w)代表m个权重值中取值最大的第五权重值，w/Max(w)代表将第二权重数据包括的m个权重值中每个权重值与该第五权重值相除，应理解，式(3)中的示例仅为方便理解本方案，不用于限定本方案。
在另一种实现方式中，电子设备还可以获取第二权重数据包括的m个权重值的和，并将第二权重数据包括的m个权重值中每个权重值与前述m个权重值的和相除，也即对m个权重值中的每个权重值进行归一化处理，从而得到了第二权重数据等，此处不对电子设备执行归一化的方式进行穷举。
可选地，电子设备在获取与第一图像数据对应的第一权重数据之后，还可以将m个权重值中的第三权重值的数值增大，将m个权重值中的第四权重值的数值减小，以得到更新后的第一权重数据。其中，第三权重值为第二权重数据包括的多个权重值中大于或等于预设阈值的权重值，第四权重值为第二权重数据包括的多个权重值中小于预设阈值的权重值，预设阈值的取值可以为0.3至0.5之间，作为示例，例如预设阈值的取值可以为0.35、0.4、0.45或其他数值等等，具体预设阈值的取值可结合实际应用场景来设定，此处不做限定。
具体的，电子设备中可以预先配置有分段映射函数，分段映射函数可以包括第一函数和第二函数，将第一权重数据包括的m个权重值中的每个权重值进行一次分段映射，得到更新后的第一权重数据；也即在第一权重数据包括的m个权重值中的一个权重值小于预设阈值的情况下，执行第一函数，在第一权重数据包括的m个权重值中的一个权重值大于或等于预设阈值的情况下，执行第二函数。为更直观地理解本方案，以下通过公式的方式来描述本实现方式：
W L＝mapping(W N)；  (4)
其中，W L代表更新后的第一权重数据，mapping(W N)指的是将第一权重数据包括的m个权重值中的每个权重值进行分段映射。进一步地，分段映射的过程中可以采用如下公式：
Figure PCTCN2020142327-appb-000002
其中，u代表W N(也即第一权重数据)中的任意一个权重值，k代表预设阈值，exp代表以e为底的指数函数，a和b均为两个超参数，作为示例，例如a的取值可以为50，b 的取值可以为0.6，具体a和b的值可以结合实际应用场景确定。
应理解，式(5)仅为方便理解本方案的一个示例，电子设备中预先存储的也可以为其他类型的分段映射函数，作为另一示例，例如在第一权重数据包括的m个权重值中的一个权重值小于预设阈值的情况下，电子设备将前述一个权重值与第一数值相除，该第一数值大于1，作为示例，例如第一数值的取值可以为1.1、1.2、1.3、2、5或其他数值等，此处不做限定。在在第一权重数据包括的m个权重值中的一个权重值大于或等于预设阈值的情况下，电子设备将前述一个权重值与第二数值相乘，第二数值大于1，作为示例，例如第二数值的取值可以为1.1、1.2、1.3、1.4、1.5等，此处不做穷举。或者，在第一权重数据包括的m个权重值中的一个权重值大于或等于预设阈值的情况下，电子设备将前述一个权重值与第三数值相加，第三数值为正数，作为示例，例如第三数值可以为0.3、0.4、0.5、0.6等等，此处不做穷举。
为更为直观地理解本方案，本申请实施例中将更新后的第一权重数据转换为权重图的形式，请参阅图3，图3为本申请实施例提供的图像处理方法中更新后的第一权重数据的一种示意图。图3包括(a)和(b)两个子示意图，图3的(a)子示意图代表第一图像数据，需要说明的是，第一图像数据为不可视的，此处仅为方便理解本方案而将第一图像数据进行可视化处理后进行展示的。图3的(b)子示意图代表与图3的(a)子示意图对应的第一图像数据的更新后的第一权重图，A1、A2、A3和A4分别与被摄对象的4个反光点对应，反光点为白色的，颜色的饱和度最低，则分配到的权重值就最高，从图3的(b)子示意图中明显可以看出A1、A2、A3和A4的颜色深度最深，代表这4个区域的像素点的权重值最高，应理解，图3中的示例仅为方便理解本方案，不用于限定本方案。
203、电子设备根据第一权重数据，生成与m个像素点一一对应的m个标签值。
本申请的一些实施例中，电子设备还可以根据第一权重数据或更新后的第一权重数据，对m个像素点进行分类，以生成与m个像素点对应的第一标签信息。其中，第一标签信息包括与m个像素点一一对应的m个标签值，用于指示m个像素点中每个像素点的类别，若第三像素点的分类结果为第一类别，则第三像素点的标签值为0，若第三像素点的分类结果为第二类别，则第三像素点的标签值为1，第三像素点为m个像素点中任一个像素点，第一类别和第二类别为不同的类别，权重值越高的像素点被分成第一类别的概率越大；进一步地，第一标签信息可以表现为p乘q的矩阵，与第一图像数据包括的n个图层(也即n个光谱通道的图像数据)中的每个图层的尺寸相同。
具体的，针对m个像素点进行分类的过程。电子设备可以采用聚类算法对第一权重数据(或者更新后的第一权重数据)包括的m个权重值进行分类，也即对与m个权重值一一对应的m个像素点的分类，得到第一分类结果。其中，前述聚类算法可以为K均值聚类算法、支持向量机(support vector machines，SVM)、逻辑回归算法或其他的聚类算法等，此处不做穷举。第一分类结果具体可以表现为p乘q的矩阵，其中包括与m个像素点一一对应的m个分类值。
进一步地，在一些实现方式中，第一分类结果中与可靠的像素点(也即第二类别)对应的分类值为1，与不可靠的像素点(也即第一类别)对应的分类值为0，则电子设备可以将生成的第一分类结果直接确定为第一标签信息。在另一些实现方式中，第一分类结果包 括的数值不可以直接确定为第一标签信息，则电子设备可以将第一分类结果进行转换，以得到第一标签信息。作为示例，例如聚类算法采用的为K均值聚类算法，得到的第一分类结果中可靠的像素点的分类值为1，不可靠的像素点的分类值为2，则可以根据第一分类结果并采用二值法(binarization)生成与m个像素点对应的第一标签信息，应理解，此处举例仅为方便理解本方案，不用于限定本方案。
为更为直观地理解本方案，本申请实施例中将第一标签信息转换为示意图的形式，请参阅图4，图4为本申请实施例提供的图像处理方法中第一标签信息的一种示意图。图4包括(a)和(b)两个子示意图，图4的(a)子示意图代表第一图像数据，图4的(b)子示意图代表与图4的(a)子示意图对应的第一图像数据中每个像素点对应的标签图(label map，LM)，由于第一标签信息中的标签值只有1或0两种取值，则第一标签信息的示意图中只有黑色和白色，图4的(b)子示意图中白色区域代表标签值为1的部分，也即与白色区域的标签值对应的像素点为可靠的像素点，图4的(b)子示意图中黑色区域代表标签值为0的部分，也即与黑色区域的标签值对应的像素点为不可靠的像素点，应理解，图4中的示例仅为方便理解本方案，不用于限定本方案。
204、电子设备将m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据。
本申请的一些实施例中，电子设备在生成与m个像素点一一对应的m个标签值之后，可以将m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据。具体的，步骤204可以包括：电子设备将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与第一标签信息进行哈达玛积运算，以将m个像素点中不可靠的像素点的像素值置零，得到更新后的第一图像数据。
更具体的，每个光谱通道的图像数据具体可以表现为p乘q的矩阵，第一图像数据表现为三维的张量，其中包括n个p乘q的矩阵；第一标签信息具体可以表现为p乘q的矩阵。则在一种实现方式中，电子设备逐个获取第一图像数据中的每个光谱通道的图像数据，将单个光谱通道的图像数据与第一标签信息进行哈达玛积运算。在另一种实现方式中，电子设备根据第一标签信息生成第一张量，第一张量为三维的张量，其中包括n个第一标签信息，电子设备将第一图像数据直接与第一张量进行哈达玛积运算。为更直观地理解本方案，如下公开了进行哈达玛积运算所采用的公式：
Figure PCTCN2020142327-appb-000003
其中，MSI d代表更新后的第一图像数据，LM 1代表第一张量，为基于第一标签信息生成的，MSI代表第一图像数据，
Figure PCTCN2020142327-appb-000004
代表将第一张量与第一图像数据进行哈达玛积运算，应理解，式(6)中的示例仅为方便理解本方案，不用于限定本方案。
为更为直观地理解本方案，本申请实施例中将第一图像数据和更新后的第一图像数据进行可视化处理，请参阅图5，图5为本申请实施例提供的图像处理方法中第一图像数据和更新后的第一图像数据的一种示意图。图5包括(a)和(b)两个子示意图，图5的(a)子示意图代表第一图像数据，图5的(b)子示意图代表更新后的第一图像数据，需要说明的是，第一图像数据和更新后的第一图像数据均为不可视的，此处仅为方便理解本方案，对第一图像数据和更新后的第一图像数据进行可视化处理后进行展示的。对比图5的(a) 子示意图和图5的(b)子示意图可知，将不可靠的像素点的像素值置零之后，需要处理的冗余信息变的更少，更加便于得到照明光源的准确的光谱信息，应理解，图5中的示例仅为方便理解本方案，不用于限定本方案。
可选地，电子设备在将m个像素点中类别为第一类别的像素点的像素值置零，以得到更新后的第一图像数据之后，还可以对更新后的第一图像数据进行去噪处理，以生成去噪后的第一图像数据。具体的，电子设备可以将更新后的第一图像数据(也即将类别为第一类别的像素点的像素值置零后的n个图层)分别输入低通滤波器中，以通过低通滤波器执行去噪操作和去除坏点操作。其中，低通滤波器包括但不限于高斯滤波器(Gaussian filter)、均值滤波器、中值滤波器或其他低通滤波器等等，此处不做限定。
为更直观地理解本方案，此处以低通滤波器采用的为高斯滤波器为例，公开了
MSI e＝GF5{MSI d}；  (7)
其中，MSI e代表去噪后的第一图像数据，GF5代表采用的为窗口大小为5的高斯滤波器，MSI d代表更新后的第一图像数据，需要说明的是，式(7)中的举例仅为方便理解本方案，实际情况中还可以采用其他类型的低通滤波器，且窗口大小也可以采用其他数值，此处均不做限定。
205、电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据。
本申请实施例中，若步骤202中生成的为第一权重数据，则电子设备根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据。
具体的，若第一权重数据为执行过归一化处理的，也即第一图像数据包括的m个像素点中每个像素点所对应的权重值均在0至1之间，则步骤205可以包括：电子设备根据第一权重数据，降低第一图像数据中第一像素点和第二像素点的像素值，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例。本申请实施例中，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例，从而实现了使第二图像数据中与低饱和度的颜色对应的像素点会更为明显的目的，且采用降低第一图像数据中第一像素点和第二像素点的像素值的方式，避免第二图像数据中出现过高的像素值。
更具体的，步骤205可以包括：电子设备将第一图像数据中第一像素点的像素值与第一权重值相乘，以得到第二图像数据中第一像素点的像素值；将第一图像数据中第二像素点的像素值与第二权重值相乘，以得到第二图像数据中第二像素点的像素值。本申请实施例中，提供了根据第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小的一种具体实现方式，通过将像素点与对应的权重值相乘的方式来实现，操作简单，易于实现。
进一步地，在一种实现方式中，电子设备可以将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与第一权重数据进行哈达玛积运算，以调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据。
更进一步地，每个光谱通道的图像数据具体可以表现为p乘q的矩阵，第一图像数据表现为三维的张量，其中包括n个p乘q的矩阵；第一权重数据具体可以表现为p乘q的矩阵。则在一种实现方式中，电子设备逐个获取第一图像数据中的每个光谱通道的图像数 据，将单个光谱通道的图像数据与第一权重数据进行哈达玛积运算。在另一种实现方式中，电子设备根据第一权重数据生成第二张量，第二张量为三维的张量，其中包括n个第一权重数据，电子设备将第一图像数据直接与第二张量进行哈达玛积运算。
在另一种实现方式中，电子设备也可以直接从第一权重数据中获取与目标像素点对应的目标权重值，并直接将目标像素点与目标权重值相乘，目标像素点为m个像素点中的任一个像素点，电子设备对m个像素点中每个像素点均执行前述操作，以调整第一图像数据中第一像素点和第二像素点的像素值的大小。
若步骤202中生成的为更新后的第一权重数据，则电子设备根据更新后的第一权重数据，调整第一图像数据中第一像素点和第二像素点的像素值的大小。具体的，电子设备根据更新后的第一权重数据，降低第一图像数据中第一像素点和第二像素点的像素值，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例。更具体的，步骤205可以包括：电子设备将第一图像数据中第一像素点的像素值与更新后的第一权重值相乘，以得到第二图像数据中第一像素点的像素值；将第一图像数据中第二像素点的像素值与更新后的第二权重值相乘，以得到第二图像数据中第二像素点的像素值。
进一步地，电子设备将第一图像数据包括的n个光谱通道的图像数据中每个光谱通道的图像数据与更新后的第一权重数据进行哈达玛积运算，以对第一图像数据中的第一像素点的像素值进行增强，对第一图像数据中的第二像素点的像素值进行削弱，得到第二图像数据。
本申请实施例中，进一步放大与高饱和度的颜色对应的像素点的权重值，并缩小与低饱和度的颜色对应的像素点的权重值，由于与高饱和度的颜色对应的像素点的低反射光谱波段，更容易受到噪声的干扰，所以进一步降低与高饱和度颜色对应的像素点的权重之后，不仅有利于进一步降低冗余像素点(也即与高饱和度的颜色对应的像素点)对生成照明光源的光谱信息的干扰，且有利于降低噪声对生成照明光源的光谱信息的干扰。
进一步地，步骤203和204为可选步骤，若执行步骤203和204，则步骤205可以包括：电子设备根据步骤202中生成的第一权重数据(或更新后的第一权重数据)，调整步骤204中生成的更新后的第一图像数据(或去噪后的第一图像数据)中的第一像素点和第二像素点的像素值的大小，得到第二图像数据。电子设备调整像素点的像素值的具体实现方式可参阅上述描述，此处不再一一赘述。
具体的，电子设备将步骤204中生成的更新后的第一图像数据(或去噪后的第一图像数据)中每个光谱通道的图像数据与步骤202中生成的第一权重数据(或更新后的第一权重数据)进行哈达玛积运算，以调整步骤204中生成的更新后的第一图像数据(或去噪后的第一图像数据)中的第一像素点和第二像素点的像素值的大小，得到第二图像数据。
本申请实施例中，根据与第一图像数据对应的第一权重数据，将第一图像数据中的m个像素点进行分类，继而将第一图像数据中类别为第一类别的像素点的像素值置零，以将第一图像数据中与高饱和度颜色对应的像素点的像素值置零，也即进一步消除第一图像数据中的冗余信息，从而可以利用更为可靠的像素点的信息，来生成照明光源的光谱信息，以提高生成的照明光源的光谱信息的准确度。
若不执行步骤203和204，则步骤205可以包括：电子设备根据步骤202中生成的第 一权重数据(或更新后的第一权重数据)，调整步骤201中获取到的第一图像数据中的第一像素点和第二像素点的像素值的大小，得到第二图像数据。
具体的，电子设备将步骤201中获取到的第一图像数据中每个光谱通道的图像数据与步骤202中生成的第一权重数据(或更新后的第一权重数据)进行哈达玛积运算，以调整步骤201中获取到的第一图像数据中的第一像素点和第二像素点的像素值的大小，得到第二图像数据。
为更为直观地理解本方案，请参阅图6和图7，图6为本申请实施例提供的图像处理方法中第二图像数据的两种示意图，图6中以不执行步骤203和204为例，图6包括(a)和(b)两个子示意图，图6的(a)子示意图代表根据第一图像数据和第一权重数据生成的第二图像数据，对前述第二图像数据进行可视化处理后得到，图6中示出的为热度图。图6的(b)子示意图代表根据第一图像数据和更新后的第一权重数据(也即将m个权重值中的第三权重值的数值增大，将m个权重值中的第四权重值的数值减小之后)生成的第二图像数据，对前述第二图像数据进行可视化处理后得到的。对比图6的(a)子示意图和图6的(b)子示意图可知，图6的(b)子示意图中的冗余信息更少，更能准确获取到照明光源的光谱信息。
继续参阅图7，图7以执行步骤203和204为例，将步骤204中生成的更新后的第一图像数据和第二图像数据进行可视化处理，请参阅图7，图7为本申请实施例提供的图像处理方法中更新后的第一图像数据和第二图像数据的一种示意图。图7为结合图3进行举例，也即与图7对应的第一图像数据为图3的(a)子示意图，图7包括(a)和(b)两个子示意图，图7的(a)子示意图代表更新后的第一图像数据，图7的(b)子示意图代表第二图像数据，需要说明的是，更新后的第一图像数据和第二图像数据均为不可视的，此处仅为方便理解本方案，对更新后的第一图像数据和第二图像数据进行可视化处理后进行展示的。对比图7的(a)子示意图和图7的(b)子示意图可知，第二图像数据中与高饱和度的颜色对应的像素点得到了进一步弱化，第二图像数据中与低饱和度的颜色对应的像素点得到了进一步地突出，从而图7的(b)子示意图中的有效信息更为集中，冗余信息更少，从而根据第二图像数据更能准确的获取到照明光源的光谱信息，应理解，图7中的示例仅为方便理解本方案，不用于限定本方案。
206、电子设备根据第二图像数据，确定照明光源的光谱信息。
本申请实施例中，电子设备在得到第二图像数据之后，可以根据第二图像数据，确定照明光源的光谱信息。其中，照明光源的光谱信息用于指示照明光源在多个光谱波段中的光谱功率分布(spectral power distribution，SPD)，照明光源的光谱信息中可以包括与多种不同波长的色光一一对应的多组强度值，每组强度值代表照明光源中每种色光的辐射能的数值。为了更加直观地理解本方案，请参阅图8，图8为本申请实施例提供的图像处理方法中照明光源的光谱信息的一种示意图。图8中以坐标图的形式来展示照明光源的部分光谱信息为例，由于一般情况下照明光源为不同波长的色光混合而成的复合光，图8为照明光源的光谱功率分布图，图8的横坐标代表多种色光中每种色光的波长值，图8的纵坐标代表照明光源中每种色光的辐射能的数值，需要说明的是，图8中的示例仅为方便理解本方案，还可以通过其他方式来表达照明光源的光谱信息，此处不做穷举。
具体的，第一图像数据为利用图像传感器得到，电子设备对第二图像数据进行特征提取，得到第一特征信息。其中，第一特征信息具体可以表现为包括n个数值的向量，前述n个数值分别为图像传感器针对照明光源在n个光谱通道的响应值的估计值。电子设备根据第一特征信息和图像传感器在n个光谱波段的光谱灵敏度，生成照明光源的光谱信息。本申请实施例中，提供了生成照明光源的光谱信息的一种具体实现方案，先生成照明光源在n个光谱通道的响应值，再求解照明光源的光谱信息，将一个大的步骤拆分为两个小的步骤，有利于提高照明光源的光谱信息的生成过程的精度。
更具体的，针对电子设备对第二图像数据进行特征提取的过程。电子设备可以将第二图像数据转换为二维数据的形式的数据，也即由三维张量(包括n个矩阵，每个矩阵为p乘q的矩阵)转换为m行n列的二维矩阵。在一种实现方式中，电子设备可以对二维数据形式的第二图像数据进行特征提取，直接得到第一特征信息。作为示例，例如在获取到m行n列的二维矩阵之后，获取前述n列数值中每列数值的平均值，从而得到第一特征信息包括的n个数值。作为另一示例，电子设备也可以将第二图像数据输入至特征提取网络中，以通过该特征提取网络输出的第一特征信息等。
在另一种实现方式中，电子设备采用第一算法对二维数据形式的第二图像数据进行特征提取(也可以称为降维处理)，得到第二特征信息，第二特征信息为k行n列的二维矩阵，进而从第二特征信息中获取第一特征信息，第一特征信息为包括n个数值的向量。其中，第一算法可以表现为主成分分析(principal components analysis，PCA)算法、奇异值分解算法或其他算法等等。为更直观地理解本方案，以下以采用PCA算法对第一图像数据进行特征提取为例，公开了生成第一特征信息过程中所采用到的公式：
Coff＝PCA(F 2D)；  (8)
其中，Coff代表第二特征信息，F 2D代表二维数据形式的第二图像数据，PCA(F 2D)代表采用PCF算法对二维数据形式的第二图像数据进行特征提取。
电子设备在通过PCA算法得到第二特征信息之后，将第二特征信息中的第一主成分信作为第一特征信息，也即：
Figure PCTCN2020142327-appb-000005
其中，
Figure PCTCN2020142327-appb-000006
代表第一特征信息，也即图像传感器针对照明光源在n个光谱通道的响应值的估计值，Coff(：；1)代表从第一特征信息包括的k行n列数据中获取第一行数据，应理解，式(8)和式(9)中的举例仅为方便理解本方案，此处均不做限定。
为了更加直观地理解本方案，请参阅图9，图9为本申请实施例提供的图像处理方法中第一特征信息的一种示意图。图9中以坐标图的形式来展示进行过归一化处理的第一特征信息为例，图9的横坐标代表与第一图像数据对应的n个光谱通道，图9中以n的取值为8为例，图9中的B1、B2、B3、B4、B5、B6、B7和B8分别代表图像传感器针对照明光源在n个光谱通道的响应值的估计值，需要说明的是，图9中的示例仅为方便理解本方案，还可以通过其他方式来表达照明光源的光谱信息，此处不做穷举。
针对生成照明光源的光谱信息的过程。电子设备在得到第一特征信息之后，可以根据第一特征信息和图像传感器在n个光谱波段的光谱灵敏度，通过求解方程式的方式，生成照明光源的光谱信息，该n个光谱波段与n个光谱通道一一对应。
为了更明确的了解本方案的实现过程，以下结合本实现方式的实现原理来解释照明光源的光谱信息的具体实现方式。生成照明光源的光谱信息的过程为利用照明光源在n个光谱通道的响应值的预估值重构出照明光源的完整的连续的光谱信息的过程。一个多光谱图像或高光谱图像的生成过程可以采用如下公式表示：
Figure PCTCN2020142327-appb-000007
其中，MSI代表第一图像数据，R代表被摄对象对n个光谱波段的光谱反射比，S代表图像传感器在n个光谱波段的光谱灵敏度，L代表照明光源的光谱信息。则图像传感器针对照明光源在n个光谱通道的响应值为S·L T，也即通过如下公式示出：
L CH＝S·L T；  (11)
其中，L CH代表图像传感器针对照明光源在n个光谱通道的响应值。式(11)可以等价于如下式(12)：
Figure PCTCN2020142327-appb-000008
其中，Φ代表正则化的约束矩阵，α为超参数，代表正则化的约束系数，I代表单位矩阵，D的取值为0，diag()代表生成对角矩阵，L CH和S的含义参阅式(10)和式(11)中的描述，此处不做赘述。为了避免解出的照明光源的光谱信息出现过拟合的问题，Φ可以有如下设置：
Figure PCTCN2020142327-appb-000009
结合上述原理可知，电子设备根据第一特征信息和图像传感器在n个光谱波段的光谱灵敏度，生成照明光源的光谱信息的过程，可以转换为根据第一特征信息、图像传感器在n个光谱波段的光谱灵敏度，采用第二算法求解照明光源的光谱信息的过程。该第二算法包括但不限于最小二乘算法、粒子群算法、遗传算法或其他的算法。
为了更直观地理解本方案，如下以通过最小二乘算法求解方程为例，则可以通过求解如下公式以生成照明光源的光谱信息：
Figure PCTCN2020142327-appb-000010
其中，
Figure PCTCN2020142327-appb-000011
代表生成的照明光源的光谱信息，式(13)中各个字母的含义可以参阅上述式(10)至式(12)中的描述，此处不做赘述，应理解，式(13)仅为方便理解本方案的一个示例，此处均不做限定。
可选地，电子设备在生成照明光源的光谱信息之后，还可以根据第一图像数据和照明光源的光谱信息，采用第三算法，生成被摄对象在n个光谱波段的光谱反射比。其中，第三算法包括但不限于最小二乘算法、粒子群算法、维纳估计算法、伪逆算法等，此处不做穷举。
其中，被摄对象在n个光谱波段的光谱反射比用于执行以下中任一项操作：对被摄对象进行分类、对被摄对象进行识别、生成被摄对象的可视化图像、对分析被摄对象的物理特性、分析被摄对象的化学特性和对被摄对象进行颜色的定量测量。
进一步地，由于不同物体对n个光谱波段的光谱反射比不同，则电子设备可以将被摄对象在n个光谱波段的光谱反射比输入用于图像分类的神经网络中，以通过神经网络对被摄对象进行分类。或者，电子设备也可以采用除神经网络之外的其他算法，根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行分类。
电子设备也可以将被摄对象在n个光谱波段的光谱反射比输入用于图像识别的神经网络中，以通过神经网络对被摄对象进行识别。或者，电子设备也可以采用除神经网络之外的其他算法，根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行识别。
电子设备还可以将被摄对象在n个光谱波段的光谱反射比输入用于物理特性分析的神经网络中，以通过神经网络对被摄对象进行物理特性分析。或者，电子设备也可以采用除神经网络之外的其他算法，根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行物理特性分析。
电子设备还可以将被摄对象在n个光谱波段的光谱反射比输入用于化学特性分析的神经网络中，以通过神经网络对被摄对象进行化学特性分析。或者，电子设备也可以采用除神经网络之外的其他算法，根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行化学特性分析。
电子设备可以采用除神经网络之外的其他算法，根据被摄对象在n个光谱波段的光谱反射比，对被摄对象进行颜色的定量测量。
电子设备还可以根据被摄对象在n个光谱波段的光谱反射比和照明光源的光谱信息，生成被摄对象的可视化图像。本申请实施例中，还会根据第一图像数据和照明光源的光谱信息，生成被摄对象在n个光谱波段的光谱反射比，进而列举了被摄对象在n个光谱波段的光谱反射比的多种应用场景，扩展了本方案的应用场景，提高了本方案的灵活性。
本申请实施例中，提供了一种从图像数据中直接获取光源的光谱信息的方案，不再需要通过高成本的专用测量仪器进行测量，不再需要执行额外操作，且节省了人工成本；此外，为与低饱和度的颜色对应的第一像素点分配更高的权重，为与高饱和度的颜色对应的第二像素点分配更低的权重，并根据每个像素点的权重值，调整第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，第二图像数据中与低饱和度的颜色对应的像素点会更为明显，也即第二图像数据中与高饱和度颜色对应的像素点所带来的干扰更少，进而根据第二图像数据，生成照明光源的光谱信息，由于被摄对象的高饱和度颜色区域会吸收照明光源大部分光谱波段的光，从而与高饱和度颜色对应的像素点中不容易提取出照明光源在所有光谱波段的光谱信息，而由于被摄对象的低饱和度颜色区域能够较为全面的反射照明光源在各个波段的光，从而从与低饱和度颜色对应的像素点中不容易提取出照明光源在所有光谱波段的光谱信息，因此，从第二图像数据中能够获取到照明光源在n个光谱波段的较为准确的光谱信息。
本申请实施例还提供了一种数据的获取方法，请参阅图10，图10为本申请实施例提供的数据的获取方法的一种流程示意图，本申请实施例提供的数据的获取方法可以包括：
1001、训练设备获取被摄对象的训练图像数据和与训练图像数据对应的第三权重数据。
本申请实施例中，训练设备中可以预先配置有多个训练图像数据和与每个训练图像数据对应的第三权重数据。其中，训练图像数据包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，每个训练图像数据的尺寸以及具体表现形式可参阅图2对应实施例中对第一图像数据的描述，请参阅上述描述，此处不做详细赘述。第三权重数据包括m个像素点中每个像素点的标注权重，m个像素点中第一像素点的权重值高于m个像素点中第二像素点的权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度；
1002、训练设备将训练图像数据与第一向量进行矩阵乘法，得到第二权重数据。
本申请实施例中，训练设备可以将训练图像数据与初始化得到的第一向量进行矩阵乘法，或者，将训练图像数据与上一次训练过程中生成的第一向量进行矩阵乘法，得到第二权重数据。其中，训练图像数据为m乘n的矩阵，第一向量包括n个元素，第二权重数据包括与m个像素点一一对应的m个权重值。步骤1002的具体实现方式可参阅图2对应实施例中步骤202中的描述，此处不做赘述。
1003、训练设备对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据。
本申请实施例中，步骤1002的具体实现方式可参阅图2对应实施例中步骤202中的描述，此处不做赘述。
1004、训练设备根据第一目标函数，对第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，第一目标函数指示第一权重数据与第三权重数据之间的相似度。
本申请实施例中，训练设备在生成第一权重数据之后，根据步骤1003生成的第一权重数据和步骤1001中获取的第三权重数据，生成第一目标函数的函数值，并根据第一目标函数的函数值进行梯度求导，以反向更新第一向量中n个元素的元素值，以完成对第一向量中n个元素的元素值的一次训练。训练设备在执行完步骤1004之后重新进入步骤1001，以对第一向量进行下一次训练，训练设备重复执行步骤1001至1004，以对第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，得到训练后的第一向量，图2对应的实施例中的第一向量为训练后的第一向量。
其中，第一目标函数指示第一权重数据与第三权重数据之间的相似度，作为示例，例如第一目标函数可以为第一权重数据与第三权重数据之间的角度差，或者，第一目标函数还可以为L2范数函数或其他类型的目标函数。预设条件可以为训练的次数达到预设次数，或者第一目标函数的数值小于预设数值。
需要说明的是，图2对应实施例中的电子设备和图10对应实施例中的训练设备可以为相同的设备，也可以为不同的设备。
本申请实施例中，还提供了第一向量的训练步骤，提高了本方案的完整性。
为了对本申请实施例所带来的有益效果有更为直观的认识，以下结合图11和图12对本申请实施例所带来的有益效果作进一步的介绍，图11和图12均以照明光源采用CIE标准光源为例。图11为采用本申请实施例提供的图像处理方法生成的照明光源在n个光谱通道的响应值的估计值与照明光源在n个光谱通道的真实响应值之间的一种对比示意图，图 11中为将前述响应值进行归一化处理后的值，图11的横坐标代表8个光谱通道，图11的纵坐标代表在每个光谱通道的响应值，C1指向的折线代表照明光源在n个光谱通道的n个真实响应值，C2指向的折线代表照明光源在n个光谱通道的n个响应值的预估值，通过图11可知，采用本申请实施例提供图像处理方法生成的照明光源在n个光谱通道的响应值的估计值的准确率较高。
继续参阅图12，图12为采用本申请实施例提供的图像处理方法生成的照明光源的光谱信息与照明光源的实际的光谱信息之间的一种对比示意图，图12的横坐标代表多种色光中每种色光的波长值，图8的纵坐标代表照明光源中每种色光的辐射能的数值，D1指向的折线代表照明光源的真实光谱信息，D2指向的折线代表采用本申请实施例提供的图像处理方法生成的照明光源的光谱信息，通过图12可知，采用本申请实施例提供图像处理方法生成的照明光源的光谱信息的准确率较高。
在图1至图10所对应的实施例的基础上，为了更好的实施本申请实施例的上述方案，下面还提供用于实施上述方案的相关设备。具体参阅图13a，图13a为本申请实施例提供的图像处理装置的一种结构示意图。图像处理装置1300可以包括获取模块1301、调整模块1302和确定模块1303，其中，获取模块1301，用于获取被摄对象在照明光源的照射下的第一图像数据，第一图像数据包括被摄对象在n个光谱通道的图像数据，第一图像数据包括m个像素点，n为大于3的整数，m为大于或等于1的整数；调整模块1302，用于根据第一权重数据，调整所述第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，其中，所述第一权重数据包括与所述m个像素点一一对应的m个权重值，所述第一像素点对应的第一权重值高于所述第二像素点对应的第二权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；确定模块1303，用于根据第二图像数据，确定照明光源的光谱信息，照明光源的光谱信息用于指示照明光源光谱功率分布。
在一种可能的设计中，调整模块1302，具体用于根据第一权重数据，降低第一图像数据中第一像素点和第二像素点的像素值，第二像素点的像素值的下降比例大于第一像素点的像素值的下降比例。
在一种可能的设计中，调整模块1302，具体用于：将第一图像数据中第一像素点的像素值与第一权重值相乘，以得到第二图像数据中第一像素点的像素值；将第一图像数据中第二像素点的像素值与第二权重值相乘，以得到第二图像数据中第二像素点的像素值。
在一种可能的设计中，请参阅图13b，图13b为本申请实施例提供的图像处理装置的一种结构示意图，装置1300还包括：生成模块1304，用于将第一图像数据与第一向量进行矩阵乘法，得到第二权重数据，其中，第一图像数据为m乘n的矩阵，第一向量包括n个元素，第二权重数据包括与m个像素点一一对应的m个权重值；生成模块1304，还用于对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据。
在一种可能的设计中，调整模块1302，还用于将m个权重值中的第三权重值的数值增大，将m个权重值中的第四权重值的数值减小，以得到更新后的第一权重数据，第三权重值为大于或等于预设阈值的权重值，第四权重值为小于预设阈值的权重值；调整模块1302，具体用于根据更新后的第一权重数据，对第一图像数据中的第一像素点的像素值进行增强， 对第一图像数据中第二像素点的像素值进行削弱。
在一种可能的设计中，第一图像数据为利用图像传感器得到，确定模块1303，具体用于：对第二图像数据进行特征提取，得到第一特征信息，第一特征信息为图像传感器针对照明光源在n个光谱通道的响应值的估计值；根据第一特征信息和图像传感器在n个光谱波段的光谱灵敏度，生成照明光源的光谱信息，n个光谱波段与n个光谱通道一一对应。
在一种可能的设计中，请参阅图13b，装置1300还包括：分类模块1305，用于根据第一权重数据，对m个像素点进行分类，以得到与m个像素点一一对应的m个标签值，用于指示m个像素点的类别；更新模块1306，用于将m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据；调整模块1302，具体用于调整更新后的第一图像数据中第一像素点和第二像素点的像素值的大小。
在一种可能的设计中，请参阅图13b，装置1300还包括：生成模块1304，用于根据第一图像数据和照明光源的光谱信息，生成被摄对象在n个光谱波段的光谱反射比，被摄对象在n个光谱波段的光谱反射比用于执行以下中任一项操作：对被摄对象进行分类、被摄对象进行识别、生成被摄对象的可视化图像、对分析被摄对象的物理特性、分析被摄对象的化学特性和对被摄对象进行颜色的定量测量。
需要说明的是，图像处理装置1300中各模块/单元之间的信息交互、执行过程等内容，与本申请中图2至图9对应的各个方法实施例基于同一构思，具体内容可参见本申请前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还提供一种数据的获取装置，参阅图14，图14为本申请实施例提供的数据的获取装置的一种结构示意图。数据的获取装置1400可以包括获取模块1401、生成模块1402和训练模块1403。获取模块1401，用于获取被摄对象的训练图像数据和与训练图像数据对应的第三权重数据，其中，训练图像数据包括被摄对象在n个光谱通道的图像数据，训练图像数据包括m个像素点，第三权重数据包括m个像素点中每个像素点的标注权重，m个像素点中第一像素点的权重值高于m个像素点中第二像素点的权重值，与第一像素点对应的颜色的饱和度低于与第二像素点对应的颜色的饱和度；生成模块1402，用于将训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，并对第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据，其中，训练图像数据为m乘n的矩阵，第一向量包括n个元素，第二权重数据包括与m个像素点一一对应的m个权重值；训练模块1403，用于根据目标函数，对第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，目标函数指示第一权重数据与第三权重数据之间的相似度。
在一种可能的设计中，生成模块1402，具体用于将m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将第一矩阵与第一向量相乘，得到第二权重数据，第一向量中的n个数值为按照从大到小的顺序排列的。
需要说明的是，数据的获取装置1400中各模块/单元之间的信息交互、执行过程等内容，与本申请中图10对应的各个方法实施例基于同一构思，具体内容可参见本申请前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还提供了一种电子设备，请参阅图15，图15为本申请实施例提供的电子设备的一种结构示意图，其中，电子设备1500用于实现图2至图9对应实施例中电子设 备的功能。具体的，电子设备1500包括：接收器1501、发射器1502、处理器1503和存储器1504(其中电子设备1500中的处理器1503的数量可以一个或多个，图15中以一个处理器为例)，其中，处理器1503可以包括应用处理器15031和通信处理器15032。在本申请的一些实施例中，接收器1501、发射器1502、处理器1503和存储器1504可通过总线或其它方式连接。
存储器1504可以包括只读存储器和随机存取存储器，并向处理器1503提供指令和数据。存储器1504的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。存储器1504存储有处理器和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。
处理器1503控制电子设备的操作。具体的应用中，电子设备的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。
上述本申请实施例揭示的方法可以应用于处理器1503中，或者由处理器1503实现。处理器1503可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1503中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1503可以是通用处理器、数字信号处理器(digital signal processing，DSP)、微处理器或微控制器，还可进一步包括专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。该处理器1503可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1504，处理器1503读取存储器1504中的信息，结合其硬件完成上述方法的步骤。
接收器1501可用于接收输入的数字或字符信息，以及产生与电子设备的相关设置以及功能控制有关的信号输入。发射器1502可用于通过第一接口输出数字或字符信息；发射器1502还可用于通过第一接口向磁盘组发送指令，以修改磁盘组中的数据；发射器1502还可以包括显示屏等显示设备。
需要说明的是，对于应用处理器15031执行图像处理方法的具体实现方式以及带来的有益效果，均可以参考图2至图9对应的各个方法实施例中的叙述，此处不再一一赘述。
请参阅图16，图16是本申请实施例提供的训练设备一种结构示意图，训练设备1600用于实现图10对应实施例中训练设备的功能。具体的，训练设备1600由一个或多个服务器实现，训练设备1600可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器(central processing units，CPU)1622(例如，一个或一个以上处理器)和存储器1632，一个或一个以上存储应用程序1642或数据1644的存储介质1630(例如一个或一个以上海量存储设备)。其中，存储器1632和存储介质1630可以是短暂存储或持久存储。 存储在存储介质1630的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对训练设备中的一系列指令操作。更进一步地，中央处理器1622可以设置为与存储介质1630通信，在训练设备1600上执行存储介质1630中的一系列指令操作。
训练设备1600还可以包括一个或一个以上电源1626，一个或一个以上有线或无线网络接口1650，一个或一个以上输入输出接口1658，和/或，一个或一个以上操作系统1641，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM等等。
需要说明的是，对于中央处理器1622执行数据的获取方法的具体实现方式以及带来的有益效果，均可以参考图10对应的各个方法实施例中的叙述，此处不再一一赘述。
本申请实施例中还提供一种计算机可读存储介质，该计算机可读存储介质中存储有用于生成车辆行驶速度的程序，当其在计算机上运行时，使得计算机执行如前述图2至图9所示实施例描述的方法中电子设备所执行的步骤，或者，使得计算机执行如前述图10所示实施例描述的方法中训练设备所执行的步骤。
本申请实施例中还提供一种包括计算机程序产品，当其在计算机上运行时，使得计算机执行如前述图2至图9所示实施例描述的方法中电子设备所执行的步骤，或者，使得计算机执行如前述图10所示实施例描述的方法中训练设备所执行的步骤。
本申请实施例中还提供一种电路系统，所述电路系统包括处理电路，所述处理电路配置为执行如前述图2至图9所示实施例描述的方法中电子设备所执行的步骤，或者，所述处理电路配置为执行如前述图10所示实施例描述的方法中电子设备所执行的步骤。
本申请实施例提供的图像处理装置或电子设备具体可以为芯片，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使芯片执行上述图2至图9所示实施例描述的图像处理方法，或者，以使芯片执行上述图10所示实施例描述的数据的获取方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述无线接入设备端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
具体的，请参阅图17，图17为本申请实施例提供的芯片的一种结构示意图，所述芯片可以表现为神经网络处理器NPU 170，NPU 170作为协处理器挂载到主CPU(Host CPU)上，由Host CPU分配任务。NPU的核心部分为运算电路1703，通过控制器1704控制运算电路1703提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路1703内部包括多个处理单元(Process Engine,PE)。在一些实现中，运算电路1703是二维脉动阵列。运算电路1703还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路1703是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器1702中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器1701中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)1708中。
统一存储器1706用于存放输入数据以及输出数据。权重数据直接通过存储单元访问控制器(Direct Memory Access Controller，DMAC)1705，DMAC被搬运到权重存储器1702中。输入数据也通过DMAC被搬运到统一存储器1706中。
BIU为Bus Interface Unit即，总线接口单元1710，用于AXI总线与DMAC和取指存储器(Instruction Fetch Buffer，IFB)1709的交互。
总线接口单元1710(Bus Interface Unit，简称BIU)，用于取指存储器1709从外部存储器获取指令，还用于存储单元访问控制器1705从外部存储器获取输入矩阵A或者权重矩阵B的原数据。
DMAC主要用于将外部存储器DDR中的输入数据搬运到统一存储器1706或将权重数据搬运到权重存储器1702中或将输入数据数据搬运到输入存储器1701中。
向量计算单元1707包括多个运算处理单元，在需要的情况下，对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。主要用于神经网络中非卷积/全连接层网络计算，如Batch Normalization(批归一化)，像素级求和，对特征平面进行上采样等。
在一些实现中，向量计算单元1707能将经处理的输出的向量存储到统一存储器1706。例如，向量计算单元1707可以将线性函数和/或非线性函数应用到运算电路1703的输出，例如对卷积层提取的特征平面进行线性插值，再例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元1707生成归一化的值、像素级求和的值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路1703的激活输入，例如用于在神经网络中的后续层中的使用。
控制器1704连接的取指存储器(instruction fetch buffer)1709，用于存储控制器1704使用的指令；
统一存储器1706，输入存储器1701，权重存储器1702以及取指存储器1709均为On-Chip存储器。外部存储器私有于该NPU硬件架构。
其中，循环神经网络中各层的运算可以由运算电路1703或向量计算单元1707执行。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或一个或多个用于控制上述第一方面方法的程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CLU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程 序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (24)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    获取被摄对象在照明光源的照射下的第一图像数据，所述第一图像数据包括所述被摄对象在n个光谱通道的图像数据，所述第一图像数据包括m个像素点，n为大于3的整数，m为正整数；
    根据第一权重数据，调整所述第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，其中，所述第一权重数据包括与所述m个像素点一一对应的m个权重值，所述第一像素点对应的第一权重值高于所述第二像素点对应的第二权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；
    根据所述第二图像数据，确定所述照明光源的光谱信息，所述照明光源的光谱信息用于指示所述照明光源光谱功率分布。
  2. 根据权利要求1所述的方法，其特征在于，所述根据第一权重数据，调整所述第一图像数据中第一像素点和第二像素点的像素值的大小，包括：
    根据所述第一权重数据，降低所述第一图像数据中所述第一像素点和所述第二像素点的像素值，所述第二像素点的像素值的下降比例大于所述第一像素点的像素值的下降比例。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据所述第一权重数据，调整所述第一图像数据中所述第一像素点和所述第二像素点的像素值的大小，包括：
    将所述第一图像数据中所述第一像素点的像素值与所述第一权重值相乘，以得到所述第二图像数据中所述第一像素点的像素值；
    将所述第一图像数据中所述第二像素点的像素值与所述第二权重值相乘，以得到所述第二图像数据中所述第二像素点的像素值。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，所述方法还包括：
    将所述第一图像数据与第一向量进行矩阵乘法，得到第二权重数据，其中，所述第一图像数据为m乘n的矩阵，所述第一向量包括n个元素，所述第二权重数据包括与所述m个像素点一一对应的m个权重值；
    对所述第二权重数据包括的m个权重值进行归一化处理，以生成所述第一权重数据。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，所述方法还包括：
    将所述m个权重值中的第三权重值的数值增大，将所述m个权重值中的第四权重值的数值减小，以得到更新后的第一权重数据，所述第三权重值为大于或等于预设阈值的权重值，所述第四权重值为小于所述预设阈值的权重值；
    所述根据所述第一权重数据，调整所述第一图像数据中所述第一像素点和所述第二像素点的像素值的大小，包括：
    根据所述更新后的第一权重数据，调整所述第一图像数据中所述第一像素点和所述第二像素点的像素值的大小。
  6. 根据权利要求1至5任一项所述的方法，其特征在于，所述第一图像数据为利用图像传感器得到，所述根据所述第二图像数据，确定所述照明光源的光谱信息，包括：
    对所述第二图像数据进行特征提取，得到第一特征信息，所述第一特征信息为图像传感器针对所述照明光源在所述n个光谱通道的响应值的估计值；
    根据所述第一特征信息和所述图像传感器在n个光谱波段的光谱灵敏度，生成所述照明光源的光谱信息，所述n个光谱波段与所述n个光谱通道一一对应。
  7. 根据权利要求1至6任一项所述的方法，其特征在于，所述方法还包括：
    根据所述第一权重数据，对所述m个像素点进行分类，以得到与所述m个像素点一一对应的m个标签值，用于指示所述m个像素点的类别；
    将所述m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据；
    所述调整所述第一图像数据中所述第一像素点和所述第二像素点的像素值的大小，包括：
    调整所述更新后的第一图像数据中所述第一像素点和所述第二像素点的像素值的大小。
  8. 根据权利要求1至7任一项所述的方法，其特征在于，所述方法还包括：
    根据所述第一图像数据和所述照明光源的光谱信息，生成所述被摄对象在所述n个光谱波段的光谱反射比，所述被摄对象在所述n个光谱波段的光谱反射比用于执行以下中任一项操作：对所述被摄对象进行分类、对所述被摄对象进行识别、生成所述被摄对象的可视化图像、对分析所述被摄对象的物理特性、分析所述被摄对象的化学特性和对所述被摄对象进行颜色的定量测量。
  9. 一种数据的获取方法，其特征在于，所述方法包括：
    获取被摄对象的训练图像数据和与所述训练图像数据对应的第三权重数据，其中，所述训练图像数据包括所述被摄对象在n个光谱通道的图像数据，所述训练图像数据包括m个像素点，所述第三权重数据包括所述m个像素点中每个像素点的标注权重，所述m个像素点中第一像素点的权重值高于所述m个像素点中第二像素点的权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；
    将所述训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，并对所述第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据，其中，所述训练图像数据为m乘n的矩阵，所述第一向量包括n个元素，所述第二权重数据包括与所述m个像素点一一对应的m个权重值；
    根据目标函数，对所述第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，所述目标函数指示所述第一权重数据与所述第三权重数据之间的相似度。
  10. 根据权利要求9所述的方法，其特征在于，所述训练图像数据包括m行数值，每行有n个数值，所述将所述训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，包括：
    将所述m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将所述第一矩阵与所述第一向量相乘，得到所述第二权重数据，所述第一向量中的n个数值为按照从大到小的顺序排列的。
  11. 一种图像处理装置，其特征在于，所述装置包括：
    获取模块，用于获取被摄对象在照明光源的照射下的第一图像数据，所述第一图像数据包括所述被摄对象在n个光谱通道的图像数据，所述第一图像数据包括m个像素点，n为大于3的整数，m为正整数；
    调整模块，用于根据第一权重数据，调整所述第一图像数据中第一像素点和第二像素点的像素值的大小，得到第二图像数据，其中，所述第一权重数据包括与所述m个像素点一一对应的m个权重值，所述第一像素点对应的第一权重值高于所述第二像素点对应的第二权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；
    确定模块，用于根据所述第二图像数据，确定所述照明光源的光谱信息，所述照明光源的光谱信息用于指示所述照明光源光谱功率分布。
  12. 根据权利要求11所述的装置，其特征在于，所述调整模块，具体用于根据所述第一权重数据，降低所述第一图像数据中所述第一像素点和所述第二像素点的像素值，所述第二像素点的像素值的下降比例大于所述第一像素点的像素值的下降比例。
  13. 根据权利要求11或12所述的装置，其特征在于，所述调整模块，具体用于：
    将所述第一图像数据中所述第一像素点的像素值与所述第一权重值相乘，以得到所述第二图像数据中所述第一像素点的像素值；
    将所述第一图像数据中所述第二像素点的像素值与所述第二权重值相乘，以得到所述第二图像数据中所述第二像素点的像素值。
  14. 根据权利要求11至13任一项所述的装置，其特征在于，所述装置还包括：
    生成模块，用于将所述第一图像数据与第一向量进行矩阵乘法，得到第二权重数据，其中，所述第一图像数据为m乘n的矩阵，所述第一向量包括n个元素，所述第二权重数据包括与所述m个像素点一一对应的m个权重值；
    所述生成模块，还用于对所述第二权重数据包括的m个权重值进行归一化处理，以生成所述第一权重数据。
  15. 根据权利要求11至14任一项所述的装置，其特征在于，
    所述调整模块，还用于将所述m个权重值中的第三权重值的数值增大，将所述m个权重值中的第四权重值的数值减小，以得到更新后的第一权重数据，所述第三权重值为大于或等于预设阈值的权重值，所述第四权重值为小于所述预设阈值的权重值；
    所述调整模块，具体用于根据所述更新后的第一权重数据，调整所述第一图像数据中所述第一像素点和所述第二像素点的像素值的大小。
  16. 根据权利要求11至15任一项所述的装置，其特征在于，所述第一图像数据为利用图像传感器得到，所述确定模块，具体用于：
    对所述第二图像数据进行特征提取，得到第一特征信息，所述第一特征信息为图像传感器针对所述照明光源在所述n个光谱通道的响应值的估计值；
    根据所述第一特征信息和所述图像传感器在n个光谱波段的光谱灵敏度，生成所述照明光源的光谱信息，所述n个光谱波段与所述n个光谱通道一一对应。
  17. 根据权利要求11至16任一项所述的装置，其特征在于，所述装置还包括：
    分类模块，用于根据所述第一权重数据，对所述m个像素点进行分类，以得到与所述m个像素点一一对应的m个标签值，用于指示所述m个像素点的类别；
    更新模块，用于将所述m个像素点中类别为第一类别的像素点的像素值置零，得到更新后的第一图像数据；
    所述调整模块，具体用于调整所述更新后的第一图像数据中所述第一像素点和所述第二像素点的像素值的大小。
  18. 根据权利要求11至17任一项所述的装置，其特征在于，所述装置还包括：
    生成模块，用于根据所述第一图像数据和所述照明光源的光谱信息，生成所述被摄对象在所述n个光谱波段的光谱反射比，所述被摄对象在所述n个光谱波段的光谱反射比用于执行以下中任一项操作：对所述被摄对象进行分类、对所述被摄对象进行识别、生成所述被摄对象的可视化图像、对分析所述被摄对象的物理特性、分析所述被摄对象的化学特性和对所述被摄对象进行颜色的定量测量。
  19. 一种数据的获取装置，其特征在于，所述装置包括：
    获取模块，用于获取被摄对象的训练图像数据和与所述训练图像数据对应的第三权重数据，其中，所述训练图像数据包括所述被摄对象在n个光谱通道的图像数据，所述训练图像数据包括m个像素点，所述第三权重数据包括所述m个像素点中每个像素点的标注权重，所述m个像素点中第一像素点的权重值高于所述m个像素点中第二像素点的权重值，与所述第一像素点对应的颜色的饱和度低于与所述第二像素点对应的颜色的饱和度；
    生成模块，用于将所述训练图像数据与第一向量进行矩阵乘法，得到第二权重数据，并对所述第二权重数据包括的m个权重值进行归一化处理，以生成第一权重数据，其中，所述训练图像数据为m乘n的矩阵，所述第一向量包括n个元素，所述第二权重数据包括与所述m个像素点一一对应的m个权重值；
    训练模块，用于根据目标函数，对所述第一向量中n个元素的元素值进行迭代训练，直至满足预设条件，所述目标函数指示所述第一权重数据与所述第三权重数据之间的相似度。
  20. 根据权利要求19所述的装置，其特征在于，所述训练图像数据包括m行数值，每行有n个数值；
    所述生成模块，具体用于将所述m行数值中的每行数值按照从大到小的顺序进行排列，得到第一矩阵，将所述第一矩阵与所述第一向量相乘，得到所述第二权重数据，所述第一向量中的n个数值为按照从大到小的顺序排列的。
  21. 一种计算机可读存储介质，其特征在于，包括程序，当其在计算机上运行时，使得计算机执行如权利要求1至8中任一项所述的方法，或者，使得计算机执行如权利要求9或10所述的方法。
  22. [根据细则91更正 24.06.2021]　
    一种电子设备，其特征在于，包括处理器，所述处理器和存储器耦合，所述存储器存储有程序指令，当所述存储器存储的程序指令被所述处理器执行时实现权利要求1至8中任一项所述的方法。
  23. [根据细则91更正 24.06.2021]　
    一种训练设备，其特征在于，包括处理器，所述处理器和存储器耦合，所述存储器存储有程序指令，当所述存储器存储的程序指令被所述处理器执行时实现权利要求9或10所述的方法。
  24. [根据细则91更正 24.06.2021]　
    一种电路系统，其特征在于，所述电路系统包括处理电路，所述处理电路配置为执行如权利要求1至8中任一项所述的方法，或者，所述处理电路配置为执行如权利要求9或10所述的方法。
PCT/CN2020/142327 2020-12-31 2020-12-31 一种图像处理方法、数据的获取方法及设备 WO2022141476A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080039169.1A CN115643811A (zh) 2020-12-31 2020-12-31 一种图像处理方法、数据的获取方法及设备
PCT/CN2020/142327 WO2022141476A1 (zh) 2020-12-31 2020-12-31 一种图像处理方法、数据的获取方法及设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/142327 WO2022141476A1 (zh) 2020-12-31 2020-12-31 一种图像处理方法、数据的获取方法及设备

Publications (1)

Publication Number Publication Date
WO2022141476A1 true WO2022141476A1 (zh) 2022-07-07

Family

ID=82258893

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/142327 WO2022141476A1 (zh) 2020-12-31 2020-12-31 一种图像处理方法、数据的获取方法及设备

Country Status (2)

Country Link
CN (1) CN115643811A (zh)
WO (1) WO2022141476A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116029936A (zh) * 2023-01-31 2023-04-28 北京小米移动软件有限公司 图像的修正方法及装置、电子设备和存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118333873B (zh) * 2024-04-15 2024-09-06 武汉纺织大学 一种基于多光谱图像重建的低照度增强方法和系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101888487A (zh) * 2010-06-02 2010-11-17 中国科学院深圳先进技术研究院 高动态范围视频成像系统和图像生成方法
CN101917633A (zh) * 2010-09-01 2010-12-15 杭州海康威视软件有限公司 图像的白平衡处理方法、装置及摄像机
CN105516606A (zh) * 2016-01-21 2016-04-20 努比亚技术有限公司 拍照装置及方法
US20160300337A1 (en) * 2015-04-08 2016-10-13 Tatung University Image fusion method and image processing apparatus
CN107239781A (zh) * 2017-05-03 2017-10-10 北京理工大学 一种基于rgb图像的超光谱反射率重建方法
CN109543701A (zh) * 2018-11-30 2019-03-29 长沙理工大学 视觉显著性区域检测方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5740147B2 (ja) * 2010-12-20 2015-06-24 三星テクウィン株式会社Ｓａｍｓｕｎｇ Ｔｅｃｈｗｉｎ Ｃｏ．， Ｌｔｄ 光源推定装置及び光源推定方法
US9996913B2 (en) * 2014-04-07 2018-06-12 Bae Systems Information And Electronic Systems Integration Inc. Contrast based image fusion
EP3510558A4 (en) * 2016-09-06 2020-05-06 B.G. Negev Technologies and Applications Ltd., at Ben-Gurion University OBTAINING HYPERSPECTRAL DATA FROM AN IMAGE
CN106840398B (zh) * 2017-01-12 2018-02-02 南京大学 一种多光谱光场成像方法
CN110660088B (zh) * 2018-06-30 2023-08-22 华为技术有限公司 一种图像处理的方法和设备
CN112153356A (zh) * 2020-09-16 2020-12-29 Oppo广东移动通信有限公司 图像参数的确定方法、图像传感器、装置、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101888487A (zh) * 2010-06-02 2010-11-17 中国科学院深圳先进技术研究院 高动态范围视频成像系统和图像生成方法
CN101917633A (zh) * 2010-09-01 2010-12-15 杭州海康威视软件有限公司 图像的白平衡处理方法、装置及摄像机
US20160300337A1 (en) * 2015-04-08 2016-10-13 Tatung University Image fusion method and image processing apparatus
CN105516606A (zh) * 2016-01-21 2016-04-20 努比亚技术有限公司 拍照装置及方法
CN107239781A (zh) * 2017-05-03 2017-10-10 北京理工大学 一种基于rgb图像的超光谱反射率重建方法
CN109543701A (zh) * 2018-11-30 2019-03-29 长沙理工大学 视觉显著性区域检测方法及装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116029936A (zh) * 2023-01-31 2023-04-28 北京小米移动软件有限公司 图像的修正方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN115643811A (zh) 2023-01-24

Similar Documents

Publication Publication Date Title
WO2022179587A1 (zh) 一种特征提取的方法以及装置
US20190108447A1 (en) Multifunction perceptrons in machine learning environments
CN111461165A (zh) 图像识别方法、识别模型的训练方法及相关装置、设备
US10803571B2 (en) Data-analysis pipeline with visual performance feedback
WO2021218471A1 (zh) 一种用于图像处理的神经网络以及相关设备
US10019788B1 (en) Machine-learning measurements of quantitative feature attributes
WO2022141476A1 (zh) 一种图像处理方法、数据的获取方法及设备
WO2022127111A1 (zh) 跨模态人脸识别方法、装置、设备及存储介质
Chaugule et al. Identification of paddy varieties based on novel seed angle features
WO2022022425A1 (en) Systems and methods for temperature measurement
WO2022012668A1 (zh) 一种训练集处理方法和装置
JP7225731B2 (ja) 多変数データシーケンスの画像化
Priya Henry et al. Convolutional neural-network-based classification of retinal images with different combinations of filtering techniques
Yin et al. Adaptive low light visual enhancement and high-significant target detection for infrared and visible image fusion
Dhiman et al. PFDI: a precise fruit disease identification model based on context data fusion with faster-CNN in edge computing environment
WO2018113206A1 (zh) 一种图像处理方法及终端
CN114187515A (zh) 图像分割方法和图像分割装置
Singh et al. Smart agriculture framework for automated detection of leaf blast disease in paddy crop using colour slicing and GLCM features based random forest approach
Wakhare et al. Using image processing and deep learning techniques detect and identify pomegranate leaf diseases
WO2021147366A1 (zh) 一种图像处理方法以及相关设备
WO2021189321A1 (zh) 一种图像处理方法和装置
Shukla et al. A hybrid CNN with transfer learning for skin cancer disease detection
WO2023029559A1 (zh) 一种数据处理方法以及装置
CN114648512A (zh) 舌下图像分析方法、装置、计算机设备和存储介质
CN118052967B (zh) 林火智能识别方法以及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20967815

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20967815

Country of ref document: EP

Kind code of ref document: A1