WO2020010638A1 - 一种图像坏点检测方法及装置 - Google Patents

一种图像坏点检测方法及装置 Download PDF

Info

Publication number
WO2020010638A1
WO2020010638A1 PCT/CN2018/095712 CN2018095712W WO2020010638A1 WO 2020010638 A1 WO2020010638 A1 WO 2020010638A1 CN 2018095712 W CN2018095712 W CN 2018095712W WO 2020010638 A1 WO2020010638 A1 WO 2020010638A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
neural network
pixel point
point
value
Prior art date
Application number
PCT/CN2018/095712
Other languages
English (en)
French (fr)
Inventor
许家誉
王维
柳海波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201880095536.2A priority Critical patent/CN112384946A/zh
Priority to PCT/CN2018/095712 priority patent/WO2020010638A1/zh
Publication of WO2020010638A1 publication Critical patent/WO2020010638A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application relates to the field of image processing, and in particular, to a method and a device for detecting an image dead point.
  • Mobile phone camera sensor due to its small sensor area, high pixel density and cost constraints, its camera sensor will have a considerable number of defects, of which the generation of dead pixels is a random event, when the dark scene is exposed for a long time, its dead pixels The probability can reach 5%, which seriously affects the quality of the image. If the correction is too strong, it will easily lead to the loss of details in the picture, and false correction will occur in the high-frequency area, which will cause false colors; if the correction is too weak, it will easily lead to the residue of bad points. At present, it is possible to detect the dead pixels based on the absolute value, gradient, direction, and threshold of the pixel, but it needs to be repeatedly debugged by experienced personnel, and the correction parameters are difficult to determine.
  • a matrix of 5 ⁇ 5 Bayer pixels around the pixel to be detected is established, and according to different color channels, the pixels of the same type of channel in different positions around the pixel to be detected in the matrix are selected.
  • the pixel value is used as a reference to obtain the pixel difference or gradient of the pixel to be detected and each pixel of the same type of channel.
  • the pixel difference or gradient exceeds a threshold, it is determined that the pixel to be detected is a bad point.
  • the position of the dead pixel can be determined according to the absolute difference between the pixels of the eight channels of the same type around the dead pixel and the dead pixel. To select the corresponding reliable pixel value, and assign the reliable pixel value to the current dead pixel for correction.
  • the embodiments of the present application provide an image dead point detection method and device, which can solve the problems of poor applicability of the dead point detection rule and poor detection effect.
  • an image dead point detection method including: selecting a pixel point in a preset area around the first pixel point around the first pixel point in the image; It is assumed that the pixel value of a pixel in the area is input to a neural network for prediction, and the probability that the first pixel is a bad point is obtained; if the probability is greater than a preset threshold, the first pixel is determined to be a bad point.
  • the combination of neural network and dead point detection can avoid the problems of poor detection of bad points in the prior art due to many rules, difficult debugging, and complex calibration. It does not require manual intervention and has a strong application. Sex.
  • the method before selecting a pixel in a preset area around the first pixel with the pixel as a center, the method further includes: receiving parameters of the neural network, and the parameters include weights and Bias, so that the first pixel point can be detected in the neural network directly according to the weight and offset of the internal connection of the neural network to obtain the probability that the first pixel point is a bad point.
  • the method before selecting a pixel point in a preset area around the first pixel point with the pixel point as the center, the method further includes: generating a bad sample of different original picture samples; The image samples and the corresponding bad point samples are input to the neural network for training, and the weights and biases of the internal connections of the neural network are obtained, so that the first pixel point can be detected in the neural network according to the weights and biases of the internal connections of the neural network. Probability that the first pixel point is a bad point.
  • Obtaining the probability that the first pixel point is a bad point includes: The pixel value of the pixel and the pixel value of the pixel point in the preset area are input to the neural network; the pixel value of the first pixel point and the pixel value of the pixel point in the preset area are calculated according to the weight and offset of the neural network internal connection. To obtain the probability that the first pixel point is a bad point.
  • the pixel points in the preset area are used to assist in detecting whether the first pixel point is a bad point.
  • This calculation process can be the sum of the product of the pixel values of the pixel points in the neuron in the previous layer and the weight of the internal connection of the neural network to obtain the value of the neuron in the next layer until the output layer is obtained, that is, the first pixel is
  • the probability of dead pixels and the overall rules for detecting dead pixels are simple, avoiding the difficulty of calibration under the influence of multiple rules, and has strong adaptability.
  • the method further includes: correcting the first pixel according to at least one second pixel point and a neural network in a same color channel as the first pixel point in a preset area. pixel.
  • the method for correcting a dead pixel using a neural network in this application has high correction accuracy.
  • correcting the first pixel according to at least one second pixel in the same color channel as the first pixel in the preset area and the neural network includes: for each second of the at least one second pixel Pixels, using a neural network to predict the probability that the updated first pixel is a bad pixel, and the updated first pixel is the first pixel after the value of the first pixel is replaced by the value of the second pixel.
  • Each second pixel point corresponds to a predicted probability; and the value of the second pixel point corresponding to the smallest probability among the predicted at least one probability is used to correct the first pixel point.
  • the pixels of the same color channel around the dead pixels are substituted into the dead pixels, and the neural network is used to predict the confidence of the non-bad pixels, and the value of the maximum confidence is used to correct the dead pixels.
  • This correction method corrects High precision.
  • the neural network is a fully connected neural network or a convolutional neural network.
  • a convolutional neural network When a convolutional neural network is used for the calculation, since the convolution kernel operation is used, it has a local receptive field, perceives the local part of the image, and mimics the biological visual connection to obtain local information. And through multi-layer connection, the local information is analyzed to obtain the global information, and its detection is more accurate.
  • convolutional neural network parameter sharing can effectively reduce the number of parameters and facilitate implementation on hardware with fewer resources.
  • a method for training a neural network which includes generating bad sample samples of different original picture samples; inputting each original picture sample and corresponding bad point samples into a neural network for training, and obtaining parameters of the neural network so that You can perform picture dead point detection based on this parameter.
  • the method further includes: sending parameters of the neural network, the parameters including weights and biases of the internal connection of the neural network, so that a device that receives the parameters of the neural network can perform picture dead point detection.
  • the receiving end it is not necessary to perform training on complex neural network parameters, thereby reducing the computational complexity of the receiving end.
  • generating bad sample samples of different original picture samples includes: collecting pictures of different brightness and different scenes as the original picture samples; determining image patches from each original picture sample, and the image patches are fixed Large-area image blocks; modify the original pixel value of a partial area of each image block to obtain the corresponding bad pixel samples of each original picture sample.
  • the present application can automatically generate the bad pixel samples of the original picture samples, thereby improving the efficiency of the bad pixel sample collection.
  • an apparatus including: an obtaining unit configured to select pixels in a preset area around the first pixel around a first pixel in the image; the obtaining unit is further configured to convert the first pixel
  • the pixel value of the pixel point and the pixel value of the pixel point in the preset area are input to the neural network to perform prediction to obtain a probability that the first pixel point is a bad point; and a determining unit is configured to determine the first pixel if the probability is greater than a preset threshold. Dots are bad.
  • it further includes a receiving unit for receiving parameters of the neural network, and the parameters include weights and biases of the internal connection of the neural network.
  • the obtaining unit is further configured to: generate a dead pixel sample of a different original picture sample; input each original picture sample and a corresponding dead pixel sample to a neural network for training, and obtain the weight of the internal connection of the neural network And offset.
  • the obtaining unit is configured to: input the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network; according to the pixel value of the first pixel point and the pixel value in the preset area The pixel value of the pixel is calculated with the weight and offset of the internal connection of the neural network to obtain the probability that the first pixel is a bad pixel.
  • the method further includes: a correction unit, configured to correct the first pixel point according to at least one second pixel point of the same color channel as the first pixel point in the preset area and the neural network.
  • the correction unit is configured to: for each second pixel point in at least one second pixel point, use a neural network to predict the probability that the updated first pixel point is a bad point, and the updated first pixel point A pixel is the first pixel after the value of the first pixel is replaced by the value of the second pixel, and each second pixel corresponds to a predicted probability; the first corresponding to the smallest probability among the predicted at least one probability is used.
  • the two-pixel value corrects the first pixel.
  • the neural network is a fully connected neural network or a convolutional neural network.
  • a device including: a generating unit for generating a dead pixel sample of a different original picture sample; and a training unit for inputting each original picture sample and a corresponding dead pixel sample into a neural network for training, Obtain the parameters of the neural network so that the picture dead pixels can be detected according to the parameters.
  • it further includes a sending unit for sending parameters of the neural network, and the parameters include weights and biases of the internal connections of the neural network.
  • the generating unit is used to: collect pictures of different brightness and different scenes as the original picture samples; determine image blocks from each original picture sample, and the image blocks are image blocks of a fixed size area; modify The original pixel value of a partial area of each image block is used to obtain a corresponding dead pixel sample of each original picture sample.
  • an apparatus including a processor and a memory.
  • the memory is used to store a program
  • the processor is used to call the program stored in the memory to execute the following process: selecting the first pixel with the first pixel point in the image as the center Pixels in a preset area around the point; input the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network for prediction, and obtain the probability that the first pixel point is a bad point; if the probability is greater than the If a threshold is set, it is determined that the first pixel point is a bad point.
  • a receiver is further included for receiving parameters of the neural network, and the parameters include weights and biases of the internal connection of the neural network.
  • the processor is further configured to: generate a dead pixel sample of a different original picture sample; input each original image sample and a corresponding dead pixel sample into a neural network for training, and obtain the weights of the internal connections of the neural network And offset.
  • the processor is configured to: input the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network; according to the pixel value of the first pixel point and the pixel in the preset area The pixel value of the point is calculated with the weight and offset of the internal connection of the neural network to obtain the probability that the first pixel point is a bad point.
  • the processor is further configured to correct the first pixel point according to at least one second pixel point of the same color channel as the first pixel point in the preset area and the neural network.
  • the processor is configured to: for each second pixel point in at least one second pixel point, use a neural network to predict a probability that the updated first pixel point is a bad point, and the updated first pixel
  • the point is the first pixel point after the value of the first pixel point is replaced by the value of the second pixel point, and each second pixel point corresponds to a predicted probability; the second corresponding to the smallest probability among the predicted at least one probability is used
  • the pixel value corrects the first pixel.
  • the neural network is a fully connected neural network or a convolutional neural network.
  • an apparatus including a processor and a memory, where the memory is used to store a program, and the processor is used to call the program stored in the memory to perform the following process: generating a dead pixel sample of a different original picture sample; The picture samples and the corresponding bad point samples are input to the neural network for training, and the parameters of the neural network are obtained.
  • a transmitter is further included for sending parameters of the neural network, and the parameters include weights and biases of the internal connections of the neural network.
  • the processor is used to: collect pictures of different brightness and different scenes as the original picture samples; determine image blocks from each original picture sample, and the image blocks are image blocks of a fixed size area; modify each The original pixel value of a partial area of each image block is used to obtain a corresponding dead pixel sample of each original picture sample.
  • an embodiment of the present application provides a computer storage medium for storing computer software instructions used by the foregoing device, which includes a program designed to execute the foregoing aspect.
  • an embodiment of the present application provides a computer program product containing instructions, which when executed on a computer, causes the computer to execute the methods described in the above aspects.
  • FIG. 1 is a schematic diagram of a network architecture according to an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a neural network training method according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of an internal structure of a neural network according to an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a dead point detection method according to an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a dead pixel correction method according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a scenario where an ISP is inside a device according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a scenario where an ISP is inside a device according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an image processing process according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a first network device according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a first network device according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a first network device according to an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a second network device according to an embodiment of the present application.
  • Bright point The signal value of the pixel point of the image is greater than the normal value, and its brightness value is significantly greater than the incident light multiplied by the corresponding ratio, and the brightness of the pixel point will increase significantly with the increase of the exposure time.
  • the signal value of the pixel point of the image is less than the normal value. After acquisition processing, the signal value is usually 0.
  • Neural network Also known as artificial neural network (ANN), it is an algorithm mathematical model that imitates the behavioral characteristics of animal neural networks and performs distributed parallel information processing. This kind of network depends on the complexity of the system and adjusts the interconnection relationship between a large number of internal nodes to achieve the purpose of processing information.
  • ANN artificial neural network
  • the embodiments of the present application can be used to detect image dead pixels.
  • neural network can be used to detect image dead pixels, and no manual intervention is required to implement the classification detection of complex rules.
  • the technical solution provided in this application may be applied to the network architecture shown in FIG. 1, and the network architecture may include a first network device 101 for performing a neural network training process and a second network device for detecting and correcting a picture dead point.
  • Network device 102 The first network device 101 can be a computer with high performance and can perform calculations on big data, and the second network device 102 can be a mobile terminal, such as a mobile phone, and can perform functions such as image shooting.
  • the main process of this application may include two parts, namely a neural network training process and an image dead point detection and correction process.
  • the two processes can be independent of each other and run on different time and equipment, or on the same equipment, which can be adjusted according to the actual situation.
  • the training process can perform a large number of image training on a high-performance computer to obtain the neural network parameters.
  • the computer is imported into the mobile phone, and the mobile phone can use the neural network parameters to perform a dead pixel detection and correction process on the captured image. In this way, for the detection of dead pixels, there are many rules, and debugging is difficult.
  • This application can combine neural networks and dead pixels detection. Using neural networks, this process can be regarded as an end-to-end solution. Code implementation, can complete the classification detection task of complex rules without manual intervention, and has high applicability.
  • An embodiment of the present application provides a method for training a neural network, including: generating a dead pixel sample of different original picture samples; inputting each original picture sample and a corresponding dead pixel sample into a neural network for training, and obtaining a weight of an internal connection of the neural network And offset.
  • the neural network training method may be executed in the foregoing first network device, and the first network device sends the weight and offset of the internal connection of the trained neural network to the second network device, so that the second network device may Neural network parameters are used to perform picture dead point detection; or the neural network training method may also be performed directly on the second network device, and the second network device may perform picture dead point detection directly based on the neural network parameters obtained by itself.
  • the method may specifically include:
  • a neural network is a mathematical or computational model that mimics the structure and function of biological neural networks and is used to estimate or approximate functions.
  • the neural network is calculated by a large number of artificial neurons.
  • Figure 3 shows the basic structure of a neural network.
  • the input layer can be the pixel values of image pixels
  • the hidden layer is the various layers composed of many neurons and links between the input layer and the output layer. Each layer can be the previous layer and the neuron.
  • the calculation result of the weight of the connection, the output layer (output layer) is the output result of the information transmission, analysis and weighing in the neuron link.
  • the artificial neural network is an adaptive system that can change the internal structure based on external information.
  • the connections between different neurons have different weights. By repeatedly training and adjusting the samples, the neural network can better fit the actual model.
  • a neural network is used for training a large number of pictures, and a neural network parameter used by the neural network to detect bad points in the picture is obtained.
  • the parameters may include weights and offsets.
  • Sample training In actual production, it is difficult to obtain enough samples of dead pixels through manual selection and other methods, resulting in insufficient recognition accuracy.
  • a first network device may be used to randomly generate a large number of dead pixel samples for training. Collect pictures of different brightness and different scenes from the gallery as the original picture samples. A variety of brightness can be achieved by adding bad points to the normal picture, for example, you can add bright points and dark points.
  • the bright point means that the pixel value of the image pixel point is greater than the normal value of the original pixel point
  • the dark point means that the pixel value of the image pixel point is less than the normal value of the original pixel point.
  • Different scenes can be, for example, pictures taken in different environments, such as pictures of different scenes taken in shopping malls, streets, and indoors, and can also include pictures taken during the day and night.
  • the fixed-size area may be a 7 ⁇ 7Bayer-sized matrix, and the image block includes 49 pixels, or may be a matrix of other sizes, for example, a 5 ⁇ 5Bayer-sized matrix, and the image block includes 25 pixels.
  • This application is not limited.
  • the pixel value of a pixel in a partial area of each image patch can be modified by a computer.
  • the modified pixel value is larger than the original pixel value, so that the modified pixel point becomes a bright point, or the modified pixel value is smaller than the original pixel value.
  • the pixel value, so that the modified pixel point becomes a dark point, and a picture sample with bright or dark points is a bad point sample.
  • the input layer is the pixel value of the pixel point of the original picture sample before modification, between the input layer and the hidden layer, between the hidden layer, and between the hidden layer and the output layer.
  • Each connection represents a neural network parameter, which includes weights and biases.
  • the pixel values of 6 pixels are assigned to the neurons 1 to 6, and the pixel values of the 6 pixels are respectively multiplied by 6 weights and summed, and then added
  • the bias value can get the value of neuron 7 in the hidden layer.
  • the pixel values of the other 6 pixels are multiplied by 6 weights and summed, and the offset value is added to obtain the hidden layer.
  • the value of neuron 8. In this way, the same calculation method is used between the hidden layer and the hidden layer and the output layer.
  • the result of the output of the neuron 14 of the output layer That is, the probability of the pixels of the modified bad sample is high enough, such as exceeding the first threshold, and the probability of the pixels of the original picture sample corresponding to the original image sample before being modified is low enough, such as lower than the second Threshold. That is to say, after the neural network is trained to obtain a sufficiently high detection accuracy, when it has the ability to detect dead pixels, the neural network parameters between the input layer and the hidden layer, between the hidden layer, and between the hidden layer and the output layer are saved.
  • the neural network After obtaining the neural network parameters obtained by training the neural network, the neural network can be used to detect dead pixels.
  • the neural network training method is executed on the first network device, the neural network parameters trained on the first network device may be imported into the second network device, for example, imported from a computer to a mobile phone. This allows online detection of pictures in your phone. Therefore, an embodiment of the present application further provides a method for detecting an image dead point, as shown in FIG. 4, including:
  • the second network device selects pixels in a preset area around the first pixel with the first pixel in the image as a center.
  • the second network device may be a mobile phone, which stores a neural network and neural network parameters imported from a computer.
  • a mobile phone When a mobile phone performs online detection of actual dead pixels in an image, it can traverse all the pixels in the picture to obtain the dead pixels in the image.
  • the first pixel point may be any pixel point in the traversal process.
  • the first pixel point When detecting the first pixel point, the first pixel point may be used as the center, and the pixel points in a preset area around the first pixel point may be intercepted.
  • the surrounding preset area is a 7 ⁇ 7 Bayer centered on the first pixel point.
  • a size matrix, or a 5 ⁇ 5 Bayer size matrix, etc. is not limited in this application.
  • a pair of pixel points in a preset area around the pixel point may be referred to as a small image block centered on the pixel point at the edge.
  • the second network device inputs the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network for prediction, and obtains a probability that the first pixel point is a bad point.
  • the second network device may input the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network, according to the pixel value of the first pixel point and the pixel value of the pixel point in the preset area, and the neural network.
  • the weights and offsets of internal network connections are calculated to obtain the probability that the first pixel point is a bad point.
  • the pixel points in the preset area are used to assist in predicting whether the first pixel point is a bad point during the neural network prediction process.
  • the mobile phone inputs the pixel value of the first pixel point and the pixel value of the pixel point in the preset area into the neural network, according to the trained neural network parameters, that is, the weights and biases between neurons in the neural network Set, multiply the pixel value of the first pixel in the input layer by the weight of the hidden layer, add the offset, and add the offset to get the pixel value of the neuron of the hidden layer. Then pass the pixel of the neuron between the hidden layer.
  • the calculation of values and weights and offsets, and the calculation of pixel values and weights and offsets of neurons between hidden layer and output layer gives the probability that the first pixel output by output layer is a bad point.
  • the second network device determines that the first pixel point is a bad point.
  • the preset threshold may be 85%, 90%, and the like, which is not limited in the present application. If the probability is greater than a preset threshold, the first pixel point is considered to be a bad point and needs to be corrected.
  • the preset threshold may be an empirical value. In order to obtain the empirical value, for example, a picture can be taken by a mobile phone, and the image dead point detection is performed through steps 401 to 403 to determine whether there is a dead point remaining. Set the threshold. If there are no remaining dead pixels, you can increase the preset threshold appropriately, and finally get a suitable preset threshold.
  • a neural network is combined with dead pixel detection, a large number of images are used to train the neural network, and the neural network is used to perform advanced features of the online dead pixel detection of the image to detect the dead pixels from a higher latitude. Higher accuracy avoids shortcomings of low-level features such as difficult threshold selection, too many parameters, and difficult adjustments.
  • an embodiment of the present application further provides a method for correcting a bad point.
  • the second network device may perform at least one second pixel point and a nerve in a same color channel with the first pixel point in a preset area.
  • the network corrects the first pixel.
  • the correction method may specifically include:
  • the second network device uses a neural network to predict the probability that the updated first pixel is a bad pixel, and the updated first pixel is a first pixel.
  • the first pixel after the value of is replaced by the value of the second pixel, and each second pixel corresponds to a predicted probability.
  • the first pixel point is a center point in the preset area.
  • the color channels can be R (red) G (green) B (blue) channels of three colors, and pixels of the same color have the same color channels.
  • the pixel values of the same color channel around the bad point in the preset area can be extracted, and the bad point positions are respectively input to the neural network for detection and prediction update. The probability that a pixel at the rear dead pixel position is a dead pixel.
  • the pixel values of the pixels of the same color channel around it are the closest to the pixel values of the first pixel point, so the pixels of the same color channel around the first pixel point can be extracted first to obtain At least one second pixel point, updating the pixel value of the first pixel point according to the pixel value of the first second pixel point of the at least one second pixel point, and inputting the updated pixel value of the first pixel point to the neural network Perform prediction to obtain the probability that the updated first pixel point is a bad point.
  • the second network device continues to update the pixel value of the first pixel at the next pixel value of the second pixel and inputs it to the neural network for prediction, and obtains the probability that the updated first pixel is a bad pixel until all the At least one second pixel corresponds to the predicted probability.
  • the second network device corrects the first pixel point by using a value of a second pixel point corresponding to a minimum probability among the predicted at least one probability.
  • the second network device uses a neural network to predict the probability that the updated first pixel point is a bad point, and then takes the second one with the highest confidence.
  • the pixel value of the pixel point modifies the pixel value of the first pixel point to complete the dead pixel correction.
  • the pixel values of the pixels in the same channel around the dead pixels are extracted, and the positions of the dead pixels are input to the neural network for detection, and the predicted probability of no dead pixels after the correction of the dead pixels is obtained.
  • the pixel value with the highest confidence is modified as the real pixel value of the dead pixel, thereby completing the dead pixel correction.
  • using the neural network to predict the confidence level of the surrounding pixel values in the same channel can improve the accuracy of dead pixel correction and the correction logic is simple.
  • a mobile phone when a mobile phone processes an image, it can perform a dead pixel correction in the Bayer domain.
  • the dead pixel detection and correction algorithm flow can be selected to run on different computing platforms according to actual needs.
  • Platforms that can run algorithms include hardware algorithms running on the Image Signal Processor (ISP) platform and software algorithms running on the Central Processing Unit (CPU) platform.
  • ISP Image Signal Processor
  • CPU Central Processing Unit
  • the algorithm When the algorithm is integrated into the ISP hardware platform, it has faster operation speed and lower power consumption, which can meet the needs of real-time processing of high-definition images, but its cost is higher, the iteration speed is slower, and the functional circuit is difficult to reuse. Integration, when using the CPU for processing, it is convenient to perform multiple software iterations, and the cost is low, but its operation speed is slow and power consumption is high.
  • the ISP can be integrated on the Application Processor (AP), as shown in Figure 6.
  • AP Application Processor
  • the image is processed by the AP, including through the AP.
  • the ISP platform performs dead pixel detection and correction on the image.
  • the ISP can also be used as an independent chip.
  • the image is first detected and corrected through the ISP platform, and then the detected and corrected image is further processed by the AP. deal with.
  • the dead pixel correction is a module in the image processing flow.
  • Image processing includes black level correction (for removing ground current noise), dead point detection and correction (for removing dead point data in the sensor), denoising (removing noise), converting raw data to RGB data (demosaic demosaic ), 3A (for auto white balance, autofocus, and autoexposure), sharpening (for adjusting sharpness), color space conversion (for converting to a different color space for processing), and color enhancement (optional, for Adjust color).
  • the picture is converted from the original digital signal into an image that meets the visual requirements of the human eye.
  • the position of the dead pixel correction in image processing is shown in FIG. 8.
  • the method for correcting dead pixels by using a neural network in this application does not cause misjudgment in a high-frequency region, can effectively prevent the occurrence of false colors, and does not affect the detailed texture of an image.
  • the neural network involved in the embodiment of the present application may be a fully connected neural network, and a deep neural network may also be used for on-line detection and correction of dead pixels.
  • the neural network structure includes convolutional neural network LeNet5, deep convolutional neural network structure AlexNet, Visual Geometry Group (VGG) 16, residual network ResNet and Inception structured convolutional neural network GoogLeNet, etc., different neural networks
  • the network has the capability of detecting dead pixels, which can perform the dead pixel detection and correction tasks better.
  • This application can use convolutional neural networks to detect and correct dead pixels based on actual computing capabilities.
  • convolutional neural network parameter sharing can effectively reduce the number of parameters and facilitate implementation on hardware with fewer resources.
  • the present application can be applied to mobile camera equipment for online detection of dead pixels, and can also be applied to the correction of holes or dead pixels in deep pictures, and the use of dead pixels in augmented reality (AR) or virtual reality (VR) display. Correct to avoid dead pixels on the display device.
  • AR augmented reality
  • VR virtual reality
  • devices such as the Internet of Things and the Internet of Things.
  • each network element such as the first network device and the second network device, includes a hardware structure and / or a software module corresponding to each function.
  • this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
  • the first network device, the second network device, and the like can be divided into function modules according to the foregoing method examples.
  • each function module can be divided corresponding to each function, or two or more functions can be integrated in In a processing module.
  • the above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be another division manner.
  • FIG. 9 shows a possible structural diagram of the first network device involved in the foregoing embodiment.
  • the first network device 90 includes a collection unit 901 and an acquisition unit. 902.
  • the acquisition unit 901 is configured to support the first network device to execute the process 201 in FIG. 2, and the acquisition unit 902 is configured to support the first network device to execute the processes 202, 203, and 204 in FIG. 2.
  • all relevant content of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
  • FIG. 10 shows a possible structural schematic diagram of the first network device involved in the foregoing embodiment.
  • the first network device 100 includes a processing module 1002 and a communication module 1003.
  • the processing module 1302 is configured to control and manage the actions of the first network device.
  • the processing module 1002 is configured to support the first network device to perform the processes 201, 202, 203, and 204 in FIG. 4 and / or used in the description herein. Other processes of technology.
  • the communication module 1003 is configured to support communication between the first network device and other network entities, for example, communication with the network entity shown in FIG. 1.
  • the first network device may further include a storage module 1001, configured to store program code and data of the first network device.
  • the processing module 1002 may be a processor or a controller.
  • the processing module 1002 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), and an application-specific integrated circuit (Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application.
  • the processor may also be a combination that realizes computing functions, for example, a combination including one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the communication module 1003 may be a transceiver, a transceiver circuit, or a communication interface.
  • the storage module 1001 may be a memory.
  • the first network device involved in this embodiment of the present application may be the first network device shown in FIG. 11.
  • the first network device 110 includes a processor 1102, a communication interface 1103, a memory 1101, and a bus 1104.
  • the transceiver communication interface 1103, the processor 1102, and the memory 1101 are connected to each other through a bus 1104.
  • the bus 1104 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA). ) Bus and so on.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only a thick line is used in FIG. 11, but it does not mean that there is only one bus or one type of bus.
  • FIG. 12 shows a possible structural schematic diagram of the second network device involved in the foregoing embodiment.
  • the second network device 120 includes an obtaining unit 1201 and a determining unit. 1202, a correction unit 1203, and a receiving unit 1204.
  • the obtaining unit 1201 is configured to support the second network device to execute the processes 401 and 402 in FIG. 4, the processes 502 and 503 in FIG. 5, and the determining unit 1202 is configured to support the second network device to execute the process 403 in FIG. 4, and the correction unit 1203
  • the receiving unit 1204 may be configured to receive a neural network parameter from the first network device.
  • FIG. 10 a possible schematic structural diagram of the second network device involved in the foregoing embodiment may refer to FIG. 10.
  • the structure is the same as that of the first network device, and may include a processing module and a communication module.
  • the processing module is configured to control and manage the actions of the second network device.
  • the processing module is configured to support the second network device to perform processes 401, 402, and 403 in FIG. 4, processes 501 and 502 in FIG. 5, and / or Other procedures for the techniques described herein.
  • the communication module is configured to support communication between the second network device and other network entities, for example, communication with the network entity shown in FIG. 1.
  • the second network device may further include a storage module for storing program code and data of the second network device.
  • the processing module in the second network device may be a processor or a controller, for example, it may be a central processing unit CPU, a general-purpose processor, a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA, or other Program logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application.
  • the processor may also be a combination that realizes computing functions, for example, a combination including one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the communication module in the second network device may be a transceiver, a transceiver circuit, or a communication interface.
  • the storage module in the second network device may be a memory.
  • the second network device may include a processor and a communication interface. , Memory, and bus.
  • the communication interface, the processor, and the memory are connected to each other through a bus;
  • the bus may be a peripheral component interconnect standard PCI bus or an extended industry standard structure EISA bus.
  • the bus can be divided into an address bus, a data bus, a control bus, and the like.
  • the steps of the method or algorithm described in combination with the disclosure of this application may be implemented in a hardware manner, or may be implemented in a manner that a processor executes software instructions.
  • Software instructions can be composed of corresponding software modules.
  • Software modules can be stored in Random Access Memory (RAM), flash memory, Read Only Memory (ROM), and erasable programmable read-only memory (ROM). Erasable (Programmable ROM, EPROM), electrically erasable programmable read-only memory (EPROM), registers, hard disks, mobile hard disks, read-only optical disks (CD-ROMs), or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium.
  • the storage medium may also be an integral part of the processor.
  • the processor and the storage medium may reside in an ASIC.
  • the ASIC can be located in a core network interface device.
  • the processor and the storage medium can also exist as discrete components in the core network interface device.
  • Computer-readable media includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种图像坏点检测方法及装置，涉及图像处理领域，能够解决坏点检测规则适用性差，以及检测效果差的问题。其方法为：以图像中的第一像素点为中心，选取第一像素点周围预设区域内的像素点；将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率；若概率大于预设阈值，则确定第一像素点为坏点。本申请实施例用于利用神经网络训练得到神经网络参数，再利用神经网络以及神经网络参数进行坏点检测以及校正。

Description

一种图像坏点检测方法及装置 技术领域
本申请涉及图像处理领域，尤其涉及一种图像坏点检测方法及装置。
背景技术
手机摄像头传感器由于其传感器面积较小，像素密度过高及成本限制等原因，其摄像头传感器会存在相当数量的缺陷，其中坏点的产生属于随机事件，当较暗场景长时间曝光，其坏点概率可达到5％，严重影响图像的质量。如果校正力度过强，容易导致图片中细节的丢失，并且在高频区域产生误校正导致伪彩出现；如果校正力度太弱，容易导致坏点的残留。目前，可以基于像素的绝对值、梯度、方向与阈值等进行比较来检测坏点，但是需要有经验的人员根据经验进行反复调试，校正参数难以确定。
例如以待检测的像素点为中心，取待检测的像素点周围5×5Bayer像素点建立矩阵，根据不同的颜色通道，选择该矩阵中待检测的像素点周围不同位置的同类型通道像素点的像素值进行参考，获取待检测的像素点与同类型通道每个像素点的像素差值或梯度，在该像素差值或梯度超过阈值时确定待检测的像素点为坏点。当确定待检测的像素点为坏点时，可以根据坏点周围8个同类型通道的像素点与坏点的绝对值差，确定坏点的位置，比如图像边缘，以根据坏点所在的位置来选择相应的可靠像素值，并将该可靠像素值赋值给当前坏点以进行校正。
其中，在根据阈值确定待检测的像素点是否为坏点的过程中，需要根据不同的图像传感器类型或亮度增益值设定不同的阈值，且具体的阈值需要根据不同的亮度增益进行微调，增加了调试人员的负担，并且在高频区域及线对处，由于图像种类的多样性，需要分别制定不同的检测规则，研究人员需要设置不同的规则进行拟合，同样增加人员的负担，适用性差。在校正力度过强的情况下，容易产生伪彩，过弱时坏点检测效果不够理想，检测效果差。
发明内容
本申请实施例提供一种图像坏点检测方法及装置，能够解决坏点检测规则适用性差，以及检测效果差的问题。
第一方面，提供一种图像坏点检测方法，包括：以图像中的第一像素点为中心，选取第一像素点周围预设区域内的像素点；将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率；若概率大于预设阈值，则确定第一像素点为坏点。这样利用神经网络与坏点检测相结合的方式，能够避免现有技术中坏点检测面临规则多、调试困难以及标定复杂导致坏点规则适用性差的问题，无需人工进行干预，具有较强的适用性。
在一种可能的设计中，在以像素点为中心，选取第一像素点周围预设区域内的像素点之前，该方法还包括:接收神经网络的参数，参数包括神经网络内部连接的权重和偏置，以便可以直接根据神经网络内部连接的权重和偏置对第一像素点在神经网络中进行检测，得到第一像素点为坏点的概率。
在一种可能的设计中，在以像素点为中心，选取第一像素点周围预设区域内的像素点之前，该方法还包括:生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络内部连接的权重和偏置，以便可以根据神经网络内部连接的权重和偏置对第一像素点在神经网络中进行检测，得到第一像素点为坏点的概率。
在一种可能的设计中，将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率包括：将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络；根据第一像素点的像素值以及预设区域内的像素点的像素值，和神经网络内部连接的权重和偏置进行计算，获取第一像素点为坏点的概率。预设区域内的像素点用于辅助检测第一像素点是否为坏点的。该计算过程可以是前一层神经元中像素点的像素值与神经网络内部连接的权重的乘积再取和，得到下一层的神经元的值，直至得到输出层，即第一像素点为坏点的概率，检测坏点的总体规则简单，避免多种规则影响下标定困难，具有较强的适应性。
在一种可能的设计中，在确定第一像素点为坏点之后，该方法还包括:根据预设区域内与第一像素点同颜色通道的至少一个第二像素点和神经网络校正第一像素点。本申请这种利用神经网络校正坏点的方法，校正精度高。
在一种可能的设计中，根据预设区域内与第一像素点同颜色通道的至少一个第二像素点和神经网络校正第一像素点包括:针对至少一个第二像素点中每个第二像素点，利用神经网络预测更新后的第一像素点为坏点的概率，更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，每个第二像素点对应一个预测的概率；利用预测出的至少一个概率中最小概率所对应的第二像素点的值校正第一像素点。即在坏点校正时采用坏点周围同颜色通道的像素点分别代入坏点中，并输入神经网络预测其非坏点的置信度，取其中最大置信度的值校正坏点，该校正方法校正精度高。
在一种可能的设计中，神经网络为全连接神经网络或卷积神经网络。其中采用卷积神经网络进行运算时，由于使用卷积核运算，拥有局部感受野，对图像局部进行感知，模仿生物视觉连接可获取局部的信息。并通过多层连接，将局部信息进行分析得到全局的信息，其检测更为准确。其次，卷积神经网络参数共享，可以有效减少参数数量，方便在资源较少的硬件上实现。
第二方面，提供一种神经网络训练方法，包括：生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络的参数，以便可以根据该参数进行图片坏点检测。
在一种可能的设计中，该方法还包括：发送神经网络的参数，该参数包括神经网络内部连接的权重和偏置，以便接收到该神经网络的参数的设备可以进行图片坏点检测，对于接收端来说，不需要进行计算量复杂的神经网络参数训练，降低接收端计算复杂度。
在一种可能的设计中，生成不同的原始图片样本的坏点样本包括：采集不同亮度和不同场景的图片作为原始图片样本；从每个原始图片样本中确定图像小块，图像小块为固定大小区域的图像块；修改每个图像块的部分区域的原像素值，得到每个原始图片样本相应的坏点样本。这样区别于现有技术中直接采集坏点样本的方法，本申请 可以自动生成原始图片样本的坏点样本，提高坏点样本采集效率。
第三方面，提供一种装置，包括：获取单元，用于以图像中的第一像素点为中心，选取第一像素点周围预设区域内的像素点；获取单元，还用于将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率；确定单元，用于若概率大于预设阈值，则确定第一像素点为坏点。
在一种可能的设计中，还包括接收单元，用于接收神经网络的参数，参数包括神经网络内部连接的权重和偏置。
在一种可能的设计中，获取单元还用于：生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络内部连接的权重和偏置。
在一种可能的设计中，获取单元用于：将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络；根据第一像素点的像素值以及预设区域内的像素点的像素值，和神经网络内部连接的权重和偏置进行计算，获取第一像素点为坏点的概率。
在一种可能的设计中，还包括：校正单元，用于根据预设区域内与第一像素点同颜色通道的至少一个第二像素点和神经网络校正第一像素点。
在一种可能的设计中，校正单元用于：针对至少一个第二像素点中每个第二像素点，利用神经网络预测更新后的第一像素点为坏点的概率，更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，每个第二像素点对应一个预测的概率；利用预测出的至少一个概率中最小概率所对应的第二像素点的值校正第一像素点。
在一种可能的设计中，神经网络为全连接神经网络或卷积神经网络。
第四方面，提供一种装置，包括：生成单元，用于生成不同的原始图片样本的坏点样本；训练单元，用于将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络的参数，以便可以根据该参数进行图片坏点检测。
在一种可能的设计中，还包括发送单元，用于发送神经网络的参数，该参数包括神经网络内部连接的权重和偏置。
在一种可能的设计中，生成单元用于：采集不同亮度和不同场景的图片作为原始图片样本；从每个原始图片样本中确定图像小块，图像小块为固定大小区域的图像块；修改每个图像块的部分区域的原像素值，得到每个原始图片样本相应的坏点样本。
第五方面，提供一种装置，包括处理器和存储器，存储器用于存储程序，处理器用于调用存储器中存储的程序以执行如下过程：以图像中的第一像素点为中心，选取第一像素点周围预设区域内的像素点；将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率；若概率大于预设阈值，则确定第一像素点为坏点。
在一种可能的设计中，还包括接收器，用于接收神经网络的参数，参数包括神经网络内部连接的权重和偏置。
在一种可能的设计中，处理器还用于：生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络内部连接的权重和偏置。
在一种可能的设计中，处理器用于：将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络；根据第一像素点的像素值以及预设区域内的像素点的像素值，和神经网络内部连接的权重和偏置进行计算，获取第一像素点为坏点的概率。
在一种可能的设计中，处理器还用于：根据预设区域内与第一像素点同颜色通道的至少一个第二像素点和神经网络校正第一像素点。
在一种可能的设计中，处理器用于：针对至少一个第二像素点中每个第二像素点，利用神经网络预测更新后的第一像素点为坏点的概率，更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，每个第二像素点对应一个预测的概率；利用预测出的至少一个概率中最小概率所对应的第二像素点的值校正第一像素点。
在一种可能的设计中，神经网络为全连接神经网络或卷积神经网络。
第六方面，提供一种装置，包括处理器和存储器，存储器用于存储程序，处理器用于调用存储器中存储的程序以执行如下过程：生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络的参数。
在一种可能的设计中，还包括发射器，用于发送神经网络的参数，该参数包括神经网络内部连接的权重和偏置。
在一种可能的设计中，处理器用于：采集不同亮度和不同场景的图片作为原始图片样本；从每个原始图片样本中确定图像小块，图像小块为固定大小区域的图像块；修改每个图像块的部分区域的原像素值，得到每个原始图片样本相应的坏点样本。
第七方面，本申请实施例提供了一种计算机存储介质，用于储存为上述装置所用的计算机软件指令，其包含用于执行上述方面所设计的程序。
第八方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述各方面所述的方法。
附图说明
图1为本申请实施例提供的一种网络架构的示意图；
图2为本申请实施例提供的一种神经网络训练方法的流程示意图；
图3为本申请实施例提供的一种神经网络的内部结构示意图；
图4为本申请实施例提供的一种坏点检测方法的流程示意图；
图5为本申请实施例提供的一种坏点校正方法的流程示意图；
图6为本申请实施例提供的一种ISP在设备内部的场景示意图；
图7为本申请实施例提供的一种ISP在设备内部的场景示意图；
图8为本申请实施例提供的一种图像处理流程的示意图；
图9为本申请实施例提供的一种第一网络设备的结构示意图；
图10为本申请实施例提供的一种第一网络设备的结构示意图；
图11为本申请实施例提供的一种第一网络设备的结构示意图；
图12为本申请实施例提供的一种第二网络设备的结构示意图。
具体实施方式
为了便于理解，示例地给出了部分与本申请相关概念的说明以供参考。如下所示：
坏点：图像传感器中每一光线采集点其阵列工艺存在缺陷，或者光信号转化数字信号过程中出现错误，会造成传感器得到的某些像素的信号有误，最终使得图像像素 值不正确，这些存在缺陷的像素点被称为图像坏点。
亮点：图像的像素点信号值大于正常值，其亮度值明显大于入射光乘以相应比例，并且随着曝光时间的增加，该像素点的亮度会显著增加。
暗点：图像的像素点信号值小于正常值，经过采集处理后，其信号值通常为0。
神经网络：又称为人工神经网络(artificial neural network，ANN)，是一种模仿动物神经网络行为特征，进行分布式并行信息处理的算法数学模型。这种网络依靠系统的复杂程度，通过调整内部大量节点之间相互连接的关系，从而达到处理信息的目的。
本申请实施例可以用于图像坏点的检测，具体可以利用神经网络对图像坏点进行检测，不需要人工干预实现复杂规则的分类检测。
本申请提供的技术方案可以应用于如图1所示的网络架构中，该网络架构可以包括用于进行神经网络训练流程的第一网络设备101和用于图片坏点检测及校正流程的第二网络设备102。其中，第一网络设备101可以为具有高性能的电脑，可以进行大数据的运算，第二网络设备102可以为移动终端，例如可以手机，可以进行图像拍摄等功能。
本申请的主要流程可以包括两个部分，分别为神经网络训练流程，和图像坏点检测及校正流程。两个流程可以相互独立，分别在不同的时间及设备上运行，也可以在同一设备上运行，具体根据实际情况可以进行调整。例如利用上述电脑和手机组成的网络架构，训练流程由于手机这一终端运算能力的限制，可以在具有高性能的电脑进行大量的图像训练，得到神经网络参数，将训练流程得到的神经网络参数从电脑导入到手机，手机可以利用神经网络参数对拍摄得到的图像进行坏点检测和校正流程。这样，对于坏点检测面临规则较多，调试困难等问题，本申请可以将神经网络和坏点检测相结合，利用神经网络可以将该过程看做端到端的解决方案，只需要将神经网络用代码实现，可完成复杂规则的分类检测任务而无需人工进行干预，适用性高。
下面对本申请的方法实施例进行说明。
本申请实施例提供一种神经网络训练方法，包括：生成不同的原始图片样本的坏点样本；将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络内部连接的权重和偏置。
需要说明的是，该神经网络训练方法可以在上述第一网络设备中执行，第一网络设备将训练得到的神经网络内部连接的权重和偏置发送给第二网络设备，以便第二网络设备根据神经网络参数进行图片坏点检测；或者，该神经网络训练方法也可以直接在上述第二网络设备中执行，第二网络设备直接根据自身得到的神经网络参数进行图片坏点检测。
无论在第一网络设备还是在第二网络设备中执行该神经网络训练方法，如图2所示，该方法可以具体包括：
201、采集不同亮度和不同场景的图片作为原始图片样本。
在机器学习和认知科学领域，神经网络是一种模仿生物神经网络的结构和功能的数学模型或计算模型，用于对函数进行估计或近似。神经网络由大量的人工神经元联结进行计算。如图3所示为神经网络基本结构。输入层(input layer)可以为图像像素 点的像素值，隐藏层(hidden layer)为输入层和输出层之间众多神经元和链接组成的各个层面，每个层面可以为前一层面与神经元联结的权重的计算结果，输出层(output layer)为信息在神经元链接中传输、分析和权衡形成的输出结果。大多数情况下，人工神经网络能在外界信息的基础上改变内部结构，是一种自适应系统。不同神经元之间的连接拥有不同的权重，通过对样本的反复训练和调整，可以使该神经网络更好地拟合实际模型。
本申请实施例利用神经网络进行大量图片的样本训练，获取神经网络用于检测图片坏点的神经网络参数，该参数可以包括权重及偏移，在此过程中，由于进行神经网络识别需要进行大样本训练，实际生产中难以通过手工选取等手段获得足够多的坏点样本，导致识别精度不够理想，在本申请实施例中，可以采用第一网络设备随机生成大量坏点样本进行训练，具体可以从图库中采集不同亮度和不同场景的图片作为原始图片样本。多种亮度可以通过在正常图片中添加坏点实现，例如可以添加亮点和暗点。亮点即图像像素点的像素值大于原像素点的正常值，暗点即图像像素点的像素值小于原像素点的正常值。不同场景例如可以不同环境中拍摄的图片，例如在商场、街道和室内拍摄的不同场景的图片，还可以包括白天和夜晚拍摄的图片等。
202、从每个原始图片样本中确定图像小块，图像小块为固定大小区域的图像块。
示例性的，固定大小区域可以为7×7Bayer大小的矩阵，图像小块包括49个像素点，也可以为其它大小的矩阵，例如为5×5Bayer大小的矩阵，图像小块包括25个像素点，本申请不做限定。
203、修改每个图像块的部分区域的原像素值，得到每个原始图片样本相应的坏点样本。
具体地，可以通过电脑修改每个图像小块的部分区域的像素点的像素值，例如修改后的像素值大于原像素值，使修改后的像素点成为亮点，或者修改后的像素值小于原像素值，使修改后的像素点成为暗点，存在亮点或暗点的图片样本即为坏点样本。
204、将每个原始图片样本和相应的坏点样本输入神经网络进行训练，获取神经网络内部连接的权重和偏置。
以图3为全连接神经网络的基本结构来说，input layer为修改前原始图片样本的像素点的像素值，input layer与hidden layer之间、hidden layer之间以及hidden layer与output layer之间的每个连接表示神经网络参数，该参数包括权重和偏置。以input layer的神经元1～6来说，对神经元1～6赋值6个像素点的像素值，将该6个像素点的像素值分别与6个权重相乘后取和，再加上偏置值可得到hidden layer中神经元7的值。同理，神经元1～6赋值为另外6个像素点的像素值时，另外6个像素点的像素值分别与6个权重相乘后取和，再加上偏置值可得到hidden layer中神经元8的值。这样，hidden layer之间以及hidden layer与output layer之间使用同样的计算方法，通过多次迭代，并修改神经元之间的权重和偏置的值，使得output layer的神经元14输出的结果，即修改后坏点样本的像素点为坏点的概率足够高，例如超过第一阈值，以及坏点样本相应的修改前原始图片样本的像素点为坏点的概率足够低，例如低于第二阈值。也即使得神经网络训练后获得足够高的检测精度，具有检测坏点的能力时，将input layer与hidden layer之间、hidden layer之间以及hidden layer与output layer之间的神经网络参 数进行保存。
在获取神经网络经过训练得到的神经网络参数后，该神经网络可以用于检测坏点。在本申请实施例中，如果神经网络训练方法在第一网络设备上执行，可以将第一网络设备上训练得到的神经网络参数导入第二网络设备，例如从电脑导入到手机中。这样就可以在手机中对图片进行在线检测。因此，本申请实施例还提供一种图像坏点检测方法，如图4所示，包括：
401、第二网络设备以图像中的第一像素点为中心，选取第一像素点周围预设区域内的像素点。
第二网络设备可以为手机，存储有神经网络以及从电脑导入的神经网络参数。手机在进行图像实际坏点在线检测时，可以遍历图片上所有的像素点，以获取图像中的坏点。以其中第一像素点来说，第一像素点可以为遍历过程中的任一像素点。对第一像素点进行检测时，可以以第一像素点为中心，截取第一像素点周围预设区域内的像素点，例如该周围预设区域为以第一像素点为中心的7×7Bayer大小的矩阵，或者5×5Bayer大小的矩阵等，本申请不做限定。其中对于图像边缘的像素点，可以将该像素点周围预设区域内的像素点对称为以该边缘的像素点为中心的图像小块。
402、第二网络设备将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络进行预测，获取第一像素点为坏点的概率。
第二网络设备可以将第一像素点的像素值以及预设区域内的像素点的像素值输入神经网络，根据第一像素点的像素值以及预设区域内的像素点的像素值，和神经网络内部连接的权重和偏置进行计算，获取第一像素点为坏点的概率。预设区域内的像素点用于在神经网络预测过程中辅助预测第一像素点是否为坏点。
示例性的，手机将第一像素点的像素值以及预设区域内的像素点的像素值输入到神经网络后，根据训练得到的神经网络参数，即神经网络中神经元之间的权重和偏置，将input layer中第一像素点的像素值与hidden layer的权重相乘取和，再加上偏置，得到hidden layer的神经元的像素值，再通过hidden layer之间的神经元的像素值与权重和偏置的计算，以及hidden layer与output layer之间的神经元的像素值与权重和偏置的计算，得到output layer输出的第一像素点为坏点的概率。
403、若概率大于预设阈值，则第二网络设备确定第一像素点为坏点。
示例性的，该预设阈值可以为85％、90％等，本申请不做限定。如果该概率大于预设阈值，则认为该第一像素点为坏点，需要进行校正。该预设阈值可以为经验值，为了得到该经验值，例如可以通过手机拍摄图片，通过步骤401至403进行图像坏点检测，确定是否残留有坏点，如果残留有坏点可以调低该预设阈值，如果没有残留坏点，可以适当调高该预设阈值，最终得到合适的预设阈值。
这样将神经网络和坏点检测相结合，通过大量多样的图像训练神经网络，利用神经网络进行图像坏点在线检测的高级特征，从更高的纬度上对坏点进行检测，使得坏点检测拥有较高的准确率，避免了低级特征如阈值难以选取、参数过多及调整困难等缺点。
若确定第一像素点为坏点，本申请实施例还提供一种坏点校正方法，第二网络设备可以根据预设区域内与第一像素点同颜色通道的至少一个第二像素点和神经网络校 正第一像素点。如图5所示，该校正方法具体可以包括：
501、第二网络设备针对至少一个第二像素点中每个第二像素点，利用神经网络预测更新后的第一像素点为坏点的概率，更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，每个第二像素点对应一个预测的概率。
第一像素点为预设区域内的中心点。颜色通道可以为R(红)G(绿)B(蓝)三种颜色的通道，相同颜色的像素点具有同颜色通道。为了获取第一像素点这一坏点的真实像素值，可以将预设区域内该坏点周围同颜色通道的像素值进行提取，依次分别代入坏点位置输入到神经网络中进行检测，预测更新后坏点位置的像素点为坏点的概率。
第一像素点为坏点时，其周围同颜色通道的像素点的像素值与第一像素点的像素值最为接近，因此可以先将第一像素点周围同颜色通道的像素点进行提取，得到至少一个第二像素点，根据该至少一个第二像素点中第一个第二像素点的像素值更新第一像素点的像素值，将更新后的第一像素点的像素值输入到神经网络进行预测，获取更新后的第一像素点为坏点的概率。而后第二网络设备继续以下一个第二像素点的像素值更新第一像素点的像素值输入到神经网络进行预测，获取此次更新后的第一像素点为坏点的概率，直至获取全部该至少一个第二像素点对应预测后的概率。
502、第二网络设备利用预测出的至少一个概率中最小概率所对应的第二像素点的值校正第一像素点。
也可以说，在第二网络设备针对至少一个第二像素点中每个第二像素点，利用神经网络预测更新后的第一像素点为坏点的概率后，取其中置信度最高的第二像素点的像素值修改第一像素点的像素值，完成坏点校正。
也即，手机进行图像校正时，将坏点周围同通道像素点的像素值进行提取，分别代入坏点位置输入到神经网络中进行检测，得到坏点校正后无坏点的预测概率，取其中置信度最高的像素值作为该坏点的真实像素值进行修改，从而完成坏点校正。这样利用神经网络预测周围同通道像素值的置信度，能够提高坏点校正的精度，校正逻辑简单。
上述检测和校正过程中，手机对图像进行处理时，可以在Bayer域进行坏点校正。坏点检测和校正算法流程可以根据实际需要选择在不同的运算平台上运行。可运行算法的平台包括硬件算法运行于图像信号处理器(Image Signal Processor，ISP)平台和软件算法运行于中央处理器(Central Processing Unit，CPU)平台。当算法集成到ISP硬件平台，其运算速度更快，功耗更低，可以满足高清图像实时处理的需要，但其成本较高，迭代速度较慢，功能电路难以复用；当算法为软件方案集成，利用CPU进行处理时，方便进行多次软件迭代，成本较低，但是其运算速度较慢，功耗较高，在选用算法时，可以根据实际情况，选择不同的算法运算平台。
针对硬件算法运行于ISP平台场景，ISP可以集成在应用处理器(Application Processor，AP)上，如图6所示，在通过相机模块拍摄得到图像后，通过AP对图像进行处理，包括通过AP中的ISP平台对图像进行坏点检测和校正。ISP也可以作为一个独立的芯片，如图7所示，在通过相机模块拍摄得到图像后，先通过ISP平台对图像进行坏点检测和校正，而后再通过AP对检测和校正后的图像进行进一步处理。
在上述ISP平台上，坏点校正为图像处理流程中的一个模块。图像处理包括黑电平校正(用于去除底电流噪声)、坏点检测及校正(用于去除传感器中坏点数据)、去噪(去除噪声)、将原始数据转为RGB数据(去马赛克Demosaic)、3A(用于自动白平衡，自动对焦及自动曝光)、锐化(用于调整锐度)、色彩空间转换(用于转换到不同色彩空间进处理)和颜色增强(可选，用于调整颜色)。经过上述处理后，图片从原始的数字信号转换为符合人眼视觉需求的图像。坏点校正在图像处理中的位置如图8所示。
需要说明的是，本申请中通过神经网络进行坏点校正的方法不会对高频区域产生误判，可以有效避免伪彩的产生，不会对图像的细节纹理产生影响。
本申请实施例中涉及到的神经网络可以为全连接神经网络，还可以采用深度神经网络进行坏点在线检测及校正。目前，神经网络结构包括卷积神经网络LeNet5、深度卷积神经网络结构AlexNet、视觉几何组(Visual Geometry Group，VGG)16、残差网络ResNet和Inception结构的卷积神经网络GoogLeNet等，不同的神经网络均拥有坏点检测能力，能较好的进行坏点检测及校正任务。本申请根据实际运算能力，可以采用卷积神经网络进行坏点检测及校正。
这是由于使用卷积核运算，拥有局部感受野，对图像局部进行感知，模仿生物视觉神经连接可获取局部的信息。并通过多层连接，将局部信息进行分析得到全局的信息，其检测更为准确，利用卷积神经网络进行坏点检测的精度相对于全连接神经网络可以提高近5％。其次卷积神经网络参数共享，可以有效减少参数数量，方便在资源较少的硬件上实现。
本申请可以应用于移动拍照设备进行坏点在线检测，还可以应用于深度图片的孔洞或坏点校正，以及增强现实(Augmented Reality，AR)或虚拟现实(Virtual Reality，VR)显示时采用坏点校正避免显示设备出现坏点。此外，还可以应用于车联网和物联网等设备上。
上述主要从各个网元之间交互的角度对本申请实施例提供的方案进行了介绍。可以理解的是，各个网元，例如第一网络设备、第二网络设备等为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对第一网络设备、第二网络设备等进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图9示出了上述实施例中所涉及的第一网络设备的一种可能的结构示意图，第一网络设备90包括：采集单元901， 获取单元902。采集单元901用于支持第一网络设备执行图2中的过程201，获取单元902用于支持第一网络设备执行图2中的过程202、203以及204。其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
在采用集成的单元的情况下，图10示出了上述实施例中所涉及的第一网络设备的一种可能的结构示意图。第一网络设备100包括：处理模块1002和通信模块1003。处理模块1302用于对第一网络设备的动作进行控制管理，例如，处理模块1002用于支持第一网络设备执行图4中的过程201、202、203以及204，和/或用于本文所描述的技术的其它过程。通信模块1003用于支持第一网络设备与其他网络实体的通信，例如与图1中示出的网络实体之间的通信。第一网络设备还可以包括存储模块1001，用于存储第一网络设备的程序代码和数据。
其中，处理模块1002可以是处理器或控制器，例如可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，专用集成电路(Application-Specific Integrated Circuit，ASIC)，现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块1003可以是收发器、收发电路或通信接口等。存储模块1001可以是存储器。
当处理模块1002为处理器，通信模块1003为通信接口，存储模块1001为存储器时，本申请实施例所涉及的第一网络设备可以为图11所示的第一网络设备。
参阅图11所示，该第一网络设备110包括：处理器1102、通信接口1103、存储器1101以及总线1104。其中，收发器通信接口1103、处理器1102以及存储器1101通过总线1104相互连接；总线1104可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图11中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
在采用对应各个功能划分各个功能模块的情况下，图12示出了上述实施例中所涉及的第二网络设备的一种可能的结构示意图，第二网络设备120包括：获取单元1201，确定单元1202、校正单元1203以及接收单元1204。获取单元1201用于支持第二网络设备执行图4中的过程401和402，图5中的过程502和503，确定单元1202用于支持第二网络设备执行图4中的过程403，校正单元1203用于支持第二网络设备执行图5中的过程501以及502，接收单元1204可以用于从第一网络设备接收神经网络参数。其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
在采用集成的单元的情况下，上述实施例中所涉及的第二网络设备的一种可能的结构示意图可以参考图10，与第一网络设备的结构相同，可以包括：处理模块和通信模块。处理模块用于对第二网络设备的动作进行控制管理，例如，处理模块用于支持第二网络设备执行图4中的过程401、402以及403，图5中的过程501和502，和/ 或用于本文所描述的技术的其它过程。通信模块用于支持第二网络设备与其他网络实体的通信，例如与图1中示出的网络实体之间的通信。第二网络设备还可以包括存储模块，用于存储第二网络设备的程序代码和数据。
其中，第二网络设备中的处理模块可以是处理器或控制器，例如可以是中央处理器CPU，通用处理器，数字信号处理器DSP，专用集成电路ASIC，现场可编程门阵列FPGA或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。第二网络设备中的通信模块可以是收发器、收发电路或通信接口等。第二网络设备中的存储模块可以是存储器。
与图11中第一网络设备的结构类似，当第二网络设备中的处理模块为处理器，通信模块为通信接口，存储模块为存储器时，该第二网络设备可以包括：处理器、通信接口、存储器以及总线。其中，通信接口、处理器以及存储器通过总线相互连接；总线可以是外设部件互连标准PCI总线或扩展工业标准结构EISA总线等。所述总线可以分为地址总线、数据总线、控制总线等。
结合本申请公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于核心网接口设备中。当然，处理器和存储介质也可以作为分立组件存在于核心网接口设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何在本发明揭露的技术范围内的变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (23)

  1. 一种图像坏点检测方法，其特征在于，包括：
    以图像中的第一像素点为中心，选取所述第一像素点周围预设区域内的像素点；
    将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入神经网络进行预测，获取所述第一像素点为坏点的概率；
    若所述概率大于预设阈值，确定所述第一像素点为坏点。
  2. 根据权利要求1所述的图像坏点检测方法，其特征在于，在以所述像素点为中心，选取所述第一像素点周围预设区域内的像素点之前，所述方法还包括:
    接收所述神经网络的参数，所述参数包括所述神经网络内部连接的权重和偏置。
  3. 根据权利要求1所述的图像坏点检测方法，其特征在于，在以所述像素点为中心，选取所述第一像素点周围预设区域内的像素点之前，所述方法还包括:
    生成不同的原始图片样本的坏点样本；
    将每个所述原始图片样本和所述相应的坏点样本输入所述神经网络进行训练，获取所述神经网络内部连接的权重和偏置。
  4. 根据权利要求2或3所述的图像坏点检测方法，其特征在于，将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入神经网络进行预测，获取所述第一像素点为坏点的概率包括：
    将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入所述神经网络；
    根据所述第一像素点的像素值以及所述预设区域内的像素点的像素值，和所述神经网络内部连接的所述权重和所述偏置进行计算，获取所述第一像素点为所述坏点的概率。
  5. 根据权利要求1-4任一项所述的图像坏点检测方法，其特征在于，在确定所述第一像素点为坏点之后，所述方法还包括:
    根据所述预设区域内与所述第一像素点同颜色通道的至少一个第二像素点和所述神经网络校正所述第一像素点。
  6. 根据权利要求5所述的图像坏点检测方法，其特征在于，所述根据所述预设区域内与所述第一像素点同颜色通道的至少一个第二像素点和所述神经网络校正所述第一像素点包括:
    针对所述至少一个第二像素点中每个第二像素点，利用所述神经网络预测更新后的第一像素点为坏点的概率，所述更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，所述每个第二像素点对应一个预测的概率；
    利用所述预测出的至少一个概率中最小概率所对应的第二像素点的值校正所述第一像素点。
  7. 根据权利要求1-6任一项所述的图像坏点检测方法，其特征在于，所述神经网络为全连接神经网络或卷积神经网络。
  8. 一种装置，其特征在于，包括：
    获取单元，用于以图像中的第一像素点为中心，选取所述第一像素点周围预设区域内的像素点；
    所述获取单元，还用于将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入神经网络进行预测，获取所述第一像素点为坏点的概率；
    确定单元，用于若所述概率大于预设阈值，则确定所述第一像素点为坏点。
  9. 根据权利要求8所述的装置，其特征在于，还包括接收单元，用于接收所述神经网络的参数，所述参数包括所述神经网络内部连接的权重和偏置。
  10. 根据权利要求8所述的装置，其特征在于，所述获取单元还用于：
    生成不同的原始图片样本的坏点样本；
    将每个所述原始图片样本和所述相应的坏点样本输入所述神经网络进行训练，获取所述神经网络内部连接的权重和偏置。
  11. 根据权利要求9或10所述的装置，其特征在于，所述获取单元用于：
    将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入所述神经网络；
    根据所述第一像素点的像素值以及所述预设区域内的像素点的像素值，和所述神经网络内部连接的所述权重和所述偏置进行计算，获取所述第一像素点为所述坏点的概率。
  12. 根据权利要求8-11任一项所述的装置，其特征在于，还包括校正单元，用于根据所述预设区域内与所述第一像素点同颜色通道的至少一个第二像素点和所述神经网络校正所述第一像素点。
  13. 根据权利要求12所述的装置，其特征在于，所述校正单元用于：
    针对所述至少一个第二像素点中每个第二像素点，利用所述神经网络预测更新后的第一像素点为坏点的概率，所述更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，所述每个第二像素点对应一个预测的概率；
    利用所述预测出的至少一个概率中最小概率所对应的第二像素点的值校正所述第一像素点。
  14. 根据权利要求8-13任一项所述的装置，其特征在于，所述神经网络为全连接神经网络或卷积神经网络。
  15. 一种装置，其特征在于，包括处理器和存储器，所述存储器用于存储程序，所述处理器用于调用所述存储器中存储的程序以执行如下过程：
    以图像中的第一像素点为中心，选取所述第一像素点周围预设区域内的像素点；将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入神经网络进行预测，获取所述第一像素点为坏点的概率；若所述概率大于预设阈值，则确定所述第一像素点为坏点。
  16. 根据权利要求15所述的装置，其特征在于，还包括接收器，用于接收所述神经网络的参数，所述参数包括所述神经网络内部连接的权重和偏置。
  17. 根据权利要求15所述的装置，其特征在于，所述处理器还用于：
    生成不同的原始图片样本的坏点样本；
    将每个所述原始图片样本和所述相应的坏点样本输入所述神经网络进行训练，获取所述神经网络内部连接的权重和偏置。
  18. 根据权利要求16或17所述的装置，其特征在于，所述处理器用于：
    将所述第一像素点的像素值以及所述预设区域内的像素点的像素值输入所述神经网络；
    根据所述第一像素点的像素值以及所述预设区域内的像素点的像素值，和所述神经网络内部连接的所述权重和所述偏置进行计算，获取所述第一像素点为所述坏点的概率。
  19. 根据权利要求15-18任一项所述的装置，其特征在于，所述处理器还用于：根据所述预设区域内与所述第一像素点同颜色通道的至少一个第二像素点和所述神经网络校正所述第一像素点。
  20. 根据权利要求19所述的装置，其特征在于，所述处理器用于：
    针对所述至少一个第二像素点中每个第二像素点，利用所述神经网络预测更新后的第一像素点为坏点的概率，所述更新后的第一像素点为第一像素点的值被第二像素点的值替换后的第一像素点，所述每个第二像素点对应一个预测的概率；
    利用所述预测出的至少一个概率中最小概率所对应的第二像素点的值校正所述第一像素点。
  21. 根据权利要求15-20任一项所述的装置，其特征在于，所述神经网络为全连接神经网络或卷积神经网络。
  22. 一种计算机存储介质，其特征在于，用于储存为上述装置所用的计算机软件指令，其包含用于执行上述权利要求1-7所设计的程序。
  23. 一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，所述计算机用于执行上述权利要求1-7所设计的指令。
PCT/CN2018/095712 2018-07-13 2018-07-13 一种图像坏点检测方法及装置 WO2020010638A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880095536.2A CN112384946A (zh) 2018-07-13 2018-07-13 一种图像坏点检测方法及装置
PCT/CN2018/095712 WO2020010638A1 (zh) 2018-07-13 2018-07-13 一种图像坏点检测方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/095712 WO2020010638A1 (zh) 2018-07-13 2018-07-13 一种图像坏点检测方法及装置

Publications (1)

Publication Number Publication Date
WO2020010638A1 true WO2020010638A1 (zh) 2020-01-16

Family

ID=69143145

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/095712 WO2020010638A1 (zh) 2018-07-13 2018-07-13 一种图像坏点检测方法及装置

Country Status (2)

Country Link
CN (1) CN112384946A (zh)
WO (1) WO2020010638A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220005168A1 (en) * 2020-07-03 2022-01-06 Samsung Electronics Co., Ltd. Image processing apparatus including neural network processor and method of operation
CN114494260A (zh) * 2022-04-18 2022-05-13 深圳思谋信息科技有限公司 对象缺陷检测方法、装置、计算机设备和存储介质
WO2023134846A1 (en) * 2022-01-12 2023-07-20 Dream Chip Technologies Gmbh Method and image processor unit for detecting defective pixels

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113657238B (zh) * 2021-08-11 2024-02-02 南京精益安防系统科技有限公司 基于神经网络的火灾早期预警方法、存储介质及终端设备
CN114677574B (zh) * 2022-05-26 2022-10-21 杭州宏景智驾科技有限公司 用于自动驾驶的诊断图像故障的方法和系统
CN116228756B (zh) * 2023-05-08 2023-07-25 常州星宇车灯股份有限公司 一种自动驾驶中相机坏点的检测方法及检测系统
CN116886894B (zh) * 2023-08-04 2024-03-15 上海宇勘科技有限公司 一种针对邻域内列式坏点的图片坏点检测及校正方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0676069A (ja) * 1992-08-27 1994-03-18 Toyota Central Res & Dev Lab Inc 表面欠陥検出装置
JPH06119454A (ja) * 1992-10-08 1994-04-28 Babcock Hitachi Kk 異常検出方法および装置
CN103475828A (zh) * 2013-10-10 2013-12-25 旗瀚科技有限公司 一种图像坏点校正方法及图像传感器
CN106845556A (zh) * 2017-02-09 2017-06-13 东华大学 一种基于卷积神经网络的织物疵点检测方法
CN107169956A (zh) * 2017-04-28 2017-09-15 西安工程大学 基于卷积神经网络的色织物疵点检测方法
CN107316295A (zh) * 2017-07-02 2017-11-03 苏州大学 一种基于深度神经网络的织物瑕疵检测方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101715050B (zh) * 2009-11-20 2014-07-09 北京中星微电子有限公司 一种图像传感器坏点检测的方法和系统
CN101807245B (zh) * 2010-03-02 2013-01-02 天津大学 基于人工神经网络的多源步态特征提取与身份识别方法
US10123268B2 (en) * 2010-06-04 2018-11-06 Board Of Regents, The University Of Texas System Wireless communication methods, systems, and computer program products
CN105868797B (zh) * 2015-01-22 2019-09-13 清华大学 识别景物类型的网络参数训练方法、景物类型识别方法及装置
CN104715282B (zh) * 2015-02-13 2017-12-05 浙江工业大学 一种基于改进pso‑bp神经网络的数据预测方法
CN106210712B (zh) * 2016-08-11 2018-07-10 上海大学 一种图像坏点检测及处理方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0676069A (ja) * 1992-08-27 1994-03-18 Toyota Central Res & Dev Lab Inc 表面欠陥検出装置
JPH06119454A (ja) * 1992-10-08 1994-04-28 Babcock Hitachi Kk 異常検出方法および装置
CN103475828A (zh) * 2013-10-10 2013-12-25 旗瀚科技有限公司 一种图像坏点校正方法及图像传感器
CN106845556A (zh) * 2017-02-09 2017-06-13 东华大学 一种基于卷积神经网络的织物疵点检测方法
CN107169956A (zh) * 2017-04-28 2017-09-15 西安工程大学 基于卷积神经网络的色织物疵点检测方法
CN107316295A (zh) * 2017-07-02 2017-11-03 苏州大学 一种基于深度神经网络的织物瑕疵检测方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220005168A1 (en) * 2020-07-03 2022-01-06 Samsung Electronics Co., Ltd. Image processing apparatus including neural network processor and method of operation
US11748862B2 (en) * 2020-07-03 2023-09-05 Samsung Electronics Co., Ltd. Image processing apparatus including neural network processor and method of operation
WO2023134846A1 (en) * 2022-01-12 2023-07-20 Dream Chip Technologies Gmbh Method and image processor unit for detecting defective pixels
CN114494260A (zh) * 2022-04-18 2022-05-13 深圳思谋信息科技有限公司 对象缺陷检测方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN112384946A (zh) 2021-02-19

Similar Documents

Publication Publication Date Title
WO2020010638A1 (zh) 一种图像坏点检测方法及装置
WO2021164234A1 (zh) 图像处理方法以及图像处理装置
CN112118388B (zh) 图像处理方法、装置、计算机设备和存储介质
CN115442515A (zh) 图像处理方法和设备
CN111062876B (zh) 用于校正模型训练和图像校正的方法和装置及电子设备
CN114972085B (zh) 一种基于对比学习的细粒度噪声估计方法和系统
WO2023125750A1 (zh) 一种图像去噪方法、装置和存储介质
CN112132172A (zh) 基于图像处理的模型训练方法、装置、设备和介质
CN110717864B (zh) 一种图像增强方法、装置、终端设备及计算机可读介质
CN113962859A (zh) 一种全景图生成方法、装置、设备及介质
CN113538223A (zh) 噪声图像生成方法、装置、电子设备及存储介质
WO2023005115A1 (zh) 图像处理方法、图像处理装置、电子设备和可读存储介质
CN114429476A (zh) 图像处理方法、装置、计算机设备以及存储介质
CN107392870A (zh) 图像处理方法、装置、移动终端及计算机可读存储介质
CN117156289A (zh) 色彩风格校正方法、系统、电子设备、存储介质及芯片
CN107424134A (zh) 图像处理方法、装置、计算机可读存储介质和计算机设备
CN112950509B (zh) 一种图像处理方法、装置及电子设备
WO2022183321A1 (zh) 图像检测方法、装置和电子设备
CN112734673B (zh) 一种基于多表达式融合的低照度图像增强方法及系统
CN115209114A (zh) 自动白平衡的方法
CN107317968A (zh) 图像去雾方法、装置、计算机可存储介质和移动终端
CN115731115A (zh) 一种数据处理方法以及装置
CN113870180A (zh) 图像处理方法、装置、计算机设备和存储介质
TWI590192B (zh) 適應性高動態範圍影像合成演算法
WO2021102947A1 (zh) 图像信号处理装置、方法、相机以及可移动平台

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18925832

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18925832

Country of ref document: EP

Kind code of ref document: A1