WO2021207879A1

WO2021207879A1 - Image processing method and apparatus, electronic device and storage medium

Info

Publication number: WO2021207879A1
Application number: PCT/CN2020/084498
Authority: WO
Inventors: 周杰旻; 江君
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-04-13
Filing date: 2020-04-13
Publication date: 2021-10-21
Also published as: CN112544074B; CN112544074A

Abstract

Provided is an image processing method. The method comprises: acquiring an image to be processed, wherein said image is generated according to image information collected by an image sensor; determining contaminated pixels in said image, wherein the contaminated pixels comprise pixels, the pixel values of which are affected by at least one dead image pixel in said image in the process of generating said image according to the image information; updating dead pixel information of the image sensor from first dead pixel information to second dead pixel information on the basis of information of the contaminated pixels; and on the basis of the second dead pixel information, processing said image so as to correct the pixel value of the dead image pixel. Also provided are an image processing apparatus, an electronic device and a storage medium.

Description

Image processing method, device, electronic equipment and storage medium

Technical field

The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic equipment, and storage medium.

Background technique

With the advancement of science and technology and the increasing requirements of users for image clarity, image sensor technology continues to develop. Both the hardware of the image sensor and the image processing algorithm have been greatly improved.

However, with the development of image sensor technology, the traditional dead pixel correction method is no longer suitable for correcting the image collected by the image sensor.

Summary of the invention

An aspect of the present disclosure provides an image processing method, including: obtaining an image to be processed, the image to be processed is generated according to image information collected by an image sensor; determining contaminated pixels in the image to be processed, Contaminated pixels include pixels whose pixel values are affected by at least one bad image in the image to be processed in the process of generating the image to be processed according to the image information; based on the information of the contaminated pixel, the image sensor The dead pixel information of is updated from the first dead pixel information to the second dead pixel information; and based on the second dead pixel information, the image to be processed is processed so as to correct the pixel value of the image dead pixel.

According to an embodiment of the present disclosure, the updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information includes: determining, according to the first dead pixel information, that at least the image to be processed is at least Location information of a dead pixel in an image; determining the location information of a contaminated pixel in the image to be processed according to the location information of the at least one dead pixel in an image; determining the credibility of the pixel value of the contaminated pixel; and The position information of the contaminated pixel and the reliability update the first dead pixel information to generate the second dead pixel information.

According to an embodiment of the present disclosure, the determining the position information of the contaminated pixel in the image to be processed includes: setting a pollution parameter of the image sensor, and the pollution parameter indicates that the to-be-processed image is generated according to the image information. In the process of processing the image, the range of pixels that can be affected by a dead pixel in the image; and determining the location information of the contaminated pixel based on the pollution parameter and the location information of the at least one dead pixel in the image.

According to an embodiment of the present disclosure, the determining the credibility of the pixel value of the contaminated pixel includes: determining the corresponding relationship between the distance and the credibility, and the distance is the amount of the contaminated pixel that affects the contaminated pixel. The Euclidean distance between the image dead pixels of the pixels; based on the location information of the contaminated pixel and the location information of the image dead pixels, determine the distance between the contaminated pixel and the image dead pixel that affects the contaminated pixel And based on the correspondence and the Euclidean distance, determine the credibility of the pixel value of the contaminated pixel.

According to an embodiment of the present disclosure, the processing the image to be processed based on the second dead pixel information so as to correct the pixel value of the image dead pixel includes: based on the location information of the at least one image dead pixel Determine a plurality of reference pixels, so as to use the respective pixel values of the plurality of reference pixels to correct the at least one bad image; determine whether there is a contaminated pixel in the plurality of reference pixels; in determining the plurality of reference pixels If there is a contaminated pixel, query the second bad pixel information to determine the credibility of the pixel value of the contaminated pixel; and based on the respective pixel values of the multiple reference pixels and the multiple reference pixels Calculate the credibility of the pixel value of the contaminated pixel, and correct the pixel value of the bad pixel of the image.

According to an embodiment of the present disclosure, based on the reliability of the respective pixel values of the plurality of reference pixels and the pixel values of the contaminated pixels in the plurality of reference pixels, correcting the pixel values of the defective image pixels includes: Determine the pixel to be eliminated, the pixel to be eliminated is a pixel whose pixel value of the contaminated pixel among the plurality of reference pixels is less than a preset threshold; and based on the plurality of reference pixels other than the eliminated pixel The pixel values of other reference pixels are used to correct the pixel values of the dead pixels of the image.

According to an embodiment of the present disclosure, the correcting the pixel value of the defective image pixel based on the pixel value of the reference pixel other than the culling pixel among the plurality of reference pixels includes: performing an image interpolation algorithm on the pixel value of the pixel. The pixel values of the reference pixels other than the excluded pixels are interpolated to generate the pixel values of the dead pixels of the image.

According to an embodiment of the present disclosure, obtaining an image to be processed includes: obtaining image information collected by a four Bayer image sensor; and an image obtained by demosaicing the image information.

According to an embodiment of the present disclosure, obtaining image data includes: obtaining image information collected by an image sensor, the image information including pixel values of a plurality of pixels; and combining the pixel values of the plurality of pixels to obtain the to-be-processed image.

According to an embodiment of the present disclosure, the updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information further includes: determining a dynamic dead pixel, wherein the dynamic dead pixel is in the When the ambient light collected by the image sensor is the first reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is less than the preset value, and the ambient light collected by the image sensor is the second reflected light In the case of the pixel value and the pixel value of the pixel in the preset area, the difference between the pixel value is greater than the preset value, wherein the ambient light is the light reflected by the object collected by the image sensor; and the dynamic damage Dot and the location information of the dynamic dead pixel are added to the first dead pixel information, so that when the pixel value of the dynamic dead pixel is not within the specific range, the image dead pixel includes the dynamic bad pixel point.

Another aspect of the present disclosure provides an image processing device, including: an acquisition module for acquiring an image to be processed, the image to be processed is generated according to image information collected by an image sensor; and a determining module for determining the Contaminated pixels in the image to be processed, where the contaminated pixels include pixels whose pixel values are affected by at least one bad image in the image to be processed during the process of generating the image to be processed according to the image information; an update module, Based on the information of the contaminated pixel, the dead pixel information of the image sensor is updated from the first dead pixel information to the second dead pixel information; and the correction module is used for correcting based on the second dead pixel information The image to be processed is processed so as to correct the pixel value of the dead pixel of the image.

According to an embodiment of the present disclosure, the update module includes: a first determining submodule, configured to determine location information of at least one image defect in the image to be processed according to the first defect information; and a second determining submodule, It is used to determine the location information of the contaminated pixel in the image to be processed according to the location information of the at least one dead pixel of the image; the credibility determination sub-module is used to determine the credibility of the pixel value of the contaminated pixel And a generating sub-module for updating the first dead pixel information based on the location information of the contaminated pixel and the credibility to generate the second dead pixel information.

According to an embodiment of the present disclosure, the second determining sub-module includes: a first determining unit configured to determine a pollution parameter of the image sensor, the pollution parameter indicating that the to-be-processed is generated according to the image information During the image process, the range of pixels that can be affected by a dead pixel in the image; and a second determining unit configured to determine the location information of the contaminated pixel based on the pollution parameter and the location information of the at least one dead pixel in the image.

According to an embodiment of the present disclosure, the credibility determination sub-module includes: a third determining unit, configured to determine the corresponding relationship between the distance and the credibility, and the distance is between the contaminated pixel and the amount that affects the contaminated pixel. The Euclidean distance between the bad pixels of the image; the fourth determining unit is used to determine, based on the position information of the contaminated pixel and the position information of the bad image of the image, that the contaminated pixel is to the extent that affects the contaminated pixel The Euclidean distance between the dead pixels of the image; and a fifth determining unit for determining the reliability of the pixel value of the contaminated pixel based on the correspondence and the Euclidean distance.

According to an embodiment of the present disclosure, the correction module includes: a third determination sub-module, configured to determine a plurality of reference pixels based on the location information of the at least one image dead pixel, so as to use the respective pixel values of the plurality of reference pixels to compare the at least A dead pixel of an image is corrected; the fourth determining sub-module is used to determine whether there are contaminated pixels in the multiple reference pixels; the query sub-module is used to determine whether there are contaminated pixels in the multiple reference pixels Next, query the second dead pixel information to determine the credibility of the pixel value of the contaminated pixel; and a correction sub-module, which is configured to be based on the respective pixel values of the multiple reference pixels and the multiple reference pixels Calculate the credibility of the pixel value of the contaminated pixel, and correct the pixel value of the bad pixel of the image.

According to an embodiment of the present disclosure, the correction sub-module includes: a sixth determining unit, configured to determine a pixel to be rejected, and the pixel to be rejected is a contaminated pixel in the plurality of reference pixels. The reliability of the pixel value of the contaminated pixel is less than a preset Threshold pixels; and a seventh determining unit, configured to correct the pixel values of the dead pixels of the image based on the pixel values of the reference pixels other than the excluded pixels among the plurality of reference pixels.

According to an embodiment of the present disclosure, the seventh determining unit includes: an interpolation sub-unit, configured to interpolate the pixel values of the reference pixels other than the excluded pixels by using an image interpolation algorithm to generate pixels of the image dead pixels value.

According to an embodiment of the present disclosure, the acquisition module includes: a first acquisition sub-module for acquiring image information collected by the Four Bayer image sensor; and a first processing sub-module for demosaicing the image information. image.

According to an embodiment of the present disclosure, the acquisition module includes: a second acquisition sub-module for acquiring image information collected by an image sensor, the image information including pixel values of a plurality of pixels; and a second processing sub-module for The pixel values of the multiple pixels are combined to obtain the image to be processed.

According to an embodiment of the present disclosure, the update module further includes: a fifth determining sub-module for determining dynamic dead pixels, where the dynamic dead pixels are when the ambient light collected by the image sensor is the first reflected light , The difference between the pixel value and the pixel value of the pixel in the preset area is less than the preset value, and when the ambient light collected by the image sensor is the second reflected light, the pixel value is different from the pixel value of the pixel in the preset area The difference between the values is greater than the preset value of pixels, where the ambient light is the light reflected by the object collected by the image sensor; Point location information is added to the first dead pixel information, so that when the pixel value of the dynamic dead pixel is not within the specific range, the image dead pixel includes the dynamic dead pixel.

Another aspect of the present disclosure provides an electronic device including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are used by the one or more When executed by two processors, the one or more processors are caused to execute the foregoing method.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above-mentioned method when executed.

Another aspect of the present disclosure provides a computer program, which includes computer-executable instructions, which are used to implement the method as described above when executed.

Description of the drawings

1A to 1C schematically show application scenarios of an image processing method according to an embodiment of the present disclosure;

Fig. 2 schematically shows a flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 3 schematically shows a flowchart of a method for updating dead pixel information of an image sensor from first dead pixel information to second dead pixel information according to an embodiment of the present disclosure;

4A schematically shows a flowchart of an example method for determining the credibility of the pixel value of a contaminated pixel according to an embodiment of the present disclosure;

FIG. 4B schematically shows the correspondence between distance and credibility according to an embodiment of the present disclosure;

Fig. 5A schematically shows a flowchart of a method for processing an image to be processed according to an embodiment of the present disclosure;

FIG. 5B schematically shows a schematic diagram of an image with dead pixels and multiple reference pixels according to an embodiment of the present disclosure;

FIG. 5C schematically shows a schematic diagram of the result of the credibility of the pixel value determined based on the dead pixels of the image according to an embodiment of the present disclosure;

FIG. 6 schematically shows a flowchart of an exemplary method for updating dead pixel information of an image sensor from first dead pixel information to second dead pixel information according to another embodiment of the present disclosure;

Fig. 7A schematically shows a flowchart of an image processing method according to another embodiment of the present disclosure;

FIG. 7B schematically shows a flowchart of an exemplary method for updating first dead pixel information of an image sensor to second dead pixel information according to an embodiment of the present disclosure;

FIG. 7C schematically shows a flowchart of an example method for image dead pixel correction according to an embodiment of the present disclosure;

Fig. 8 schematically shows a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure; and

Fig. 9 schematically shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, it should be understood that these descriptions are only exemplary, and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concept of the present disclosure.

The terms used here are only for describing specific embodiments, and are not intended to limit the present disclosure. The terms "including", "including", etc. used herein indicate the existence of the described features, steps, operations and/or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted as having meanings consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

In the case of using an expression similar to "at least one of A, B and C, etc.", generally speaking, it should be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "having A, B and C" At least one of the "systems" shall include, but is not limited to, systems having A alone, B alone, C alone, A and B, A and C, B and C, and/or systems having A, B, C, etc. ). In the case of using an expression similar to "at least one of A, B or C, etc.", generally speaking, it should be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "having A, B or C" At least one of the "systems" shall include, but is not limited to, systems having A alone, B alone, C alone, A and B, A and C, B and C, and/or systems having A, B, C, etc. ).

The embodiment of the present disclosure provides an image processing method. The image processing method may include: obtaining a to-be-processed image, the to-be-processed image is generated according to image information collected by an image sensor, and determining the contaminated pixels in the to-be-processed image. Based on the information of the contaminated pixels, the dead pixel information of the image sensor is updated from the first dead pixel information to the second dead pixel information. Based on the second dead pixel information, the image to be processed is processed to correct the pixel value of the image dead pixel. Wherein, the contaminated pixel includes a pixel whose pixel value is affected by at least one bad image in the image to be processed in the process of generating the image to be processed according to the image information.

Figures 1A to 1C schematically show application scenarios of an image processing method according to an embodiment of the present disclosure. It should be noted that FIGS. 1A to 1C are only examples of application scenarios in which the embodiments of the present disclosure can be applied to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot Used in other devices, systems, environments or scenarios.

As shown in FIG. 1A, the application scenario includes a camera 100. The camera 100 may be configured with, for example, a four-Bayer image sensor, and the four-Bayer image sensor refers to an image sensor using a four-Bayer array.

Fig. 1B schematically shows a schematic diagram of a four Bayer array and a standard Bayer array.

As shown in FIG. 1B, the four-Bayer array can be a filter array with four adjacent basic pixel units of the same color, while the standard Bayer array can be a filter array with four adjacent basic pixel units sequentially G (Green, Green), The filter array of R (Red), B (Blue, blue), G, or the standard Bayer array can be a filter array with four adjacent basic pixel units being B, G, G, R in turn, or a standard The Bayer array can be a filter array with four adjacent basic pixel units sequentially G, B, R, G, or a standard Bayer array can be a filter array with four adjacent basic pixel units sequentially R, G, G, B Filter array.

According to an embodiment of the present disclosure, the four Bayer image sensor has two image output modes. The first is to directly perform demosic processing (Demosic) on the Four Bayer images collected by the Four Bayer array to obtain the output image. The second is to demosaic the four Bayer images collected by the four Bayer array to obtain a standard Bayer image based on the standard Bayer array, and then perform demosaic processing on the standard Bayer image to obtain an output image. Demosaicing processing refers to processing a four Bayer image based on a four Bayer array into a standard Bayer image based on a standard Bayer array, and demosaicing processing refers to processing a standard Bayer image into an RGB image.

However, for the second image output mode mentioned above, in the process of demosaicing the Four Bayer image to obtain the standard Bayer image, a dead pixel in the Four Bayer image will affect the pixel values of multiple pixels in the standard Bayer image. . Image dead pixels refer to pixels with inaccurate pixel values in the image collected by the image sensor. Image dead pixels may be caused by defects in the process of the image acquisition unit of the image sensor, or errors in the process of converting optical signals into electrical signals.

FIG. 1C schematically shows a schematic diagram of a standard Bayer image obtained by demosaicing a four-Bayer image.

As shown in FIG. 1C, for example, the four Bayer image includes image dead pixels 111. As shown above with reference to FIG. 1B, the actual output color of the dead pixels in the four Bayer image should be red. However, because the image acquisition unit corresponding to the image dead pixel 111 in the image sensor is damaged, the output color of the image dead pixel 111 is not red, for example, the output is black.

Referring to FIG. 1B, it can be seen that the

pixels

114, 115, and 116 in the four Bayer image are blue, green, and green, respectively, while in the standard Bayer image, the

pixels

114, 115, and 116 should be red. In order to convert the four-Bayer image into a standard Bayer image, the four-Bayer image needs to be demosaiced, so that the pixel 114, the pixel 115, and the pixel 116 are all converted into red pixel values after the demosaicing process.

According to the embodiments of the present disclosure, for example, demosaicing can be realized by an interpolation algorithm. The following is an example of determining the pixel value of the pixel 114 in the standard Bayer image to illustrate an implementation of the demosaicing process. For example, the red pixel adjacent to the pixel 114 (for example, the pixel in the ellipse and dotted line in FIG. 1C) can be used as the reference pixel of the pixel 114, and the pixel value of the reference pixel can be interpolated by an interpolation algorithm to obtain the pixel 114 in the standard The pixel value in the Bayer image. Similarly, the pixel values of other pixels in the standard Bayer image can also be determined by interpolation.

As shown in FIG. 1C, in the process of determining the pixel value of the pixel 114 in the standard Bayer image, since the reference pixel of the pixel 114 includes the image dead pixel 111, the pixel value of the pixel 114 in the standard Bayer image is affected by the image damage. The influence of the point 111, that is, the pixel 114 is a contaminated pixel. Similarly, the

pixels

115 and 116 are also contaminated pixels contaminated by the image dead pixels 111. Therefore, in the process of demosaicing the Four Bayer image to obtain the standard Bayer image, a dead pixel in the Four Bayer image will affect the pixel values of multiple pixels in the standard Bayer image.

The image processing method of the present disclosure can be used, for example, to correct the standard Bayer image converted from the four Bayer image in FIG. 1C, thereby correcting the defective pixels in the standard Bayer image, and ultimately improving the image quality of the RGB image output by the camera 100.

Fig. 2 schematically shows a flowchart of an image processing method according to an embodiment of the present disclosure.

As shown in FIG. 2, the method may include operations S201 to S204.

In operation S201, an image to be processed is obtained, and the image to be processed is generated according to the image information collected by the image sensor.

According to an embodiment of the present disclosure, the image sensor may be, for example, a four-Bayer image sensor. For example, the original image collected by the four Bayer image sensor may be subjected to the demosaic processing described above to obtain the image to be processed.

According to another embodiment of the present disclosure, obtaining a to-be-processed image may be a to-be-processed image obtained by combining pixel values of multiple pixels collected by an image sensor. For example, the binning operation may be performed on multiple pixels collected by the image sensor, and the binning operation may be adding the pixel values of adjacent pixels of the same color as the pixel value of the new pixel in the image after the binning operation. Specifically, for example, the image sensor has a binning mode. When the binning mode of the image sensor is turned on, the image sensor performs a binning operation on the collected image information to obtain the image to be processed, where the image information may include the pixel values of multiple pixels .

In operation S202, a contaminated pixel in the image to be processed is determined, and the contaminated pixel includes a pixel whose pixel value is affected by at least one defective image in the image to be processed during the process of generating the image to be processed according to the image information.

For example, in the scenario described above with reference to FIG. 1C, the image information may be a four-Bayer image collected by an image sensor. In the process of demosaicing the Four Bayer image to obtain the image to be processed, the

pixels

114, 115, 116, etc. in the image to be processed are affected by the defective image 111. The to-be-processed image obtained by demosaicing the four Bayer image includes at least one image dead pixel 111, a plurality of

contaminated pixels

114, 115, 116, etc., where, as shown in FIG. 1C, the image dead pixel 111 is in four The position in the Bayer image is consistent with the position of the image dead pixel 111 in the standard Bayer image.

In operation S203, based on the information of the contaminated pixel, the dead pixel information of the image sensor is updated from the first dead pixel information to the second dead pixel information.

According to an embodiment of the present disclosure, the first dead pixel information may be, for example, a dead pixel table provided by a manufacturer of the image sensor. The dead pixel table may include, for example, the identifier of the dead pixel and the coordinates of each dead pixel.

According to the embodiment of the present disclosure, for example, the second dead pixel information can be generated based on the dead pixel table and the contaminated pixels. The second dead pixel information may also be a table, for example. The table may include, for example, two types of information, the first type may be the identification of the dead pixel and the coordinates of each dead pixel, and the second type may be the identification of the contaminated pixel and the coordinates of the contaminated pixel and other information. Among them, the dead pixels and the contaminated pixels are respectively marked with two different types of identification, so as to distinguish whether the pixel is a dead pixel or a contaminated pixel.

In operation S204, the image to be processed is processed based on the second dead pixel information, so as to correct the pixel value of the image dead pixel.

For example, when a correction algorithm is used to correct the image defect in the image to be processed, the influence of the pixel value of the contaminated pixel on the pixel value of the image defect can be reduced according to the second defect information. Specifically, for example, the pixel value of the contaminated pixel may be multiplied by a weight less than 1 and greater than 0 and then applied to the correction algorithm.

According to the embodiments of the present disclosure, it is possible to determine the contaminated pixels in the image to be processed, and update the dead pixel information of the image sensor according to the contaminated pixels. Thus, in the process of correcting image defects in the image to be processed according to the information of the defect pixels, the influence of contaminated pixels on the image defect correction process can be reduced, the effect of image defect correction can be improved, and the image quality after correction can be improved. Image Quality.

Fig. 3 schematically shows a flowchart of a method for updating the dead pixel information of an image sensor from the first dead pixel information to the second dead pixel information according to an embodiment of the present disclosure.

As shown in FIG. 3, the method may include operation S213 to operation S243.

In operation S213, according to the first dead pixel information, position information of at least one image dead pixel in the image to be processed is determined.

The first dead pixel information may be, for example, a dead pixel table of the image sensor, and the location information may be, for example, the coordinates of the image dead pixel. For example, the dead pixel table of the image sensor can be queried to determine the coordinates of the image dead pixel.

In operation S223, the position information of the contaminated pixel in the image to be processed is determined according to the position information of the at least one dead pixel of the image.

According to the embodiment of the present disclosure, for example, the pollution parameter of the image sensor can be determined first. The pollution parameter indicates the range of pixels that can be affected by a dead pixel in an image during the process of generating the image to be processed based on the image information, and the location information of the contaminated pixel is determined based on the pollution parameter and the location information of at least one dead pixel in the image.

According to an embodiment of the present disclosure, the pollution parameter may be provided by the manufacturer of the image sensor. Specifically, the pollution parameter may indicate, for example, that the pixels within a circular area where the Euclidean distance to the defective image pixel is P (P>0) is a pixel range that can be affected by the defective image pixel. In the case of determining the pixel range that can be affected by a dead pixel of an image, it can be determined that the pixel whose Euclidean distance from the dead pixel of the image is less than P is the contaminated pixel contaminated by the dead pixel of the image.

In operation S233, the reliability of the pixel value of the contaminated pixel is determined.

According to the embodiments of the present disclosure, the credibility of the pixel value of the contaminated pixel represents the degree to which the contaminated pixel is affected by the image defect. The more the contaminated pixel is affected by the image defect, the lower the credibility of the pixel value of the contaminated pixel. Conversely, the less the contaminated pixel is affected by the image defect, the higher the credibility of the contaminated pixel's pixel value.

According to an embodiment of the present disclosure, the credibility of the pixel value of the contaminated pixel may be related to the distance between the contaminated pixel and the dead pixel of the image, for example. In this embodiment, the credibility of the pixel value of the contaminated pixel can be calculated based on the distance between the contaminated pixel and the dead pixel of the image.

According to another embodiment of the present disclosure, the reliability of the pixel value of the contaminated pixel may be pre-defined by the manufacturer of the image sensor, and written into the parameter information of the image sensor, for example. In this embodiment, for example, the parameter information of the image sensor can be directly read to obtain the credibility of the pixel value of the contaminated pixel.

In operation S243, the first dead pixel information is updated based on the location information and reliability of the contaminated pixel to generate second dead pixel information.

According to an embodiment of the present disclosure, the second dead pixel information may include, but is not limited to, the position of the dead pixel of the image, the position of the contaminated pixel, and the credibility of the pixel value of the contaminated pixel, for example. Wherein, the location of the image dead pixel can be determined according to the first dead pixel information.

According to the embodiments of the present disclosure, for example, the contaminated pixels and the credibility of the contaminated pixels can be added to the first dead pixel information, thereby generating the second dead pixel information.

According to the embodiments of the present disclosure, on the basis of determining the contaminated pixel, the credibility of the pixel value of the contaminated pixel can also be determined, so that the image defect can be corrected according to the credibility of the contaminated pixel, which further improves The accuracy of image dead pixel correction.

Fig. 4A schematically shows a flowchart of an example method for determining the credibility of the pixel value of a contaminated pixel according to an embodiment of the present disclosure.

As shown in FIG. 4A, the method may include operations S2331 to S2333.

In operation S2331, the corresponding relationship between the distance and the credibility is determined, and the distance is the Euclidean distance between the contaminated pixel and the defective image that affects the contaminated pixel.

According to an embodiment of the present disclosure, the corresponding relationship between the distance and the credibility can be represented by a confidence function, for example, and the confidence function can be a log function, an exponential function, or the like, for example.

Fig. 4B schematically shows the correspondence between distance and credibility according to an embodiment of the present disclosure.

As shown in Figure 4B, when the distance is less than P, the credibility is positively correlated with the distance. When the distance is greater than P, the credibility does not change, which is the same as the credibility when the distance is P. This distance is the Euclidean distance between the contaminated pixel and the image dead pixel that affects the contaminated pixel.

In operation S2332, based on the location information of the contaminated pixel and the location information of the defective image, the Euclidean distance between the contaminated pixel and the defective image that affects the contaminated pixel is determined.

For example, the Euclidean distance can be calculated based on the coordinates of the contaminated pixels and the coordinates of the dead pixels of the image.

In operation S2333, based on the correspondence and the Euclidean distance, the reliability of the pixel value of the contaminated pixel is determined.

For example, the Euclidean distance can be brought into the confidence function described above to determine the credibility of the pixel value of the contaminated pixel.

Fig. 5A schematically shows a flow chart of a method for processing an image to be processed based on second dead pixel information according to an embodiment of the present disclosure.

As shown in FIG. 5A, the method may include operations S214 to S244.

In operation S214, a plurality of reference pixels are determined based on the location information of the at least one image dead pixel, so as to correct the at least one image dead pixel by using respective pixel values of the plurality of reference pixels.

According to an embodiment of the present disclosure, the plurality of reference pixels may be, for example, a plurality of pixels whose distance to the dead pixel of the image is less than Q. The multiple reference pixels can be used to correct the dead pixels of the image.

FIG. 5B schematically shows a schematic diagram of a dead pixel and a plurality of reference pixels in an image according to an embodiment of the present disclosure.

As shown in FIG. 5B, the scene includes image dead pixels 510, and the reference pixels determined according to the location of the image dead pixels 510 for correcting the image dead pixels 510 may be, for example, European styles to the image dead pixels 510. Multiple pixels whose distance is less than Q.

In operation S224, it is determined whether there is a contaminated pixel among the plurality of reference pixels.

According to the embodiment of the present disclosure, for example, the position of the contaminated pixel can be obtained by querying the second bad pixel information described above, and the position of the contaminated pixel is compared with the position of the reference pixel. When the contaminated pixel is within the region of the reference pixel determined in operation S214 above, it is determined that the contaminated pixel exists in the plurality of reference pixels.

FIG. 5B is taken as an example to illustrate operation S224. For example, according to the second dead pixel information, it can be determined that the pixels in the area of the black solid line frame 520 in FIG. 5B are contaminated pixels contaminated by the image dead pixels 510. In operation S214, it is determined that the pixels in the area of the black dashed line frame 530 are reference pixels of the dead pixels 510 of the image. Obviously, the area of the black dashed line frame 530 includes the area inside the black solid line frame 520, and therefore, there are polluted pixels in the reference pixels corresponding to the dead pixels 510 of the image.

In operation S234, in a case where it is determined that there is a polluted pixel in the plurality of reference pixels, the second defective pixel information is queried to determine the credibility of the pixel value of the polluted pixel.

In operation S244, based on the respective pixel values of the plurality of reference pixels and the credibility of the pixel values of the contaminated pixels in the plurality of reference pixels, the pixel values of the bad pixels of the image are corrected.

According to the embodiments of the present disclosure, for example, it can be determined that the pixel to be rejected is a pixel whose pixel value of the contaminated pixel among the multiple reference pixels has a credibility less than a preset threshold; and the pixel to be rejected is based on the multiple reference pixels. The pixel value of other reference pixels is used to correct the pixel value of the image's bad pixels.

According to an embodiment of the present disclosure, the preset threshold may be determined by those skilled in the art based on experience and actual conditions of the image sensor.

FIG. 5C schematically shows a schematic diagram of the result of the credibility of the pixel value determined based on the image dead pixel 510 according to an embodiment of the present disclosure.

As shown in FIG. 5C, if the credibility of the pixel marked with the "X" mark is less than the preset threshold, the pixel marked with the "X" mark is the pixel to be excluded. The credibility of the pixels marked with "C" is greater than the preset threshold. The pixels marked with an “O” mark are pixels that are not contaminated by the dead pixels 510 of the image.

The pixel value of the remaining pixels (that is, the pixels marked with the "C" mark and the pixels marked with the "O" mark) after removing the pixels marked with the "X" mark in the reference pixels can be used to correct the image dead pixels 510.

According to the embodiments of the present disclosure, for example, the corresponding relationship between the range to which the credibility belongs and the pixel value weight (0<pixel value weight<1) can be determined, so that the weight of the pixel value can be determined according to the range to which the credibility belongs and then The pixel value of the contaminated pixel is multiplied by the weight and then applied to the correction algorithm. For example, in the scenario shown in FIG. B, the pixel value obtained by multiplying the pixel marked with "C" by 0.7 can be applied to the correction algorithm for correcting the image defect 510.

According to an embodiment of the present disclosure, based on the pixel values of the reference pixels other than the excluded pixels among the plurality of reference pixels, correcting the pixel values of the dead pixels of the image includes: using an image interpolation algorithm to refer to other reference pixels other than the excluded pixels The pixel value of the pixel is interpolated to generate the pixel value of the image dead pixel. Specifically, for example, the average value of the pixel values of the reference pixels other than the excluded pixels among the plurality of reference pixels may be used as the pixel value of the defective image of the image.

Fig. 6 schematically shows a flowchart of a method for updating dead pixel information of an image sensor from first dead pixel information to second dead pixel information according to another embodiment of the present disclosure.

As shown in FIG. 6, the method may further include operation 253 and operation S263 on the basis of operations S213 to S243 described in FIG. 3.

In operation S253, dynamic dead pixels are determined, where the dynamic dead pixels are when the ambient light collected by the image sensor is the first reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is less than A preset value, and when the ambient light collected by the image sensor is the second reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is greater than the pixel with the preset value, wherein the The ambient light is the light reflected by the object collected by the image sensor.

According to an embodiment of the present disclosure, the preset area may be a neighborhood of a pixel. When the ambient light collected by the image sensor is the first reflected light, if the difference between the pixel value of a certain pixel and the pixel value of the pixel in the neighborhood of the pixel is less than the preset value, and the environment collected by the image sensor When the light is the second reflected light, and the difference between the pixel value of the pixel and the pixel value of the pixel in the neighborhood of the pixel is greater than the preset value, the pixel is a dynamic dead pixel. In other words, a dynamic dead pixel is a pixel whose output pixel value is normal within a certain pixel value range, but the pixel whose output pixel value is abnormal if it is not within this pixel value range.

According to an embodiment of the present disclosure, the preset value may be a value determined by a person skilled in the art based on experience. The preset value may be any value between 0 and 255, for example.

According to the embodiments of the present disclosure, those skilled in the art can detect the image sensor to determine the dynamic dead pixels before using the image sensor, and write the detected dynamic dead pixel information into the software program of the image sensor, thereby When the image sensor executes the image processing method, the dynamic dead pixels can be read from the software program.

In operation S263, the dynamic dead pixels and the location information of the dynamic dead pixels are added to the first dead pixel information, so that when the pixel values of the dynamic dead pixels are not within a specific range, the image dead pixels include dynamic dead pixels.

According to an embodiment of the present disclosure, the location information of the dynamic dead pixel may be, for example, the coordinates of the dynamic dead pixel. For example, the coordinates of the dynamic dead pixels and the pixel range where the dynamic dead pixels behave abnormally can be added to the first dead pixel information.

For example, when the average value of the pixel values of the pixels in the neighborhood of the dynamic dead pixel is greater than the first value, it is determined that the output of the pixel value of the dynamic dead pixel is abnormal, and then at least one image dead pixel in operation S202 includes the dynamic dead pixel . Or, for example, when the average value of the pixel values of the pixels in the neighborhood of the dynamic dead pixel is less than the second value, it is determined that the output of the pixel value of the dynamic dead pixel is abnormal, and then at least one image dead pixel in operation S202 includes the dynamic dead pixel. Dead pixels.

Fig. 7A schematically shows a flowchart of an image processing method according to another embodiment of the present disclosure.

As shown in FIG. 7A, the image processing method may include operations S701 to S705.

In operation S701, an image to be processed is acquired.

In operation S702, it is determined whether the remote function is turned on by the image sensor. If the Remote function is enabled, perform operation S703, and if the remote function is not enabled, perform operation S704.

In operation S703, the first dead pixel information of the image sensor is updated to the second dead pixel information.

In operation S704, image dead pixel correction is performed.

In operation S705, an output image is generated based on the corrected image. For example, the image to be processed may be a standard Bayer image, and an RGB image is generated according to the corrected standard Bayer image.

FIG. 7B schematically shows a flowchart of a method for updating the first dead pixel information of the image sensor to the second dead pixel information in operation S703 according to an embodiment of the present disclosure.

As shown in FIG. 7B, the method may include operations S713 to S733.

In operation S713, the contaminated pixels are determined according to the dead pixels of the image.

In operation S723, the credibility of the contaminated pixel is determined.

In operation S733, the contaminated pixel and the credibility of the contaminated pixel are added to the first dead pixel information to generate second dead pixel information.

Fig. 7C schematically shows a flowchart of a method for correcting image dead pixels in operation S704 according to an embodiment of the present disclosure.

As shown in FIG. 7C, the method may include operations S714 to S754.

In operation S714, the pixel values of the reference pixels of the dead pixels of the image are acquired. The reference pixels of the dead pixels of the image may be, for example, multiple pixels whose distance to the dead pixels of the image is less than Q.

In operation S724, it is determined whether there is a contaminated pixel among the reference pixels. If there is a contaminated pixel, perform operation S734. If there is no contaminated pixel, perform operation S764.

In operation S734, the credibility of the contaminated pixel is compared with a preset threshold to determine whether the credibility of the contaminated pixel is less than the preset threshold. If the credibility of the contaminated pixel is less than the preset threshold, perform operation S754, and if the credibility of the contaminated pixel is greater than or equal to the preset threshold, perform operation S764.

In operation S744, the contaminated pixels whose credibility is less than the preset threshold are eliminated.

In operation S754, the pixel values of the multiple reference pixels are used for interpolation calculation to correct the pixel values of the dead pixels of the image. Among them, the multiple reference pixels do not include contaminated pixels whose credibility is less than a preset threshold.

FIG. 8 schematically shows a schematic diagram of an image processing apparatus 800 according to an embodiment of the present disclosure.

As shown in FIG. 8, the image processing apparatus 800 may include an acquisition module 810, a determination module 820, an update module 830, and a correction module 840.

The acquisition module 810, for example, executes the operation S201 described above with reference to FIG. 2 to acquire an image to be processed, which is generated according to image information collected by an image sensor.

The determining module 820, for example, executes the operation S202 described above with reference to FIG. 2 for determining the contaminated pixels in the image to be processed, and the contaminated pixels are included in the process of generating the image to be processed according to the image information. , A pixel whose pixel value is affected by at least one bad pixel in the image to be processed.

The update module 830, for example, executes the operation S203 described above with reference to FIG. 2 for updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information based on the information of the contaminated pixel.

The correction module 840, for example, executes the operation S204 described above with reference to FIG. 2 for processing the image to be processed based on the second dead pixel information, so as to correct the pixel value of the image dead pixel.

According to an embodiment of the present disclosure, the second determining sub-module includes: a first determining unit configured to determine a pollution parameter of the image sensor, the pollution parameter indicating that the image to be processed is generated according to the image information In the process, a pixel range that can be affected by a bad image pixel; and a second determining unit configured to determine the location information of the contaminated pixel based on the pollution parameter and the location information of the at least one image bad pixel.

According to an embodiment of the present disclosure, the credibility determination sub-module includes: a third determining unit, configured to determine the corresponding relationship between the distance and the credibility, and the distance is between the contaminated pixel and the amount that affects the contaminated pixel. The Euclidean distance between the dead pixels of the image; the fourth determining unit is used to determine the contaminated pixel to the value that affects the contaminated pixel based on the location information of the contaminated pixel and the location information of the image dead pixel. The Euclidean distance between the dead pixels of the image; and a fifth determining unit for determining the reliability of the pixel value of the contaminated pixel based on the correspondence and the Euclidean distance.

According to an embodiment of the present disclosure, the update module further includes: a fifth determining sub-module, configured to determine dynamic dead pixels, wherein, in the first condition, the pixel value of the dynamic dead pixel is the same as the pixel value of the pixel in the preset area. The difference between the values is less than the preset value, and under the second condition, the difference between the pixel value of the dynamic dead pixel and the pixel value of the pixel in the preset area is greater than the preset value; and the dead pixel adding sub Module for adding the dynamic dead pixels and the location information of the dynamic dead pixels to the first dead pixel information, so that when the pixel values of the dynamic dead pixels are not within the specific range, the The dead pixels of the image include the dynamic dead pixels.

According to the embodiments of the present disclosure, any number of the modules, sub-modules, units, and sub-units, or at least part of the functions of any number of them, may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), System-on-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way that integrates or encapsulates the circuit, or by software, hardware, and firmware. Any one of these implementations or an appropriate combination of any of them can be implemented. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be performed.

For example, any number of the acquisition module 810, the determination module 820, the update module 830, and the correction module 840 can be combined into one module for implementation, or any one of them can be split into multiple modules. Or, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the determination module 820, the update module 830, and the correction module 840 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA) and a programmable logic array. (PLA), system on chip, system on substrate, system on package, application-specific integrated circuit (ASIC), or any other reasonable way to integrate or package the circuit, or other hardware or firmware, or by software, It can be implemented in any one of the three implementation modes of hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 810, the determination module 820, the update module 830, and the correction module 840 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

Fig. 9 schematically shows a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device shown in FIG. 9 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 9, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can be loaded into a random access memory (RAM) 903 according to a program stored in a read only memory (ROM) 902 or from a storage part 908 The program executes various appropriate actions and processing. The processor 901 may include, for example, a general-purpose microprocessor (for example, a CPU), an instruction set processor and/or a related chipset and/or a special purpose microprocessor (for example, an application specific integrated circuit (ASIC)), and so on. The processor 901 may also include on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for executing different actions of a method flow according to an embodiment of the present disclosure.

In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. The processor 901 executes various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 902 and/or RAM 903. It should be noted that the program may also be stored in one or more memories other than ROM 902 and RAM 903. The processor 901 may also execute various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 900 may further include an input/output (I/O) interface 905, and the input/output (I/O) interface 905 is also connected to the bus 904. The electronic device 900 may also include one or more of the following components connected to the I/O interface 905: an input part 906 including a keyboard, a mouse, etc.; including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and An output section 907 of a speaker and the like; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, and the like. The communication section 909 performs communication processing via a network such as the Internet. The drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that the computer program read therefrom is installed into the storage portion 908 as needed.

According to the embodiment of the present disclosure, the method flow according to the embodiment of the present disclosure may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 909, and/or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to the embodiments of the present disclosure, the systems, devices, devices, modules, units, etc. described above may be implemented by computer program modules.

The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be included in the device/device/system described in the above embodiment; or it may exist alone without being assembled into the device/ In the device/system. The aforementioned computer-readable storage medium carries one or more programs, and when the aforementioned one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but not limited to: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM) , Erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include one or more memories other than ROM 902 and/or RAM 903 and/or ROM 902 and RAM 903 described above.

The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more for realizing the specified logic function. Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by It is realized by a combination of dedicated hardware and computer instructions.

Those skilled in the art can understand that the various embodiments of the present disclosure and/or the features described in the claims can be combined or and/or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the various embodiments of the present disclosure and/or the features described in the claims can be combined and/or combined in various ways. All these combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only, and are not intended to limit the scope of the present disclosure. Although the respective embodiments are described above separately, this does not mean that the measures in the respective embodiments cannot be advantageously used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should fall within the scope of the present disclosure.

Claims

An image processing method, including:

Obtaining a to-be-processed image, the to-be-processed image being generated based on image information collected by an image sensor;

Determining a contaminated pixel in the image to be processed, where the contaminated pixel includes a pixel whose pixel value is affected by at least one bad image in the image to be processed in the process of generating the image to be processed according to the image information;

Based on the information of the contaminated pixels, updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information; and

Based on the second bad pixel information, processing the image to be processed so as to correct the pixel value of the image bad pixel.
The method according to claim 1, wherein said updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information comprises:

Determine the location information of at least one image dead pixel in the image to be processed according to the first dead pixel information;

Determine the location information of the contaminated pixels in the image to be processed according to the location information of the at least one image dead pixel;

Determining the credibility of the pixel value of the contaminated pixel; and

The first dead pixel information is updated based on the location information of the contaminated pixel and the credibility to generate the second dead pixel information.
The method according to claim 2, wherein said determining the location information of the contaminated pixels in the image to be processed comprises:

Determining a pollution parameter of the image sensor, the pollution parameter indicating a pixel range that can be affected by a bad image pixel in the process of generating the image to be processed according to the image information; and

Based on the pollution parameter and the location information of the at least one image dead pixel, the location information of the polluted pixel is determined.
The method according to claim 2, wherein said determining the credibility of the pixel value of the contaminated pixel comprises:

Determine the correspondence between the distance and the credibility, where the distance is the Euclidean distance between the contaminated pixel and the dead pixel of the image that affects the contaminated pixel;

Determine the Euclidean distance from the contaminated pixel to the image defect that affects the contaminated pixel based on the location information of the contaminated pixel and the location information of the image dead pixel; and

Based on the correspondence and the Euclidean distance, the reliability of the pixel value of the contaminated pixel is determined.
The method according to claim 2, wherein the processing the image to be processed based on the second dead pixel information so as to correct the pixel value of the image dead pixel comprises:

Determining a plurality of reference pixels based on the location information of the at least one image dead pixel, so as to correct the at least one image dead pixel by using respective pixel values of the plurality of reference pixels;

Determining whether there are polluted pixels in the plurality of reference pixels;

In the case where it is determined that there is a polluted pixel in the plurality of reference pixels, query the second bad pixel information to determine the credibility of the pixel value of the polluted pixel; and

Based on the respective pixel values of the plurality of reference pixels and the reliability of the pixel values of the contaminated pixels in the plurality of reference pixels, the pixel values of the bad pixels of the image are corrected.
5. The method according to claim 5, wherein, based on the respective pixel values of the plurality of reference pixels and the credibility of the pixel values of the contaminated pixels in the plurality of reference pixels, correcting the defect of the image Pixel values include:

Determining a pixel to be eliminated, where the pixel to be eliminated is a pixel in which the credibility of the pixel value of the contaminated pixel among the plurality of reference pixels is less than a preset threshold; and

Based on the pixel values of the reference pixels other than the eliminated pixels among the plurality of reference pixels, the pixel values of the dead pixels of the image are corrected.
7. The method according to claim 6, wherein the correcting the pixel value of the dead pixel of the image based on the pixel value of the reference pixels other than the excluded pixel among the plurality of reference pixels comprises:

Interpolating the pixel values of the reference pixels other than the excluded pixels through an image interpolation algorithm to generate the pixel values of the image dead pixels.
The method according to claim 1, wherein said obtaining the image to be processed comprises:

Obtain the image information collected by the Four Bayer image sensor; and

An image obtained by demosaicing the image information.
The method according to claim 1, wherein said obtaining the image to be processed comprises:

Obtaining image information collected by the image sensor, the image information including pixel values of a plurality of pixels; and

The pixel values of the multiple pixels are combined to obtain the image to be processed.
The method according to claim 2, wherein said updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information further comprises:

Determining dynamic dead pixels, wherein the dynamic dead pixels are that when the ambient light collected by the image sensor is the first reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is less than the preset value , And when the ambient light collected by the image sensor is the second reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is greater than the preset value of the pixel, wherein the ambient light is all The light reflected by the object collected by the image sensor; and

The dynamic dead pixel and the location information of the dynamic dead pixel are added to the first dead pixel information, so that when the pixel value of the dynamic dead pixel is not within the specific range, the image dead pixel Including the dynamic dead pixels.
An image processing device, including:

An acquisition module for acquiring an image to be processed, the image to be processed is generated based on image information collected by an image sensor;

The determining module is used to determine the contaminated pixel in the image to be processed, and the contaminated pixel is included in the process of generating the image to be processed according to the image information, and the pixel value is damaged by at least one image in the image to be processed. Pixels affected by a point;

An update module for updating the dead pixel information of the image sensor from the first dead pixel information to the second dead pixel information based on the information of the contaminated pixel; and

The correction module is configured to process the image to be processed based on the second bad pixel information, so as to correct the pixel value of the image bad pixel.
The device according to claim 11, wherein the update module comprises:

The first determining submodule is configured to determine the location information of at least one image of the to-be-processed image according to the first dead pixel information;

The second determining sub-module is configured to determine the location information of the contaminated pixels in the image to be processed according to the location information of the at least one dead pixel of the image;

The credibility determination sub-module is used to determine the credibility of the pixel value of the contaminated pixel; and

A generating submodule is used to update the first dead pixel information based on the location information of the contaminated pixel and the credibility to generate the second dead pixel information.
The device according to claim 12, wherein the second determining submodule comprises:

The first determining unit is configured to determine a pollution parameter of the image sensor, where the pollution parameter indicates a pixel range that can be affected by a bad image pixel in the process of generating the image to be processed according to the image information; as well as

The second determining unit is configured to determine the position information of the contaminated pixel based on the pollution parameter and the position information of the at least one image dead pixel.
The apparatus according to claim 12, wherein said credibility determination sub-module comprises:

The third determining unit is configured to determine the corresponding relationship between the distance and the credibility, where the distance is the Euclidean distance between the contaminated pixel and the defective image that affects the contaminated pixel;

A fourth determining unit, configured to determine the Euclidean distance between the contaminated pixel and the image defect that affects the contaminated pixel based on the location information of the contaminated pixel and the location information of the image dead pixel; as well as

The fifth determining unit is configured to determine the credibility of the pixel value of the contaminated pixel based on the correspondence and the Euclidean distance.
The device according to claim 12, wherein the correction module comprises:

The third determining sub-module is configured to determine a plurality of reference pixels based on the location information of the at least one image dead pixel, so as to correct the at least one image dead pixel by using respective pixel values of the plurality of reference pixels;

A fourth determining sub-module, configured to determine whether there is a polluted pixel among the multiple reference pixels;

The query sub-module is configured to query the second dead pixel information to determine the credibility of the pixel value of the polluted pixel when it is determined that there is a polluted pixel in the multiple reference pixels; and

The correction sub-module is configured to correct the pixel value of the bad image of the image based on the respective pixel values of the multiple reference pixels and the credibility of the pixel values of the contaminated pixels in the multiple reference pixels.
The device according to claim 15, wherein the correction sub-module comprises:

A sixth determining unit, configured to determine a pixel to be removed, where the pixel to be removed is a pixel in which the credibility of the pixel value of the contaminated pixel among the plurality of reference pixels is less than a preset threshold; and

The seventh determining unit is configured to correct the pixel value of the dead pixel of the image based on the pixel values of the reference pixels other than the eliminated pixels among the plurality of reference pixels.
The apparatus according to claim 16, wherein the seventh determining unit comprises:

The interpolation subunit is used for interpolating the pixel values of the reference pixels other than the excluded pixels through an image interpolation algorithm to generate the pixel values of the image dead pixels.
The apparatus according to claim 11, wherein the acquisition module comprises:

The first obtaining sub-module is used to obtain the image information collected by the four Bayer image sensor; and

The first processing sub-module is used for the image obtained by demosaicing the image information.
The apparatus according to claim 11, wherein the acquisition module comprises:

The second obtaining sub-module is used to obtain image information collected by the image sensor, the image information including pixel values of a plurality of pixels; and

The second processing sub-module is configured to combine the pixel values of the multiple pixels to obtain the image to be processed.
The device according to claim 12, wherein the update module further comprises:

The fifth determining sub-module is configured to determine dynamic dead pixels, where the dynamic dead pixels are the pixel value and the pixel value of the pixel in the preset area when the ambient light collected by the image sensor is the first reflected light The difference between is smaller than the preset value, and in the case where the ambient light collected by the image sensor is the second reflected light, the difference between the pixel value and the pixel value of the pixel in the preset area is greater than the preset value, Wherein, the ambient light is light reflected by an object collected by the image sensor; and

The dead pixel adding sub-module is used to add the dynamic dead pixel and the location information of the dynamic dead pixel to the first dead pixel information, so that the pixel value of the dynamic dead pixel is not within the specific range In this case, the dead pixels of the image include the dynamic dead pixels.
An electronic device including:

One or more processors;

Storage device for storing one or more programs,

Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1-10.
A computer-readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method according to any one of claims 1-10.