CN111340714A - Moire pattern processing method and device and electronic equipment - Google Patents
Moire pattern processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN111340714A CN111340714A CN201910807521.0A CN201910807521A CN111340714A CN 111340714 A CN111340714 A CN 111340714A CN 201910807521 A CN201910807521 A CN 201910807521A CN 111340714 A CN111340714 A CN 111340714A
- Authority
- CN
- China
- Prior art keywords
- image
- moire
- interpolation
- similarity
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000001914 filtration Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
The embodiment of the invention provides a Moire pattern processing method and device and electronic equipment. The method comprises the following steps: counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, so as to obtain a statistical result; determining a re-interpolation direction corresponding to the moire image area according to the statistical result; and in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain an image processed by the moire image area. The influence of moire on the interpolation direction can be suppressed by means of re-interpolation of the original Bayer image region corresponding to the moire image region, and then moire of the moire image region can be suppressed. And the filter processing is not only carried out on the moire image area, so that the influence of moire on the image quality can be effectively reduced and restrained.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a moire processing method and apparatus, and an electronic device.
Background
When the sampling frequency of the image sensor of the camera is lower than the spatial frequency of the photographed scene, moire may be generated in the photographed image due to stacking of high-frequency image information in a low-frequency pixel space, affecting the image quality.
In the related art, a gaussian filtering process may be performed on an image region where moire exists to obtain a moire-removed image of the image region. But gaussian filtering will filter some of the high frequency image information in the image area, resulting in a loss of image quality.
Disclosure of Invention
The embodiment of the invention aims to provide a moire processing method, a moire processing device and an electronic device, so as to reduce and inhibit the influence of moire on the picture quality. The specific technical scheme is as follows:
in a first aspect of embodiments of the present invention, there is provided a moire processing method, including:
counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, so as to obtain a statistical result;
determining a re-interpolation direction corresponding to the moire image area according to the statistical result;
and in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain an image processed by the moire image area.
With reference to the first aspect, in a first possible implementation manner, the determining, according to the statistical result, a re-interpolation direction corresponding to the moire image region includes:
and if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction as the designated direction is greater than a preset number threshold, determining the re-interpolation direction as the direction orthogonal to the designated direction, wherein the designated direction is the horizontal direction or the vertical direction.
With reference to the first aspect, in a second possible implementation manner, the determining, according to the statistical result, a re-interpolation direction corresponding to the moire image region includes:
if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction being the designated direction is not more than a preset number threshold, respectively interpolating the original Bayer image region in the horizontal direction and the vertical direction to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner, when counting, for each moire image region in the image to be processed, an interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on an original Bayer image interpolation, the obtaining a statistical result includes:
until an image processed in each moire image area in the image to be processed is obtained, counting the interpolation direction of original Bayer image pixel points corresponding to each moire pixel point in the moire image area when the image to be processed is obtained based on the original Bayer image interpolation aiming at each moire image area in the image to be processed, and obtaining a statistical result;
the method further comprises the step of uniformly dividing the moire image area into a plurality of sub-areas if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the sub-areas as a new moire image area of the image to be processed.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the dividing the moire image region into a plurality of sub-regions, and taking the plurality of sub-regions as a new moire image region includes:
if the moire image area is not a sub-area obtained by dividing other moire image areas for a preset number of times, dividing the moire image area into a plurality of sub-areas, and taking the plurality of sub-areas as a new image area of the image to be processed;
the method further comprises the following steps:
and if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times, carrying out filtering processing on the moire image area to obtain a processed image of the moire image area.
In a second aspect of embodiments of the present invention, there is provided a moire processing apparatus, comprising:
the interpolation direction counting module is used for counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the interpolation of the original Bayer image aiming at each moire image region in the image to be processed, so as to obtain a counting result;
the interpolation direction determining module is used for determining a re-interpolation direction corresponding to the moire image area according to the statistical result;
and the re-interpolation module is used for interpolating the original Bayer image area corresponding to the moire image area in the re-interpolation direction to obtain an image processed by the moire image area.
With reference to the second aspect, in a first possible implementation manner, the interpolation direction determining module is specifically configured to determine the re-interpolation direction as a direction orthogonal to the specified direction if the statistical result indicates that the number of pixels of the original Bayer image whose interpolation direction is the specified direction is greater than a preset number threshold, where the specified direction is a horizontal direction or a vertical direction.
With reference to the second aspect, in a second possible implementation manner, the interpolation direction determining module is specifically configured to, if the statistical result indicates that the number of pixels of the original Bayer image whose interpolation direction is the designated direction is not greater than a preset number threshold, interpolate the original Bayer image region in a horizontal direction and a vertical direction respectively to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner, the interpolation direction statistics module is specifically configured to, until an image after processing of each moire image region in the image to be processed is obtained, count, for each moire image region in the image to be processed, an interpolation direction of a pixel point of the original Bayer image corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on interpolation of the original Bayer image, and obtain a statistical result;
the device further comprises a region dividing module, which is used for uniformly dividing the moire image region into a plurality of sub-regions if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the plurality of sub-regions as new moire image regions of the image to be processed.
With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the region dividing module is specifically configured to, if the moire image region is not a sub-region obtained by dividing other moire image regions by a preset number of times, divide the moire image region into a plurality of sub-regions, and use the plurality of sub-regions as a new image region of the image to be processed;
the device further comprises a filtering module, wherein the filtering module is used for performing filtering processing on the moire image area to obtain a processed image of the moire image area if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times.
In a third aspect of embodiments of the present invention, there is provided an electronic device, including:
a memory for storing a computer program;
a processor for implementing the moire processing method according to any one of the above first aspects when executing a program stored in a memory.
In a fourth aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the moire processing method according to any one of the above first aspects.
According to the moire processing method, the moire processing device and the electronic equipment, the influence of moire on the interpolation direction can be inhibited (even eliminated) in a mode of re-interpolating the original Bayer image area corresponding to the moire image area, and then moire of the moire image area is inhibited (even eliminated). The filtering processing is not only carried out on the moire image area, so that the influence of moire on the image quality can be effectively reduced. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a Moire processing method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of another Moire processing method according to an embodiment of the present invention;
FIG. 3a is a schematic diagram of an image to be processed according to an embodiment of the present invention;
FIG. 3b is a schematic diagram of an image to be processed including moire image regions according to an embodiment of the present invention;
FIG. 3c is a schematic diagram of an image to be processed according to an embodiment of the present invention, in which a moire image region is divided into a plurality of sub-regions;
FIG. 4 is a schematic structural diagram of a moire processing device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a moire suppression method provided in an embodiment of the present invention, which may include:
s101, counting the interpolation direction of original Bayer image pixel points corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation aiming at each moire image region in the image to be processed, and obtaining a counting result.
The moire image area is an image area with moire in the image to be processed. It can be understood that, when an image to be processed is obtained based on interpolation of an original Bayer image, the interpolation direction of pixel points of the original Bayer image affects the interpolation result, that is, the interpolation direction of one or more pixel points of the original Bayer image in the original Bayer image area is changed under the condition that the pixel value of each pixel point of the original Bayer image in the original Bayer image area is not changed, and the obtained interpolation results may be different.
The interpolation direction may include different directions according to different application scenarios, and for example, in an embodiment of the present invention, the interpolation direction may include a horizontal direction, a vertical direction, and a non-direction. In other optional embodiments, the interpolation direction may also include a horizontal direction, a vertical direction, a first oblique direction, a second oblique direction, and a non-direction, where the first oblique direction may be an angular bisector direction of a positive direction of the horizontal direction and a positive direction of the vertical direction, and the second oblique direction may be an angular bisector direction of a negative direction of the horizontal direction and a positive direction of the vertical direction, which is not limited in this embodiment.
The original Bayer image includes red pixel points, green pixel points, and blue pixel points, where the red pixel points only include red components, the green pixel points only include green components, and the blue pixel points only include blue components, so that interpolation needs to be performed on the original Bayer image to obtain color components missing by each pixel point (for example, the red pixel points lack the blue components and the green components). According to different selected interpolation methods, the determination mode of the interpolation direction of a pixel point of an original Bayer image can be different, and exemplarily, the horizontal direction gradient and the vertical direction gradient of the pixel point of the original Bayer image can be respectively calculated according to the following formula:
DirH＝|P(i，j-1)-P(i，j+1)|
DirV＝|P(i-1，j)-P(i+1，j)|
wherein i is the row coordinate of the pixel point of the original Bayer image, and j is the column coordinate of the pixel point of the original Bayer image. P (i, j-1) is the pixel value of the pixel point (i, j-1), DirHFor a gradient in the horizontal direction, DirVIs a vertical gradient. In the present embodiment, the horizontal direction may refer to a scanning direction of an image sensor of the camera, and the vertical direction is a direction orthogonal to the horizontal direction.
If so: dirH-DirV≥thLWhich isMiddle thLPresetting a difference threshold value, wherein the horizontal direction gradient is far larger than the vertical direction gradient, namely the texture trend at the pixel point of the original Bayer image is in the vertical direction with high probability, so that the interpolation direction of the pixel point of the original Bayer image can be determined to be in the vertical direction;
if so: dirV-DirH≥thLThe gradient in the vertical direction is far greater than that in the horizontal direction, that is, the trend of the texture at the pixel point of the original Bayer image is in the horizontal direction with a high probability, so that the interpolation direction of the pixel point of the original Bayer image can be determined to be in the vertical direction;
if the above two conditions are not satisfied, namely the conditions are satisfied: | DirH-DirV|<thLThe vertical gradient and the horizontal gradient are considered to be similar, that is, the probability that the texture trend of the pixel point of the original Bayer image is in the horizontal direction is equivalent to the probability that the texture trend is in the vertical direction, and the interpolation direction of the pixel point of the original Bayer image can be determined to be non-directional.
And S102, determining a re-interpolation direction corresponding to the moire image area according to the statistical result.
It can be understood that the statistical result may represent the number of original Bayer image pixel points in each interpolation direction in each original Bayer image region, so that the trend of the interpolation direction of the original Bayer image pixel points in the original Bayer image region may be reflected. For example, if the number of original Bayer image pixels having an interpolation direction in the horizontal direction is far greater than that of original Bayer image pixels having an interpolation direction in other directions in the statistical result, it may be considered that the interpolation direction of most of the original Bayer image pixels in the original Bayer image region is in the horizontal direction, and because the interpolation direction of the most of the original Bayer image pixels having the interpolation direction in the horizontal direction is wrong, a moire pattern exists in the moire pattern image region obtained by interpolation, so that the re-interpolation direction may be determined as a direction orthogonal to the horizontal direction, that is, the vertical direction. How to determine the re-interpolation direction according to the statistical result will be described in detail in the following embodiments, and will not be described herein again.
And S103, in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain an image processed by the moire image area.
As described in the foregoing analysis, the statistical result may reflect whether the reason for the existence of moire in the moire image region is that the interpolation direction of the pixel points of the original Bayer image in the original Bayer image region is incorrect. Therefore, according to the re-interpolation direction obtained by the statistical result, more accurate interpolation can be carried out on the original Bayer image area.
By adopting the embodiment, the influence of the moire on the interpolation direction can be inhibited (even eliminated) in a mode of re-interpolating the original Bayer image area corresponding to the moire image area, and then the moire of the moire image area is inhibited (even eliminated). Since the moire image region is not subjected to the filtering process, the image quality is not degraded by the suppression of moire.
Referring to fig. 2, fig. 2 is a schematic flow chart of another moire processing method according to an embodiment of the present invention, which may include:
s201, counting the interpolation direction of original Bayer image pixel points corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation aiming at each moire image region in the image to be processed, and obtaining a counting result.
The step is the same as S101, and reference may be made to the foregoing description about S101, which is not described herein again.
S202, if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction as the designated direction is larger than a preset number threshold, determining the re-interpolation direction as the direction orthogonal to the designated direction.
The designated direction is a horizontal direction or a vertical direction, and the preset number threshold value can be set according to actual requirements. For convenience of description, it is assumed that a statistical result represents that the number of pixel points of an original Bayer image, the interpolation direction of which is the horizontal direction, in an original Bayer image region obtained by statistics is T1Original Bayer image with vertical interpolation directionThe number of prime points is T2The number of original Bayer image pixel points with the interpolation direction being non-directional is T3For the case where the interpolation direction includes only the horizontal direction, the vertical direction, and no direction, the following relationship exists:
T＝T1+T2+T3
and T is the total number of moire pixel points in the moire image area. The preset number threshold may be
Wherein th is a preset value, since T is T ═ T1+T2+T3And therefore, in this case,
is equivalent to T1-(T1+T3)>th, i.e., in this case, it can be considered that if:
T1-(T2+T3)>th
determining that the number of pixel points of the original Bayer image in the horizontal direction is greater than a preset number threshold value, and determining the re-interpolation direction as a direction orthogonal to the specified direction, namely determining the re-interpolation direction as the vertical direction.
Similarly, it can be considered that if:
T2-(T1+T3)>th
determining that the number of pixel points of the original Bayer image in the vertical direction is greater than a preset number threshold value, and determining the original interpolation direction as a direction orthogonal to the specified direction, namely determining the re-interpolation direction as the horizontal direction.
S203, if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction being the designated direction is not more than a preset number threshold, respectively interpolating the original Bayer image region corresponding to the moire image region in the original Bayer image in the horizontal direction and the vertical direction to obtain a horizontal direction interpolation result and a vertical direction interpolation result.
As with the foregoing analysis, in an alternative embodimentIn the examples, it can be said that T is satisfied3-(T1+T2)>th, the number of pixels of the original Bayer image with the interpolation direction being the designated direction is not more than a preset number threshold. It is understood that in the case where this condition is satisfied, T can be regarded as1、T2Close, difficult to simply follow T1、T2And the trend of the interpolation direction of each original Bayer image pixel point in the original Bayer image area is shown. The re-interpolation direction can be further determined based on other criteria.
And S204, calculating the similarity between the horizontal direction interpolation result and the image of the moire area as a first similarity.
In an alternative embodiment, the first similarity may be calculated based on the horizontal direction interpolation result and components of a red channel, a green channel, and a blue channel of the image of the moire area, and in other alternative embodiments, other image similarity calculation methods may also be used to calculate the first similarity, which is not limited in this embodiment.
For example, assuming that the component of the red channel of the horizontal direction interpolation result is Ar, the component of the green channel is Ag, the component of the blue channel is Ab, the component of the red channel of the image of the moire region is Ir, the component of the green channel is Ig, and the component of the blue channel is Ib, the first similarity may be calculated according to the following formula:
wherein AI is the first similarity.
And S205, calculating the similarity between the vertical direction interpolation result and the image of the Moire pattern region as a second similarity.
In order to make the first similarity and the second similarity have comparability, the first similarity and the second similarity should be calculated by the same calculation method, for example, the second similarity may be calculated according to the following formula:
wherein, BI is the second similarity, Br is the component of the red channel of the vertical direction interpolation result, Bg is the component of the green channel of the vertical direction interpolation result, and Bb is the component of the blue channel of the vertical direction interpolation result. In other alternative embodiments, S205 may also be executed before S204, or may also be executed in parallel with S205 or executed alternately, which is not limited in this embodiment.
S206, determining whether the absolute value of the interpolation of the first similarity and the second similarity is greater than a preset judgment threshold, if the absolute value of the interpolation of the first similarity and the second similarity is greater than the preset judgment threshold, executing S207, and if the absolute value of the interpolation of the first similarity and the second similarity is not greater than the preset judgment threshold, executing S209.
S207, the re-interpolation direction is determined as the interpolation direction corresponding to the greater similarity of the first similarity and the second similarity.
The interpolation direction corresponding to the first similarity is the horizontal direction, and the interpolation direction corresponding to the second similarity is the vertical direction. And judging whether the absolute value of the difference value between the first similarity and the second similarity is greater than a preset judgment threshold, and not calculating the absolute value of the difference value between the first similarity and the second similarity.
For example, assuming that the preset determination threshold is Distance, when the condition is satisfied: when AI-BI > Distance, the absolute value of the difference between the first similarity and the second similarity may be considered to be greater than a preset determination threshold, and the first similarity is a greater similarity, so that the re-interpolation direction may be determined as a horizontal direction:
when the condition BI-AI > Distance is satisfied, it may be considered that an absolute value of a difference between the first similarity and the second similarity is greater than a preset determination threshold, and the second similarity is a greater similarity, so that the re-interpolation direction may be determined as the vertical direction.
And S208, in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain the image processed by the moire image area.
It can be understood that if the absolute value of the interpolation of the first similarity and the second similarity is not greater than the preset judgment threshold, it may be considered that the moire has a smaller influence on the interpolation direction of the original Bayer image region, whereas if the absolute value of the interpolation of the first similarity and the second similarity is greater than the preset judgment threshold, it may be considered that the moire has a larger influence on the interpolation direction of the original Bayer image region, and therefore, the moire of the moire image region may be further suppressed by changing the interpolation direction to re-interpolate.
And S209, uniformly dividing the moire image area into a plurality of sub-areas, and taking the plurality of sub-areas as new image areas, or filtering the moire image area to obtain a processed image of the moire image area.
As in the foregoing analysis, if the absolute value of the interpolation of the first similarity and the second similarity is not greater than the preset judgment threshold, it may be considered that the moire has a small influence on the interpolation direction of the original Bayer image region, in this case, the moire image region may be divided into a plurality of sub-regions as new image regions, and the processing as described in S201 to S209 may be performed on the moire image region in the new image region until each of the processed images of each moire image region is obtained.
It is also considered that in this case, the moire image region is not generated due to the inaccuracy of the interpolation direction of the original Bayer image region, and therefore, the moire image region may be processed by a filtering process (such as gaussian filtering) to suppress moire of the moire image region.
The two processing modes can be selected according to actual requirements. In an optional embodiment, if the moire image region is obtained by splitting a historical moire image region, the moire image region is processed through a filtering process, and if the moire image region is obtained by directly dividing an image to be processed, the moire image region is uniformly split into a plurality of new moire image regions.
By selecting the embodiment, the main reason for generating the moire in the moire image area can be judged based on the original interpolation direction of the moire image area in the image to be processed, and a more appropriate method is selected to inhibit the moire in the moire image area.
In the following, a moire processing method for an image provided by an embodiment of the present invention will be described with reference to a specific application scenario, assuming that an image to be processed is shown in fig. 3a, where a shaded portion is a moire in the image to be processed, the image to be processed can be uniformly divided into a plurality of sub-regions, assuming that the image to be processed is divided into 16 sub-regions in total of 4 × 4, as shown in fig. 3b, for convenience of discussion, the 16 sub-regions are numbered from left to right and from top to bottom, exemplarily, a sub-region in a first row and a first column is denoted as sub-region 1, a sub-region in a first row and a second column is denoted as sub-region 2, and a sub-region in a second row and a first column is denoted as sub-. Fig. 3b shows that sub-areas 1, 4, 7, 10, 16 are moire image areas.
According to the moire processing method provided by the embodiment of the invention, the moire image area is processed to obtain the processed image of the moire image area, and the image with moire suppressed (even eliminated) can be obtained without processing the non-moire image area. As described in the foregoing S209, in some application scenarios, if the absolute value of the interpolation of the first similarity and the second similarity is not greater than the preset determination threshold, the moire image region may be divided into a plurality of sub-regions, as shown in fig. 3c, and these sub-regions are used as new image regions, and the moire processing method provided in the embodiment of the present invention is used to process these sub-regions to obtain processed images, and if the moire image regions in these new image regions still satisfy that the absolute value of the interpolation of the first similarity and the second similarity is not greater than the preset determination threshold, these new moire image regions may be subjected to filtering processing (in other optional embodiments, the separation may also be continued, and the principle is the same, so that no more discussion is made).
Referring to fig. 4, fig. 4 is a schematic structural diagram of a moire processing apparatus according to an embodiment of the present invention, which may include:
an interpolation direction statistical module 401, configured to count, for each moire image region in the image to be processed, an interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, so as to obtain a statistical result;
an interpolation direction determining module 402, configured to determine, according to the statistical result, a re-interpolation direction corresponding to the moire image region;
and a re-interpolation module 403, configured to interpolate an original Bayer image region corresponding to the moire image region in the re-interpolation direction, to obtain an image after the moire image region is processed.
In a possible embodiment, the interpolation direction determining module 402 is specifically configured to determine the re-interpolation direction as a direction orthogonal to the specified direction if the statistical result indicates that the number of original Bayer image pixel points whose interpolation direction is the specified direction is greater than a preset number threshold, where the specified direction is a horizontal direction or a vertical direction.
In a possible embodiment, the interpolation direction determining module 402 is specifically configured to, if the statistical result indicates that the number of pixels of the original Bayer image whose interpolation direction is the designated direction is not greater than a preset number threshold, interpolate the original Bayer image region in the horizontal direction and the vertical direction, respectively, to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
In a possible embodiment, the interpolation direction statistics module 401 is specifically configured to, until an image processed in each moire image region in the image to be processed is obtained, count, for each moire image region in the image to be processed, an interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, and obtain a statistical result;
the device further comprises a region dividing module, which is used for uniformly dividing the moire image region into a plurality of sub-regions if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the plurality of sub-regions as new moire image regions of the image to be processed.
In a possible embodiment, the region dividing module is specifically configured to, if the moire image region is not a sub-region obtained by dividing other moire image regions by a preset number of times, divide the moire image region into a plurality of sub-regions, and use the plurality of sub-regions as a new image region of the image to be processed;
the device further comprises a filtering module, wherein the filtering module is used for performing filtering processing on the moire image area to obtain a processed image of the moire image area if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times.
An embodiment of the present invention further provides an electronic device, as shown in fig. 5, including:
a memory 501 for storing a computer program;
the processor 502 is configured to implement the following steps when executing the program stored in the memory 501:
counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, so as to obtain a statistical result;
determining a re-interpolation direction corresponding to the moire image area according to the statistical result;
and in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain an image processed by the moire image area.
In a possible embodiment, the determining, according to the statistical result, a re-interpolation direction corresponding to the moire image region includes:
and if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction as the designated direction is greater than a preset number threshold, determining the re-interpolation direction as the direction orthogonal to the designated direction, wherein the designated direction is the horizontal direction or the vertical direction.
In a possible embodiment, the determining, according to the statistical result, a re-interpolation direction corresponding to the moire image region includes:
if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction being the designated direction is not more than a preset number threshold, respectively interpolating the original Bayer image region in the horizontal direction and the vertical direction to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
In a possible embodiment, when counting, for each moire image region in the image to be processed, an interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, to obtain a statistical result, the method includes:
until an image processed in each moire image area in the image to be processed is obtained, counting the interpolation direction of original Bayer image pixel points corresponding to each moire pixel point in the moire image area when the image to be processed is obtained based on the original Bayer image interpolation aiming at each moire image area in the image to be processed, and obtaining a statistical result;
the method further comprises the step of uniformly dividing the moire image area into a plurality of sub-areas if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the sub-areas as a new moire image area of the image to be processed.
In a possible embodiment, the dividing the moire image region into a plurality of sub-regions and regarding the plurality of sub-regions as a new moire image region includes:
if the moire image area is not a sub-area obtained by dividing other moire image areas for a preset number of times, dividing the moire image area into a plurality of sub-areas, and taking the plurality of sub-areas as a new image area of the image to be processed;
the method further comprises the following steps:
and if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times, carrying out filtering processing on the moire image area to obtain a processed image of the moire image area.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
In another embodiment of the present invention, a computer-readable storage medium is further provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the moire processing methods in the above embodiments.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the moire processing methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, the computer-readable storage medium, and the computer program product, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A moire processing method, comprising:
counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the original Bayer image interpolation, so as to obtain a statistical result;
determining a re-interpolation direction corresponding to the moire image area according to the statistical result;
and in the re-interpolation direction, interpolating the original Bayer image area corresponding to the moire image area to obtain an image processed by the moire image area.
2. The method of claim 1, wherein determining the re-interpolation direction corresponding to the moire image region according to the statistical result comprises:
and if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction as the designated direction is greater than a preset number threshold, determining the re-interpolation direction as the direction orthogonal to the designated direction, wherein the designated direction is the horizontal direction or the vertical direction.
3. The method of claim 1, wherein determining the re-interpolation direction corresponding to the moire image region according to the statistical result comprises:
if the statistical result shows that the number of the pixel points of the original Bayer image with the interpolation direction being the designated direction is not more than a preset number threshold, respectively interpolating the original Bayer image region in the horizontal direction and the vertical direction to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
4. The method according to claim 3, wherein when the statistics is performed on each moire image region in the image to be processed based on the interpolation of the original Bayer image to obtain the image to be processed, the statistics is obtained by counting the interpolation direction of the original Bayer image pixel point corresponding to each moire pixel point in the moire image region, and the method comprises:
until an image processed in each moire image area in the image to be processed is obtained, counting the interpolation direction of original Bayer image pixel points corresponding to each moire pixel point in the moire image area when the image to be processed is obtained based on the original Bayer image interpolation aiming at each moire image area in the image to be processed, and obtaining a statistical result;
the method further comprises the step of uniformly dividing the moire image area into a plurality of sub-areas if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the sub-areas as a new moire image area of the image to be processed.
5. The method according to claim 4, wherein the dividing the moire image region into a plurality of sub-regions and regarding the plurality of sub-regions as new moire image regions comprises:
if the moire image area is not a sub-area obtained by dividing other moire image areas for a preset number of times, dividing the moire image area into a plurality of sub-areas, and taking the plurality of sub-areas as a new image area of the image to be processed;
the method further comprises the following steps:
and if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times, carrying out filtering processing on the moire image area to obtain a processed image of the moire image area.
6. A moire processing device, said device comprising:
the interpolation direction counting module is used for counting the interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on the interpolation of the original Bayer image aiming at each moire image region in the image to be processed, so as to obtain a counting result;
the interpolation direction determining module is used for determining a re-interpolation direction corresponding to the moire image area according to the statistical result;
and the re-interpolation module is used for interpolating the original Bayer image area corresponding to the moire image area in the re-interpolation direction to obtain an image processed by the moire image area.
7. The apparatus according to claim 6, wherein the interpolation direction determining module is specifically configured to determine the re-interpolation direction as a direction orthogonal to the specified direction if the statistical result indicates that the number of original Bayer image pixel points whose interpolation direction is the specified direction is greater than a preset number threshold, where the specified direction is a horizontal direction or a vertical direction.
8. The apparatus according to claim 6, wherein the interpolation direction determining module is specifically configured to, if the statistical result indicates that the number of pixels of the raw Bayer image whose interpolation direction is the specified direction is not greater than a preset number threshold, interpolate the raw Bayer image region in a horizontal direction and a vertical direction respectively to obtain a horizontal direction interpolation result and a vertical direction interpolation result;
calculating the similarity between the horizontal direction interpolation result and the moire pattern image area as a first similarity;
calculating the similarity between the interpolation result in the vertical direction and the image of the moire area as a second similarity;
and if the absolute value of the difference value between the first similarity and the second similarity is larger than a preset judgment threshold, taking the interpolation direction corresponding to the larger similarity of the first similarity and the second similarity as a re-interpolation direction.
9. The apparatus according to claim 8, wherein the interpolation direction statistics module is specifically configured to, until an image after processing of each moire image region in the image to be processed is obtained, count, for each moire image region in the image to be processed, an interpolation direction of an original Bayer image pixel point corresponding to each moire pixel point in the moire image region when the image to be processed is obtained based on interpolation of the original Bayer image, and obtain a statistical result;
the device further comprises a region dividing module, which is used for uniformly dividing the moire image region into a plurality of sub-regions if the absolute value of the difference value between the first similarity and the second similarity is not larger than a preset judgment threshold value, and taking the plurality of sub-regions as new moire image regions of the image to be processed.
10. The apparatus according to claim 9, wherein the region dividing module is specifically configured to, if the moire image region is not a sub-region obtained by dividing other moire image regions by a preset number of times, divide the moire image region into a plurality of sub-regions, and use the plurality of sub-regions as a new image region of the image to be processed;
the device further comprises a filtering module, wherein the filtering module is used for performing filtering processing on the moire image area to obtain a processed image of the moire image area if the moire image area is a sub-area obtained by dividing other moire image areas for a preset number of times.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910807521.0A CN111340714B (en) | 2019-08-29 | 2019-08-29 | Moire pattern processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910807521.0A CN111340714B (en) | 2019-08-29 | 2019-08-29 | Moire pattern processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111340714A true CN111340714A (en) | 2020-06-26 |
| CN111340714B CN111340714B (en) | 2023-08-08 |
Family
ID=71183531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910807521.0A Active CN111340714B (en) | 2019-08-29 | 2019-08-29 | Moire pattern processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111340714B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554561A (en) * | 2021-07-01 | 2021-10-26 | Oppo广东移动通信有限公司 | Image generation method and device, electronic equipment and computer-readable storage medium |
| CN116074484A (en) * | 2023-01-15 | 2023-05-05 | 山东产研卫星信息技术产业研究院有限公司 | Bayer color reconstruction method of CMOS satellite image |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007202916A (en) * | 2006-02-03 | 2007-08-16 | Hitachi Medical Corp | Medical image display device |
| US20150235725A1 (en) * | 2014-02-14 | 2015-08-20 | Konica Minolta, Inc. | X-ray imaging system and image processing method |
| CN105046631A (en) * | 2014-04-25 | 2015-11-11 | 佳能株式会社 | Image processing apparatus, and image processing method |
| CN105163098A (en) * | 2015-09-29 | 2015-12-16 | 联想(北京)有限公司 | Image processing method and device |
| JP2016005166A (en) * | 2014-06-18 | 2016-01-12 | リコーイメージング株式会社 | Video imaging device and video imaging method |
| JP2016220176A (en) * | 2015-05-26 | 2016-12-22 | ハンファテクウィン株式会社Ｈａｎｗｈａ Ｔｅｃｈｗｉｎ Ｃｏ．，Ｌｔｄ． | Image processing device, image processing method and imaging device |
| CN108615227A (en) * | 2018-05-08 | 2018-10-02 | 浙江大华技术股份有限公司 | A kind of suppressing method and equipment of image moire fringes |
| CN108734668A (en) * | 2017-04-21 | 2018-11-02 | 展讯通信（上海）有限公司 | Image color restoration methods, device, computer readable storage medium and terminal |
| CN108921823A (en) * | 2018-06-08 | 2018-11-30 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
-
2019
- 2019-08-29 CN CN201910807521.0A patent/CN111340714B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007202916A (en) * | 2006-02-03 | 2007-08-16 | Hitachi Medical Corp | Medical image display device |
| US20150235725A1 (en) * | 2014-02-14 | 2015-08-20 | Konica Minolta, Inc. | X-ray imaging system and image processing method |
| CN105046631A (en) * | 2014-04-25 | 2015-11-11 | 佳能株式会社 | Image processing apparatus, and image processing method |
| JP2016005166A (en) * | 2014-06-18 | 2016-01-12 | リコーイメージング株式会社 | Video imaging device and video imaging method |
| JP2016220176A (en) * | 2015-05-26 | 2016-12-22 | ハンファテクウィン株式会社Ｈａｎｗｈａ Ｔｅｃｈｗｉｎ Ｃｏ．，Ｌｔｄ． | Image processing device, image processing method and imaging device |
| CN105163098A (en) * | 2015-09-29 | 2015-12-16 | 联想(北京)有限公司 | Image processing method and device |
| CN108734668A (en) * | 2017-04-21 | 2018-11-02 | 展讯通信（上海）有限公司 | Image color restoration methods, device, computer readable storage medium and terminal |
| CN108615227A (en) * | 2018-05-08 | 2018-10-02 | 浙江大华技术股份有限公司 | A kind of suppressing method and equipment of image moire fringes |
| CN108921823A (en) * | 2018-06-08 | 2018-11-30 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
Non-Patent Citations (5)
| Title |
|---|
| DAE-CHUL KIM, ET.AL: "Moire reduction using inflection point and color variation in digital camera of No optical low pass filter" * |
| DU SIC YOO, ET.AL: "Joint deblurring and demosaicing using edge information from bayer images" * |
| HWAMOG KIM, ET.AL: "Digital elevation modeling via curvature interpolation for lidar data" * |
| 邱菊: "基于插值算法的去彩色摩尔纹效应的研究" * |
| 邱香香: "基于CCD静态图像的摩尔纹去除算法研究" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554561A (en) * | 2021-07-01 | 2021-10-26 | Oppo广东移动通信有限公司 | Image generation method and device, electronic equipment and computer-readable storage medium |
| CN113554561B (en) * | 2021-07-01 | 2024-08-16 | Oppo广东移动通信有限公司 | Image generation method, device, electronic equipment and computer readable storage medium |
| CN116074484A (en) * | 2023-01-15 | 2023-05-05 | 山东产研卫星信息技术产业研究院有限公司 | Bayer color reconstruction method of CMOS satellite image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111340714B (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108833785B (en) | Fusion method and device of multi-view images, computer equipment and storage medium | |
| US10762655B1 (en) | Disparity estimation using sparsely-distributed phase detection pixels | |
| WO2019233264A1 (en) | Image processing method, computer readable storage medium, and electronic device | |
| CN111383186B (en) | Image processing method and device and terminal equipment | |
| CN112019768B (en) | Video generation method and device and electronic equipment | |
| CN108257186B (en) | Method and device for determining calibration image, camera and storage medium | |
| JP6830712B1 (en) | Random sampling Consistency-based effective area extraction method for fisheye images | |
| CN111340714B (en) | Moire pattern processing method and device and electronic equipment | |
| CN111598777A (en) | Sky cloud image processing method, computer device and readable storage medium | |
| CN111091572A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN111340693B (en) | Interpolation method and device of Bayer image and electronic equipment | |
| CN111767752B (en) | Two-dimensional code identification method and device | |
| CN109285122A (en) | A kind of method and apparatus carrying out image procossing | |
| CN110689565B (en) | Depth map determination method and device and electronic equipment | |
| CN112384945B (en) | Super resolution using natural hand-held motion applied to user equipment | |
| WO2020098325A1 (en) | Image synthesis method, electronic device and storage medium | |
| CN117392161B (en) | Calibration plate corner point for long-distance large perspective distortion and corner point number determination method | |
| CN111340863B (en) | Moire pixel point determination method and device and electronic equipment | |
| CN111885371A (en) | Image occlusion detection method and device, electronic equipment and computer readable medium | |
| CN110874814B (en) | Image processing method, image processing device and terminal equipment | |
| WO2020133538A1 (en) | Circular feature detection method, processing system and apparatus having storage function | |
| CN112312108A (en) | White balance abnormity determining method and device, storage medium and electronic equipment | |
| CN114140362B (en) | Thermal imaging image correction method and device | |
| JP5683153B2 (en) | Image processing apparatus and image processing method | |
| CN114596220A (en) | Method for correcting image lateral chromatic aberration, electronic device and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |