CN112070671A - Mosaic removal method, system, terminal and storage medium based on spectrum analysis - Google Patents
Mosaic removal method, system, terminal and storage medium based on spectrum analysis Download PDFInfo
- Publication number
- CN112070671A CN112070671A CN202010921922.1A CN202010921922A CN112070671A CN 112070671 A CN112070671 A CN 112070671A CN 202010921922 A CN202010921922 A CN 202010921922A CN 112070671 A CN112070671 A CN 112070671A
- Authority
- CN
- China
- Prior art keywords
- mosaic
- image
- peak
- area
- spectrogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010183 spectrum analysis Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000012216 screening Methods 0.000 claims abstract description 21
- 238000010586 diagram Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 11
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 2
- 238000001228 spectrum Methods 0.000 abstract description 9
- 238000010191 image analysis Methods 0.000 abstract description 6
- 238000003708 edge detection Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4015—Demosaicing, e.g. colour filter array [CFA], Bayer pattern
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention discloses a mosaic removal method, a mosaic removal system, a mosaic removal terminal and a storage medium based on spectrum analysis. The method comprises the following steps: carrying out Fourier transform on an original mosaic image to obtain a spectrogram of the mosaic image; respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic area in the original mosaic image according to the coordinates of the screened peak points; and clipping the mosaic area according to the positioning to obtain an image with the mosaic area removed. According to the embodiment of the application, the mosaic image is subjected to spectrum analysis by utilizing Fourier transform, and the spectrum difference between the mosaic region and other regions is detected, so that the mosaic region in the image is automatically, quickly and accurately positioned and removed, the introduction of noise is avoided, and the accuracy of image analysis is favorably improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, a system, a terminal, and a storage medium for removing mosaics based on spectrum analysis.
Background
In the internet era, in order to display the privacy of information on pictures or videos, mosaics need to be printed on certain specific parts; however, large area of mosaic will bring noise pollution, and affect the accuracy of image analysis and other techniques. Taking the face video heart rate estimation as an example, due to the personal privacy problem, glasses and a mouth part in a face image are usually mosaiced, but since the face video heart rate estimation needs to perform signal detection based on slight variation of the skin of the face, a large area of mosaics brings a significant challenge to the face video heart rate estimation. The removal of mosaics therefore appears to be crucial in the image analysis task.
Most of the researches on mosaic removal in the prior art are to recover the mosaic in the picture by using a generation countermeasure network, such as a PLUSE (Photo Upsampling via Space Exploration self-supervision on picture sampling) method. However, the methods are models trained by using face data sets in countries in europe and america, so that great limitations exist in practical use, and the face restored by using the PLUSE method cannot improve the model accuracy, but introduces greater noise.
Disclosure of Invention
The invention provides a mosaic removal method, a mosaic removal system, a mosaic removal terminal and a mosaic removal storage medium based on spectrum analysis, which can overcome the defects in the prior art to a certain extent.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a mosaic removal method based on spectral analysis comprises the following steps:
carrying out Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions, wherein the preset conditions are as follows: the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
positioning a mosaic area in the mosaic image according to the coordinate of the screened peak point;
and clipping the mosaic area according to the positioning to obtain an image with the mosaic area removed.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: before the fourier transform is performed on the mosaic image, the method further comprises the following steps:
and carrying out edge pixel point detection on the mosaic image by using a sobel operator to obtain a boundary image of the mosaic image.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the Fourier transforming the mosaic image comprises:
and grouping the pixel values in the boundary image according to rows, and performing Fourier transform on each group of pixel values respectively to obtain a spectrogram corresponding to each group of pixel values.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the positioning the mosaic area in the mosaic image according to the position of the screened peak point comprises:
calculating to obtain the frequency ratio of the low frequency and the high frequency of each spectrogram according to the set high-low frequency critical point value, and obtaining a high-low frequency ratio map of each spectrogram;
detecting a peak point of the high-low frequency ratio map by using a peak detection method to obtain a peak point map;
screening all peak points with the longitudinal axis larger than a first preset value from the peak point diagram, and recording the abscissa of the peak points;
the first preset value is set according to the position and the size of the mosaic area;
sequencing the horizontal coordinates of the peak points, counting the interval values of the horizontal coordinates of the peak points, screening out the peak points with the interval values larger than a second preset value, and recording the vertical coordinates of the peak points to obtain the vertical coordinate position information of the mosaic area in the mosaic image;
and the second preset value is set according to the length of the mosaic area.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the positioning the mosaic area in the mosaic image according to the screened coordinates of the peak point further comprises:
and rotating the mosaic image by 90 degrees, and carrying out vertical coordinate positioning on the mosaic area on the rotated mosaic image again, wherein the vertical coordinate positioning result obtained after the rotation is the horizontal coordinate of the mosaic area in the mosaic image.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the number of the mosaic areas is at least one.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: when the number of the mosaic areas is two or more, rotating the mosaic image by 90 ° and performing vertical coordinate positioning of the mosaic areas on the rotated mosaic image again further includes:
and dividing the mosaic image into two or more images according to the position of the mosaic area, and respectively rotating the two or more divided images by 90 degrees and positioning the vertical coordinate.
The embodiment of the invention adopts another technical scheme that: a mosaic removal system based on spectral analysis, comprising:
a Fourier transform module: the mosaic image processing device is used for carrying out Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
the mosaic positioning module: the mosaic image processing device is used for respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic region in the mosaic image according to the coordinates of the screened peak points;
wherein the preset conditions are as follows:
the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
a mosaic cutting module: and the mosaic area is cut according to the positioning to obtain an image with the mosaic area removed.
The embodiment of the invention adopts another technical scheme that: a terminal comprising a processor, a memory coupled to the processor, wherein,
the memory stores program instructions for implementing the above-described mosaic removal method based on spectral analysis;
the processor is to execute the program instructions stored by the memory to perform the spectral analysis based mosaic removal operation.
The embodiment of the invention adopts another technical scheme that: a storage medium storing program instructions executable by a processor to perform the spectral analysis based mosaic removal method described above.
The invention has the beneficial effects that: according to the embodiment of the application, the frequency spectrum analysis is carried out on the mosaic image by utilizing the Fourier transform, and the frequency spectrum difference between the mosaic region and other regions is detected, so that the mosaic region in the image is automatically, quickly and accurately positioned and removed, the introduction of noise is avoided, the accuracy of image analysis is favorably improved, and the high efficiency and the practicability are well embodied.
Drawings
Fig. 1 is a schematic flow chart of a mosaic removal method based on spectrum analysis according to a first embodiment of the present invention;
FIG. 2 is a schematic flowchart of a mosaic removal method based on spectrum analysis according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a frame of face image captured from a video;
FIG. 4 is a flowchart illustrating a mosaic removal method based on spectral analysis according to a third embodiment of the present invention;
fig. 5 is a schematic diagram of a face boundary image detected in the embodiment of the present application;
FIG. 6 is a frequency spectrum diagram obtained by performing Fourier transform on a face boundary image according to an embodiment of the present invention; wherein, (a), (b), (c) and (d) are respectively the spectrogram of lines 180, 184, 200 and 294 in the face boundary image;
FIG. 7 is a graph of high and low frequency ratios calculated based on a spectrogram in accordance with an embodiment of the present invention;
FIG. 8 is a high-low frequency ratio map obtained based on the high-low frequency ratio map detection in accordance with an embodiment of the present invention;
fig. 9 is a face image with mosaic areas removed according to a third embodiment of the present invention;
fig. 10 is a schematic flowchart of a mosaic removal method based on spectrum analysis according to a fourth embodiment of the present invention;
fig. 11 is a face image with mosaic areas removed according to a fourth embodiment of the present invention;
FIG. 12 is a schematic structural diagram of a mosaic removal system based on spectrum analysis according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of a terminal according to an embodiment of the present invention;
FIG. 14 is a schematic diagram of a storage medium structure according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first", "second" and "third" in the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of the feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise. All directional indicators (such as up, down, left, right, front, and rear … …) in the embodiments of the present invention are only used to explain the relative positional relationship between the components, the movement, and the like in a specific posture (as shown in the drawings), and if the specific posture is changed, the directional indicator is changed accordingly. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Fig. 1 is a schematic flow chart of a mosaic removing method based on spectrum analysis according to a first embodiment of the present invention. The mosaic removal method based on the spectrum analysis of the first embodiment of the invention comprises the following steps:
s10: carrying out Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
the fourier transform of the mosaic image is specifically as follows:
carrying out edge pixel point detection on the mosaic image by using a sobel operator to obtain a boundary image of the mosaic image;
and grouping the pixel values in the boundary image according to rows, and performing Fourier transform on each group of pixel values respectively to obtain a spectrogram corresponding to each group of pixel values.
S11, respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions;
the spectrum of the area without the mosaic is relatively smooth, and the spectrum of the mosaic area is relatively vibrated, so that the spectrum analysis is carried out based on the rule. The preset conditions are as follows: the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value.
S12: positioning a mosaic area in the mosaic image according to the coordinate of the screened peak point;
the mosaic area positioning specifically comprises the following steps:
calculating to obtain the frequency ratio of the low frequency and the high frequency of each spectrogram according to the set high-low frequency critical point value, and obtaining a high-low frequency ratio map of each spectrogram;
detecting a peak point of the high-low frequency ratio map by using a peak detection method to obtain a peak point map;
screening all peak points with the longitudinal axis larger than a first preset value from the peak point diagram, and recording the abscissa of the peak points;
and sequencing the horizontal coordinates of the peak points, counting the interval values of the horizontal coordinates of the peak points, screening out the peak points with the interval values larger than a second preset value, and recording the vertical coordinates of the peak points to obtain the coordinate position information of the mosaic area in the original mosaic image.
S13: and clipping the mosaic area according to the positioning to obtain an image with the mosaic area removed.
Based on the above, the first embodiment of the present application performs spectrum analysis on a mosaic image by using fourier transform, and detects the spectrum difference between a mosaic region and other regions, so as to automatically, quickly and accurately locate the mosaic region in the image, and remove the mosaic region located in the image, thereby avoiding the introduction of noise, facilitating the improvement of the accuracy of image analysis, and achieving good representation in terms of efficiency and practicability.
Further, please refer to fig. 2, which is a flowchart illustrating a mosaic removing method based on spectrum analysis according to a second embodiment of the present invention. The mosaic removal method based on the spectrum analysis of the second embodiment of the invention comprises the following steps:
s20: reading a mosaic image;
s21: carrying out edge pixel point detection on the mosaic image by using a sobel operator to obtain a boundary image of the mosaic image;
in this step, the sobel operator is a discrete difference operator, and is used for calculating the approximate gradient of the brightness value of the image pixel point. The edge detection algorithm specifically comprises the following steps:
the image of the memorial mosaic is I, and operators of the sobel operator in the horizontal direction and the vertical direction are respectively as follows:
the edge detection of the mosaic image by using the sobel operator is as follows:
s22: grouping pixel values in the boundary image according to lines, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values;
in this step, the pixel value grouping method is: the grouping is performed by the height value of the image, and the width value of the image is set as the pixel value data of each group.
S23: respectively carrying out spectrum analysis on each spectrogram, and calculating according to a set high-low frequency critical point value to obtain a low-frequency and high-frequency ratio of each spectrogram so as to obtain a high-low frequency ratio map of each spectrogram;
in the step, because the frequency spectrum of the area without the mosaic is relatively smooth and the frequency spectrum of the mosaic area is relatively vibrated, the frequency spectrum analysis is carried out based on the rule. The high and low frequency critical point values may be set based on empirical statistics.
S24: respectively carrying out peak point detection on the high-frequency ratio graph and the low-frequency ratio graph of each frequency spectrogram by using a peak detection method to obtain a peak point graph, screening all peak points with the longitudinal axis larger than a first preset value from the peak point graph, and recording the abscissa of the screened peak points;
in this step, the first preset value may be set according to the position and size of the mosaic area in the mosaic image.
S25: sorting the horizontal coordinates of the screened peak points, screening the peak points with the horizontal coordinate interval value larger than a second preset value, and recording the vertical coordinates of the screened peak points to obtain the vertical coordinate position information of the mosaic area in the mosaic image;
in this step, the peak point screening method is: respectively counting the abscissa interval value of the peak point obtained by screening in each peak point diagram, if the abscissa interval value of one peak point and the other peak point in one peak point diagram is greater than a second preset value, considering the peak point as the initial position of the mosaic area in the mosaic image, and the ordinate corresponding to the peak point is the ordinate position information of the initial position of the mosaic area; and repeating the steps until all peak point diagrams are screened, and obtaining the vertical coordinate position information of the whole mosaic area in the mosaic image. The second preset value can be set according to the length or the size of the mosaic area.
S26: rotating the mosaic image by 90 degrees, re-executing S21-S25, and positioning the vertical coordinates of the mosaic area of the rotated mosaic image again;
in this step, it can be understood that the vertical coordinate positioning result of the mosaic area obtained after rotation is the horizontal coordinate of the mosaic area in the original mosaic image.
S27: clipping the mosaic area according to the two vertical coordinate positioning results to obtain an image with the mosaic area removed;
based on the above, the second embodiment of the present application performs edge detection on a mosaic image, and then performs spectrum analysis on the edge detection image by using fourier transform to detect out a spectrum difference between a mosaic region and other regions, so as to automatically, quickly and accurately locate the mosaic region in the image, and remove the mosaic region located in the image, thereby avoiding introduction of noise, facilitating improvement of accuracy of image analysis, and achieving good representation in terms of efficiency and practicability.
In order to more clearly illustrate the implementation process of the present application, the following embodiments specifically describe the application to the removal of eye mosaics in a face image. As shown in fig. 3, in a frame of face image captured from a video, it can be clearly seen that the eyes in the face image are covered by the mosaic, and the color transformation of the mosaic region is consistent with the color change of the key part of the face, so that the mosaic cannot be directly removed by using the color difference of the mosaic in the image. The mosaic area is a large rectangular block spliced by a plurality of small rectangular color blocks, so that the mosaic area can be positioned only by finding the difference between the mosaic area and other areas in the face image, and the mosaic area is removed.
Specifically, please refer to fig. 4, which is a flowchart illustrating a mosaic removing method based on spectrum analysis according to a third embodiment of the present invention. The mosaic removal method based on the spectral analysis of the third embodiment of the present invention includes the steps of:
s30: reading a face image containing a mosaic area;
s31: carrying out edge pixel point detection on the face image by using a sobel operator to obtain a face boundary image;
in the example of the face avatar shown in fig. 2, the detected face boundary image is shown in fig. 5.
S32: grouping pixel values in the face boundary image according to rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values;
assuming that the face boundary image shown in fig. 5 is an image with a height 473 and a width 373, the image is divided into 473 groups of pixel data, each group of pixel data includes 373 pixel values, and then each group of pixel data is subjected to fourier transform, so as to obtain a spectrogram as shown in fig. 6, where (a), (b), (c), and (d) are spectrograms of lines drawn in lines 180, 184, 200, and 294 in fig. 5, respectively, and in the spectrogram, the horizontal axis represents frequency and the vertical axis represents amplitude.
S33: carrying out spectrum analysis on the spectrogram, and calculating according to a set high-low frequency critical point value to obtain a low-frequency and high-frequency ratio of each spectrogram so as to obtain a high-low frequency ratio map of each spectrogram;
as can be seen from fig. 6, in the spectrogram after fourier transform, most of the information is concentrated in the low frequency part, such as lines 180 and 200 shown in (a) and (c) of fig. 6; the frequency distribution is wider in the split line where the mosaic is located, as shown in lines 184 and 200 in fig. 6 (b) and (d). In the embodiment of the present application, the high and low frequency critical point values are set to 15 according to the empirical statistical value of the face image, and a high and low frequency ratio graph calculated according to the high and low frequency critical point values is shown in fig. 7.
S34: respectively carrying out peak point detection on the high-frequency ratio graph and the low-frequency ratio graph of each frequency spectrogram by using a peak detection method to obtain a peak point graph, screening all peak points with the longitudinal axis larger than a first preset value from the peak point graph, and recording the abscissa of the screened peak points;
fig. 8 shows a peak point diagram obtained by peak point detection. According to the prior values counted by all the face image data sets, mosaics do not appear in the first 80 rows of the face images, so peak points with the horizontal axis smaller than 80 in the high-low frequency ratio image are removed firstly, and then the peak points are screened. Preferably, the first preset value is set to 0.5.
S35: sorting the horizontal coordinates of the screened peak points, screening out the peak points with the horizontal coordinate interval value larger than a second preset value, and recording the vertical coordinates of the screened peak points to obtain the vertical coordinate position information of the eye mosaic in the mosaic image;
wherein the second preset value is preferably set to 25.
S36: rotating the face image by 90 degrees, re-executing S31-S35, and positioning the vertical coordinates of the eye mosaic area of the rotated face image again;
it can be understood that the vertical coordinate positioning result of the eye mosaic region obtained after rotation is the horizontal coordinate of the mosaic region in the original face image.
S37: clipping the eye mosaic region according to the two-time vertical coordinate positioning result to obtain a face image with the eye mosaic region removed;
the face image with the mosaic area removed is shown in fig. 9.
It can be understood that the embodiments of the present application can be applied to various types of mosaic images, and can locate and remove multiple mosaic areas in one image. The following embodiment specifically describes an example of eye mosaic removal and mouth mosaic removal in the face image shown in fig. 2.
Fig. 10 is a flowchart illustrating a mosaic removing method based on spectrum analysis according to a fourth embodiment of the present invention. The mosaic removal method based on the spectral analysis of the fourth embodiment of the invention comprises the following steps:
s40: reading a face image containing a mosaic area;
s41: carrying out edge pixel point detection on the face image by using a sobel operator to obtain a face boundary image;
the detected face boundary image is shown in fig. 5.
S42: grouping pixel values in the face boundary image according to rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values;
assuming that the face boundary image shown in fig. 5 is an image with a height 473 and a width 373, the image is divided into 473 groups of pixel data, each group of pixel data includes 373 pixel values, and then each group of pixel data is subjected to fourier transform, so as to obtain a spectrogram as shown in fig. 6, where (a), (b), (c), and (d) are spectrograms of lines drawn in lines 180, 184, 200, and 294 in fig. 5, respectively, and in the spectrogram, the horizontal axis represents frequency and the vertical axis represents amplitude.
S33: carrying out spectrum analysis on the spectrogram, and calculating according to a set high-low frequency critical point value to obtain a low-frequency and high-frequency ratio of each spectrogram so as to obtain a high-low frequency ratio map of each spectrogram;
as can be seen from fig. 6, in the spectrogram after fourier transform, most of the information is concentrated in the low frequency part, such as lines 180 and 200 shown in (a) and (c) of fig. 6; the frequency distribution is wider in the split line where the mosaic is located, as shown in lines 184 and 200 in fig. 6 (b) and (d). In the embodiment of the present application, the high and low frequency critical point values are set to 15 according to the empirical statistical value of the face image, and a high and low frequency ratio graph calculated according to the high and low frequency critical point values is shown in fig. 7.
S44: respectively carrying out peak point detection on the high-frequency ratio graph and the low-frequency ratio graph of each frequency spectrogram by using a peak detection method to obtain a peak point graph, screening all peak points with the longitudinal axis larger than a first preset value from the peak point graph, and recording the abscissa of the screened peak points;
fig. 8 shows a peak point diagram obtained by peak point detection. According to the prior values counted by all the face image data sets, mosaics do not appear in the first 80 rows of the face images, so peak points with the horizontal axis smaller than 80 in the high-low frequency ratio image are removed firstly, and then the peak points are screened. Preferably, the first preset value is set to 0.5.
S45: sorting the horizontal coordinates of the screened peak points, screening out the peak points with the horizontal coordinate interval value larger than a second preset value, recording the vertical coordinates of the screened peak points, and respectively obtaining the vertical coordinate position information of the two mosaic areas of the eye part and the mouth part in the mosaic image;
wherein the second preset value is preferably set to 25.
S46: dividing two mosaic areas of eyes and a mouth in the human face image into two images, respectively rotating the two images by 90 degrees, and re-executing S41-S45 to respectively position the vertical coordinates of the mosaic areas of the eyes and the mouth of the two rotated images;
it can be understood that the vertical coordinate positioning result of the eye mosaic region and the mouth mosaic region obtained after rotation is the horizontal coordinate of the two mosaic regions in the original face image.
S47: respectively cutting mosaic areas of eyes and a mouth according to the two-time vertical coordinate positioning results to obtain a face image with the mosaic areas removed;
the face image with the mosaic area removed is shown in fig. 11.
In an alternative embodiment, it is also possible to: and uploading the result of the mosaic removal method based on the spectrum analysis to a block chain.
Specifically, the corresponding digest information is obtained based on the result of the mosaic removing method based on the spectrum analysis, and specifically, the digest information is obtained by performing hash processing on the result of the mosaic removing method based on the spectrum analysis, for example, the hash processing is performed by using the sha256s algorithm. Uploading summary information to the blockchain can ensure the safety and the fair transparency of the user. The user may download the summary information from the blockchain to verify whether the result of the spectral analysis-based mosaic removal method is tampered. The blockchain referred to in this example is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Please refer to fig. 12, which is a schematic structural diagram of a mosaic removal system based on spectrum analysis according to an embodiment of the present invention. The mosaic removal system 40 based on spectrum analysis according to the embodiment of the present invention includes:
the fourier transform module 41: the mosaic image processing method comprises the steps of carrying out Fourier transform on an original mosaic image to obtain a spectrogram of the mosaic image;
the mosaic positioning module 42: the mosaic image processing device is used for respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic region in the original mosaic image according to the coordinates of the screened peak points;
wherein the preset conditions are as follows:
the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
the mosaic clipping module 43: and the mosaic area is cut according to the positioning to obtain an image with the mosaic area removed.
Fig. 13 is a schematic diagram of a terminal structure according to an embodiment of the present invention. The terminal 50 comprises a processor 51, a memory 52 coupled to the processor 51.
The memory 52 stores program instructions for implementing the spectral analysis based mosaic removal method described above.
The processor 51 is operative to execute program instructions stored in the memory 52 to perform a spectral analysis based mosaic removal operation.
The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having signal processing capabilities. The processor 51 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Referring to fig. 14, fig. 14 is a schematic structural diagram of a storage medium according to an embodiment of the invention. The storage medium of the embodiment of the present invention stores a program file 61 capable of implementing all the methods described above, wherein the program file 61 may be stored in the storage medium in the form of a software product, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices, such as a computer, a server, a mobile phone, and a tablet.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A mosaic removal method based on spectral analysis is characterized by comprising the following steps:
carrying out Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions, wherein the preset conditions are as follows: the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
positioning a mosaic area in the mosaic image according to the coordinate of the screened peak point;
and clipping the mosaic area according to the positioning to obtain an image with the mosaic area removed.
2. The method for removing mosaic based on spectral analysis according to claim 1, wherein said fourier transform of the mosaic image further comprises:
and carrying out edge pixel point detection on the mosaic image by using a sobel operator to obtain a boundary image of the mosaic image.
3. The method according to claim 2, wherein the fourier transforming the mosaic image comprises:
and grouping the pixel values in the boundary image according to rows, and performing Fourier transform on each group of pixel values respectively to obtain a spectrogram corresponding to each group of pixel values.
4. The method according to claim 3, wherein the locating the mosaic region in the mosaic image according to the position of the filtered peak point comprises:
calculating to obtain the frequency ratio of the low frequency and the high frequency of each spectrogram according to the set high-low frequency critical point value, and obtaining a high-low frequency ratio map of each spectrogram;
detecting a peak point of the high-low frequency ratio map by using a peak detection method to obtain a peak point map;
screening all peak points with the longitudinal axis larger than a first preset value from the peak point diagram, and recording the abscissa of the peak points;
the first preset value is set according to the position and the size of the mosaic area;
sequencing the horizontal coordinates of the peak points, counting the interval values of the horizontal coordinates of the peak points, screening out the peak points with the interval values larger than a second preset value, and recording the vertical coordinates of the peak points to obtain the vertical coordinate position information of the mosaic area in the original mosaic image;
and the second preset value is set according to the length of the mosaic area.
5. The method according to claim 4, wherein the locating the mosaic area in the mosaic image according to the filtered coordinates of the peak point further comprises:
and rotating the original mosaic image by 90 degrees, and carrying out vertical coordinate positioning on the mosaic area on the rotated mosaic image again, wherein the vertical coordinate positioning result obtained after the rotation is the horizontal coordinate of the mosaic area in the mosaic image.
6. The spectral analysis based mosaic removal method according to any one of claims 1 to 5, wherein said number of mosaic regions is at least one.
7. The spectral analysis-based mosaic removal method of claim 6, wherein when said number of mosaic regions is two or more, said rotating said mosaic image by 90 ° and re-aligning said rotated mosaic image by the ordinate of said mosaic region further comprises:
and dividing the mosaic image into two or more images according to the position of the mosaic area, and respectively rotating the two or more divided images by 90 degrees and positioning the vertical coordinate.
8. A mosaic removal system based on spectral analysis, comprising:
a Fourier transform module: the mosaic image processing device is used for carrying out Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
the mosaic positioning module: the mosaic image processing device is used for respectively carrying out spectrum analysis on the spectrogram, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic region in the mosaic image according to the coordinates of the screened peak points;
wherein the preset conditions are as follows:
the longitudinal axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
a mosaic cutting module: and the mosaic area is cut according to the positioning to obtain an image with the mosaic area removed.
9. A terminal, comprising a processor, a memory coupled to the processor, wherein,
the memory stores program instructions for implementing the spectral analysis based mosaic removal method of any one of claims 1 to 7;
the processor is configured to execute the program instructions stored by the memory to perform the spectral analysis based mosaic removal method.
10. A storage medium having stored thereon program instructions executable by a processor to perform the spectral analysis based mosaic removal method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921922.1A CN112070671A (en) | 2020-09-04 | 2020-09-04 | Mosaic removal method, system, terminal and storage medium based on spectrum analysis |
| PCT/CN2020/124814 WO2021143272A1 (en) | 2020-09-04 | 2020-10-29 | Spectral analysis-based de-mosaicing method, system, terminal, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921922.1A CN112070671A (en) | 2020-09-04 | 2020-09-04 | Mosaic removal method, system, terminal and storage medium based on spectrum analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112070671A true CN112070671A (en) | 2020-12-11 |
Family
ID=73665064
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010921922.1A Pending CN112070671A (en) | 2020-09-04 | 2020-09-04 | Mosaic removal method, system, terminal and storage medium based on spectrum analysis |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112070671A (en) |
| WO (1) | WO2021143272A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116453173A (en) * | 2022-12-16 | 2023-07-18 | 南京奥看信息科技有限公司 | Picture processing method based on picture region segmentation technology |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02295354A (en) * | 1989-05-10 | 1990-12-06 | Canon Inc | Picture processor |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3136071B2 (en) * | 1995-04-14 | 2001-02-19 | シャープ株式会社 | Image processing device and imaging device |
| JPH09128529A (en) * | 1995-10-30 | 1997-05-16 | Sony Corp | Removing method for noise in digital image based on projection |
| EP1534016A3 (en) * | 2003-11-21 | 2007-03-07 | Matsushita Electric Industrial Co., Ltd. | Using motion vectors and the occurrence of coefficient quantization levels to drive a noise removal image postprocessing |
| CN104517269A (en) * | 2014-12-25 | 2015-04-15 | 深圳市一体太赫兹科技有限公司 | Terahertz image strip processing method and system |
| CN109410123B (en) * | 2018-10-15 | 2023-08-18 | 深圳市能信安科技股份有限公司 | Deep learning-based mosaic removing method and device and electronic equipment |
| CN110415175B (en) * | 2019-08-07 | 2023-06-02 | 杭州微帧信息科技有限公司 | Method for rapidly removing flat region coding mosaic |
| CN110517206B (en) * | 2019-09-05 | 2021-10-15 | 锐芯微电子股份有限公司 | Method and device for eliminating color moire |
-
2020
- 2020-09-04 CN CN202010921922.1A patent/CN112070671A/en active Pending
- 2020-10-29 WO PCT/CN2020/124814 patent/WO2021143272A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02295354A (en) * | 1989-05-10 | 1990-12-06 | Canon Inc | Picture processor |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
Non-Patent Citations (2)
| Title |
|---|
| E. DUBOIS: "Frequency-domain methods for demosaicking of Bayer-sampled color images", IEEE SIGNAL PROCESSING LETTERS, vol. 12, no. 12, 31 December 2005 (2005-12-31), pages 847, XP002518020, DOI: 10.1109/LSP.2005.859503 * |
| 章化冰等: "基于模板匹配的马赛克视频检测方法研究", 中国有线电视, no. 11, 15 November 2010 (2010-11-15), pages 1254 - 1256 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116453173A (en) * | 2022-12-16 | 2023-07-18 | 南京奥看信息科技有限公司 | Picture processing method based on picture region segmentation technology |
| CN116453173B (en) * | 2022-12-16 | 2023-09-08 | 南京奥看信息科技有限公司 | Picture processing method based on picture region segmentation technology |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021143272A1 (en) | 2021-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2020533654A (en) | Holographic anti-counterfeit code inspection method and equipment | |
| CN112102164B (en) | Image processing method, device, terminal and storage medium | |
| CN103353881B (en) | Method and device for searching application | |
| CN109903302A (en) | A kind of altering detecting method for stitching image | |
| CN110765799B (en) | Client code scanning identification method, device, equipment and storage medium | |
| US11816946B2 (en) | Image based novelty detection of material samples | |
| US10657421B2 (en) | Shape detection | |
| JP2022550195A (en) | Text recognition method, device, equipment, storage medium and computer program | |
| CN111131688B (en) | Image processing method and device and mobile terminal | |
| CN112070671A (en) | Mosaic removal method, system, terminal and storage medium based on spectrum analysis | |
| CN114355467A (en) | Detection model establishment and multi-image-based track foreign matter detection method | |
| CN113570725A (en) | Three-dimensional surface reconstruction method and device based on clustering, server and storage medium | |
| CN111222446B (en) | Face recognition method, face recognition device and mobile terminal | |
| Gupta et al. | A simplistic global median filtering forensics based on frequency domain analysis of image residuals | |
| CN113269752A (en) | Image detection method, device terminal equipment and storage medium | |
| Mehta et al. | Near-duplicate detection for LCD screen acquired images using edge histogram descriptor | |
| CN111833285A (en) | Image processing method, image processing device and terminal equipment | |
| CN110880023A (en) | Method and device for detecting certificate picture | |
| CN114758145A (en) | Image desensitization method and device, electronic equipment and storage medium | |
| CN112487943A (en) | Method and device for removing duplicate of key frame and electronic equipment | |
| CN113554035A (en) | Feature matching method based on optical flow, intelligent terminal and computer storage medium | |
| CN109286798B (en) | Frame position identification method and system of projection screen and terminal equipment | |
| CN112995488B (en) | High-resolution video image processing method and device and electronic equipment | |
| CN110246227B (en) | Virtual-real fusion simulation experiment image data collection method and system | |
| US20220198208A1 (en) | Systems and methods for masking biometric information in images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |