CN108769523B - Image processing method and device, electronic equipment and computer readable storage medium - Google Patents
Image processing method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN108769523B CN108769523B CN201810579249.0A CN201810579249A CN108769523B CN 108769523 B CN108769523 B CN 108769523B CN 201810579249 A CN201810579249 A CN 201810579249A CN 108769523 B CN108769523 B CN 108769523B
- Authority
- CN
- China
- Prior art keywords
- frequency components
- images
- image
- spectrogram
- spectrograms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The application relates to an image processing method and device, an electronic device and a computer readable storage medium. The method comprises the following steps: acquiring a plurality of images with consistent shooting contents; respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image; obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms. The image processing method can obtain a plurality of images without moire.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a computer-readable storage medium.
Background
When an object is photographed by an electronic device, if the texture of a dense grain exists in the photographed object, a photographed image often has a color grain like a water wave, which is a moire grain. For example, when a display screen of a display screen is photographed by an electronic device, moire fringes often appear in an image photographed by the electronic device. The existence of moire in the photographed image may seriously affect the photographing effect.
Disclosure of Invention
The embodiment of the application provides an image processing method and device, electronic equipment and a computer readable storage medium, which can eliminate moire fringes in an image.
An image processing method comprising:
acquiring a plurality of images with consistent shooting contents;
respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
An image processing apparatus comprising:
the acquisition module is used for acquiring a plurality of images with consistent shooting contents;
the extraction module is used for respectively extracting the frequency components of the images; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
the processing module is used for obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the image processing method as described above.
A computer-readable storage medium, on which a computer program is stored, which computer program is executed by a processor for performing the steps of the image processing method as described above.
The image processing method and device, the electronic device and the computer-readable storage medium in the embodiment of the application acquire a plurality of images with consistent shooting contents and respectively extract frequency components of the plurality of images. The moire in each image is generated by different frequency components in a plurality of images, and the frequency components generating the moire in each image can be removed by keeping the same frequency components in each image and removing different frequency components, so that a plurality of images with the moire eliminated are obtained.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of an electronic device in one embodiment;
FIG. 2 is a flow diagram of a method of image processing in one embodiment;
FIG. 3 is a flow chart of an image processing method in another embodiment;
FIG. 4 is a block diagram showing the configuration of an image processing apparatus according to an embodiment;
FIG. 5 is a schematic diagram of an image processing circuit in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The image processing method in the embodiment of the present application is applied to the electronic device shown in fig. 1. The electronic device shown in fig. 1 includes a processor, a memory, and a network interface connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory is used for storing data, programs and the like, and at least one computer program is stored on the memory, and can be executed by the processor to realize the application processing method suitable for the electronic device provided in the embodiment of the application. The Memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random-Access-Memory (RAM). For example, in one embodiment, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement an image processing method provided in the following embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The network interface may be an ethernet card or a wireless network card, etc. for communicating with an external electronic device. The electronic device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device, etc.
FIG. 2 is a flow diagram of a method of image processing in one embodiment. The image processing method in this embodiment is described by taking the electronic device operating in fig. 1 as an example. As shown in fig. 2, the image processing method includes steps 202 to 206.
Specifically, when the electronic device is used for shooting an object with a microgroove texture, the electronic device can shoot consistent shooting contents for multiple times by setting a continuous shooting function of the electronic device, and acquire multiple images with consistent shooting contents.
For step 204, the frequency components of each image are indicators that characterize how strongly the intensity of the gray scale changes in each image. Such as: the large-area desert is an area with slow gray level change in the image, and the frequency component values in the corresponding image are lower; and for the edge area with violent surface attribute transformation, the image is an area with violent gray scale change, and each frequency component value in the corresponding image is higher.
For the above steps, the electronic device may extract the frequency components of the respective images for comparison. If the images are images without moire, the frequency components corresponding to the images can be completely the same; if there is a moire-containing image, the moire in the image is generated by different frequency components in the plurality of images. Moire fringes in each image can be eliminated by removing different frequency components in each image, for example, image 1, image 2 and image 3 respectively contain four frequency components, the frequency component of image 1 is ABCDEF, and the frequency component of image 2 is ABCDE1F, the frequency component of the image 3 is ABCDEF1Then the same frequency components ABCD in image 1, image 2 and image 3 may be retained and the remaining different frequency components removed. The removal of the different frequency components simultaneously removes moir é patterns created by these different frequency components.
In the above embodiment, a plurality of images having the same shooting content are acquired, and the frequency components of the plurality of images are extracted respectively. The moire fringes in each image are generated by different frequency components in a plurality of images, and the frequency components of the moire fringes in each image can be removed by keeping the same frequency components in each image and removing different frequency components, so that a plurality of images with the moire fringes eliminated are obtained, and the image quality is improved.
In one embodiment, for step 206, the multiple images with moire removed can be obtained by: comparing the frequency components of the first spectrograms; obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram; and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
In the above embodiment, the first spectrogram may be obtained through Fast Fourier Transform (FFT), where the FFT is obtained by improving the discrete Fourier transform algorithm according to the characteristics of odd, even, imaginary, and real of the discrete Fourier transform. A two-dimensional spectrogram can be obtained by fast fourier transform, in which the horizontal axis represents frequency, the vertical axis represents phase, and the luminance value of each point represents the amplitude of frequency. If the brightness values of the points at the same position in each spectrogram are the same, the frequency components corresponding to the points are the same.
In the above embodiment, a plurality of images having the same shooting content are acquired, and the frequency components of the plurality of images are extracted respectively. The frequency components of the moire fringes generated in each image can be removed by keeping the same frequency components in each image and removing different frequency components, so that a plurality of images with the moire fringes eliminated are obtained.
In one embodiment, when removing moir é in multiple images, the following steps may also be performed: reserving the same frequency components in each first spectrogram; and extracting different frequency components in each first spectrogram, and removing the different frequency components with the frequency less than or equal to a specified threshold value in all the first spectrograms to obtain a plurality of second spectrograms by retaining the different frequency components with the frequency greater than the specified threshold value in all the first spectrograms.
In the above embodiment, whether to retain the different frequency components may be determined by counting the number of occurrences of the different frequency components in all of the first spectrograms. Whether different frequency components are reserved can also be determined by counting the proportion of the first spectrogram containing the different frequency components in all the first spectrograms; for example, when the proportion of the first spectrogram containing the different frequency components in all the first spectrograms exceeds a specified threshold, the different frequency components in the first spectrogram are retained, otherwise, the different frequency components in the first spectrogram are removed. The specified threshold value can be adjusted correspondingly according to different application scenarios.
For example, assuming that the threshold is 50%, the first spectrogram 1, the first spectrogram 2 and the first spectrogram 3 respectively include four frequency components, the frequency component of the first spectrogram 1 is ABCDEF, and the frequency component of the first spectrogram 2 is ABCDE1F, the frequency component of the first spectrogram 3 is ABCDEF1The same frequency components ABCD in the first spectrogram 1, 2 and 3 may be retained first, and then the different frequency components (E, E) in all first spectrograms may be retained1F and F1) And (6) carrying out statistics. The proportion of the first spectrogram containing the frequency component E in all the first spectrograms is about 67%, the preset threshold value is exceeded by 50%, and the frequency component E in the first spectrogram 1 and the first spectrogram 3 is reserved; all the first spectrograms contain a frequency component E1Is about 33% and does not exceed a predetermined threshold of 50%, removing the frequency component E in 2 of the first spectrogram1(ii) a The proportion of the first spectrogram containing the frequency component F in all the first spectrograms is about 67%, the preset threshold value is exceeded by 50%, and the frequency component F in the first spectrogram 1 and the first spectrogram 3 is reserved; all the first spectrograms contain a frequency component F1Is about 33% and does not exceed a predetermined threshold of 50%, removing the frequency component F in 2 of the first spectrogram1. The frequency component of the first spectrogram 1 is ABCDEF, the frequency component of the first spectrogram 2 is ABCDF, and the frequency component of the first spectrogram 3 is ABCDE.
In the above embodiment, by counting different frequency components in all the first spectrograms, when the proportion of the first spectrograms containing the different frequency components in all the first spectrograms exceeds a specified threshold, different frequency components whose occurrence times in all the first spectrograms are greater than the specified threshold are retained, otherwise, the different frequency components in the first spectrograms are removed, so that the frequency components which do not generate moire patterns are prevented from being removed when the different frequency components in the first spectrograms are removed, and the definition of the image can be better guaranteed.
In one embodiment, for step 202, multiple images with consistent shooting content may be acquired by: acquiring the time interval of two adjacent shots; if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent; and acquiring a plurality of images with consistent shooting contents.
When the camera is used for shooting, a plurality of images with consistent shooting contents can be obtained through high dynamic range shooting. High-Dynamic Range (HDR) photographing is to take multiple photographs of the same photographing content to obtain multiple photographs, and then sequentially increase the exposure time of the photographs. After Moire of a plurality of images obtained by high dynamic range shooting is eliminated, the images can be synthesized by using the optimal image details of different images corresponding to different exposure times, and the visual effect in a real environment can be better reflected.
Specifically, the preset interval threshold may be 100ms, and in the high dynamic range shooting process, if the time interval between two adjacent shots is less than 100ms, it may be determined that the shot contents corresponding to the two adjacent shots are consistent. The number of acquired images having matching shooting contents may be 3 or more.
Whether the shot contents shot in two adjacent times are consistent or not is judged through the time interval of two adjacent times of shooting, so that the electronic equipment can be ensured to further process a plurality of images with consistent shot contents, the same frequency components in the plurality of images with consistent shot contents are reserved, different frequency components are removed, the frequency components generating moire in each image can be removed, and then the plurality of images with moire eliminated are obtained.
In one embodiment, each image may be down-sampled before step 204.
In the above embodiment, before performing frequency domain transformation on each image to obtain each first spectrogram, each image may be down-sampled. Meanwhile, upsampling can be carried out after frequency domain inverse transformation, and a plurality of images with Moire fringes eliminated are obtained. The image data needs to be sampled before frequency domain transformation, the down-sampling is to reduce the sampling rate of the image data sampling process, the size of the image data to be processed can be reduced through the down-sampling, and the transformation efficiency of the frequency domain transformation can be greatly improved by carrying out the frequency domain transformation on the down-sampled image. The up-sampling is to increase the sampling rate of the image by interpolation, and the up-sampling of the image data after the frequency domain transformation can restore the size of the image data by interpolation, thereby avoiding the over-low resolution of the image caused by down-sampling.
In the above embodiment, before the frequency domain transformation is performed on the multiple images to obtain the multiple spectrograms, the multiple images are down-sampled respectively, and then the frequency domain inverse transformation is performed on the multiple images to obtain the multiple images without moire.
In one embodiment, before step 204, frequency components of each first spectrogram having frequency amplitudes greater than a preset amplitude threshold may be rejected.
The frequency component generated by the environmental noise is generally a high-frequency component, the frequency component of the image moire is also generally a high-frequency component, and the high-frequency component generated by the environmental noise and the high-frequency component generated by the moire can be filtered in advance through a preset amplitude threshold. The preset amplitude threshold may be determined according to the highest frequency component in the spectrogram, for example, the preset amplitude threshold may be 80% of the highest frequency component in the spectrogram of the frequency domain, i.e., information that the frequency exceeds 80% of the highest frequency component is filtered.
When one or more high-frequency components generating moire fringes exist in a plurality of first spectrograms at the same time and the high-frequency components are higher than a preset amplitude threshold, the high-frequency components are preferentially filtered by setting the amplitude threshold, more high-frequency components generating moire fringes can be removed, the probability of moire occurrence of the finally obtained image is lower, more high-frequency components generated by environmental noise can also be removed, and the interference of the image by the environmental noise is smaller.
In one embodiment, after obtaining the processed multiple images, the electronic device may further perform the following steps: respectively extracting the optimal exposure areas in the processed multiple images; and combining the processed multiple images into one image according to each optimal exposure area and outputting the image.
For the above embodiment, the optimal pixel value of each pixel point in the finally output image can be obtained according to the set Camera Response Curve (Camera Response Curve). The camera response curve can reflect the mapping relation between the image data corresponding to the shot image and the actual Scene illumination (Scene radiation), and the actual Scene illumination can be determined according to the response curve. The optimal pixel value of each pixel point of the image can be obtained according to the actual scene illumination. After the optimal pixel value of each pixel point is determined, the pixel value of each pixel point in each image with different exposure degrees can be compared with the optimal pixel value, and the area where the pixel point with the pixel value difference lower than a preset pixel value difference threshold value is located can be set as the optimal exposure area.
In the embodiment, the electronic device extracts the optimal exposure areas in each image without moire fringes, and combines a plurality of images into one image according to each optimal exposure area for output, so that the image details of each image area can be better reflected, and the visual effect of the finally output image is improved.
In one embodiment, an image processing method is provided, as shown in fig. 3, comprising the steps of:
And step 304, acquiring a plurality of images with time intervals meeting preset conditions.
In step 306, the frequency components of each image are extracted. The frequency domain transformation can be respectively carried out on the plurality of images to obtain a plurality of first spectrograms, and the frequency components in the first spectrograms are obtained.
And 308, eliminating frequency components of which the frequency amplitude is greater than a preset amplitude threshold value in each first spectrogram.
And 310, eliminating the frequency components generating the moire fringes to obtain a plurality of images without the moire fringes. Comparing the frequency components of each first spectrogram, reserving the same frequency components in each first spectrogram, extracting different frequency components in each first spectrogram, reserving different frequency components of which the occurrence frequency in all first spectrograms is greater than a specified threshold value, removing different frequency components of which the occurrence frequency in all first spectrograms is less than or equal to the specified threshold value to obtain a plurality of second spectrograms, and performing frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
In the embodiment, a plurality of images shot in a high dynamic range are obtained, the frequency components of the images are respectively extracted, the frequency components of which the frequency amplitude exceeds the amplitude threshold are removed, more high-frequency components which generate moire fringes can be removed, the probability of moire occurrence of the finally obtained image is lower, more high-frequency components generated by environmental noise can be removed, and the interference of the image by the environmental noise is smaller. The frequency components of the moire fringes generated in each image can be removed by keeping the same frequency components in each image and removing different frequency components, so that a plurality of images with the moire fringes eliminated are obtained. The optimal exposure areas in the images with the moire fringes eliminated are extracted, and then a plurality of images are combined into one image according to the optimal exposure areas to be output, so that the image details of each image area can be better reflected, and the visual effect of the finally output image is improved.
It should be understood that although the steps in the flowcharts of fig. 2 and 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 and 3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
Fig. 4 is a block diagram of an image processing apparatus according to an embodiment. As shown in fig. 4, the image processing apparatus of the present embodiment includes:
an obtaining module 402, configured to obtain multiple images with consistent shooting contents;
an extracting module 404, configured to extract frequency components of each image respectively; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
the processing module 406 is configured to obtain a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
The image processing apparatus acquires a plurality of images having identical contents, and extracts frequency components of the plurality of images. The moire in each image is generated by different frequency components in a plurality of images, and the frequency components generating the moire in each image can be removed by keeping the same frequency components in each image and removing different frequency components, so that a plurality of images with the moire eliminated are obtained.
In one embodiment, the processing module 406 is further configured to compare frequency components of the respective first spectrogram; obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram; and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
In one embodiment, the processing module 406 is further configured to reserve the same frequency components in each first spectrogram; and extracting different frequency components in each first spectrogram, and removing the different frequency components with the frequency less than or equal to a specified threshold value in all the first spectrograms to obtain a plurality of second spectrograms by retaining the different frequency components with the frequency greater than the specified threshold value in all the first spectrograms.
In one embodiment, the obtaining module 402 is further configured to obtain a time interval between two adjacent shots; if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent; and acquiring a plurality of images with consistent shooting contents.
In one embodiment, the image processing apparatus further includes a down-sampling module, and the down-sampling module is configured to down-sample each image.
In an embodiment, the image processing apparatus further includes a rejecting module, where the rejecting module is configured to reject frequency components in each first spectrogram, whose frequency amplitudes are greater than a preset amplitude threshold.
In the image processing apparatus of an embodiment, the image processing apparatus further includes an exposure area extraction module and an image synthesis module, wherein the exposure area extraction module is configured to respectively extract optimal exposure areas in the processed multiple images; the image synthesis module is used for synthesizing the processed multiple images into one image according to each optimal exposure area and outputting the image.
The division of each module in the image processing apparatus is only for illustration, and in other embodiments, the image moir é removal apparatus may be divided into different modules as needed to complete all or part of the functions of the image moir é removal apparatus.
For specific limitations of the image moire removing device, reference may be made to the above limitations of the image processing method, and details thereof are not repeated here. The modules in the image moire elimination device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
The implementation of each module in the image moir é elimination apparatus provided in the embodiments of the present application may be in the form of a computer program. The computer program may be run on a terminal or a server. The program modules constituted by the computer program may be stored on the memory of the terminal or the server. Which when executed by a processor, performs the steps of the method described in the embodiments of the present application.
The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of:
acquiring a plurality of images with consistent shooting contents;
respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: comparing the frequency components of the first spectrograms; obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram; and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: reserving the same frequency components in each first spectrogram; and extracting different frequency components in each first spectrogram, and removing the different frequency components with the frequency less than or equal to a specified threshold value in all the first spectrograms to obtain a plurality of second spectrograms by retaining the different frequency components with the frequency greater than the specified threshold value in all the first spectrograms.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: acquiring the time interval of two adjacent shots; if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent; and acquiring a plurality of images with consistent shooting contents.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: and respectively carrying out down-sampling processing on each image.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: and eliminating frequency components of which the frequency amplitude is greater than a preset amplitude threshold value in each first frequency spectrogram.
In one embodiment, the computer executable instructions, when executed by the one or more processors, further cause the processors to perform the steps of: respectively extracting the optimal exposure areas in the processed multiple images; and combining the processed multiple images into one image according to each optimal exposure area and outputting the image.
A computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of:
acquiring a plurality of images with consistent shooting contents;
respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
The embodiment of the application also provides the electronic equipment. The electronic device includes therein an Image Processing circuit, which may be implemented using hardware and/or software components, and may include various Processing units defining an ISP (Image Signal Processing) pipeline. FIG. 5 is a schematic diagram of an image processing circuit in one embodiment. As shown in fig. 5, for convenience of explanation, only aspects of the image processing technology related to the embodiments of the present application are shown.
As shown in fig. 5, the image processing circuit includes an ISP processor 540 and control logic 550. The image data captured by the imaging device 510 is first processed by the ISP processor 540, and the ISP processor 540 analyzes the image data to capture image statistics that may be used to determine and/or image one or more control parameters of the imaging device 510. The imaging device 510 may include a camera having one or more lenses 512 and an image sensor 514. Image sensor 514 may include an array of color filters (e.g., Bayer filters), and image sensor 514 may acquire light intensity and wavelength information captured with each imaging pixel of image sensor 514 and provide a set of raw image data that may be processed by ISP processor 540. The sensor 520 (e.g., gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 540 based on the type of sensor 520 interface. The sensor 520 interface may utilize an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.
In addition, the image sensor 514 may also send raw image data to the sensor 520, the sensor 520 may provide the raw image data to the ISP processor 540 based on the sensor 520 interface type, or the sensor 520 may store the raw image data in the image memory 530.
The ISP processor 540 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 540 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.
Upon receiving raw image data from image sensor 514 interface or from sensor 520 interface or from image memory 530, ISP processor 540 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 530 for additional processing before being displayed. ISP processor 540 receives the processed data from image memory 530 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed by ISP processor 540 may be output to display 570 for viewing by a user and/or further processed by a Graphics Processing Unit (GPU). Further, the output of ISP processor 540 may also be sent to image memory 530, and display 570 may read image data from image memory 530. In one embodiment, image memory 530 may be configured to implement one or more frame buffers. In addition, the output of the ISP processor 540 may be transmitted to an encoder/decoder 560 to encode/decode image data. The encoded image data may be saved and decompressed before being displayed on the display 570 device. The encoder/decoder 560 may be implemented by a CPU or GPU or coprocessor.
The statistics determined by ISP processor 540 may be sent to control logic 550 unit. For example, the statistical data may include image sensor 514 statistical information such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 512 shading correction, and the like. Control logic 550 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters of imaging device 510 and ISP processor 540 based on the received statistical data. For example, the control parameters of imaging device 510 may include sensor 520 control parameters (e.g., gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 512 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 512 shading correction parameters.
The following steps are used for realizing the image processing method by using the image processing technology in the figure 5:
acquiring a plurality of images with consistent shooting contents;
respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency component is a frequency component with the same coordinate position and the same frequency amplitude between the first spectrograms.
In one embodiment, the following steps may also be implemented: comparing the frequency components of the first spectrograms; obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram; and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
In one embodiment, the following steps may also be implemented: reserving the same frequency components in each first spectrogram; and extracting different frequency components in each first spectrogram, and removing the different frequency components with the frequency less than or equal to a specified threshold value in all the first spectrograms to obtain a plurality of second spectrograms by retaining the different frequency components with the frequency greater than the specified threshold value in all the first spectrograms.
In one embodiment, the following steps may also be implemented: acquiring the time interval of two adjacent shots; if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent; and acquiring a plurality of images with consistent shooting contents.
In one embodiment, the following steps may also be implemented: and respectively carrying out down-sampling processing on each image.
In one embodiment, the following steps may also be implemented: and eliminating frequency components of which the frequency amplitude is greater than a preset amplitude threshold value in each first frequency spectrogram.
In one embodiment, the following steps may also be implemented: respectively extracting the optimal exposure areas in the processed multiple images; and combining the processed multiple images into one image according to each optimal exposure area and outputting the image.
Any reference to memory, storage, database, or other medium used herein may include non-volatile and/or volatile memory. Suitable non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An image processing method, comprising:
acquiring a plurality of images with consistent shooting contents;
respectively extracting frequency components of each image; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency components are frequency components with the same coordinate positions and the same frequency amplitudes among the first frequency spectrograms;
the acquiring of the plurality of images with consistent shooting contents comprises:
acquiring the time interval of two adjacent shots;
if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent;
and acquiring a plurality of images with consistent shooting contents.
2. The method according to claim 1, wherein obtaining the processed images by obtaining the same frequency component among the images comprises:
comparing the frequency components of each of the first spectrogram;
obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram;
and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
3. The image processing method of claim 2, wherein the obtaining a plurality of second spectrograms by preserving same frequency components in each of the first spectrograms and removing different frequency components in each of the first spectrograms comprises:
reserving the same frequency components in each first spectrogram;
and extracting different frequency components in each first spectrogram, and removing the different frequency components with the frequency less than or equal to a specified threshold value in all the first spectrograms to obtain a plurality of second spectrograms by retaining the different frequency components with the frequency greater than the specified threshold value in all the first spectrograms.
4. The image processing method according to any one of claims 1 to 3, wherein before the extracting the frequency components of the respective images, respectively, comprising:
and respectively carrying out down-sampling processing on each image.
5. The image processing method according to any one of claims 1 to 3, wherein after the extracting the frequency components of the respective images, respectively, comprises:
and eliminating frequency components of which the frequency amplitude is greater than a preset amplitude threshold value in each first frequency spectrogram.
6. The image processing method according to any one of claims 1 to 3, characterized in that the method further comprises:
respectively extracting the optimal exposure areas in the processed multiple images;
and combining the processed multiple images into one image according to each optimal exposure area and outputting the image.
7. An image processing apparatus characterized by comprising:
the acquisition module is used for acquiring a plurality of images with consistent shooting contents;
the extraction module is used for respectively extracting the frequency components of the images; the frequency components are frequency components in a first spectrogram obtained by frequency domain transformation of each image;
the processing module is used for obtaining a plurality of processed images by obtaining the same frequency components among the plurality of images; the same frequency components are frequency components with the same coordinate positions and the same frequency amplitudes among the first frequency spectrograms;
the acquisition module is also used for acquiring the time interval between two adjacent shots; if the time interval is smaller than a preset interval threshold, judging that the corresponding shooting contents of the two adjacent shot images are consistent; and acquiring a plurality of images with consistent shooting contents.
8. The image processing apparatus according to claim 7, wherein the processing module is further configured to compare frequency components of the first spectrograms; obtaining a plurality of second spectrograms by reserving the same frequency components in each first spectrogram and removing different frequency components in each first spectrogram; and carrying out frequency domain inverse transformation on each second spectrogram to obtain a plurality of processed images.
9. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the image processing method according to any one of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the image processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810579249.0A CN108769523B (en) | 2018-06-07 | 2018-06-07 | Image processing method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810579249.0A CN108769523B (en) | 2018-06-07 | 2018-06-07 | Image processing method and device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108769523A CN108769523A (en) | 2018-11-06 |
| CN108769523B true CN108769523B (en) | 2020-06-30 |
Family
ID=63999245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810579249.0A Active CN108769523B (en) | 2018-06-07 | 2018-06-07 | Image processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108769523B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798380A (en) * | 2019-04-09 | 2020-10-20 | Tcl集团股份有限公司 | Image processing method, image processing device and terminal equipment |
| CN110300325B (en) * | 2019-08-06 | 2022-04-15 | 北京字节跳动网络技术有限公司 | Video processing method and device, electronic equipment and computer readable storage medium |
| CN110661969B (en) * | 2019-08-19 | 2021-09-10 | 厦门快商通科技股份有限公司 | Image acquisition method and device for display screen, electronic equipment and medium |
| CN111062922B (en) * | 2019-12-14 | 2024-02-20 | 创新奇智(北京)科技有限公司 | Method and system for distinguishing flip image and electronic equipment |
| CN111311511B (en) * | 2020-01-22 | 2023-08-29 | 凌云光技术股份有限公司 | Method and device for removing moire patterns |
| CN112351205B (en) * | 2020-10-28 | 2022-08-16 | 维沃移动通信有限公司 | Shooting control method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1894169A2 (en) * | 2005-06-23 | 2008-03-05 | SDU Identification B.V. | Identification card |
| CN104010174A (en) * | 2009-10-13 | 2014-08-27 | 佳能株式会社 | Image processing device and image processing method |
| CN107424123A (en) * | 2017-03-29 | 2017-12-01 | 北京粉笔未来科技有限公司 | A kind of moire fringes minimizing technology and device |
| CN107749953A (en) * | 2017-11-17 | 2018-03-02 | 嘉兴学院 | A kind of double camera image based on binocular lens tube increases frequency device and control method |
-
2018
- 2018-06-07 CN CN201810579249.0A patent/CN108769523B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1894169A2 (en) * | 2005-06-23 | 2008-03-05 | SDU Identification B.V. | Identification card |
| CN104010174A (en) * | 2009-10-13 | 2014-08-27 | 佳能株式会社 | Image processing device and image processing method |
| CN107424123A (en) * | 2017-03-29 | 2017-12-01 | 北京粉笔未来科技有限公司 | A kind of moire fringes minimizing technology and device |
| CN107749953A (en) * | 2017-11-17 | 2018-03-02 | 嘉兴学院 | A kind of double camera image based on binocular lens tube increases frequency device and control method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108769523A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108769523B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| US11430103B2 (en) | Method for image processing, non-transitory computer readable storage medium, and electronic device | |
| CN109089047B (en) | Method and device for controlling focusing, storage medium and electronic equipment | |
| CN109767467B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN108012080B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| US11431915B2 (en) | Image acquisition method, electronic device, and non-transitory computer readable storage medium | |
| CN110473185B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110290323B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| EP3849170B1 (en) | Image processing method, electronic device, and computer-readable storage medium | |
| CN113766125B (en) | Focusing method and device, electronic equipment and computer readable storage medium | |
| US8860816B2 (en) | Scene enhancements in off-center peripheral regions for nonlinear lens geometries | |
| CN108198152B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN107395991B (en) | Image synthesis method, image synthesis device, computer-readable storage medium and computer equipment | |
| CN114586337B (en) | Video anti-shake optimization processing method and device and electronic equipment | |
| CN110349163B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| KR20200031169A (en) | Image processing method and device | |
| CN109685853B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN108616700B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110177212B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| WO2017113917A1 (en) | Imaging method, imaging apparatus, and terminal | |
| CN109559353B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN110213498B (en) | Image generation method and device, electronic equipment and computer readable storage medium | |
| CN110866486B (en) | Subject detection method and apparatus, electronic device, and computer-readable storage medium | |
| WO2021139635A1 (en) | Method and apparatus for generating super night scene image, and electronic device and storage medium | |
| WO2020107995A1 (en) | Imaging method and apparatus, electronic device, and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |