CN115278069A - Image processing method and device, computer readable storage medium and terminal - Google Patents
Image processing method and device, computer readable storage medium and terminal Download PDFInfo
- Publication number
- CN115278069A CN115278069A CN202210868040.2A CN202210868040A CN115278069A CN 115278069 A CN115278069 A CN 115278069A CN 202210868040 A CN202210868040 A CN 202210868040A CN 115278069 A CN115278069 A CN 115278069A
- Authority
- CN
- China
- Prior art keywords
- image
- frame
- original
- photographing
- adjustment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000000694 effects Effects 0.000 claims abstract description 33
- 238000003384 imaging method Methods 0.000 claims abstract description 21
- 238000007499 fusion processing Methods 0.000 claims abstract description 14
- 230000004044 response Effects 0.000 claims abstract description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 27
- 230000004927 fusion Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000003705 background correction Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 22
- 238000004364 calculation method Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- -1 gray scale value Chemical compound 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005375 photometry Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
An image processing method and device, a computer readable storage medium and a terminal are provided, the method comprises the following steps: in response to receiving a photographing instruction, determining multiple frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity; respectively calculating image adjusting parameters of each frame of the photographed original image; based on the image adjustment parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjustment parameters are used for expressing direct parameters for adjusting display effects; and carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image. According to the scheme, the image adjustment parameters are strictly matched with the original photographing images of the frames, so that the problem of parameter applicability accuracy in a multi-exposure photographing system is solved, and the multi-exposure imaging effect is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to an image processing method and device, a computer readable storage medium and a terminal.
Background
In the field of multi-exposure imaging/photographing technology, images with different exposure intensities in the same scene are collected, image Signal Processing (ISP) is performed on original images with different exposure intensities by using Image adjustment parameters to obtain YUV (Y represents brightness or Luma, i.e. gray scale value, "U" and "V" represent Chroma or Chroma) images with multiple exposures, and then Image fusion Processing is performed to obtain final photographed images.
In the prior art, after entering a photographing mode (preview mode), a camera system may count image adjustment statistical information of each frame of original image (or called preview original image) acquired by an image sensor, such as Auto Exposure (AE) statistical information, auto White Balance (AWB) statistical information, auto Focus (AF) statistical information (hereinafter referred to as 3A statistical information), and after the statistics is completed, an interrupt may be generated, and then the 3A statistical information is sent to a 3A algorithm module to perform analysis and calculation to obtain an image adjustment parameter (hereinafter referred to as 3A parameter); after receiving a photographing instruction, directly multiplexing the 3A parameters obtained in the preview process to perform ISP processing on a plurality of frames of original images (or called photographing original images) with different exposure intensities to obtain YUV images with multiple exposures. However, the above-mentioned prior art has the following disadvantages:
on one hand, in order to ensure the preview effect, the 3A statistical information of each frame of original image is filtered after being counted in the preview mode, and only the 3A statistical information within the normal exposure intensity is selected, for example, the 3A statistical information of the strong exposure image and the weak exposure image is removed). Therefore, in the photographing mode, the 3A parameters calculated by the 3A statistical information in the preview mode are directly multiplexed to process the photographed original images with different exposure intensities (which may include strong exposure, normal exposure, and weak exposure) in each frame, which may cause poor processing effect due to inaccurate parameter usage and affect the final imaging effect (e.g., color cast may occur). On the other hand, due to the influence of delay factors such as 3A parameter calculation and transmission, the 3A parameter used in performing ISP processing often has a certain delay, and the 3A parameter used in processing the current frame may be a parameter calculated based on the 3A statistical information of the previous frame or two frames. Therefore, if the 3A parameter processing of the simple multiplexing preview mode takes a photograph of an original image, it is affected by the delay problem, and the imaging effect is also reduced due to inaccurate use of the parameter.
Disclosure of Invention
The technical problem solved by the embodiment of the invention is that in the prior art, the image adjustment parameters calculated in the preview process are directly reused in the photographing process, and the image signal processing is carried out on the photographed original images with different exposure intensities of each frame, so that the imaging effect is reduced due to inaccurate parameter use.
In order to achieve the above object, an embodiment of the present invention provides an image processing method, including: in response to receiving a photographing instruction, determining multiple frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity; respectively calculating image adjusting parameters of each frame of the photographed original image; based on the image adjustment parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjustment parameters are used for expressing direct parameters for adjusting display effects; and carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
Optionally, the respectively calculating the image adjustment parameters of each frame of the photographed original image includes: determining image adjustment statistical information of each frame of the photographed original image, wherein the image adjustment statistical information is used for expressing indirect parameters for adjusting display effects; for each frame of the photographed original image, determining the image adjustment parameters based on the image adjustment statistical information.
Optionally, the determining the image adjustment statistical information of each frame of the photographed original image includes: acquiring preview adjustment statistical information of a preview original image to which the same frame number belongs as the image adjustment statistical information based on the frame number of each frame of the photographed original image; or acquiring preview adjustment statistical information of a preview original image to which the same acquisition time stamp belongs as the image adjustment statistical information based on the acquisition time stamp of each frame of the photographed original image; the preview original image and the preview adjustment statistical information are determined after receiving an instruction of entering a photographing mode.
Optionally, the image adjustment statistical information is selected from one or more of the following: auto exposure AE statistics, auto white balance AWB statistics, auto focus AF statistics.
Optionally, the image adjustment statistical information includes AE statistical information and AWB statistical information, and the image adjustment parameters include an automatic exposure AE adjustment parameter, an automatic white balance AWB adjustment parameter, and an automatic lens shading correction ALSC parameter; the determining the image adjustment parameters based on the image adjustment statistical information for each frame of the photographed original image comprises: determining an AE adjusting parameter based on AE statistical information of the frame photographing original image by adopting an automatic exposure algorithm; determining an AWB adjusting parameter based on the AWB statistical information of the original photographed image of the frame and the AE adjusting parameter by adopting an automatic white balance algorithm; determining the ALSC parameter based on the AE adjustment parameter and the AWB adjustment parameter.
Optionally, the determining the multiple frames of original photographed images from the time to which the current photographing instruction belongs includes: determining a plurality of frames of the original photographing images within a preset time length from the moment of the current photographing instruction; or, determining the original photographing image with the preset frame number from the time of the current photographing instruction.
Optionally, the image fusion processing on the obtained multi-frame processed image includes: and performing image fusion processing on the obtained multi-frame processed image by adopting a multi-exposure image fusion algorithm.
Optionally, the processed image is selected from: YUV images, RGB images, and YIQ images.
An embodiment of the present invention further provides an image processing apparatus, including: the photographing original image determining module is used for responding to the received photographing instruction and determining a plurality of frames of photographing original images from the moment of the current photographing instruction, wherein each frame of photographing original image has respective exposure intensity; the image adjustment parameter determining module is used for respectively calculating the image adjustment parameters of each frame of the photographed original image; the image signal processing module is used for carrying out image signal processing on the photographed original image of each frame based on the image adjusting parameters to obtain a processed image, wherein the image adjusting parameters are used for expressing direct parameters for adjusting the display effect; and the fusion imaging module is used for carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the image processing method.
The embodiment of the present invention further provides a terminal, which includes a memory and a processor, where the memory stores a computer program capable of running on the processor, and the processor executes the steps of the image processing method when running the computer program.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, in response to receiving a photographing instruction, determining a plurality of frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity; respectively calculating image adjusting parameters of each frame of the photographed original image; and based on the image adjusting parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjusting parameters are used for expressing direct parameters for adjusting the display effect. Compared with the image adjustment parameters obtained by calculating the image adjustment statistical information which is counted under the preview mode and is directly multiplexed in the photographing process in the prior art, ISP processing is carried out on the photographed original images of different exposure intensities of each frame, and the imaging effect is reduced due to inaccurate parameter use. According to the embodiment of the invention, the image adjustment parameters of each frame of photographed original image are respectively calculated after the photographing instruction is received through frame-level control over the image adjustment parameters, instead of directly multiplexing the image adjustment parameters determined in the preview process. Therefore, the image adjustment parameters are strictly matched with the original photographed image of each frame, the problem of parameter applicability accuracy in a multi-exposure photographing system is solved, and the multi-exposure imaging effect is improved.
Further, the calculating the image adjustment parameters of the photographed original images for each frame respectively includes: determining image adjustment statistical information (such as 3A statistical information) of each frame of the photographed original image, wherein the image adjustment statistical information is used for expressing indirect parameters for adjusting display effect; for each frame of the photographed original image, determining the image adjustment parameters based on the image adjustment statistical information. The 3A statistical information can directly multiplex the 3A statistical information of the preview original image which is determined in the preview process and has the same frame number or the same collection timestamp as the photographed original image, and the 3A statistical information of each frame of photographed original image does not need to be recalculated, so that the calculation efficiency of the 3A parameter can be improved, the imaging efficiency is improved, and the operation cost is reduced.
Drawings
FIG. 1 is a flow chart of an image processing method in an embodiment of the invention;
FIG. 2 is a flowchart of one embodiment of step S12 of FIG. 1;
fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
Detailed Description
As mentioned above, in the field of multi-exposure imaging/photographing technology, in the process of performing Image Signal Processing (ISP) on original images with different exposure intensities, whether the Image adjustment parameters are used accurately or not directly affects the final photographed Image effect.
In the prior art, after entering a photographing mode (preview mode), a camera system may count image adjustment statistical information of each frame of original image (or referred to as a preview original image) acquired by an image sensor, for example, auto Exposure (AE) statistical information, auto White Balance (AWB) statistical information, auto Focus (AF) statistical information (hereinafter referred to as 3A statistical information), and after the statistics is completed, an interrupt may be generated, and then the 3A statistical information is sent to a 3A algorithm module for analysis and calculation to obtain image adjustment parameters (hereinafter referred to as 3A parameters); after receiving a photographing instruction, directly multiplexing the 3A parameters obtained in the preview process to perform ISP processing on a plurality of frames of original images (or called photographing original images) with different exposure intensities to obtain YUV images with multiple exposures.
The inventor of the present invention has found through research that the above prior art has the following disadvantages: on one hand, in order to ensure the preview effect, the 3A statistical information of each frame of original image is filtered after being counted in the preview mode, and only the 3A statistical information within the normal exposure intensity (the 3A statistical information of the strong exposure image and the weak exposure image is not included) is selected. Therefore, in the photographing mode, the 3A parameters calculated by the 3A statistical information in the preview mode are directly multiplexed to process the original photographed image with different exposure intensities (which may include strong exposure, normal exposure, and weak exposure) of each frame, which may cause poor processing effect due to inaccurate parameter usage and affect the final imaging effect (e.g., color cast may occur). On the other hand, due to the influence of delay factors such as 3A parameter calculation and transmission, the 3A parameter used in performing ISP processing often has a certain delay, and the 3A parameter used in processing the current frame may be a parameter calculated based on the 3A statistical information of the previous frame or the previous two frames. Therefore, if the 3A parameter process of the simple multiplexing preview mode takes a photograph of an original image, it is affected by the delay problem and the imaging effect is also reduced due to inaccurate use of the parameters.
To solve the foregoing technical problem, an embodiment of the present invention provides an image processing method, which specifically includes: in response to receiving a photographing instruction, determining multiple frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity; respectively calculating image adjustment parameters of each frame of the photographed original image; and based on the image adjusting parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjusting parameters are used for expressing direct parameters for adjusting the display effect.
In view of the above, the embodiment of the present invention performs frame-level control on the image adjustment parameters, and after receiving the photographing instruction, calculates the image adjustment parameters of each frame of the original photographed image, instead of directly multiplexing the image adjustment parameters determined in the preview process, so as to ensure that the image adjustment parameters are strictly matched with the current photographing frame, which is helpful for solving the problem of accuracy of parameter applicability in the multi-exposure photographing system and improving the multi-exposure imaging effect.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Referring to fig. 1, fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention. The method may include steps S11 to S14:
step S11: in response to receiving a photographing instruction, determining multiple frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity;
step S12: respectively calculating image adjusting parameters of each frame of the photographed original image;
step S13: based on the image adjustment parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjustment parameters are used for expressing direct parameters for adjusting display effects;
step S14: and carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
In a specific implementation, the image processing method may be executed by a terminal, and may specifically be various existing terminals with image processing capability, such as various types of cameras, and may also be a mobile phone, a computer, a tablet computer, a wearable device (such as a smart watch), and the like, which are installed with photographing software, but is not limited thereto. Specifically, the image processing method can be applied to a multi-exposure photographing system of various types of terminals listed above, such as: high Dynamic Range (HDR) photographing, night scene photographing and other modes, and is mainly realized by relying on flow optimization of camera basic software.
It is understood that, in a specific implementation, the image processing method may be implemented by using a software program, where the software program runs in a processor integrated inside a chip or a chip module; alternatively, the method can be implemented in hardware or a combination of hardware and software.
In a specific implementation of step S11, the multiple frames of photographed RAW images may be several frames of images with different exposure intensities selected from RAW image (RAW image) data streams buffered in real time by a camera or a photographing system, and the RAW image may be unprocessed image data acquired in real time by an image sensor.
Further, the determining the plurality of frames of photographed original images from the time to which the current photographing instruction belongs includes: determining a plurality of frames of the original photographing images within a preset time length from the moment of the current photographing instruction; or, determining the original photographing image with the preset frame number from the time of the current photographing instruction.
Specifically, a plurality of frames of images with different exposure intensities selected from RAW images acquired by an image sensor within a preset time period from the moment to which the current photographing instruction belongs can be used as the multi-frame original photographing image. The exposure intensity level, the preset duration or the preset number of frames specifically included in the multiple frames of the original photographed images may be different according to different specific application scenes, for example, may be determined according to the needs of a subsequent multi-exposure fusion algorithm, and is not limited herein.
In a specific implementation of step S12, the image adjustment parameter may be used to represent a direct parameter for adjusting the display effect, and may be understood as a camera setting, an image processing setting, or a combination thereof. Among them, the image adjustment parameter as the camera setting may be a parameter related to a change in the focus, exposure time, gain, aperture, or depth of field of the camera. The image adjustment parameter as the image processing setting may be a parameter that changes a color scheme, a signal-to-noise ratio, or a contrast.
As one non-limiting example, the image adjustment parameters may include 3A parameters, wherein the 3A parameters may include: auto Focus (AF) parameters, auto Exposure (AE) parameters, and Auto White Balance (AWB) parameters, and may further include Auto Lens Shading Correction (ALSC) parameters. After the image adjustment parameters are acquired, the image adjustment parameters can be used for directly adjusting the display effect of the image and improving the photographing imaging quality.
It can be understood that, in the specific implementation, in order to ensure that the image adjustment parameters strictly correspond to the original photographed images (frame-level alignment of the image adjustment parameters) in consideration of the delay in the processes of software and hardware processing, network transmission, etc., the image adjustment parameters of each frame of the original photographed images may be calculated in an off-line manner, and in the subsequent steps, image signal processing may be performed on each frame of the original photographed images in an off-line manner based on the image adjustment parameters.
Referring to fig. 2, fig. 2 is a flowchart of an embodiment of step S12 in fig. 1. The calculating of the image adjustment parameters of the photographed original image for each frame may include steps S21 to S22, which are described below.
In step S21, image adjustment statistical information for each frame of the photographed original image is determined, the image adjustment statistical information being used to indicate an indirect parameter for adjusting a display effect.
Further, the image adjustment statistics may be selected from one or more of: automatic exposure AE statistical information, automatic white balance AWB statistical information and automatic focusing AF statistical information.
The image adjustment statistical information may be a statistical result obtained by performing mathematical statistics (for example, histogram statistics) on some information related to image signal processing in the photographed original image by the hardware analysis module. It should be noted that, as an indirect parameter for indicating adjustment of the display effect, the statistical result may be used as input data of a subsequent software computing module, and after being processed by a related algorithm, an image adjustment parameter that is specifically applicable to camera setting and image processing setting may be obtained, that is, a direct parameter for indicating adjustment of the display effect may be obtained after being processed by the related algorithm.
As a non-limiting example, the Lens (Lens) of the camera may first project the shot on the image Sensor (Sensor), and at the same time, the hardware module of the image processor (ISP) may obtain the image adjustment statistical information through photometry, distance measurement, etc., and then the software module may analyze and calculate the appropriate image adjustment statistical parameters and instruct the Lens to focus, and as the user presses the photographing key, the image Sensor (Sensor) may complete one exposure and become a picture through the image processor (ISP), and then the picture is finally displayed on the screen through the post-processing (e.g., multi-exposure fusion processing) of the application program.
Further, the determining the image adjustment statistical information of each frame of the photographed original image comprises: acquiring preview adjustment statistical information of a preview original image to which the same frame number belongs as the image adjustment statistical information based on the frame number of each frame of the photographed original image; or acquiring preview adjustment statistical information of the preview original image to which the same acquisition timestamp belongs based on the acquisition timestamp of each frame of the photographed original image, and taking the preview adjustment statistical information as the image adjustment statistical information.
Wherein, the preview original image and the preview adjustment statistical information are determined in response to receiving an instruction to enter a photographing mode (or a preview mode).
It should be noted that, for each frame of RAW image acquired by the image sensor, the RAW image can be accessed by taking a picture of the original image and previewing the original image.
In a specific implementation, in addition to the above listed methods, other methods may be used to determine the adjustment statistical information of the original photographed image in each frame, which is not limited in this embodiment of the present invention.
It is understood that, in a specific implementation, the camera or the photographing system may further include a preview process (corresponding to the preview mode) in addition to the photographing process (corresponding to the photographing mode) included in the image processing method. The preview process may be executed in an online manner, and mainly includes the following steps:
when a camera or a photographing system enters a photographing start mode/preview mode, it usually receives an original image stream (or referred to as a preview original image stream) acquired by an image sensor in real time and counts image adjustment statistical information, such as 3A statistical information, of each frame of preview original image (meanwhile, the received original image stream and the 3A statistical information thereof may also be stored in a memory for use in the photographing mode). Then the camera or the photographing system selects appropriate 3A statistical information to calculate image adjustment parameters for the ISP to process image signals to obtain a processed primary preview image (such as YUV image), and then the processed primary preview image is input into a preview algorithm module to be processed to obtain a final preview image.
It can be understood that, in the embodiment of the present invention, the 3A statistical information of the preview original image determined in the preview process is directly used, where the frame number of the preview original image is the same as the frame number of the photographed original image or the capture timestamp of the photographed original image is the same as the frame number of the photographed original image. Because the 3A statistical information of each frame of photographed original image does not need to be recalculated, the calculation efficiency of the 3A parameters can be improved, the imaging efficiency is improved, and the operation overhead is reduced.
In step S22, for each frame of the photographed original image, the image adjustment parameter is determined based on the image adjustment statistical information.
Further, the image adjustment statistical information includes AE statistical information and AWB statistical information, and the image adjustment parameters include an automatic exposure AE adjustment parameter, an automatic white balance AWB adjustment parameter, and an automatic lens shading correction ALSC parameter; the determining the image adjustment parameters based on the image adjustment statistical information for each frame of the photographed original image comprises: determining an AE adjusting parameter based on AE statistical information of the frame photographing original image by adopting an automatic exposure algorithm; determining an AWB adjusting parameter based on the AWB statistical information of the original photographed image of the frame and the AE adjusting parameter by adopting an automatic white balance algorithm; determining the ALSC parameter based on the AE adjustment parameter and the AWB adjustment parameter.
The automatic exposure algorithm and the automatic white balance algorithm both belong to the algorithm type in the 3A algorithm.
The automatic exposure algorithm can analyze and calculate the automatic exposure AE statistical information of the original photographed image, and output automatic exposure AE parameters which can be used for adjusting shutter speed, aperture value, sensitivity and the like, so that the ISP can adjust the brightness or darkness of the original photographed image to solve the problem that some areas in the image are over-exposed and other areas are under-exposed.
The automatic white balance algorithm can analyze and calculate the automatic white balance AWB statistical information of the photographed original image, and output automatic white balance AWB parameters for the ISP to correct the color temperature of the photographed original image, so that the real color of the photographed scene/object can be restored as much as possible by final imaging. For example, since the color temperature is low, the image sensor captures dusk as a yellow hue, and in an environment with sufficient sunlight, a higher blue hue is presented, and by outputting appropriate AWB parameters using the AWB algorithm, the camera can be enabled to not only capture a better image in a low-light environment, but also provide better color restoration and color cast compensation functions.
With continued reference to fig. 1, in a specific implementation of step S13, the Image Signal processing may be a process of performing Signal processing on raw Image data output by the Image sensor by using an Image Processor or an Image Signal Processor (ISP), which is a chip used in a camera or a device with a shooting function. The main functions of the image signal processing process are, for example, linear correction, noise removal, dead pixel removal, interpolation, white balance, automatic exposure control, and the like. The field details can be well restored under different optical conditions only by depending on image signal processing, and the imaging quality of the camera is determined to a great extent by the image signal processing technology.
In a specific implementation, the image signal processor ISP supports real-time processing of preview data, and may also write a RAW image output by the image sensor into the memory. And the ISP supports the independent off-line processing of the photographed stream data, the RAW image is read from the memory to perform the processing of the complete assembly line flow, namely the ISP has at least two flows, the preview flow can work in an on-line mode, and the photographing flow can work in an off-line mode. The camera software supports storing 3A statistical information of corresponding data frames in an aligned manner while storing the RAW image sequence. The 3A control software supports independent offline link processing of the photographed data frames, and frame-level accurate alignment of the 3A parameters is achieved.
Further, the multi-frame processed image obtained after ISP processing may be referred to as a multiple exposure (multiple exposure) image (generally, YUV image at multiple exposure intensity levels). In other embodiments, the image may also be an RGB image, a YIQ image, or the like at multiple exposure intensity levels, and the format of the processed image output by the ISP is not limited in the embodiments of the present invention.
In a specific implementation of step S14, a multi-exposure image fusion algorithm may be adopted to perform image fusion processing on the obtained multi-frame processed image.
The multiple exposure fusion (multiple exposure fusion) technique is a technique that two or more independent exposures are adopted in photography, and then the results are fused according to some image processing algorithm to obtain a single high dynamic range photographed image. The multi-exposure image fusion algorithm can be selected according to different specific application scenes. For example, a traditional multi-exposure image fusion algorithm such as laplacian Pyramid Laplace Pyramid Blending fusion algorithm, and a machine learning-based multi-exposure image fusion algorithm such as depth fusion DeepFuse.
In the embodiment of the invention, compared with the image adjustment parameters obtained by calculating the image adjustment statistical information which is directly multiplexed and counted in the preview mode in the photographing process in the prior art, ISP processing is carried out on the photographed original images of different exposure intensities of each frame, so that the imaging effect is reduced due to inaccurate parameter use; according to the embodiment of the invention, the image adjustment parameters of each frame of photographed original image are respectively calculated after the photographing instruction is received through frame-level control over the image adjustment parameters, instead of directly multiplexing the image adjustment parameters determined in the preview process. Therefore, the image adjustment parameters are strictly matched with the current photographing frame, the problem of accuracy in parameter applicability in a multi-exposure photographing system is solved, and the multi-exposure imaging effect is improved.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus may include:
the original photographing image determining module 31 is configured to determine, in response to receiving a photographing instruction, a plurality of frames of original photographing images from a time to which the current photographing instruction belongs, where each frame of original photographing image has respective exposure intensity;
an image adjustment parameter determining module 32, configured to calculate image adjustment parameters of each frame of the photographed original image respectively;
the image signal processing module 33 is configured to perform image signal processing on the photographed original image of each frame based on the image adjustment parameter to obtain a processed image, where the image adjustment parameter is used to indicate a direct parameter for adjusting a display effect;
and the fusion imaging module 34 is configured to perform image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
For the principle, specific implementation and beneficial effects of the image processing apparatus, reference is made to the foregoing description related to the image processing method shown in fig. 1 to fig. 2, and details are not repeated here.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the image processing method shown in fig. 1 and fig. 2. The computer-readable storage medium may include non-volatile (non-volatile) or non-transitory (non-transitory) memory, and may also include optical disks, mechanical hard disks, solid state disks, and the like.
Specifically, in the embodiment of the present invention, the processor may be a Central Processing Unit (CPU), and the processor may also be another general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will also be appreciated that the memory in the embodiments of the subject application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. Volatile memory may be Random Access Memory (RAM) which acts as external cache memory. By way of example and not limitation, many forms of Random Access Memory (RAM) are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (enhanced SDRAM), synchronous DRAM (SLDRAM), synchronous Link DRAM (SLDRAM), and direct bus RAM (DR RAM).
An embodiment of the present invention further provides a terminal, which includes a memory and a processor, where the memory stores a computer program capable of running on the processor, and the processor executes the steps of the image processing method shown in fig. 1 and fig. 2 when running the computer program. The terminal can include but is not limited to a mobile phone, a computer, a tablet computer and other terminal devices, and can also be a server, a cloud platform and the like. Alternatively, the terminal may include the image processing apparatus shown in fig. 3 described above.
It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in this document indicates that the former and latter related objects are in an "or" relationship.
The "plurality" appearing in the embodiments of the present application means two or more.
The descriptions of the first, second, etc. appearing in the embodiments of the present application are only for illustrating and differentiating the objects, and do not represent the order or the particular limitation of the number of the devices in the embodiments of the present application, and do not constitute any limitation to the embodiments of the present application.
It should be noted that the sequence numbers of the steps in this embodiment do not represent a limitation on the execution sequence of the steps.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (11)
1. An image processing method, characterized by comprising:
in response to receiving a photographing instruction, determining multiple frames of original photographing images from the moment of the current photographing instruction, wherein each frame of original photographing image has respective exposure intensity;
respectively calculating image adjustment parameters of each frame of the photographed original image;
based on the image adjustment parameters, carrying out image signal processing on the photographed original images of the frames to obtain processed images, wherein the image adjustment parameters are used for expressing direct parameters for adjusting display effects; and carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
2. The method of claim 1, wherein the calculating the image adjustment parameters of the original photographed image for each frame comprises:
determining image adjustment statistical information of each frame of the photographed original image, wherein the image adjustment statistical information is used for expressing indirect parameters for adjusting display effects;
for each frame of the photographed original image, determining the image adjustment parameters based on the image adjustment statistical information.
3. The method of claim 2, wherein determining image adjustment statistics for each frame of the photographed raw image comprises:
acquiring preview adjustment statistical information of a preview original image to which the same frame number belongs as the image adjustment statistical information based on the frame number of each frame of the photographed original image;
alternatively, the first and second electrodes may be,
acquiring preview adjustment statistical information of a preview original image to which the same acquisition timestamp belongs based on the acquisition timestamp of each frame of the photographed original image, and taking the preview adjustment statistical information as the image adjustment statistical information;
the preview original image and the preview adjustment statistical information are determined after receiving an instruction of entering a photographing mode.
4. The method of claim 2, wherein the image adjustment statistics are selected from one or more of:
automatic exposure AE statistical information, automatic white balance AWB statistical information and automatic focusing AF statistical information.
5. The method of claim 2, wherein the image adjustment statistics comprise AE statistics and AWB statistics, and the image adjustment parameters comprise automatic exposure AE adjustment parameters, automatic white balance AWB adjustment parameters, and automatic lens shading correction ALSC parameters;
the determining the image adjustment parameters based on the image adjustment statistical information for each frame of the photographed original image comprises:
determining an AE adjusting parameter based on AE statistical information of the frame photographing original image by adopting an automatic exposure algorithm;
determining AWB adjustment parameters based on AWB statistical information of the photographed original image of the frame and the AE adjustment parameters by adopting an automatic white balance algorithm;
determining the ALSC parameter based on the AE adjustment parameter and the AWB adjustment parameter.
6. The method according to claim 1, wherein the determining the plurality of frames of original photographing images from the time to which the current photographing instruction belongs comprises:
determining a plurality of frames of the original photographing images within a preset time length from the moment of the current photographing instruction; alternatively, the first and second electrodes may be,
and determining the original photographing image with the preset frame number from the moment of the current photographing instruction.
7. The method according to claim 1, wherein the image fusion processing on the obtained multi-frame processed image comprises:
and performing image fusion processing on the obtained multi-frame processed image by adopting a multi-exposure image fusion algorithm.
8. The method of claim 1 or 7, wherein the processed image is selected from the group consisting of:
YUV images, RGB images, and YIQ images.
9. An image processing apparatus characterized by comprising:
the photographing original image determining module is used for responding to the received photographing instruction and determining a plurality of frames of photographing original images from the moment of the current photographing instruction, wherein each frame of photographing original image has respective exposure intensity; the image adjustment parameter determining module is used for respectively calculating the image adjustment parameters of each frame of the photographed original image;
the image signal processing module is used for carrying out image signal processing on the photographed original image of each frame based on the image adjusting parameters to obtain a processed image, wherein the image adjusting parameters are used for expressing direct parameters for adjusting the display effect;
and the fusion imaging module is used for carrying out image fusion processing on the obtained multi-frame processed image to obtain a fused photographed image.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the image processing method according to any one of claims 1 to 8.
11. A terminal comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, wherein the processor executes the computer program to perform the steps of the image processing method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210868040.2A CN115278069A (en) | 2022-07-22 | 2022-07-22 | Image processing method and device, computer readable storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210868040.2A CN115278069A (en) | 2022-07-22 | 2022-07-22 | Image processing method and device, computer readable storage medium and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115278069A true CN115278069A (en) | 2022-11-01 |
Family
ID=83768302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210868040.2A Pending CN115278069A (en) | 2022-07-22 | 2022-07-22 | Image processing method and device, computer readable storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115278069A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115589539A (en) * | 2022-11-29 | 2023-01-10 | 荣耀终端有限公司 | Image adjusting method, device and storage medium |
| CN117156293A (en) * | 2023-01-19 | 2023-12-01 | 荣耀终端有限公司 | Photographing method and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578042A (en) * | 2015-12-18 | 2016-05-11 | 深圳市金立通信设备有限公司 | Image data transmission method and terminal |
| CN109167930A (en) * | 2018-10-11 | 2019-01-08 | Oppo广东移动通信有限公司 | Image display method, device, electronic equipment and computer readable storage medium |
| WO2019071613A1 (en) * | 2017-10-13 | 2019-04-18 | 华为技术有限公司 | Image processing method and device |
| CN113810593A (en) * | 2020-06-15 | 2021-12-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
-
2022
- 2022-07-22 CN CN202210868040.2A patent/CN115278069A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578042A (en) * | 2015-12-18 | 2016-05-11 | 深圳市金立通信设备有限公司 | Image data transmission method and terminal |
| WO2019071613A1 (en) * | 2017-10-13 | 2019-04-18 | 华为技术有限公司 | Image processing method and device |
| CN109167930A (en) * | 2018-10-11 | 2019-01-08 | Oppo广东移动通信有限公司 | Image display method, device, electronic equipment and computer readable storage medium |
| CN113810593A (en) * | 2020-06-15 | 2021-12-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115589539A (en) * | 2022-11-29 | 2023-01-10 | 荣耀终端有限公司 | Image adjusting method, device and storage medium |
| CN117156293A (en) * | 2023-01-19 | 2023-12-01 | 荣耀终端有限公司 | Photographing method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110225248B (en) | Image acquisition method and device, electronic equipment and computer readable storage medium | |
| CN109089047B (en) | Method and device for controlling focusing, storage medium and electronic equipment | |
| CN109862269B (en) | Image acquisition method and device, electronic equipment and computer readable storage medium | |
| CN108337445B (en) | Photographing method, related device and computer storage medium | |
| CN107846556B (en) | Imaging method, imaging device, mobile terminal and storage medium | |
| CN110166705B (en) | High dynamic range HDR image generation method and device, electronic equipment and computer readable storage medium | |
| CN108337446B (en) | High dynamic range image acquisition method, device and equipment based on double cameras | |
| CN110213494B (en) | Photographing method and device, electronic equipment and computer readable storage medium | |
| CN115278069A (en) | Image processing method and device, computer readable storage medium and terminal | |
| CN110198418B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110049240B (en) | Camera control method and device, electronic equipment and computer readable storage medium | |
| CN108156369B (en) | Image processing method and device | |
| CN110290325B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US9071766B2 (en) | Image capturing apparatus and control method thereof | |
| CN110213498B (en) | Image generation method and device, electronic equipment and computer readable storage medium | |
| US20190347776A1 (en) | Image processing method and image processing device | |
| CN110430370B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110881108B (en) | Image processing method and image processing apparatus | |
| CN110177212B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN109005346B (en) | Control method, control device, electronic equipment and computer-readable storage medium | |
| JP2015144475A (en) | Imaging apparatus, control method of the same, program and storage medium | |
| CN101998059A (en) | Photographing apparatus and method | |
| CN108881876B (en) | Method and device for carrying out white balance processing on image and electronic equipment | |
| CN110266965B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110266967B (en) | Image processing method, image processing device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |