CN107454340B - Image synthesis method and device based on high dynamic range principle and mobile terminal - Google Patents
Image synthesis method and device based on high dynamic range principle and mobile terminal Download PDFInfo
- Publication number
- CN107454340B CN107454340B CN201710630424.XA CN201710630424A CN107454340B CN 107454340 B CN107454340 B CN 107454340B CN 201710630424 A CN201710630424 A CN 201710630424A CN 107454340 B CN107454340 B CN 107454340B
- Authority
- CN
- China
- Prior art keywords
- pixel
- pixels
- dynamic range
- high dynamic
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an image synthesis method, an image synthesis device and a mobile terminal based on a high dynamic range principle, wherein the method comprises the following steps: collecting a plurality of images with different exposure parameters; synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image; carrying out tone mapping on the high dynamic range image to re-compress pixels of the high dynamic range image; the device comprises an image acquisition module, an image synthesis module and a tone mapping module; the mobile terminal comprises a processor and a memory for storing a processor executable program, and when the processor executes the program stored in the memory, the operation of the method is realized. The invention synthesizes a plurality of images with different exposure parameters to obtain a high dynamic range image, and performs tone mapping, thereby not only improving the internal details of the image, but also keeping the edge details of the image.
Description
Technical Field
The invention relates to an image synthesis method, in particular to an image synthesis method, an image synthesis device and a mobile terminal based on a high dynamic range principle, and belongs to the field of image processing.
Background
With the popularization of certificates, information such as Hongkong and Macau passes, Taiwan passes, passports, identity cards and the like all need to be verified for personal information. The applicant needs to shoot the certificate photo meeting the certificate making requirement again every time of certificate making.
With the popularization of mobile phone shooting, the shooting function is increasingly powerful, but the following problems still exist:
1) the system is difficult to extract detailed information when the photo processing system reads information due to uneven overall lighting caused by different shooting environments or light reflection on the surface of a shot object and the like;
2) in the process of reading a person image, a large amount of noise points appear in the shot image due to light problems during shooting, so that the color channel of the image deviates, and an error appears when a system reads face or background area information;
3) it is difficult to ensure the light stability, and the system needs to process the shot photo to separate the external environment from the character image, and at the same time, the character detail information can be separated, but the processing method is troublesome.
Therefore, when the photos shot by the mobile phone are used for subsequent image processing, the detailed information of the photos cannot be completely separated, and the photos shot with the assistance of other people cannot meet the requirements of certificate making; when the shooting mechanism with perfect configuration is used for shooting, the charging is expensive, the shooting effect is poor, and when the shooting effect is unsatisfactory and the photographer wants to take a picture again, the shooting mechanism needs to communicate with the photographer again and trade the photo, which is time-consuming and labor-consuming.
For a High-dynamic range (HDR) image, two stages of long and short exposures are usually adopted to realize the image, but the effect is not ideal, and particularly in a scene where brightness and darkness are mixed, the transition effect between a bright place and a dark place is not good.
Disclosure of Invention
The first objective of the present invention is to solve the above-mentioned drawbacks of the prior art, and provide an image synthesis method based on the high dynamic range principle, which synthesizes a plurality of images with different exposure parameters to obtain a high dynamic range image, and performs tone mapping, thereby improving the details inside the image and preserving the details of the edges of the image.
It is a second object of the present invention to provide an image composition system based on the high dynamic range principle.
A third object of the present invention is to provide a mobile terminal.
The first purpose of the invention can be achieved by adopting the following technical scheme:
a method of image synthesis based on the high dynamic range principle, the method comprising:
collecting a plurality of images with different exposure parameters;
synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
tone mapping is performed on the high dynamic range image, so that pixels of the high dynamic range image are recompressed.
Further, the synthesizing a plurality of images with different exposure parameters based on the high dynamic range principle to obtain a high dynamic range image specifically includes:
expanding the pixel levels of a plurality of images with different exposure parameters;
and taking the average value of the pixel values of the extended images with different exposure parameters to obtain a high dynamic range image.
Further, the expanding the pixel levels in the images with different exposure parameters specifically includes:
multiplying each pixel in a plurality of images of different exposure parameters by 28～264Each pixel is made to be able to accommodate 216～272A level of pixels.
Further, tone mapping is performed on the high dynamic range image, so that the pixels of the high dynamic range image are recompressed, specifically including:
constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image;
for each 3 x 3 pixel grid, comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the periphery respectively;
according to the comparison result, when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two, the pixels of the central pixel grid are different from those of the eight pixel grids surrounding the periphery, the central pixel grid is judged as edge information; otherwise, judging the central pixel grid as common information;
and mapping all pixels of the central pixel grid which are judged to be common information to recompress the pixels of the high dynamic range image.
Further, the pixel squares with different pixels from those of the central pixel square in the eight pixel squares surrounding the periphery refer to:
the pixel grids of the pixels in the eight pixel grids surrounding the periphery are larger than m times of the pixels in the central pixel grid; wherein 1< m < 1.2;
or the pixel grids of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are smaller than n times the pixels of the central pixel grid; wherein 0.8< n <1.
Further, the mapping processing of the pixels of the central pixel grid determined as the common information to recompress the pixels of the high dynamic range image specifically includes:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter, and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl)
wherein l (x, y) is a pixel value;
the pixel values l (x, y) are normalized to 0-255 levels.
The second purpose of the invention can be achieved by adopting the following technical scheme:
an image synthesis apparatus based on the high dynamic range principle, the apparatus comprising:
the image acquisition module is used for acquiring a plurality of images with different exposure parameters;
the image synthesis module is used for synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
and the tone mapping module is used for carrying out tone mapping on the high dynamic range image so as to recompress the pixels of the high dynamic range image.
Further, the image synthesis module specifically includes:
the pixel level expansion unit is used for expanding the pixel levels of a plurality of images with different exposure parameters;
and the averaging unit is used for averaging the pixel values of the extended images with different exposure parameters to obtain a high dynamic range image.
Further, the pixel-level expansion unit specifically includes:
multiplying each pixel in a plurality of images of different exposure parameters by 28～264Each pixel is made to be able to accommodate 216～272A level of pixels.
Further, the tone mapping module specifically includes:
a pixel grid constructing unit, which is used for constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image;
the comparison unit is used for comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the periphery of the central pixel grid aiming at each pixel grid of 3 x 3;
the judging unit is used for judging the central pixel grid as edge information when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two and the number of the pixel grids is different from that of the pixels in the central pixel grid; otherwise, judging the central pixel grid as common information;
and the mapping unit is used for mapping all the pixels of the central pixel grid which are judged to be the common information so as to recompress the pixels of the high dynamic range image.
Further, the pixel squares with different pixels from those of the central pixel square in the eight pixel squares surrounding the periphery refer to:
the pixel grids of the pixels in the eight pixel grids surrounding the periphery are larger than m times of the pixels in the central pixel grid; wherein 1< m < 1.2;
or the pixel grids of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are smaller than n times the pixels of the central pixel grid; wherein 0.8< n <1.
Further, in the mapping unit, performing tone mapping processing on all the central pixel squares determined as the common information to recompress the pixels of the high dynamic range image specifically includes:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter, and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl)
wherein l (x, y) is a pixel value;
the pixel values l (x, y) are normalized to 0-255 levels.
The third purpose of the invention can be achieved by adopting the following technical scheme:
a mobile terminal comprising a processor and a memory for storing processor-executable programs, the processor, when executing the programs stored in the memory, performing the following:
collecting a plurality of images with different exposure parameters;
synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
tone mapping is performed on the high dynamic range image, so that pixels of the high dynamic range image are recompressed.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention collects a plurality of images with different exposure parameters, synthesizes the images with different exposure parameters based on the high dynamic range principle, reduces noise generated by environmental illumination when the images are shot by the synthesized images, solves the offset on a color channel in the image processing process, and then carries out tone mapping on the obtained high dynamic range images, so that the pixels of the high dynamic range images are recompressed to restore to a level capable of being displayed on the mobile terminal again, and the details are retained to the maximum extent.
2. In the synthesis process based on the high dynamic range principle, the invention expands the pixel level of a plurality of images with different exposure parameters to ensure that each pixel can contain 216～272Pixels of one level, i.e. pixel level up to 216～272The image detail can be better preserved by multiple levels, and the image can better restore the current shooting light.
3. After obtaining the high dynamic range image, the invention constructs a plurality of 3 x 3 pixel grids in the whole high dynamic range image, analyzes and judges the central pixel grid of each 3 x 3 pixel square, if the central pixel grid is edge information, the central pixel grid is not processed, if the central pixel grid is common information, the pixels of the central pixel grid are mapped, and weight is added as the measuring standard of tone mapping, thus the image can keep edge details.
Drawings
Fig. 1 is a flowchart of an image synthesis method according to embodiment 1 of the present invention.
FIG. 2a is a schematic diagram of a normal exposure parameter image in embodiment 1 of the present invention.
Fig. 2b is a schematic diagram of an under-exposed image according to embodiment 1 of the present invention.
Fig. 2c is a schematic diagram of an overexposed image in embodiment 1 of the invention.
Fig. 3 is a flowchart of obtaining a high dynamic range image according to embodiment 1 of the present invention.
FIG. 4 is a flowchart of tone mapping according to embodiment 1 of the present invention.
Fig. 5 is a schematic diagram of constructing a plurality of 3 × 3 pixel grids in the whole high dynamic range image according to embodiment 1 of the present invention.
Fig. 6 is a diagram showing the final effect of embodiment 1 of the present invention.
Fig. 7 is a block diagram of an image synthesizing apparatus according to embodiment 2 of the present invention.
Fig. 8 is a block diagram of an image synthesis module according to embodiment 2 of the present invention.
Fig. 9 is a block diagram of a tone mapping module according to embodiment 2 of the present invention.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.
Example 1:
the embodiment provides an image synthesis method which improves the synthesis of shot images based on the high dynamic range principle to obtain an image with more details and better light conditions.
As shown in fig. 1, the image synthesis method of the present embodiment includes the steps of:
s101, collecting three images with different exposure parameters
In the embodiment, if two images with different exposure parameters are acquired, the effect is poor, and if four or more images with different exposure parameters are acquired, the displacement generated in the shooting process is too large, so that the subsequent correction is influenced, and therefore, the acquisition of three images with different exposure parameters is determined, namely, the image with the normal exposure parameters, the underexposed image and the overexposed image.
Since the exposure parameters of the built-in cameras of different mobile terminals (such as smart phones) are different, even if the exposure parameters are the same, when the shooting environment changes, the exposure effect of the image is also affected, so in this embodiment, under the same shooting environment, three images with different exposure parameters are rapidly shot by using the same mobile terminal, the exposure parameter of the first image is the exposure parameter of the environment during shooting, and the image with the exposure parameter is taken as the image with the normal exposure parameter, as shown in fig. 2 a; the exposure of the second image is adjusted downward with reference to the normal exposure parameters to obtain an under-exposed image, preferably 2EV (-2EV) to obtain an under-exposed image, as shown in FIG. 2 b; the exposure of the second image is adjusted up with reference to the normal exposure parameters resulting in an overexposed image, preferably 2EV (+2EV) as shown in fig. 2 c.
S102, synthesizing three images with different exposure parameters based on the High Dynamic Range principle to obtain a High-Dynamic Range image (HDRI for short)
As shown in fig. 3, step S102 specifically includes:
s1021, expanding the pixel level of the three images with different exposure parameters
At present, when a mobile terminal displays an image, an 8-bit (RGB signal transmission) display mode is mostly used, which results in that an image level can only display 256 levels of image pixels from 0 to 255, and in an actual scene, a luminance difference between the brightest level and the darkest level is far greater than the 256 levels of pixels, and a real luminance level difference reaches 108Therefore, the pixels beyond 256 levels are omitted when the mobile terminal displays the image.
In order to retain more details in the image compression process, the three images with different exposure parameters are imagedThe elementary level is expanded, and specifically comprises the following steps: each pixel (2) of three images8) Are all multiplied by 28～264Each pixel is made to be able to accommodate 216～272Pixels of each level, this embodiment, to accommodate most mobile terminal capabilities, each pixel of the three images (2)8) Multiplying by 224I.e. each pixel has reached capacity to accommodate 232A level of pixels.
S1022, taking the average value of the pixel values of the three expanded images with different exposure parameters to obtain the high dynamic range image
The pixel values of the three expanded images with different exposure parameters are superposed, and then divided by the number of the images (namely 3) to obtain an average value, so that a high dynamic range image with multi-level brightness information at the same time can be obtained, wherein the high dynamic range image is an image close to natural illumination.
S103, tone mapping is carried out on the high dynamic range image, and the pixels of the high dynamic range image are compressed again
After obtaining the high dynamic range image, the pixels of the image need to be recompressed to a level where they can be displayed again on the mobile terminal, which requires Tone Mapping (Tone Mapping) of the image to preserve the details to the maximum extent.
Due to the limitation of a built-in camera of the mobile terminal, the acquired image cannot obtain accurate exposure parameters, and an accurate camera response curve cannot be fitted to perform tone mapping of the image, so that a new tone mapping method is adopted to compress the high dynamic range image.
As shown in fig. 4, step S103 specifically includes:
s1031, constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image
The pixel grid of 3 × 3 refers to a pixel grid at the center and eight grids surrounding the pixel grid, as shown in fig. 5, taking two pixel grids of 3 × 3 as an example, the center pixel grid of one pixel grid of 3 × 3 is an a pixel grid, the a pixel grid and the eight pixel grids surrounding the a pixel grid form a nine-square grid together, the center pixel grid of the other pixel grid of 3 × 3 is a B pixel grid, and the B pixel grid and the eight pixel grids surrounding the B pixel grid form a nine-square grid together.
S1032, for each 3 × 3 pixel grid, comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the central pixel grid
In this step, taking an a pixel grid and eight pixel grids surrounding it, and a B pixel grid and eight pixel grids surrounding it as examples, the pixels of the a pixel grid and the pixels of the eight pixel grids surrounding it are compared, and the pixels of the B pixel grid and the pixels of the eight pixel grids surrounding it are compared.
S1033, according to the comparison result, when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two, the number of the pixel grids with different pixels from that of the central pixel grid is judged as edge information; otherwise, judging the central pixel grid as common information
In the eight pixel squares surrounding the periphery, the pixels of the pixel squares are different from those of the central pixel square, and this embodiment defines them as: pixel grids of pixels in the eight pixel grids surrounding the periphery, wherein the pixels are larger than m times of the pixels in the central pixel grid, or pixel grids of pixels in the eight pixel grids surrounding the periphery, wherein the pixels are smaller than n times of the pixels in the central pixel grid; where 1< m <1.2, 0.8< n <1, for example, m may have a value of 1.02, 1.05, 1.08, 1.1, 1.12, 1.15, 1.18, etc., and n may have a value of 0.82, 0.85, 0.88, 0.9, 0.92, 0.95, 0.98, etc., where m is 1.1 and n is 0.9, which is the most preferable value obtained by the experiment.
Taking an a pixel grid and eight pixel grids surrounding the a pixel grid as an example, if there are pixels of a certain pixel grid larger than 1.1 times the a pixel grid or smaller than 0.9 times the a pixel grid in the eight pixel grids surrounding the a pixel grid, it is considered that the pixels of the pixel grid are different from the pixels of the a pixel grid, and the number of the pixel grids is counted, as shown in fig. 5, in the eight pixel grids, there are pixels of five pixel grids different from the pixels of the a pixel grid, because the number of the grids is more than two, the a pixel grid is determined as edge information, and the edge information is not processed in order to retain more image details.
Taking B pixel squares and eight pixel squares around the B pixel squares as an example, if there are pixels of a certain pixel square larger than 1.1 times the pixels of the B pixel squares or smaller than 0.9 times the pixels of the B pixel squares in the eight pixel squares around the B pixel squares, the pixels of the pixel squares are considered to be different from the pixels of the B pixel squares, and the number of the pixel squares is counted, as shown in fig. 5, in the eight pixel squares, only the pixels of one pixel square are different from the pixels of the B pixel squares, and since the number of the squares is less than two, the B pixel squares are judged to be common information.
S1034, mapping all the pixels of the central pixel grid which are judged to be the common information to re-compress the pixels of the high dynamic range image
Step S1034 specifically includes:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl) (1)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter (set to 5 according to an empirical value), and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y) (3)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y) (4)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl) (5)
wherein l (x, y) is a pixel value;
the pixel value l (x, y) of the final image is obtained by calculation of formula (5), in order to enable the pixel value to be displayed in the mobile terminal, the pixel value l (x, y) is normalized to 0-255 levels, namely the pixel value of the mapped 3 × 3 pixel grid is filled back into the original 3 × 3 pixel grid, the compressed image pixel is a Low-dynamic range (Low-dynamic range) image suitable for the mobile terminal to display, and the final effect is as shown in fig. 6.
Example 2:
as shown in fig. 7, the present embodiment provides an image synthesis apparatus based on the high dynamic range principle, the apparatus includes an image acquisition module 701, an image synthesis module 702, and a tone mapping module 703, and specific functions of each module are as follows:
the image acquisition module 701 is configured to acquire a plurality of images with different exposure parameters.
The image synthesis module 702 is configured to synthesize a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image, and the structure of the module is as shown in fig. 8, and specifically includes:
the pixel level expansion unit 7021 is configured to expand pixel levels of a plurality of images with different exposure parameters, specifically: multiplying each pixel in a plurality of images of different exposure parameters by 28～264Each pixel is made to be able to accommodate 216～272A plurality of levels of pixels;
the averaging unit 7022 is configured to obtain an average value of pixel values of the extended images with different exposure parameters, so as to obtain a high dynamic range image.
The tone mapping module 703 is configured to perform tone mapping on the high dynamic range image, so as to recompress the pixels of the high dynamic range image, and the structure of the module is as shown in fig. 9, and specifically includes:
a pixel grid constructing unit 7031, configured to construct a plurality of 3 × 3 pixel grids in the entire high dynamic range image;
a comparing unit 7032, configured to compare, for each 3 × 3 pixel grid, pixels of the central pixel grid with pixels of eight pixel grids surrounding the periphery, respectively;
a determining unit 7033, configured to determine, according to the comparison result, when the number of pixel squares, in the eight pixel squares surrounding the four sides, where pixels of the pixel squares are different from those of the central pixel square, is more than two, the central pixel square is determined as edge information; otherwise, judging the central pixel grid as common information;
whether the pixels of the eight pixel grids surrounding the periphery are the same as the pixels of the central pixel grid or not is judged by adopting the following judgment method:
a. in eight pixel grids surrounding the periphery, if the pixels of a certain pixel grid are larger than m (m is a set value, 1< m <1.2) times of the pixels of the central pixel grid, the pixels of the pixel grid are different from the pixels of the central pixel grid;
b. in the eight pixel grids surrounding the periphery, if the pixels of a certain pixel grid are smaller than n (n is a set value, 0.8< n <1) times of the pixels of the central pixel grid, the pixels of the pixel grid are different from the pixels of the central pixel grid;
c. in the case other than the cases a and b, that is, in the case where the pixels in a certain pixel grid are not less than m times the pixels in the central pixel grid, but not less than n times the pixels in the central pixel grid, the pixels in the pixel grid are the same as the pixels in the central pixel grid.
The mapping unit 7034 is configured to perform mapping processing on all pixels of the central pixel grid determined as the common information, so as to recompress the pixels of the high dynamic range image, and specifically includes:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter, and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl)
wherein l (x, y) is a pixel value;
and normalizing the pixel values l (x, y) to 0-255 levels, namely filling the pixel values of the mapped 3-3 pixel grids back to the original 3-3 pixel grids.
It should be noted that, the apparatus of this embodiment is only exemplified by the division of the above functional modules, and in practical applications, the above functions may be distributed by different functional modules as needed, that is, the internal structure may be divided into different functional modules to complete all or part of the above described functions.
Example 3:
the embodiment provides a mobile terminal, which comprises a processor and a memory, wherein the memory stores one or more programs, and when the processor executes the programs stored in the memory, the following operations are realized:
collecting a plurality of images with different exposure parameters;
synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
tone mapping is performed on the high dynamic range image, so that pixels of the high dynamic range image are recompressed.
The mobile terminal described in this embodiment may be a smart phone, a PDA handheld terminal, a tablet computer, or other handheld terminal devices with shooting and displaying functions.
In summary, the present invention collects a plurality of images with different exposure parameters, and synthesizes the images with different exposure parameters based on the high dynamic range principle, so that the synthesized image reduces noise caused by environmental illumination when the image is shot, and simultaneously solves the offset on the color channel in the image processing process, and then performs tone mapping on the obtained high dynamic range image, so that the pixels of the high dynamic range image are recompressed to restore to the level capable of being displayed on the mobile terminal again, and the details are retained to the maximum extent.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and for example, the present invention can be implemented by acquiring two, four or more images with different exposure parameters for synthesis, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (11)
1. An image synthesis method based on the high dynamic range principle is characterized in that: the method comprises the following steps:
collecting a plurality of images with different exposure parameters;
synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
performing tone mapping on the high dynamic range image to recompress pixels of the high dynamic range image, specifically comprising:
constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image;
for each 3 x 3 pixel grid, comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the periphery respectively;
according to the comparison result, when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two, the pixels of the central pixel grid are different from those of the eight pixel grids surrounding the periphery, the central pixel grid is judged as edge information; otherwise, judging the central pixel grid as common information;
and mapping all pixels of the central pixel grid which are judged to be common information to recompress the pixels of the high dynamic range image.
2. An image synthesis method based on the high dynamic range principle as claimed in claim 1, characterized in that: the synthesizing of a plurality of images with different exposure parameters based on the high dynamic range principle to obtain the high dynamic range image specifically comprises:
expanding the pixel levels of a plurality of images with different exposure parameters;
and taking the average value of the pixel values of the extended images with different exposure parameters to obtain a high dynamic range image.
3. An image synthesis method based on the high dynamic range principle as claimed in claim 2, characterized in that: the expanding the pixel levels in the images with different exposure parameters specifically comprises:
multiplying each pixel in a plurality of images of different exposure parameters by 28～264Each pixel is made to be able to accommodate 216～272A level of pixels.
4. An image synthesis method based on the high dynamic range principle as claimed in claim 1, characterized in that: the pixel grid of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are different from those of the central pixel grid, is as follows:
the pixel grids of the pixels in the eight pixel grids surrounding the periphery are larger than m times of the pixels in the central pixel grid; wherein 1< m < 1.2;
or the pixel grids of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are smaller than n times the pixels of the central pixel grid; wherein 0.8< n <1.
5. An image synthesis method based on the high dynamic range principle as claimed in claim 1, characterized in that: the mapping processing of the pixels of the central pixel grid which are judged to be the common information to re-compress the pixels of the high dynamic range image specifically comprises the following steps:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter, and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
w(x,y)＝e-|f(x,y)|5*cl
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl)
wherein l (x, y) is a pixel value;
the pixel values l (x, y) are normalized to 0-255 levels.
6. An image synthesizing apparatus based on the high dynamic range principle, characterized in that: the device comprises:
the image acquisition module is used for acquiring a plurality of images with different exposure parameters;
the image synthesis module is used for synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
the tone mapping module is used for carrying out tone mapping on the high dynamic range image so as to recompress the pixels of the high dynamic range image;
the tone mapping module specifically includes:
a pixel grid constructing unit, which is used for constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image;
the comparison unit is used for comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the periphery of the central pixel grid aiming at each pixel grid of 3 x 3;
the judging unit is used for judging the central pixel grid as edge information when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two and the number of the pixel grids is different from that of the pixels in the central pixel grid; otherwise, judging the central pixel grid as common information;
and the mapping unit is used for mapping all the pixels of the central pixel grid which are judged to be the common information so as to recompress the pixels of the high dynamic range image.
7. An image synthesis apparatus based on the high dynamic range principle as claimed in claim 6, characterized in that: the image synthesis module specifically includes:
the pixel level expansion unit is used for expanding the pixel levels of a plurality of images with different exposure parameters;
and the averaging unit is used for averaging the pixel values of the extended images with different exposure parameters to obtain a high dynamic range image.
8. An image synthesis apparatus based on the high dynamic range principle as claimed in claim 7, wherein: the pixel-level expansion unit specifically includes:
for multiplying each pixel in a plurality of images of different exposure parameters by 28～264Each pixel is made to be able to accommodate 216～272A level of pixels.
9. An image synthesis apparatus based on the high dynamic range principle as claimed in claim 6, characterized in that: the pixel grid of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are different from those of the central pixel grid, is as follows:
the pixel grids of the pixels in the eight pixel grids surrounding the periphery are larger than m times of the pixels in the central pixel grid; wherein 1< m < 1.2;
or the pixel grids of the eight pixel grids surrounding the periphery, wherein the pixels of the eight pixel grids are smaller than n times the pixels of the central pixel grid; wherein 0.8< n <1.
10. An image synthesis apparatus based on the high dynamic range principle as claimed in claim 1, characterized in that: in the mapping unit, tone mapping is performed on all the central pixel squares determined as the common information, so that the pixels of the high dynamic range image are recompressed, specifically including:
calculating the mapping values of all pixels of the central pixel grid which are judged to be common information as follows:
f(x,y)＝ln(p(x,y))/ln(cl)-ln(p)/ln(cl)
wherein p (x, y) is the pixels of eight pixel grids around the 3 × 3 pixel grid, p is the pixels of the central pixel grid of the 3 × 3 pixel grid, cl is a contrast parameter, and f (x, y) is the pixel mapping value of the 3 × 3 pixel grid;
calculating the weight of all the central pixel squares judged as common information as follows:
wherein, w (x, y) is the pixel weight of 3 x 3 pixel grids;
substituting the calculated mapping value f (x, y) and the weight value w (x, y) into the following formula:
f'(x,y)＝f(x,y)*w(x,y)
substituting the calculated f' (x, y) into the following equation:
wm＝∑f'(x,y)/∑w(x,y)
wherein wm is a numerical value in operation;
substituting the calculated wm into the following formula:
l(x,y)＝e(wm+(p(x,y)/log(cl)))*log(cl)
wherein l (x, y) is a pixel value;
the pixel values l (x, y) are normalized to 0-255 levels.
11. A mobile terminal comprising a processor and a memory for storing processor-executable programs, characterized in that: when the processor executes the program stored in the memory, the following operations are realized:
collecting a plurality of images with different exposure parameters;
synthesizing a plurality of images with different exposure parameters based on a high dynamic range principle to obtain a high dynamic range image;
performing tone mapping on the high dynamic range image to recompress pixels of the high dynamic range image, specifically comprising:
constructing a plurality of 3 x 3 pixel grids in the whole high dynamic range image;
for each 3 x 3 pixel grid, comparing the pixels of the central pixel grid with the pixels of the eight pixel grids surrounding the periphery respectively;
according to the comparison result, when the number of the pixel grids in the eight pixel grids surrounding the periphery is more than two, the pixels of the central pixel grid are different from those of the eight pixel grids surrounding the periphery, the central pixel grid is judged as edge information; otherwise, judging the central pixel grid as common information;
and mapping all pixels of the central pixel grid which are judged to be common information to recompress the pixels of the high dynamic range image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710630424.XA CN107454340B (en) | 2017-07-28 | 2017-07-28 | Image synthesis method and device based on high dynamic range principle and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710630424.XA CN107454340B (en) | 2017-07-28 | 2017-07-28 | Image synthesis method and device based on high dynamic range principle and mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107454340A CN107454340A (en) | 2017-12-08 |
| CN107454340B true CN107454340B (en) | 2019-12-27 |
Family
ID=60489443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710630424.XA Active CN107454340B (en) | 2017-07-28 | 2017-07-28 | Image synthesis method and device based on high dynamic range principle and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107454340B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163816B (en) * | 2019-04-24 | 2021-08-31 | Oppo广东移动通信有限公司 | Image information processing method and device, storage medium and electronic equipment |
| CN111131722A (en) * | 2019-12-30 | 2020-05-08 | 维沃移动通信有限公司 | Image processing method, electronic device, and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514588A (en) * | 2013-10-10 | 2014-01-15 | 广东威创视讯科技股份有限公司 | Image enhancement method and system |
| CN104717432A (en) * | 2013-12-13 | 2015-06-17 | 柯尼卡美能达美国研究所有限公司 | Method for processing input image, image processing equipment, and digital camera |
| CN104835131A (en) * | 2015-04-20 | 2015-08-12 | 中国科学技术大学先进技术研究院 | Method and system for achieving HDR image generation and tone mapping based on IC |
| CN105450939A (en) * | 2015-11-19 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | High dynamic range image display method and mobile terminal |
| CN106162131A (en) * | 2015-04-28 | 2016-11-23 | 艾韬 | A kind of real time image processing |
| CN106488141A (en) * | 2015-08-13 | 2017-03-08 | 汤姆逊许可公司 | HDR is to the method for HDR inverse tone mapping (ITM), system and equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101420531A (en) * | 2007-10-23 | 2009-04-29 | 鸿富锦精密工业（深圳）有限公司 | High dynamic range photo acquisition apparatus and method |
| US9704269B2 (en) * | 2014-11-21 | 2017-07-11 | Texas Instruments Incorporated | Efficient methodology to process wide dynamic range images |
-
2017
- 2017-07-28 CN CN201710630424.XA patent/CN107454340B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514588A (en) * | 2013-10-10 | 2014-01-15 | 广东威创视讯科技股份有限公司 | Image enhancement method and system |
| CN104717432A (en) * | 2013-12-13 | 2015-06-17 | 柯尼卡美能达美国研究所有限公司 | Method for processing input image, image processing equipment, and digital camera |
| CN104835131A (en) * | 2015-04-20 | 2015-08-12 | 中国科学技术大学先进技术研究院 | Method and system for achieving HDR image generation and tone mapping based on IC |
| CN106162131A (en) * | 2015-04-28 | 2016-11-23 | 艾韬 | A kind of real time image processing |
| CN106488141A (en) * | 2015-08-13 | 2017-03-08 | 汤姆逊许可公司 | HDR is to the method for HDR inverse tone mapping (ITM), system and equipment |
| CN105450939A (en) * | 2015-11-19 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | High dynamic range image display method and mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107454340A (en) | 2017-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110033418B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US9451173B2 (en) | Electronic device and control method of the same | |
| US9172888B2 (en) | Determining exposure times using split paxels | |
| US8964060B2 (en) | Determining an image capture payload burst structure based on a metering image capture sweep | |
| CN108337445B (en) | Photographing method, related device and computer storage medium | |
| US11431915B2 (en) | Image acquisition method, electronic device, and non-transitory computer readable storage medium | |
| CN104883504B (en) | Open the method and device of high dynamic range HDR functions on intelligent terminal | |
| CN110022469B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN106981054B (en) | Image processing method and electronic equipment | |
| CN112565636B (en) | Image processing method, device, equipment and storage medium | |
| CN110266954B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US9087391B2 (en) | Determining an image capture payload burst structure | |
| CN108616700B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110047060B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113411498B (en) | Image shooting method, mobile terminal and storage medium | |
| CN110881108B (en) | Image processing method and image processing apparatus | |
| CN113012081A (en) | Image processing method, device and electronic system | |
| CN107682611B (en) | Focusing method and device, computer readable storage medium and electronic equipment | |
| CN107147851B (en) | Photo processing method and device, computer readable storage medium and electronic equipment | |
| CN102469248B (en) | Device for filming image and image synthesis method thereof | |
| CN107454340B (en) | Image synthesis method and device based on high dynamic range principle and mobile terminal | |
| CN104869319A (en) | Image processing method and image processing device | |
| CN115147304A (en) | Image fusion method and device, electronic equipment, storage medium and product | |
| CN108401119B (en) | Image processing method, mobile terminal and related medium product | |
| TW201525942A (en) | Image processing method and image processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |