CN109257540B - Photographing correction method of multi-photographing lens group and photographing device - Google Patents
Photographing correction method of multi-photographing lens group and photographing device Download PDFInfo
- Publication number
- CN109257540B CN109257540B CN201811307126.8A CN201811307126A CN109257540B CN 109257540 B CN109257540 B CN 109257540B CN 201811307126 A CN201811307126 A CN 201811307126A CN 109257540 B CN109257540 B CN 109257540B
- Authority
- CN
- China
- Prior art keywords
- image
- block
- sharpness
- blocks
- gray
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
Abstract
The invention relates to a shooting correction method and a shooting device of a multi-shooting lens group, comprising the following steps: s1, enabling one lens group in a multi-lens group to take a first image, and generating a first sharpness index based on the first image; s2, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting another lens group in the multi-lens group to take a second image; and S3, acquiring the number of pixels which are added into the first image by the second image based on the first sharpness index, and finishing the correction of the first image. Whether the first lens group is blurred or not is evaluated based on the sharpness index, when part or all of the first image is blurred, a second lens with a focal range different from that of the first lens group is started to shoot a second image, and the blurred area in the first image is replaced or synthesized through the second image, so that under the condition that the image quality is guaranteed, the calculated amount of the method is greatly reduced, and the calculation efficiency is improved.
Description
Technical Field
The present invention relates to the field of optical lens imaging, and more particularly, to a photographing correction method and a photographing apparatus for a multi-lens assembly.
Background
In recent years, with the development of science and technology, portable electronic products such as mobile phones and tablet computers have been increasingly popular, and therefore, demands for camera lenses suitable for portable electronic products in the market have been increasing. As portable electronic products tend to be miniaturized and light and thin, the total length of the lens is limited, thereby increasing the design difficulty of the lens. Meanwhile, with the improvement of performance and size reduction of elements such as a common photosensitive element CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), higher requirements are put forward for high imaging quality and miniaturization of the lens. At present, a double-shooting module formed by providing two lens groups is applied to a portable electronic product, so that the requirements of part of the double-shooting module can be met, and the functions of optical zooming, wide-angle assistance, depth-of-field assistance, 3D imaging, augmented reality and the like which are difficult to realize by other single-shooting modules are realized. However, the application of the two shots in the prior art is still not perfect, and various use scenes which can be realized through the double shot are waiting for mining. Especially, the double-shooting lens group is difficult to realize dynamic zooming, and the images shot by the two lens groups respectively need to be fused to simulate the zooming effect by a software means, so that the image shot by the existing double-shooting module has low definition and poor effect.
Disclosure of Invention
The present invention provides a photographing correction method and a photographing apparatus for a multi-lens assembly, which can improve the imaging quality of the multi-lens assembly.
To achieve the above object, the present invention provides a method for correcting a photographing lens assembly, comprising:
s1, enabling one lens group in a multi-lens group to take a first image, and generating a first sharpness index based on the first image;
s2, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting another lens group in the multi-lens group to take a second image;
and S3, acquiring the number of pixels which are added into the first image by the second image based on the first sharpness index, and finishing the correction of the first image.
According to an aspect of the present invention, step S1 includes:
s11, a first image is shot through one of the multiple shooting lens groups, and the first image is divided into a plurality of blocks;
s12, converting the first image to obtain a first gray scale image, and obtaining a first gray scale gradient image according to the first gray scale image;
s13, obtaining the first sharpness index of at least one block in the first image according to the first gray-scale image and the first gray-scale gradient image.
According to an aspect of the invention, the first sharpness indicator is:
wherein G ismnIs the average of the maximum and minimum gray values in the block, G0Is the average of all gray values in the block, gmnIs the average of the maximum and minimum gray gradient values in the block, g0Is the average of all the gray scale gradient values in the block.
According to an aspect of the invention, in step S2, the first threshold condition is that the first sharpness index among the blocks in the first image whose ratio exceeds a preset ratio is less than or equal to a preset sharpness threshold.
According to an aspect of the invention, in step S2, the first threshold condition is that the sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
According to an aspect of the present invention, the step of taking the second image by the other lens group of the multi-lens group in the step S2 further includes:
s21, mapping the second image to a plurality of blocks in the first image according to a block division mode consistent with that of the first image;
s22, converting the second image to obtain a second gray scale map, and obtaining a second gray scale gradient map according to the second gray scale image;
s13, obtaining a second sharpness index of at least one block in the second image according to the second gray-scale image and the second gray-scale gradient image.
According to an aspect of the present invention, in step S3, the number of pixels in at least one block of the second image used for complementing a corresponding block in the first image is obtained based on the difference between the first sharpness index and the second sharpness index.
According to an aspect of the present invention, in step S3, when the difference between the first sharpness index and the second sharpness index of a block is greater than a predetermined upper limit of sharpness difference, all pixels in the block of the first image are replaced with pixels of a corresponding block in the second image.
According to an aspect of the present invention, in step S3, when the difference between the first sharpness index and the second sharpness index is less than a predetermined lower sharpness difference, the lens group taking the second image is disabled.
According to an aspect of the present invention, in step S21, the second image is mapped to a block division manner on the first image by using a coordinate transformation method using pre-stored calibration information or by using an image feature matching method.
According to an aspect of the invention, in step S11, at least one of the blocks in the first image has a size different from the sizes of the other blocks.
To achieve the above object, the present invention provides a method for correcting a photographing lens assembly, comprising:
s1, starting a first lens group of a multi-camera lens group to shoot a first image, and dividing the first image into a plurality of blocks;
s2, converting the first image into a first gray-scale image, and calculating a corresponding first gray-scale gradient image;
s3, calculating a first sharpness index of at least one block through the first gray-scale image and the first gray-scale gradient image;
s4, enabling a second lens group of the multi-shooting lens group to shoot a second image in response to the first sharpness index not meeting a preset first threshold condition; and
determining a number of pixels in the second image in the at least one block for compositing with the first image based on the first sharpness indicator.
To achieve the above object, the present invention provides an image pickup apparatus including: a multi-taking lens group including a first lens for taking a first image, a second lens for taking a second image;
the focal length range of the first lens is different from that of the second lens;
generating a first sharpness index based on the first image, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting the second lens to shoot the second image;
and acquiring the number of pixels which are added into the first image by the second image based on the first sharpness index, and correcting the first image. .
According to an aspect of the invention, further comprising: the processor is electrically connected with the first lens and the second lens, the memory is electrically connected with the processor, the display is used for displaying images, and the communication interface is used for receiving the images;
the memory stores a first image shot by the first lens, and the processor divides the first image into a plurality of blocks;
the processor converts the first image to obtain a first gray-scale image, and obtains a first gray-scale gradient image according to the first gray-scale image;
the processor obtains the first sharpness index of at least one block in the first image according to the first gray scale map and the first gray scale gradient map.
According to an aspect of the invention, the first sharpness indicator is:
wherein G ismnIs the average of the maximum and minimum gray values in the block, G0Is the average of all gray values in the block, gmnIs the average of the maximum and minimum gray gradient values in the block, g0Is the average of all the gray scale gradient values in the block.
According to an aspect of the invention, the first threshold condition is that the first sharpness index among the blocks in the first image whose ratio exceeds a preset ratio is less than or equal to a preset sharpness threshold.
According to an aspect of the invention, the first threshold condition is that the sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
According to one aspect of the invention, the processor maps the second image to a plurality of the blocks in the first image according to a block division mode consistent with the first image;
the processor converts the second image to obtain a second gray scale map, and obtains a second gray scale gradient map according to the second gray scale image;
and the processor acquires a second sharpness index of at least one block in the second image according to the second gray scale map and the second gray scale gradient map.
According to an aspect of the invention, the processor obtains the number of pixels in at least one block of the second image used for complementing a corresponding block in the first image based on the difference between the first sharpness index and the second sharpness index.
According to one aspect of the invention, when the difference between the first sharpness index and the second sharpness index of a block is greater than a preset sharpness difference upper limit, all pixels in the block of the first image are replaced by pixels of a corresponding block in the second image.
According to one aspect of the invention, the second lens is deactivated when the difference between the first sharpness index and the second sharpness index is less than a predetermined lower sharpness difference limit.
According to one aspect of the invention, the second image is mapped to the block division mode on the first image by using a coordinate transformation mode of pre-stored calibration information or an image feature matching mode.
According to an aspect of the invention, the size of at least one of the blocks of the first image is different from the sizes of the other blocks.
To achieve the above object, the present invention provides an image pickup apparatus including:
a multi-lens group including a first lens group and a second lens group, a memory for storing images, and a processor, wherein the processor is configured to:
enabling the first lens group to capture a first image for storage in a memory;
dividing the first image into a plurality of blocks;
after the first image is converted into a first gray scale image, calculating a corresponding first gray scale gradient image;
calculating a first sharpness index of at least one block through the first gray scale map and the first gray scale gradient map;
enabling the second lens group to capture a second image in memory in response to the first sharpness indicator not meeting a preset first threshold condition; and
determining a number of pixels in the second image in the at least one block for compositing with the first image based on the first sharpness indicator.
According to one scheme of the invention, the first image is divided into different sizes or shapes, so that the parts with important features in the first image are divided according to the importance degree, the method can accurately optimize the definition of the important features, the calibration precision of the image is improved, and the quality of the image after calibration is ensured. Meanwhile, the calculation amount of the method can be further reduced through the division mode, and the method is favorable for improving the calibration efficiency of the image.
According to one scheme of the invention, the sharpness index is generated by simultaneously adopting the gray level image and the gray level gradient image, whether the first lens group is blurred or not is evaluated based on the sharpness index, when part or all of the first image is blurred, the second lens with the focal length range different from that of the first lens group is started to shoot the second image, and the blurred area in the first image is replaced or synthesized through the second image, so that the zooming effect can be realized. In addition, by means of proper block division, the operation time and the power consumption are reduced, and only a part of blocks in the first image can be synthesized by the method without synthesizing the whole image. Meanwhile, the fuzzy area which needs to be replaced or synthesized is processed, so that the calculated amount of the method is greatly reduced and the calculation efficiency is improved under the condition of ensuring the image quality. In addition, according to the specific size of the sharpness index, it is determined how many pixels of the block in the second image need to be replaced or synthesized into the block in the first image, and the smaller the sharpness is, the more the pixels need to be replaced. Therefore, the number of pixels to be supplemented into the first image can be effectively and accurately judged through the sharpness index, so that the calibration process of the image is more refined, the calculated amount is less, the accuracy of the supplemented pixels is ensured, and the calibration quality of the image is improved.
According to an aspect of the present invention, the sharpness index is defined based on a difference between an extreme value of the gray scale values and the gray scale gradient values in the block and the overall average value. Therefore, the error of the result of the multiple calculation of the sharpness index is smaller, so that the method can be better applied to the judgment of whether the block is fuzzy due to defocusing, the accuracy of the judgment process of the method is improved, the calculation result of the number of pixels needing to be supplemented is also improved, and the method is further beneficial to ensuring the calibration result of the image.
According to one scheme of the invention, the calibration of the first image can be flexibly and conveniently realized aiming at different scenes or requirements by setting different first threshold conditions, so that the flexibility of the calibration method is improved, and the method can be more effectively applied to various scenes. In addition, the accuracy of image calibration can be effectively influenced according to different first threshold conditions, so that the method can be suitable for processing images of different types and with different calibration accuracy requirements through the arrangement, and the calibration effectiveness of the method is improved.
According to the scheme of the invention, the second sharpness index is obtained, so that the sharpness of the image shot by the second lens group can be further calculated under the condition of better zooming effect, and the image can be more accurately compared with the first image, so that the replacement or synthesis of the image in the process of supplementing the pixel is more accurate, and the replacement is more effective particularly on some blocks with obviously larger differences.
Drawings
FIG. 1 schematically represents a block diagram of steps according to an embodiment of the present invention;
fig. 2 is a block diagram schematically showing a configuration of a photographing apparatus according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
In describing embodiments of the present invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship that is based on the orientation or positional relationship shown in the associated drawings, which is for convenience and simplicity of description only, and does not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus, the above-described terms should not be construed as limiting the present invention.
The present invention is described in detail below with reference to the drawings and the specific embodiments, which are not repeated herein, but the embodiments of the present invention are not limited to the following embodiments.
As shown in fig. 1, according to an embodiment of the present invention, a photographing correction method for a multi-lens group of the present invention includes:
s1, enabling one lens group in a multi-lens group to take a first image, and generating a first sharpness index based on the first image;
s2, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting another lens group in the multi-lens group to take a second image;
and S3, acquiring the number of pixels which are added into the first image by the second image based on the first sharpness index, and finishing the correction of the first image.
The method of the present invention is implemented by a multi-shot lens group, which can be used for two, three or more lens groups. To explain the present invention in detail, the present invention will be explained in detail with a multi-lens having two lens groups.
According to an embodiment of the present invention, step S1 includes:
s11, a first image is shot through one lens group in the multi-shooting lens group, and the first image is divided into a plurality of blocks. In the present embodiment, the multi-lens group includes a first lens group 11 and a second lens group 12. In the present embodiment, when image capturing is performed, the first lens group 11 first performs image capturing by default, and a first image can be acquired through the first lens group 11. In the present embodiment, the first image is divided into a plurality of blocks according to a mesh division manner, and the division of the blocks may be performed according to the image characteristics of the first image, for example, when the first image is divided into blocks, the blocks may be divided according to different image characteristics such as color distribution and face recognition. Meanwhile, in the process of dividing the first image, the part in which the user is interested can be divided more finely, and the parts such as edges and the like in which the user is not interested can be divided more coarsely, for example, when there are four parts in which the user is interested on the first image, the blocks containing the four parts in which the user is interested are divided more finely, and it is not necessary to define each block to be the same size. Therefore, in the plurality of blocks of the first image, the size of at least one block is different from the sizes of the rest blocks. That is, the size of at least two blocks in the plurality of blocks of the first image is different, and the size of one block is larger or smaller than that of another block. By dividing the first image into different sizes or shapes, the parts with important features in the first image are divided according to the importance degree, the method can accurately optimize the definition of the important features, and further improve the calibration precision of the image and ensure the quality of the image after calibration. Meanwhile, the calculation amount of the method can be further reduced through the division mode, and the method is favorable for improving the calibration efficiency of the image.
And S12, converting the first image to obtain a first gray-scale image, and obtaining a first gray-scale gradient image according to the first gray-scale image. In the present embodiment, the first image is converted into the first gray scale map, and the gradient of the first gray scale map is calculated by a convolution algorithm such as laplace operator or a difference method, thereby obtaining the first gray scale gradient map. And stores the resulting first gray scale map and first gray scale gradient map,
s13, acquiring a first sharpness index of at least one block in the first image according to the first gray-scale image and the first gray-scale gradient image. In the present embodiment, the first sharpness index is calculated for each block in the first image according to the acquired first gray scale map and the acquired first gray scale gradient map. And after the first sharpness index is obtained, whether the first image is blurred to the extent that image synthesis is needed can be evaluated.
Through the arrangement, the sharpness index is generated by simultaneously adopting the gray level image and the gray level gradient image, whether the first lens group is blurred or not is evaluated based on the sharpness index, when part or all of the first image is blurred, the second lens with the focal length range different from that of the first lens group is started to shoot the second image, and the blurred area in the first image is replaced or synthesized through the second image, so that the zooming effect can be realized. In addition, by means of proper block division, the operation time and the power consumption are reduced, and only a part of blocks in the first image can be synthesized by the method without synthesizing the whole image. Meanwhile, the fuzzy area which needs to be replaced or synthesized is processed, so that the calculated amount of the method is greatly reduced and the calculation efficiency is improved under the condition of ensuring the image quality. In addition, according to the specific size of the sharpness index, it is determined how many pixels of the block in the second image need to be replaced or synthesized into the block in the first image, and the smaller the sharpness is, the more the pixels need to be replaced. Therefore, the number of pixels to be supplemented into the first image can be effectively and accurately judged through the sharpness index, so that the calibration process of the image is more refined, the calculated amount is less, the accuracy of the supplemented pixels is ensured, and the calibration quality of the image is improved.
According to an embodiment of the invention, the first sharpness index is:
wherein G ismnIs the average of the maximum and minimum gray values in the block, G0Is the average of all gray values in the block, gmnIs the average of the maximum and minimum gray gradient values in the block, g0Is the average of all the gray scale gradient values in the block. In the present embodiment, a smaller value of the first sharpness index indicates a blurry edge, and it is possible to evaluate whether the second lens group 12 needs to be enabled to capture the second image to correct the first image by comparing the first sharpness index with a relative or fixed comparison criterion. Of course, the definition of the first sharpness metric may vary and is not limited to using the above-defined expression.
With the above arrangement, the definition of the sharpness index is based on the difference between the extreme values of the gray scale values and the gray scale gradient values in the block and the overall average value. Therefore, the error of the result of the multiple calculation of the sharpness index is smaller, so that the method can be better applied to the judgment of whether the block is fuzzy due to defocusing, the accuracy of the judgment process of the method is improved, the calculation result of the number of pixels needing to be supplemented is also improved, and the method is further beneficial to ensuring the calibration result of the image.
According to an embodiment of the present invention, in step S2, when the first sharpness index does not satisfy the preset first threshold condition, the second lens module 12 is directly activated to capture a second image. In this embodiment, the first threshold condition is that the first sharpness index in the block with the ratio exceeding the preset ratio in the first image is smaller than or equal to the preset sharpness threshold. In the present embodiment, one of the determination conditions for enabling the second lens group 12 to perform photographing is specified, the preset ratio may be set to 20%, 40%, or 60%, etc., and the preset sharpness threshold may be set to 15%, 30%, or 50%, etc. In the present embodiment, the value of the preset ratio and the preset sharpness threshold in the determination condition are adjusted according to the situation of the specific shooting scene, for example, when the sharpness of more than 20% of the blocks is less than or equal to 15% in the case of shooting a close-range still portrait and the like requiring higher sharpness, the second lens group is enabled to replace the blocks with insufficient sharpness. And under the condition that high definition is not needed, such as long-range shooting, the second lens group can be started to replace only in the area exceeding 60 percent and under the condition that the sharpness is less than 50 percent.
According to another embodiment of the present invention, the first threshold condition may be further set such that a sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold. In the present embodiment, the second determination condition for enabling the second lens group to capture is defined, and for example, in the case of dividing the first image into 144 blocks, if the sum of the first sharpness indexes of all the blocks is greater than the sum threshold set by 50, 60, or 70, etc., it is determined that the entire image needs to be corrected, and all the blocks are replaced or synthesized in a correlated manner.
Through the setting, the calibration of the first image can be flexibly and conveniently realized aiming at different scenes or requirements by setting different first threshold conditions, so that the flexibility of the calibration method is improved, and the method can be more effectively applied to various scenes. In addition, the accuracy of image calibration can be effectively influenced according to different first threshold conditions, so that the method can be suitable for processing images of different types and with different calibration accuracy requirements through the arrangement, and the calibration effectiveness of the method is improved.
In the present embodiment, the step of taking the second image by the other lens group (i.e., the second lens group) of the multi-taking lens group in step S2 further includes:
and S21, mapping the second image to a plurality of blocks in the first image according to a block division mode consistent with the first image. In this embodiment, the second image performs coordinate matching between the first image and the second image by performing coordinate transformation using pre-stored calibration information, and maps the second image to each of the blocks divided on the first image in a block division manner of the first image. According to another embodiment of the present invention, the step of mapping the second image into the block of the first image may be performed according to an actual image feature matching manner.
And S22, converting the second image to obtain a second gray scale map, and obtaining a second gray scale gradient map according to the second gray scale image.
And S23, acquiring a second sharpness index of at least one block in the second image according to the second gray map and the second gray gradient map. Through obtaining the second sharpness index, can guarantee under the circumstances of better zooming effect, can further calculate the sharpness of the image that the second lens group shot, and then can realize more accurate and carry out the contrast with first image, and then make the replacement or the synthesis of image interpolation pixel in-process more accurate, it is more effective especially to replace some blocks that the gap is obviously great.
According to an embodiment of the present invention, in step S3, the number of pixels of the second image added to the first image is obtained based on the first sharpness index, and the correction of the first image is completed. Therefore, how many pixels in the second image are to be filled in the first image is determined according to the first sharpness index in the blurred block of the first image. In this embodiment, in the process of supplementing pixels to the first image, the pixels are supplemented at a rate of b1 pixels from the corresponding tile in the second image inserted every a1 pixels in the tile, and each replaced pixel is inserted in the tile at an interval of a1/b1 in the first image. For example, in a tile of size a1 × a2 in the first image, the number of pixels in the corresponding tile of the second image is b1 × b2 and the number of original pixels in the tile of the first image is a1 × a2-b1 × b 2. Generally, the image feature in the second image only includes a part of the image feature in the first image, so there may be some blocks in the first image that cannot be compensated by the second lens group.
In this embodiment, in step S3, the number of pixels in at least one block of the second image used to fill the corresponding block in the first image is obtained based on the difference between the first sharpness index and the second sharpness index. In this embodiment, pixel replacement is performed according to a difference between a first sharpness index of the first image and a second sharpness index of the second image in each block, and the larger the difference is, the larger the number of pixels to be replaced is.
In this embodiment, when the difference between the first sharpness index and the second sharpness index of a block is greater than the predetermined upper sharpness difference limit, all pixels in the block of the first image are replaced with pixels of a corresponding block in the second image. In this embodiment, an upper limit of the difference between the first sharpness index and the second sharpness index may be specified, for example, a value of 50%, 100%, etc., and if this upper limit is exceeded, it may be considered that the block is unusable in the first image, i.e., all pixels of the block in the first image are replaced with pixels in the second image, and below this upper limit, the number of pixels to be replaced may be reduced stepwise in a linear manner.
In this embodiment, when the difference between the first sharpness index and the second sharpness index is less than the predetermined lower limit of sharpness difference, the lens group capturing the second image is disabled. In the present embodiment, a lower limit of the difference between the first sharpness index and the second sharpness index, for example, a value of 10% or the like, may be specified, and if below the lower limit, it indicates that the sharpness of the first image is sufficient, replacement is temporarily not necessary, the second lens group may be disabled, only shooting with the first lens group is utilized to save resources, and power may be saved.
As shown in fig. 2, according to an embodiment of the present invention, a multi-lens group of the present invention includes: a first lens 11 for acquiring a first image, a second lens 12 for acquiring a second image. In this embodiment, the focal length range of the first lens 11 is different from the focal length range of the second lens 12, that is, the focal length range of the first lens 11 is different from the focal length range of the second lens 12, and the focal length range of the second lens 12 is longer than the focal length range of the first lens 11, so that the second lens 12 can still clearly image under the condition that the first lens 11 cannot clearly image. Of course, the present invention can also adopt more lens groups for image calibration, only the lens groups with different focal length ranges need to be selected, the image calibration process is the same as the above process, and the details are not repeated herein. The first lens 11 and the second lens may each employ a zoom lens. In the present embodiment, the first lens 11 is a wide-angle lens or a normal lens.
As shown in fig. 2, according to an embodiment of the present invention, a photographing apparatus of the present invention includes: the multi-taking lens group 1, a processor 2 electrically connected with the multi-taking lens group 1, a memory 3 electrically connected with the processor 2, a display 4 for displaying images taken by the multi-taking lens group 1, and a communication interface 5. In the embodiment, the first image or the second image captured in the multi-camera lens group 1 needs to be processed by the processor 2 and then transmitted to the memory 3 for storage, so that the image calibration process can be conveniently called, and the processing speed of the invention is improved. In the present embodiment, the communication interface 5 may employ an I/O interface.
As shown in fig. 2, according to one embodiment of the present invention, the multi-lens group includes a first lens 11 for acquiring a first image, and a second lens 12 for acquiring a second image. In the embodiment, the focal length range of the first lens 11 is different from the focal length range of the second lens 12, so that the second lens 12 can still clearly image under the condition that the first lens 11 cannot clearly image. Of course, the present invention can also adopt more lens groups for image calibration, only the lens groups with different focal length ranges need to be selected, the image calibration process is the same as the above process, and the details are not repeated herein. The first lens 11 and the second lens may each employ a zoom lens. In the present embodiment, the first lens 11 is a wide-angle lens or a normal lens.
In this embodiment, a first sharpness index is generated based on the first image, and whether the first sharpness index meets a preset first threshold condition is determined, if not, the second lens 12 is enabled to capture the second image;
in the present embodiment, the number of pixels of the second image to be added to the first image is acquired based on the first sharpness index, and the first image is corrected. .
As shown in fig. 2, according to an embodiment of the present invention, the memory 3 stores a first image captured by the first lens 11, and the processor 2 divides the first image into a plurality of blocks; in the embodiment, the processor 2 converts the first image to obtain a first gray scale image, and obtains a first gray scale gradient map according to the first gray scale image; in this embodiment, the processor 2 obtains a first sharpness index of at least one block in the first image according to the first gray scale map and the first gray scale gradient map.
According to an embodiment of the invention, the first sharpness index is:
wherein G ismnIs the average of the maximum and minimum gray values in the block, G0Is the average of all gray values in the block, gmnIs the average of the maximum and minimum gray gradient values in the block, g0Is the average of all the gray scale gradient values in the block.
According to an embodiment of the present invention, the first threshold condition is that the first sharpness index in a block of the first image with a ratio exceeding a preset ratio is less than or equal to a preset sharpness threshold.
According to an embodiment of the present invention, the first threshold condition is that a sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
According to one embodiment of the invention, the processor maps the second image to a plurality of blocks in the first image according to a block division mode consistent with the first image;
in this embodiment, the processor 2 converts the second image to obtain a second gray scale map, and obtains a second gray scale gradient map according to the second gray scale image;
in this embodiment, the processor 2 obtains a second sharpness index of at least one block in the second image according to the second gray map and the second gray gradient map.
According to an embodiment of the present invention, the processor 2 obtains the number of pixels in at least one block of the second image for complementing a corresponding block in the first image based on the difference between 2 the first sharpness index and 2 the second sharpness index.
According to one embodiment of the present invention, when the difference between the first sharpness index and the second sharpness index of a block is greater than the predetermined upper sharpness difference limit, all pixels in the block of the first image are replaced with pixels of a corresponding block of the second image. When the difference between the first sharpness index and the second sharpness index is less than the predetermined lower sharpness difference, the second lens 12 is deactivated.
According to one embodiment of the invention, the second image is mapped to the block division mode on the first image by using the pre-stored calibration information to perform coordinate transformation or by using the image feature matching mode.
According to an embodiment of the present invention, at least one of the blocks of the first image has a size different from sizes of the other blocks.
According to an embodiment of the present invention, an imaging apparatus of the present invention includes:
a multi-lens group 1 including a first lens group 11 and a second lens group 12, a memory 3 for storing images, and a processor 2, wherein the processor 2 is configured to: the first lens group 11 is enabled to take a first image stored in the memory 3. In this embodiment, the first image is divided into a plurality of blocks. And after the first image is converted into the first gray-scale image, calculating a corresponding first gray-scale gradient image. And calculating a first sharpness index of the at least one block through the first gray-scale map and the first gray-scale gradient map. Enabling the second lens group 12 to capture a second image memory in the memory 3 in response to the first sharpness indicator not meeting a preset first threshold condition; and determining the number of pixels in a second image for compositing with the first image in the at least one block based on the first sharpness indicator.
The foregoing is merely exemplary of particular aspects of the present invention and devices and structures not specifically described herein are understood to be those of ordinary skill in the art and are intended to be implemented in such conventional ways.
The above description is only one embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A method for correcting the photographing of a multi-photographing lens group comprises the following steps:
s1, starting one lens group in a multi-lens group to take a first image, dividing the first image into a plurality of blocks, and generating a first sharpness index based on the first image of the divided blocks; dividing the first image into a plurality of blocks according to the image characteristics of the first image in a grid division mode, wherein the size of at least one block in the plurality of blocks of the first image is different from the sizes of the rest blocks, and the block with the small size is positioned on the part, interested by a user, of the first image;
s2, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting another lens group in the multi-lens group to take a second image; mapping a plurality of blocks in the first image according to the second image in a block division mode consistent with the first image, and acquiring a second sharpness index of at least one block in the second image;
s3, acquiring the number of pixels which are added into the first image by the second image based on the first sharpness index, and finishing the correction of the first image; acquiring the number of pixels used for supplementing a corresponding block in the first image in at least one block of the second image based on the difference value of the first sharpness index and the second sharpness index, wherein in the process of supplementing the pixels to the first image, b1 pixels from the corresponding block in the second image are inserted into the block every a1 pixels for pixel supplementation, and each replaced pixel in the first image is inserted into the block at the interval of a1/b 1;
in step S2, the first threshold condition is that the first sharpness index in a block of the first image with a ratio exceeding a preset ratio is less than or equal to a preset sharpness threshold; or, the first threshold condition is that the sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
2. The image correction method according to claim 1, wherein step S1 includes:
converting the first image to obtain a first gray-scale image, and obtaining a first gray-scale gradient image according to the first gray-scale image;
and acquiring the first sharpness index of at least one block in the first image according to the first gray-scale image and the first gray-scale gradient image.
3. The photography correction method of claim 1 or 2, wherein the first sharpness indicator is:
wherein G ismnIs the average of the maximum and minimum gray values in the block, G0Is the average of all gray values in the block, gmnIs the maximum gray gradient value and the minimum gray gradient in the blockAverage of the values, g0Is the average of all the gray scale gradient values in the block.
4. The photographing correction method of claim 1, wherein the step of taking the second image through the other lens group of the multi-lens group in the step S2 further comprises:
converting the second image to obtain a second gray scale map, and obtaining a second gray scale gradient map according to the second gray scale image;
and acquiring a second sharpness index of at least one block in the second image according to the second gray map and the second gray gradient map.
5. The image correction method of claim 4, wherein in step S3, when the difference between the first sharpness index and the second sharpness index of a block is greater than a predetermined upper limit of sharpness difference, all pixels in the block of the first image are replaced with pixels of a corresponding block in the second image;
and when the difference value of the first sharpness index and the second sharpness index is smaller than the preset lower sharpness difference value, stopping the lens group taking the second image.
6. The image correction method of claim 4, wherein in step S21, the second image is mapped to the block division on the first image by coordinate transformation using pre-stored calibration information or by image feature matching.
7. A method for correcting the photographing of a multi-photographing lens group comprises the following steps:
s1, starting a first lens group of a multi-camera lens group to shoot a first image, and dividing the first image into a plurality of blocks; dividing the first image into a plurality of blocks according to the image characteristics of the first image in a grid division mode, wherein the size of at least one block in the plurality of blocks of the first image is different from the sizes of the rest blocks, and the block with the small size is positioned on the part, interested by a user, of the first image;
s2, converting the first image into a first gray-scale image, and calculating a corresponding first gray-scale gradient image;
s3, calculating a first sharpness index of at least one block through the first gray-scale image and the first gray-scale gradient image;
s4, enabling a second lens group of the multi-shooting lens group to shoot a second image in response to the first sharpness index not meeting a preset first threshold condition; mapping a plurality of blocks in the first image according to the second image in a block division mode consistent with the first image, and acquiring a second sharpness index of at least one block in the second image; and
determining a number of pixels in the second image in the at least one block for compositing with the first image based on the first sharpness indicator; acquiring the number of pixels used for supplementing a corresponding block in the first image in at least one block of the second image based on the difference value of the first sharpness index and the second sharpness index, wherein in the process of supplementing the pixels to the first image, b1 pixels from the corresponding block in the second image are inserted into the block every a1 pixels for pixel supplementation, and each replaced pixel in the first image is inserted into the block at the interval of a1/b 1;
the first threshold condition is that the first sharpness index in a block with a ratio exceeding a preset ratio in the first image is smaller than or equal to a preset sharpness threshold; or, the first threshold condition is that the sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
8. A photographing apparatus, comprising: a multi-shot lens group (1), the multi-shot lens group (1) including a first lens (11) for taking a first image, a second lens (12) for obtaining a second image;
the focal length range of the first lens (11) is different from the focal length range of the second lens (12);
dividing the first image into a plurality of blocks according to the image characteristics of the first image in a grid division mode, wherein the size of at least one block in the plurality of blocks of the first image is different from the sizes of the rest blocks, and the block with the small size is positioned on the part, interested by a user, of the first image;
generating a first sharpness index based on the first image, judging whether the first sharpness index meets a preset first threshold condition, and if not, starting the second lens (12) to shoot the second image;
the first threshold condition is that the first sharpness index in blocks with the ratio exceeding a preset ratio in the first image is smaller than or equal to a preset sharpness threshold, or the first threshold condition is that the sum of the first sharpness indexes of a plurality of blocks is smaller than or equal to a preset sharpness sum threshold;
the processor (2) maps a plurality of blocks in the first image according to the second image in a block division mode consistent with the first image, and obtains a second sharpness index of at least one block in the second image;
acquiring the number of pixels of the second image to be supplemented to the first image based on the first sharpness index, and correcting the first image; wherein the processor (2) obtains the number of pixels in at least one block of the second image for filling in the corresponding block in the first image based on the difference between the first sharpness index and the second sharpness index, and during the process of filling in the pixels to the first image, the filling in of the pixels is replaced by inserting b1 pixels from the corresponding block in the second image at intervals of a1 pixels in the block, and each replaced pixel in the first image is inserted in the block at intervals of a1/b 1.
9. The camera device according to claim 8, further comprising: a processor (2) electrically interconnected with the first lens (11) and the second lens (12), respectively, a memory (3) electrically interconnected with the processor (2), a display (4) for displaying images, and a communication interface (5);
the memory (3) stores a first image shot by the first lens (11), and the processor (2) divides the first image into a plurality of blocks;
the processor (2) converts the first image to obtain a first gray-scale image, and obtains a first gray-scale gradient image according to the first gray-scale image;
the processor (2) obtains the first sharpness indicator of at least one of the blocks in the first image from the first gray scale map and the first gray scale gradient map.
10. The camera device according to claim 9,
the processor (2) converts the second image to obtain a second gray scale map, and obtains a second gray scale gradient map according to the second gray scale image;
the processor (2) obtains a second sharpness indicator of at least one of the blocks in the second image according to the second gray map and the second gray gradient map.
11. The camera device as claimed in claim 10, wherein when a difference between the first sharpness index and the second sharpness index of a block is greater than a predetermined upper limit of sharpness difference, all pixels in the block of the first image are replaced with pixels of a corresponding block in the second image;
and when the difference value of the first sharpness index and the second sharpness index is smaller than a preset sharpness difference lower limit, the second lens (12) is deactivated.
12. The camera device according to claim 10, wherein the second image is mapped to the block division on the first image by performing coordinate transformation using pre-stored calibration information or by performing image feature matching.
13. A photographing apparatus, comprising:
a multi-lens group (1) comprising a first lens group (11) and a second lens group (12), a memory (3) for storing images and a processor (2), wherein the processor (2) is configured to:
enabling the first lens group (11) to take a first image stored in a memory (3);
dividing the first image into a plurality of blocks; dividing the first image into a plurality of blocks according to the image characteristics of the first image in a grid division mode, wherein the size of at least one block in the plurality of blocks of the first image is different from the sizes of the rest blocks, and the block with the small size is positioned on the part, interested by a user, of the first image;
after the first image is converted into a first gray scale image, calculating a corresponding first gray scale gradient image;
calculating a first sharpness index of at least one block through the first gray scale map and the first gray scale gradient map;
enabling the second lens group (12) to take a second image memory in a memory (3) in response to the first sharpness indicator not meeting a preset first threshold condition; the processor (2) maps a plurality of blocks in the first image of the second image according to a block division mode consistent with that of the first image, and obtains a second sharpness index of at least one block in the second image; and
determining a number of pixels in the second image in the at least one block for compositing with the first image based on the first sharpness indicator; wherein the processor (2) obtains the number of pixels in at least one tile of the second image for filling a corresponding tile in the first image based on the difference between the first sharpness index and the second sharpness index, and during the pixel filling of the first image, the filling of pixels is replaced by inserting b1 pixels from the corresponding tile in the second image at intervals of a1 pixels in the tile, and each replaced pixel in the first image is inserted in the tile at intervals of a1/b 1;
the first threshold condition is that the first sharpness index in a block with a ratio exceeding a preset ratio in the first image is smaller than or equal to a preset sharpness threshold; or, the first threshold condition is that the sum of the first sharpness indexes of the plurality of blocks is less than or equal to a preset sharpness sum threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811307126.8A CN109257540B (en) | 2018-11-05 | 2018-11-05 | Photographing correction method of multi-photographing lens group and photographing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811307126.8A CN109257540B (en) | 2018-11-05 | 2018-11-05 | Photographing correction method of multi-photographing lens group and photographing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109257540A CN109257540A (en) | 2019-01-22 |
| CN109257540B true CN109257540B (en) | 2021-06-18 |
Family
ID=65044116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811307126.8A Active CN109257540B (en) | 2018-11-05 | 2018-11-05 | Photographing correction method of multi-photographing lens group and photographing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109257540B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112188096A (en) * | 2020-09-27 | 2021-01-05 | 北京小米移动软件有限公司 | Photographing method and device, terminal and storage medium |
| CN112532884B (en) * | 2020-11-27 | 2022-10-14 | 维沃移动通信有限公司 | Identification method and device and electronic equipment |
| CN115348390A (en) * | 2022-08-23 | 2022-11-15 | 维沃移动通信有限公司 | Shooting method and shooting device |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1177298C (en) * | 2002-09-19 | 2004-11-24 | 上海交通大学 | Multiple focussing image fusion method based on block dividing |
| US7073908B1 (en) * | 2005-01-11 | 2006-07-11 | Anthony Italo Provitola | Enhancement of depth perception |
| CN103186894B (en) * | 2013-03-22 | 2015-10-07 | 南京信息工程大学 | A kind of multi-focus image fusing method of self-adaptation piecemeal |
| JP6241192B2 (en) * | 2013-10-18 | 2017-12-06 | 株式会社リコー | Image processing apparatus, image processing system, image processing method, program, and recording medium |
| CN103702024B (en) * | 2013-12-02 | 2017-06-20 | 宇龙计算机通信科技(深圳)有限公司 | Image processing apparatus and image processing method |
| CN104299198A (en) * | 2014-10-14 | 2015-01-21 | 嘉应学院 | Fast image defogging method based on dark channels of pixels |
| CN104363379B (en) * | 2014-11-28 | 2017-06-06 | 广东欧珀移动通信有限公司 | The method and terminal taken pictures using different focal camera |
| CN106331497B (en) * | 2016-08-31 | 2019-06-11 | 宇龙计算机通信科技(深圳)有限公司 | A kind of image processing method and terminal |
| CN107872614A (en) * | 2016-09-27 | 2018-04-03 | 中兴通讯股份有限公司 | A kind of image pickup method and filming apparatus |
| CN106709882A (en) * | 2016-12-16 | 2017-05-24 | 努比亚技术有限公司 | Image fusion method and device |
-
2018
- 2018-11-05 CN CN201811307126.8A patent/CN109257540B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109257540A (en) | 2019-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107948519B (en) | Image processing method, device and equipment | |
| CN111246089B (en) | Jitter compensation method and apparatus, electronic device, computer-readable storage medium | |
| CN108055452B (en) | Image processing method, device and equipment | |
| CN107945105B (en) | Background blurring processing method, device and equipment | |
| US9288392B2 (en) | Image capturing device capable of blending images and image processing method for blending images thereof | |
| CN106210501B (en) | Image synthesizing method and image processing apparatus | |
| CN107493432B (en) | Image processing method, image processing device, mobile terminal and computer readable storage medium | |
| CN107509031B (en) | Image processing method, image processing device, mobile terminal and computer readable storage medium | |
| US9076214B2 (en) | Image acquisition apparatus and image processing apparatus using selected in-focus image data | |
| CN111028190A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| KR20160090373A (en) | Photographing method for dual-camera device and dual-camera device | |
| CN111726521B (en) | Photographing method and photographing device of terminal and terminal | |
| JP6452360B2 (en) | Image processing apparatus, imaging apparatus, image processing method, and program | |
| CN109257540B (en) | Photographing correction method of multi-photographing lens group and photographing device | |
| CN109685853B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN108053438B (en) | Depth of field acquisition method, device and equipment | |
| CN107704798B (en) | Image blurring method and device, computer readable storage medium and computer device | |
| CN110866486B (en) | Subject detection method and apparatus, electronic device, and computer-readable storage medium | |
| CN109559353B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN107872631B (en) | Image shooting method and device based on double cameras and mobile terminal | |
| CN113313661A (en) | Image fusion method and device, electronic equipment and computer readable storage medium | |
| CN109559352B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109951641B (en) | Image shooting method and device, electronic equipment and computer readable storage medium | |
| CN111932587A (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN111385461B (en) | Panoramic shooting method and device, camera and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |