CN110310251B - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN110310251B CN110310251B CN201910593850.XA CN201910593850A CN110310251B CN 110310251 B CN110310251 B CN 110310251B CN 201910593850 A CN201910593850 A CN 201910593850A CN 110310251 B CN110310251 B CN 110310251B
- Authority
- CN
- China
- Prior art keywords
- image
- images
- target
- target image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 230000004927 fusion Effects 0.000 claims abstract description 41
- 238000006243 chemical reaction Methods 0.000 claims abstract description 31
- 238000013507 mapping Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims description 23
- 238000004364 calculation method Methods 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 abstract description 10
- 238000003384 imaging method Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The embodiment of the disclosure discloses an image processing method and device. One embodiment of the method comprises: acquiring at least two frames of images, wherein the at least two frames of images present the same scene; selecting a target image from at least two frame images; mapping image coordinates of a non-target image in at least two frames of images to a target image to obtain an image after coordinate conversion; fusing the image after coordinate conversion and the target image; based on the fusion result, a fused image is generated. The embodiment can reduce the noise generated in the image shooting process and improve the image imaging effect.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to an image processing method and device.
Background
With the development of scientific technology and the popularization of image processing technology, more and more users enjoy taking images by using terminal equipment. In general, when picture taking is performed in, for example, an outdoor scene, noise exists in some of the taken images due to the influence of the illumination intensity, or the influence of the parameters of the photographing apparatus.
In order to obtain higher imaging quality, improve the definition of an image, and the like, noise reduction processing or denoising processing is generally performed on an image with noise. In the related art, image denoising is generally performed by using a method such as median filtering.
Disclosure of Invention
The embodiment of the disclosure provides an image processing method and device.
In a first aspect, an embodiment of the present disclosure provides an image processing method, including: acquiring at least two frames of images, wherein the at least two frames of images present the same scene; selecting a target image from at least two frame images; mapping image coordinates of a non-target image in at least two frames of images to a target image to obtain an image after coordinate conversion; fusing the image after coordinate conversion and the target image; based on the fusion result, a fused image is generated.
In some embodiments, mapping image coordinates of a non-target image of the at least two frame images into the target image comprises: extracting key points of each frame of image in at least two frames of images; for each frame of image in the non-target image, matching the key points of the image with the key points of the target image; based on the matching result, generating a homography matrix for converting the image coordinates of the key points in the image into the target image; image coordinates of the image are mapped into a target image based on the homography matrix.
In some embodiments, the image fusion of the coordinate-transformed image and the target image comprises: calculating the average value of pixel values at the same coordinate position in the image and the target image after the coordinate conversion; and generating a fused image based on the fusion result, comprising: based on the average calculation result, a fused image is generated.
In some embodiments, generating the fused image based on the mean calculation comprises: determining a processed image obtained after calculation based on the average value; for each frame of image in the image after coordinate conversion and the target image, determining the difference between the image and the processed image; setting a weight to the image based on the difference result; fusing the image after coordinate conversion and the target image based on the obtained weight of each frame of image, the pixel value of each frame of image in the image after coordinate conversion and the target image; based on the fusion result, a fused image is generated.
In some embodiments, determining the difference between the image and the processed image comprises: comparing the pixel value of each pixel in the image with the pixel value of a pixel at the same image coordinate position in the processed image, and generating a difference value between the pixel of the image and the pixel of the processed image based on the comparison result; and setting a weight to the image based on the difference result, including: based on the difference, a weight is set for the image.
In a second aspect, an embodiment of the present disclosure provides an image processing apparatus, including: an acquisition unit configured to acquire at least two frames of images, wherein the at least two frames of images present the same scene; a selection unit configured to select a target image from at least two frame images; a mapping unit configured to map image coordinates of a non-target image of the at least two frame images into a target image to obtain a coordinate-converted image; a fusion unit configured to fuse the coordinate-converted image and the target image; a production unit configured to generate a fused image based on the fusion result.
In some embodiments, the mapping unit is further configured to: extracting key points of each frame of image in at least two frames of images; for each frame of image in the non-target image, matching the key points of the image with the key points of the target image; based on the matching result, generating a homography matrix for converting the image coordinates of the key points in the image into the target image; image coordinates of the image are mapped into a target image based on the homography matrix.
In some embodiments, a fusion unit, comprises: a calculation subunit configured to perform average value calculation on pixel values at the same coordinate position in the coordinate-converted image and the target image; and a generating unit comprising: a generation subunit configured to generate a fused image based on the average value calculation result.
In some embodiments, generating the sub-unit comprises: a determination module configured to determine a processed image calculated based on the average value; a setting module configured to determine, for each frame of the image after the coordinate conversion and the target image, a difference between the image and the processed image; setting a weight to the image based on the difference result; a generating module configured to fuse the coordinate-converted image and the target image based on the obtained weight of each frame image, the pixel value of each frame image in the coordinate-converted image and the target image; based on the fusion result, a fused image is generated.
In some embodiments, the setup module is further configured to: comparing the pixel value of each pixel in the image with the pixel value of a pixel at the same image coordinate position in the processed image, and generating a difference value between the pixel of the image and the pixel of the processed image based on the comparison result; based on the difference, a weight is set for the image.
In a third aspect, an embodiment of the present disclosure provides a terminal device, including: one or more processors; storage means for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any implementation of the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, which computer program, when executed by a processor, implements the method as described in any of the implementations of the first aspect.
According to the image processing method and the image processing device provided by the embodiment of the disclosure, at least two frames of images shot in the same scene are obtained, then the at least two frames of images are fused, and the fused images are generated based on the fusion result, so that the noise generated in the image shooting process can be reduced, and the image imaging effect is improved; and performing image coordinate mapping on at least two frames of images to enable the same positions of the images to correspond, so that the image fusion speed and the image fusion effect can be improved.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present disclosure may be applied;
FIG. 2 is a flow diagram for one embodiment of an image processing method according to the present disclosure;
FIG. 3 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present disclosure;
FIG. 4 is a flow diagram of yet another embodiment of an image processing method according to the present disclosure;
FIG. 5 is a schematic block diagram of one embodiment of an image processing apparatus according to the present disclosure;
FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows an exemplary architecture 100 to which embodiments of the image processing method or image processing apparatus of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
Various client applications may be installed on the terminal devices 101, 102, 103. Such as image capture-type applications, image processing-type applications, search-type applications, beauty-picture-type applications, instant messaging-type applications, and the like. The terminal devices 101, 102, 103 may interact with the server 105 via the network 104 to receive or send messages or the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices having an image capturing function, and may also be various electronic devices that can receive user operations, including but not limited to cameras, smart phones, tablet computers, electronic book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a background server supporting client applications installed on the terminal devices 101, 102, 103. The server 105 may provide a background server for downloading various functions and using functions for the client applications installed on the terminal devices 101, 102, 103. The client applications installed on the terminal devices 101, 102, 103 can be made to use the respective image processing functions by making downloads, such as image processing (e.g., image deduplication) functions, with the client applications from the servers for which support is provided.
The server 105 may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the image processing method provided by the embodiments of the present disclosure is executed by the terminal devices 101, 102, 103. Accordingly, the image processing apparatus may be provided in the terminal devices 101, 102, 103.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. In the case where data used in the image processing process (e.g., certain image processing functions) need not be acquired from a remote location, the system architecture described above may include no network, and only terminal devices.
With continued reference to FIG. 2, a flow 200 of one embodiment of an image processing method according to the present disclosure is shown. The image processing method comprises the following steps:
In the present embodiment, the execution subject of the above-described image processing method (e.g., the terminal apparatus 101, 102, 103 or the server 105 shown in fig. 1) may be mounted with or connected to a photographing apparatus. The at least two frames of images may be captured by the capturing device and then transmitted to the executing body. Alternatively, the at least two images may be pre-stored locally. The execution main body may acquire the at least two frames of images through path information indicating a location where each image is stored.
Here, the at least two images present the same scene. That is, the information of the object, the shooting background, and the like presented by the at least two frames of images are the same. As an example, the at least two frames of images are both images taken of building a.
In this embodiment, according to the at least two frames of images acquired in step 201, the executing entity may select one frame from the at least two frames of images as the target image.
Specifically, the quality detection may be performed on the at least two frames of images, and one image with the highest imaging quality may be selected as the target image. Quality tests may include, but are not limited to: image color saturation detection, detection of the position of a target object in an image represented by the image, and the like. Here, the execution subject may first calculate a pixel value of each image, and determine the color saturation of each image based on the calculated pixel value. Then, the calculated color saturation values of the respective images are compared with a preset optimum saturation value, and an image having a color saturation closest to the preset optimum saturation value is saved as a target image based on the comparison result. Alternatively, the execution subject may detect an object presented in the image and determine the position of the target object in the image. Specifically, the distance of the presented target object from the center point of the image and the proportion of the presented target object in the image may be calculated. Then, an image with the proportion of the presented target object in the image larger than a preset threshold value is selected. And selecting the image closest to the position of the central point of the image as the target image from the images of which the proportion of the selected target object to the image is greater than a preset threshold value.
In this embodiment, when the same scene is photographed, based on the influence of the surrounding environment, there may occur a case where the photographing of the photographing apparatus is unstable, or a case where the photographing apparatus is rotated, or the like, resulting in a difference in the position of the same object included in a plurality of images in the images. Therefore, each image in the first image sequence needs to be coordinate mapped. Therefore, the situation that the imaging effect of the fused image is influenced due to the fact that the pixel value cannot be accurately calculated in the image fusion process caused by the fact that the same object is located at the image coordinate positions of different images can be avoided.
Specifically, the in-focus point of the target image and the in-focus point of the non-target image may be determined separately. In general, when a shooting device shoots a scene, focusing is performed based on the shot scene. An object to be focused can be set. Then, the photographing apparatus can perform focus photographing based on the object. Generally, after the photographing apparatus has photographed an image, an in-focus point is generally presented. The coordinates of the focus point are typically camera coordinates. Since the focus is obtained by focusing on a specified object, the focus of each image is used to indicate the same object. The executive may then compare the respective in-focus points of the remaining images with the in-focus point of the target image, and determine the deviation between the in-focus point of each non-target image and the in-focus point of the target image. Based on the deviation, a camera coordinate mapping relationship of the non-target image and the target image is determined. For example, a transfer matrix for converting camera coordinates of the non-target image into camera coordinates of the target image may be determined based on the deviation. Then, based on the transfer matrix, the camera coordinates and the conversion relationship of the image coordinates, the image coordinates of the non-target image can be mapped into the image coordinates of the target image, so as to obtain an image after coordinate conversion.
In some optional implementation manners of this embodiment, the mapping the image coordinates of the non-target image in the at least two frame images to the target image may specifically include: extracting key points of each frame of image in at least two frames of images; for each frame of image in the non-target image, matching the key points of the image with the key points of the target image, and generating a homography matrix for converting the image coordinates of the key points in the image into the target image based on the matching result; image coordinates of the image are mapped into a target image based on the homography matrix.
Specifically, the above-mentioned key point extraction may be, for example, sift-based key point extraction. Sift is an image local feature description operator based on scale space, which remains invariant to image scaling, rotation, and even affine transformations. Firstly, the interest points which are local extreme points in the scale space and the two-dimensional image space can be extracted, and unstable and wrong interest points with low energy are filtered to obtain the final stable feature points. Then, the feature points are described. The feature point description may include a feature point direction assignment and a 128-dimensional vector description. Thus, the key points of the target image are obtained based on the determined feature points and the description of the feature points. The keypoints for each of the remaining non-target images can also be determined in the same manner. Then, for each frame of image in the non-target image, the determined key points of the image are matched with the key points of the target image. Here, the keypoint matching may be specifically implemented by calculating euclidean distances of 128-dimensional vectors between the keypoints of the image and the keypoints of the target image. Wherein, the smaller the Euclidean distance is, the higher the matching degree is. When the euclidean distance is smaller than the set threshold, it can be determined that the matching is successful. Then, based on the matching result of the image and the key points of the target image, determining a homography matrix which maps the image coordinates of the key points in the image to the target image. Finally, based on the calculated homography matrix, the image coordinates of each pixel in the image may be multiplied by the homography matrix to map the image coordinates of the image into the target image.
And step 204, fusing the image after coordinate conversion and the target image.
In this embodiment, the executing entity may perform image fusion of the target image and the coordinate-converted image according to the coordinate-converted image obtained in step 203.
Specifically, for each non-target image, based on the calculated conversion matrix between the non-target image and the target image, the image coordinates of the same key points in the non-target image as the target image are mapped to the image coordinates of the target image, so as to obtain an image after coordinate conversion. Then, the average value of the pixel values at the same image coordinate position in the target image and the image obtained after the coordinate conversion is calculated.
In the present embodiment, according to the fusion of the coordinate-converted image and the target image in step 204, a fused image may be generated based on the fusion result.
Specifically, based on the average value calculation result of each pixel value, an image after the average value calculation can be generated. The image is taken as a fused image of the at least two frames of images. By utilizing the method to process the image, compared with the method of processing the single-frame image to remove noise, the processed image is closer to a real scene, and the scene reducibility is improved.
Further referring to fig. 3, it shows an application scenario diagram of the image processing method of the present disclosure.
In the application scenario shown in fig. 3, the electronic device acquires two frames of images, image a and image B. Where both image a and image B are taken for the same scene. As can be seen from the figure, in the picture presented by the image a, an image noise area a exists; in the picture presented by image B, there is an image noise area B. Then, the image coordinates of the image B are mapped into the image a with the image a as a target image, resulting in an image C after coordinate conversion. And finally, carrying out image fusion on the image A and the image C to obtain a fused image D. As can be seen from the figure, in the image D obtained after the image A and the image C are fused, the image noise which does not exist in the image A and the image C is reduced, and the imaging quality of the image is improved.
According to the image processing method provided by the embodiment of the disclosure, at least two frames of images shot in the same scene are obtained, then the at least two frames of images are fused, and the fused image is generated based on the fusion result, so that the noise generated in the image shooting process can be reduced, and the image imaging effect is improved; and performing image coordinate mapping on at least two frames of images to enable the same positions of the images to correspond, so that the image fusion speed and the image fusion effect can be improved.
With further reference to fig. 4, a flow 400 of yet another embodiment of an image processing method according to the present disclosure is shown. The image processing method comprises the following steps:
In the present embodiment, the execution subject of the above-described image processing method (e.g., the terminal apparatus 101, 102, 103 or the server 105 shown in fig. 1) may be mounted with or connected to a photographing apparatus. The at least two frames of images may be captured by the capturing device and then transmitted to the executing body. Alternatively, the at least two images may be pre-stored locally. The execution main body may acquire the at least two frames of images through path information indicating a location where each image is stored.
Here, the at least two images present the same scene. That is, the information of the object, the shooting background, and the like presented by the at least two frames of images are the same. As an example, the at least two frames of images are both images taken of building a.
And 403, mapping the image coordinates of the non-target image in the at least two frame images to the target image to obtain an image after coordinate conversion.
The specific implementation of step 401, step 402, and step 403 and the beneficial effects thereof may refer to the related description of step 201, step 202, and step 203 shown in fig. 2, and are not described herein again.
And step 404, calculating the average value of the pixel values at the same coordinate position in the image after the coordinate conversion and the target image.
In this embodiment, according to the coordinate-converted image determined in step 402 corresponding to each non-target image, the executing entity may perform average calculation on the pixel values of the pixels at the same image coordinate position in the target image and the coordinate-converted image, so as to obtain a new pixel value corresponding to each image coordinate position.
In step 405, a processed image based on the average value calculation is determined.
In this embodiment, a new processed image may be generated based on the new pixel values corresponding to each image coordinate position obtained in step 404.
In step 406, for each frame of image in the coordinate-converted image and the target image, the difference between the image and the processed image is determined, and a weight is set for the image based on the difference result.
In this embodiment, determining the difference between the image and the processed image may specifically include: for the target image, the image is directly compared with the processed image, and based on the comparison result, the difference between the frame image and the processed image is determined. For the non-target image, the image after coordinate conversion corresponding to the frame image is compared with the processed image, and based on the comparison result, the difference between the frame image and the processed image is determined. Since the image coordinates of the feature points in the image without coordinate conversion have a deviation from the image coordinates of the same feature points in the processed image, direct comparison results in a large error in comparison results, and the accuracy of the determined difference is reduced. By transforming the coordinates of the non-target image and then determining the difference between the non-target image and the target image, the accuracy of the determined difference can be improved.
Here, for each frame of the image participating in the image fusion, the difference values of the pixel values at the same coordinate position in the image and the processed image may be determined first, and then the sum of all the difference values obtained based on the image may be determined. Then, an index of the sum of the differences is determined as a first value. And adding the indexes of the sum of the difference values between each frame of image participating in image fusion and the processed image to obtain a second value. And finally, taking the ratio of the first value to the second value as the weight corresponding to the image after the frame coordinate conversion. Here, the image participating in the image fusion specifically includes a target image and a coordinate-converted image corresponding to each frame of the non-target image.
In this embodiment, the preprocessed image may be obtained by multiplying the pixel values of each frame of image by the weight based on the weight corresponding to the frame of image. Then, the pixel values at the same image coordinate position in all the obtained preprocessed images are averaged.
In this embodiment, an average value calculation is performed according to the pixel values at the same image coordinate position in step 407, and an average value calculation result can be obtained. Then, a new image is generated based on each pixel value obtained as a result of the average value calculation. And taking the generated new image as a fusion image.
It can be seen from fig. 4 that, different from the embodiment shown in fig. 2, the embodiment highlights the step of setting the weight for each frame of image, and can determine the proportion of the image participating in the fusion based on the weight of each frame of image, so as to avoid that a certain position area still has large noise in the obtained fusion image due to an excessive noise or an excessive proportion of the image with large color difference participating in the fusion, and the fusion image obtained by the method shown in the embodiment can effectively reduce the noise, thereby improving the image imaging effect.
With further reference to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an image processing apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 5, the image processing apparatus 500 provided by the present embodiment includes an acquisition unit 501, a selection unit 502, a mapping unit 503, a fusion unit 504, and a generation unit 505. The acquiring unit 501 is configured to acquire at least two frames of images, where the at least two frames of images represent the same scene; a selecting unit 502 configured to select a target image from at least two frame images; a mapping unit 503 configured to map image coordinates of a non-target image of the at least two frame images into the target image to obtain a coordinate-converted image; a fusion unit 504 configured to fuse the coordinate-converted image and the target image; a generating unit 505 configured to generate a fused image based on the fusion result.
In the present embodiment, in the image processing apparatus 500: the specific processing of the obtaining unit 501, the selecting unit 502, the mapping unit 503, the fusing unit 504, and the generating unit 505 and the technical effects thereof can refer to the related descriptions of step 201, step 202, step 203, step 204, and step 205 in the corresponding embodiment of fig. 2, which are not described herein again.
In some optional implementations of this embodiment, the mapping unit 502 is further configured to: extracting key points of each frame of image in at least two frames of images; for each frame of image in the non-target image, matching the key points of the image with the key points of the target image, and generating a homography matrix for converting the image coordinates of the key points in the image into the target image based on the matching result; image coordinates of the image are mapped into a target image based on the homography matrix.
In some optional implementations of this embodiment, the fusion unit 504 includes: a calculation subunit (not shown in the figure) configured to perform average calculation on pixel values at the same coordinate position in the coordinate-converted image and the target image; and a generating unit 505 comprising: a generation subunit (not shown in the figure) configured to generate a fused image based on the average value calculation result.
In some optional implementations of this embodiment, the generating a sub-unit (not shown in the figure) includes: a determination module (not shown in the figure) configured to determine a processed image calculated based on the average value; a setting module (not shown in the figure) configured to determine, for each frame image of the coordinate-converted image and the target image, a difference between the image and the processed image; setting a weight to the image based on the difference result; a generating module (not shown in the figure) configured to fuse the coordinate-converted image and the target image based on the obtained weight of each frame image, the pixel value of each frame image in the coordinate-converted image and the target image; based on the fusion result, a fused image is generated.
In some optional implementations of this embodiment, the setting module (not shown in the figure) is further configured to: comparing the pixel value of each pixel in the image with the pixel value of a pixel at the same image coordinate position in the processed image, and generating a difference value between the pixel of the image and the pixel of the processed image based on the comparison result; based on the difference, a weight is set for the image.
According to the image processing device provided by the embodiment of the disclosure, at least two frames of images shot in the same scene are acquired, then the at least two frames of images are fused, and the fused image is generated based on the fusion result, so that the noise generated in the image shooting process can be reduced, and the image imaging effect is improved; and performing image coordinate mapping on at least two frames of images to enable the same positions of the images to correspond, so that the image fusion speed and the image fusion effect can be improved.
Referring now to fig. 6, shown is a schematic diagram of an electronic device (e.g., terminal device in fig. 1) 600 suitable for use in implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the use range of the embodiments of the present disclosure.
As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of embodiments of the present disclosure.
It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be included in the terminal device; or may exist separately without being assembled into the terminal device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two frames of images, wherein the at least two frames of images present the same scene; selecting a target image from at least two frame images; mapping image coordinates of a non-target image in at least two frames of images to a target image to obtain an image after coordinate conversion; and fusing the image subjected to the coordinate conversion and the target image, and generating a fused image based on a fusion result.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a processor including an acquisition unit, a selection unit, a mapping unit, a fusion unit, and a generation unit. The names of the units do not in some cases constitute a limitation on the units themselves, and for example, the acquiring unit may also be described as a "unit that acquires at least two frames of images".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (8)
1. An image processing method comprising:
acquiring at least two frames of images, wherein the at least two frames of images present the same scene;
selecting a target image from the at least two frame images;
mapping image coordinates of a non-target image in the at least two frames of images to the target image to obtain an image after coordinate conversion;
calculating the average value of pixel values at the same coordinate position in the image after the coordinate conversion and the target image, and determining a processed image obtained after the average value calculation;
determining the difference between the image after the coordinate conversion and each frame of image in the target image and the processed image; setting a weight to the image based on the difference result;
and performing image fusion on the image subjected to the coordinate conversion and the target image based on the obtained weight of each frame of image, the pixel value of each frame of image in the image subjected to the coordinate conversion and the target image, and generating a fused image based on a fusion result.
2. The method of claim 1, wherein said mapping image coordinates of non-target images of said at least two frame images into said target image comprises:
extracting key points of each frame of image in the at least two frames of images;
for each frame of image in the non-target image,
matching the key points of the image with the key points of the target image;
based on the matching result, generating a homography matrix for converting the image coordinates of the key points in the image into the target image;
mapping the image coordinates of the image into the target image based on the homography matrix.
3. The method of claim 1, wherein said determining a difference between the image and the processed image comprises:
comparing the pixel value of each pixel in the image with the pixel value of a pixel at the same image coordinate location in the processed image;
generating a difference between the pixels of the image and the pixels of the processed image based on the comparison result; and
the setting of the weight to the image based on the difference result includes:
setting a weight to the image based on the difference.
4. An image processing apparatus comprising:
an acquisition unit configured to acquire at least two frames of images, wherein the at least two frames of images present the same scene;
a selection unit configured to select a target image from the at least two frame images;
a mapping unit configured to map image coordinates of a non-target image of the at least two frame images into the target image to obtain a coordinate-converted image;
a calculation subunit configured to perform average calculation on pixel values at the same coordinate position in the coordinate-converted image and the target image;
a determination module configured to determine a processed image calculated based on the average value;
a setting module configured to determine, for each frame of the coordinate-converted image and the target image, a difference between the image and the processed image; setting a weight to the image based on the difference result;
a generating module configured to fuse the coordinate-converted image and the target image based on the obtained weight of each frame of image, the pixel value of each frame of image in the coordinate-converted image and the target image.
5. The apparatus of claim 4, wherein the mapping unit is further configured to:
extracting key points of each frame of image in the at least two frames of images;
for each frame of image in the non-target image,
matching the key points of the image with the key points of the target image;
based on the matching result, generating a homography matrix for converting the image coordinates of the key points in the image into the target image;
mapping the image coordinates of the image into the target image based on the homography matrix.
6. The apparatus of claim 4, wherein the setup module is further configured to:
comparing the pixel value of each pixel in the image with the pixel value of a pixel at the same image coordinate position in the processed image, and generating a difference value between the pixel of the image and the pixel of the processed image based on the comparison result;
setting a weight to the image based on the difference.
7. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-3.
8. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910593850.XA CN110310251B (en) | 2019-07-03 | 2019-07-03 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910593850.XA CN110310251B (en) | 2019-07-03 | 2019-07-03 | Image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310251A CN110310251A (en) | 2019-10-08 |
| CN110310251B true CN110310251B (en) | 2021-10-29 |
Family
ID=68079548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910593850.XA Active CN110310251B (en) | 2019-07-03 | 2019-07-03 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310251B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853524A (en) * | 2010-05-13 | 2010-10-06 | 北京农业信息技术研究中心 | Method for generating corn ear panoramic image by using image sequence |
| CN102075679A (en) * | 2010-11-18 | 2011-05-25 | 无锡中星微电子有限公司 | Method and device for acquiring image |
| CN104065854A (en) * | 2014-06-18 | 2014-09-24 | 联想(北京)有限公司 | Image processing method and electronic device |
| CN106251365A (en) * | 2016-07-22 | 2016-12-21 | 北京邮电大学 | Many exposure video fusion method and device |
| CN108898567A (en) * | 2018-09-20 | 2018-11-27 | 北京旷视科技有限公司 | Image denoising method, apparatus and system |
-
2019
- 2019-07-03 CN CN201910593850.XA patent/CN110310251B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853524A (en) * | 2010-05-13 | 2010-10-06 | 北京农业信息技术研究中心 | Method for generating corn ear panoramic image by using image sequence |
| CN102075679A (en) * | 2010-11-18 | 2011-05-25 | 无锡中星微电子有限公司 | Method and device for acquiring image |
| CN104065854A (en) * | 2014-06-18 | 2014-09-24 | 联想(北京)有限公司 | Image processing method and electronic device |
| CN106251365A (en) * | 2016-07-22 | 2016-12-21 | 北京邮电大学 | Many exposure video fusion method and device |
| CN108898567A (en) * | 2018-09-20 | 2018-11-27 | 北京旷视科技有限公司 | Image denoising method, apparatus and system |
Non-Patent Citations (1)
| Title |
|---|
| A real time pixel-level based image fusion via adaptive weight averaging;Lallier E et al;《International Conference on Information Fusion》;20001231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310251A (en) | 2019-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7118244B2 (en) | Graphic code recognition method and device, terminal and program | |
| CN108492364B (en) | Method and apparatus for generating image generation model | |
| CN109829432B (en) | Method and apparatus for generating information | |
| CN109741388B (en) | Method and apparatus for generating a binocular depth estimation model | |
| CN110111241B (en) | Method and apparatus for generating dynamic image | |
| CN110516678B (en) | Image processing method and device | |
| CN110033423B (en) | Method and apparatus for processing image | |
| CN110059623B (en) | Method and apparatus for generating information | |
| CN110288625B (en) | Method and apparatus for processing image | |
| CN110062157B (en) | Method and device for rendering image, electronic equipment and computer readable storage medium | |
| CN111402404B (en) | Panorama complementing method and device, computer readable storage medium and electronic equipment | |
| CN113256742A (en) | Interface display method and device, electronic equipment and computer readable medium | |
| CN112085775A (en) | Image processing method, device, terminal and storage medium | |
| CN111325792A (en) | Method, apparatus, device, and medium for determining camera pose | |
| CN109034214B (en) | Method and apparatus for generating a mark | |
| CN109919220B (en) | Method and apparatus for generating feature vectors of video | |
| CN109816791B (en) | Method and apparatus for generating information | |
| CN109840059B (en) | Method and apparatus for displaying image | |
| CN115170395A (en) | Panoramic image stitching method, panoramic image stitching device, electronic equipment, panoramic image stitching medium and program product | |
| CN110310251B (en) | Image processing method and device | |
| CN111385460A (en) | Image processing method and device | |
| CN115393423A (en) | Target detection method and device | |
| CN112492230B (en) | Video processing method and device, readable medium and electronic equipment | |
| CN110084835B (en) | Method and apparatus for processing video | |
| CN111314627B (en) | Method and apparatus for processing video frames |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. |