TWI526987B - Method for real-time simultaneous image stitching based on visual content maximization - Google Patents

Description

Instant image stitching method based on maximizing visual content

The invention relates to an instant image splicing method, in particular to an instant image splicing method that covers the maximization of visual content, and selects less important images that are more important and not repeated in the continuous captured image sequence for splicing processing, and can cover observations. A wider range of visual information for the scene, in order to achieve instant and good quality stitching results.

Image Stitching is a very important image processing technology. It is commonly used to combine similar images into a single image to display the entire image of the entire scene, rather than just a small part of the partial image. For example, a 360-degree panoramic photography taken with a camera or a digital camera, where successively captured images require appropriate stitching, removing overlapping portions of the subsequent images, or the same, similar portions, and retaining the differences, The stitching is then stitched to extend the displayable range of the image.

However, when the camera is shaken or shaken by external force, the captured image will be blurred. Although the image difference of the scene is not large, it will be mistaken by the processing system as the difference between the front and back images becomes larger, resulting in stitching. Perform unnecessary calculations, waste valuable computing resources, reduce processing speed and efficiency, and affect image quality, resulting in unclear images.

In addition, as the resolution of the image sensor increases, the amount of data of a single image is quite large, which causes considerable computational challenges for image processing that requires continuous image stitching, especially for instant applications.

Therefore, there is a great need for an innovative method for instant image stitching, which first removes the blurred image to avoid affecting subsequent image processing, and filters out similar images to reduce the number of images, which not only can effectively improve the quality of the stitched image, reduce the cumulative error, and can speed up The processing speed improves the processing efficiency of the overall image splicing, thereby solving the problems of the above-mentioned conventional techniques.

The main object of the present invention is to provide a real-time image splicing method based on maximizing visual content, which mainly includes an input image step, a step of determining a blurred image, a step of determining a color similarity, a step of finding a suture, and a feature. The point matching step, the calculation of the homography transfer matrix step and the stitching image step are used for splicing and locating the continuous input image to cover the visual information of the wider scene, thereby achieving an instant and good quality splicing result.

Specifically, in the input image step, a series of continuous images are mainly input into the system, such as a system implemented by a microprocessor, a central processing unit, a personal computer, a notebook computer, a tablet computer, a mobile phone, and continuously. Images can be generated by shooting from a camera, digital camera, tablet, or mobile phone, or input from a memory medium such as a hard disk or a compact disc.

Then enter the step of determining the blurred image, using the difference of the edge amount of the image in different sizes, or using the amount of the edge of the image area to determine whether it is a blurred image, such as the sudden shaking of the camera or the speed of the target image during the shooting. The blurred image generated by the change, and if it is a blurred image, is directly removed, so as to avoid the blurred image affecting the subsequent stitching processing and image quality.

In addition, in order to reduce the computing time, achieve the effect of instant splicing, and at the same time maintain the quality of the spliced image, it is necessary to select images with important and maximum amount of rich information for all continuous images for splicing. Therefore, it is necessary to determine the similarity between images as reference data.

In the step of determining the color similarity, the input image is first converted from the RGB color space to the HSV (Hue-Saturation-Value) color space to obtain a color histogram of the input image, and then The Bhattacharyya distance is used to calculate the color similarity between the two images before and after, and the color similarity is used to determine whether the current input image is a similar image with respect to the previous input image, and if it is a similar image, the current image is deleted. Input image. Therefore, the number of stitched images can be reduced, the cumulative error of the stitching can be reduced, and the calculation time can be reduced, thereby speeding up the operation speed of the system and improving the overall efficiency.

Here, in order to remove the color mixture afterimage caused by the coincidence of the overlapping areas between the stitched images, to improve the quality of the stitched image, the stitching between the stitched images can be searched for, and the image is utilized. The suture in the image is used for image cutting and color mixing.

Therefore, the first step to find the suture is mainly based on the difference in color or color in the image, that is, the information richness of the overlapping regions in the front and rear images, and using the preset ideal suture path, such as a plurality of equally spaced verticals. A line or horizontal line to find stitches in an image, and to retain images with more information about the content of the image.

Then, in the feature point matching step, an optical flow estimation (Optical Flow Estimation) is used, and the feature points are uniformly distributed to perform matching and tracking of the image feature points, thereby obtaining at least one feature point. And in the step of calculating the homography transfer matrix, the feature points are utilized to calculate the homography transfer matrix. Finally, the step of splicing the image is performed, and according to the homography transfer matrix, each input image is instantly positioned on the spliced image, thereby completing the entire image splicing.

Since the method of the invention first eliminates the blurred image, the accuracy of the subsequent image processing can be avoided, the unnecessary interference is avoided, and the similar image is filtered to reduce the number of images, so that the processing speed is accelerated, and the quality of the stitched image is effectively improved. Improve the processing efficiency of the overall image stitching.

S10~S70‧‧‧Steps

The first figure shows a schematic diagram of an operational flow based on an instant image stitching method that covers visual content maximization in accordance with an embodiment of the present invention.

The embodiments of the present invention will be described in more detail below with reference to the drawings and the reference numerals, which can be implemented by those skilled in the art after having studied this specification.

Referring to the first figure, a schematic diagram of an operation flow based on a method for realizing instant image mosaic covering visual content maximization according to an embodiment of the present invention. As shown in the first figure, the present invention is based on an image mosaic method for maximizing visual content, which mainly includes an input image step S10, a fuzzy image step S20, a color similarity step S30, and a suture finding step S40. The feature point matching step S50, the calculation of the homography transfer matrix step S60, and the stitching image step S70 are performed for image stitching and positioning processing for a series of consecutive input images to cover the observation of a relatively large scene visual information, thereby achieving instant And the quality of the splicing results.

First, the instant image stitching method of the present invention starts from step S10 and will lose The input image is input into the system, such as a system implemented by a microprocessor, a central processing unit, a personal computer, a notebook computer, a tablet computer, a mobile phone, and the continuous image can be photographed by a camera, a digital camera, a tablet computer, a mobile phone. Generated, or entered by a memory medium.

Then, the step of determining the blurred image step S20 is performed to prevent the blurred image from affecting the subsequent stitching processing and the image quality is mainly by using the difference of the edge amount of the input image in different sizes, or by using the amount of the edge of the area of the input image to determine the input image. Whether it is a blurred image. If the input image is a blurred image, the input image is rejected, the subsequent steps are not performed, and the input image is returned to step S10. If the input image is not a blurred image, step S30 is determined to select an input image having an important and maximum amount of rich information for subsequent processing, which can reduce the number of images and the operation time, and speed up the processing to achieve an instant. The effect of splicing.

In the determining color similarity step S30, the input image is first converted from the RGB color space to the HSV (Hue-Saturation-Value) color space, so that the color distribution histogram of the input image can be obtained (Color Histogram) Then, the Bhattacharyya distance is used to calculate the color similarity between the two images before and after, and the color similarity is used to determine whether the current input image is a similar image with respect to the previous input image. If it is a similar image, deleting the current input image and returning to the input image step S10 can reduce the number of stitched images, reduce the cumulative error of the stitching, and reduce the calculation time, thereby speeding up the operation speed of the system and improving the overall efficiency. .

Here, in order to remove the color-mixing residual image caused by the coincidence of the overlapping regions between the stitched images, to improve the quality of the stitched image, the stitching between the stitched images can be searched, and the stitching and color mixing of the image can be performed by using the stitching in the image.

In the step of finding the suture step S40, according to the color or color difference in the image, that is, the information richness of the overlapping regions in the front and rear images, and using the preset ideal suture path, such as a plurality of equally spaced vertical lines and / Or a horizontal line to find the stitches in the image, and to retain the image with more information about the image observation.

Then, the feature point matching step S50 is entered, and the optical flow estimation (Optical Flow Estimation) is used, and the feature points are uniformly distributed to perform matching and tracking of the image feature points to obtain at least one feature point. Then, in the calculation of the homography transfer matrix step S60, the homography transfer matrix (Homography Matrix) is calculated using the obtained feature points. Finally, the splicing image step S70 is executed, and each input image is instantly positioned on the spliced image according to the homography transfer matrix, thereby completing image splicing and returning to the input image step S10.

The image splicing method of the invention is very suitable for use in related electronic products with image functions such as mobile phones and cameras.

In summary, the main feature of the present invention is that the blurred image is first removed, which can avoid affecting the accuracy of subsequent image processing, avoid unnecessary interference, and eliminate similar images to reduce the number of images and select continuous captured images. The less important images in the sequence are not spliced for splicing, which can cover the observation of a large amount of scene visual information, which makes the processing speed faster, reduces the accumulated error, effectively improves the quality of the spliced image and improves the processing efficiency of the overall image splicing.

The above is only a preferred embodiment for explaining the present invention, and is not intended to limit the present invention in any way, and any modifications or alterations to the present invention made in the spirit of the same invention. All should still be included in the scope of the intention of the present invention.

S10~S70‧‧‧Steps

Claims (2)

An instant image splicing method based on maximizing visual content is used for performing image splicing processing on a series of consecutive input images, including: an input image step for inputting the input image to a system; The image step, using the difference in the edge amount of the input image at different sizes, or using the amount of the edge of the region describing the input image to determine whether the input image is a blurred image, and if the input image is a blurred image, then the image is rejected. Input image, return to the input image step; a color similarity step, first convert the input image from RGB color space to HSV (Hue-Saturation-Value) color space, and obtain input image Color Histogram, and then use the Bhattacharyya distance to calculate the color similarity between the two consecutive input images, and then determine the current input image relative to the previous input based on the color similarity. Whether the image is a similar image, and if it is a similar image, delete the current input Image, and returning to the input image step; a step of finding a suture according to a color or color difference in the input image, and using a preset ideal suture path to find a suture in the input image, wherein The preset ideal suture path includes a plurality of equally spaced vertical lines and/or horizontal lines; a feature point matching step uses an Optical Flow Estimation and is evenly distributed with feature points (Feature Points) The method for performing matching and tracking of the feature points of the input image to obtain at least one feature point; and calculating a homography transfer matrix step, using the at least one feature point to calculate a Homography transfer matrix; And a stitching image step, according to the homography transfer matrix, instantly positioning the input image to be spelled On the connected image, the image stitching is completed, and the input image step is returned. According to the instant image splicing method described in claim 1, wherein the system is implemented by a microprocessor, a central processing unit, a personal computer, a notebook computer, a tablet computer, a mobile phone, and the continuous image is a camera, A digital camera, tablet, or mobile phone is produced by shooting, or input by a memory medium including a hard disk or a compact disc.