CN112102168A - Image splicing method and system based on multiple threads - Google Patents
Image splicing method and system based on multiple threads Download PDFInfo
- Publication number
- CN112102168A CN112102168A CN202010913588.5A CN202010913588A CN112102168A CN 112102168 A CN112102168 A CN 112102168A CN 202010913588 A CN202010913588 A CN 202010913588A CN 112102168 A CN112102168 A CN 112102168A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- images
- preprocessed
- average
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000007781 pre-processing Methods 0.000 claims description 61
- 238000001914 filtration Methods 0.000 claims description 12
- 230000007613 environmental effect Effects 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 8
- 239000000126 substance Substances 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention provides an image splicing method and system based on multithreading, which are characterized in that the noise reduction and image area division are carried out on sub-images obtained by shooting at different visual angles, so that overlapped sub-image blocks among different sub-images are determined and are used as corresponding spliced image areas, and then the multithreading correction of the brightness, the chromaticity and the resolution ratio is carried out on the overlapped sub-image blocks, so that the overlapped sub-image blocks can be matched with each other, and the image quality after splicing and the efficiency and the reliability of image splicing are greatly improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image stitching method and system based on multithreading.
Background
Currently, a single camera cannot capture a panoramic image of a target object well due to a limited range of a shooting angle, which severely restricts the panoramic reproduction of the target object. However, with the continuous development of computer image processing technology and the continuous improvement of computer image processing performance, in the prior art, a wide-view panoramic image is formed by stitching images with different viewing angles through a computer, and in the stitching process, the image pixel fusion degree of the images with different viewing angles in a stitching area is generally only considered, and the images are not subjected to multi-thread stitching fusion on different image parameters such as image brightness, image chromaticity and image resolution, so that the quality of the stitched images and the efficiency and reliability of image stitching are greatly reduced.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention provides an image splicing method and system based on multithreading, which comprises the steps of shooting a target object in multiple visual angles to obtain a plurality of sub-images corresponding to each shooting visual angle one by one, carrying out noise reduction preprocessing on the sub-images to obtain a plurality of preprocessed sub-images, carrying out image area division on the preprocessed sub-images to obtain a plurality of sub-image blocks, determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images, determining the difference information of at least one overlapped sub-image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution, and finally carrying out at least one of brightness correction, chroma correction and resolution correction on at least one overlapped sub-image block corresponding to any two preprocessed images respectively according to the difference information, splicing any two corrected preprocessed images; therefore, the multithreading-based image splicing method and the multithreading-based image splicing system have the advantages that the image quality after splicing and the efficiency and the reliability of image splicing are greatly improved.
The invention provides an image splicing method based on multithreading, which is characterized by comprising the following steps:
step S1, carrying out multi-view shooting on the target object so as to obtain a plurality of sub-images corresponding to each shooting view one by one, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
step S2, dividing the image area of the preprocessed sub-images to obtain a plurality of sub-image blocks, and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
step S3, determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
step S4, according to the difference information, at least one of brightness correction, chroma correction and resolution correction is carried out on at least one overlapped sub image block corresponding to any two preprocessed images, and then any two preprocessed images after correction are spliced;
further, in step S1, performing multi-view shooting on the target object to obtain a plurality of sub-images corresponding to each shooting view one to one, and performing noise reduction preprocessing on the sub-images to obtain a plurality of preprocessed sub-images specifically includes:
step S101, performing fixed-focus shooting on the target object along a plurality of different view angle directions which are mutually crossed, so as to obtain a plurality of sub-images which correspond to the different view angle directions in a preset depth, wherein included angles of any two adjacent view angle directions in the different view angle directions are the same;
step S102, distinguishing a target object from an environment background of the sub-image, so as to determine a target object image area and an environment background image area contained in the sub-image;
step S103, performing Kalman filtering processing on the target object image area so as to realize noise reduction preprocessing on the target object image area, and sequentially performing Fourier transform processing and high-frequency filtering processing on the environment background image area so as to realize noise reduction preprocessing on the environment background image area;
further, in step S2, the image area division is performed on the preprocessed sub-images to obtain a plurality of sub-image blocks, and the determining at least one overlapping sub-image block commonly included in any two preprocessed sub-images specifically includes:
step S201, carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
step S202, sequentially numbering and calibrating each sub image block of the image edge area of any two pre-processing sub images, comparing image profiles of two sub image blocks with the same calibration number in any two pre-processing sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is greater than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapping sub image blocks commonly included between any two pre-processing sub images, and otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapping sub image blocks commonly included between any two pre-processing sub images;
further, in step S3, the determining the difference information of at least one overlapping sub image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance, and resolution specifically includes:
step S301, in any two pre-processed images, determining an average brightness value, an average chroma value and an average resolution value of each overlapping sub-image block respectively contained in one pre-processed image and the other pre-processed image;
step S302, determining at least one of an average luminance difference value, an average chrominance difference value, and an average resolution difference value of each overlapping sub-image block included in each of the one pre-processed image and the other pre-processed image according to the average luminance value, the average chrominance value, and the average resolution value, as the difference information;
further, in step S4, according to the difference information, at least one of luminance correction, chrominance correction, and resolution correction is performed on at least one overlapping sub-image block corresponding to each of any two preprocessed images, and then splicing any two preprocessed images after correction specifically includes:
step S401, according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapping sub image block included in at least one of any two pre-processed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapping sub image block included in any two pre-processed images are all lower than corresponding difference threshold values;
and step S402, taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially performing pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing splicing of any two preprocessed images.
The invention also provides an image splicing system based on multithreading, which is characterized by comprising an image shooting and preprocessing module, an image area dividing module, an image difference information determining module and an image splicing module; wherein the content of the first and second substances,
the image shooting and preprocessing module is used for carrying out multi-view shooting on a target object so as to obtain a plurality of sub-images which are in one-to-one correspondence with each shooting view, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
the image area division module is used for carrying out image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
the image difference information determining module is used for determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
the image splicing module is used for performing at least one of brightness correction, chroma correction and resolution correction on at least one overlapped sub-image block corresponding to any two preprocessed images respectively according to the difference information, and splicing any two preprocessed images after correction;
further, the image capturing and preprocessing module performs multi-view capturing on the target object to obtain a plurality of sub-images corresponding to each capturing view one to one specifically includes:
performing fixed-focus shooting on the target object along a plurality of mutually crossed different view angle directions to acquire a plurality of sub-images which correspond to the different view angle directions in a preset depth of field one by one, wherein included angles of any two adjacent view angle directions in the different view angle directions are the same
The image capturing and preprocessing module performs noise reduction preprocessing on the sub-images, so as to obtain a plurality of preprocessed sub-images specifically includes:
distinguishing the target object from the environmental background to determine the target object image area and the environmental background image area contained in the sub-image,
performing Kalman filtering processing on the target object image area so as to realize noise reduction preprocessing on the target object image area, and sequentially performing Fourier transform processing and high-frequency filtering processing on the environment background image area so as to realize noise reduction preprocessing on the environment background image area;
further, the image area dividing module performs image area division on the preprocessed sub-images, so as to obtain a plurality of sub-image blocks specifically includes:
carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
the determining, by the image area dividing module, at least one overlapping sub-image block commonly included in any two pre-processed sub-images specifically includes:
numbering and calibrating each sub image block of the image edge area of any two pre-processing sub images in sequence, comparing image profiles of two sub image blocks with the same calibration number in any two pre-processing sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is larger than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapped sub image blocks commonly contained between any two pre-processing sub images, and otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapped sub image blocks commonly contained between any two pre-processing sub images;
further, the determining, by the image difference information determining module, difference information of at least one overlapping sub-image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance, and resolution specifically includes:
determining an average luminance value, an average chrominance value and an average resolution value of each overlapping sub-image block respectively comprised by one of the pre-processed images and the other pre-processed image, in any two of the pre-processed images,
determining at least one of an average luminance difference value, an average chrominance difference value and an average resolution difference value of each overlapping sub-image block included in the one preprocessed image and the other preprocessed image respectively according to the average luminance value, the average chrominance value and the average resolution value, and using the at least one of the average luminance difference value, the average chrominance difference value and the average resolution difference value as the difference information;
further, the image stitching module, according to the difference information, at least one of performing luminance correction, chrominance correction, and resolution correction on at least one overlapping sub-image block corresponding to each of any two preprocessed images specifically includes:
according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapped sub image block contained in at least one of any two preprocessed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapped sub image block contained in any two preprocessed images are all lower than corresponding difference threshold values;
the image stitching module stitching any two corrected preprocessed images specifically comprises the following steps:
and taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially carrying out pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing the splicing of any two preprocessed images.
Compared with the prior art, the multithreading-based image splicing method and the multithreading-based image splicing system acquire a plurality of sub-images corresponding to each shooting view angle one by shooting a target object at multiple views, perform noise reduction preprocessing on the sub-images to acquire a plurality of preprocessed sub-images, perform image area division on the preprocessed sub-images to acquire a plurality of sub-image blocks, determine at least one overlapped sub-image block commonly contained between any two preprocessed sub-images, determine difference information of at least one overlapped sub-image block corresponding to any two preprocessed images in at least one of brightness, chromaticity and resolution, and perform at least one of brightness correction, chromaticity correction and resolution correction on at least one overlapped sub-image block corresponding to any two preprocessed images respectively according to the difference information, splicing any two corrected preprocessed images; therefore, the multithreading-based image splicing method and the multithreading-based image splicing system have the advantages that the image quality after splicing and the efficiency and the reliability of image splicing are greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments or technical descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of an image stitching method based on multithreading according to the present invention.
FIG. 2 is a schematic structural diagram of an image stitching system based on multiple threads according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of the image stitching method based on multithreading according to the present invention. The image stitching method based on multithreading comprises the following steps:
step S1, carrying out multi-view shooting on the target object so as to obtain a plurality of sub-images corresponding to each shooting view one by one, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
step S2, dividing the image area of the preprocessed sub-images to obtain a plurality of sub-image blocks, and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
step S3, determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
and step S4, according to the difference information, at least one of brightness correction, chroma correction and resolution correction is carried out on at least one overlapped sub-image block corresponding to any two preprocessed images, and then any two preprocessed images after correction are spliced.
The multithreading-based image splicing method is characterized in that the image splicing method is used for reducing noise and dividing image areas of the sub-images shot from different visual angles, so that overlapped sub-image blocks of the different sub-images are determined and serve as corresponding spliced image areas, and then multithreading correction of brightness, chromaticity and resolution is carried out on the overlapped sub-image blocks, so that the overlapped sub-image blocks can be matched with one another, and the quality of the spliced image and the efficiency and reliability of image splicing are greatly improved.
Preferably, in step S1, the multi-view shooting of the target object to obtain a plurality of sub-images corresponding to each shooting view one-to-one, and performing noise reduction preprocessing on the sub-images to obtain a plurality of preprocessed sub-images specifically includes:
step S101, performing fixed-focus shooting on the target object along a plurality of different view angle directions which are mutually crossed, so as to obtain a plurality of sub-images which correspond to the plurality of different view angle directions in a preset depth of field one by one, wherein included angles of any two adjacent view angle directions in the plurality of different view angle directions are the same;
step S102, distinguishing a target object from an environment background for the sub-image, so as to determine a target object image area and an environment background image area contained in the sub-image;
step S103, Kalman filtering processing is carried out on the target object image area, so that denoising preprocessing of the target object image area is realized, Fourier transform processing and high-frequency filtering processing are carried out on the environment background image area in sequence, and denoising preprocessing of the environment background image area is realized.
The target object is shot along a plurality of different view angle directions which are mutually crossed, so that the image information of the target object in different view angle directions can be obtained, the target object can be panned to obtain image acquisition, and the integrity of a subsequent spliced image is ensured; in addition, because the pixel information contained in the target object image area and the environmental background image area in the sub-image are different, and different types of noise reduction preprocessing are respectively carried out on the two different image areas, the optimal image noise elimination processing can be ensured to be obtained in both the target object image area and the environmental background image area.
Preferably, in step S2, the image area dividing the preprocessed sub-images to obtain sub-image blocks, and the determining at least one overlapping sub-image block commonly included in any two preprocessed sub-images specifically includes:
step S201, carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
step S202, sequentially numbering and calibrating each sub image block of the respective image edge area of any two pre-processed sub images, and comparing the image profiles of two sub image blocks with the same calibration number in any two pre-processed sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is greater than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapping sub image blocks commonly included between any two pre-processed sub images, otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapping sub image blocks commonly included between any two pre-processed sub images.
By dividing the grid image area of the preprocessed subimages, the subimage blocks with uniform shapes and sizes are obtained, and the convenience and reliability of calibrating the overlapped subimage blocks of any two preprocessed subimages can be improved.
Preferably, in step S3, the determining the difference information of at least one overlapping sub image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance and resolution specifically includes:
step S301, in any two pre-processed images, determining an average brightness value, an average chroma value and an average resolution value of each overlapping sub-image block respectively contained in one pre-processed image and the other pre-processed image;
step S302, determining at least one of an average luminance difference value, an average chrominance difference value, and an average resolution difference value of each overlapping sub-image block included in each of the one pre-processed image and the other pre-processed image according to the average luminance value, the average chrominance value, and the average resolution value, as the difference information.
By determining the difference of each overlapped sub-image block in the average brightness value, the average chroma value and the average resolution value, the multithreading image state synchronous judgment of different preprocessed sub-images can be realized, so that the overlapped sub-image blocks can be ensured to be matched with each other in the brightness, the chroma and the resolution of the image, and the calculation workload of subsequent image splicing is greatly reduced.
Preferably, in step S4, according to the difference information, at least one of luminance correction, chrominance correction, and resolution correction is performed on at least one overlapping sub-image block corresponding to each of any two preprocessed images, and then stitching any two preprocessed images after correction specifically includes:
step S401, according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapping sub image block included in at least one of any two pre-processed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapping sub image block included in any two pre-processed images are all lower than corresponding difference threshold values;
and step S402, taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially performing pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing splicing of any two preprocessed images.
The pre-processed sub-images can be matched and corrected in visual sense by performing average brightness value correction, average colorimetric value correction and average resolution value correction on the overlapped sub-image blocks, so that two different overlapped sub-image blocks can be spliced and fused accurately in the subsequent pixel convolution processing and pixel smoothing processing processes, and the quality of the spliced image and the efficiency and reliability of image splicing are improved.
Fig. 2 is a schematic structural diagram of the image stitching system based on multithreading according to the present invention. The multithreading-based image splicing system comprises an image shooting and preprocessing module, an image area dividing module, an image difference information determining module and an image splicing module; wherein the content of the first and second substances,
the image shooting and preprocessing module is used for carrying out multi-view shooting on a target object so as to obtain a plurality of sub-images which are in one-to-one correspondence with each shooting view, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
the image area division module is used for carrying out image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
the image difference information determining module is used for determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
the image splicing module is used for performing at least one of brightness correction, chroma correction and resolution correction on at least one overlapped sub-image block corresponding to any two preprocessed images respectively according to the difference information, and splicing any two preprocessed images after correction.
The multithreading-based image splicing system is used for determining overlapped sub-image blocks among different sub-images by performing noise reduction and image area division on the sub-images shot from different visual angles, and using the overlapped sub-image blocks as corresponding spliced image areas, and performing multithreading correction on the brightness, the chromaticity and the resolution of the overlapped sub-image blocks to ensure that the overlapped sub-image blocks can be matched with each other, so that the quality of spliced images and the efficiency and the reliability of image splicing are greatly improved.
Preferably, the image capturing and preprocessing module performs multi-view capturing on the target object, so as to obtain a plurality of sub-images corresponding to each capturing view one to one, specifically includes:
the target object is shot in a fixed focus mode along a plurality of different view angle directions which are mutually crossed, so that a plurality of sub-images which correspond to the different view angle directions in a preset depth of field in a one-to-one mode are obtained, wherein included angles of any two adjacent view angle directions in the different view angle directions are the same
The image capturing and preprocessing module performs noise reduction preprocessing on the sub-images, so as to obtain a plurality of preprocessed sub-images specifically includes:
distinguishing the target object from the environment background to determine the target object image area and the environment background image area contained in the sub-image,
and then performing Kalman filtering processing on the target object image area so as to realize the noise reduction preprocessing of the target object image area, and sequentially performing Fourier transform processing and high-frequency filtering processing on the environment background image area so as to realize the noise reduction preprocessing of the environment background image area.
The target object is shot along a plurality of different view angle directions which are mutually crossed, so that the image information of the target object in different view angle directions can be obtained, the target object can be panned to obtain image acquisition, and the integrity of a subsequent spliced image is ensured; in addition, because the pixel information contained in the target object image area and the environmental background image area in the sub-image are different, and different types of noise reduction preprocessing are respectively carried out on the two different image areas, the optimal image noise elimination processing can be ensured to be obtained in both the target object image area and the environmental background image area.
Preferably, the image area dividing module performs image area division on the preprocessed sub-images, so as to obtain a plurality of sub-image blocks specifically includes:
carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
the determining, by the image area dividing module, at least one overlapping sub-image block commonly included in any two pre-processed sub-images specifically includes:
the method comprises the steps of numbering and calibrating each sub image block of the image edge area of any two pre-processing sub images in sequence, comparing image profiles of two sub image blocks with the same calibration number in any two pre-processing sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is larger than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapped sub image blocks commonly contained between any two pre-processing sub images, and otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapped sub image blocks commonly contained between any two pre-processing sub images.
By dividing the grid image area of the preprocessed subimages, the subimage blocks with uniform shapes and sizes are obtained, and the convenience and reliability of calibrating the overlapped subimage blocks of any two preprocessed subimages can be improved.
Preferably, the determining, by the image difference information determining module, difference information of at least one overlapping sub image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance, and resolution specifically includes:
determining an average luminance value, an average chrominance value and an average resolution value of each overlapping sub-image block respectively comprised by one of the pre-processed images and the other pre-processed image, in any two of the pre-processed images,
and determining at least one of an average luminance difference value, an average chrominance difference value and an average resolution difference value of each overlapping sub-image block respectively included in the one preprocessed image and the other preprocessed image according to the average luminance value, the average chrominance value and the average resolution value, so as to serve as the difference information.
By determining the difference of each overlapped sub-image block in the average brightness value, the average chroma value and the average resolution value, the multithreading image state synchronous judgment of different preprocessed sub-images can be realized, so that the overlapped sub-image blocks can be ensured to be matched with each other in the brightness, the chroma and the resolution of the image, and the calculation workload of subsequent image splicing is greatly reduced.
Preferably, the image stitching module performs at least one of luminance correction, chrominance correction and resolution correction on at least one overlapping sub-image block corresponding to each of any two preprocessed images according to the difference information, specifically including:
according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapped sub image block contained in at least one of any two preprocessed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapped sub image block contained in any two preprocessed images are all lower than corresponding difference threshold values;
the image splicing module specifically splices any two corrected preprocessed images, and comprises the following steps:
and taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially performing pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing the splicing of any two preprocessed images.
The pre-processed sub-images can be matched and corrected in visual sense by performing average brightness value correction, average colorimetric value correction and average resolution value correction on the overlapped sub-image blocks, so that two different overlapped sub-image blocks can be spliced and fused accurately in the subsequent pixel convolution processing and pixel smoothing processing processes, and the quality of the spliced image and the efficiency and reliability of image splicing are improved.
It can be known from the content of the above embodiments that the image stitching method and system based on multithreading perform multi-view shooting on a target object to obtain a plurality of sub-images corresponding to each shooting view one by one, perform noise reduction preprocessing on the sub-images to obtain a plurality of preprocessed sub-images, perform image area division on the preprocessed sub-images to obtain a plurality of sub-image blocks, determine at least one overlapped sub-image block commonly included between any two preprocessed sub-images, determine difference information of at least one overlapped sub-image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance and resolution, and perform at least one of luminance correction, chrominance correction, and resolution correction on at least one overlapped sub-image block corresponding to each of any two preprocessed images according to the difference information, splicing any two corrected preprocessed images; therefore, the multithreading-based image splicing method and the multithreading-based image splicing system have the advantages that the image quality after splicing and the efficiency and the reliability of image splicing are greatly improved.
Claims (10)
1. The image stitching method based on multithreading is characterized by comprising the following steps:
step S1, carrying out multi-view shooting on the target object so as to obtain a plurality of sub-images corresponding to each shooting view one by one, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
step S2, dividing the image area of the preprocessed sub-images to obtain a plurality of sub-image blocks, and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
step S3, determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
and step S4, according to the difference information, at least one of brightness correction, chroma correction and resolution correction is carried out on at least one overlapped sub image block corresponding to any two preprocessed images, and then any two preprocessed images after correction are spliced.
2. The multithread-based image stitching method of claim 1, wherein:
in step S1, performing multi-view shooting on the target object to obtain a plurality of sub-images corresponding to each shooting view one-to-one, and performing noise reduction preprocessing on the sub-images to obtain a plurality of preprocessed sub-images specifically includes:
step S101, performing fixed-focus shooting on the target object along a plurality of different view angle directions which are mutually crossed, so as to obtain a plurality of sub-images which correspond to the different view angle directions in a preset depth, wherein included angles of any two adjacent view angle directions in the different view angle directions are the same;
step S102, distinguishing a target object from an environment background of the sub-image, so as to determine a target object image area and an environment background image area contained in the sub-image;
step S103, performing Kalman filtering processing on the target object image area so as to realize denoising preprocessing on the target object image area, and sequentially performing Fourier transform processing and high-frequency filtering processing on the environment background image area so as to realize denoising preprocessing on the environment background image area.
3. The multithread-based image stitching method of claim 1, wherein:
in step S2, the image area division is performed on the preprocessed sub-images to obtain a plurality of sub-image blocks, and the determining at least one overlapping sub-image block commonly included in any two preprocessed sub-images specifically includes:
step S201, carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
step S202, sequentially numbering and calibrating each sub image block of the image edge area of any two pre-processing sub images, comparing image profiles of two sub image blocks with the same calibration number in any two pre-processing sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is greater than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapping sub image blocks commonly included between any two pre-processing sub images, and otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapping sub image blocks commonly included between any two pre-processing sub images.
4. The multithread-based image stitching method of claim 1, wherein:
in step S3, the determining the difference information of at least one overlapping sub image block corresponding to each of any two pre-processed images in at least one of luminance, chrominance and resolution specifically includes:
step S301, in any two pre-processed images, determining an average brightness value, an average chroma value and an average resolution value of each overlapping sub-image block respectively contained in one pre-processed image and the other pre-processed image;
step S302, determining at least one of an average luminance difference value, an average chrominance difference value, and an average resolution difference value of each overlapping sub-image block included in each of the one pre-processed image and the other pre-processed image according to the average luminance value, the average chrominance value, and the average resolution value, as the difference information.
5. The multithread-based image stitching method of claim 1, wherein:
in step S4, according to the difference information, at least one of luminance correction, chrominance correction, and resolution correction is performed on at least one overlapping sub image block corresponding to each of any two preprocessed images, and then splicing any two preprocessed images after correction specifically includes:
step S401, according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapping sub image block included in at least one of any two pre-processed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapping sub image block included in any two pre-processed images are all lower than corresponding difference threshold values;
and step S402, taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially performing pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing splicing of any two preprocessed images.
6. The multithreading-based image splicing system is characterized by comprising an image shooting and preprocessing module, an image area dividing module, an image difference information determining module and an image splicing module; wherein the content of the first and second substances,
the image shooting and preprocessing module is used for carrying out multi-view shooting on a target object so as to obtain a plurality of sub-images which are in one-to-one correspondence with each shooting view, and carrying out noise reduction preprocessing on the sub-images so as to obtain a plurality of preprocessed sub-images;
the image area division module is used for carrying out image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks and determining at least one overlapped sub-image block commonly contained between any two preprocessed sub-images;
the image difference information determining module is used for determining difference information of at least one overlapping sub image block corresponding to any two preprocessed images in at least one of brightness, chroma and resolution;
the image splicing module is used for performing at least one of brightness correction, chroma correction and resolution correction on at least one overlapped sub-image block corresponding to any two preprocessed images respectively according to the difference information, and then splicing any two preprocessed images after correction.
7. The multithread-based image stitching system of claim 6, wherein:
the image shooting and preprocessing module performs multi-view shooting on the target object so as to obtain a plurality of sub-images corresponding to each shooting view one by one, specifically comprising:
performing fixed-focus shooting on the target object along a plurality of mutually crossed different view angle directions to acquire a plurality of sub-images which correspond to the different view angle directions in a preset depth of field one by one, wherein included angles of any two adjacent view angle directions in the different view angle directions are the same
The image capturing and preprocessing module performs noise reduction preprocessing on the sub-images, so as to obtain a plurality of preprocessed sub-images specifically includes:
distinguishing the target object from the environmental background to determine the target object image area and the environmental background image area contained in the sub-image,
and performing Kalman filtering processing on the target object image area so as to realize noise reduction preprocessing on the target object image area, and sequentially performing Fourier transform processing and high-frequency filtering processing on the environment background image area so as to realize noise reduction preprocessing on the environment background image area.
8. The multithread-based image stitching system of claim 6, wherein:
the image area dividing module performs image area division on the preprocessed sub-images, so as to obtain a plurality of sub-image blocks specifically comprises:
carrying out grid image area division on the preprocessed sub-images so as to obtain a plurality of sub-image blocks which are rectangular and have the same area;
the determining, by the image area dividing module, at least one overlapping sub-image block commonly included in any two pre-processed sub-images specifically includes:
the method comprises the steps of numbering and calibrating each sub image block of the image edge area of any two pre-processing sub images in sequence, comparing image profiles of two sub image blocks with the same calibration number in any two pre-processing sub images, if the comparison process determines that the image profile similarity between the two sub image blocks with the same calibration number is larger than or equal to a preset similarity threshold, determining that the two sub image blocks with the same calibration number belong to overlapped sub image blocks commonly contained between any two pre-processing sub images, and otherwise, determining that the two sub image blocks with the same calibration number do not belong to overlapped sub image blocks commonly contained between any two pre-processing sub images.
9. The multithread-based image stitching system of claim 6, wherein:
the determining, by the image difference information determining module, difference information of at least one overlapping sub image block corresponding to each of any two preprocessed images in at least one of luminance, chrominance, and resolution specifically includes:
determining an average luminance value, an average chrominance value and an average resolution value of each overlapping sub-image block respectively comprised by one of the pre-processed images and the other pre-processed image, in any two of the pre-processed images,
and determining at least one of an average luminance difference value, an average chrominance difference value and an average resolution difference value of each overlapping sub-image block included in the one preprocessed image and the other preprocessed image respectively according to the average luminance value, the average chrominance value and the average resolution value, so as to serve as the difference information.
10. The multithread-based image stitching system of claim 6, wherein:
the image stitching module performs at least one of luminance correction, chrominance correction and resolution correction on at least one overlapping sub-image block corresponding to each of any two preprocessed images according to the difference information, and specifically includes:
according to the difference information, performing average brightness value correction, average chroma value correction and average resolution value correction on each overlapped sub image block contained in at least one of any two preprocessed images, so that the average brightness difference value, the average chroma difference value and the average resolution difference value of each overlapped sub image block contained in any two preprocessed images are all lower than corresponding difference threshold values;
the image stitching module stitching any two corrected preprocessed images specifically comprises the following steps:
and taking all overlapped sub image blocks contained in any two corrected preprocessed images as a spliced image area, and sequentially carrying out pixel convolution processing and pixel smoothing processing on the spliced image area, thereby realizing the splicing of any two preprocessed images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010913588.5A CN112102168B (en) | 2020-09-03 | 2020-09-03 | Image splicing method and system based on multiple threads |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010913588.5A CN112102168B (en) | 2020-09-03 | 2020-09-03 | Image splicing method and system based on multiple threads |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112102168A true CN112102168A (en) | 2020-12-18 |
| CN112102168B CN112102168B (en) | 2023-03-28 |
Family
ID=73758479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010913588.5A Active CN112102168B (en) | 2020-09-03 | 2020-09-03 | Image splicing method and system based on multiple threads |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112102168B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114025088A (en) * | 2021-10-31 | 2022-02-08 | 中汽院(重庆)汽车检测有限公司 | Method for realizing all-around image safety monitoring by arranging intelligent camera on commercial vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100165087A1 (en) * | 2008-12-31 | 2010-07-01 | Corso Jason J | System and method for mosaicing endoscope images captured from within a cavity |
| CN101888484A (en) * | 2010-06-28 | 2010-11-17 | 浙江大华技术股份有限公司 | Automatic correction method for matching deviation of double-channel CCD images |
| CN103269416A (en) * | 2013-05-03 | 2013-08-28 | 四川虹视显示技术有限公司 | Device and method for achieving video image tiled display by adoption of parallel processing mode |
| CN103971352A (en) * | 2014-04-18 | 2014-08-06 | 华南理工大学 | Rapid image splicing method based on wide-angle lenses |
| CN105574838A (en) * | 2014-10-15 | 2016-05-11 | 上海弘视通信技术有限公司 | Image registration and splicing method of multi-view camera and device thereof |
| CN106157304A (en) * | 2016-07-01 | 2016-11-23 | 成都通甲优博科技有限责任公司 | A kind of Panoramagram montage method based on multiple cameras and system |
| CN110782394A (en) * | 2019-10-21 | 2020-02-11 | 中国人民解放军63861部队 | Panoramic video rapid splicing method and system |
-
2020
- 2020-09-03 CN CN202010913588.5A patent/CN112102168B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100165087A1 (en) * | 2008-12-31 | 2010-07-01 | Corso Jason J | System and method for mosaicing endoscope images captured from within a cavity |
| CN101888484A (en) * | 2010-06-28 | 2010-11-17 | 浙江大华技术股份有限公司 | Automatic correction method for matching deviation of double-channel CCD images |
| CN103269416A (en) * | 2013-05-03 | 2013-08-28 | 四川虹视显示技术有限公司 | Device and method for achieving video image tiled display by adoption of parallel processing mode |
| CN103971352A (en) * | 2014-04-18 | 2014-08-06 | 华南理工大学 | Rapid image splicing method based on wide-angle lenses |
| CN105574838A (en) * | 2014-10-15 | 2016-05-11 | 上海弘视通信技术有限公司 | Image registration and splicing method of multi-view camera and device thereof |
| CN106157304A (en) * | 2016-07-01 | 2016-11-23 | 成都通甲优博科技有限责任公司 | A kind of Panoramagram montage method based on multiple cameras and system |
| CN110782394A (en) * | 2019-10-21 | 2020-02-11 | 中国人民解放军63861部队 | Panoramic video rapid splicing method and system |
Non-Patent Citations (2)
| Title |
|---|
| YANYING WANG 等: ""An Optimized Image Mosaic Algorithm with Parallel IO and Dynamic Grouped Parallel Strategy Based on Minimal Spanning Tree"" * |
| 马孝贺: ""多幅图像的高分辨率无缝快速拼接方法"" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114025088A (en) * | 2021-10-31 | 2022-02-08 | 中汽院(重庆)汽车检测有限公司 | Method for realizing all-around image safety monitoring by arranging intelligent camera on commercial vehicle |
| CN114025088B (en) * | 2021-10-31 | 2023-08-22 | 中汽院(重庆)汽车检测有限公司 | Method for realizing safety monitoring of all-round image by arranging intelligent camera on operating automobile |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112102168B (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9811946B1 (en) | High resolution (HR) panorama generation without ghosting artifacts using multiple HR images mapped to a low resolution 360-degree image | |
| CN108760767B (en) | Large-size liquid crystal display defect detection method based on machine vision | |
| CN103517041B (en) | Based on real time panoramic method for supervising and the device of polyphaser rotation sweep | |
| US10306141B2 (en) | Image processing apparatus and method therefor | |
| CN105516614B (en) | Information processing method and electronic equipment | |
| CN106469444B (en) | Rapid image fusion method for eliminating splicing gap | |
| CN111583116A (en) | Video panorama stitching and fusing method and system based on multi-camera cross photography | |
| CN107085842B (en) | Self-learning multipath image fusion real-time correction method and system | |
| CN103733608B (en) | Image processing apparatus and control method therefor | |
| CN112085659B (en) | Panorama splicing and fusing method and system based on dome camera and storage medium | |
| CN103986875A (en) | Image acquiring device, method and terminal and video acquiring method | |
| CN109658427B (en) | Image processing method and device | |
| US20120243776A1 (en) | Image processing apparatus, image processing method, and program | |
| CN103167223A (en) | Mobile device with wide-angle shooting function and image acquisition method thereof | |
| CN107358577A (en) | A kind of quick joining method of cubic panorama | |
| US8731327B2 (en) | Image processing system and image processing method | |
| CN112102168B (en) | Image splicing method and system based on multiple threads | |
| CN114331835A (en) | Panoramic image splicing method and device based on optimal mapping matrix | |
| CN115376028A (en) | Target detection method based on dense feature point splicing and improved YOLOV5 | |
| CN117114997B (en) | Image stitching method and device based on suture line search algorithm | |
| CN113989392A (en) | Color chessboard calibration method and device of splicing camera and camera | |
| CN112037128B (en) | Panoramic video stitching method | |
| CN116051681B (en) | Processing method and system for generating image data based on intelligent watch | |
| CN105704363B (en) | Image data processing method and device | |
| CN115619636A (en) | Image stitching method, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |