CN112017117A - Panoramic image acquisition method and system based on thermal infrared imager - Google Patents

Abstract

The invention discloses a panoramic image acquisition method and a panoramic image acquisition system based on a thermal infrared imager. The method comprises the following steps: and carrying out multiple times of infrared thermal imaging on the target to obtain N images, wherein N is a preset value, overlapping areas exist among the N images, and the N images are spliced by using computer software to obtain a panoramic image. According to the invention, the infrared images shot by the user are spliced in a full-automatic seamless manner through the image panoramic splicing algorithm, and the overall observation and temperature measurement of the user on the large building are realized on the basis of not increasing the hardware cost and basically not increasing the complexity of the user operation.

Description

A method and system for acquiring panoramic image based on infrared thermal imager

technical field

The invention belongs to the technical field of infrared imaging, and more particularly, relates to a panoramic image acquisition method and system based on an infrared thermal imager.

Background technique

Limited by the size of the infrared device, the field of view of the infrared thermal imager is generally small. For objects of normal size, infrared can easily take pictures and measure the temperature of the target, but for large building targets, it is difficult to obtain the whole picture of the building through a single photo. .

When shooting large buildings, visible light cameras can directly use wide-angle lenses to shoot, and can achieve better shooting results. For the infrared thermal imager, it generally needs to have the temperature measurement function, and the premise that the temperature measurement curve can be accurately calibrated is that all the pixels of the entire detector area array have the same or similar ability to accept external infrared radiation, but it is obvious Wide angle lenses do not have this feature. Therefore, additionally equipped with a wide-angle lens to assist ordinary thermal imager products to take panoramic photos of large architectural targets, not only is expensive, but also difficult to ensure temperature measurement accuracy.

SUMMARY OF THE INVENTION

In view of at least one defect or improvement requirement of the prior art, the present invention provides a panoramic image acquisition method and system based on an infrared thermal imager, which realizes the overall observation and temperature measurement functions of a large-format target building on a common thermal imager. .

In order to achieve the above object, according to the first aspect of the present invention, there is provided a panoramic image acquisition method based on an infrared thermal imager, comprising: performing infrared thermal imaging on a target for multiple times, and acquiring N images, where N is a preset value, And there is an overlapping area between the N images, and the N images are spliced to obtain a panoramic image;

The splicing includes steps:

The two images to be spliced are recorded as the first image and the second image, and the FAST algorithm is used to extract the first feature point of the first image and the second feature point of the second image respectively;

Perform non-maximum suppression processing on the acquired first feature point and the second feature point, respectively, to obtain the first local maximum point and the second local maximum point;

The ORB description algorithm is used to describe the first local maximum point and the second local maximum point respectively, and the first descriptor of the first feature point and the second descriptor of the second feature point are respectively generated;

Perform Hamming distance bidirectional matching on the first feature point and the second feature point;

The matching results are further matched using the RANSAC criterion, and the perspective transformation model T and the inverse matrix Tinv of T are obtained in this further matching process;

Expand the edges of the first image to obtain the first image matrix, initialize a second image matrix that is the same size as the first image matrix, and perform an inverse bilinear interpolation operation on the initialized second image matrix according to the inverse matrix Tinv, and complete The coordinate transformation of the second image, the second image after the coordinate transformation is stored in the second image matrix;

Perform luminance value compensation on the second image matrix according to the difference between the mean gray values of the overlapping regions of the first image matrix and the second image matrix;

Count the seams of the first image matrix and the second image matrix, initialize the first fusion weight and the second fusion weight according to the position of the seam, and the first fusion weight and the second fusion weight are equal to the first image matrix;

Perform Gaussian smoothing on the first fusion weight and the second fusion weight, and use the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain a fusion image IM, where IM1 is the first image matrix, and IM2 is the second image matrix. Image matrix, W1 is the first fusion weight, IM2 is the second fusion weight;

The fused image IM is trimmed with the smallest circumscribed rectangle to obtain a stitched image.

Preferably, in the imaging process of step S1, the current field of view of the infrared thermal imager is obtained, the current field of view of the infrared thermal imager is displayed in the first area of the operation interface of the infrared thermal imager, and the current field of view of the infrared thermal imager is displayed in the first area of the operation interface of the infrared thermal imager. The second area displays the image that has been imaged in the form of thumbnails, and the user operation prompt is displayed in the third area of the operation interface of the infrared thermal imager.

According to a second aspect of the present invention, a panoramic image acquisition system based on an infrared thermal imager is provided, comprising:

The infrared thermal imager is used to perform multiple infrared thermal imaging on the target, and obtain N images, where N is a preset value, and the N images have overlapping areas with each other; the panoramic stitching module is used to perform the N images. Stitching to get a panoramic image;

The panoramic stitching module includes:

The feature point extraction module is used to record the two images to be spliced as the first image and the second image, and uses the FAST algorithm to extract the first feature point of the first image and the second feature point of the second image respectively;

The non-maximum value suppression processing module is used to perform non-maximum value suppression processing on the acquired first feature point and the second feature point respectively, and obtain the first local maximum point and the second local maximum point respectively;

A descriptor generating module is used to describe the first local maximum point and the second local maximum point respectively by using the ORB description algorithm, and respectively generate the first descriptor of the first feature point and the second descriptor of the second feature point;

The rough matching module is used to perform Hamming distance bidirectional matching on the first feature point and the second feature point;

The precise matching module is used to further match the matching results by using the RANSAC criterion, and in this further matching process, the perspective transformation model T and the inverse matrix Tinv of T are obtained;

The coordinate change module is used to expand the edge of the first image to obtain the first image matrix, and initialize a second image matrix that is the same size as the first image matrix, and perform reverse double on the initialized second image matrix according to the inverse matrix Tinv Linear interpolation operation, complete the coordinate transformation of the second image, and store the second image after the coordinate transformation in the second image matrix;

a brightness compensation module, configured to perform brightness value compensation on the second image matrix according to the difference between the mean gray values of the overlapping regions of the first image matrix and the second image matrix;

The fusion weight initialization module is used to count the seams of the first image matrix and the second image matrix, initialize the first fusion weight and the second fusion weight according to the position of the seam, and the first fusion weight and the second fusion weight are the same as The first image matrix is the same size;

The fusion module performs Gaussian smoothing on the first fusion weight and the second fusion weight, and uses the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain a fusion image IM, where IM1 is the first image matrix, IM2 is the second image matrix, W1 is the first fusion weight, and IM2 is the second fusion weight;

The stitching module is used for trimming the fused image IM with the smallest circumscribed rectangle to obtain a stitched image.

Preferably, the infrared thermal imager is also used to obtain the current field of view of the infrared thermal imager during the imaging process, and displays the current field of view of the infrared thermal imager in the first area of the operation interface of the infrared thermal imager. The second area of the operation interface of the thermal imager displays the image that has been imaged in the form of thumbnails, and the user operation prompt is displayed in the third area of the operation interface of the infrared thermal imager.

In general, compared with the prior art, the present invention realizes the user's global observation and temperature measurement of large buildings on the basis of not increasing the hardware cost and basically not increasing the operational complexity. Through this solution, users can obtain wider-format images on the basis of ensuring image quality and temperature measurement accuracy, which effectively improves product competitiveness. Specifically, the following beneficial effects are included:

(1) The system cost is low. The entire processing process of this solution is completed by a software algorithm. Only a small amount of additional operation is required by the user, and no additional peripheral hardware equipment is required to realize the overall observation and temperature measurement of the large-format target building on an ordinary thermal imager.

(2) The image distortion is small. Compared with the severe image distortion when using a wide-angle lens to obtain a large-format image, this solution seamlessly combines multiple sub-images originally captured into a large-format image through the panoramic stitching algorithm, which can ensure that the image has no obvious distortion.

(3) The temperature measurement is accurate and reliable. Compared with the temperature measurement curve of the wide-angle lens, it is difficult to accurately calibrate the temperature measurement curve. This solution directly performs panoramic stitching on the raw data of the sub-image captured by the ordinary lens, and can generate a 16-bit large-format image, which theoretically does not affect the accuracy of the temperature measurement result.

Description of drawings

1 is a schematic diagram of a panoramic image acquisition according to an embodiment of the present invention;

2 is a schematic diagram of an imaging operation interface according to an embodiment of the present invention;

3 is a schematic diagram of an imaging operation according to an embodiment of the present invention;

4 is a schematic diagram of a splicing method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a minimum circumscribed rectangle trimming process according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

A panoramic image acquisition method based on an infrared thermal imager according to an embodiment of the present invention includes an imaging step and a panoramic stitching step. The imaging step realizes multiple infrared thermal imaging of the target, and obtains N images, where N is any preset value greater than or equal to 2, and the N images have overlapping areas with each other. The panorama stitching step realizes stitching the N images to obtain a panorama image. The panorama stitching step can be realized based on computer software.

Taking N as 9, that is, 3×3 type panoramic stitching as an example, the main process is shown in Figure 1: when the user uses the infrared thermal imager with the panoramic photo function, he enters the panoramic photo mode through the menu option, and presses the operation prompts. Control the infrared thermal imager at different viewing angles in the horizontal and vertical directions, and shoot 3 × 3 sub-images with an overlap area of about 30% with each other; Select panorama stitching on the analysis software, and the module can automatically stitch the captured sub-images seamlessly to generate a large-format panoramic image of the target building. Through the above solution, the user can directly observe, measure the temperature and other subsequent processing of the panoramic image of the target building, thereby effectively improving the user experience when using the thermal imager to photograph large buildings.

Preferably, the operation interface of the infrared thermal imager is divided into three areas. During the imaging process, the current field of view of the infrared thermal imager is obtained, the current field of view of the infrared thermal imager is displayed in the first area of the operation interface of the infrared thermal imager, and the second area of the operation interface of the infrared thermal imager is displayed in the form of a thumbnail The image that has been imaged is displayed, and a user operation prompt is displayed in the third area of the operation interface of the infrared thermal imager. Taking the stitching of 9 sub-images of 3×3 as an example, the user enters the panoramic photo mode through the menu option. During the photo-taking process, the operation interface of the display screen of the infrared thermal imager is shown in Figure 2, and then the user presses Figure 3. Process to take a panoramic photo.

After acquiring N images, the process of stitching N images includes:

Preferably, stitching the N images includes the steps:

S1, read two images in the N images for splicing to obtain a spliced image.

S2, if N>2, continue to read the unstitched images in the N images, and stitch the unstitched images with the stitched images obtained last time. That is, the image that has not been stitched is stitched with the stitched image obtained in step S1.

S3, if N>3, repeat step S2 until the stitching of N images is completed, and a panoramic image is obtained. That is, the image that has not been stitched is stitched with the stitched image obtained in the previous step S2, and the stitching is performed cyclically in this way until the stitching of N images is completed.

Preferably, in the above steps S1, S2 and S3, the splicing method is shown in Figure 4, including the steps:

(1) Denote the two images to be spliced as Image1 and Image2, and use the FAST algorithm to extract the feature points points1 of the image Image1 and the feature points2 of the image Image2 respectively; if the number of feature points does not meet the preset requirements, adjust the FAST algorithm features Click the extraction threshold to re-extract.

(2) Perform non-maximum suppression processing on the acquired feature points points1 and points2 to obtain local maximum points loc1 and loc2.

(3) The ORB description algorithm is used to describe the local maximum points loc1 and loc2 respectively, and the descriptor Des1 of the feature point points1 and the descriptor Des2 of the feature point points2 are generated respectively.

(4) Perform Hamming (Hamming distance) two-way matching on the feature points points1 and points2. If the number of matching point pairs is too small, adjust the Hamming matching threshold to re-match.

(5) Use RANSAC (Random Sampling Consistency) criterion to perform further precise matching on the Hamming matching results, and obtain the perspective transformation model T and the inverse matrix Tinv of T in the precise matching process.

(6) Perform Padding (edge expansion) on Image1 to obtain the image matrix IM1, and initialize an image matrix IM2 that is the same size as the image matrix IM1, and perform reverse bilinear interpolation on the initialized image IM2 according to the inverse matrix Tinv to complete the image. For the coordinate transformation of Image2, the image Image2 after the coordinate transformation is stored in the image matrix IM2.

(7) Perform luminance value compensation on the image matrix IM2 according to the difference between the mean gray values of the overlapping regions of the image matrix IM1 and the image matrix IM2.

(8) Count the seams of the image matrix IM1 and the image matrix IM2, initialize the fusion weights W1 and W2 according to the positions of the seams, and the weights W1 and W2 are equal to the image matrix IM1.

(9) Perform Gaussian smoothing on the weights W1 and W2, and use the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain a fusion image IM.

(10) As shown in Fig. 5, the fused image IM is trimmed with the minimum circumscribed rectangle to obtain the stitched image PIC.

A panoramic image acquisition system based on an infrared thermal imager according to an embodiment of the present invention includes a panoramic photographing module and a panoramic stitching module. The panoramic camera module is an infrared thermal imager. The panoramic camera module is used to perform multiple infrared thermal imaging on the target to obtain N images, where N is a preset value, and the N images have overlapping areas with each other. The panoramic stitching module is used to stitch N images to obtain a panoramic image.

The panoramic stitching module includes:

The feature point extraction module is used to record the two images to be spliced as Image1 and Image2, and extract the feature points points1 of the image Image1 and the feature points2 of the image Image2 respectively;

The non-maximum suppression processing module is used to perform non-maximum suppression processing on the acquired feature points points1 and points2 respectively, and obtain the local maximum points loc1 and loc2 respectively;

The descriptor generation module is used to describe the local maximum points loc1 and loc2 respectively by using the ORB description algorithm, and respectively generate the descriptor Des1 of the feature point points1 and the descriptor Des2 of the feature point points2;

Coarse matching module, used for bidirectional matching of Hamming distance between feature points points1 and points2;

The precise matching module is used to further match the matching results by using the RANSAC criterion, and obtain the perspective transformation model T and the inverse matrix Tinv of T in the further matching process;

The coordinate change module is used to expand the image Image1 to obtain the image matrix IM1, and initialize an image matrix IM2 that is the same size as the image matrix IM1, and perform the reverse bilinear interpolation operation on the initialized image IM2 according to the inverse matrix Tinv, and complete The coordinate transformation of the image Image2, and the image Image2 after the coordinate transformation is stored in the image matrix IM2;

a brightness compensation module, configured to compensate the brightness value of the image matrix IM2 according to the difference between the average gray values of the overlapping regions of the image matrix IM1 and the image matrix IM2;

The fusion weight initialization module is used to count the seam between the image matrix IM1 and the image matrix IM2, and initialize the fusion weights W1 and W2 according to the position of the seam, and the weights W1 and W2 are equal to the image matrix IM1;

The fusion module is used to perform Gaussian smoothing on the weights W1 and W2, and use the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain the fusion image IM;

The stitching module is used for trimming the fused image IM with the smallest circumscribed rectangle to obtain a stitched image.

Preferably, the infrared thermal imager is also used to obtain the current field of view of the infrared thermal imager during the imaging process, and the current field of view of the infrared thermal imager is displayed in the first area of the operation interface of the infrared thermal imager. The second area of the operation interface displays the image that has been imaged in the form of thumbnails, and the user operation prompt is displayed in the third area of the operation interface of the infrared thermal imager.

The realization principle and technical effect of the panoramic image acquisition system are similar to those of the above-mentioned method, which will not be repeated here.

It must be noted that, in any of the above embodiments, the methods are not necessarily executed in sequence, and as long as it cannot be inferred from the execution logic that the methods must be executed in a certain sequence, it means that the methods can be executed in any other possible sequence.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (6)

1. A panoramic image acquisition method based on an infrared thermal imager, characterized in that, comprising: performing infrared thermal imaging on a target multiple times, acquiring N images, where N is a preset value, and the N images exist between each other. Overlapping area, stitching the N images to obtain a panoramic image; The splicing includes steps: The two images to be spliced are recorded as the first image and the second image, and the FAST algorithm is used to extract the first feature point of the first image and the second feature point of the second image respectively; Perform non-maximum suppression processing on the acquired first feature point and the second feature point, respectively, to obtain the first local maximum point and the second local maximum point; The ORB description algorithm is used to describe the first local maximum point and the second local maximum point respectively, and the first descriptor of the first feature point and the second descriptor of the second feature point are respectively generated; Perform Hamming distance bidirectional matching on the first feature point and the second feature point; The matching results are further matched using the RANSAC criterion, and the perspective transformation model T and the inverse matrix Tinv of T are obtained in this further matching process; Expand the edges of the first image to obtain the first image matrix, initialize a second image matrix that is the same size as the first image matrix, and perform an inverse bilinear interpolation operation on the initialized second image matrix according to the inverse matrix Tinv, and complete The coordinate transformation of the second image, the second image after the coordinate transformation is stored in the second image matrix; Perform luminance value compensation on the second image matrix according to the difference between the mean gray values of the overlapping regions of the first image matrix and the second image matrix; Count the seams of the first image matrix and the second image matrix, initialize the first fusion weight and the second fusion weight according to the position of the seam, and the first fusion weight and the second fusion weight are equal to the first image matrix; Perform Gaussian smoothing on the first fusion weight and the second fusion weight, and use the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain a fusion image IM, where IM1 is the first image matrix, and IM2 is the second image matrix. Image matrix, W1 is the first fusion weight, IM2 is the second fusion weight; The fused image IM is trimmed with the smallest circumscribed rectangle to obtain a stitched image. 2. The method for obtaining a panoramic image based on an infrared thermal imager according to claim 1, wherein in the imaging process of step S1, the current field of view of the infrared thermal imager is acquired, and the infrared thermal imager The first area of the operation interface of the infrared thermal imager displays the current field of view of the infrared thermal imager, the second area of the operation interface of the infrared thermal imager displays the image that has been imaged in the form of thumbnails, and the third area of the operation interface of the infrared thermal imager is displayed. Displays user action prompts. 3. The method for obtaining a panoramic image based on an infrared thermal imager as claimed in claim 1, wherein the stitching of the N images comprises the steps of: S1, read 2 images in the N images for splicing to obtain a spliced image; S2, if N>2, then continue to read the unfinished splicing images in the N images, and splicing the unfinished splicing images with the spliced images obtained last time; S3, if N>3, repeat step S2 until the stitching of N images is completed, and a panoramic image is obtained. 4. A panoramic image acquisition system based on an infrared thermal imager, characterized in that, comprising: The infrared thermal imager is used to perform multiple infrared thermal imaging on the target, and obtain N images, where N is a preset value, and the N images have overlapping areas with each other; the panoramic stitching module is used to perform the N images. Stitching to get a panoramic image; The panoramic stitching module includes: The feature point extraction module is used to record the two images to be spliced as the first image and the second image, and uses the FAST algorithm to extract the first feature point of the first image and the second feature point of the second image respectively; The non-maximum value suppression processing module is used to perform non-maximum value suppression processing on the acquired first feature point and the second feature point respectively, and obtain the first local maximum point and the second local maximum point respectively; A descriptor generating module is used to describe the first local maximum point and the second local maximum point respectively by using the ORB description algorithm, and respectively generate the first descriptor of the first feature point and the second descriptor of the second feature point; The rough matching module is used to perform Hamming distance bidirectional matching on the first feature point and the second feature point; The precise matching module is used to further match the matching results by using the RANSAC criterion, and in this further matching process, the perspective transformation model T and the inverse matrix Tinv of T are obtained; The coordinate change module is used to expand the edge of the first image to obtain the first image matrix, and initialize a second image matrix that is the same size as the first image matrix, and perform reverse double on the initialized second image matrix according to the inverse matrix Tinv Linear interpolation operation, complete the coordinate transformation of the second image, and store the second image after the coordinate transformation in the second image matrix; a brightness compensation module, configured to perform brightness value compensation on the second image matrix according to the difference between the mean gray values of the overlapping regions of the first image matrix and the second image matrix; The fusion weight initialization module is used to count the seams of the first image matrix and the second image matrix, initialize the first fusion weight and the second fusion weight according to the position of the seam, and the first fusion weight and the second fusion weight are the same as The first image matrix is the same size; The fusion module performs Gaussian smoothing on the first fusion weight and the second fusion weight, and uses the formula IM=IM1*W1+IM2*W2 to complete the fusion of the two images to obtain a fusion image IM, where IM1 is the first image matrix, IM2 is the second image matrix, W1 is the first fusion weight, and IM2 is the second fusion weight; The stitching module is used for trimming the fused image IM with the smallest circumscribed rectangle to obtain a stitched image. 5. The infrared thermal imager-based panoramic image acquisition system according to claim 4, wherein the infrared thermal imager is also used to acquire the current field of view of the infrared thermal imager during the imaging process , the current field of view of the infrared thermal imager is displayed in the first area of the operation interface of the infrared thermal imager, and the image that has been imaged is displayed in the form of a thumbnail in the second area of the operation interface of the infrared thermal imager. The third area of the operation interface displays user operation prompts. 6. The system for obtaining panoramic images based on an infrared thermal imager as claimed in claim 4, wherein the stitching of the N images comprises the steps of: S1, read 2 images in the N images for splicing to obtain a spliced image; S2, if N>2, then continue to read the unfinished splicing images in the N images, and splicing the unfinished splicing images with the spliced images obtained last time; S3, if N>3, repeat step S2 until the stitching of N images is completed, and a panoramic image is obtained.

Images