CN117876217A - Infrared image stitching method, system and storage medium based on small overlapping area - Google Patents
Infrared image stitching method, system and storage medium based on small overlapping area Download PDFInfo
- Publication number
- CN117876217A CN117876217A CN202311742619.5A CN202311742619A CN117876217A CN 117876217 A CN117876217 A CN 117876217A CN 202311742619 A CN202311742619 A CN 202311742619A CN 117876217 A CN117876217 A CN 117876217A
- Authority
- CN
- China
- Prior art keywords
- infrared
- image
- frame
- images
- infrared image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000006835 compression Effects 0.000 claims abstract description 33
- 238000007906 compression Methods 0.000 claims abstract description 33
- 238000013507 mapping Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 20
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 12
- 230000000694 effects Effects 0.000 abstract description 8
- 230000007704 transition Effects 0.000 abstract description 8
- 238000007500 overflow downdraw method Methods 0.000 abstract description 4
- 238000005728 strengthening Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000003331 infrared imaging Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 210000003128 head Anatomy 0.000 description 5
- 238000011282 treatment Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an infrared image stitching method, an infrared image stitching system and a storage medium based on a small overlapping area, wherein the method comprises the following steps: respectively carrying out dynamic range compression on a plurality of frames of high-order wide infrared images to obtain a plurality of frames of low-order wide infrared compressed images; respectively scaling the resolution of the infrared image corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image stitching condition to obtain multi-frame infrared images to be stitched; and respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize the infrared image splicing. According to the invention, the brightness of the images to be spliced tends to be consistent on the premise of strengthening details by utilizing the dynamic range compression of the infrared images, then a gradual-in gradual-out image fusion method is utilized, a more natural and smooth transition effect can be realized in a narrow image overlapping area, finally the infrared images are scaled to a target resolution, and then the panoramic image output with a large field of view is realized in a frame-by-frame splicing mode, so that the time consumption of the infrared image splicing process is reduced.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, a system, and a storage medium for stitching infrared images based on a small overlapping area.
Background
The image stitching technology combines a plurality of input views with a certain position relationship into a panoramic output image with a large field of view and without stitching marks, and is one of the most common technologies in the field of computer vision. In order to meet the requirements of different application fields and usage scenes, particularly some large dynamic range infrared scenes, on an infrared imaging system, a high-performance infrared imaging system is designed, and an A/D converter with a bit depth of 14 bits or more is adopted to sample and quantify the output signal of a detector. The gray scale display dynamic range of the current general display device is only 256 levels, and the gray scale number which can be resolved by human eyes is only 128 levels at most. In order to facilitate the observation of the target in the infrared scene by the human eye and simultaneously consider the display of the display device, 14 or higher original infrared data obtained by the infrared imaging system are often required to be compressed to be 8-bit data width by a specific algorithm for display, which is a dynamic range compression technology in the infrared imaging system.
The traditional image stitching technology causes great difficulty for image registration, and in the process of image fusion, the overlapping area of the images is used as a buffer area for smooth transition between two images, so that the diversity of selectable algorithms is limited, and in addition, the difficulty of pursuing good visual consistency effect in the small smooth transition area is further increased due to the influence of the dynamic range compression effect of the infrared image on the final display of the images. Therefore, how to improve the infrared image stitching effect in the case of small overlapping area is a urgent problem to be solved under the condition of reducing the time consumption of the infrared image stitching process.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide an infrared image stitching method, an infrared image stitching system and a storage medium based on a small overlapping area, and aims to solve the technical problem of how to improve the infrared image stitching effect under the condition of the small overlapping area under the condition of reducing the time consumption of an infrared image stitching process.
In order to achieve the above object, the present invention provides an infrared image stitching method based on a small overlapping region, which includes:
uniformly collecting multi-frame high-position wide infrared images around an axis by an infrared image detector mounted on a holder;
respectively carrying out dynamic range compression on a plurality of frames of high-order wide infrared images to obtain a plurality of frames of low-order wide infrared compressed images;
respectively scaling the resolution of the infrared image corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image stitching condition to obtain multi-frame infrared images to be stitched;
and respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize the infrared image splicing.
Optionally, the step of respectively performing dynamic range compression on the multi-frame high-bit wide infrared image to obtain the multi-frame low-bit wide infrared compressed image includes:
determining infrared image sequence information corresponding to multi-frame high-bit wide infrared images;
dividing the high-order wide infrared image blocks of each frame respectively by limiting contrast histogram equalization according to the infrared image sequence information to obtain a plurality of image sub-blocks corresponding to the high-order wide infrared image of each frame;
and respectively compressing the dynamic range of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image.
Optionally, the step of compressing the dynamic range of the multiple image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image includes:
respectively compressing dynamic ranges of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain multi-frame infrared compressed images;
determining a plurality of compressed image sub-blocks corresponding to each frame of infrared compressed image, and determining gray level differences between adjacent sub-blocks in the plurality of compressed image sub-blocks;
and respectively processing the plurality of compressed image sub-blocks according to the gray level difference to obtain a multi-frame low-bit-width infrared compressed image.
Optionally, the step of processing the plurality of compressed image sub-blocks according to the gray level difference to obtain a multi-frame low-bit-width infrared compressed image includes:
judging whether the gray level difference is larger than a preset gray level threshold value or not;
and when the gray level difference is smaller than or equal to the preset gray level threshold value, smoothing the plurality of compressed image sub-blocks in a gradual-in gradual-out thought correction mode to obtain a multi-frame low-bit-width infrared compressed image.
Optionally, after the step of determining whether the gray level difference is greater than a preset gray level threshold, the method further includes:
and when the gray level difference is larger than the preset gray level threshold value, performing brightness compensation on the plurality of compressed image sub-blocks in a gray level weighting mode to obtain a multi-frame low-bit wide infrared compressed image.
Optionally, before the step of respectively storing the multiple frames of to-be-spliced infrared images to the corresponding DDR storage addresses to realize the splicing of the infrared images, the method further includes:
determining the total frame number corresponding to a plurality of frames of high-order wide infrared images, and determining the image size corresponding to a plurality of frames of infrared images to be spliced;
and determining DDR storage addresses corresponding to the multi-frame infrared images to be spliced according to the infrared image sequence information, the total frame number, the image size and the panoramic large-view-field image frame information.
In addition, in order to achieve the above object, the present invention further provides an infrared image stitching system based on a small overlapping area, where the infrared image stitching system based on the small overlapping area includes:
the acquisition module is used for uniformly acquiring multi-frame high-position wide infrared images around the axis by an infrared image detector mounted on the holder at fixed points;
the compression module is used for respectively carrying out dynamic range compression on the multi-frame high-bit wide infrared image to obtain a multi-frame low-bit wide infrared compressed image;
the scaling module is used for respectively scaling the infrared image resolution corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image splicing condition to obtain multi-frame infrared images to be spliced;
and the splicing module is used for respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize infrared image splicing.
In addition, in order to achieve the above object, the present invention also proposes an infrared image stitching device based on a small overlapping area, the device comprising: the system comprises a memory, a processor and a small overlap region-based infrared image stitching program stored on the memory and executable on the processor, the small overlap region-based infrared image stitching program configured to implement the steps of the small overlap region-based infrared image stitching method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a small overlap region-based infrared image stitching program, which when executed by a processor, implements the steps of the small overlap region-based infrared image stitching method as described above.
According to the invention, firstly, a plurality of frames of high-order wide infrared images are uniformly collected around an axis by an infrared image detector mounted on a cradle head, then, the dynamic range compression is respectively carried out on the plurality of frames of high-order wide infrared images to obtain a plurality of frames of low-order wide infrared compressed images, then, the infrared image resolution corresponding to the plurality of frames of low-order wide infrared compressed images is respectively scaled to a target resolution according to the infrared image splicing condition to obtain a plurality of frames of infrared images to be spliced, and finally, the plurality of frames of infrared images to be spliced are respectively stored to corresponding DDR storage addresses to realize the infrared image splicing. According to the invention, the dynamic range compression of the infrared image is utilized to enable the brightness of the images to be spliced to be consistent on the premise of strengthening details, and on the basis, a gradual-in gradual-out image fusion method is utilized, so that a more natural and smooth transition effect can be realized in a narrow image overlapping area, finally, the infrared image is scaled to a target resolution, and then, the panoramic image output with a large field of view is realized in a frame-by-frame splicing mode, thereby reducing the time consumption of the infrared image splicing process.
Drawings
FIG. 1 is a schematic structural diagram of an infrared image stitching device based on a small overlap region in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of the method for stitching infrared images based on small overlapping areas according to the present invention;
FIG. 3 is a flowchart of a technique for stitching infrared images facing a small overlapping region according to a first embodiment of the method for stitching infrared images based on a small overlapping region of the present invention;
FIG. 4 is a schematic diagram illustrating the compression of the dynamic range of an infrared image according to a first embodiment of the method for stitching an infrared image based on a small overlapping region of the present invention;
FIG. 5 is a schematic diagram of an infrared image stitching process according to a first embodiment of the present invention;
fig. 6 is a block diagram of a first embodiment of an infrared image stitching system based on a small overlap region.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an infrared image stitching device based on a small overlapping region in a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the infrared image splicing apparatus based on the small overlap region may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage system separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 does not constitute a limitation of the small overlap region based infrared image stitching device, and may include more or fewer components than illustrated, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and an infrared image stitching program based on a small overlap area may be included in the memory 1005 as one storage medium.
In the infrared image stitching device based on the small overlapping area shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the infrared image stitching device based on the small overlapping area can be arranged in the infrared image stitching device based on the small overlapping area, and the infrared image stitching device based on the small overlapping area calls an infrared image stitching program based on the small overlapping area stored in the memory 1005 through the processor 1001 and executes the infrared image stitching method based on the small overlapping area.
The embodiment of the invention provides an infrared image stitching method based on a small overlapping area, and referring to fig. 2, fig. 2 is a schematic flow diagram of a first embodiment of the infrared image stitching method based on the small overlapping area.
In this embodiment, the method for stitching infrared images based on small overlapping areas includes the following steps:
step S10: and uniformly acquiring multi-frame high-position wide infrared images around the axis by using an infrared image detector carried on the cradle head.
It is to be understood that the execution subject of the present embodiment may be an infrared image stitching system based on a small overlapping area with functions of data processing, network communication, program running, etc., or may be other computer devices with similar functions, etc., and the present embodiment is not limited thereto.
The present embodiment is realized based on SOC. The SOC is a heterogeneous chip of an FPGA+ARM architecture, program development on the SOC is divided into a PL end (Programmable Logic), the PL end refers to an FPGA part of the SOC, a PS end (Processing System) refers to an ARM part of the SOC.
In this embodiment, referring to fig. 3, fig. 3 is a flowchart of an infrared image stitching technique facing a small overlapping area according to a first embodiment of the infrared image stitching method based on a small overlapping area, and the present invention mainly includes 3 parts:
infrared image dynamic range compression: the high-bit-width infrared image is processed into an 8-bit infrared image.
Infrared image scaling processing: the infrared image is scaled to a target resolution.
And (3) infrared image stitching treatment: and splicing the infrared images into a panoramic output image with a large field of view and without splicing marks.
In a specific implementation, the infrared image is core input data of the embodiment, and the multi-frame high-bit-width infrared image (for example, multi-frame 14-bit infrared image) is obtained by uniformly shooting and collecting around an axis and fixed point through an infrared image detector carried on the holder. Due to the function of the cradle head, multiple frames of high-level wide infrared images are positioned on the same horizontal reference, the motion between adjacent infrared images can be approximately regarded as translational motion, and the acquired images do not need to be subjected to displacement, image rotation and other treatments.
It should be noted that the collected multi-frame high-bandwidth infrared images are sequentially arranged multi-frame high-bandwidth infrared images.
Step S20: and respectively carrying out dynamic range compression on the multi-frame high-order wide infrared images to obtain multi-frame low-order wide infrared compressed images.
Further, respectively carrying out dynamic range compression on the multi-frame high-order wide infrared images to obtain multi-frame low-order wide infrared compressed images, wherein the processing mode of the multi-frame low-order wide infrared compressed images is to determine infrared image sequence information corresponding to the multi-frame high-order wide infrared images; dividing the line blocks of each frame of high-bit-width infrared image by limiting contrast histogram equalization according to the infrared image sequence information to obtain a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image; and respectively compressing the dynamic range of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image.
It should be understood that the infrared image sequence information is the sequential arrangement information of the multi-frame high-bit-width infrared images.
In a specific implementation, the partial function is implemented at the PL end, and each frame of high-order wide infrared image is respectively subjected to column block division by limiting contrast histogram equalization according to sequential arrangement information of multiple frames of high-order wide infrared images, so as to obtain a plurality of image sub-blocks corresponding to each frame of high-order wide infrared image, and referring to fig. 4, fig. 4 is an infrared image dynamic range compression schematic diagram of a first embodiment of the infrared image stitching method based on a small overlapping region.
It should be further noted that the column block division may be user-defined, and may be divided into N blocks, where N is greater than or equal to 4.
Further, respectively compressing a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image in a dynamic range according to the bit-width conversion mapping table to obtain a plurality of frames of low-bit-width infrared compressed images; determining a plurality of compressed image sub-blocks corresponding to each frame of infrared compressed image, and determining gray level differences between adjacent sub-blocks in the plurality of compressed image sub-blocks; and respectively processing the plurality of compressed image sub-blocks according to the gray level difference to obtain a multi-frame low-bit-width infrared compressed image.
It should also be noted that a mapping table (i.e., a bit width conversion mapping table) of 14 bits to 8 bits is constructed for each image sub-block. And carrying out dynamic range compression on a plurality of image sub-blocks (14 bit image sub-blocks) corresponding to each frame of high-bit-width infrared image (namely, 14bit infrared image) to obtain multi-frame infrared compressed images (namely, 8bit infrared images), wherein each frame of infrared compressed image is provided with a plurality of corresponding compressed image sub-blocks, and the plurality of compressed image sub-blocks are all 8bit image sub-blocks.
Further, processing the plurality of compressed image sub-blocks according to the gray level difference respectively to obtain a multi-frame low-bit-width infrared compressed image in a processing mode of judging whether the gray level difference is larger than a preset gray level threshold value or not; when the gray level difference is smaller than or equal to a preset gray level threshold value, smoothing the plurality of compressed image sub-blocks in a gradual-in gradual-out thought correction mode to obtain a multi-frame low-bit wide infrared compressed image; when the gray level difference is larger than a preset gray level threshold value, brightness compensation is carried out on the plurality of compressed image sub-blocks in a gray level weighting mode, so that a multi-frame low-bit wide infrared compressed image is obtained.
Wherein the gray level difference between adjacent sub-blocks of the plurality of compressed image sub-blocks can be calculated according to the following pixels:
pixel 1= (a1×frame N region 3 mapping table+b1×frame N region 2 mapping table)/(a1+b1);
……
pixel n= (an×frame n+1 region 1 map+bn×frame N region 8 map)/(an+bn) +luminance compensation. (note: a1, b1, an, bn represent image pixel coordinate distances).
The preset gray threshold may be set by user, which is not limited in this embodiment.
In a specific implementation, due to an inherent mechanism of infrared imaging, a phenomenon of inconsistent brightness inevitably exists in an acquired infrared image sequence (multi-frame high-bit-width infrared image with sequential arrangement), so that in order to realize smooth transition between frames, the final dynamic range compression output of each pixel of an image edge sub-block is corrected by adopting the concept of gradual-in gradual-out based on a mapping table of two frames of adjacent sub-blocks, and gray weighting is additionally introduced to carry out brightness compensation.
Step S30: and respectively scaling the resolution of the infrared image corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image stitching condition to obtain multi-frame infrared images to be stitched.
It should be noted that the infrared image stitching condition is the actual infrared image stitching requirement.
In a specific implementation, this part of the functionality is implemented at the PL end. According to the actual infrared image splicing requirement, the resolution of an original infrared image (namely a multi-frame low-bit-width infrared compressed image) is scaled to a target resolution by utilizing bilinear interpolation.
Step S40: and respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize the infrared image splicing.
Further, respectively storing the multiple frames of infrared images to be spliced to corresponding DDR storage addresses, and determining the total frame number corresponding to the multiple frames of high-order wide infrared images and the image size corresponding to the multiple frames of infrared images to be spliced before the step of splicing the infrared images; and determining DDR storage addresses corresponding to the multi-frame infrared images to be spliced according to the infrared image sequence information, the total frame number, the image size and the panoramic large-view-field image frame information.
In a specific implementation, the partial functions are implemented in cooperation with the PL end and the PS end. The method comprises the steps of completing splicing of large-view-field infrared images in a frame-by-frame splicing mode, calculating and obtaining DDR storage addresses corresponding to each frame of images (namely, multi-frame infrared images to be spliced) in space according to information such as total frame number (namely, total frame number corresponding to multi-frame high-order wide infrared images), scaled infrared image size, panoramic large-view-field image frame and the like of a circle of infrared image shot uniformly around an AXIs by an infrared image detector carried on a cloud deck, storing the images subjected to pre-processing to the corresponding storage addresses in the DDR by using an AXI_DataMover IP core, and then reading the spliced images according to the space positions by using the IPs such as AXI Video Direct Memory Access, video Timing Controller, AXI4-Stream to Video Out and the like after updating of the images. Referring to fig. 5, fig. 5 is a schematic diagram of an infrared image stitching flow according to a first embodiment of the infrared image stitching method based on a small overlapping region of the present invention.
In this embodiment, firstly, multiple frames of high-order wide infrared images are uniformly collected around an axis by an infrared image detector mounted on a holder, then dynamic range compression is performed on the multiple frames of high-order wide infrared images respectively to obtain multiple frames of low-order wide infrared compressed images, then the infrared image resolution corresponding to the multiple frames of low-order wide infrared compressed images is scaled to a target resolution according to the infrared image splicing condition respectively to obtain multiple frames of to-be-spliced infrared images, and finally the multiple frames of to-be-spliced infrared images are stored to corresponding DDR storage addresses respectively to realize infrared image splicing. According to the embodiment, the brightness of the images to be spliced tends to be consistent on the premise of strengthening details by utilizing the dynamic range compression of the infrared images, a gradual-in gradual-out image fusion method is utilized on the basis, a more natural and smooth transition effect can be achieved in a narrow image overlapping area, finally, the infrared images are scaled to a target resolution, then, panoramic image output with a large view field is achieved in a frame-by-frame splicing mode, and the time consumption of an infrared image splicing process is reduced.
Referring to fig. 6, fig. 6 is a block diagram illustrating a first embodiment of an infrared image stitching system based on a small overlap region according to the present invention.
As shown in fig. 6, an infrared image stitching system based on a small overlapping area according to an embodiment of the present invention includes:
the acquisition module 6001 is used for uniformly acquiring multi-frame high-position wide infrared images around an axis by an infrared image detector mounted on the cradle head.
The present embodiment is realized based on SOC. The SOC is a heterogeneous chip of an FPGA+ARM architecture, program development on the SOC is divided into a PL end (Programmable Logic), the PL end refers to an FPGA part of the SOC, a PS end (Processing System) refers to an ARM part of the SOC.
In this embodiment, referring to fig. 3, fig. 3 is a flowchart of an infrared image stitching technique facing a small overlapping area according to a first embodiment of the infrared image stitching method based on a small overlapping area, and the present invention mainly includes 3 parts:
infrared image dynamic range compression: the high-bit-width infrared image is processed into an 8-bit infrared image.
Infrared image scaling processing: the infrared image is scaled to a target resolution.
And (3) infrared image stitching treatment: and splicing the infrared images into a panoramic output image with a large field of view and without splicing marks.
In a specific implementation, the infrared image is core input data of the embodiment, and the multi-frame high-bit-width infrared image (for example, multi-frame 14-bit infrared image) is obtained by uniformly shooting and collecting around an axis and fixed point through an infrared image detector carried on the holder. Due to the function of the cradle head, multiple frames of high-level wide infrared images are positioned on the same horizontal reference, the motion between adjacent infrared images can be approximately regarded as translational motion, and the acquired images do not need to be subjected to displacement, image rotation and other treatments.
It should be noted that the collected multi-frame high-bandwidth infrared images are sequentially arranged multi-frame high-bandwidth infrared images.
The compression module 6002 is configured to perform dynamic range compression on multiple frames of high-order wide infrared images respectively, so as to obtain multiple frames of low-order wide infrared compressed images.
Further, respectively carrying out dynamic range compression on the multi-frame high-order wide infrared images to obtain multi-frame low-order wide infrared compressed images, wherein the processing mode of the multi-frame low-order wide infrared compressed images is to determine infrared image sequence information corresponding to the multi-frame high-order wide infrared images; dividing the line blocks of each frame of high-bit-width infrared image by limiting contrast histogram equalization according to the infrared image sequence information to obtain a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image; and respectively compressing the dynamic range of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image.
It should be understood that the infrared image sequence information is the sequential arrangement information of the multi-frame high-bit-width infrared images.
In a specific implementation, the partial function is implemented at the PL end, and each frame of high-order wide infrared image is respectively subjected to column block division by limiting contrast histogram equalization according to sequential arrangement information of multiple frames of high-order wide infrared images, so as to obtain a plurality of image sub-blocks corresponding to each frame of high-order wide infrared image, and referring to fig. 4, fig. 4 is an infrared image dynamic range compression schematic diagram of a first embodiment of the infrared image stitching method based on a small overlapping region.
It should be further noted that the column block division may be user-defined, and may be divided into N blocks, where N is greater than or equal to 4.
Further, respectively compressing a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image in a dynamic range according to the bit-width conversion mapping table to obtain a plurality of frames of low-bit-width infrared compressed images; determining a plurality of compressed image sub-blocks corresponding to each frame of infrared compressed image, and determining gray level differences between adjacent sub-blocks in the plurality of compressed image sub-blocks; and respectively processing the plurality of compressed image sub-blocks according to the gray level difference to obtain a multi-frame low-bit-width infrared compressed image.
It should also be noted that a mapping table (i.e., a bit width conversion mapping table) of 14 bits to 8 bits is constructed for each image sub-block. And carrying out dynamic range compression on a plurality of image sub-blocks (14 bit image sub-blocks) corresponding to each frame of high-bit-width infrared image (namely, 14bit infrared image) to obtain multi-frame infrared compressed images (namely, 8bit infrared images), wherein each frame of infrared compressed image is provided with a plurality of corresponding compressed image sub-blocks, and the plurality of compressed image sub-blocks are all 8bit image sub-blocks.
Further, processing the plurality of compressed image sub-blocks according to the gray level difference respectively to obtain a multi-frame low-bit-width infrared compressed image in a processing mode of judging whether the gray level difference is larger than a preset gray level threshold value or not; when the gray level difference is smaller than or equal to a preset gray level threshold value, smoothing the plurality of compressed image sub-blocks in a gradual-in gradual-out thought correction mode to obtain a multi-frame low-bit wide infrared compressed image; when the gray level difference is larger than a preset gray level threshold value, brightness compensation is carried out on the plurality of compressed image sub-blocks in a gray level weighting mode, so that a multi-frame low-bit wide infrared compressed image is obtained.
Wherein the gray level difference between adjacent sub-blocks of the plurality of compressed image sub-blocks can be calculated according to the following pixels:
pixel 1= (a1×frame N region 3 mapping table+b1×frame N region 2 mapping table)/(a1+b1);
……
pixel n= (an×frame n+1 region 1 map+bn×frame N region 8 map)/(an+bn) +luminance compensation. (note: a1, b1, an, bn represent image pixel coordinate distances).
The preset gray threshold may be set by user, which is not limited in this embodiment.
In a specific implementation, due to an inherent mechanism of infrared imaging, a phenomenon of inconsistent brightness inevitably exists in an acquired infrared image sequence (multi-frame high-bit-width infrared image with sequential arrangement), so that in order to realize smooth transition between frames, the final dynamic range compression output of each pixel of an image edge sub-block is corrected by adopting the concept of gradual-in gradual-out based on a mapping table of two frames of adjacent sub-blocks, and gray weighting is additionally introduced to carry out brightness compensation.
The scaling module 6003 is configured to scale the resolution of the infrared image corresponding to the multi-frame low-bit-width infrared compressed image to the target resolution according to the infrared image stitching condition, so as to obtain a multi-frame infrared image to be stitched.
It should be noted that the infrared image stitching condition is the actual infrared image stitching requirement.
In a specific implementation, this part of the functionality is implemented at the PL end. According to the actual infrared image splicing requirement, the resolution of an original infrared image (namely a multi-frame low-bit-width infrared compressed image) is scaled to a target resolution by utilizing bilinear interpolation.
And the splicing module is used for respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize infrared image splicing.
Further, respectively storing the multiple frames of infrared images to be spliced to corresponding DDR storage addresses, and determining the total frame number corresponding to the multiple frames of high-order wide infrared images and the image size corresponding to the multiple frames of infrared images to be spliced before the step of splicing the infrared images; and determining DDR storage addresses corresponding to the multi-frame infrared images to be spliced according to the infrared image sequence information, the total frame number, the image size and the panoramic large-view-field image frame information.
In a specific implementation, the partial functions are implemented in cooperation with the PL end and the PS end. The method comprises the steps of completing splicing of large-view-field infrared images in a frame-by-frame splicing mode, calculating and obtaining DDR storage addresses corresponding to each frame of images (namely, multi-frame infrared images to be spliced) in space according to information such as total frame number (namely, total frame number corresponding to multi-frame high-order wide infrared images), scaled infrared image size, panoramic large-view-field image frame and the like of a circle of infrared image shot uniformly around an AXIs by an infrared image detector carried on a cloud deck, storing the images subjected to pre-processing to the corresponding storage addresses in the DDR by using an AXI_DataMover IP core, and then reading the spliced images according to the space positions by using the IPs such as AXI Video Direct Memory Access, video Timing Controller, AXI4-Stream to Video Out and the like after updating of the images. Referring to fig. 5, fig. 5 is a schematic diagram of an infrared image stitching flow according to a first embodiment of the infrared image stitching method based on a small overlapping region of the present invention.
In this embodiment, firstly, multiple frames of high-order wide infrared images are uniformly collected around an axis by an infrared image detector mounted on a holder, then dynamic range compression is performed on the multiple frames of high-order wide infrared images respectively to obtain multiple frames of low-order wide infrared compressed images, then the infrared image resolution corresponding to the multiple frames of low-order wide infrared compressed images is scaled to a target resolution according to the infrared image splicing condition respectively to obtain multiple frames of to-be-spliced infrared images, and finally the multiple frames of to-be-spliced infrared images are stored to corresponding DDR storage addresses respectively to realize infrared image splicing. According to the embodiment, the brightness of the images to be spliced tends to be consistent on the premise of strengthening details by utilizing the dynamic range compression of the infrared images, a gradual-in gradual-out image fusion method is utilized on the basis, a more natural and smooth transition effect can be achieved in a narrow image overlapping area, finally, the infrared images are scaled to a target resolution, then, panoramic image output with a large view field is achieved in a frame-by-frame splicing mode, and the time consumption of an infrared image splicing process is reduced.
Other embodiments or specific implementation manners of the infrared image stitching system based on the small overlapping area of the present invention may refer to the above method embodiments, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. The infrared image stitching method based on the small overlapping area is characterized by comprising the following steps of:
uniformly collecting multi-frame high-position wide infrared images around an axis by an infrared image detector mounted on a holder;
respectively carrying out dynamic range compression on a plurality of frames of high-order wide infrared images to obtain a plurality of frames of low-order wide infrared compressed images;
respectively scaling the resolution of the infrared image corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image stitching condition to obtain multi-frame infrared images to be stitched;
and respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize the infrared image splicing.
2. The method of claim 1, wherein the step of respectively performing dynamic range compression on the plurality of frames of high-bandwidth infrared images to obtain a plurality of frames of low-bandwidth infrared compressed images comprises:
determining infrared image sequence information corresponding to multi-frame high-bit wide infrared images;
dividing the high-order wide infrared image blocks of each frame respectively by limiting contrast histogram equalization according to the infrared image sequence information to obtain a plurality of image sub-blocks corresponding to the high-order wide infrared image of each frame;
and respectively compressing the dynamic range of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image.
3. The method of claim 2, wherein the step of performing dynamic range compression on the plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain a multi-frame low-bit-width infrared compressed image comprises:
respectively compressing dynamic ranges of a plurality of image sub-blocks corresponding to each frame of high-bit-width infrared image according to the bit-width conversion mapping table to obtain multi-frame infrared compressed images;
determining a plurality of compressed image sub-blocks corresponding to each frame of infrared compressed image, and determining gray level differences between adjacent sub-blocks in the plurality of compressed image sub-blocks;
and respectively processing the plurality of compressed image sub-blocks according to the gray level difference to obtain a multi-frame low-bit-width infrared compressed image.
4. A method according to claim 3, wherein said step of processing each of the plurality of compressed image sub-blocks according to said gray scale difference to obtain a multi-frame low-bit-width infrared compressed image comprises:
judging whether the gray level difference is larger than a preset gray level threshold value or not;
and when the gray level difference is smaller than or equal to the preset gray level threshold value, smoothing the plurality of compressed image sub-blocks in a gradual-in gradual-out thought correction mode to obtain a multi-frame low-bit-width infrared compressed image.
5. The method of claim 4, wherein after the step of determining whether the gray level difference is greater than a preset gray level threshold, further comprising:
and when the gray level difference is larger than the preset gray level threshold value, performing brightness compensation on the plurality of compressed image sub-blocks in a gray level weighting mode to obtain a multi-frame low-bit wide infrared compressed image.
6. The method of claim 2, wherein before the step of storing the plurality of frames of the infrared images to be spliced to the corresponding DDR memory addresses to achieve the infrared image splicing, the method further comprises:
determining the total frame number corresponding to a plurality of frames of high-order wide infrared images, and determining the image size corresponding to a plurality of frames of infrared images to be spliced;
and determining DDR storage addresses corresponding to the multi-frame infrared images to be spliced according to the infrared image sequence information, the total frame number, the image size and the panoramic large-view-field image frame information.
7. An infrared image stitching system based on a small overlap region, wherein the infrared image stitching system based on a small overlap region comprises:
the acquisition module is used for uniformly acquiring multi-frame high-position wide infrared images around the axis by an infrared image detector mounted on the holder at fixed points;
the compression module is used for respectively carrying out dynamic range compression on the multi-frame high-bit wide infrared image to obtain a multi-frame low-bit wide infrared compressed image;
the scaling module is used for respectively scaling the infrared image resolution corresponding to the multi-frame low-bit wide infrared compressed image to the target resolution according to the infrared image splicing condition to obtain multi-frame infrared images to be spliced;
and the splicing module is used for respectively storing the multi-frame infrared images to be spliced to the corresponding DDR storage addresses so as to realize infrared image splicing.
8. A storage medium, wherein a small overlap region based infrared image stitching program is stored on the storage medium, and the small overlap region based infrared image stitching program, when executed by a processor, implements the steps of the small overlap region based infrared image stitching method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311742619.5A CN117876217A (en) | 2023-12-15 | 2023-12-15 | Infrared image stitching method, system and storage medium based on small overlapping area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311742619.5A CN117876217A (en) | 2023-12-15 | 2023-12-15 | Infrared image stitching method, system and storage medium based on small overlapping area |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117876217A true CN117876217A (en) | 2024-04-12 |
Family
ID=90596030
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311742619.5A Pending CN117876217A (en) | 2023-12-15 | 2023-12-15 | Infrared image stitching method, system and storage medium based on small overlapping area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117876217A (en) |
-
2023
- 2023-12-15 CN CN202311742619.5A patent/CN117876217A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113034358B (en) | Super-resolution image processing method and related device | |
| CN112529775A (en) | Image processing method and device | |
| CN108171662B (en) | Method for reading image compression data and anti-distortion method comprising same | |
| US11983848B2 (en) | AI frame engine for mobile edge | |
| CN112055875A (en) | Partial image frame update system and method for electronic display | |
| US20070104360A1 (en) | System and method for capturing 3D face | |
| CN112184877B (en) | Method and system for rendering optimization of glow effect | |
| CN108564551A (en) | Fish eye images processing method and fish eye images processing unit | |
| CN111489322A (en) | Method and device for adding sky filter to static picture | |
| WO2019153731A1 (en) | Image processing method and system | |
| KR20140107044A (en) | Image subsystem | |
| US9928577B2 (en) | Image correction apparatus and image correction method | |
| CN117876217A (en) | Infrared image stitching method, system and storage medium based on small overlapping area | |
| WO2018092715A1 (en) | Image processing device, image processing method and program | |
| CN114820370A (en) | Picture conversion method of ink screen equipment, electronic equipment and storage medium | |
| CN114785957A (en) | Shooting method and device thereof | |
| CN114519753A (en) | Image generation method, system, electronic device, storage medium and product | |
| CN114612295A (en) | Image super-resolution processing method and computer equipment | |
| CN113961844B (en) | Method for displaying color DICOM image, user equipment and medium | |
| US11367167B2 (en) | Neural network-based image processing with artifact compensation | |
| CN118317144B (en) | Video processing method, device, computer equipment and storage medium | |
| CN108696670A (en) | Tile reuse is carried out in imaging | |
| CN117745531B (en) | Image interpolation method, apparatus and readable storage medium | |
| WO2023197284A1 (en) | Saliency-based adaptive color enhancement | |
| WO2024130715A1 (en) | Video processing method, video processing apparatus and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |