CN108257090B - High-dynamic image splicing method for airborne line-scan camera - Google Patents
High-dynamic image splicing method for airborne line-scan camera Download PDFInfo
- Publication number
- CN108257090B CN108257090B CN201810031563.5A CN201810031563A CN108257090B CN 108257090 B CN108257090 B CN 108257090B CN 201810031563 A CN201810031563 A CN 201810031563A CN 108257090 B CN108257090 B CN 108257090B
- Authority
- CN
- China
- Prior art keywords
- image
- line
- scan
- time
- scanning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
Abstract
The invention discloses a high dynamic image splicing method for an airborne line-scan camera, and belongs to the field of signal processing. Firstly, gray level conversion is carried out on a line scanning image acquired by a line scanning camera, and an interested area in the line scanning image is enhanced. Secondly, the line scanning image passes through a band-pass filter to extract an interested part. And then, reconstructing a line scan image, and subtracting the product of the interested part and the signal displacement coefficient from the original line scan image to obtain a result image. Finally, the line-scanning images are spliced to generate a blank matrix with fixed line number and column number, and the two-dimensional matrix of the processed images is written into the matrix in sequence. And finally obtaining the panoramic image until the image to be spliced is the last line scanning image. The method provided by the invention can adapt to the change of factors such as light rays in a scene, can process the line scanning image acquired under the low illumination condition, improves the robustness of the system, and can ensure the alignment of seams while completing the splicing of the line scanning image.
Description
Technical Field
The invention belongs to the field of signal processing, and relates to a high-dynamic image splicing method for an airborne line scan camera.
Background
The airborne camera is important equipment for aerial survey of the airplane, and has the task of acquiring high-resolution target image information from the air in real time, processing the information and then transmitting the information to the control center in real time to finish the acquisition of ground target images.
The airplane is high in dynamic, the speed can reach 205m/s, the view field is large when the onboard camera shoots, and the required precision is high. The linear array camera has ultrahigh resolution and high image acquisition speed, so that the prior aircraft mostly adopts the linear array camera to acquire ground images. And (3) as the airplane moves forward, the linear array camera scans the ground target line by line and uniformly detects the whole target. The acquired line scanning images are directly sent to a computer for storage, and because one line of image data is acquired at each moment, the line of images is required to be preprocessed and then combined into a complete image, and then the required information is acquired by utilizing the image, so that the whole process from measurement to imaging is realized in a digital mode.
The precondition for the normal shooting of the airborne camera is that the visual axis of the camera is kept stable relative to the geographic coordinate system, that is, the airborne camera is required to always fly horizontally, linearly and at a constant speed in the shooting process of the camera, and the stability of the flying attitude is also kept. However, changes of air current, wind direction and wind speed in the air cause changes of attitude roll, pitch and yaw of the carrier, and the changes of the attitude of the carrier directly affect the geometric distortion of line scan images and the relative translation between adjacent line scan images, so that the result of directly splicing the line scan images is not ideal.
Disclosure of Invention
The invention aims to solve the problems and provides a high-dynamic image splicing method facing an airborne line scan camera.
The invention discloses a high dynamic image splicing method facing an airborne line scan camera, which comprises the following specific steps:
s1, acquiring a line scanning image through a line camera carried on the airplane; setting the scanning frequency of the line scan camera to be K hertz and the time length to be T seconds, and scanning to obtain a video stream of line scan images of K multiplied by T lines; K. t is a positive integer;
s2, starting from the first frame of the video stream, sequentially carrying out gray level transformation on each frame of line scanning image;
s3, I (x, t) represents the intensity of the line scan image processed by S2 in space x and at time t, and I (x,0) ═ f (x) represents the initial time. The displacement function is denoted by δ (t), i.e. the change of a pixel from 0 to time x t, the image intensity at time t is denoted as I (x, t) ═ f (x + δ (t)). Expanding the one-dimensional signal by a first-order Taylor series about x to obtain
S4, the line scan image represented by I (x, t) in S3 is passed through a time-domain band-pass filter, in order to extract all signals except f (x), and the extracted interesting part is B (x, t).
S5, reconstructing the line scan image, and obtaining the intensity of the reconstructed image at the space x and the time t as I' (x, t);
i' (x, t) ═ I (x, t) - α B (x, t); where the factor alpha is a coefficient of signal displacement.
The factor α in the present invention is a coefficient of signal displacement, and f (x) is transformed from δ (t) to (1- α) δ (t) by a displacement function at time t through operation. The value of α is not constant, and is different according to the line scan image to be processed.
And S6, splicing the line-scan images. An empty matrix with the number of rows K x T is generated, and the two-dimensional matrix of the image processed at S5 is written into the empty matrix.
And S7, continuously repeating the steps S3-S5 for the ith row-scanned image input subsequently, circularly writing the matrix of the ith row-scanned image processed in the step S5 into the matrix generated in the step S6 in sequence until the image to be spliced is the KxT row-scanned image, and finally obtaining the K xT image spliced panoramic image.
In S2, the grayscale transformation may be linear transformation, logarithmic transformation, power transformation, or piecewise linear transformation. When a linear transformation of a proportional function is used, the output luminance and the input luminance of the linear transformation are interchangeable and are completely included in the graph.
In S4, the time domain band-pass filter is a second-order IIR filter, and the center frequency of the filter is set to be the scanning frequency of the line camera. A bandpass filter is used for I (x, t) in S3, and B (x, t) ═ H (x, t) × I (x, t) × denotes convolution. Extracting all signals except f (x), i.e.
Instant game
In S5, I' (x, t) is further represented as:
then, using taylor's formula, the output I' (x, t) ═ f (x + (1- α) δ (t)) is obtained, and the displacement function of f (x) at time t is converted from δ (t) to (1- α) δ (t).
The invention has the advantages and beneficial effects that: a signal processing method is adopted instead of the traditional image processing method, the line scanning image is processed according to the angle of the signal changing along with time, and the line scanning image is spliced by using Taylor series expansion and meanwhile the joint alignment is ensured. The method is simple and easy to implement, and the calculated amount is small. The method provided by the invention can adapt to the change of factors such as light rays in a scene, can process the line scanning image acquired under the low-light condition, and improves the robustness of the system.
Drawings
FIG. 1 is a flow chart of the high dynamic image stitching method for an airborne line-scan camera according to the present invention;
FIG. 2 is a line scan image stitching without gray scale transformation according to the present invention;
FIG. 3 is a line scan image stitching process after gray scale conversion processing according to the present invention;
FIG. 4 is a schematic diagram of images that are directly stitched according to a conventional method;
FIG. 5 is a detail view of the portion framed in FIG. 4;
FIG. 6 is a schematic diagram of images spliced using the method of the present invention;
fig. 7 is a detailed view of the portion framed in fig. 6.
Detailed Description
Most of the existing image splicing methods are used for splicing images with overlapped parts, and the disclosed method for splicing non-overlapped parts is determined according to the similarity degree of the contents of two images near the splicing position. The two methods are characterized in that the method starts from the angle of image processing, does not relate to the change of image signals in a time domain, and has more complex processing process and larger calculated amount. The existing image splicing method is to perform spatial registration on an image sequence, and then form a wide-view image containing each image after image transformation, resampling and image fusion. Compared with the splicing of line scan images acquired by an airborne line scan camera, the line scan images acquired by the line scan camera are obviously different because the line scan images acquired by the line scan camera have no overlapping part.
The invention can consider that at least one part of pixel points with the same data characteristics of two adjacent lines of images. The resolution of the airborne linear array camera is 1 multiplied by 2048, the scanning frequency of the linear array camera is set to KHz, and the time length is set to T seconds. Because the line-scan camera can only acquire one line-scan image with the line number of 1 and the column number of 2048 at each moment, the change of each pixel point in the line-scan image in the shooting process of the camera can be regarded as the change along with time, namely the translational motion of a one-dimensional signal. The intensity of the line-scan image at space x and at time t is therefore denoted by I (x, t), and I (x,0) ═ f (x) denotes the initial instant. After the relative displacement is eliminated by signal processing, the images are sequentially spliced according to the time sequence of collecting the line scanning images to obtain a complete ground target image.
In order to better understand the technical scheme of the invention, the method of the invention is described in detail below by combining the line scan data of the airport runway entrance obtained by the line camera (the scanning frequency is 1kHz) when the plane flies through the airport and the attached drawings.
The specific implementation flow of the high-dynamic image stitching method for the airborne line-scan camera is shown in fig. 1, and the implementation steps are described below.
S1, setting the scanning frequency of the line camera to be K hertz (Hz) and the time length to be T seconds, namely scanning K lines of data per second, and scanning K multiplied by T lines of images with the line width of 1 multiplied by 2048 in total. And inputting the acquired line scanning data into a computer through a line scan camera carried on the airplane to obtain a video stream with the resolution of 1 multiplied by 2048 and K frames per second.
And S2, starting from the first frame (the first line scan image), sequentially carrying out gray level transformation on each frame of line scan image, enhancing the region of interest in the line scan image, and eliminating the influence of low illumination to make the line scan image clearer. This step is repeated for the input ith line scan image, i being 1,2, …, K × T. As shown in fig. 2, the line scan image stitching is performed without the gray level conversion, and the line scan image stitching shown in fig. 3 is obtained after the gray level conversion.
The gray scale transformation is commonly linear transformation, logarithmic transformation, power transformation, piecewise linear transformation and the like, and the image can be enhanced through histogram equalization. The present invention employs a proportional linear transformation whose output luminance and input luminance are interchangeable and are fully included in the graphics. When the illumination condition is not ideal, and the scanning image contrast is low, the processing process is simple and effective.
S3, I (x, t) represents the intensity of the line scan image processed by S2 in space x and at time t, and I (x,0) ═ f (x) represents the initial time. The displacement function is denoted by δ (t), i.e. the change of a pixel from 0 to time x t, the image intensity at time t is denoted as I (x, t) ═ f (x + δ (t)). The one-dimensional signal is expanded about x with a first order taylor series, resulting in:
s4, the line scan image indicated by I (x, t) in S3 is passed through a time-domain bandpass filter, i.e., B (x, t) ═ H (x, t) × I (x, t). Where B (x, t) represents the extracted portion of interest, H (x, t) represents the time domain bandpass filter, and the symbol x represents the convolution.
The second-order IIR filter is adopted in the present invention, since K pictures per second are known, and K is known to be 1kHz, the center frequency of the filter is set to 1kHz, and the passband is set to 0.8kHz to 1.2 kHz. Using a band-pass filter for I (x, t) in S3, all signals except f (x) are extracted, i.e.
And order
And S5, reconstructing a line scan image, and subtracting the interested part extracted in the S4 from the original line scan image and multiplying the factor alpha to obtain a result image I' (x, t). The factor a is the coefficient of the signal displacement.
The original signal minus the bandpass signal multiplied by a factor α is denoted by I' (x, t) ═ I (x, t) - α B (x, t).
The result of the further transformation is that,
further, the taylor equation is used to obtain an output I' (x, t) ═ f (x + (1- α) δ (t)). And (x) converting the displacement function of f (x) at the time t from delta (t) to (1-alpha) delta (t). Since the ideal goal of the S4 filter is to extract all but f (x), this process is well within the bandwidth of the filter for the video signal to be processed. However, experiments find that a very small portion of the signal is outside the bandwidth of the filter, so the actual value of the factor α needs to be determined experimentally. Through a plurality of experiments, the actual value of the factor alpha in the embodiment of the invention is 0.18, so as to ensure that the video signal to be processed is within the bandwidth of the filter as much as possible.
And S6, splicing the line-scan images. An empty matrix with the number of rows K × T and the number of columns 2048 is generated, and the two-dimensional matrix of the line-scan image processed in S5 is written into the empty matrix.
And S7, continuously repeating the step of the ith row-scanning image which is input subsequently through S3-S5, circularly performing, sequentially writing the two-dimensional matrix of the row-scanning image processed by the S5 into the matrix generated by the S6 until the image to be spliced is the KxT row-scanning image, and finally obtaining the K xT image spliced panoramic image.
As shown in fig. 4, the images obtained by direct stitching according to the conventional method are shown in fig. 6. Fig. 5 and 7 are enlarged images of frame portions of images obtained by the conventional method and the method of the present invention, respectively, in which a vertical dot-dash line is a reference line. According to the comparison of the two images, the accuracy of splicing and aligning the line scanning images cannot be guaranteed in the traditional method, and the right side of the zebra crossing is shifted and bent when the line scanning images are visible to the naked eye; in the images spliced by the method, the vertical alignment effect of the zebra stripes is obviously superior to that of the traditional method.
Claims (6)
1. A high dynamic image stitching method facing an airborne line scan camera is characterized by comprising the following steps:
s1, acquiring a line scanning image through a line camera carried on the airplane; setting the scanning frequency of the line scan camera to be K hertz and the time length to be T seconds, and scanning to obtain a video stream of line scan images of K multiplied by T lines; K. t is a positive integer;
s2, starting from the first frame of the video stream, sequentially carrying out gray level transformation on each frame of line scanning image;
s3, representing the intensity of the line scan image processed by the S2 at the space x and the time t by I (x, t); i (x,0) ═ f (x) denotes the initial time image intensity; the displacement function is represented by δ (t), which is the change of pixel from 0 to time x t; then at time t the image intensity is expressed as I (x, t) ═ f (x + δ (t)); for I (x, t), a first order Taylor series is used for expansion about x to obtain
S4, passing the line scanning image represented by I (x, t) in S3 through a time domain band-pass filter H (x, t), wherein the aim is to extract all signals except f (x), and the extracted interesting part is B (x, t);
s5, reconstructing the line scan image, and obtaining the intensity of the reconstructed image at the space x and the time t as I' (x, t);
i' (x, t) ═ I (x, t) - α B (x, t); wherein the factor α is a coefficient of signal displacement;
the I' (x, t) is represented as:
then, using Taylor formula to obtain output I' (x, t) ═ f (x + (1-alpha) delta (t)), and converting the displacement function of f (x) at time t from delta (t) to (1-alpha) delta (t);
s6, splicing the line scanning images; the method comprises the following steps: generating a blank matrix with K multiplied by T rows, and writing the two-dimensional matrix of the image processed by S5 into the blank matrix; when the resolution of the line scan image is 1 × 2048, the column of the empty matrix is 2048;
and S7, continuously repeating the steps S3-S5 for the ith row-scanned image input subsequently, sequentially writing the matrix of the ith row-scanned image processed in the step S5 into the matrix generated in the step S6 until the image to be spliced is the KxT row-scanned image, and finally obtaining the K xT image spliced panoramic image.
2. The method for stitching high-dynamic images facing an onboard line-scan camera according to claim 1, wherein the resolution of the line-scan image is 1 x 2048.
3. The method for stitching high-dynamic images facing an onboard line-scan camera according to claim 1, wherein in the step S2, the gray scale transformation is linear transformation, logarithmic transformation, power transformation or piecewise linear transformation.
4. The method for stitching high-dynamic images facing an onboard line-scan camera of claim 1, wherein in step S2, the gray scale transformation is a linear transformation of a proportional function, and the output brightness and the input brightness are interchangeable.
5. The method for stitching high-dynamic images facing an airborne line-scan camera according to claim 1, wherein in S4, the time-domain band-pass filter is a second-order recursive filter, and the center frequency of the filter is set to the scanning frequency of the line-scan camera;
using a bandpass filter for I (x, t), B (x, t) ═ H (x, t) × I (x, t) —, denotes convolution; let the interested part
6. The method for stitching high-dynamic images facing an airborne line-scan camera according to claim 1, wherein in S5, the value of the factor α is 0.18.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810031563.5A CN108257090B (en) | 2018-01-12 | 2018-01-12 | High-dynamic image splicing method for airborne line-scan camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810031563.5A CN108257090B (en) | 2018-01-12 | 2018-01-12 | High-dynamic image splicing method for airborne line-scan camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108257090A CN108257090A (en) | 2018-07-06 |
| CN108257090B true CN108257090B (en) | 2021-03-30 |
Family
ID=62726372
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810031563.5A Active CN108257090B (en) | 2018-01-12 | 2018-01-12 | High-dynamic image splicing method for airborne line-scan camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108257090B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109685845B (en) * | 2018-11-26 | 2023-04-07 | 普达迪泰（天津）智能装备科技有限公司 | POS system-based real-time image splicing processing method for FOD detection robot |
| CN111612690B (en) * | 2019-12-30 | 2023-04-07 | 苏州纽迈分析仪器股份有限公司 | Image splicing method and system |
| CN113378672A (en) * | 2021-05-31 | 2021-09-10 | 扬州大学 | Multi-target detection method for defects of power transmission line based on improved YOLOv3 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184560A (en) * | 2011-03-25 | 2011-09-14 | 南昌航空大学 | Template-based CCD-DR (charge coupled device-digital radiography) image splicing method |
| CN104463886A (en) * | 2014-12-18 | 2015-03-25 | 西南交通大学 | Processing method and device for images shot by line-scan digital camera |
| CN105376485A (en) * | 2015-11-09 | 2016-03-02 | 南京大学 | Bidirectional real-time vehicle chassis image synthetic method based on linear array type camera |
| US20180005392A1 (en) * | 2016-06-30 | 2018-01-04 | Datalogic ADC, Inc. | Item image stitching from multiple line-scan images for barcode scanning systems |
-
2018
- 2018-01-12 CN CN201810031563.5A patent/CN108257090B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184560A (en) * | 2011-03-25 | 2011-09-14 | 南昌航空大学 | Template-based CCD-DR (charge coupled device-digital radiography) image splicing method |
| CN104463886A (en) * | 2014-12-18 | 2015-03-25 | 西南交通大学 | Processing method and device for images shot by line-scan digital camera |
| CN105376485A (en) * | 2015-11-09 | 2016-03-02 | 南京大学 | Bidirectional real-time vehicle chassis image synthetic method based on linear array type camera |
| US20180005392A1 (en) * | 2016-06-30 | 2018-01-04 | Datalogic ADC, Inc. | Item image stitching from multiple line-scan images for barcode scanning systems |
Non-Patent Citations (3)
| Title |
|---|
| LI WANG 等.A robust multisource image automatic registration system based on the SIFT descriptor.《International Journal of Remote Sensing》.2012,第33卷(第12期),第3850–3869页. * |
| 刘洪兴 等.反射式拼接CCD相机渐晕图像校正.《光电工程》.2015,第42卷(第2期),第9-14页. * |
| 王长缨 等.一种多CCD图像拼接的快速算法.《半导体光电》.2006,第27卷(第2期),第206-209页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108257090A (en) | 2018-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10301041B2 (en) | Systems and methods for tracking moving objects | |
| CN108257090B (en) | High-dynamic image splicing method for airborne line-scan camera | |
| CN111062905A (en) | Infrared and visible light fusion method based on saliency map enhancement | |
| US20120027371A1 (en) | Video summarization using video frames from different perspectives | |
| CN105182678B (en) | A kind of system and method based on multichannel camera observation space target | |
| CN111693025B (en) | Remote sensing image data generation method, system and equipment | |
| Lee et al. | Cloud removal of satellite images using convolutional neural network with reliable cloudy image synthesis model | |
| CN111899345B (en) | Three-dimensional reconstruction method based on 2D visual image | |
| CN107845145B (en) | Three-dimensional reconstruction system and method under electron microscopic scene | |
| CN112288637A (en) | Unmanned aerial vehicle aerial image rapid splicing device and rapid splicing method | |
| CN110532853B (en) | Remote sensing time-exceeding phase data classification method and device | |
| CN104156977B (en) | Point target movement velocity detection method based on multiple linear moveout scanning, extending and sampling | |
| CN108109118B (en) | Aerial image geometric correction method without control points | |
| Huang et al. | Image registration among UAV image sequence and Google satellite image under quality mismatch | |
| WO2014056473A2 (en) | Method for image processing and method that can be performed therewith for the automatic detection of objects, observation device and method for high-precision tracking of the course followed by launched rockets over large distances | |
| CN108876755B (en) | Improved method for constructing color background of gray level image | |
| CN116883235A (en) | Distributed photoelectric oriented image stitching method and device | |
| CN112154484A (en) | Ortho image generation method, system and storage medium | |
| CN115830064A (en) | Weak and small target tracking method and device based on infrared pulse signals | |
| CN104143196B (en) | A kind of point target detection method based on many alignment moveout scans extension sampling | |
| Luo et al. | An Evolutionary Shadow Correction Network and A Benchmark UAV Dataset for Remote Sensing Images | |
| Whitley | Unmanned aerial vehicles (UAVs) for documenting and interpreting historical archaeological Sites: Part II—return of the drones | |
| CN114630024B (en) | Retina-imitating non-uniform imaging method based on array camera system | |
| Madadikhaljan et al. | Single-Image Dehazing on Aerial Imagery Using Convolutional Neural Networks | |
| EP4276429A1 (en) | Multi-spectral and panchromatic imaging apparatus and associated system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |