CN101819677B - Fusion processing system of multi-sensor images - Google Patents
Fusion processing system of multi-sensor images Download PDFInfo
- Publication number
- CN101819677B CN101819677B CN2010101437417A CN201010143741A CN101819677B CN 101819677 B CN101819677 B CN 101819677B CN 2010101437417 A CN2010101437417 A CN 2010101437417A CN 201010143741 A CN201010143741 A CN 201010143741A CN 101819677 B CN101819677 B CN 101819677B
- Authority
- CN
- China
- Prior art keywords
- image
- image data
- matched
- images
- target location
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to a fusion processing system of multi-sensor images, comprising image data acquisition devices, multi-characteristic image data processing devices and data fuse devices, wherein the number of the image data acquisition devices and the multi-characteristic image data processing devices corresponds to that of sensors; image data output by the sensors are preprocessed by the image data acquisition devices and are cached in a mode of whole page burst; the multi-characteristic image data processing devices are used for extracting the image data stored by the image data acquisition devices, finishing the multi-characteristic operation, generating images to be matched, calculating the position of a target in the images to be matched and storing the images to be matched and the target position into the image data acquisition devices; and the data fusion devices are used for reading the images to be matched and the target positions of various image data acquisition devices and carrying out image matching and fusion processing on the images to generate final fusion images. The invention has the advantages of high practicability, capability of effectively meeting the service requirements of real-time property and excellent fuse effect, wide application range and strong feasibility.
Description
Technical field
The present invention relates to a kind of image processing system, particularly a kind of fusion processing system of multi-sensor images.
Background technology
In recent years, no matter in military field or in non-military field, sensor Data Fusion has all obtained to pay close attention to widely, is applied in the growing field.Yet the data fusion platform that adopts at present can't effectively satisfy in the application needs to image real time fusion display process.Make up a kind ofly can handle in real time, data fusion platform that syncretizing effect is good is extremely urgent.
Summary of the invention
The fusion processing system of multi-sensor images that the technical matters that the present invention will solve provides and a kind ofly can handle in real time, syncretizing effect is good.
In order to solve the problems of the technologies described above, fusion processing system of multi-sensor images of the present invention comprises and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device; Said image data acquiring device carries out after the pre-service pattern buffer memory with whole page or leaf burst to the view data of sensor output; The multi-characteristic image data processing equipment extracts image data acquiring device image stored data; Accomplish many characteristic operation of target; Generate image to be matched, calculate target position in image to be matched, and deposit image to be matched and target location in the image data acquiring device; Data fusion device reads the image to be matched and the target location of each road image data acquiring device storage, carries out images match, fusion treatment, generates last fused images.
The present invention utilizes the image data acquiring device to accomplish image reception, transmission and the preprocessing function of multisensor; Utilize the multi-characteristic image data processing equipment to generate image to be matched; And calculating target position in image to be matched; Utilize data fusion device to each road image mate, fusion treatment, generate last fused images.The present invention has very high practicality, can effectively satisfy the request for utilization real-time, that syncretizing effect is good, be widely used, and exploitativeness is strong.
Said multi-characteristic image treating apparatus comprises:
The device that is used for reads image data;
The device that is used for read sensor pixel resolution multiplying power, focal length value and sensor visual angle;
The device of the image identical with the image aspects of regulation sensor acquisition is arrived in the image transformation that is used for obtaining in real time;
Be used to generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it device of image data acquiring device in;
Be used for device according to gradation of image information calculations image gradient;
Be used for confirming the device of object edge based on image gradient;
Be used for device according to the object edge calculated target positions;
Be used for whether putting effective marker, and deposit the target location device of image data acquiring device in according to the target location.
Said data fusion device comprises:
Be used to read the device of the image to be matched of each road image data acquiring device storage;
Be used to read the device of target position information;
Be used to judge whether effectively device of target location;
Be used to accomplish the device of image registration:
If the target location is effective, then according to this road image of target location storage of target location in the image to be matched of this road with respect to the specified standard image; If the target location is invalid; Then setting big or small image with the specified standard picture centre is matching template, in this road image, finds the image identical with this matching template gray-scale value, calculates the position of this image with respect to standard picture; And, accomplish image registration according to this road image of result of calculation storage;
Be used for the image behind each width of cloth registration is carried out the device that laplacian pyramid decomposes;
Be used for device that each tomographic image after decomposing is merged:
For the top layer exploded view as the peaked pixel of chosen area average gradient as the pixel of fused image on this zone; For other layers exploded view as the peaked pixel of favored area average energy as the pixel of fused image on this zone;
Be used for doing the device that the laplacian pyramid inverse transformation obtains final fused images to merging pyramid.
Description of drawings
Below in conjunction with accompanying drawing and embodiment the present invention is done further explain.
Fig. 1 is a sensor image fusion processing system structured flowchart of the present invention.
Fig. 2 is for realizing hardware circuit synoptic diagram of the present invention.
Fig. 3 is a multi-characteristic image treating apparatus program flow diagram.
Fig. 4 is the data fusion device program flow diagram.
Embodiment:
As shown in Figure 1, fusion processing system of multi-sensor images of the present invention comprises and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device;
As shown in Figure 2, said image data acquiring device adopts PLD (FPGA) and storer (SDRAM); The image that sensor is taken in is sent into FPGA, and image deposits SDRAM through pre-service (promptly view data being organized) back in the pattern that whole page or leaf happens suddenly in FPGA, through the FPGA sequential control, makes SDRAM realize the function of multiport image buffer.
Said multi-characteristic image data processing equipment adopts DSPs or other common flush bonding processors to realize.
Said data fusion device adopts microcomputer or other common flush bonding processors to realize.
First port of SDRAM supplies the image input to use, and the view data of sensor output collects among the SDRAM through FPGA.Second port supplies the real-time reading images of DSPs, and the image to be matched and the target location of generation are temporarily stored among the SDRAM through FPGA; DSPs can control FPGA through this port, like control images acquired size, image memory location etc.The 3rd port supplies PCI-bridge (PCI bridge) to use, and computing machine can read the image to be matched of arbitrary frame at any time through pci bus, and the target location that DSPs calculates and the information such as image to be matched of generation are also delivered in the computing machine through this port.
As shown in Figure 3, DSPs is responsible for multi-characteristic image and handles, and multi-characteristic image handling procedure flow process comprises the steps:
Initialization;
Reads image data;
Orientation, the luffing angle of read sensor pixel resolution multiplying power, focal length value and sensor;
Regulation is a standard transducer with one of them sensor; Based on the orientation angles of orientation angles, luffing angle and the required standard sensor of the sensor of real-time acquisition, the difference between the luffing angle, image is transformed to the image identical with the image aspects of regulation sensor acquisition;
Generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it in SDRAM assigned address;
According to gradation of image information calculations image gradient to be matched;
Confirm object edge according to image gradient;
According to the object edge calculated target positions;
Judge whether the target location is effective,, then deposit the target location in SDRAM if the target location is effective;
Return.
Microcomputer reads image to be matched and target position information through PCI with dma mode, and carries out fusion treatment, and idiographic flow comprises the steps:
Initialization:
Read each road image to be matched of storing among the SDRAM through pci bus;
Read target position information in the image to be matched of each road through pci bus;
Judge whether the target location is effective,, then calculate target in the image to be matched of this road with respect to the position of target in the specified standard image if the target location is effective, and based on this road image of result of calculation storage; If the target location is invalid; Then the image with specified standard picture centre position 96*96 size is a matching template; In this road image, find the image of the 96*96 size identical with this matching template gray value; Calculate the position of this image with respect to standard picture, and based on this road image of result of calculation storage; Accomplish the image registration before merging;
Image behind each width of cloth registration is carried out laplacian pyramid to be decomposed;
Adopt different fusion criterions to merge to each tomographic image after decomposing; Concrete fusion rule is: select regional area for being that the territory is faced at 3 * 3 of center with its each pixel, adopt the peaked pixel of chosen area average gradient as the pixel of fused image on this zone for top layer exploded view picture; The pixel that the method that adopts provincial characteristics to measure for other layers exploded view picture is selected the energy maximum region is as the pixel of fused image on this zone;
The fusion pyramid is done the laplacian pyramid inverse transformation obtain final fused images.
Advantage of the present invention: comprise information such as half-tone information, target location because the present invention adopts DSPs to calculate respectively in the view data that different sensors collects; Give microcomputer with result of calculation and carry out images match, fusion treatment, generate last fused images.Computing velocity can satisfy real-time processing requirements, help the raising of syncretizing effect more.
Claims (3)
1. a fusion processing system of multi-sensor images is characterized in that comprising and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device; Said image data acquiring device carries out after the pre-service pattern buffer memory with whole page or leaf burst to the view data of sensor output; The multi-characteristic image data processing equipment extracts image data acquiring device image stored data; Accomplish many characteristic operation of target; Generate image to be matched, calculate target position in image to be matched, and deposit image to be matched and target location in the image data acquiring device; Data fusion device reads the image to be matched and the target location of each road image data acquiring device storage, carries out images match, fusion treatment, generates last fused images; Said image data acquiring device adopts FPGA and SDRAM to realize; The image that sensor is taken in is sent into FPGA, and image deposits SDRAM in through the pattern with whole page or leaf burst after the pre-service in FPGA, through the FPGA sequential control, make SDRAM realize the function of multiport image buffer; Said multi-characteristic image data processing equipment adopts DSPs to realize, data fusion device adopts microcomputer to realize; First port of SDRAM supplies the image input to use; The view data of sensor output collects among the SDRAM through FPGA; Second port supplies the real-time reading images of DSPs; The image to be matched and the target location that generate are temporarily stored among the SDRAM through FPGA, and DSPs controls FPGA through this port, and the 3rd port supplies the PCI bridge to use; Computing machine reads the image to be matched of arbitrary frame at any time through pci bus, the target location that DSPs calculates and the image information to be matched of generation are also delivered in the computing machine through this port.
2. fusion processing system of multi-sensor images according to claim 1 is characterized in that said multi-characteristic image treating apparatus comprises:
The device that is used for reads image data;
The device that is used for read sensor pixel resolution multiplying power, focal length value and sensor visual angle;
The device of the image identical with the image aspects of regulation sensor acquisition is arrived in the image transformation that is used for obtaining in real time;
Be used to generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it device of image data acquiring device in;
Be used for device according to gradation of image information calculations image gradient;
Be used for confirming the device of object edge based on image gradient;
Be used for device according to the object edge calculated target positions;
Be used for whether putting effective marker, and deposit the target location device of image data acquiring device in according to the target location.
3. fusion processing system of multi-sensor images according to claim 1 and 2 is characterized in that said data fusion device comprises:
Be used to read the device of the image to be matched of each road image data acquiring device storage;
Be used to read the device of target position information;
Be used to judge whether effectively device of target location;
Be used to accomplish the device of image registration:
If the target location is effective, then according to this road image of target location storage of target location in the image to be matched of this road with respect to the specified standard image; If the target location is invalid; Then setting big or small image with the specified standard picture centre is matching template, in this road image, finds the image identical with this matching template gray-scale value, calculates the position of this image with respect to standard picture; And, accomplish image registration according to this road image of result of calculation storage;
Be used for the image behind each width of cloth registration is carried out the device that laplacian pyramid decomposes;
Be used for device that each tomographic image after decomposing is merged:
For the top layer exploded view as the peaked pixel of chosen area average gradient as the pixel of fused image on this zone; For other layers exploded view as the peaked pixel of favored area average energy as the pixel of fused image on this zone;
Be used for doing the device that the laplacian pyramid inverse transformation obtains final fused images to merging pyramid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101437417A CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101437417A CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101819677A CN101819677A (en) | 2010-09-01 |
| CN101819677B true CN101819677B (en) | 2012-06-13 |
Family
ID=42654764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101437417A Expired - Fee Related CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101819677B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750247B (en) * | 2012-06-05 | 2015-01-14 | 中国科学院光电技术研究所 | Signal processing platform applicable to multi-sensor self-adaption optical system |
| CN104168455B (en) * | 2014-08-08 | 2018-03-09 | 北京航天控制仪器研究所 | A kind of space base large scene camera system and method |
| CN114915731A (en) * | 2021-02-07 | 2022-08-16 | 华为技术有限公司 | Method and apparatus for processing image data of image sensor |
| CN113611112B (en) * | 2021-07-29 | 2022-11-08 | 中国第一汽车股份有限公司 | Target association method, device, equipment and storage medium |
-
2010
- 2010-04-12 CN CN2010101437417A patent/CN101819677B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN101819677A (en) | 2010-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110599540B (en) | Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera | |
| CN104484648B (en) | Robot variable visual angle obstacle detection method based on outline identification | |
| CN109389578A (en) | Railroad track abnormality detection | |
| Godse et al. | Wavelet based image fusion using pixel based maximum selection rule | |
| CN101819677B (en) | Fusion processing system of multi-sensor images | |
| CN107851196B (en) | Image pattern matching method and device | |
| CN104899894B (en) | A kind of method that use multiple cameras carries out motion target tracking | |
| CN112967345B (en) | External parameter calibration method, device and system of fish-eye camera | |
| CN107610164A (en) | A kind of No. four Image registration methods of high score based on multiple features mixing | |
| CN110113560A (en) | The method and server of video intelligent linkage | |
| CN102096913B (en) | Multi-strategy image fusion method under compressed sensing framework | |
| CN106649607A (en) | Remote sensing image online target recognition and monitoring method and system based on vector mask | |
| CN103260008A (en) | Projection converting method from image position to actual position | |
| CN110411449B (en) | Aviation reconnaissance load target positioning method and system and terminal equipment | |
| CN114092771A (en) | Multi-sensing data fusion method, target detection device and computer equipment | |
| Huang et al. | A real-time image matching algorithm for integrated navigation system | |
| CN116976721A (en) | Power distribution operation behavior normalization evaluation method, system and computing equipment | |
| CN106403951A (en) | Computer vision based positioning system and positioning method thereof | |
| CN102778684A (en) | Embedded monocular passive target tracking positioning system and method based on FPGA (Field Programmable Gate Array) | |
| Zhuang et al. | A novel method for 3D measurement of RFID multi-tag network based on matching vision and wavelet | |
| Ressl et al. | Multi-epoch bundle block adjustment for processing large dataset of historical aerial images | |
| CN115049745B (en) | Calibration method, device, equipment and medium for roadside sensor | |
| Sun et al. | Infrared and visible image fusion based on object extraction and adaptive pulse coupled neural network via non-subsampled Shearlet transform | |
| Zhu et al. | Research on infrared and visible images registration algorithm based on graph | |
| CN116982998B (en) | Method and system for dynamically adjusting X-ray radiation dose of CT system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120613 Termination date: 20140412 |