CN101819677A - Fusion processing system of multi-sensor images - Google Patents
Fusion processing system of multi-sensor images Download PDFInfo
- Publication number
- CN101819677A CN101819677A CN201010143741A CN201010143741A CN101819677A CN 101819677 A CN101819677 A CN 101819677A CN 201010143741 A CN201010143741 A CN 201010143741A CN 201010143741 A CN201010143741 A CN 201010143741A CN 101819677 A CN101819677 A CN 101819677A
- Authority
- CN
- China
- Prior art keywords
- image
- image data
- matched
- images
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a fusion processing system of multi-sensor images, comprising image data acquisition devices, multi-characteristic image data processing devices and data fuse devices, wherein the number of the image data acquisition devices and the multi-characteristic image data processing devices corresponds to that of sensors; image data output by the sensors are preprocessed by the image data acquisition devices and are cached in a mode of whole page burst; the multi-characteristic image data processing devices are used for extracting the image data stored by the image data acquisition devices, finishing the multi-characteristic operation, generating images to be matched, calculating the position of a target in the images to be matched and storing the images to be matched and the target position into the image data acquisition devices; and the data fusion devices are used for reading the images to be matched and the target positions of various image data acquisition devices and carrying out image matching and fusion processing on the images to generate final fusion images. The invention has the advantages of high practicability, capability of effectively meeting the service requirements of real-time property and excellent fuse effect, wide application range and strong feasibility.
Description
Technical field
The present invention relates to a kind of image processing system, particularly a kind of fusion processing system of multi-sensor images.
Background technology
In recent years, no matter in military field or in non-military field, sensor Data Fusion has all obtained to pay close attention to widely, is applied in the growing field.Yet at present the data fusion platform that adopts can't effectively satisfy in the application needs to image real time fusion display process.Make up a kind ofly can handle in real time, data fusion platform that syncretizing effect is good is extremely urgent.
Summary of the invention
The fusion processing system of multi-sensor images that the technical problem to be solved in the present invention provides and a kind ofly can handle in real time, syncretizing effect is good.
In order to solve the problems of the technologies described above, fusion processing system of multi-sensor images of the present invention comprises and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device; Described image data acquiring device carries out after the pre-service pattern buffer memory with whole page or leaf burst to the view data of sensor output; The multi-characteristic image data processing equipment extracts the view data of image data acquiring device storage, finish many characteristic operation of target, generate image to be matched, calculate target position in image to be matched, and deposit image to be matched and target location in the image data acquiring device; Data fusion device reads the image to be matched and the target location of each road image data acquiring device storage, carries out images match, fusion treatment, generates last fused images.
The present invention utilizes the image data acquiring device to finish image reception, transmission and the preprocessing function of multisensor, utilize the multi-characteristic image data processing equipment to generate image to be matched, and calculating target position in image to be matched, utilize data fusion device to each road image mate, fusion treatment, generate last fused images.The present invention has very high practicality, can effectively satisfy the request for utilization real-time, that syncretizing effect is good, be widely used, and exploitativeness is strong.
Described multi-characteristic image treating apparatus comprises:
The device that is used for reads image data;
The device that is used for read sensor pixel resolution multiplying power, focal length value and sensor visual angle;
The device of the image identical with the image aspects of regulation sensor acquisition is arrived in the image transformation that is used for obtaining in real time;
Be used to generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it device of image data acquiring device in;
Be used for device according to gradation of image information calculations image gradient;
Be used for determining the device of object edge according to image gradient;
Be used for device according to the object edge calculated target positions;
Be used for whether putting effective marker, and the target location deposited in the device of image data acquiring device according to the target location.
Described data fusion device comprises:
Be used to read the device of the image to be matched of each road image data acquiring device storage;
Be used to read the device of target position information;
Be used to judge whether effectively device of target location;
Be used to finish the device of image registration:
If the target location is effective, then store this road image with respect to the target location of specified standard image according to target location in the image to be matched of this road; If the target location is invalid, then setting big or small image with the specified standard picture centre is matching template, finds the image identical with this matching template gray-scale value in this road image, calculates the position of this image with respect to standard picture, and store this road image according to result of calculation, finish image registration;
Be used for the image behind each width of cloth registration is carried out the device that laplacian pyramid decomposes;
Be used for device that each tomographic image after decomposing is merged:
For the top layer exploded view as the peaked pixel of chosen area average gradient as the pixel of fused image on this zone; For other layers exploded view as the peaked pixel of favored area average energy as the pixel of fused image on this zone;
Be used for doing the device that the laplacian pyramid inverse transformation obtains final fused images to merging pyramid.
Description of drawings
Below in conjunction with the drawings and specific embodiments the present invention is described in further detail.
Fig. 1 is a sensor image fusion processing system structured flowchart of the present invention.
Fig. 2 is for realizing hardware circuit synoptic diagram of the present invention.
Fig. 3 is a multi-characteristic image treating apparatus program flow diagram.
Fig. 4 is the data fusion device program flow diagram.
Embodiment:
As shown in Figure 1, fusion processing system of multi-sensor images of the present invention comprises and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device;
As shown in Figure 2, described image data acquiring device adopts programmable logic device (PLD) (FPGA) and storer (SDRAM); The image that sensor is taken in is sent into FPGA, and image deposits SDRAM through pre-service (promptly view data being organized) back in the pattern that whole page or leaf happens suddenly in FPGA, by the FPGA sequential control, makes SDRAM realize the function of multiport image buffer.
Described multi-characteristic image data processing equipment adopts DSPs or other common flush bonding processors to realize.
Described data fusion device adopts microcomputer or other common flush bonding processors to realize.
First port of SDRAM uses for the image input, and the view data of sensor output collects among the SDRAM through FPGA.Second port is for the real-time reading images of DSPs, and the image to be matched and the target location of generation are temporarily stored among the SDRAM by FPGA; DSPs can control FPGA by this port, as control images acquired size, image memory location etc.The 3rd port uses for PCI-bridge (PCI bridge), and computing machine can read the image to be matched of arbitrary frame at any time by pci bus, and the target location that DSPs calculates and the information such as image to be matched of generation are also delivered in the computing machine by this port.
As shown in Figure 3, DSPs is responsible for multi-characteristic image and handles, and multi-characteristic image handling procedure flow process comprises the steps:
Initialization;
Reads image data;
Orientation, the luffing angle of read sensor pixel resolution multiplying power, focal length value and sensor;
Regulation is a standard transducer with one of them sensor, according to the orientation angles of orientation angles, luffing angle and the required standard sensor of the sensor of real-time acquisition, the difference between the luffing angle, with image transformation to and the identical image of image aspects of regulation sensor acquisition;
Generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it in SDRAM assigned address;
According to gradation of image information calculations image gradient to be matched;
Determine object edge according to image gradient;
According to the object edge calculated target positions;
Judge whether the target location is effective,, then deposit the target location in SDRAM if the target location is effective;
Return.
Microcomputer reads image to be matched and target position information by PCI with dma mode, and carries out fusion treatment, and idiographic flow comprises the steps:
Initialization:
Read each road image to be matched of storing among the SDRAM by pci bus;
Read target position information in the image to be matched of each road by pci bus;
Judge whether the target location is effective, if the target location is effective, then calculate target in the image to be matched of this road, and store this road image according to result of calculation with respect to the position of target in the specified standard image; If the target location is invalid, then the image with specified standard picture centre position 96*96 size is a matching template, in this road image, find the image of the 96*96 size identical with this matching template gray-scale value, calculate the position of this image, and store this road image according to result of calculation with respect to standard picture; Finish the image registration before merging;
Image behind each width of cloth registration is carried out laplacian pyramid to be decomposed;
Adopt different fusion criterions to merge to each tomographic image after decomposing, concrete fusion rule is: select regional area for being that the territory is faced at 3 * 3 of center with its each pixel, adopt the peaked pixel of chosen area average gradient as the pixel of fused image on this zone for top layer exploded view picture; Select the pixel of energy maximum region as the pixel of fused image on this zone for the method that other layers exploded view picture adopts provincial characteristics to measure;
The fusion pyramid is done the laplacian pyramid inverse transformation obtain final fused images.
Advantage of the present invention: comprise the information such as half-tone information, target location because the present invention adopts DSPs to calculate respectively in the view data that different sensors collects, give microcomputer with result of calculation and carry out images match, fusion treatment, generate last fused images. Computational speed can satisfy real-time processing requirements, more be conducive to the raising of syncretizing effect.
Claims (3)
1. a fusion processing system of multi-sensor images is characterized in that comprising and corresponding image data acquiring device of number of sensors and multi-characteristic image data processing equipment, and data fusion device; Described image data acquiring device carries out after the pre-service pattern buffer memory with whole page or leaf burst to the view data of sensor output; The multi-characteristic image data processing equipment extracts the view data of image data acquiring device storage, finish many characteristic operation of target, generate image to be matched, calculate target position in image to be matched, and deposit image to be matched and target location in the image data acquiring device; Data fusion device reads the image to be matched and the target location of each road image data acquiring device storage, carries out images match, fusion treatment, generates last fused images.
2. fusion processing system of multi-sensor images according to claim 1 is characterized in that described multi-characteristic image treating apparatus comprises:
The device that is used for reads image data;
The device that is used for read sensor pixel resolution multiplying power, focal length value and sensor visual angle;
The device of the image identical with the image aspects of regulation sensor acquisition is arrived in the image transformation that is used for obtaining in real time;
Be used to generate and the image equal resolution multiplying power of regulation sensor acquisition, the image to be matched of the same focal length value, and deposit it device of image data acquiring device in;
Be used for device according to gradation of image information calculations image gradient;
Be used for determining the device of object edge according to image gradient;
Be used for device according to the object edge calculated target positions;
Be used for whether putting effective marker, and the target location deposited in the device of image data acquiring device according to the target location.
3. fusion processing system of multi-sensor images according to claim 1 and 2 is characterized in that described data fusion device comprises:
Be used to read the device of the image to be matched of each road image data acquiring device storage;
Be used to read the device of target position information;
Be used to judge whether effectively device of target location;
Be used to finish the device of image registration:
If the target location is effective, then store this road image with respect to the target location of specified standard image according to target location in the image to be matched of this road; If the target location is invalid, then setting big or small image with the specified standard picture centre is matching template, finds the image identical with this matching template gray-scale value in this road image, calculates the position of this image with respect to standard picture, and store this road image according to result of calculation, finish image registration;
Be used for the image behind each width of cloth registration is carried out the device that laplacian pyramid decomposes;
Be used for device that each tomographic image after decomposing is merged:
For the top layer exploded view as the peaked pixel of chosen area average gradient as the pixel of fused image on this zone; For other layers exploded view as the peaked pixel of favored area average energy as the pixel of fused image on this zone;
Be used for doing the device that the laplacian pyramid inverse transformation obtains final fused images to merging pyramid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101437417A CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101437417A CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101819677A true CN101819677A (en) | 2010-09-01 |
| CN101819677B CN101819677B (en) | 2012-06-13 |
Family
ID=42654764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101437417A Expired - Fee Related CN101819677B (en) | 2010-04-12 | 2010-04-12 | Fusion processing system of multi-sensor images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101819677B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750247A (en) * | 2012-06-05 | 2012-10-24 | 中国科学院光电技术研究所 | Signal processing platform applicable to multi-sensor self-adaption optical system |
| CN104168455A (en) * | 2014-08-08 | 2014-11-26 | 北京航天控制仪器研究所 | Air-based large-scene photographing system and method |
| CN113611112A (en) * | 2021-07-29 | 2021-11-05 | 中国第一汽车股份有限公司 | Target association method, device, equipment and storage medium |
| CN114915731A (en) * | 2021-02-07 | 2022-08-16 | 华为技术有限公司 | Method and apparatus for processing image data of image sensor |
-
2010
- 2010-04-12 CN CN2010101437417A patent/CN101819677B/en not_active Expired - Fee Related
Non-Patent Citations (5)
| Title |
|---|
| 《中国优秀硕士学位论文全文数据库》 20070915 尹乐 基于FPGA和DSP的图像多功能卡的设计与实现 P16-P18 1-3 , 第3期 2 * |
| 《大连理工大学硕士学位论文》 20080131 陈东 一种结合边缘特征和互信息的图像配准方法 全文 1-3 , 2 * |
| 《武汉理工大学工学硕士学位论文》 20080930 晏植 多源遥感图像融合技术研究 正文第2.2.4节、第2.3.5节、第4.4.1节以及图2-5、图4-3 1-3 , 2 * |
| 《激光与红外》 20090430 陈浩，王延杰 基于拉普拉斯金字塔变换的图像融合算法研究 正文第3节 3 第39卷, 第4期 2 * |
| 《红外与激光工程》 20050831 陶冰洁，王敬儒等 一种实用的小目标配准方法 正文第2节、第3节 1-3 第34卷, 第4期 2 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750247A (en) * | 2012-06-05 | 2012-10-24 | 中国科学院光电技术研究所 | Signal processing platform applicable to multi-sensor self-adaption optical system |
| CN102750247B (en) * | 2012-06-05 | 2015-01-14 | 中国科学院光电技术研究所 | Signal processing platform applicable to multi-sensor self-adaption optical system |
| CN104168455A (en) * | 2014-08-08 | 2014-11-26 | 北京航天控制仪器研究所 | Air-based large-scene photographing system and method |
| CN104168455B (en) * | 2014-08-08 | 2018-03-09 | 北京航天控制仪器研究所 | A kind of space base large scene camera system and method |
| CN114915731A (en) * | 2021-02-07 | 2022-08-16 | 华为技术有限公司 | Method and apparatus for processing image data of image sensor |
| CN113611112A (en) * | 2021-07-29 | 2021-11-05 | 中国第一汽车股份有限公司 | Target association method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101819677B (en) | 2012-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107194962B (en) | Point cloud and plane image fusion method and device | |
| CN104089697B (en) | Real-time online visual vibration measurement method based on thread pool concurrent technology | |
| CN102855649B (en) | Method for splicing high-definition image panorama of high-pressure rod tower on basis of ORB (Object Request Broker) feature point | |
| CN105551064B (en) | A kind of method based on characteristics of image estimation windrow volume change | |
| CN107851196B (en) | Image pattern matching method and device | |
| CN101819677B (en) | Fusion processing system of multi-sensor images | |
| CN107610164A (en) | A kind of No. four Image registration methods of high score based on multiple features mixing | |
| CN104299231B (en) | Method and system for registering images of multiple sensors in real time | |
| CN108230402B (en) | Three-dimensional calibration method based on triangular pyramid model | |
| CN110363211A (en) | Detect network model and object detection method | |
| CN114764778A (en) | Target detection method, target detection model training method and related equipment | |
| CN114694011A (en) | Fog penetrating target detection method and device based on multi-sensor fusion | |
| CN102096913B (en) | Multi-strategy image fusion method under compressed sensing framework | |
| Yuan et al. | Faster light detection algorithm of traffic signs based on yolov5s-a2 | |
| CN103366348A (en) | Processing method and processing device for restraining bone image in X-ray image | |
| CN103260008A (en) | Projection converting method from image position to actual position | |
| Galantucci et al. | Coded targets and hybrid grids for photogrammetric 3D digitisation of human faces | |
| CN110084851A (en) | A kind of binocular point cloud generation method and system | |
| CN102778684A (en) | Embedded monocular passive target tracking positioning system and method based on FPGA (Field Programmable Gate Array) | |
| Wang et al. | An image fusion method based on NSCT and dual-channel PCNN model | |
| Belan et al. | Machine vision system based on the parallel shift technology and multiple image analysis | |
| CN108122220A (en) | Infrared image and LLL image fusion system and method | |
| Sun et al. | Infrared and visible image fusion based on object extraction and adaptive pulse coupled neural network via non-subsampled Shearlet transform | |
| CN115049745B (en) | Calibration method, device, equipment and medium for roadside sensor | |
| Wu et al. | Expansion of the visual angle of a car rear-view image via an image mosaic algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120613 Termination date: 20140412 |