CN101976429A - Cruise image based imaging method of water-surface aerial view - Google Patents

Abstract

The invention relates to a cruise image based imaging method of a water-surface aerial view, which comprises the steps of: firstly, establishing an imaging system which comprises a server image processing system, a client interface display and interactive system, and a video collecting system of a cruise boat; installing a video camera and a GPS (Global Position System) device on the boat; collecting water-surface video data; processing the video data by the server image processing system to obtain a water aerial view; and inputting the water-surface aerial view into the client interface display and interactive system. A water-surface video is collected by the cruise boat, and is converted into the water aerial view. The method has the advantages of small distortion, short period, high aging performance, low investment and good compatibility, and is disturbed by less noise.

Description

Water surface general view formation method based on the image that cruises

Technical field

The invention belongs to technical field of image processing, relate to mentioned image stabilization techniques, camera calibration technology, image repair technology etc., to be converted to the general view of the water surface by the video frame images that the patrol of water surface ships and light boats is taken, show the large tracts of land buoyance, as blue-green algae etc., be specially a kind of water surface general view formation method based on the image that cruises.

Background technology

Since the sixties in 20th century, the economy of Taihu Lake basin develops rapidly, brings immense pressure to environment, has caused the havoc of the ecosystem and the decline of environmental quality.By existing " water environment quality standard GB3838-88 ", the mid-90 in 20th century Taihu Lake water quality on average reached the IV class, 1/3 lake region is the V class, shows as average rank of per 10 years deteriorating water qualities, decline rate was obviously accelerated in nearly more than 10 years.Along with the quickening of eutrophication process, waters blue-green alga blooms such as Taihu Lake Mei Liangwan, silk Shan Hu, western Taihu Lake bank frequently take place.Blue-green algae misgrowth is very easily piled up, rotten sedimentation forms wawter bloom, in the river mouth and offshore alluvial, not only destroy water landscape and ecosystem balance, and because blue-green algae discharges toxin in growth course, consume dissolved oxygen DO, cause the biological mass mortality of water body, the lake water quality deterioration, serious threat the safe drinking water of surrounding area, lake.2007, the blue-green algae incident of Taihu Lake in 2008 outburst is particularly outstanding, its break out the time early, scale intensity is big, consequence serious threat the safe drinking water in cities such as Wuxi, make economy be subjected to tremendous loss.Therefore, control lake water pollution is kept ecological balance, and to guarantee human normal living environment, has become one of modern society's sixty-four dollar question.

Except blue-green algae, also have other status of the water area to need monitoring on a large scale, as the red tide on sea, the water pollution that Oil spills etc. cause etc.

On July 16th, 2010, pipe laying of Dalian Bay explodes suddenly, cause estimation at least up to ten thousand tons of crude oil leakages go into the sea, ecological safety is caused serious harm, caused incalculable damage to sea-farming and tourist industry etc.Large quantities of sea life are because of poisoning or death by suffocation, and toxic compounds such as benzene that crude oil is contained and toluene can enter food chain.In addition, potential infringement further expands in the local ecosystem, and the biology of survival can entail the offspring to the influence of poisonous substance in several years.

On October 11st, 2010, southeaster is brought than landing the Zhangpu, Fujian in all Asias of typhoon, causes red tide on a large scale at salt port, Huidong, and more than 2500 the net cage fish in salt port are dead in 3 days because of the red tide anoxic, cause more than 1,700 ten thousand yuan of direct economic losses.

Water quality monitoring is the main foundation of water quality assessment and water prevention and cure of pollution, day by day serious along with water pollution problem, and water quality monitoring becomes the significant problem that social economy's sustainable development must solve.Four kinds of methods are mainly taked in the landlocked water body water quality monitoring of China at present:

(1) direct Detection Method.Water sample is gathered in the lake that will monitor, carried out water analysis, obtain relevant chemical parameters, represent near waters or adopt its spatial and temporal distributions of statistical methods analyst with the parameter of sampled point in the laboratory.The advantage of this method is to detect multiple water quality parameter, comprise dissolved oxygen DO, water temperature, pH value, conductivity, transparency, permanganate index, ammonia nitrogen, total nitrogen, total phosphorus, chlorophyll A, algae gross density or the like and the closely-related information of water quality, weak point is to large-area waters, adopt that this method is consuming time, expensive, consumption power, be difficult to obtain waters state parameter on a large scale, can not satisfy, on a large scale monitoring and evaluation requirement real-time, continuous water quality.

(2) fixed cameras monitoring.With Taihu Lake is example, and reach an agreement with China Telecom in Wuxi City environment measuring center at present, adopts " global eyes " technology to realize the real-time video monitoring of the Mei Liang lake stretch of coastal water.This system tentatively finishes, but still comes with some shortcomings.

1. expense costliness: both sides adopt the mode of rental service, all build by China Telecom and provide funds, and comprise maintenance in the future, and its service is rented at environmental protection tests center, Wuxi, pays general 500,000 yuan network rent expense every year.

2. image quality is not good: telecommunications " global eyes " adopts 10Mbps optical fiber to insert, and is not very smooth at the picture of inspection center.At present the sharpness that adopts is 4CIF, and frame per second is not less than 20 frames, and camera adopts cradle head control, can long-rangely carry out level, rotate and the remote adjustment lens focus up and down.

3. the monitoring territory is limited: close shot can be seen clearly substantially, but camera lens when zooming out picture waterborne can't distinguish that it is relevant with the camera quality on the one hand, the height that sets up with The Cloud Terrace is relevant on the other hand.

(3) artificial patrol monitors.Artificial patrol monitors by finishing jointly along each department, waters, all departments.The tour point is set in the maritime belt, and wind direction, water colour, contaminated area, the algae in essential record observation station periphery waters assemble situation, determine the observation frequency according to incident generation rank.Manual patrol has strong subjectivity, criterions such as water colour, algae gathering situation is difficult to unified, and parameters such as contaminated area can only estimate and draw that the monitoring frequency need consume a large amount of human and material resources when higher.

(4) remote sensing monitoring method.Landlocked water quality remote sensing monitoring at present commonly used is based on selections remote sensing wave band datas such as experience, statistical study, water quality parameter spectral signature and carries out statistical study, sets up water quality parameter inversion algorithm realization.At the initial stage seventies, the Remote Sensing Study of land water body developed into from simple waters identification water quality parameter is carried out remote sensing monitoring, drawing and prediction.Along with to the substance spectra The Characteristics deeply, the improvement of algorithm and the continuous innovation of remote sensing technology itself, remote sensing monitoring water quality develops into quantitatively from qualitative, and can increase gradually by the water quality parameter kind of remote sensing prediction, the vertical attenuation coefficient and some comprehensive pollution indexs such as the nutritional status index etc. that comprise incident and emergent light in suspended particulate substance, water transparency, chlorophyll-a concentration and dissolved organic matter, the water.

Remote sensing monitoring is divided into two classes again: one is based on high spatial resolution, as TM (Thematic Mapper) data of the HRV data of French SPOT satellite, the U.S. land explorer satellite LANDSAT of system or carry the direct extraction to the lake relevant information such as ASTER data on the Terro satellite; Two are based on the MODIS data such as (Moderate-resolution Imaging Spectroradiometer) of carrying on Terra (EOS-AMI) satellite of AVHRR, U.S. emission in 1999 of resolution between low-to-medium altitude such as weather satellite NOAA extracts target information in conjunction with relevant algorithm.

Remote sensing technology is the method that gets most of the attention in the water quality monitoring on a large scale, but still has following problem:

1. the high-resolution remote sensing image temporal resolution is low and cost an arm and a leg.Repeating covering cycle as ASTER is 4～16 days, and the cycle of operation of TM is 16 days, the track of SPOT be " stationary state " (phased), repeating covering cycle is 26 days.The ASTER image is approximately 800 yuan/scape; 4000 yuan/scape of TM image; SPOT1,2,3,9900 yuan/scape of 4 image datas, the SOPT5 of resolution 10m and 5m then is 14900 a yuan/scape, 2.5m up to 29800 yuan/scape.

2. there is the mixed pixel problem in the remote sensing monitoring data of middle low resolution, and precision is limited.Remote sensing is to be unit with the pixel to the detection of atural object, and pixel has certain area, and seldom is made up of single and uniform face of land covering class, all is the mixture of several atural objects generally.Although different natural feature on a maps has features such as its different spectrum, time, angle, the pixel of remote sensing record has only features such as single spectrum, time, angle, the feature after promptly mixing, thus cause puzzlement to remote Sensing Interpretation.The ground resolution of MODIS instrument is 250m, 500m and 1000m; The substar resolution of AVHRR is 1.1km, because scan angle is big, part distortion in image border is bigger, in fact its most useful part (15 ° are located ground resolution is 1.5km) in ± 15 ° of scopes.The mixed pixel problem is the significant obstacle that remote sensing technology deeply develops to quantification.

3. remote sensing images are subject to cloud layer, atmospheric effect, cause obtaining the ground image or disturb greatlyyer, are difficult to the dynamic change of real-time continuous trackable surface rainy season.Taihu Lake basin is rainy, average annual quantity of precipitation is 1100 milliliters, it belongs to the subtropics monsoon climatic region, and every year, at the end of spring and the beginning of summer current of warm air of going up north and the cold airflow of going down south met in this, both are evenly matched, one month locked time, even five or six ten days, plum rains without stop formed, and also blue-green algae high-incidence season just this moment, remote sensing monitoring is difficult to play effectiveness.What satellite sensor received is the signal of clutter reflections solar radiation, in the interaction of solar radiation and earth atmosphere, can produce effects such as absorption, reflection, scattering and emission, these all can cause the distorted signals that sensor receives, make image quality decrease, water body particularly, its relative land of reflectance spectrum signal a little less than, so atmospheric condition can cause larger interference to monitoring result.

All there is top problem in various waters in the time need the monitoring of whole visual state being arranged to the water surface.

Summary of the invention

The problem to be solved in the present invention is: prior art exists not enough to the monitoring of the water surface, and as not satisfying the requirement of water field of big area monitoring, the monitoring real-time is not enough, monitor with high costs, monitoring accuracy susceptible to etc.; Need a kind of easy easy realization, the high respond well water surface monitoring method of precision.

Technical scheme of the present invention is: based on the water surface general view formation method of the image that cruises, at first set up imaging system, comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises, the ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device, the server images disposal system is handled video data and is obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module, client end interface shows and interactive system comprises that map filling and display module and grid map make up module, may further comprise the steps:

1), video camera and GPS locating device are set on ships and light boats, video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route, the output video frame sequence, the GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one by the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system, the video stabilization module is carried out pre-service by overall motion estimation, motion compensation and image repair technology to sequence of frames of video, because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, finishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module, comprise camera attitudes such as the video camera inside and outside parameter and the angle of pitch, the visual angle of sequence of frames of video is converted to from side-looking overlooks, utilize the image reconstruction technique realization that hole-filling, the image sharpening of overhead view image are handled again, obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template with based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is according to handling the LIP model based on Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation, generation is based on the electronics grid map of Geographic Information System GIS, wherein use scalable vector graphics SVG as format map, gps coordinate is converted to water surface map reference, with the corresponding relation of the water surface map reference map is divided into different grids by the GPS longitude and latitude, general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

Step 2) in, the treatment scheme of video stabilization module is as follows:

21) overall motion estimation: adopt the Block Matching Algorithm and the parameter model estimation technique;

22) image motion compensation: at first moving unintentionally and intentionally of sequence of frames of video carried out parameter estimation, adopt Kalman's state filtering method to have a mind to the kinematic parameter estimation, utilize statistical method structural physical state-space model, the dynamic change of having a mind to and being not intended to kinematic parameter is described, and estimate to have a mind to kinematic parameter based on Kalman filter, compensate by image transformation then and be not intended to motion, the motion that makes image sequence with estimate to have a mind to motion model consistent, thereby reach the purpose of video stabilization, suppose n two field picture f _nTransformation parameter is (T _n, d _n), then transformation model is as follows:

${\stackrel{\‾}{p}}_n={\stackrel{\‾}{T}}_n{p}_n+{\stackrel{\‾}{d}}_n---\left(1\right)$

Wherein, p _nBe the point coordinate vector after the image transformation, state equation and observation equation according to Kalman filtering obtain:

${\stackrel{\‾}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\′}}_n\left(p\right)=f_n({\left({\stackrel{\‾}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\‾}{T}}_n\right)}^{-1}{\stackrel{\‾}{d}}_n\mathrm{})---\left(3\right);$

23) image repair: from view data, obtain iconic model through filtering, parameter or non-parametric estmation and entropy method, employing is based on the digital picture repairing technique of partial differential equation, the marginal information of repairing area is treated in utilization, adopt the direction of estimating isophote to smart method by thick simultaneously, the information around the repairing area for the treatment of is propagated in the repairing area, to realize image repair.

As optimal way, video camera of the present invention is a pinhole camera, on traditional camera marking method basis, inside and outside ginseng is demarcated respectively during the step 3) camera calibration, with the known calibrating block of structure as the space object of reference, utilize the three-dimensional coordinate of one group of non degenerate point on the calibrating block and corresponding image coordinate by a series of mathematic(al) manipulations and computing method, ask for the inner parameter and the external parameter of camera model, confidential reference items are demarcated and are finished under the laboratory experiment environment, adopt traditional method based on calibrating template; Outer ginseng is carried out on-line proving, realizes by the fixed character object of reference that extracts in the scene.

In the step 4), the image transformation module realizes the visual angle conversion of frame of video by the reverse mapping techniques of image, the mapping relations matrix of foundation from the purpose territory to reference field, the purpose territory is a general view, reference field is a video frame images, the information that obtains with camera calibration is rebuild the depth information in purpose territory in advance, according to certain any coordinate (X in the general view of required structure, Y), by the mapping relations matrix computations obtain this point the picture planimetric coordinates (x, y), again with (x, y) locate the pixel value assignment in (X Y), is converted to the visual angle of sequence of frames of video and overlooks from side-looking; Image reconstruction adopts the algorithm based on template.

Further, the treatment scheme of the image mosaic module of step 5) comprises image pre-service, image registration, image co-registration, wherein:

The image pre-service is carried out pre-service to reference picture and image to be spliced, reference picture is got first two field picture of video sequence, pre-service comprises the basic operation of Flame Image Process, set up the matching template of image and image is carried out conversion, wherein with reference picture as matching template, can do some shearings according to actual conditions, image is carried out conversion in order to extract characteristics of image, comprise Fourier transform, wavelet transformation, the Gabor conversion, extract the characteristic set of image afterwards, utilize the rough position relation of feature calculation reference picture and image to be spliced, promptly treat stitching image and carry out coarse localization, find overlapping region roughly, dwindle matching range, raising speed;

Image registration adopts improved Normalized Grey Level level correlation technique NGC to carry out image registration in conjunction with the GPS locator data, improved Normalized Grey Level level correlation technique NGC is: the gray level that the gray level of pixel in the matching template of image be multiply by the respective pixel that is covered by matching template in the image to be spliced, the summation NC value that obtains is stored in the two-dimensional array, the general view that described matching template and image to be spliced have not all transformed through step 4):

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For mobile matching template starting point to image to be spliced (i, j) image is called subgraph by the zone that template covers during the position, the position of matching template in image to be spliced is by the subgraph decision the most similar to matching template, with matching template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S of θ minimum ^{I, j}The position at place is the position of template in image:

$\cos {\mathrm{\θ}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left[T(m,n)\right]}^2}}---\left(11\right);$

Image co-registration adopts log-domain Flame Image Process model LIP, and image transitions to be spliced is handled to the LIP territory, realizes figure image intensifying, gamma correction.

As preferably, in the image amalgamation, being positioned at

The brightness value of the pixel at place is considered as a tri-vector, is referred to as color vectors, uses Expression,

$\stackrel{\mathrm{ur}}{F}\stackrel{r}{\left(v\right)}=\left(\begin{array}{c}{F}_r\stackrel{r}{\left(v\right)}\\ F_g\stackrel{r}{\left(v\right)}\\ F_b\stackrel{r}{\left(v\right)}\end{array}\right)---\left(18\right)$

In calculating, quote LIP territory definition

With

Algorithm, and mapping function ψ are handled three passages of R, G, B of coloured image respectively, on the LIP of gray level image model, set up the logarithm of coloured image and handle the CLIP model, in the CLIP territory, handle Surface Picture, make image more press close to true visual signature.

As preferably, in the step 6) map of the water surface during by mesh segmentation, is divided into hexagonal mesh map of uniform size.

The present invention as system front end, gathers Surface Picture with the boat-carrying video camera, and with the video sequence input system, system at first utilizes the video stabilization technology, because the float that causes of jolting of ship is removed, obtains stable video sequence image; Then by the image transformation module, utilize camera attitudes such as the automatic calibrating camera inside and outside parameter of camera calibration technology and the mensuration angle of pitch, utilize the image transformation technology with the visual angle from test conversion for overlooking, utilize technical finesse vertical views such as hole-filling, image sharpening again, obtain the clear general view behind the view transformation; Next be the image mosaic module, utilize the image mosaic technology video image that one width of cloth width of cloth is isolated to piece together, obtain a big figure, splicing is according to being treated to the master with Image Feature Point Matching and CLIP, and the GPS geographic coordinate is auxilliary; The big figure of the water surface that splicing is finished issues on WEB with the form of grid map.

Can't obtain the ground image down or disturb bigger deficiency at the overcast and rainy cloudy weather of satellite remote sensing, the present invention proposes to cruise and the status monitoring new method of waters on a large scale of video camera based on ships and light boats, has complementary advantages with existing monitoring method.The high-resolution remote sensing image price is extremely expensive, and the boat-carrying video camera cost that the present invention is based on existing patrol ships and light boats is moderate, and can gather equal resolution even high-definition picture more, and its closely imaging disturbed by atmosphere, weather less, the Taihu Lake blue-green algae distribution plan system that makes up based on the image that cruises has broad application prospects.

The present invention makes up Video processing, analysis, integrated, delivery system, has patrol videograph, location, map reproduction and human-computer interaction function.The system integration GPS geo-location data and the view data of cruising provide map to show, can be used for instructing ships and light boats patrol direction, and auxiliary its exhaustively travelled the territory, lake, in order to avoid blindly cruise.

The present invention proposes the new method that makes up territory, lake general view, alleviate dependence satellite remote sensing system.On the camera calibration technical foundation, realize image mapped and reconstruction, skeleton view is converted into general view,, and eliminate between the piece image luminance difference that causes because of uneven illumination etc., form territory, whole lake general view based on the image mosaic technology.

The gridding map mode that proposes makes up and has high-resolution grid image piece, its independent processing, dynamically updates, and has and handles advantages such as convenient, real-time.Based on GIS map form and SVG technique construction Taihu Lake grid image, can represent the distributed intelligence of blue-green algae universe in real time, can extract independent gridblock again, represent the high resolving power detailed information.

The Flame Image Process mode based on log-domain LIP and CLIP that proposes solves problems such as overflowing in the image calculation, distortion, makes it approach the human visual system more.Common addition and the scalar multiplication computing used always in the Flame Image Process, inconsistent with image formation rule, its image to forming by transmitted light, to human visual perception and a lot of real figure image all is unaccommodated, usually cause calculating and overflow, the log-domain image processing techniques preferably resolves this problem, and its sealing operational pattern can not cause calculating to be overflowed.

The present invention has the following advantages:

1. distortion is little, and noise is few.Be different from remotely sensed image, very short based on light radiation path in the video camera imaging of patrol ships and light boats, influence quite little, not influenced by cloud layer, there is not the mixed pixel problem yet, need not to carry out complicated pre-service such as atmosphere compensation, even overcast and rainy cloudy weather also can be caught real-time lake condition image.

2. the cycle is lacked ageing height.According to investigations, be to hold in real time water surface situation, routine monitoring needs report Monitoring Data 1 time every day, breaks out the most serious period as blue-green algae, and emergency monitoring project monitoring at least 1 day 2 times, automatic on-line monitoring then need every 2h～4h reported data 1 time.The satellite remote-sensing image excessive cycle can not satisfy the real-time demand.The present invention can issue by handling in several seconds by the lake area image that the patrol ships and light boats are gathered, and is ageing very high.

3. drop into low, compatible good.Utilize existing patrol ships and light boats, the configuration video camera can be realized the waters condition monitoring, with respect to the satellite remote-sensing image that needs are given a long price for, and the monitoring means of its Cheap highly effective of can yet be regarded as.

Description of drawings

Fig. 1 is the structural drawing of imaging system of the present invention.

Fig. 2 is the process flow diagram of the inventive method.

Fig. 3 is the processing flow chart of video stabilization module.

Fig. 4 is the desirable little pore model of video camera.

Fig. 5 demarcates the perspective projection model based on the plane template demarcation that adopts for camera calibration module confidential reference items of the present invention.

Fig. 6 mixes the video camera town and country model of demarcation outward for camera calibration module of the present invention.

Fig. 7 is the forward mapping synoptic diagram in the image mapped.

Fig. 8 is the reverse mapping synoptic diagram in the image mapped.

The reverse mapping synoptic diagram that Fig. 9 adopts for the present invention.

Figure 10 is an image reconstructing method synoptic diagram of the present invention.

Figure 11 is an image mosaic process flow diagram of the present invention.

Figure 12 is the linear excessively synoptic diagram of method in overlay region in the image mosaic.

Figure 13 is the flow path switch synoptic diagram of gps coordinate of the present invention and electronic chart coordinate.

The sexangle grid synoptic diagram that Figure 14 adopts for the present invention.

Figure 15 is a WebGIS platform structure synoptic diagram.

Figure 16 for the present invention in the electronics grid map being carried out WEB issue, the application function synoptic diagram of Ajax.

Figure 17 for the present invention in the electronics grid map being carried out WEB issue, the synoptic diagram of Ajax asynchronous communication process.

Embodiment

The present invention is based on the boat-carrying video camera and make up surface, waters general view, be different from land fixed cameras supervisory system, its image cruise characteristic and water surface image be different from terrestrial landscape characteristics determined native system need adopt the particular processing mode.Fig. 1 has shown imaging system framework of the present invention, comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises, the ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device, the server images disposal system is handled video data and is obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module, and client end interface shows and interactive system comprises that map filling and display module and grid map make up module.As Fig. 2, the present invention includes following steps:

1), video camera and GPS locating device are set on ships and light boats, video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route, the output video frame sequence, the GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one by the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system, the video stabilization module is carried out pre-service by overall motion estimation, motion compensation and image repair technology to sequence of frames of video, because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, finishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module, comprise camera attitudes such as the video camera inside and outside parameter and the angle of pitch, the visual angle of sequence of frames of video is converted to from side-looking overlooks, utilize the image reconstruction technique realization that hole-filling, the image sharpening of overhead view image are handled again, obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template with based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is according to handling the LIP model based on Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation, generation is based on the electronics grid map of Geographic Information System GIS, wherein use scalable vector graphics SVG as format map, gps coordinate is converted to water surface map reference, with the corresponding relation of the water surface map reference map is divided into different grids by the GPS longitude and latitude, general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

Blue-green algae distribution monitoring with Tai Lake specifies enforcement of the present invention below.

1.1.1. video stabilization technology

The video stabilization technology has three main steps: image overall estimation, image motion compensation, video reparation, algorithm flow chart as shown in Figure 3:

1) image overall is estimated

Because the motion of the relative background of video camera is a kind of global motion, promptly its image change of causing is a kind of global change, so can compensate with GLOBAL MOTION ESTIMATION TECHNOLOGY in the video stabilization algorithm.Mainly being divided into of the estimation of global motion based on Block Matching Algorithm with based on the parameter model estimation technique.

The basic thought of block matching method is that present frame is divided into square, to each piece of present frame certain zone in reference frame all, be in the search window, the piece that has minimum match error with it according to the search of certain matching criterior, this piece is the match block of current block, and the coordinate displacement between match block and the current block is exactly a motion vector.Complexity based on the estimation of block matching method depends primarily on the calculated amount of matching criterior and these two aspects of searching algorithm of employing.This method is calculated simple, but algorithm can not be estimated the motion that the rotation, zoom of video camera etc. cause usually well.

The parameter model algorithm for estimating is meant by setting up different parameter models can describe different forms of motion, parameter model is intended to describe quadrature or the perspective projection of three-dimensional motion at the plane of delineation, model uses the wherein motion of each pixel of a spot of parametric description, compare with block matching method, parameter model can be described motion more compactly, and is not easy to be subjected to The noise.At present main use six parametric methods based on parallel projection are arranged, eight parametric methods based on perspective projection are also arranged, this class algorithm can be described motions such as the rotation of video camera and zoom.

2) image motion compensation

The purpose of video stabilization be to alleviate or eliminate the video camera carrier rock the image sequence interframe that caused unintentionally, harmful motion, the main motion that exists in the reservation queue image or have a mind to motion.Motion compensation comprises two steps, at first the actual motion of image sequence is carried out parameter estimation with having a mind to move, and compensates by suitable image transformation then and is not intended to motion, thereby reach the purpose of video stabilization.

Traditional kinematic parameter algorithm for estimating of having a mind to has curve fitting method and moving average filter method, and the curve fitting fado adopts lower-order model to carry out least square method the actual parameter that overall motion estimation obtains is carried out curve fitting, thereby estimates kinematic parameter intentionally.Curve movement is to have a mind to the stack of motion and randomized jitter, and having a mind to motion, often to represent low frequency component, randomized jitter be high fdrequency component, and the moving average filter method can be used for filtering and represent the high fdrequency component of randomized jitter.But all there is bigger defective in above-mentioned two kinds of methods, and classic method can be used for the present invention, and just calculated amount is huge, are difficult to guarantee its real-time with the arithmetic speed of current hardware.And the real-time that Target Tracking System is handled data is had relatively high expectations, and should finish the processing to former frame before the next frame image arrives.

Kalman's state filtering method can be used to have a mind to kinematic parameter and estimates that its main thought is to utilize statistical method structural physical state-space model, describes the dynamic change of having a mind to and being not intended to kinematic parameter, and estimates to have a mind to kinematic parameter based on the Kalman wave filter.The dynamic motion model that this algorithm is constructed can be described practically owing to the interframe movement that camera motion caused, and estimates to obtain the best kinematic parameter of having a mind to by recurrence.In addition, this method applying flexible can utilize various obtainable prioris by suitable correction state-space model, as having a mind to motion and be not intended to the characteristics etc. of motion

Traditional curve fitting method and moving average filter method have all been used the kinematic parameter of subsequent image frames, obviously can't satisfy higher real-time requirement.In addition, curve fitting method is difficult to adapt to comparatively complicated forms of motion, and there is the problem of " steady excessively " and " owing steady " in the moving average filter rule.And Kalman's state filtering method is estimated the current kinematic parameter of having a mind to according to up-to-date real image kinematic parameter of past, and calculated amount is less, thereby has overcome the defective that the conventional motion filtering algorithm is difficult to satisfy the high real-time requirement.

The purpose of image transformation is that compensation is not intended to motion, the motion that makes image sequence with estimate motion model is consistent intentionally.Suppose that n two field picture transformation parameter is (T _n, d _n), then transformation model is as follows:

${\stackrel{\‾}{p}}_n={\stackrel{\‾}{T}}_n{p}_n+{\stackrel{\‾}{d}}_n---\left(1\right)$

Wherein, p _nBe the point coordinate vector after the image transformation, state equation and observation equation according to Kalman filtering can obtain:

${\stackrel{\‾}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\′}}_n\left(p\right)=f_n({\left({\stackrel{\‾}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\‾}{T}}_n\right)}^{-1}{\stackrel{\‾}{d}}_n\mathrm{})---\left(3\right)$

3) image repair technology

Have two big class image repair technology at present, two kinds of image mending methods can be applied to the present invention: a class is to be used to repair the damaged digital picture of small scale to repair (inpainting) technology.The marginal information of repairing area is treated in its utilization, adopt simultaneously a kind ofly to estimate the direction of isophote (isophote) by thick to smart method, and the employing mechanism of transmission propagates into information in the zone to be repaired, so that obtain repair efficiency preferably.In essence, it is a kind of based on partial differential equation (Partialdifferential Equation, PDE) inpainting algorithm, the main thought of these class methods are to utilize the thermal diffusion equation in the physics will treat that the information around the repairing area propagates in the repairing area.Another kind of is image completion (completion) technology that is used for blank map picture bulk drop-out.At present, this class technology also comprises following two kinds of methods: a kind of recovery technique that is based on picture breakdown, its main thought is to be structure division and texture part with picture breakdown, and wherein structure division is repaired with the inpainting algorithm, and texture part is filled with texture synthesis method.

The present invention mainly handles water surface view, and its texture information is less, and video sequence provides many data sources for image repair, can not produce the situation of bulk information dropout, adopts simply to find the solution partial differential equation and get final product with the method for diffusion neighborhood.

From the mathematics angle, image repair is exactly to treat in the repairing area according to treating that repairing area information on every side is filled into image.Yet, image mending is an ill-conditioning problem normally, because still there is not enough information can guarantee unique part of correctly recovering to be damaged at present, so, people analyze from the angle of psychology of vision, have proposed various hypothesis qualifications and have been used for addressing this problem.As seen, image mending belongs to the research field of image restoration.

Usually image often is subjected to the influence of some factors in acquisition process, makes deteriroation of image quality.In the image restoration field, degradation model commonly used is

I ⁰＝I+N (4)

Wherein, I ⁰Be the observation image that is obtained, I is original image (I={I (x) }), and N is an additive white noise.Concerning most image mending problem, data model has following form:

I ⁰| _Ω/D＝[I+N] _Ω/D (5)

Wherein, Ω represents the entire image zone, and D represents the repairing area for the treatment of of information dropout, Ω D represent not have the zone of drop-out, I ⁰For Ω available image section on the D, the target image of I for needing to restore.Suppose N for Gauss, the energy function E about data model can use the least mean-square error definition always so:

$E\left[I^0\right|I]=\frac{\mathrm{\λ}}{2}\underset{\mathrm{\Ω}/D}{\∫}{(I-I^0)}^2\mathrm{dx}---\left(6\right)$

Because any available data of repairing area D, therefore image (priori) model is concerning the image mending algorithm, than other traditional recovery problems (as denoising, go to fall clear) become even more important, and iconic model can obtain through filtering, parameter or non-parametric estmation and entropy method from view data.

1.1.2. camera calibration technology

Pin-hole model is to be suitable for the simplest being similar to that a lot of computer visions are used, and pinhole camera is finished perspective projection.Fig. 4 has drawn the pin-hole model of video camera, and it has defined following four coordinate systems ^[2]:

European world coordinate system (being designated as ω down): initial point is at O _w

European camera coordinate system (being designated as c down): initial point is at camera lens focus C=O _c, coordinate axis Z _cWith optical axis coincidence and point to outside the plane of delineation.

European plane of delineation coordinate system (being designated as i down): with video camera principal point (intersection point of the optical axis and the plane of delineation) is initial point, X _iAnd Y _iBe positioned on the plane of delineation, parallel with the X-axis of camera coordinate system respectively with Y-axis.

Pattern matrix coordinate system (being designated as f down): the image pixel coordinate during computer frame is deposited.

Based on above-mentioned four coordinate systems, pin-hole model is finished the linear transformation of 3D world coordinates to the 2D pixel coordinate:

$\left(\begin{array}{c}U\\ V\\ W\end{array}\right)=K(R,T)\left(\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right)---\left(7\right)$

$X_f=\frac{U}{W},Y_f=\frac{V}{W}---\left(8\right)$

Wherein K is video camera confidential reference items matrixes, and (R is outer ginseng matrix T), and wherein R is the rotation matrix of 3*3, and T is a translation vector.Linear camera calibration problem is promptly found the solution inside and outside ginseng matrix.In the high accuracy vision detection system, also need to consider the camera lens distortion.

The most general division is to the camera calibration algorithm: traditional camera calibration algorithm, camera self-calibration method and based on the scaling method of active vision.This several method can both be used for camera calibration of the present invention, and active vision is demarcated owing to more to limit movement, and therefore boat-carrying camera motion complexity needs to draw its advantage, adopts suitable calibration algorithm at actual conditions.

Traditional camera marking method is at specific experiment condition, with the known calibrating block of structure as the space object of reference, utilize the three-dimensional coordinate of one group of non degenerate point on the calibrating block and corresponding image coordinate by a series of mathematic(al) manipulations and computing method, ask for the inner parameter and the external parameter of camera model.The tradition scaling method has had ripe method and theory.

Self-calibration technology does not need known object of reference, only utilizes the image of video camera surrounding environment in motion process and the corresponding relation between the image that video camera is demarcated.Its advantage is not rely on caliberating device and online carrying out.Desirable self-calibration technology not by any with reference to calibrating block, a series of scenes of only rely on taking demarcate according to the corresponding relation of each feature in the scene, but this require high image recognition and characteristic matching technology.Actual camera self-calibration has been used specific object of reference more or less, and as circle, straight line, angle point etc., some that perhaps just directly utilize the objective world have the object of certain regular geometric shapes, as door, window, box, chest etc.

Self-calibration technology based on active vision does not need known object of reference, only obtains specific constraint condition by the motion of control video camera and determines inner parameter, and calibration process is greatly simplified.But this method is more to the camera motion restriction for fear of finding the solution nonlinear equation.

World coordinate system constantly changes in the process in view of ships and light boats are travelled, and its confidential reference items are constant when adopting fixed cameras, the method that the present invention adopts inside and outside ginseng to demarcate respectively, and confidential reference items are demarcated and are finished under the laboratory experiment environment, adopt traditional method, to guarantee precision based on calibrating template; Outer ginseng is carried out on-line proving, by extracting correlated characteristic such as realizations such as position, local horizon, fore unique point in the scene.

● based on the confidential reference items scaling method of plane template

The plane template method of Zhang only needs a printing to have the template (as shown in Figure 5) of grid pattern to rotate before video camera (to contain three times) more than three times to finish demarcation, need not to understand the kinematic parameter of template, concise flow, and precision is higher, can be rated as representative.Under the plane template condition, can think Z _w=0, formula (7) can be rewritten as

$\left(\begin{array}{c}U\\ V\\ W\end{array}\right)=H\left(\begin{array}{c}{X}_{\mathrm{<figure-callout\; id="1"\; label="w\; Y\; w"\; filenames="HDA0000029614080000051.png"\; state="\{\{state\}\}">w</figure-callout>}}& \\ Y_w{}_{}\\ 1\end{array}\right),$ Promptly $s\begin{array}{c}\left(\begin{array}{c}\mathrm{<figure-callout\; id="1"\; label="u\; v"\; filenames="HDA0000029614080000051.png"\; state="\{\{state\}\}">u</figure-callout>}& \\ v\\ 1\end{array}\right)\end{array}=H\left(\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right)---\left(9\right)$

S is a scale-up factor, and (u v) is a pixel coordinate, and H is the Homography matrix of stencil plane to video camera, and it has set up the perspective relation of the stencil plane and the plane of delineation.To calibration point, adopt maximum-likelihood criterion can find the solution the Homography matrix by known N.

● online outer ginseng scaling method

The patrol ships and light boats travel with 20km/h speed, it constantly changes with respect to the Taihu Lake coordinate, can realize the ships and light boats location by the GPS positioning system, and the video camera that is set up on the ship is static with respect to ship itself, therefore, the former point selection hull point of world coordinate system gets final product in the demarcation, and local water surface can be approximately the plane simultaneously, and calibration process can be reduced to finds the solution two interplanar mapping relations.Fig. 6 has shown abstract boat-carrying camera model:

Supposing that camera lens swing angle with respect to the horizontal plane is 0, is 0 with respect to the drift angle of foot direction, if not zero can make so by adjustment, so a demand separates inclination angle t, when the local horizon is visible in the image, but base area horizontal line ordinate value is found the solution t; If the local horizon is invisible, and fore as seen, and the demarcation thing can on the bow be set, and directly finds the solution imaging array.

1.1.3. image transformation technology

● image mapped (image warping) algorithm

If spatial point X projection is in reference view C _rWith purpose viewpoint C _d, obtain reference pixel X respectively _rWith purpose pixel X _dIn the forward mapping process, mapping is from reference field R _rTo purpose territory R _dCarry out,, set up from X according to Given information _rTo X _dMapping relations: X _d=f _Forward(X _r), as shown in Figure 7, otherwise reverse mapping is promptly set up from X _dTo X _rMapping relations: X _r=f _Backward(X _d), as shown in Figure 8.

Because the degree of depth is known in the reference field, the forward mapping has higher execution efficient, but the following problem of ubiquity: (1) mapping out-of-bounds problem, because the pixel in the reference field is not all to be mapped to the purpose territory, can calculate a large amount of invalid out-of-bounds pixels; (2) empty problem numerous tiny cavities can occur because the information that the image expansion forms is not enough; (3) block the problem of penetrating more, can't avoid redundant computation for the pixel that is blocked, and the double counting of a plurality of reference pixel when being mapped to a purpose pixel.

Reverse mapping is adopted and is drawn the principle that drives, oppositely set up mapping from the purpose territory, the problems in the forward mapping have been solved effectively, but because degree of depth the unknown in purpose territory, it is the process of the pixel search of O (width/2) that reverse mapping usually needs an average complexity, thereby causes the bottleneck of computing velocity.Can adopt two kinds of methods to the optimization of reverse mapping: (1) is according to polar curve geometric relationship optimization searching process; (2) utilize the three-dimensional information of scene to rebuild the depth information that obtains the purpose territory in advance, thereby reverse mapping is converted into a forward mapping problems.

Because the present invention has adopted the camera calibration technology, the depth information in purpose territory can be rebuild in advance, therefore simplified the image mapped process, flow process is as shown in Figure 9: (X Y), calculates the picture planimetric coordinates (x of this point by imaging array inverse mapping relation according to this coordinate in the general view of required structure, y), with (x y) locates the pixel value assignment in (X Y) gets final product.May run in the actual mechanical process round, problem such as out-of-bounds, need to adopt suitable strategy to solve.

● image reconstruction algorithm

Image reconstruction of the present invention can adopt based on template (splat) with based on the algorithm of grid (mesh).As shown in figure 10:

(1) based on the algorithm of template.Such algorithm uses the earliest in volume data is drawn, and main thought is parameters such as the degree of depth, direction according to reference pixel, adopts the kernel of a variable size to be reconstructed to reference pixel.Pixel can be a zone by Gaussian kernel or other kernel extensions like this, again superposed part is merged drafting, and wherein the calculating of kernel size can be estimated at the bounding box of single pixel or scenario node.Because the diversity of reconstruct kernel, algorithm based on template can obtain to draw preferably effect, but also come with some shortcomings: at first do not distinguish space structure information, the problem of losing shape appears easily, and, the calculating of template relies on CPU to carry out fully, makes that calculated amount is bigger under high resolving power or complex core situation; In addition, the size of reconstruct kernel be difficult for to be handled, and crosses conference and reduces display effect and increase the calculated amount that the edge merges, and too smallly then can stay too much thin hole in edge.

(2) based on the algorithm of grid.This class algorithm is a continuous grid surface according to the space continuity of hypothesis with discrete pixel establishment, with the summit of pixel as grid, fills grid surface with the result of interpolation calculation.Such algorithm can be finished by hardware, but can't differentiate the topological relation in space effectively.When neighbor is not positioned at the same space continuous surface, will produces non-existent spatial surface, thereby between the prospect of scene and background, often cause " erasing rubber phenomenon " (Rubber Sheet).

The present invention mainly is reconstructed Surface Picture, the water surface can be approximately the plane, three-D space structure information is less, adopt CUDA (compute unified device architecture in addition, statistical computation equipment framework) etc. the high speed calculate platform can solve the big problem of calculated amount, and therefore adopting the algorithm based on template is better selection.

1.1.4. image mosaic technology

The image mosaic technology mainly is divided into three key steps: image pre-service, image registration, image co-registration, as shown in figure 11.Wherein the pretreated purpose of image is the precision that guarantees next step image registration, original image is done some folding variations and coordinate transform, comprise the basic operation of Flame Image Process, as histogram handle, the smothing filtering of image etc., set up the matching template of image, image carried out conversion, as Fourier transform, wavelet transformation, Gabor conversion etc. and the operations such as characteristic set of extracting image, coarse localization, find overlapping region roughly, dwindle matching range, raising speed.

● image registration algorithm

Up to now, produced many method for registering images both at home and abroad, the whole bag of tricks all is the application towards certain limit, also has characteristics separately.According to the information of the image that image registration utilized, image registration can be divided into four classes: based on the image registration of half-tone information, based on the image registration of transform domain, based on the image registration of feature with based on the image registration of model.These four class methods may be used to figure of the present invention and are complementary, and are best based on the effect of gray scale, but computation complexity is also the highest, if want efficient and quality to take into account, often several method merges use.

Directly utilize the half-tone information of image to carry out registration based on the method for gradation of image, by pixel the global optimization of certain similarity measurement (as mutual information, Minimum Mean Square Error etc.) is therebetween realized registration, this method does not need image is cut apart and feature extraction, so precision height, robustness are good.But this method for registering is very responsive to grey scale change, does not make full use of the gray-scale statistical characteristic, and is bigger to the half-tone information dependence of every bit.

A classical way based on the image registration of transform domain is a phase correlation method, promptly utilizes the method for Fourier transform, with image by space field transformation to frequency field, realize the registration of image according to the translation principle of Fourier transform.Because distortion such as translation, rotation, convergent-divergent all have correspondent transform at frequency domain, therefore can utilize fourier method in frequency domain, to carry out images match fully.The method of transform domain has the insensitivity to noise, and the counting yield height has ripe fast algorithm (fft algorithm) and is easy to characteristics such as hardware realization.In general, adopt the method for transform domain to provide a good initial registration parameter for image mosaic.

Method based on characteristics of image at first will be handled two width of cloth images subject to registration, extracts and satisfies the feature set that application-specific requires, and these features as control structure, are sought the mapping relations of two width of cloth image holding bodies.Based on the method for characteristics of image, the quantity of the unique point that obtains after feature extraction will significantly reduce, and therefore can improve the speed of registration, but the effect of its registration also depends on the extraction precision of unique point and the accuracy of Feature Points Matching to a great extent.

Method for registering based on model is the registration that comes image is carried out gamma correction according to the mathematical model of image fault.Typical algorithm is the conversion optimization that Szeliski proposes, and at first sets up the transformation model between the image sequence, obtains running parameter in the model by the optimized Algorithm iteration then, thereby realizes treating the registration of stitching image.The conversion optimization can be handled the splicing of geometric transformations such as having translation, rotation, convergent-divergent between the image sequence, without any need for unique point, and fast convergence rate, and on statistics, be optimum.But make whole algorithm reach the convergent requirement, the initial estimate of certain precision must be arranged, the promptly just artificial initial corresponding point of determining want enough accurate, otherwise will cause the failure of image registration.

Consider that Surface Picture feature angle point is less, and blue-green algae often is the changeable color of enriching such as red, yellow, green, brown, black because of the old children of individuality, physiological situation and environment light quality are different, directly cause Surface Picture color, luminance difference, the red tide on sea, water surface situations such as Oil spills also have obvious color difference, so the present invention adopts improved Normalized Grey Level level correlation technique (NGC) to carry out image registration in conjunction with the GPS locator data.

Original NGC method multiply by the gray level of pixel in the matching template gray level of the respective pixel that is covered by template in the image to be spliced, the summation that obtains (NC value) is stored in the two-dimensional array, as the formula (10), the maximum positions of elements of NC value is exactly the position of template in image in this array.

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For the movable platen starting point to image to be spliced (i, j) image is called subgraph by the zone that template covers during the position, i, j are the coordinate of subgraph in image.The position of matching template in image to be spliced is by the subgraph decision the most similar to template, with template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S of θ minimum ^{I, j}The position at place is the position of template in image.

In view of the color change feature of blue-green algae, consider not adopt merely the brightness parameter among the present invention, adopt the matching result possibility of parameters such as tone H, saturation degree S better.

Because dot product itself can not locate exactly, because the size in the NC value also is subjected to the influence of pixel grayscale size in the two-dimensional array, if the very bright pixel in a certain zone has bigger gray level in the image, its NC value also just becomes big naturally so.This algorithm is concerned about more is angle between vector, and changes location influence with this removal of images brightness.

According to above analysis, can obtain following computing formula:

$\cos {\mathrm{\&theta;}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[T(m,n)\right]}^2}}---\left(11\right)$

● image amalgamation algorithm

The piece of image will be fairly obvious when two width of cloth images to be spliced there are differences, as: even two doubling of the image districts that cause of uneven illumination have bigger brightness inconsistent during owing to shooting, perhaps because image geometry deformation that lens distortion causes or the like.For the sudden change of eliminating light intensity and make the light intensity smooth excessiveness, method commonly used has three kinds:

(1) method of weighted mean

Szeliski uses one " cap shape function " to come weighted mean on the respective pixel of each overlapping frame, and this function is minimum at the place, image border, and in the center contribution at most.Weight distribution to each image sampling of synthetic result defines like this: the pixel near more from picture centre is big more to final synthetic result's contribution.The weight distribution function shows as triangle.Picture one cap is so claim cap shape function (hat function).

(2) based on effective weight of Euclidean distance

Each pixel of image all assigns weight, and this weight is proportional with the distance to edge (or nearest invisible point).When splicing, fundamental purpose is to reduce the light intensity contribution of the near pixel of isolated edge.(x y), utilizes piece distance and Euclidean distance, calculates the nearest brocken spectrum or the distance on limit to calculate distance map d in the blending algorithm.Merge all deformation pattern formula:

$C(x,y)=\frac{\underset{k}{\mathrm{\&Sigma;}}w(x,y)I_k^{\&prime;}(x,y)}{\underset{k}{\mathrm{\&Sigma;}}w\left(d(x,y)\right)}---\left(12\right)$

W is a monotonic quantity, w (x)=x, I _kBe the light intensity function of k amplitude variation shape image, the calculating of d is very simple, gets from rectangle four edges minimum value and value.

(3) multiresolution batten technology

Bert.PJ adopts Laplace multiresolution pyramid structure, picture breakdown is become set of diagrams picture on the different frequency territory, on the frequency field of each decomposition, with doubling of the image boundary vicinity weighted mean.At last the composograph on all frequencies is aggregated into piece image.The batten methods of differentiating are to handle the boundary vicinity zone on all frequency fields more.Therefore workload is big, but the quality height.

In order to eliminate the piece problem of overlay region, effect is preferably the linear excessive method in overlay region at present, as shown in figure 12.The concrete grammar of realizing is that hypothesis overlapping region width is L, and getting the transition factor is σ (0≤σ≤1), and the x in two width of cloth doubling of the image districts and the minimum and maximum value of y axle are respectively X _Min, X _MaxAnd Y _Min, Y _Max, the then transition factor:

$\mathrm{\&sigma;}=\frac{X_{\max }-x}{X_{\max }-X_{\min }}---\left(13\right)$

The image intensity of overlapping region is:

I＝σI _A(x，y)+(1-σ)I _B(x，y) (14)

I _A, I _BBe respectively the corresponding pixel value of figure A and B, this method makes transition portion smoother, does not have tangible step.

● log-domain Flame Image Process (LIP) technology

The present invention introduces a kind of new Flame Image Process framework---log-domain Flame Image Process model LIP.With respect to the digital image processing techniques of general domain, LIP area image disposal route has its unique advantage:

(1) the LIP model is a complete mathematical theory, and in bounded gray scale treatment of picture, the LIP model has been stipulated a special computing collection.Common addition that uses in present most of Flame Image Process and scalar multiplication computing, inconsistent with image formation rule, its image to being formed by transmitted light all is unaccommodated to human vision sensation and a lot of actual digital pictures.

(2) LIP area image processing operational criterion is sealed.In to Digital Image Processing, dispersing and may drawing a value that exceeds the gradation of image scope of two grey scale pixel values promptly overflowed, and the sealing computing of LIP can address this problem preferably.

(3) introduce spatial mappings function ψ among the LIP, finish gray-scale value and in conversion to the number space, can be with complexity Computing is converted to directly adding and subtracting mutually of mapping function with ⊙, has significantly reduced operand.

(4) to be that the brightness size is inversely proportional to the characteristics of saturation degree consistent for the rule of the verified and multiple in practice transmission imaging model of LIP logarithmic image disposal route, multipath reflection imaging model and human brightness sensation, it is consistent with the contrast notion on the psychology, approach the vision system of human eye more, more approaching with the true environment of human eye impression.

In handled in the LIP territory, the gray scale function definition of image claimed that D is that the space is supported on a non-NULL spatial domain D.D is the real zone of a bounded [0, M], and general M gets 256.LIP fundamental operation criterion is defined as follows:

$\&ForAll;f,g\&Element;S$ $f\&CirclePlus;g=f+g-\frac{\mathrm{fg}}{M}$

$\&ForAll;f\&Element;S,\&ForAll;\mathrm{\&lambda;}\&Element;R$ $\mathrm{\&lambda;}\&CircleTimes;f=M-M{(1-\frac{f}{M})}^{\mathrm{\&lambda;}}$

Subtraction has defined the difference between two gray scale function f and the g:

Has special computing

With Be defined among the vector space E, space E is being shone upon ψ:

$\mathrm{\&Psi;}\left(f\right)=-M\ln (1-\frac{f}{M})---\left(17\right)$

Space E the mapping ψ under with real number space R be algebraically isomorphic.Can be with complexity

With

Computing is converted to directly adding and subtracting mutually of mapping function, has significantly reduced operand.The LIP technology can be used for image background renewal, image reconstruction, figure image intensifying, picture contrast estimation, rim detection and image segmentation etc.The present invention adopts LIP technology solution figure image intensifying, gamma correction problem, and image transitions to be spliced to LIP territory is handled, and can better solve because the big problem of luminance difference between image to be spliced under the situations such as solar irradiation inequality, imaging backlight.The application in the present invention of log-domain Flame Image Process has very, and obvious effects promotes.

● colored logarithmic image is handled (CLIP) technology

In traditional Color Image Processing process, a width of cloth coloured image can be considered a common two dimensional image, and just the brightness of each pixel has three degree of freedom (R, G, B) in the image.In the colored log-domain treatment of picture of CLIP, being positioned at The brightness value of the pixel at place is considered as a tri-vector, is referred to as color vectors, uses

Expression,

$\stackrel{\mathrm{ur}}{F}\stackrel{r}{\left(v\right)}=\left(\begin{array}{c}{F}_r\stackrel{r}{\left(v\right)}\\ F_g\stackrel{r}{\left(v\right)}\\ F_b\stackrel{r}{\left(v\right)}\end{array}\right)---\left(18\right)$

In calculating, quote LIP territory definition

With Algorithm, and mapping function ψ are handled R, G, three passages of B respectively, on the LIP model of gray level image, set up the logarithm of coloured image and handle the CLIP model, handle the blue-green algae image in the CLIP territory, make image more press close to true visual signature.

1.1.5. grid map constructing technology

● the grid image based on the SVG map generates

SVG (Scalable Vector Graphics) is that W3C is organized as the develop rapidly needs that adapt to the Internet application and a two-dimentional scalable vector graphics language description standard of overlapping based on the XML language of formulating.The mark of the employing of traditional HTML static page descriptive language fixes, limited and do not have intension, do not support shortcoming exposed day by day such as vector graphics to come out, more and more do not satisfied the WebGIS demand for development.At present the SWF file layout that proposes of the MacroMiedia company on the network is with its image vector, and file is less and have interactivity and gain great popularity, but it is than SVG, and the some shortcomings part is still arranged.XML is used by growing field as the following consolidation form standard in the world that generally acknowledges.SVG is the nature static that solution WebGIS faces as the appearance of describing the subclass of vector graphics of XML, the data layout diversity, and web content performance that platform is relevant and shortage interactivity, Network Transmission waits problem that a brand-new solution is provided slowly.

SVG compares with HTML, has following advantage:

(1) broken through the constraint of HTML fixation mark set, made the content of file abundanter, more complicated, easier composition complete information systems;

(2) SVG is the vector image form, is highly suitable for transmission and application in the network.Generally speaking, (as GIF, JPEG) littler, speed of download is faster than other network image forms for the SVG image;

(3) constitute vector image by text.Its Textuality makes the SVG file that good professional platform independence be arranged and can edit, revise it easily by DOM (Document Obiect Model).Literal in the very outstanding advantage SVG file of another one also can be arrived as keyword search by network search engines.

(4) has dynamic interaction.The SVG image can be made different responses to user action, for example highlighted, sound effect, special efficacy, animation etc.In addition, because the integrated plug-in unit of browsing the SVG file among the IE6.0 of Microsoft, this makes that browsing of SVG is convenient, easily.

SWF compares with SVG, and its deficiency is presented as:

(1) the non-opening of SWF standard.SWF is the technology of a relative closure, and the scheme that neither one merges fully between other the open standard.Along with the development of XML and other open standards, the incoordination of SWF will become increasingly conspicuous.

(2) the relatively poor editability of SWF.SWF is the output file form of Flash, generates form as final animation, and its production process is enclosed in the SWF file, can't carry out the secondary editor again.And SVG adopts a kind of text formatting, just can open the edlin of going forward side by side with common edit tool.

(3) SWF can't carry out picture search.Because SWF is a non-textual format, text can not be independent of image and exist, and therefore can't set up the picture search function that is similar to SVG.

In view of above characteristic and the advantage of SVG, the present invention selects to use SVG to show the blue-green algae distribution situation image of storage, transmission and demonstration Tai Lake as format map.

● GPS longitude and latitude data are to the conversion of local map reference

The locating information that GPS receives is based on WGS one 84 coordinate systems, and the data that WGS 1 receives can't directly apply on the electronic chart.So, be mapped GPS locating information under WGS one 84 coordinate systems and the coordinate on the electronic chart, could accurately realize mesh segmentation like this to the SVG map.

The locator data that is obtained by the GPS receiver is the three-dimensional information that comprises longitude L, latitude B and elevation H, and (the Xt of the point on the electronic chart that finally will obtain, Yt) be the picture element of two dimension, and consider altitude figures benchmark disunity, therefore can take to give up the shortcut calculation of elevation H.At first with WGS one 84 coordinates (L, B H) project on the Gaussian plane, obtain its Gaussian plane respective coordinates point (x, y), again with plane four parametric methods of simplifying with its rotation translation be scaled to the electronic chart planimetric rectangular coordinates (Xt, Yt).The locator data that receives of GPS receiver just can intuitively show on electronic chart like this.Its flow path switch as shown in figure 13.

WGS one 84 coordinate systems project to the Gaussian plane mathematical model, i.e. Gauss projection is just being calculated formula as shown in Equation 19, and wherein, B is the geodetic latitude of subpoint; L=L-L0, L are the geodetic longitude of subpoint, and L0 is the geodetic longitude of central meridian; N is the radius of curvature in prime vertical of subpoint

T=tanB, η=e*cosB, e are ellipsoid second excentricity.

$\left\{\begin{array}{c}x=X^0+1/2\&times;N\&times;t\&times;{\cos }^{2}B\&times;l^2+1/24\&times;N\&times;t(5-t^2+9{\mathrm{\&eta;}}^2+4{\mathrm{\&eta;}}^4){\cos }^{4}B\&times;l^4\\ +1/720\&times;N\&times;t(61-58t^2+t^4+270{\mathrm{\&eta;}}^2-330{\mathrm{\&eta;}}^2{t}^2){\cos }^{6}B\&times;l^6\\ y=N\&times;\cos B\&times;l+1/6N(1-t^2+{\mathrm{\&eta;}}^2){\cos }^{3}B\&times;l^3\\ +1/120N(5-18t^2+t^4+14{\mathrm{\&eta;}}^2-58{\mathrm{\&eta;}}^2{t}^2){\cos }^{5}B\&times;l^5\end{array}\right.---\left(19\right)$

We only need carry out the coordinate transform on the two dimensional surface, therefore can adopt four more easy Parameters Transformation models.The mathematics implication of its four parameters in Gaussian plane is respectively: Δ X is the translational component of coordinate x; Δ Y is the translational component of coordinate y; M is a scale factor; θ is a rotation amount.

In order to find the solution the parameter of coordinate transform, we need know the coordinate of corresponding point under three above WGS one 84 coordinate system map coordinates systems, establish this group common point P1, P2 ... the coordinate of PN (N 〉=3) under two coordinate systems be respectively (Xg, Yg) _i(Xd, Yd) _i,, relation is as shown in Equation 20 arranged then for a Pi.

$\left[\begin{array}{c}\mathrm{Xd}-\mathrm{Xg}\\ \mathrm{Yd}-\mathrm{Yg}\end{array}\right]={\left[\begin{array}{ccc}1& 0\mathrm{Xg}& \mathrm{Yg}\\ 0& 1\mathrm{Yg}& \mathrm{Xg}\end{array}\right]}_i\left[\begin{array}{c}\mathrm{\&Delta;X}\\ \mathrm{\&Delta;Y}\\ m\\ \mathrm{\&theta;}\end{array}\right]---\left(20\right)$

Take advantage of principle by least square, obtain conversion parameter (Δ X, Δ Y, m, θ).With the point coordinate under the WGS 1 of needs conversions (x, y) for people's formula (21), utilize four parameters can get the planimetric rectangular coordinates of tested point (Xt, Yt).

$\left[\begin{array}{c}\mathrm{Xt}\\ \mathrm{Yt}\end{array}\right]=\left[\begin{array}{c}\mathrm{\&Delta;X}\\ \mathrm{\&Delta;Y}\end{array}\right]+(1+m)\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}0\mathrm{\&theta;}\\ \mathrm{\&theta;}0\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]---\left(21\right)$

Through conversion, the map reference that the coordinate conversion under the WGS 1 can be become use, thus realize the conversion of longitude and latitude to map reference.

● grid image generates

Invention is drawn the Tai Lake map with the SVG format-pattern, with the corresponding relation of map reference map is divided into different grids by longitude and latitude, local blue-green algae real-time distribution situation map is filled in the corresponding map grid, generates the blue-green algae population distribution figure in waters.With respect to rectangular node, hexagonal mesh more approaches circle, meets the video camera imaging visual angle more, helps image block cutting and splicing, therefore, adopts hexagonal mesh to divide the territory, lake as shown in figure 14.

● merge the lightweight WebGIS platform of MapTile technology and vector data technology

In MapTile Technology in Web GIS platform, earlier the grid map of map is processed into image pyramid and carries out stripping and slicing, then picture is carried out quadtree coding, leave on the server, when client is carried out operation such as convergent-divergent, roaming, only need the URL of dynamic calculation map tile to conduct interviews and show and get final product, the volume of these tiles is all very little, when network condition is reasonable, browse very smooth.Because picture all is that prior generation is placed on server end well, so server end needs the memory device of huge memory capacity.Simultaneously all properties key element on the map all concentrates on the tile, just seems very difficult so realize effect such as highlighted.

Based on the lightweight WebGIS platform of vector data technology, mainly adopt the exhibition method of SVG (Scalable Vector Graphics) as map.The map vector data that platform uses just leaves server end in, the disposable client that sends to during client-requested, utilize the interactivity of SVG image and script JavaScript to finish basic operations such as convergent-divergent, roaming, inquiry in client, do not need and server interaction.

In conjunction with characteristics of the present invention, the WebGIS platform issue Tai Lake blue-green algae real-time distribution information that the MapTile technology is combined with the vector data technology and makes up lightweight.Platform utilizes vector data technology SVG issue Tai Lake map, and uses the MapTile technology that local blue-green algae distribution situation figure is filled in the SVG map of having cut apart, and the platform structure synoptic diagram as shown in figure 15.

Server end can be divided into map publisher server and grid image publisher server by function, and the two can be realized on same machine in the present invention.When client is used the Web browser access server, its SVG map with Tai Lake of map issuing service sends to client and shows, after this, the basic operations such as splicing of the convergent-divergent of map, roaming, inquiry grid image are then finished in client by script JavaScript.The grid image server then is responsible for providing the local blue-green algae distribution plan of zones of different, sends to client, is spliced by client script.

Client is realized by the Ajax technology with communicating by letter of grid image server, Ajax (Asynchronous Javascript and XML) is the comprehensive of multiple technologies, also can be described as a kind of design, it combines the technology such as JavaScript, XHTML and CSS, DOM, XML, XSTL, XMLHttpRequest that comprise.Various technology act on as shown in figure 16 in Ajax.

Whole communication process is asynchronous carrying out, communication process as shown in figure 17, the mode that asynchronous Ajax of referring to and server are got in touch.If the use traditional mode, whenever the user when server sends the local blue-green algae image distribution of longitude and latitude request image block, Web browser will refresh current window.Figure X has showed the asynchronous mode that Ajax adopted, and browser need not wait for that the user operates, and also need not refresh the image block that whole window just can obtain needs.As long as transmit the data that adopt the encapsulation of XML form back and forth, operation JavaScript code just can be got in touch with server.When execution result arrives client, just can notify browser image block to be filled in the corresponding grid in the suitable time.

Claims (7)

1. based on the water surface general view formation method of the image that cruises, it is characterized in that at first setting up imaging system, comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises, the ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device, the server images disposal system is handled video data and is obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module, client end interface shows and interactive system comprises that map filling and display module and grid map make up module, may further comprise the steps:

1), video camera and GPS locating device are set on ships and light boats, video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route, the output video frame sequence, the GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one by the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system, the video stabilization module is carried out pre-service by overall motion estimation, motion compensation and image repair technology to sequence of frames of video, because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, finishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module, comprise camera attitudes such as the video camera inside and outside parameter and the angle of pitch, the visual angle of sequence of frames of video is converted to from side-looking overlooks, utilize the image reconstruction technique realization that hole-filling, the image sharpening of overhead view image are handled again, obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template with based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is according to handling the LIP model based on Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation, generation is based on the electronics grid map of Geographic Information System GIS, wherein use scalable vector graphics SVG as format map, gps coordinate is converted to water surface map reference, with the corresponding relation of the water surface map reference map is divided into different grids by the GPS longitude and latitude, general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

2. the water surface general view formation method based on the image that cruises according to claim 1 is characterized in that step 2) in, the treatment scheme of video stabilization module is as follows:

21) overall motion estimation: adopt the Block Matching Algorithm and the parameter model estimation technique;

22) image motion compensation: at first moving unintentionally and intentionally of sequence of frames of video carried out parameter estimation, adopt Kalman's state filtering method to have a mind to the kinematic parameter estimation, utilize statistical method structural physical state-space model, the dynamic change of having a mind to and being not intended to kinematic parameter is described, and estimate to have a mind to kinematic parameter based on Kalman filter, compensate by image transformation then and be not intended to motion, the motion that makes image sequence with estimate to have a mind to motion model consistent, thereby reach the purpose of video stabilization, suppose n two field picture f _nTransformation parameter is (T _n, d _n), then transformation model is as follows:

${\stackrel{\&OverBar;}{p}}_n={\stackrel{\&OverBar;}{T}}_n{p}_n+{\stackrel{\&OverBar;}{d}}_n---\left(1\right)$

Wherein, p _nBe the point coordinate vector after the image transformation, state equation and observation equation according to Kalman filtering obtain:

${\stackrel{\&OverBar;}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\&prime;}}_n\left(p\right)=f_n({\left({\stackrel{\&OverBar;}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\&OverBar;}{T}}_n\right)}^{-1}{\stackrel{\&OverBar;}{d}}_n\mathrm{})---\left(3\right);$

23) image repair: from view data, obtain iconic model through filtering, parameter or non-parametric estmation and entropy method, employing is based on the digital picture repairing technique of partial differential equation, the marginal information of repairing area is treated in utilization, adopt the direction of estimating isophote to smart method by thick simultaneously, the information around the repairing area for the treatment of is propagated in the repairing area, to realize image repair.

3. the water surface general view formation method based on the image that cruises according to claim 1, it is characterized in that video camera is a pinhole camera, on traditional camera marking method basis, inside and outside ginseng is demarcated respectively during the step 3) camera calibration, with the known calibrating block of structure as the space object of reference, utilize the three-dimensional coordinate of one group of non degenerate point on the calibrating block and corresponding image coordinate by a series of mathematic(al) manipulations and computing method, ask for the inner parameter and the external parameter of camera model, confidential reference items are demarcated and are finished under the laboratory experiment environment, adopt traditional method based on calibrating template; Outer ginseng is carried out on-line proving, realizes by the fixed character object of reference that extracts in the scene.

4. the water surface general view formation method based on the image that cruises according to claim 1, it is characterized in that in the step 4), the image transformation module realizes the visual angle conversion of frame of video by the reverse mapping techniques of image, the mapping relations matrix of foundation from the purpose territory to reference field, the purpose territory is a general view, reference field is a video frame images, the information that obtains with camera calibration is rebuild the depth information in purpose territory in advance, according to certain any coordinate in the general view of required structure (X, Y), obtain the picture planimetric coordinates (x of this point by the mapping relations matrix computations, y), again with (x y) locates the pixel value assignment in (X, Y), the visual angle of sequence of frames of video is converted to from side-looking overlooks; Image reconstruction adopts the algorithm based on template.

5. the water surface general view formation method based on the image that cruises according to claim 1 is characterized in that the treatment scheme of the image mosaic module of step 5) comprises image pre-service, image registration, image co-registration, wherein:

The image pre-service is carried out pre-service to reference picture and image to be spliced, reference picture is got first two field picture of video sequence, pre-service comprises the basic operation of Flame Image Process, set up the matching template of image and image is carried out conversion, wherein with reference picture as matching template, can do some shearings according to actual conditions, image is carried out conversion in order to extract characteristics of image, comprise Fourier transform, wavelet transformation, the Gabor conversion, extract the characteristic set of image afterwards, utilize the rough position relation of feature calculation reference picture and image to be spliced, promptly treat stitching image and carry out coarse localization, find overlapping region roughly, dwindle matching range, raising speed;

Image registration adopts improved Normalized Grey Level level correlation technique NGC to carry out image registration in conjunction with the GPS locator data, improved Normalized Grey Level level correlation technique NGC is: the gray level that the gray level of pixel in the matching template of image be multiply by the respective pixel that is covered by matching template in the image to be spliced, the summation NC value that obtains is stored in the two-dimensional array, the general view that described matching template and image to be spliced have not all transformed through step 4):

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For mobile matching template starting point to image to be spliced (i, j) image is called subgraph by the zone that template covers during the position, the position of matching template in image to be spliced is by the subgraph decision the most similar to matching template, with matching template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S of θ minimum ^{I, j}The position at place is the position of template in image:

$\cos {\mathrm{\&theta;}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[T(m,n)\right]}^2}}---\left(11\right);$

Image co-registration adopts log-domain Flame Image Process model LIP, and image transitions to be spliced is handled to the LIP territory, realizes figure image intensifying, gamma correction.

6. the water surface general view formation method based on the image that cruises according to claim 5 is characterized in that in the image amalgamation, being positioned at

The brightness value of the pixel at place is considered as a tri-vector, is referred to as color vectors, uses

Expression,

$\stackrel{\mathrm{ur}}{F}\stackrel{r}{\left(v\right)}=\left(\begin{array}{c}{F}_r\stackrel{r}{\left(v\right)}\\ F_g\stackrel{r}{\left(v\right)}\\ F_b\stackrel{r}{\left(v\right)}\end{array}\right)---\left(18\right)$

In calculating, quote LIP territory definition With

Algorithm, and mapping function ψ are handled three passages of R, G, B of coloured image respectively, on the LIP of gray level image model, set up the logarithm of coloured image and handle the CLIP model, in the CLIP territory, handle Surface Picture, make image more press close to true visual signature.

7. the water surface general view formation method based on the image that cruises according to claim 1 when it is characterized in that in the step 6) that map with the water surface is by mesh segmentation, is divided into hexagonal mesh map of uniform size.

Images