CN101976429B - Cruise image based imaging method of water-surface aerial view - Google Patents

Abstract

The invention relates to a cruise image based imaging method of a water-surface aerial view, which comprises the steps of: firstly, establishing an imaging system which comprises a server image processing system, a client interface display and interactive system, and a video collecting system of a cruise boat; installing a video camera and a GPS (Global Position System) device on the boat; collecting water-surface video data; processing the video data by the server image processing system to obtain a water aerial view; and inputting the water-surface aerial view into the client interface display and interactive system. A water-surface video is collected by the cruise boat, and is converted into the water aerial view. The method has the advantages of small distortion, short period, high aging performance, low investment and good compatibility, and is disturbed by less noise.

Description

Water surface general view formation method based on the image that cruises

Technical field

The invention belongs to technical field of image processing; Relate to mentioned image stabilization techniques, camera calibration technology, image repair technology etc.; To convert the general view of the water surface through the video frame images that the patrol of water surface ships and light boats is taken into; Show the large tracts of land buoyance,, be specially a kind of water surface general view formation method based on the image that cruises like blue-green algae etc.

Background technology

Since the sixties in 20th century, the economy of Taihu Lake basin develops rapidly, brings immense pressure to environment, has caused the havoc of the ecosystem and the decline of environmental quality.By existing " water environment quality standard GB3838-88 ", the mid-90 in 20th century Taihu Lake water quality on average reached the IV class, 1/3 lake region is the V class, shows as average rank of per 10 years deteriorating water qualities, decline rate was obviously accelerated in nearly more than 10 years.Along with the quickening of eutrophication process, waters blue-green alga blooms such as Taihu Lake Mei Liangwan, silk Shan Hu, western Taihu Lake bank frequently take place.Blue-green algae misgrowth is very easily piled up, rotten sedimentation forms wawter bloom; In the river mouth and offshore alluvial, not only destroy water landscape and ecosystem balance, and owing to blue-green algae discharges toxin in growth course; Consume dissolved oxygen DO; Cause the biological mass mortality of water body, the lake water quality deterioration, serious threat the safe drinking water of surrounding area, lake.2007, the blue-green algae incident of Taihu Lake in 2008 outburst is particularly outstanding, its break out the time early, scale intensity is big, consequence serious threat the safe drinking water in cities such as Wuxi, make economy be subjected to tremendous loss.Therefore, control lake water pollution is kept ecological balance, and to guarantee human normal living environment, has become one of modern society's sixty-four dollar question.

Except blue-green algae, also have other status of the water area to need monitoring on a large scale, like the red tide on sea, the water pollution that Oil spills etc. cause etc.

On July 16th, 2010, pipe laying of Dalian Bay explodes suddenly, cause estimation at least up to ten thousand tons of crude oil leakages go into the sea, ecological safety is caused serious harm, caused incalculable damage to sea-farming and tourist industry etc.Large quantities of sea life are because of poisoning or death by suffocation, and toxic compounds such as benzene that crude oil is contained and toluene can get into food chain.In addition, potential infringement further expands in the local ecosystem, and the biology of survival can entail the offspring to the influence of poisonous substance in several years.

On October 11st, 2010, southeaster is brought than landing the Zhangpu, Fujian in all Asias of typhoon, and red tide is on a large scale caused at the salt port in the Huidong, and more than 2500 the net cage fish in salt port are dead in 3 days because of the red tide anoxic, cause more than 1,700 ten thousand yuan of direct economic losses.

Water quality monitoring is the main foundation of water quality assessment and water prevention and cure of pollution, day by day serious along with water pollution problem, and water quality monitoring becomes the significant problem that social economy's sustainable development must solve.Four kinds of methods are mainly taked in the landlocked water body water quality monitoring of China at present:

(1) direct Detection Method.Water sample is gathered in lake to monitoring, carries out water analysis in the laboratory, obtains relevant chemical parameters, represents near waters or adopts its spatial and temporal distributions of statistical methods analyst with the parameter of sampled point.The advantage of this method is to detect multiple water quality parameter; Comprise dissolved oxygen DO, water temperature, pH value, conductivity, transparency; Permanganate index, ammonia nitrogen, total nitrogen, total phosphorus, chlorophyll A, algae gross density or the like are to large-area waters with the closely-related information of water quality, weak point, adopt that this method is consuming time, expensive, consumption power; Be difficult to obtain waters state parameter on a large scale, can not satisfy, on a large scale monitoring and evaluation requirement real-time, continuous water quality.

(2) fixed cameras monitoring.With Taihu Lake is example, and reach an agreement with China Telecom in Wuxi City environment measuring center at present, adopts " global eyes " technology to realize the real-time video monitoring of the Mei Liang lake stretch of coastal water.This system tentatively accomplishes, but still comes with some shortcomings.

1. expensive: both sides adopt the mode of rental service, all build by China Telecom and provide funds, and comprise maintenance in the future, and its service is rented at environmental protection tests center, Wuxi, pays general 500,000 yuan network rent expense every year.

2. image quality is not good: telecommunications " global eyes " adopts 10Mbps optical fiber to insert, and is not very smooth at the picture of inspection center.The sharpness that adopts at present is 4CIF, and frame per second is not less than 20 frames, and camera adopts cradle head control, can long-rangely carry out level, rotate and the remote adjustment lens focus up and down.

3. the monitoring territory is limited: close shot can be seen clearly basically, but camera lens when zooming out picture waterborne can't distinguish that it is relevant with the camera quality on the one hand, the height that sets up with The Cloud Terrace on the other hand is relevant.

(3) artificial patrol is kept watch on.Artificial patrol is kept watch on by accomplishing jointly along each department, waters, all departments.The tour point is set in the maritime belt, and wind direction, water colour, contaminated area, the algae in essential record observation station periphery waters assemble situation, confirm the observation frequency according to incident generation rank.Manual patrol has strong subjectivity, criterions such as water colour, algae gathering situation is difficult to unified, and parameters such as contaminated area can only estimate and draw that the monitoring frequency need consume a large amount of human and material resources when higher.

(4) remote sensing monitoring method.Landlocked water quality remote sensing monitoring at present commonly used is based on selections remote sensing wave band datas such as experience, statistical study, water quality parameter spectral signature and carries out statistical study, sets up water quality parameter inversion algorithm realization.At the initial stage seventies, the Remote Sensing Study of land water body developed into from simple waters identification water quality parameter is carried out remote sensing monitoring, drawing and prediction.Along with to the substance spectra The Characteristics deeply, the improvement of algorithm and the continuous innovation of remote sensing technology itself; Remote sensing monitoring water quality develops into quantitatively from qualitative; And can increase gradually through the water quality parameter kind of remote sensing prediction; The vertical attenuation coefficient and some comprehensive pollution indexs such as the nutritional status index etc. that comprise incident and emergent light in suspended particulate substance, water transparency, chlorophyll-a concentration and dissolved organic matter, the water.

Remote sensing monitoring is divided into two types again: one is based on high spatial resolution, like TM (Thematic Mapper) data of the HRV data of French SPOT satellite, the U.S. land explorer satellite LANDSAT of system or carry the direct extraction to the lake relevant information such as ASTER data on the Terro satellite; Two are based on the MODIS data such as (Moderate-resolution Imaging Spectroradiometer) of carrying on Terra (EOS-AMI) satellite of AVHRR, U.S. emission in 1999 of resolution between low-to-medium altitude such as weather satellite NOAA combines relevant algorithm to extract target information.

Remote sensing technology is the method that gets most of the attention in the water quality monitoring on a large scale, but still has following problem:

1. the high-resolution remote sensing image temporal resolution is low and cost an arm and a leg.Repeating covering cycle like ASTER is 4～16 days, and the cycle of operation of TM is 16 days, the track of SPOT be " stationary state " (phased), repeating covering cycle is 26 days.The ASTER image is approximately 800 yuan/scape; 4000 yuan/scape of TM image; SPOT1,2,3,9900 yuan/scape of 4 image datas, the SOPT5 of resolution 10m and 5m then is 14900 a yuan/scape, 2.5m up to 29800 yuan/scape.

2. there is the mixed pixel problem in the remote sensing monitoring data of middle low resolution, and precision is limited.Remote sensing is to be unit with the pixel to the detection of atural object, and pixel has certain area, and seldom is made up of single and uniform face of land covering class, all is the amalgam of several kinds of atural objects generally.Although different natural feature on a maps has characteristics such as its different spectrum, time, angle, the pixel of remote sensing record has only characteristics such as single spectrum, time, angle, the characteristic after promptly mixing, thus cause puzzlement to remote Sensing Interpretation.The ground resolution of MODIS instrument is 250m, 500m and 1000m; The substar resolution of AVHRR is 1.1km, because scan angle is big, part distortion in image border is bigger, in fact its most useful part (15 ° are located ground resolution is 1.5km) in ± 15 ° of scopes.The mixed pixel problem is the significant obstacle that remote sensing technology deeply develops to quantification.

3. remote sensing images are subject to cloud layer, atmospheric effect, cause obtaining the ground image or disturb greatlyyer, are difficult to the dynamic change of real-time continuous trackable surface rainy season.Taihu Lake basin is rainy, and average annual quantity of precipitation is 1100 milliliters, and it belongs to the subtropics monsoon climatic region; Every year, at the end of spring and the beginning of summer the current of warm air of going up north met in this with the cold airflow of going down south, and both are evenly matched; One month locked time, even five or six ten days, plum rains without stop formed; And also blue-green algae high-incidence season just this moment, remote sensing monitoring is difficult to play effectiveness.What satellite sensor received is the signal of clutter reflections solar radiation; In the interaction of solar radiation and earth atmosphere, can produce effects such as absorption, reflection, scattering and emission, these all can cause the distorted signals that sensor receives; Make image quality decrease; Water body particularly, its relative land of reflectance spectrum signal a little less than, so atmospheric condition can cause larger interference to monitoring result.

All there is top problem in various waters in the time need the monitoring of whole visibility status being arranged to the water surface.

Summary of the invention

The problem that the present invention will solve is: prior art exists not enough to the monitoring of the water surface, and as not satisfying the requirement of water field of big area monitoring, the monitoring real-time is not enough, monitor with high costs, monitoring accuracy susceptible to etc.; Need a kind of easy easy realization, the high respond well water surface monitoring method of precision.

Technical scheme of the present invention is: based on the water surface general view formation method of the image that cruises; At first set up imaging system; Comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises; The ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device, and the server images disposal system is handled video data and obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module; Client end interface shows and interactive system comprises that map filling and display module and grid map make up module, may further comprise the steps:

1), video camera and GPS locating device are set on ships and light boats; Video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route; The output video frame sequence; The GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one through the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system; The video stabilization module is carried out pre-service through overall motion estimation, motion compensation and image repair technology to sequence of frames of video; Because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, accomplishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module; Comprise camera attitudes such as the video camera inside and outside parameter and the angle of pitch; The visual angle of sequence of frames of video converted into from side-looking overlook; Utilize image reconstruction technique to realize that hole-filling, image sharpening to overhead view image handle again, obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template with based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is main according to handling the LIP model with Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation; Generation is based on the electronics grid map of GIS-Geographic Information System GIS; Wherein use scalable vector graphics SVG as format map; Gps coordinate converts water surface map reference into; Corresponding relation through GPS longitude and latitude and water surface map reference is divided into different grid with map, and general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

Step 2) in, the treatment scheme of video stabilization module is following:

21) overall motion estimation: adopt the BMA and the parameter model estimation technique;

22) image motion compensation: at first moving unintentionally and intentionally of sequence of frames of video carried out parameter estimation; Adopt Kalman's state filtering method to have a mind to the kinematic parameter estimation; Utilize statistical method structural physical state-space model, describe the dynamic change of having a mind to and being not intended to kinematic parameter, and estimate kinematic parameter intentionally based on Kalman filter; Compensate through image transformation then and be not intended to motion; The motion that makes image sequence with estimate to have a mind to motion model consistent, thereby reach the purpose of video stabilization, suppose n two field picture f _nTransformation parameter is (T _n, d _n), then transformation model is following:

${\stackrel{\‾}{p}}_n={\stackrel{\‾}{T}}_n{p}_n+{\stackrel{\‾}{d}}_n---\left(1\right)$

Wherein, p _nBe the vector of the point coordinate after the image transformation, state equation and observation equation according to Kalman filtering obtain:

${\stackrel{\‾}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\′}}_n\left(p\right)=f_n({\left({\stackrel{\‾}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\‾}{T}}_n\right)}^{-1}{\stackrel{\‾}{d}}_n\mathrm{})---\left(3\right);$

23) image repair: from view data, obtain iconic model through filtering, parameter or non-parametric estmation and entropy method; Employing is based on the digital picture repairing technique of PDE; The marginal information of repairing area is treated in utilization; Adopt the direction of estimating isophote by thick method simultaneously, the information around the repairing area of treating is propagated in the repairing area, to realize image repair to essence.

As optimal way; Video camera of the present invention is a pinhole camera; On traditional camera marking method basis, inside and outside ginseng is demarcated respectively during the step 3) camera calibration; As the space object of reference, the three-dimensional coordinate that utilizes one group of non degenerate point on the calibrating block and corresponding image coordinate are asked for the inner parameter and the external parameter of camera model through a series of mathematic(al) manipulations and computing method with the known calibrating block of structure; Confidential reference items are demarcated under the laboratory experiment environment and are accomplished, and adopt traditional method based on calibrating template; Outer ginseng is carried out on-line proving, realizes through the fixed character object of reference that extracts in the scene.

In the step 4), the image transformation module is set up the mapping relations matrix from the purpose territory to reference field through the visual angle conversion of the reverse mapping techniques realization of image frame of video; The purpose territory is a general view, and reference field is a video frame images, and the information that obtains with camera calibration is rebuild the depth information in purpose territory in advance; (X Y), obtains the picture planimetric coordinates (x of this point through the mapping relations matrix computations according to certain any coordinate in the general view of required structure; Y), again with (x y) locates the pixel value assignment in (X; Y), the visual angle of sequence of frames of video is converted into from side-looking overlook; Image reconstruction adopts the algorithm based on template.

Further, the treatment scheme of the image mosaic module of step 5) comprises image pre-service, image registration, image co-registration, wherein:

The image pre-service is carried out pre-service to reference picture and image to be spliced, and reference picture is got first two field picture of video sequence, pre-service comprise Flame Image Process basic operation, set up the matching template of image and image carried out conversion; Wherein with reference picture as matching template, can do some shearings according to actual conditions, image is carried out conversion in order to extract characteristics of image; Comprise Fourier transform, wavelet transformation, Gabor conversion, extract the characteristic set of image afterwards, utilize the rough position relation of feature calculation reference picture and image to be spliced; Promptly treat stitching image and carry out coarse localization; Find overlapping region roughly, dwindle matching range, raising speed;

Image registration adopts improved Normalized Grey Level level correlation technique NGC to combine the GPS locator data to carry out image registration; Improved Normalized Grey Level level correlation technique NGC is: the gray level that the gray level of pixel in the matching template of image multiply by the respective pixel that is covered by matching template in the image to be spliced; The summation NC value that obtains is stored in the two-dimensional array, the general view that said matching template and image to be spliced have not all transformed through step 4):

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For move matching template starting point to image to be spliced (i, j) image is called subgraph by the template region covered during position, the position of matching template in image to be spliced is by the subgraph decision the most similar with matching template, with matching template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S that θ is minimum ^{I, j}The position at place is the position of template in image:

$\cos {\mathrm{\θ}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left[T(m,n)\right]}^2}}---\left(11\right);$

Image co-registration adopts log-domain Flame Image Process model LIP, and image transitions to be spliced to LIP territory is handled, and realizes figure image intensifying, gamma correction.

As preferably; In the image amalgamation; The brightness value of the pixel that is positioned at

and locates is considered as a three-dimensional vector; Be referred to as color vectors; The expression with

$\stackrel{\mathrm{ur}}{F}\stackrel{r}{\left(v\right)}=\left(\begin{array}{c}{F}_r\stackrel{r}{\left(v\right)}\\ F_g\stackrel{r}{\left(v\right)}\\ F_b\stackrel{r}{\left(v\right)}\end{array}\right)---\left(18\right)$

In calculating, quote

and algorithm of LIP territory definition; And mapping function ψ; Three passages of R, G, B of coloured image are handled respectively; On the LIP of gray level image model; Set up the logarithm of coloured image and handle the CLIP model; In the CLIP territory, handle Surface Picture, make image more press close to true visual signature.

As preferably, in the step 6) map of the water surface during by mesh segmentation, is divided into hexagonal mesh map of uniform size.

The present invention as system front end, gathers Surface Picture with the boat-carrying video camera, and with the video sequence input system, system at first utilizes the video stabilization technology, because the float that causes of jolting of ship is removed, obtains stable video sequence image; Then through the image transformation module; Utilize camera calibration technology camera attitude such as the calibrating camera inside and outside parameter and the mensuration angle of pitch automatically; Utilize the image transformation technology with the visual angle from test conversion for overlooking; Utilize technical finesse vertical views such as hole-filling, image sharpening again, obtain the clear general view behind the view transformation; Next be the image mosaic module, utilize the image mosaic technology video image that one width of cloth width of cloth is isolated to piece together, obtain a big figure, splicing is according to being treated to the master with Image Feature Point Matching and CLIP, and the GPS geographic coordinate is auxilliary; The big figure of the water surface that splicing is accomplished issues on WEB with the form of grid map.

Can't obtain the ground image down or disturb bigger deficiency to the overcast and rainy cloudy weather of satellite remote sensing, the present invention proposes to have complementary advantages with existing monitoring method based on the ships and light boats status monitoring new method of waters on a large scale with video camera of cruising.The high-resolution remote sensing image price is extremely expensive; And the boat-carrying video camera cost that the present invention is based on existing patrol ships and light boats is moderate; And can gather equal resolution even high-definition picture more; And it closely forms images, and disturbed by atmosphere, weather less, and the Taihu Lake blue-green algae distribution plan system that makes up based on the image that cruises has broad application prospects.

The present invention makes up Video processing, analysis, integrated, delivery system, has patrol videograph, location, map reproduction and human-computer interaction function.System integration GPS geo-location data provide map to show with the view data of cruising, and can be used for instructing ships and light boats patrol direction, and auxiliary its exhaustively travelled the territory, lake, in order to avoid blindly cruise.

The present invention proposes the new method that makes up territory, lake general view, alleviate dependence satellite remote sensing system.On the camera calibration technical foundation, realize image mapped and reconstruction, skeleton view is converted into general view,, and eliminate between the piece image luminance difference that causes because of uneven illumination etc., form territory, whole lake general view based on the image mosaic technology.

The gridding map mode that proposes makes up and has high-resolution grid image piece, its independent processing, dynamically updates, and has and handles advantages such as convenient, real-time.Based on GIS map form and SVG technique construction Taihu Lake grid image, can represent the distributed intelligence of blue-green algae universe in real time, can extract independent gridblock again, represent the high resolving power detailed information.

The Flame Image Process mode based on log-domain LIP and CLIP that proposes, problems such as overflowing in the solution image calculation, distortion make it approach the human visual system more.Common addition and the scalar multiplication computing used always in the Flame Image Process; Inconsistent with image formation rule; Its image to being formed by transmitted light all is unaccommodated to human visual perception with a lot of real figure images, usually causes calculating and overflows; The log-domain image processing techniques preferably resolves this problem, and its sealing operational pattern can not cause calculating to be overflowed.

The present invention has the following advantages:

1. distortion is little, and noise is few.Be different from remotely sensed image, very short based on light radiation path in the video camera imaging of patrol ships and light boats, influence quite little; Not influenced by cloud layer; There is not the mixed pixel problem yet, need not to carry out complicated pre-service such as atmosphere compensation, even overcast and rainy cloudy weather also can be caught real-time lake condition image.

2. the cycle is lacked ageing height.According to investigations, be to hold in real time water surface situation, routine monitoring needs report 1 time every day Monitoring Data, breaks out the most serious period like blue-green algae, and emergency monitoring project monitoring at least 1 day 2 times, automatic on-line monitoring then need every 2h～4h reported data 1 time.The satellite remote-sensing image excessive cycle can not satisfy the real-time demand.The present invention can issue through handling in several seconds through the lake area image that the patrol ships and light boats are gathered, and is ageing very high.

3. drop into low, compatible good.Utilize existing patrol ships and light boats, the configuration video camera can be realized the waters condition monitoring, with respect to the satellite remote-sensing image that needs are given a long price for, and the monitoring means of its Cheap highly effective of can yet be regarded as.

Description of drawings

Fig. 1 is the structural drawing of imaging system of the present invention.

Fig. 2 is the process flow diagram of the inventive method.

Fig. 3 is the processing flow chart of video stabilization module.

Fig. 4 is the desirable little pore model of video camera.

Fig. 5 demarcates the perspective projection model based on the plane template demarcation that adopts for camera calibration module confidential reference items of the present invention.

Fig. 6 mixes the video camera town and country model of demarcation outward for camera calibration module of the present invention.

Fig. 7 is the forward mapping synoptic diagram in the image mapped.

Fig. 8 is the reverse mapping synoptic diagram in the image mapped.

The reverse mapping synoptic diagram that Fig. 9 adopts for the present invention.

Figure 10 is an image reconstructing method synoptic diagram of the present invention.

Figure 11 is an image mosaic process flow diagram of the present invention.

Figure 12 is the linear excessively synoptic diagram of method in overlay region in the image mosaic.

Figure 13 is the flow path switch synoptic diagram of gps coordinate of the present invention and electronic chart coordinate.

The sexangle grid synoptic diagram that Figure 14 adopts for the present invention.

Figure 15 is a WebGIS platform structure synoptic diagram.

Figure 16 for the present invention in the electronics grid map being carried out WEB issue, the application function synoptic diagram of Ajax.

Figure 17 for the present invention in the electronics grid map being carried out WEB issue, the synoptic diagram of Ajax asynchronous communication process.

Embodiment

The present invention is based on the boat-carrying video camera and make up surface, waters general view, be different from land fixed cameras supervisory system, its image cruise characteristic and water surface image be different from terrestrial landscape characteristics determined native system need adopt the particular processing mode.Fig. 1 has shown imaging system framework of the present invention; Comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises; The ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device; The server images disposal system is handled video data and is obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module, and client end interface shows and interactive system comprises that map filling and display module and grid map make up module.Like Fig. 2, the present invention includes following steps:

1), video camera and GPS locating device are set on ships and light boats; Video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route; The output video frame sequence; The GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one through the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system; The video stabilization module is carried out pre-service through overall motion estimation, motion compensation and image repair technology to sequence of frames of video; Because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, accomplishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module; Comprise camera attitudes such as the video camera inside and outside parameter and the angle of pitch; The visual angle of sequence of frames of video converted into from side-looking overlook; Utilize image reconstruction technique to realize that hole-filling, image sharpening to overhead view image handle again, obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template with based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is main according to handling the LIP model with Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation; Generation is based on the electronics grid map of GIS-Geographic Information System GIS; Wherein use scalable vector graphics SVG as format map; Gps coordinate converts water surface map reference into; Corresponding relation through GPS longitude and latitude and water surface map reference is divided into different grid with map, and general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

Blue-green algae distribution monitoring with Tai Lake specifies enforcement of the present invention below.

1.1.1. video stabilization is technological

The video stabilization technology has three main steps: Global Motion estimation, image motion compensation, video reparation, and algorithm flow chart is as shown in Figure 3:

1) image overall is estimated

Because the motion of the relative background of video camera is a kind of global motion, promptly its image change of causing is a kind of global change, so can compensate with GLOBAL MOTION ESTIMATION TECHNOLOGY in the video stabilization algorithm.Mainly being divided into of the estimation of global motion based on BMA with based on the parameter model estimation technique.

The basic thought of block matching method is to be divided into square to present frame; To each piece of present frame certain zone in reference frame all; Be in the search window; According to the piece that certain matching criterior search has minimum match error with it, this piece is the match block of current block, and the coordinate displacement between match block and the current block is exactly a motion vector.Complexity based on the estimation of block matching method depends primarily on the calculated amount of matching criterior and these two aspects of searching algorithm of employing.This method is calculated simple, but algorithm can not be estimated the motion that the rotation, zoom of video camera etc. cause usually well.

The parameter model algorithm for estimating is meant through setting up different parameter models can describe different forms of motion; Parameter model is intended to describe quadrature or the perspective projection of three-dimensional motion at the plane of delineation; Model uses the wherein motion of each pixel of a spot of parametric description; Compare with block matching method, parameter model can be described motion more compactly, and is not easy to receive The noise.At present main use six parametric methods based on parallel projection are arranged, eight parametric methods based on perspective projection are also arranged, this type algorithm can be described motions such as rotation and the zoom of video camera.

2) image motion compensation

The purpose of video stabilization be to alleviate or eliminate the video camera carrier rock the image sequence interframe that caused unintentionally, harmful motion, the main motion that exists in the reservation queue image or have a mind to motion.Motion compensation comprises two steps, at first the actual motion of image sequence is carried out parameter estimation with having a mind to move, and compensates through the appropriate image conversion then and is not intended to motion, thereby reach the purpose of video stabilization.

Traditional kinematic parameter algorithm for estimating of having a mind to has curve fitting method and moving average filter method, and the curve fitting fado adopts lower-order model to carry out least square method the actual parameter that overall motion estimation obtains is carried out curve fitting, thereby estimates kinematic parameter intentionally.Curve movement is to have a mind to the stack of motion and randomized jitter, and having a mind to motion, often to represent low frequency component, randomized jitter be high fdrequency component, and the moving average filter method can be used for filtering and represent the high fdrequency component of randomized jitter.But all there is bigger defective in above-mentioned two kinds of methods, and classic method can be used for the present invention, and just calculated amount is huge, are difficult to guarantee its real-time with the arithmetic speed of current hardware.And the real-time that Target Tracking System is handled data is had relatively high expectations, and should before the next frame image arrives, accomplish the processing to former frame.

Kalman's state filtering method can be used to have a mind to kinematic parameter and estimates that its main thought is to utilize statistical method structural physical state-space model, describes the dynamic change of having a mind to and being not intended to kinematic parameter, and estimates to have a mind to kinematic parameter based on the Kalman wave filter.The dynamic motion model that this algorithm is constructed can be described owing to the interframe movement that camera motion caused practically, and estimates to obtain best kinematic parameter intentionally through recurrence.In addition, this method applying flexible can utilize various obtainable prioris through suitable correction state-space model, as having a mind to motion and be not intended to the characteristics etc. of motion

Traditional curve fitting method and moving average filter method have all been used the kinematic parameter of subsequent image frames, obviously can't satisfy higher real-time requirement.In addition, curve fitting method is difficult to adapt to comparatively complicated motion form, and there is the problem of " steady excessively " and " owing steady " in the moving average filter rule.And Kalman's state filtering method is estimated the current kinematic parameter of having a mind to according to up-to-date real image kinematic parameter of past, and calculated amount is less, thereby has overcome the defective that the conventional motion filtering algorithm is difficult to satisfy the high real-time requirement.

The purpose of image transformation is that compensation is not intended to motion, the motion that makes image sequence with estimate motion model is consistent intentionally.Suppose that n two field picture transformation parameter is (T _n, d _n), then transformation model is following:

${\stackrel{\‾}{p}}_n={\stackrel{\‾}{T}}_n{p}_n+{\stackrel{\‾}{d}}_n---\left(1\right)$

Wherein, p _nBe the vector of the point coordinate after the image transformation, state equation and observation equation according to Kalman filtering can obtain:

${\stackrel{\‾}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\′}}_n\left(p\right)=f_n({\left({\stackrel{\‾}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\‾}{T}}_n\right)}^{-1}{\stackrel{\‾}{d}}_n\mathrm{})---\left(3\right)$

3) image repair technology

Have two big types of image repair technology at present, two kinds of image mending methods can be applied to the present invention: one type is to be used to repair the damaged digital picture of small scale to repair (inpainting) technology.The marginal information of repairing area is treated in its utilization, adopt simultaneously a kind ofly to estimate the direction of isophote (isophote) by thick to smart method, and the employing mechanism of transmission propagates into information in the zone to be repaired, so that obtain repair efficiency preferably.In essence; It is a kind of based on PDE (Partialdifferential Equation; PDE) inpainting algorithm, the main thought of these class methods are to utilize the thermal diffusion equation in the physics will treat that the information around the repairing area propagates in the repairing area.Another kind of is image completion (completion) technology that is used for blank map picture bulk drop-out.At present; This type technology also comprises following two kinds of methods: a kind of recovery technique that is based on picture breakdown; Its main thought is to be structure division and texture part with picture breakdown, and wherein structure division is repaired with the inpainting algorithm, and texture part is filled with texture synthesis method.

Main processing water surface view of the present invention, its texture information is less, and video sequence is that image repair provides many data sources, can not produce the situation of bulk information dropout, adopts simply to find the solution PDE and get final product with the method for diffusion neighborhood.

From the mathematics angle, image repair is exactly to treat in the repairing area according to treating that repairing area information on every side is filled into image.Yet; Image mending is an ill-conditioning problem normally, because still there is not enough information can guarantee unique part of correctly recovering to be damaged at present, so; People analyze from the angle of psychology of vision, have proposed various hypothesis qualifications and have been used for addressing this problem.It is thus clear that image mending belongs to the research field of image restoration.

Usually image often receives the influence of some factors in acquisition process, makes deteriroation of image quality.In the image restoration field, degradation model commonly used is

I ⁰＝I+N (4)

Wherein, I ⁰Be the observation image that is obtained, I is original image (I={I (x) }), and N is an additive white noise.Concerning most image mending problem, data model has following form:

I ⁰| _Ω/D＝[I+N] _Ω/D (5)

Wherein, Ω representes entire image zone, and D representes the repairing area of treating of information dropout, Ω D represent not have the zone of drop-out, I ⁰For Ω D on available image section, the target image of I for need restoring.Suppose N for Gauss, the energy function E about data model can use the least mean-square error definition always so:

$E\left[I^0\right|I]=\frac{\mathrm{\λ}}{2}\underset{\mathrm{\Ω}/D}{\∫}{(I-I^0)}^2\mathrm{dx}---\left(6\right)$

Because any available data of repairing area D; Therefore image (priori) model is concerning the image mending algorithm; Than other traditional recovery problems (like denoising; Go to fall clear) become even more important, and iconic model can obtain through filtering, parameter or non-parametric estmation and entropy method from view data.

1.1.2. camera calibration is technological

Pin-hole model is be suitable for a lot of computer visions application the most approximate, and pinhole camera is accomplished perspective projection.Fig. 4 has drawn the pin-hole model of video camera, and it has defined following four coordinate systems ^[2]:

European world coordinate system (being designated as ω down): initial point is at O _w

European camera coordinate system (being designated as c down): initial point is at camera lens focus C=O _c, coordinate axis Z _cWith optical axis coincidence and point to outside the plane of delineation.

European plane of delineation coordinate system (being designated as i down): with video camera principal point (intersection point of the optical axis and the plane of delineation) is initial point, X _iAnd Y _iBe positioned on the plane of delineation, parallel with the X axle of camera coordinate system respectively with the Y axle.

Pattern matrix coordinate system (being designated as f down): the image pixel coordinate during computer frame is deposited.

Based on above-mentioned four coordinate systems, pin-hole model is accomplished the linear transformation of 3D world coordinates to the 2D pixel coordinate:

$\left(\begin{array}{c}U\\ V\\ W\end{array}\right)=K(R,T)\left(\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right)---\left(7\right)$

$X_f=\frac{U}{W},Y_f=\frac{V}{W}---\left(8\right)$

Wherein K is video camera confidential reference items matrixes, and (R is outer ginseng matrix T), and wherein R is the rotation matrix of 3*3, and T is a translation vector.Linear camera calibration problem is promptly found the solution inside and outside ginseng matrix.In the high accuracy vision detection system, also need consider the camera lens distortion.

The most general division is to the camera calibration algorithm: traditional camera calibration algorithm, camera self-calibration method and based on the scaling method of active vision.This several method can both be used for camera calibration of the present invention, and active vision is demarcated owing to more to limit movement, and the boat-carrying camera motion is complicated, therefore needs to draw its advantage, adopts suitable calibration algorithm to actual conditions.

Traditional camera marking method is at specific experiment condition; With the known calibrating block of structure as the space object of reference; The three-dimensional coordinate that utilizes one group of non degenerate point on the calibrating block and corresponding image coordinate are asked for the inner parameter and the external parameter of camera model through a series of mathematic(al) manipulations and computing method.The tradition scaling method has had ripe method and theory.

Self-calibration technology does not need known object of reference, only utilizes the image of video camera surrounding environment in motion process and the corresponding relation between the image that video camera is demarcated.Its advantage is not rely on caliberating device and online carrying out.Desirable self-calibration technology not by any with reference to calibrating block, a series of scenes of only rely on taking are demarcated according to the corresponding relation of each characteristic in the scene, but this requires high image recognition and characteristic matching technology.Actual camera self-calibration has been used specific object of reference more or less, and like circle, straight line, angle point etc., some that perhaps just directly utilize the objective world have the object of certain regular geometric shapes, like door, window, box, chest etc.

Self-calibration technology based on active vision does not need known object of reference, only obtains specific constraint condition through the motion of control video camera and confirms inner parameter, and calibration process is greatly simplified.But this method is more to the camera motion restriction for fear of finding the solution nonlinear equation.

World coordinate system constantly changes in the process in view of ships and light boats are travelled; And its confidential reference items are constant when adopting fixed cameras, the method that the present invention adopts inside and outside ginseng to demarcate respectively, and confidential reference items are demarcated completion under the laboratory experiment environment; Adopt traditional method, to guarantee precision based on calibrating template; Outer ginseng is carried out on-line proving, through extracting correlated characteristic such as realizations such as position, local horizon, fore unique point in the scene.

● based on the confidential reference items scaling method of plane template

The plane template method of Zhang only needs a printing to have the template (as shown in Figure 5) of grid pattern before video camera, to rotate (to contain three times) more than three times to accomplish demarcation, need not to understand the kinematic parameter of template, concise flow, and precision is higher, can be rated as representative.Under the plane template condition, can think Z _w=0, formula (7) can be rewritten as

$\left(\begin{array}{c}U\\ V\\ W\end{array}\right)=H\left(\begin{array}{c}{X}_{\mathrm{<figure-callout\; id="1"\; label="w\; Y\; w"\; filenames="HDA0000029614080000051.png"\; state="\{\{state\}\}">w</figure-callout>}}& \\ Y_w{}_{}\\ 1\end{array}\right),$ Promptly $s\begin{array}{c}\left(\begin{array}{c}\mathrm{<figure-callout\; id="1"\; label="u\; v"\; filenames="HDA0000029614080000051.png"\; state="\{\{state\}\}">u</figure-callout>}& \\ v\\ 1\end{array}\right)\end{array}=H\left(\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right)---\left(9\right)$

S is a scale-up factor, and (u v) is a pixel coordinate, and H is the Homography matrix of stencil plane to video camera, and it has set up the perspective relation of the stencil plane and the plane of delineation.To calibration point, adopt maximum-likelihood criterion can find the solution the Homography matrix through known N.

● online outer ginseng scaling method

The patrol ships and light boats are with the 20km/h speed; It constantly changes with respect to the Taihu Lake coordinate, can realize the ships and light boats location through the GPS positioning system, and the video camera that is set up on the ship is static with respect to ship itself; Therefore; The former point selection hull point of world coordinate system gets final product in the demarcation, and local water surface can be approximately the plane simultaneously, and calibration process can be reduced to finds the solution two interplanar mapping relations.Fig. 6 has shown abstract boat-carrying camera model:

Supposing that camera lens swing angle with respect to the horizontal plane is 0, is 0 with respect to the drift angle of foot direction, if not zero can make so through adjustment, therefore a demand is separated inclination angle t, when local horizon in the image thus it is clear that the time, but base area horizontal line ordinate value is found the solution t; If the local horizon is invisible, and fore is visible, and the demarcation thing can on the bow be set, and directly finds the solution imaging array.

1.1.3. image transformation is technological

● image mapped (image warping) algorithm

If spatial point X projection is in reference view C _rWith purpose viewpoint C _d, obtain reference pixel X respectively _rWith purpose pixel X _dIn the forward mapping process, mapping is from reference field R _rTo purpose territory R _dCarry out,, set up from X according to Given information _rTo X _dMapping relations: X _d=f _Forward(X _r), as shown in Figure 7, on the contrary reverse mapping is promptly set up from X _dTo X _rMapping relations: X _r=f _Backward(X _d), as shown in Figure 8.

Because the degree of depth is known in the reference field, the forward mapping has higher execution efficient, but the following problem of ubiquity: (1) mapping out-of-bounds problem, because the pixel in the reference field is not all to be mapped to the purpose territory, can calculate a large amount of invalid out-of-bounds pixels; (2) empty problem numerous tiny cavities can occur because the information that the image expansion forms is not enough; (3) block the problem of penetrating more, can't avoid redundant computation for the pixel that is blocked, and the double counting of a plurality of reference pixel when being mapped to a purpose pixel.

Reverse mapping is adopted and is drawn the principle that drives; Oppositely set up mapping from the purpose territory; The problems in the forward mapping have been solved effectively; But because the degree of depth in purpose territory is unknown, the process of the pixel search that it is O (width/2) that reverse mapping usually needs an average complexity, thus cause the bottleneck of computing velocity.Can adopt two kinds of methods to the optimization of reverse mapping: (1) is according to polar curve geometric relationship optimization searching process; (2) utilize the three-dimensional information of scene to rebuild the depth information that obtains the purpose territory in advance, thereby reverse mapping is converted into a forward mapping problems.

Because the present invention has adopted the camera calibration technology, the depth information in purpose territory can be rebuild in advance, has therefore simplified the image mapped process; Flow process is as shown in Figure 9: (X Y), calculates the picture planimetric coordinates (x of this point through imaging array inverse mapping relation according to this coordinate in the general view of required structure; Y); With (x y) locates the pixel value assignment in (X Y) gets final product.May run in the actual mechanical process round, problem such as out-of-bounds, need to adopt suitable strategy to solve.

● image reconstruction algorithm

Image reconstruction of the present invention can adopt based on template (splat) with based on the algorithm of grid (mesh).Shown in figure 10:

(1) based on the algorithm of template.Such algorithm uses the earliest in volume data is drawn, and main thought is parameters such as the degree of depth, direction according to reference pixel, adopts the kernel of a variable size to carry out reconstruct to reference pixel.Pixel can be a zone by Gaussian kernel or other kernel extensions like this, again superposed part is merged drafting, and wherein the calculating of kernel size can be estimated to the bounding box of single pixel or scenario node.Because the diversity of reconstruct kernel; Algorithm based on template can obtain to draw preferably effect; But also come with some shortcomings: at first do not distinguish space structure information, occur the problem of losing shape easily, and; The calculating of template relies on CPU to carry out fully, makes that calculated amount is bigger under high resolving power or complex core situation; In addition, the size of reconstruct kernel be difficult for to be handled, and crosses conference and reduces display effect and increase the calculated amount that the edge merges, and too smallly then can locate to stay too much thin hole on the edge of.

(2) based on the algorithm of grid.This type algorithm is a continuous grid surface according to the space continuity of hypothesis with discrete pixel establishment, with the summit of pixel as grid, fills grid surface with the result of interpolation calculation.Such algorithm can be accomplished by hardware, but can't differentiate the topological relation in space effectively.When neighbor is not positioned at the same space continuous surface, will produces non-existent spatial surface, thereby between the prospect of scene and background, often cause " erasing rubber phenomenon " (Rubber Sheet).

The present invention mainly carries out reconstruct to Surface Picture; The water surface can be approximately the plane; Three-D space structure information is less; Adopt CUDA high-speed computation platforms such as (compute unified device architecture, statistical computation equipment frameworks) can solve the big problem of calculated amount in addition, therefore adopting the algorithm based on template is better selection.

1.1.4. image mosaic is technological

The image mosaic technology mainly is divided into three key steps: image pre-service, image registration, image co-registration, and shown in figure 11.Wherein the pretreated purpose of image is the precision that guarantees next step image registration, and original image is done some folding variations and coordinate transform, comprises the basic operation of Flame Image Process; As histogram handle, the smothing filtering of image etc., set up the matching template of image, image carried out conversion, like Fourier transform, wavelet transformation, Gabor conversion etc. and the operations such as characteristic set of extracting image; Coarse localization; Find overlapping region roughly, dwindle matching range, raising speed.

● image registration algorithm

Up to now, produced many method for registering images both at home and abroad, the whole bag of tricks all is the application towards certain limit, also has characteristics separately.According to the information of the image that image registration utilized, can image registration be divided into four types: based on the image registration of half-tone information, based on the image registration of transform domain, based on the image registration of characteristic with based on the image registration of model.These four class methods may be used to figure of the present invention and are complementary, and are best based on the effect of gray scale, but computation complexity is also the highest, if want efficient and quality to take into account, often several method merges use.

Directly utilize the half-tone information of image to carry out registration based on the method for gradation of image; Through pixel the global optimization of certain similarity measurement (like mutual information, Minimum Mean Square Error etc.) is therebetween realized registration; This method need not cut apart and feature extraction image, so precision is high, robustness is good.But this method for registering is very responsive to grey scale change, does not make full use of the gray-scale statistical characteristic, and is bigger to the half-tone information dependence of every bit.

A classical way based on the image registration of transform domain is a phase correlation method, promptly utilizes the method for Fourier transform, with image by space field transformation to frequency field, realize the registration of image according to the translation principle of Fourier transform.Because distortion such as translation, rotation, convergent-divergent all have correspondent transform at frequency domain, therefore can utilize fourier method in frequency domain, to carry out images match fully.The method of transform domain has the insensitivity to noise, and counting yield is high, ripe fast algorithm (fft algorithm) is arranged and be easy to characteristics such as hardware realization.In general, adopt the method for transform domain a good initial registration parameter to be provided for image mosaic.

At first to treat two width of cloth images of registration based on the method for characteristics of image and handle, extract and satisfy the feature set that application-specific requires, these characteristics as control structure, are sought the mapping relations of two width of cloth image holding bodies.Based on the method for characteristics of image, the quantity of the unique point that after feature extraction, obtains will significantly reduce, and therefore can improve the speed of registration, but the effect of its registration also depends on the extraction precision of unique point and the accuracy of Feature Points Matching to a great extent.

Method for registering based on model is the registration that comes image is carried out gamma correction according to the mathematical model of image fault.Typical algorithm is the conversion optimization that Szeliski proposes, and at first sets up the transformation model between the image sequence, obtains the running parameter in the model through the optimized Algorithm iteration then, thereby realizes treating the registration of stitching image.The conversion optimization can be handled the splicing of geometric transformations such as having translation, rotation, convergent-divergent between the image sequence, without any need for unique point, and fast convergence rate, and on statistics, be optimum.But make whole algorithm reach the convergent requirement, the initial estimate of certain precision must be arranged, the promptly just artificial initial corresponding point of confirming want enough accurate, otherwise will cause the failure of image registration.

Consider that Surface Picture characteristic angle point is less; And blue-green algae often is the changeable color of enriching such as red, yellow, green, brown, black because of the old children of individuality, physiological situation and environment light quality are different; Directly cause Surface Picture color, luminance difference; The red tide on sea, water surface situations such as Oil spills also have obvious color difference, so the present invention adopts improved Normalized Grey Level level correlation technique (NGC) to combine the GPS locator data to carry out image registration.

Original NGC method multiply by the gray level of pixel in the matching template gray level of the respective pixel that is covered by template in the image to be spliced; The summation that obtains (NC value) is stored in the two-dimensional array; Shown in (10), the maximum positions of elements of NC value is exactly the position of template in image in this array.

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For movable platen starting point to image to be spliced (i, j) image is called subgraph by the template region covered during position, i, j are the coordinate of subgraph in image.The position of matching template in image to be spliced is by the subgraph decision the most similar with template, with template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S that θ is minimum ^{I, j}The position at place is the position of template in image.

In view of the color change characteristic of blue-green algae, consider not adopt merely the brightness parameter among the present invention, adopt the matching result possibility of parameters such as tone H, saturation degree S better.

Because dot product itself can not be located exactly, because the size in the NC value also receives the big or small influence of pixel grayscale in the two-dimensional array, if the very bright pixel in a certain zone has bigger gray level in the image, its NC value also just becomes big naturally so.What this algorithm was concerned about more is the angle between vector, and changes location influence with this removal of images brightness.

According to above analysis, can obtain following computing formula:

$\cos {\mathrm{\&theta;}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[T(m,n)\right]}^2}}---\left(11\right)$

● image amalgamation algorithm

The piece of image will be fairly obvious when two width of cloth images to be spliced there are differences, as: even two doubling of the image districts that cause of uneven illumination have bigger brightness inconsistent during owing to shooting, perhaps because image geometry deformation that lens distortion causes or the like.For the sudden change of eliminating light intensity and make the light intensity smooth excessiveness, method commonly used has three kinds:

(1) method of weighted mean

Szeliski uses one " cap shape function " to come weighted mean on the respective pixel of each overlapping frame, and this function is minimum at the place, image border, and contribution is maximum in the center.Weight distribution to each image sampling of synthetic result defines like this: the pixel near more from picture centre is big more to final synthetic result's contribution.The weight distribution function shows as triangle.Picture one cap is so claim cap shape function (hat function).

(2) based on effective weight of Euclidean distance

Each pixel of image all assigns weight, and this weight is proportional with the distance to edge (or nearest invisible point).When splicing, fundamental purpose is to reduce the light intensity contribution of the near pixel of isolated edge.(x y), utilizes piece distance and Euclidean distance, calculates the nearest brocken spectrum or the distance on limit to calculate distance map d in the blending algorithm.Merge all deformation pattern formula:

$C(x,y)=\frac{\underset{k}{\mathrm{\&Sigma;}}w(x,y)I_k^{\&prime;}(x,y)}{\underset{k}{\mathrm{\&Sigma;}}w\left(d(x,y)\right)}---\left(12\right)$

W is a monotonic quantity, w (x)=x, I _kBe the light intensity function of k amplitude variation shape image, the calculating of d is very simple, gets from rectangle four edges minimum value and value.

(3) multiresolution spline technique

Bert.PJ adopts Laplace multiresolution pyramid structure, picture breakdown is become the set of diagrams picture on the different frequency territory, on the frequency field of each decomposition, with doubling of the image boundary vicinity weighted mean.At last the composograph on all frequencies is gathered into piece image.The batten methods of differentiating are on all frequency fields, to handle the boundary vicinity zone more.Therefore workload is big, but quality is high.

In order to eliminate the piece problem of overlay region, effect is preferably the linear excessive method in overlay region at present, and is shown in figure 12.The concrete grammar of realizing is that hypothesis overlapping region width is L, and getting the transition factor is σ (0≤σ≤1), and the x in two width of cloth doubling of the image districts and the minimum and maximum value of y axle are respectively X _Min, X _MaxAnd Y _Min, Y _Max, the then transition factor:

$\mathrm{\&sigma;}=\frac{X_{\max }-x}{X_{\max }-X_{\min }}---\left(13\right)$

The image intensity of overlapping region is:

I＝σI _A(x，y)+(1-σ)I _B(x，y) (14)

I _A, I _BBe respectively the corresponding pixel value of figure A and B, this method makes transition portion smoother, does not have tangible step.

● log-domain Flame Image Process (LIP) technology

The present invention introduces a kind of new Flame Image Process framework---log-domain Flame Image Process model LIP.With respect to the digital image processing techniques of general domain, LIP area image disposal route has its unique advantage:

(1) the LIP model is a complete mathematical theory, and in bounded gray scale treatment of picture, the LIP model has been stipulated a special computing collection.Common addition that uses in present most of Flame Image Process and scalar multiplication computing, inconsistent with image formation rule, its image to being formed by transmitted light all is unaccommodated to the human vision sensation with a lot of actual digital pictures.

(2) LIP area image processing operations criterion is sealed.In to Digital Image Processing, dispersing and possibly drawing a value that exceeds the gradation of image scope of two grey scale pixel values promptly overflowed, and the sealing computing of LIP can address this problem preferably.

(3) introduce spatial mappings function ψ among the LIP; Accomplish gray-scale value and in conversion to the number space; Can computing convert the directly plus-minus mutually of mapping function into ⊙ with

of complicacy, significantly reduced operand.

(4) to be that the brightness size is inversely proportional to the characteristics of saturation degree consistent for the rule of the verified and multiple in practice transmission imaging model of LIP logarithmic image disposal route, multipath reflection imaging model and human brightness sensation; It is consistent with the contrast notion on the psychology; Approach the vision system of human eye more, more approaching with the true environment of human eye impression.

In handled in the LIP territory, the gray scale function definition of image claimed that D is that the space is supported on a non-NULL spatial domain D.D is the real zone of a bounded [0, M], and general M gets 256.LIP fundamental operation criterion defines as follows:

$\&ForAll;f,g\&Element;S$ $f\&CirclePlus;g=f+g-\frac{\mathrm{fg}}{M}$

$\&ForAll;f\&Element;S,\&ForAll;\mathrm{\&lambda;}\&Element;R$ $\mathrm{\&lambda;}\&CircleTimes;f=M-M{(1-\frac{f}{M})}^{\mathrm{\&lambda;}}$

Subtraction has defined the difference between two gray scale function f and the g:

and

with special computing is defined among the vector space E, and space E is being shone upon ψ:

$\mathrm{\&Psi;}\left(f\right)=-M\ln (1-\frac{f}{M})---\left(17\right)$

Space E mapping ψ down and real number space R be algebraically isomorphic.Can convert

of complicacy into mapping function directly add and subtract mutually with

computing, significantly reduce operand.The LIP technology can be used for image background renewal, image reconstruction, figure image intensifying, picture contrast estimation, rim detection and image segmentation etc.The present invention adopts LIP technology solution figure image intensifying, gamma correction problem, and image transitions to be spliced to LIP territory is handled, and can better solve because luminance difference problem greatly between image to be spliced under the situation such as solar irradiation inequality, imaging backlight.The application in the present invention of log-domain Flame Image Process has very, and obvious effects promotes.

● colored logarithmic image is handled (CLIP) technology

In traditional Color Image Processing process, a width of cloth coloured image can be considered a common two dimensional image, and just the brightness of each pixel has three degree of freedom (R, G, B) in the image.In the colored log-domain treatment of picture of CLIP; The brightness value of the pixel that is positioned at

and locates is considered as a three-dimensional vector; Be referred to as color vectors; The expression with

$\stackrel{\mathrm{ur}}{F}\stackrel{r}{\left(v\right)}=\left(\begin{array}{c}{F}_r\stackrel{r}{\left(v\right)}\\ F_g\stackrel{r}{\left(v\right)}\\ F_b\stackrel{r}{\left(v\right)}\end{array}\right)---\left(18\right)$

In calculating, quote and

algorithm of LIP territory definition; And mapping function ψ; R, G, three passages of B are handled respectively; On the LIP model of gray level image; Set up the logarithm of coloured image and handle the CLIP model; In the CLIP territory, handle the blue-green algae image, make image more press close to true visual signature.

1.1.5. grid map constructing technology

● the grid image based on the SVG map generates

SVG (Scalable Vector Graphics) is that W3C is organized as a two-dimentional scalable vector graphics language description standard of overlapping based on the XML language that adapts to develop rapidly needs that Internet uses and formulate.The mark of the employing of traditional HTML static page descriptive language fixes, limited and do not have intension, do not support shortcoming exposed day by day such as vector graphics to come out, more and more satisfied not the WebGIS demand for development.At present the SWF file layout that proposes of the MacroMiedia company on the network is with its image vector, and file is less and have interactivity and gain great popularity, but it is than SVG, and the some shortcomings part is still arranged.XML is used by growing field as the following consolidation form standard in the world that generally acknowledges.SVG is the nature static that solution WebGIS faces as the appearance of describing the subclass of vector graphics of XML, the data layout diversity, and web content performance that platform is relevant and shortage interactivity, Network Transmission waits problem that a brand-new solution is provided slowly.

SVG compares with HTML, has following advantage:

(1) broken through the constraint of HTML fixation mark set, make the content of file abundanter, more complicated, form complete information systems more easily;

(2) SVG is the vector image form, is highly suitable for transmission through network and application.Generally speaking, (like GIF, JPEG) littler, speed of download is faster than other network image forms for the SVG image;

(3) constitute vector image by text.Its Textuality makes the SVG file good professional platform independence arranged and can edit, revise it easily through DOM (Document Obiect Model).Literal in the very outstanding advantage SVG file of another one also can be arrived as keyword search by network search engines.

(4) has dynamic interaction property.The SVG image energy is made different responses, for example Gao Liang, sound effect, special efficacy, animation etc. to user action.In addition, because the integrated plug-in unit of browsing the SVG file among the IE6.0 of Microsoft, this makes that browsing of SVG is convenient, easily.

SWF compares with SVG, and its deficiency is presented as:

(1) the non-opening of SWF standard.SWF is the technology of a relative closure, and the scheme that neither one merges fully between other the open standard.Along with the development of XML and other open standards, the incoordination of SWF will become increasingly conspicuous.

(2) the relatively poor editability of SWF.SWF is the output file form of Flash, generates form as final animation, and its production process is enclosed in the SWF file, can't carry out the secondary editor again.And SVG adopts a kind of text formatting, just can open the edlin of going forward side by side with common edit tool.

(3) SWF can't carry out picture search.Because SWF is a non-textual format, text can not be independent of image and exist, and therefore can't set up the picture search function that is similar to SVG.

In view of above characteristic and the advantage of SVG, the present invention selects to use SVG to show the blue-green algae distribution situation image of storage, transmission and demonstration Tai Lake as format map.

● GPS longitude and latitude data are to the conversion of local map reference

The locating information that GPS receives is based on WGS one 84 coordinate systems, and the data that WGS 1 receives can't directly apply on the electronic chart.So, be mapped GPS locating information under WGS one 84 coordinate systems and the coordinate on the electronic chart, could accurately realize mesh segmentation like this to the SVG map.

The locator data that is obtained by the GPS receiver is the three-dimensional information that comprises longitude L, latitude B and elevation H; And (the Xt of the point on the electronic chart that finally will obtain; Yt) be the picture element of two dimension, and consider altitude figures benchmark disunity, therefore can take to give up the shortcut calculation of elevation H.At first with WGS one 84 coordinates (L, B H) project on the Gaussian plane, obtain its Gaussian plane respective coordinates point (x, y), again with plane four parametric methods of simplifying with its rotation translation be scaled to the electronic chart planimetric rectangular coordinates (Xt, Yt).The locator data that receives of GPS receiver just can intuitively show on electronic chart like this.Its flow path switch is shown in figure 13.

WGS one 84 coordinate systems project to the Gaussian plane mathematical model, i.e. Gauss projection is just being calculated formula suc as formula shown in 19, and wherein, B is the geodetic latitude of subpoint; L=L-L0, L are the geodetic longitude of subpoint, and L0 is the geodetic longitude of central meridian; N is radius of curvature in prime vertical t=tanB of subpoint; η=e*cosB, e are ellipsoid second excentricity.

$\left\{\begin{array}{c}x=X^0+1/2\&times;N\&times;t\&times;{\cos }^{2}B\&times;l^2+1/24\&times;N\&times;t(5-t^2+9{\mathrm{\&eta;}}^2+4{\mathrm{\&eta;}}^4){\cos }^{4}B\&times;l^4\\ +1/720\&times;N\&times;t(61-58t^2+t^4+270{\mathrm{\&eta;}}^2-330{\mathrm{\&eta;}}^2{t}^2){\cos }^{6}B\&times;l^6\\ y=N\&times;\cos B\&times;l+1/6N(1-t^2+{\mathrm{\&eta;}}^2){\cos }^{3}B\&times;l^3\\ +1/120N(5-18t^2+t^4+14{\mathrm{\&eta;}}^2-58{\mathrm{\&eta;}}^2{t}^2){\cos }^{5}B\&times;l^5\end{array}\right.---\left(19\right)$

We only need carry out the coordinate transform on the two dimensional surface, therefore can adopt four more easy Parameters Transformation models.The mathematics implication of its four parameters in Gaussian plane is respectively: Δ X is the translational component of coordinate x; Δ Y is the translational component of coordinate y; M is a scale factor; θ is a rotation amount.

In order to find the solution the parameter of coordinate transform, we need know the coordinate of corresponding point under three above WGS one 84 coordinate system map coordinates systems, establish this group common point P1, P2 ... the coordinate of PN (N>=3) under two coordinate systems be respectively (Xg, Yg) _i(Xd, Yd) _i,, have suc as formula relation shown in 20 then for a Pi.

$\left[\begin{array}{c}\mathrm{Xd}-\mathrm{Xg}\\ \mathrm{Yd}-\mathrm{Yg}\end{array}\right]={\left[\begin{array}{ccc}1& 0\mathrm{Xg}& \mathrm{Yg}\\ 0& 1\mathrm{Yg}& \mathrm{Xg}\end{array}\right]}_i\left[\begin{array}{c}\mathrm{\&Delta;X}\\ \mathrm{\&Delta;Y}\\ m\\ \mathrm{\&theta;}\end{array}\right]---\left(20\right)$

Take advantage of principle through least square, obtain conversion parameter (Δ X, Δ Y, m, θ).With the point coordinate under the WGS 1 of needs conversions (x, y) for people's formula (21), utilize four parameters can get the planimetric rectangular coordinates of tested point (Xt, Yt).

$\left[\begin{array}{c}\mathrm{Xt}\\ \mathrm{Yt}\end{array}\right]=\left[\begin{array}{c}\mathrm{\&Delta;X}\\ \mathrm{\&Delta;Y}\end{array}\right]+(1+m)\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}0\mathrm{\&theta;}\\ \mathrm{\&theta;}0\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]---\left(21\right)$

Through conversion, can the coordinate conversion under the WGS 1 be become the map reference of use, thereby realize the conversion of longitude and latitude to map reference.

● grid image generates

Invention is drawn the Tai Lake map with the SVG format-pattern; Corresponding relation through longitude and latitude and map reference is divided into different grid with map; Local blue-green algae real-time distribution situation map is filled in the corresponding map grid, generates the blue-green algae population distribution figure in waters.With respect to rectangular node, hexagonal mesh more approaches circle, meets the video camera imaging visual angle more, helps image block cutting and splicing, and therefore, it is shown in figure 14 to adopt hexagonal mesh to divide the territory, lake.

● merge the lightweight WebGIS platform of MapTile technology and vector data technology

In MapTile Technology in Web GIS platform, be processed into image pyramid to the grid map of map earlier and carry out stripping and slicing, then picture is carried out quadtree coding; Leave on the server; When client is carried out operation such as convergent-divergent, roaming, only need the URL of dynamic calculation map tile to conduct interviews and show and get final product, the volume of these tiles is all very little; When network condition is reasonable, browse very smooth.Because picture all is that prior generation is placed on server end well, so server end needs the memory device of huge memory capacity.All properties key element on the map all concentrates on the tile simultaneously, the very difficulty so effects such as realization Gao Liang just seem.

Lightweight WebGIS platform based on the vector data technology mainly adopts the exhibition method of SVG (Scalable Vector Graphics) as map.The map vector data that platform uses just leaves server end in; The disposable client that sends to during client-requested; Utilize the interactivity of SVG image and script JavaScript to accomplish basic operations such as convergent-divergent, roaming, inquiry in client, do not need and server interaction.

In conjunction with characteristics of the present invention, with the technological WebGIS platform issue Tai Lake blue-green algae real-time distribution information that combines with the vector data technology and make up lightweight of MapTile.Platform utilizes vector data technology SVG issue Tai Lake map, and uses the MapTile technology that local blue-green algae distribution situation figure is filled in the SVG map of having cut apart, and the platform structure synoptic diagram is shown in figure 15.

Server end can be divided into map publisher server and grid image publisher server by function, and the two can be realized on same machine in the present invention.When client is used the Web browser access server; Its SVG map with Tai Lake of map issuing service sends to client and shows; After this, the basic operations such as splicing of the convergent-divergent of map, roaming, inquiry grid image are then accomplished in client by script JavaScript.The grid image server then is responsible for providing the local blue-green algae distribution plan of zones of different, sends to client, is spliced by client script.

Client is realized by the Ajax technology with communicating by letter of grid image server; Ajax (Asynchronous Javascript and XML) is the comprehensive of multiple technologies; Also can be described as a kind of design, it combines technology such as comprising JavaScript, XHTML and CSS, DOM, XML, XSTL, XMLHttpRequest.Various technology act on shown in figure 16 in Ajax.

Whole communication process is asynchronous carrying out, and communication process is shown in figure 17, the mode that asynchronous Ajax of referring to and server are got in touch.If the use traditional mode, whenever the user when server sends the local blue-green algae image distribution of longitude and latitude request image block, Web browser will refresh current window.Figure X has showed the asynchronous mode that Ajax adopted, and browser need not wait for that the user operates, and also need not refresh the image block that whole window just can obtain needs.As long as transmit the data that adopt the encapsulation of XML form back and forth, operation JavaScript code just can be got in touch with server.When execution result arrives client, just can notify browser image block to be filled in the corresponding grid in the suitable time.

Claims (7)

1. based on the water surface general view formation method of the image that cruises; It is characterized in that at first setting up imaging system; Comprise server images disposal system, client end interface demonstration and interactive system and the ships and light boats video acquisition system that cruises; The ships and light boats video acquisition system that cruises comprises video acquisition device and GPS locating device, and the server images disposal system is handled video data and obtained water surface general view picture, and the input client end interface shows and interactive system; Wherein: the server images disposal system comprises video stabilization module, camera calibration module, image transformation module and image mosaic module; Client end interface shows and interactive system comprises that map filling and display module and grid map make up module, may further comprise the steps:

1), video camera and GPS locating device are set on ships and light boats; Video camera record patrol ships and light boats are travelled the water surface scene in the ken in the route; The output video frame sequence; The GPS locating device obtains the GPS information of ships and light boats in cruising in real time, and GPS information is corresponding one by one through the time with sequence of frames of video;

2) sequence of frames of video is imported the server images disposal system; The video stabilization module is carried out pre-service through overall motion estimation, motion compensation and image repair technology to sequence of frames of video; Because the float that causes of jolting of ships and light boats is removed, obtain stable video sequence image;

3) image information obtained from video camera of camera calibration module is calculated the geological information the Between Air and Water coordinate system, accomplishes camera calibration, and the information that obtains is used to rebuild water surface general view image information;

4) image transformation module is utilized the calibration information of camera calibration module; Comprise video camera inside and outside parameter and camera attitude; Said camera attitude comprises the angle of pitch, the visual angle of sequence of frames of video is converted into from side-looking overlook, and utilizes image reconstruction technique to realize that hole-filling, image sharpening to overhead view image handle again; Obtain the clear general view behind the view transformation, wherein image reconstruction adopts based on template or based on the algorithm of grid;

5) the image mosaic module sequence of frames of video vertical view that one width of cloth width of cloth is isolated pieces together, and obtains a big figure, and splicing is main according to handling the LIP model with Image Feature Point Matching and logarithmic image, and the geographic coordinate of GPS information is auxilliary;

6) grid map makes up module and according to the corresponding relation of latitude and longitude information and map reference the map of the water surface is pressed mesh segmentation; Generation is based on the electronics grid map of GIS-Geographic Information System GIS; Wherein use scalable vector graphics SVG as format map; Gps coordinate converts water surface map reference into; Corresponding relation through GPS longitude and latitude and water surface map reference is divided into different grid with map, and general view is filled in the corresponding water surface map grid according to gps coordinate, and the overall general view that generates the waters is issued on WEB.

2. the water surface general view formation method based on the image that cruises according to claim 1 is characterized in that step 2) in, the treatment scheme of video stabilization module is following:

21) overall motion estimation: adopt the BMA or the parameter model estimation technique;

22) image motion compensation: at first moving unintentionally and intentionally of sequence of frames of video carried out parameter estimation; Adopt Kalman's state filtering method to have a mind to the kinematic parameter estimation; Utilize statistical method structural physical state-space model, describe the dynamic change of having a mind to and being not intended to kinematic parameter, and estimate kinematic parameter intentionally based on Kalman filter; Compensate through image transformation then and be not intended to motion; The motion that makes image sequence with estimate to have a mind to motion model consistent, thereby reach the purpose of video stabilization, suppose n two field picture f _nTransformation parameter is (T _n, d _n), then transformation model is following:

${\stackrel{\&OverBar;}{p}}_n={\stackrel{\&OverBar;}{T}}_n{p}_n+{\stackrel{\&OverBar;}{d}}_n---\left(1\right)$

Wherein, p _nBe the vector of the point coordinate after the image transformation, state equation and observation equation according to Kalman filtering obtain:

${\stackrel{\&OverBar;}{p}}_n={\hat{T}}_n{p}_n+{\hat{d}}_n=\left({\hat{T}}_n{\left({\tilde{T}}_n\right)}^{-1}\right)p_n-{\left({\hat{T}}_n({\tilde{T}}_n\right)}^{-1}{\tilde{d}}_n+{\hat{d}}_n---\left(2\right)$

Therefore, n two field picture f _nConversion process can be described below:

${f^{\&prime;}}_n\left(p\right)=f_n({\left({\stackrel{\&OverBar;}{T}}_n\right)}^{-1}{p}_n-{\left({\stackrel{\&OverBar;}{T}}_n\right)}^{-1}{\stackrel{\&OverBar;}{d}}_n)---\left(3\right);$

23) image repair: from view data, obtain iconic model through filtering, parameter or non-parametric estmation and entropy method; Employing is based on the digital picture repairing technique of PDE; The marginal information of repairing area is treated in utilization; Adopt the direction of estimating isophote by thick method simultaneously, the information around the repairing area of treating is propagated in the repairing area, to realize image repair to essence.

3. the water surface general view formation method based on the image that cruises according to claim 1; It is characterized in that video camera is a pinhole camera; On traditional camera marking method basis, inside and outside ginseng is demarcated respectively during the step 3) camera calibration; As the space object of reference, the three-dimensional coordinate that utilizes one group of non degenerate point on the calibrating block and corresponding image coordinate are asked for the inner parameter and the external parameter of camera model through a series of mathematic(al) manipulations and computing method with the known calibrating block of structure; Confidential reference items are demarcated under the laboratory experiment environment and are accomplished, and adopt traditional method based on calibrating template; Outer ginseng is carried out on-line proving, realizes through the fixed character object of reference that extracts in the scene.

4. the water surface general view formation method based on the image that cruises according to claim 1 is characterized in that in the step 4), and the image transformation module realizes the visual angle conversion of frame of video through the reverse mapping techniques of image; The mapping relations matrix of foundation from the purpose territory to reference field, the purpose territory is a general view, reference field is a video frame images; The information that obtains with camera calibration is rebuild the depth information in purpose territory in advance, according to certain any coordinate in the general view of required structure (X, Y); Through the mapping relations matrix computations obtain this point the picture planimetric coordinates (x, y), again with (x; Y) locate the pixel value assignment in (X Y), converts the visual angle of sequence of frames of video into and to overlook from side-looking; Image reconstruction adopts the algorithm based on template.

5. the water surface general view formation method based on the image that cruises according to claim 1 is characterized in that the treatment scheme of the image mosaic module of step 5) comprises image pre-service, image registration, image co-registration, wherein:

The image pre-service is carried out pre-service to reference picture and image to be spliced, and reference picture is got first two field picture of video sequence, pre-service comprise Flame Image Process basic operation, set up the matching template of image and image carried out conversion; Wherein with reference picture as matching template, can do some shearings according to actual conditions, image is carried out conversion in order to extract characteristics of image; Comprise Fourier transform, wavelet transformation, Gabor conversion, extract the characteristic set of image afterwards, utilize the rough position relation of feature calculation reference picture and image to be spliced; Promptly treat stitching image and carry out coarse localization; Find overlapping region roughly, dwindle matching range, raising speed;

Image registration adopts improved Normalized Grey Level level correlation technique NGC to combine the GPS locator data to carry out image registration; Improved Normalized Grey Level level correlation technique NGC is: the gray level that the gray level of pixel in the matching template of image multiply by the respective pixel that is covered by matching template in the image to be spliced; The summation NC value that obtains is stored in the two-dimensional array, and said matching template and image to be spliced are the general view that has transformed through step 4):

$\mathrm{NC}(i,j)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)---\left(10\right)$

Wherein, T is a matching template, comprises M * N pixel, S ^{I, j}For move matching template starting point to image to be spliced (i, j) image is called subgraph by the template region covered during position, the position of matching template in image to be spliced is by the subgraph decision the most similar with matching template, with matching template T and subgraph S ^{I, j}Regard two vectors as, utilize the cosine formula (11) of vector angle to find the solution S ^{I, j}With the angle theta of T, make the S that θ is minimum ^{I, j}The position at place is the position of template in image:

$\cos {\mathrm{\&theta;}}_{i,j}=\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{S}^{i,j}(m,n)T(m,n)}{\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[S(m,n)\right]}^2}\sqrt{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[T(m,n)\right]}^2}}---\left(11\right);$

Image co-registration adopts log-domain Flame Image Process model LIP, and image transitions to be spliced to LIP territory is handled, and realizes figure image intensifying, gamma correction.

6. the water surface birds-eye view formation method based on the image that cruises according to claim 5; It is characterized in that in the image amalgamation; The brightness value of the pixel that is positioned at

and locates is considered as a three-dimensional vector; Be referred to as color vectors; The expression with

$\stackrel{\&RightArrow;}{F}\left(\stackrel{\&RightArrow;}{v}\right)=\left(\begin{array}{c}{F}_r\left(\stackrel{\&RightArrow;}{v}\right)\\ F_g\left(\stackrel{\&RightArrow;}{v}\right)\\ F_b\left(\stackrel{\&RightArrow;}{v}\right)\end{array}\right)---\left(18\right)$

In calculating, quote the ⊕ and

algorithm of the definition of LIP territory; And mapping function ψ; Three passages of R, G, B of coloured image are handled respectively; On the LIP of gray level image model; Set up the logarithm of coloured image and handle the CLIP model, in the CLIP territory, handle Surface Picture, make image more press close to true visual signature.

7. the water surface general view formation method based on the image that cruises according to claim 1 when it is characterized in that in the step 6) that map with the water surface is by mesh segmentation, is divided into hexagonal mesh map of uniform size.

Images