CN106296584A - A kind of method that panoramic mosaic figure shows with local video or image co-registration - Google Patents

Abstract

The present invention relates to a kind of method that panoramic mosaic figure shows with local video or image co-registration.The method is by PTZ camera current video or shooting angle (horizontal angle Pan of certain sectional drawing, angle of pitch Tilt), and coverage (angle of view, zoom Zoom), calculate the image-region particular location at panoramic mosaic model, then the content updated in panoramic picture in respective regions, when shooting angle and scope change, the respective regions in change panoramic picture, and the restore-zone that original video covers returns panoramic mosaic figure.The method improves the man-machine experience that solution panoramic mosaic figure shows with local video or image co-registration.

Description

A kind of method that panoramic mosaic figure shows with local video or image co-registration

Technical field

The present invention relates to a kind of method that panoramic mosaic figure shows with local video or image co-registration.

Background technology

In conventional CCTV camera, there is a kind of PTZ camera such as Fig. 1, although the coverage of camera lens is limited, but due to dress On rotatable The Cloud Terrace, it is possible to achieve level and the rotation of pitching, expand monitoring range.

Understanding to realize the summary to surrounding, splicing frequently appears in the application of this kind of PTZ camera, By being spliced by the picture that camera shoots in different angles, obtain the picture of a coverage the most even panorama. Such as Fig. 2.But the image of splicing is static, it is impossible to real-time update, thus need nonetheless remain for connecing a camera video synthesis and use.

Video image and panoramic mosaic image with the use of, need user switch, and abstract location fusion video exist The content of panoramic mosaic image.Still inconvenience is had in man-machine experience.

The split-join model being currently employed for PTZ camera includes cylinder model more, and Sphere Measurement Model etc., by characteristic matching, ball The methods such as coordinate transform, it is achieved PTZ camera sequence of pictures splicing and combining on model.Such as Fig. 3, pass through cylinder for ptz camera The annulus panorama sketch of model splicing.

Panoramic mosaic figure can directly show by 2D, it is also possible to is converted by 3D engine, becomes virtual PTZ and shows.As Fig. 4,3D engine the virtual PTZ image on 4 directions obtained.

In addition, also there is a kind of product adding full shot in PTZ device, directly generated panorama by attachment lens Figure.

Summary of the invention

In order to solve above-mentioned technical problem, it is an object of the invention to provide a kind of panoramic mosaic figure and local video or figure Method shown in merging, the method improves the man-machine experience solving this separate type.

In order to realize above-mentioned purpose, present invention employs following technical scheme:

A kind of method that panoramic mosaic figure shows with local video or image co-registration, the method comprises the following steps:

A) according to the algorithm of panoramic mosaic, panoramic mosaic pixel coordinate system R is obtained_panoWith split-join model coordinate system R_model Corresponding relation F_p-m；

B) PTZ camera video present frame or shooting angle Pan of certain sectional drawing and Tilt are obtained；

C) according to PTZ camera coverage and zoom, present frame or the horizontal direction shooting angle model of certain sectional drawing are obtained Enclose Range_panAnd the Range in vertical direction_tilt, and resolution (Wid_v,Hei_v)；

D) according to Pan, Tilt, Range_panAnd Range_tilt, obtain model coordinate systems R_modelSit with the pixel of PTZ camera Mark system R_ptCorresponding relation F_m-pt；

E) selected display mode, while display panoramic mosaic figure, according to corresponding relation F_p-mAnd F_m-ptCalculate panorama sketch In each position whether in the range of the current shooting of PTZ camera, if in the range of, then by content replace, content cover And/or content mergence method, the current video frame of display PTZ camera or sectional drawing；If outside scope, then show original spliced map Content.

As preferably, the present invention panoramic mosaic figure be 2D display panoramic mosaic figure or 3D shows panoramic mosaic figure.

As preferably, panoramic mosaic figure is 2D when showing panoramic mosaic figure,

R_panoFor cartesian coordinate system

R_modelFor normalization polar coordinate system

$F_{p-m}:\left\{\begin{array}{c}{P}_m=x_{pano}*\frac{P_{max}-P_{min}}{{\mathrm{Wid}}_{pano}}+P_{\min }\\ T_m=y_{pano}*\frac{T_{\max }-T_{min}}{{\mathrm{Hei}}_{pano}}+T_{min}\end{array}\right..$

As improving further, when panoramic mosaic figure is 2D display panoramic mosaic figure, in step d),

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*\tan \left(\mathrm{\Δ}Pan\right)+Wi{d}_v*0.5\\ y_{pt}=Dis*\tan \left(\mathrm{\Δ}Tilt\right)+He{i}_v*0.5\end{array}\right.,$

Wherein:

Dis=Wid_v*0.5/tan(R_pan* 0.5) or

Dis=Hei_v*0.5/tan(R_tilt* 0.5),

$\left\{\begin{array}{c}\mathrm{\Δ}Pan=P_m-P_v\\ \mathrm{\Δ}Tilt=T_m-T_v\end{array}\right..$

As preferably, panoramic mosaic figure is 3D when showing panoramic mosaic figure,

R_panoFor cartesian coordinate system

R_modelFor cartesian coordinate system

$\left\{\begin{array}{c}{P}_m=\left\{\begin{array}{c}\arccos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m>0\\ 180+\arccos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m<0\end{array}\right.\\ T_m=\arcsin \left(\frac{z_m}{\left|\right|\begin{array}{ccc}{x}_m& y_m& z_m\end{array}\left|\right|}\right)\\ 1\end{array}\right.,$

$F_{m-p}:\left\{\begin{array}{c}{x}_{pano}=(P_m-P_{min})*Wi{d}_{pano}/(P_{max}-P_{min})\\ y_{pano}=(T_m-T_{min})*He{i}_{pano}/(T_{max}-T_{\min })\end{array}\right.;$

As improving further, when panoramic mosaic figure is 3D display panoramic mosaic figure, in step d),

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]*\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]=\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]={\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]}^{-1}*\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*y_d/\left|\right|\begin{array}{cc}{x}_d& y_d\left|\right|+{\mathrm{Wid}}_v*0.5\end{array}\\ y_{pt}=Dis*z_d/\left|\right|\begin{array}{cc}\begin{array}{ccc}{x}_d& y_d& z_d\end{array}& \left|\right|+\end{array}{\mathrm{Hei}}_v*0.5\end{array}\right.,$

Wherein:

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]=\left[\begin{array}{ccc}\cos \left(T_v\right)& \sin \left(T_v\right)& 0\\ -\sin \left(T_v\right)& \cos \left(T_v\right)& 0\\ 0& 0& 1\end{array}\right]*\left[\begin{array}{ccc}\cos \left(P_v\right)& 0& -\sin \left(P_v\right)\\ 0& 1& 0\\ \sin \left(P_v\right)& 0& \cos \left(P_v\right)\end{array}\right].$

Due to the fact that and have employed above-mentioned technical scheme, by PTZ camera current video or the shooting angle of certain sectional drawing Degree (horizontal angle Pan, angle of pitch Tilt), and coverage (angle of view, zoom Zoom), calculate image-region at panorama The particular location of split-join model, the content then updated in panoramic picture in respective regions, when shooting angle and scope change, Respective regions in change panoramic picture, and the restore-zone that original video covers returns panoramic mosaic figure.The method improves solution The man-machine experience that panoramic mosaic figure and local video or image co-registration show.

Accompanying drawing explanation

Fig. 1 is Pan/Tilt/Zoom camera image.

Fig. 2 is Pan/Tilt/Zoom camera stitching image.

Fig. 3 is annulus panorama sketch.

Fig. 4 is virtual PTZ technology figure under multi-angle.

Fig. 5 is cartesian coordinate system video-projection figure.

Fig. 6 is annulus splicing figure and head-shaking machine video frame content figure.

Fig. 7 is frame of video in-scope figure.

Fig. 8 is 2D display panoramic mosaic and frame of video fusion results figure.

Fig. 9 is 3D display panoramic mosaic and frame of video fusion results figure.

Detailed description of the invention

Below in conjunction with the accompanying drawings the detailed description of the invention of the present invention is made a detailed explanation.

The present invention to relate to parameter interpretation as follows:

x_pano: panoramic mosaic pixel coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_pano

y_pano: panoramic mosaic pixel coordinate figure Y-component, the upper left corner is 0, and the lower right corner is Hei_pano

Wid_pano: panoramic mosaic figure width

Hei_pano: panoramic mosaic figure is high

P_m: the horizontal angle component of polar coordinate system split-join model

T_m: the angle of pitch component of polar coordinate system split-join model

P_v: the horizontal angle component of head-shaking machine The Cloud Terrace

T_v: the angle of pitch component of head-shaking machine The Cloud Terrace

R_pan: the field of view angle of video image horizontal direction

R_tilt: the field of view angle of video image vertical direction

Wid_v: video image width

Hei_v: video image water is high

x_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_v

y_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Hei_v

x_m: during 3D display, the X-component of the normalization direction vector of cartesian coordinate system split-join model

y_m: during 3D display, the Y-component of the normalization direction vector of cartesian coordinate system split-join model

z_m: during 3D display, the Z component of the normalization direction vector of cartesian coordinate system split-join model

x_m: during 3D display, the X-component of the normalization direction vector of cartesian coordinate system video-projection

y_m: during 3D display, the Y-component of the normalization direction vector of cartesian coordinate system video-projection

z_m: during 3D display, the Z component of the normalization direction vector of cartesian coordinate system video-projection

Cartesian coordinate system video-projection Fig. 5, camera lens optical axis is X-axis.

Embodiment 1 2D display panoramic mosaic and the fusion of head-shaking machine video

Head-shaking machine only has two degree of freedom of Pan, Tilt, does not has Zoom

A) cylindrical panoramic annulus and split-join model coordinate system transformational relation F are obtained_p-m

R_panoFor cartesian coordinate system

R_modelFor normalization polar coordinate system $\left\{\begin{array}{c}{P}_m|\&Element;(P_{\min },P_{\max })\\ T_m|\&Element;(T_{\min },T_{\max })\\ 1\end{array}\right.,$

$F_{p-m}:\left\{\begin{array}{c}{P}_m=x_{pano}*\frac{P_{max}-P_{min}}{{\mathrm{Wid}}_{pano}}+P_{\min }\\ T_m=y_{pano}*\frac{T_{\max }-T_{min}}{{\mathrm{Hei}}_{pano}}+T_{min}\end{array}\right.;$

B) by private information or agreement, the shooting angle of head-shaking machine video present frame is obtained

C) head-shaking machine coverage is obtainedAnd video resolutionWherein it must is fulfilled for

Hei_v=Wid_v*tan(R_tilt*0.5)/tan(R_pan* 0.5) or

R_tilt=atctan (Hei_v/Wid_v*tan(R_pan*0.5))*2；

D) model coordinate systems R is obtained_modelWith head-shaking machine video pixel coordinates system R_ptCorresponding relation F_m-pt,

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*\tan \left(\mathrm{\&Delta;}Pan\right)+Wi{d}_v*0.5\\ y_{pt}=Dis*\tan \left(\mathrm{\&Delta;}Tilt\right)+He{i}_v*0.5\end{array}\right.;$

Wherein:

Dis=Wid_v*0.5/tan(R_pan* 0.5) or

Dis=Hei_v*0.5/tan(R_tilt*0.5)；

$\left\{\begin{array}{c}\mathrm{\&Delta;}Pan=P_m-P_v\\ \mathrm{\&Delta;}Tilt=T_m-T_v\end{array}\right.;$

If x_pt∈(0,Wid_v) and y_pt∈(0,Hei_v), then in panoramic mosaic figure, this location conten is fused and replaces to The response contents of video present frame.Below figure: Fig. 6 shows panoramic mosaic figure and head-shaking machine current video frame；Fig. 7 shows and calculates Head-shaking machine current video position；Fig. 8 is for merging final result.

Embodiment 2 3D display panoramic mosaic and the fusion of head-shaking machine video

Head-shaking machine only has two degree of freedom of Pan, Tilt, does not has Zoom.And 3D shows, having model coordinate systems is original coordinates System, distinguishes sphere spliced map position corresponding to computation model diverse location and head-shaking machine video location.

A) the coordinate system transformational relation F of sphere split-join model and spliced map is obtained_m-p

R_panoFor cartesian coordinate system

R_modelFor cartesian coordinate system

$\left\{\begin{array}{c}{P}_m=\left\{\begin{array}{c}\arccos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m>0\\ 180+\arccos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m<0\end{array}\right.\\ T_m=\arcsin \left(\frac{z_m}{\left|\right|\begin{array}{ccc}{x}_m& y_m& z_m\end{array}\left|\right|}\right)\\ 1\end{array}\right.,$

$F_{m-p}:\left\{\begin{array}{c}{x}_{pano}=(P_m-P_{min})*Wi{d}_{pano}/(P_{max}-P_{min})\\ y_{pano}=(T_m-T_{min})*He{i}_{pano}/(T_{max}-T_{\min })\end{array}\right.;$

B) by private information or agreement, the shooting angle of head-shaking machine video present frame is obtained

C) head-shaking machine coverage is obtainedAnd video resolutionWherein it must is fulfilled for

Hei_v=Wid_v*tan(R_tilt*0.5)/tan(R_pan0.5) or

R_tilt=atctan (Hei_v/Wid_v*tan(R_pan*0.5))*2；

D) model coordinate systems R is obtained_modelWith head-shaking machine video pixel coordinates system R_ptCorresponding relation F_m-pt

$\left[\begin{array}{c}{x}_u\\ y_u\\ z_u\end{array}\right]=\left[\begin{array}{c}cos\left(P_v\right)cos\left(T_v\right)\\ sin\left(P_v\right)cos\left(T_v\right)\\ \sin \left(T_v\right)\end{array}\right],$

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]*\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]=\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]={\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]}^{-1}*\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*y_d/\left|\right|\begin{array}{cc}{x}_d& y_d\left|\right|+{\mathrm{Wid}}_v*0.5\end{array}\\ y_{pt}=Dis*z_d/\left|\right|\begin{array}{cc}\begin{array}{ccc}{x}_d& y_d& z_d\end{array}& \left|\right|+\end{array}{\mathrm{Hei}}_v*0.5\end{array}\right.,$

Wherein:

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]=\left[\begin{array}{ccc}\cos \left(T_v\right)& \sin \left(T_v\right)& 0\\ -\sin \left(T_v\right)& \cos \left(T_v\right)& 0\\ 0& 0& 1\end{array}\right]*\left[\begin{array}{ccc}\cos \left(P_v\right)& 0& \sin \left(P_v\right)\\ 0& 1& 0\\ \sin \left(P_v\right)& 0& \cos \left(P_v\right)\end{array}\right].$

If x_pt∈(0,Wid_v) and y_pt∈(0,Hei_v), then in panoramic mosaic figure, this location conten is fused and replaces to The response contents of video present frame.If Fig. 9 is for merging final result.

Claims (6)

1. the method that a panoramic mosaic figure shows with local video or image co-registration, it is characterised in that the method includes following Step:

A) according to the algorithm of panoramic mosaic, panoramic mosaic pixel coordinate system R is obtained_panoWith split-join model coordinate system R_modelRight F should be related to_p-m；

B) PTZ camera video present frame or shooting angle Pan of certain sectional drawing and Tilt are obtained；

C) according to PTZ camera coverage and zoom, present frame or the horizontal direction shooting angle scope of certain sectional drawing are obtained Range_panAnd the Range in vertical direction_tilt, and resolution (Wid_v,Hei_v)；

D) according to Pan, Tilt, Range_panAnd Range_tilt, obtain model coordinate systems R_modelPixel coordinate system with PTZ camera R_ptCorresponding relation F_m-pt；

E) selected display mode, while display panoramic mosaic figure, according to corresponding relation F_p-mAnd F_m-ptCalculate in panorama sketch each Whether individual position in the range of the current shooting of PTZ camera, if in the range of, then by content replace, content cover and/or Content mergence method, the current video frame of display PTZ camera or sectional drawing；If outside scope, then show the interior of original spliced map Hold.

The method that a kind of panoramic mosaic figure the most according to claim 1 shows with local video or image co-registration, its feature It is that panoramic mosaic figure is 2D display panoramic mosaic figure or 3D display panoramic mosaic figure.

The method that a kind of panoramic mosaic figure the most according to claim 1 shows with local video or image co-registration, its feature It is when panoramic mosaic figure is 2D display panoramic mosaic figure,

R_panoFor cartesian coordinate system

R_modelFor normalization polar coordinate system

$F_{p-m}:\left\{\begin{array}{c}{P}_m=x_{pano}*\frac{P_{max}-P_{min}}{{\mathrm{Wid}}_{pano}}+P_{\min }\\ T_m=y_{pano}*\frac{T_{\max }-T_{\min }}{{\mathrm{Hei}}_{pano}}+T_{\min }\end{array}\right.;$

Wherein:

x_pano: panoramic mosaic pixel coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_pano；

y_pano: panoramic mosaic pixel coordinate figure Y-component, the upper left corner is 0, and the lower right corner is Hei_pano；

P_m: the horizontal angle component of polar coordinate system split-join model；

T_m: the angle of pitch component of polar coordinate system split-join model.

The method that a kind of panoramic mosaic figure the most according to claim 3 shows with local video or image co-registration, its feature It is in step d),

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*tan\left(\mathrm{\&Delta;}Pan\right)+{\mathrm{Wid}}_v*0.5\\ y_{pt}=Dis*tan\left(\mathrm{\&Delta;}Tilt\right)+{\mathrm{Hei}}_v*0.5\end{array}\right.,$

Dis=Wid_v*0.5/tan(R_pan* 0.5) or

Dis=Hei_v*0.5/tan(R_tilt* 0.5),

$\left\{\begin{array}{c}\mathrm{\&Delta;}Pan=P_m-P_v\\ \mathrm{\&Delta;}Tilt=T_m-T_v\end{array}\right.;$

Wherein:

x_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_v

y_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Hei_v

R_pan: the field of view angle of video image horizontal direction

R_tilt: the field of view angle of video image vertical direction

Wid_v: video image width

Hei_v: video image is high

P_m: the horizontal angle component of polar coordinate system split-join model

T_m: the angle of pitch component of polar coordinate system split-join model

P_v: the horizontal angle component of The Cloud Terrace

T_v: the angle of pitch component of The Cloud Terrace.

The method that a kind of panoramic mosaic figure the most according to claim 1 shows with local video or image co-registration, its feature It is when panoramic mosaic figure is 3D display panoramic mosaic figure,

R_panoFor cartesian coordinate system

R_modelFor cartesian coordinate system

$\left\{\begin{array}{c}{P}_m=\left\{\begin{array}{c}arc\cos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m>0\\ 180+arc\cos \left(\frac{x_m}{\left|\right|\begin{array}{cc}{x}_{dm}& y_{dm}\end{array}\left|\right|}\right):y_m<0\end{array}\right.\\ T_m=arc\sin \left(\frac{z_m}{\left|\right|\begin{array}{ccc}{x}_m& y_m& z_m\end{array}\left|\right|}\right)\\ 1\end{array}\right.,$

$F_{m-p}:\left\{\begin{array}{c}{x}_{pano}=(P_m-P_{min})*{\mathrm{Wid}}_{pano}/(P_{max}-P_{min})\\ y_{pano}=(T_m-T_{min})*{\mathrm{Hei}}_{pano}/(T_{max}-T_{min})\end{array}\right.;$

Wherein:

x_pano: panoramic mosaic pixel coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_pano

y_pano: panoramic mosaic pixel coordinate figure Y-component, the upper left corner is 0, and the lower right corner is Hei_pano

x_m: during 3D display, the X-component of the normalization direction vector of cartesian coordinate system split-join model

y_m: during 3D display, the Y-component of the normalization direction vector of cartesian coordinate system split-join model

z_m: during 3D display, the Z component of the normalization direction vector of cartesian coordinate system split-join model

P_m: the horizontal angle component of polar coordinate system split-join model

T_m: the angle of pitch component of polar coordinate system split-join model

Wid_pano: panoramic mosaic figure width

Hei_pano: panoramic mosaic figure is high.

The method that a kind of panoramic mosaic figure the most according to claim 5 shows with local video or image co-registration, its feature It is in step d),

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]*\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]=\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$\left[\begin{array}{c}{x}_d\\ y_d\\ z_d\end{array}\right]={\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]}^{-1}*\left[\begin{array}{c}{x}_m\\ y_m\\ z_m\end{array}\right],$

$F_{m-pt}:\left\{\begin{array}{c}{x}_{pt}=Dis*y_d/\left|\right|\begin{array}{cc}{x}_d& y_d\end{array}\left|\right|+{\mathrm{Wid}}_v*0.5\\ y_{pt}=Dis*z_d/\left|\right|\begin{array}{ccc}{x}_d& y_d& z_d\end{array}\left|\right|+{\mathrm{Hei}}_v*0.5\end{array}\right.,$

Wherein:

$\left[\begin{array}{ccc}{m}_{00}& m_{01}& m_{02}\\ m_{10}& m_{11}& m_{12}\\ m_{20}& m_{21}& m_{22}\end{array}\right]=\left[\begin{array}{ccc}cos\left(T_v\right)& sin\left(T_v\right)& 0\\ -\sin \left(T_v\right)& cos\left(T_v\right)& 0\\ 0& 0& 1\end{array}\right]*\left[\begin{array}{ccc}cos\left(P_v\right)& 0& -sin\left(P_v\right)\\ 0& 1& 0\\ sin\left(P_v\right)& 0& \cos \left(P_v\right)\end{array}\right];$

Wherein:

Wid_v: video image width

Hei_v: video image water is high

x_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Wid_v

y_pt: video image element coordinate figure X-component, the upper left corner is 0, and the lower right corner is Hei_v。