CN105335977B - The localization method of camera system and target object - Google Patents
The localization method of camera system and target object Download PDFInfo
- Publication number
- CN105335977B CN105335977B CN201510711384.2A CN201510711384A CN105335977B CN 105335977 B CN105335977 B CN 105335977B CN 201510711384 A CN201510711384 A CN 201510711384A CN 105335977 B CN105335977 B CN 105335977B
- Authority
- CN
- China
- Prior art keywords
- image
- target
- layer
- video camera
- subgraph
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention provides a kind of localization method of target object, and for camera chain, the camera chain includes：First video camera, for obtaining the first image；And second video camera, for obtaining the second image；The localization method includes：A. the described first image and the second image for including target object are obtained；B. according to the first image acquisition first object image, first object image includes target object；C. according to the status information of the targeted message and the second video camera of the first video camera and the second video camera, initial homography matrix is calculated；D. according to initial homography matrix, the second image is mapped in the first image, obtains the second target image；E. light stream matching is carried out to first object image and the second target image, calculates Optic flow information；F. calculated according to Optic flow information and correct homography matrix；G. according to homography matrix is corrected, first object image is mapped in the second image, obtains and corrects the second target image, to position target object in the second image.
Description
Technical field
The present invention relates to Computer Applied Technology field more particularly to the positioning sides of a kind of camera system and target object
Method.
Background technology
Pan/Tilt/Zoom camera (Pan-Tilt-Zoom), abbreviation ball-shaped camera are integrated with clouds terrace system and camera chain.
Camera chain can carry out the stretching in the visual field, and holder can make camera chain horizontally rotate and vertically rotate.
Therefore, Pan/Tilt/Zoom camera can be realized to the target in monitoring scene into line trace and amplification, played in monitoring system important
Effect.
In rifle ball linked system, wide-angle gun shaped video camera carries out background modeling to monitoring area, detects moving target, so
Controlling ball-shaped camera afterwards, control here includes the P (horizontally rotating) of ball-shaped camera, T is controlled (to tilt and turn into line trace
It is dynamic), Zoom (scaling) and speed during ball-shaped camera rotation etc..So first with the big visual field pair of wide-angle gun shaped video camera
Large scene carry out target detection, recycle P, T, Zoom of ball-shaped camera and the velocity of rotation of ball-shaped camera to target into
Line trace and scaling have reached and have not only monitored the big visual field, but also do not omit the purpose of Small object details.
The image of ball-shaped camera is widescreen, and target object is not mutually to be fitted with the shape of the image of ball-shaped camera
Should, such as some target objects are tall and thin pedestrian, if view picture ball-shaped camera image is captured, are exported to attributive analysis
Module had not only stored substantial amounts of invalid information around target, but also has influenced attributive analysis result.If only preserve the centre of ball machine image
Part and unreasonable.In the application of actual rifle ball linked system, ball machine is moved under gunlock control, to target
Into line trace, at the same time target is also ceaselessly moving.Therefore, target is likely to appear in the difference of ball-shaped camera image
Position.
The content of the invention
The present invention provides the positioning of a kind of camera system and target object to overcome the problems of the above-mentioned prior art
Method can quickly and effectively position target object, and obtain the target image available for image procossing.
The present invention provides a kind of localization method of target object, and for camera chain, the camera chain includes：The
One video camera, for obtaining the first image, described first image is the wide angle picture of a scene ken；And second video camera,
For obtaining the second image, second image is the partial enlarged view of the scene ken；The localization method includes：A. obtain
Take the described first image and the second image for including target object；B. first object image, institute are obtained according to described first image
It states first object image and includes the target object；C. according to first video camera and the targeted message of second video camera
And the status information of second video camera, calculate initial homography matrix；D. according to the initial homography matrix, by described second
Image is mapped in described first image, obtains the second target image；E. to the first object image and second target
Image carries out light stream matching, calculates Optic flow information；F. calculated according to the Optic flow information and correct homography matrix；G. repaiied according to
The first object image is mapped in second image by positive homography matrix, is obtained and is corrected the second target image, in institute
It states and the target object is positioned in the second image.
Preferably, the step b includes：In described first image, centered on the center of the target object, interception
Rectangular target image is as the first object image.
Preferably, the rectangular target image intercepted is the square target image of 96*96 pixel.
Preferably, the initial homography matrix is transformed into the homography matrix of described first image for second image.
Preferably, the step c includes：C1. the targeted message is obtained, the targeted message includes the described first camera shooting
The pixel coordinate of first image of machine is transformed into the second homography matrix of the physical coordinates of second video camera；C2. according to institute
The status information of the pixel coordinate and second video camera of stating the second image of the second video camera calculates corresponding described second and takes the photograph
The physical coordinates of second video camera of the pixel coordinate of second image of camera；C3. according to second homography matrix
The physical coordinates of inverse matrix and second video camera calculate the pixel coordinate of the second image corresponding to second video camera
First video camera the first image pixel coordinate；And c4. is according to the picture of the first image of first video camera
The pixel coordinate of second image of plain coordinate and second video camera calculates the initial homography matrix.
Preferably, the step c2 includes：The picture of at least four not conllinear pixels is chosen according to second image
Pixel coordinate of the plain coordinate as second video camera.
Preferably, the step e includes：E1. the Gauss of the first object image and second target image is calculated
Pyramid；E2. the gradient information of the gaussian pyramid of second target image is successively calculated；E3. according to the gradient information
Light stream matching successively is carried out to the first object image and second target image, calculates Optic flow information.
Preferably, the step e1 includes：Using Gaussian kernel to the first object image and second target image
Carry out convolution operation；According to the first object image and second target image, the gaussian pyramid that height is 3 is established,
First object image collection A and the second target image set B are denoted as respectively, wherein, the first object image collection A includes big
The small first layer first object subgraph A being gradually reduced1, second layer first object subgraph A2, third layer first object subgraph
A3；The second target image set B includes first layer the second target subgraph B that size is gradually reduced1, the second mesh of the second layer
Mark subgraph B2, third layer the second target subgraph B3。
Preferably, the Gaussian kernel is [1,/16 1/4 3/8 1/4 1/16] x [1,/16 1/4 3/8 1/4 1/16]T。
Preferably, the first layer first object subgraph A1And first layer the second target subgraph B1For 96*96 pictures
The image of vegetarian refreshments；The second layer first object subgraph A2And the second layer the second target subgraph B2For 48*48 pixels
Image；The third layer first object subgraph A3And third layer the second target subgraph B3For the figure of 24*24 pixels
Picture.
Preferably, the step e2 includes：
The gradient information of the second target image set B is successively calculated according to equation below：
Wherein,Represent i-th layer of second target subgraph BiGradient information, gradxRepresent described i-th layer
Two target subgraph BiIn the gradient information of X-direction, gradyRepresent i-th layer of second target subgraph BiLadder in the Y direction
Information is spent, i takes 3,2,1 successively.
Preferably, the step e3 includes：
The Optic flow information is calculated according to equation below
Wherein, dxRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in X
The offset in direction, dyRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in X
The offset in direction,Represent i-th layer of first object subgraph Ai's and i-th layer of second target subgraph Bi
Optic flow information,
ΣNgxx、ΣNgyy、ΣNgxy、errxAnd erryIt is calculated respectively according to equation below：
ΣNGxx=ΣNgradx*gradx；
ΣNGyy=ΣNgrady*grady；
ΣNGxy=ΣNgradx*grady；
errx=ΣNDiff*gradx；
erry=ΣNDiff*grady；
Wherein, N represents the neighborhood of characteristic point P, and characteristic point P chooses in each layer of the first object image collection A,
Diff represents the gray scale difference value of pixel in the N of field.
Preferably, the field N be centered on characteristic point P, odd number of pixels point for the length of side square area.
Preferably, the Optic flow information of i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi according to
The Optic flow information of i+1 layer first object subgraph Ai+1 and i+1 layer the second target subgraph Bi+1 calculates.
Preferably, the step f includes：N number of pixel is chosen in the first object image as the first pixel；
Chosen in second target image respectively N number of pixel corresponding with first pixel as the second pixel；Profit
The pixel coordinate of N number of first pixel is corrected with the Optic flow information, obtains the first pixel of N number of amendment；Using described
The pixel coordinate for correcting the first pixel and second pixel calculates the amendment homography matrix.
Preferably, the first object image has first object frame, and the first object frame is the first object figure
The boundary rectangle of picture, the step g include：According to the amendment homography matrix, the first object frame is mapped to described
In two images, using the boundary rectangle of the mapping objects frame of acquisition as the second target frame, by the image in the second target frame
As the second target image of the amendment.
Preferably, the image identification and graphical analysis for correcting the second target image for the target object.
According to another aspect of the invention, a kind of camera system is also provided, including：First video camera, for obtaining first
Image, described first image are the wide angle picture of a scene ken；Second video camera, for obtaining the second image, described second
Image is the partial enlarged view of the scene ken；And positioner, using above-mentioned localization method, according to first figure
As second video camera is controlled to position the target object in second image.
Preferably, first video camera is gun shaped video camera, and second video camera is ball-shaped camera.
Compared with prior art, the present invention obtains wide angle picture and partial enlargement image by two kinds of video cameras, according to wide-angle
Image and the mapping of partial enlargement image and light stream matching, calculate the offset between image, and then mesh will be included in wide angle picture
The target image of mark object is mapped in partial enlargement image, to position the target object in partial enlargement image.The present invention is only
The target image that target object is included in partial enlargement image is used for the image procossing for target object.The target of the present invention
The target image that object positioning method is provided contains target object exactly, and it will not include a large amount of invalid information
To increase the time of image procossing and load.
Description of the drawings
Its example embodiment is described in detail by referring to accompanying drawing, above and other feature of the invention and advantage will become
It is more obvious.
Fig. 1 shows the schematic diagram of camera system according to embodiments of the present invention.
Fig. 2 shows the flow chart of the localization method of target object according to embodiments of the present invention.
Fig. 3 shows the first image according to embodiments of the present invention.
Fig. 4 shows the second image according to embodiments of the present invention.
Fig. 5 shows first object image according to embodiments of the present invention.
Fig. 6 shows the second target image according to embodiments of the present invention.
Fig. 7 shows the second target image of amendment according to embodiments of the present invention.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to embodiment set forth herein；On the contrary, these embodiments are provided so that the present invention will
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.It is identical attached in figure
Icon note represents same or similar structure, thus will omit repetition thereof.
The localization method of camera system provided by the invention and target object is described with reference to Fig. 1 to Fig. 7.
Camera system 100 is preferably twin camera linked system, including the first video camera 110, the second video camera 120
With positioner 130.First video camera 110 is used to obtain the first image 200 of a scene ken wide angle picture.Preferably,
One video camera 110 is for shooting the gun shaped video camera of wide angle picture.Second video camera 120 is for the second image 300 of acquisition.The
Two images 300 are the partial enlarged views of the scene ken of the first image 200.Second video camera 120 is ball-type camera shooting preferably
Machine.Positioner 130 controls the second video camera 120 with the by localization method provided by the invention, according to the first image 200
Target object 900 is positioned in two images 300.In one embodiment, positioner 130 can be integrated with the first video camera 110
Together.In another embodiment, positioner 130 can be integrated with the second video camera 120.In another implementation
In example, positioner 130 is an independent device, and passes through wired or wireless mode and the first video camera 110 and second
120 connecting communication of video camera.In other embodiments, positioner 130 can also in a distributed manner respectively with the first video camera 110
It integrates to perform different steps in the first video camera 110 and the second video camera 120 with the second video camera 120.
The localization method of target object provided by the invention flow chart shown in Figure 2.The localization method includes as follows
Step：
S210：Obtain 200 and second image 300 of described first image for including target object.This step is taken the photograph by first
110 and second video camera 120 of camera obtains the shooting of the Same Scene comprising target object 900.Second image 300 is first
The partial enlarged view of image 200.
S220：First object image 210 is obtained according to described first image 200, first object image 210 includes target pair
As 900.
Specifically, first object image 210 is the rectangle that is intercepted in the first image 200 centered on target's center
Target image.The rectangular target image is preferably the target image of a square.For example, first object image 210 can be
The square target image of 96*96 pixels.
In a specific embodiment, first object image 210 has first object frame 220.First object image 210
In in first object frame 220.In this specific embodiment, positioner 130 also obtain first object frame 220 location information and
Size information.Location information can be the pixel coordinate of first object frame central point or each vertex of first object frame
Pixel coordinate.Size information can be provided in units of pixel.
S230：According to the state of the targeted message and the second video camera 120 of the first video camera 110 and the second video camera 120
Information calculates initial homography matrix.
Initial homography matrix is transformed into the homography matrix of the first image 200 for the second image 300.Specifically initial list should
Matrix is calculated according to following manner.
First, the targeted message of the first video camera 110 and the second video camera 120 is obtained.The targeted message is taken the photograph including first
The pixel coordinate of first image 200 of camera 110 is transformed into the second homography matrix of the physical coordinates of the second video camera 120.The
Two homography matrixs can be obtained by existing mode, such as can be according to invention " one kind of Patent No. CN103198487A
For the automatic marking method in video monitoring system " in calibrating method obtain the second homography matrix.This second singly answers square
Battle array is preferably the matrix of 3*3, can convert the pixel coordinate of the first image 200 of first video camera 110 at any point
For the physical coordinates of the second video camera 120.
Then, believed according to the state of the pixel coordinate of the second image 300 of the second video camera 120 and the second video camera 120
Breath calculates the physical coordinates of second video camera 120 of the pixel coordinate of the second image 300 of corresponding second video camera 120
(horizontal and vertical deflection).Specifically, the pixel coordinate of N number of point is chosen on the second image 300.N is at least chosen at second
Not conllinear four pixels on image 300.
Then, calculated according to the physical coordinates of the inverse matrix of the second homography matrix and the second video camera 120 and correspond to second
The pixel coordinate of first image 200 of the first video camera 110 of the pixel coordinate of the second image 300 of video camera 120.Finally,
It is sat according to the pixel of the pixel coordinate of the first image 200 of the first video camera 110 and the second image 300 of the second video camera 120
Mark calculates initial homography matrix.
Specifically, in a specific embodiment, in order to improve computational accuracy and computational efficiency, in the second image 300
5 pixels of upper selection, the central pixel point for being respectively the second image 300 and the pixel close to four vertex.This 5 pictures
Vegetarian refreshments is denoted as p respectivelyi(i=1-5), pixel coordinate is denoted as (X respectivelydi, Ydi).Join further according to the inside of the second video camera 120
Current physical coordinates (the P of number, the second video camera 120c、Tc)(PcFor horizontal deflection coordinate, TcFor vertical deflection coordinate) and the
The current focal length value of two video cameras 120 corresponds to this 5 pixels to calculate in 120 the second current image 300 of the second video camera
The second video camera 120 physical coordinates.Then according to the pixel coordinate of the first image 200 to the physics of the second video camera 120
The inverse matrix of second homography matrix of coordinate system calculates first video camera 110 corresponding with the physical coordinates of the second video camera 120
The first image 200 5 pixels pixel coordinate (Xbi, Ybi).Then according to the second image 300 of the second video camera 120
Pixel coordinate (the X of upper 5 pointsdi, Ydi) and the first image 200 of the first video camera 120 on corresponding 5 points pixel coordinate
(Xbi, Ybi), calculate the initial homography matrix that the second image 300 is transformed into the first image 200.
S240：According to initial homography matrix, the second image 300 is mapped in the first image 200, obtains the second target figure
As 310.
Specifically, that is, according to initial homography matrix, the second image 300 is mapped to the scale of the first image 200
On, as shown in Figure 6.Wherein, when part of second image 300 there are no initial data, this sentences gray value 0 to fill.
When the second image 300 maps, image interpolation can be utilized.To make the image effect after interpolation more preferable, the present invention is excellent
Select cubic curve interpolation.
S250：Light stream matching is carried out to the first object image and second target image, calculates Optic flow information.
Specifically, it is the light stream matching that solves big displacement, therefore uses gaussian pyramid to first object image 210 and the
Two target images 310 are handled.Gaussian pyramid is an image collection, each image will originate from same in set
Original image, by the down-sampled acquisition of the Gauss of the image.Preferably, in the present embodiment, [1,/16 1/4 3/8 1/4 are utilized
1/16]x[1/16 1/4 3/8 1/4 1/16]TGaussian kernel 210 and second target image 310 of first object image is carried out
Convolution operation, wherein T representing matrixes transposition.According to 210 and second target image 310 of first object image, height is established as 3
Gaussian pyramid is denoted as first object image collection A and first object image collection B respectively.First object image collection A includes
The first layer first object subgraph A that size is gradually reduced1, second layer first object subgraph A2, third layer first object subgraph
As A3.Second target image set B includes first layer the second target subgraph B that size is gradually reduced1, the second target of the second layer
Subgraph B2, third layer the second target subgraph B3.In a specific embodiment, first layer first object subgraph A1And the
One layer of second target subgraph B1For the image of 96*96 pixels.Second layer first object subgraph A2And the second target of the second layer
Subgraph B2For the image of 48*48 pixels.Third layer first object subgraph A3And third layer the second target subgraph B3For
The image of 24*24 pixels.
After establishing the gaussian pyramid of 210 and second target image 310 of first object image, the second target figure is successively calculated
The gradient information of the gaussian pyramid of picture.Specifically, the gradient letter of the second target image set B is successively calculated according to equation below
Breath：
Wherein,Represent i-th layer of second target subgraph BiGradient information, gradxRepresent i-th layer of second target
Image BiIn the gradient information of X-direction, gradyRepresent i-th layer of second target subgraph BiGradient information in the Y direction, i
Take 3 successively, 2,1, T representing matrix transposition.It can use in some embodiments and gradient letter is calculated the methods of centered difference, Sharr
Cease gradxAnd grady.The present invention preferably, using Sharr methods calculates gradient information grad to calculatexAnd grady。
Then, light stream matching, meter are successively carried out to first object subgraph and the second target subgraph according to gradient information
Calculate Optic flow information.
The matched principle of light stream is represented with equation below：
Zd=err,
Wherein, Z represents gradient matrix, and d represents offset, and err represents difference.Gradient matrix Z, offset d and difference e rr
Respectively：
Wherein, dxRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in X
The offset in direction, dyRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in X
The offset in direction,Represent i-th layer of first object subgraph Ai's and i-th layer of second target subgraph Bi
Optic flow information.
ΣNgxx、ΣNgyy、ΣNgxy、errxAnd erryIt is calculated respectively according to equation below：
ΣNGxx=ΣNgradx*gradx；
ΣNGyy=ΣNgrady*grady；
ΣNGxy=ΣNgradx*grady；
errx=ΣNDiff*gradx；
erry=ΣNDiff*grady,
Wherein, N represents the neighborhood of characteristic point P, and characteristic point P chooses in each layer of first object image collection A, Diff tables
Show the gray scale difference value of pixel in the N of field.Field N be centered on characteristic point P, odd number of pixels point for the length of side square region
Domain.Preferably, field N is the square area of 15*15 pixels.
By gradient matrixOffsetAnd difference
Bring formula Z intod=err is obtained：
Correspondingly, Optic flow information is：
It is further simplified：
Wherein, det (Z) represents the value of the determinant of gradient matrix Z.
Correspondingly, the Optic flow information of X-direction and the Optic flow information of Y-direction are calculated according to equation below：
Specifically, the accurate of this feature point P can be calculated using Newton-Raphson iterative methods in this step
Solution, obtains the center point P of first object image 210cOptic flow information, be denoted as [dx, dy].Above formula describes specific every
The Optic flow information computational methods of one tomographic image, in pyramid figure layer, when calculating the Optic flow information of a certain layer, it is necessary to upper strata
Optic flow information as a result, light stream initial estimate as lower floor, wherein the light stream initial estimate of top layer's image are 0.Most
Precalculated is the Optic flow information of Gauss pyramid top layer image, and the output of this layer is as a result, as next layer of input.Now
Use the recursive operation of two adjacent interlayers of denotational description.It is assumed that two adjacent layers are L and L+1 respectively, and L+1 layers
Optic flow information has calculated, and is dL+1, then L layers of light stream initial estimate g is calculated from L+1 tomographic imagesLExpression formula
For：
gL=2 (gL+1+dL+1)
In which it is assumed that algorithm does not have believable light stream initial estimate top, i.e.,：
According to above-mentioned formula, when L layers of figure layer calculate light stream vector, do not sat in the characteristic point position of this layer of target image
Mark starts search matching, but the characteristic point pixel coordinate in this layer of target image translates gLPlace starts search matching, calculates residual
Poor minimum position, the light stream vector that so each layer searches all are thin tail sheeps.
Same method can calculate L-1 layers of displacement vector dL+1, this process is performed until image bottom L=1,
Until reaching original image, image and displacement vector are all original resolution ratio at this time.The then light stream displacement vector of the bottom
For：
D=g1+d1
It can also be represented with each layer of light stream vector：
So operation, to ensure in the Optic flow information calculating process of each layer of Gauss pyramid, the displacement of characteristic point P is all
It is thin tail sheep.
S260：It is calculated according to the Optic flow information and corrects homography matrix.Wherein, homography matrix is corrected to be used for first object
Image 210 is mapped in the second image 300.
Specifically, this step calculates amendment homography matrix in the following way：First, in first object image 210
N number of pixel is chosen as the first pixel.It is chosen in the second target image 310 corresponding with the first pixel N number of respectively
Pixel is as the second pixel.The pixel coordinate of N number of first pixel is corrected using Optic flow information, obtains N number of amendment first
Pixel.The amendment homography matrix is calculated using the pixel coordinate for correcting the first pixel and the second pixel.
In a specific embodiment, N preferably, takes 5.Then this step first, 5 is chosen in first object image 210
A pixel is as the first pixel pbi.5 pixels corresponding with the first pixel respectively are chosen in the second target image 310
Point is as the second pixel pdi.The pixel coordinate of 5 the first pixels is subtracted to the Optic flow information of step S250 calculating acquisitions
[dx, dy], obtain 5 the first pixel p of amendmentbi' pixel coordinate.Utilize the first pixel p of amendmentbi' and the second pixel
pdiPixel coordinate calculate correct homography matrix.
S270：According to homography matrix is corrected, first object image 210 is mapped in the second image 300, obtains and corrects the
Two target images, to position target object 900 in the second image 300.
Specifically, first object image 210 has first object frame 220.First object frame 220 is first object image
210 boundary rectangle.This step further includes：According to homography matrix is corrected, first object frame 220 is mapped to the second image 300
In, using the boundary rectangle of the mapping objects frame of acquisition as the second target frame 320.Using the image in the second target frame 320 as
Correct the second target image.In some specific embodiments, it is rectangle that mapping objects frame, which is not, it is therefore preferred that by mapping objects
The boundary rectangle of frame is as the second target frame 320.The second target image, which is corrected, in second target frame 320 can be used for target object
900 subsequent image identifications and graphical analysis.
The present invention is by carrying out the first image and the second image gaussian pyramid and the matched operation of light stream in the second figure
Position and the size of target object 900 are accurately positioned in picture 300, and reduces in the second target image of amendment finally obtained
Invalid information.
Compared with prior art, the present invention obtains wide angle picture and partial enlargement image by two kinds of video cameras, according to wide-angle
Image and the mapping of partial enlargement image and light stream matching, calculate the offset between image, and then mesh will be included in wide angle picture
The target image of mark object is mapped in partial enlargement image, to position the target object in partial enlargement image.The present invention is only
The target image that target object is included in partial enlargement image is used for the image procossing for target object.The target of the present invention
The target image that object positioning method is provided contains target object exactly, and it will not include a large amount of invalid information
To increase the time of image procossing and load.
Exemplary embodiments of the present invention are particularly shown and described above.It should be understood that the invention is not restricted to institute
Disclosed embodiment, on the contrary, it is intended to cover comprising various modifications within the scope of the appended claims and equivalent put
It changes.
Claims (19)
1. a kind of localization method of target object, for camera chain, the camera chain includes：
First video camera, for obtaining the first image, described first image is the wide angle picture of a scene ken；And
Second video camera, for obtaining the second image, second image is the partial enlarged view of the scene ken；
The localization method includes：
A. the described first image and the second image for including target object are obtained；
B. first object image is obtained according to described first image, the first object image includes the target object；
C. according to the status information of the targeted message and second video camera of first video camera and second video camera,
Calculate initial homography matrix；
D. according to the initial homography matrix, second image is mapped in described first image, obtains the second target figure
Picture；
E. light stream matching is carried out to the first object image and second target image, calculates Optic flow information；
F. calculated according to the Optic flow information and correct homography matrix；
G. according to the amendment homography matrix, the first object image is mapped in second image, obtains and corrects the
Two target images, to position the target object in second image.
2. localization method as described in claim 1, which is characterized in that the step b includes：
In described first image, centered on the center of the target object, interception rectangular target image is as described first
Target image.
3. localization method as claimed in claim 2, which is characterized in that the rectangular target image intercepted is 96*96 pixel
Square target image.
4. localization method as described in claim 1, which is characterized in that the initial homography matrix is converted for second image
To the homography matrix of described first image.
5. localization method as claimed in claim 4, which is characterized in that the step c includes：
C1. the targeted message is obtained, the pixel coordinate that the targeted message includes the first image of first video camera turns
Change to the second homography matrix of the physical coordinates of second video camera；
C2. according to the calculating pair of the status information of the pixel coordinate of the second image of second video camera and second video camera
Answer the physical coordinates of second video camera of the pixel coordinate of the second image of second video camera；
C3. calculated according to the physical coordinates of the inverse matrix of second homography matrix and second video camera and correspond to described the
The pixel coordinate of first image of first video camera of the pixel coordinate of the second image of two video cameras；And
C4. according to the pixel of the pixel coordinate of the first image of first video camera and the second image of second video camera
Coordinate calculates the initial homography matrix.
6. localization method as claimed in claim 5, which is characterized in that the step c2 includes：
The pixel coordinate of at least four not conllinear pixels is chosen as second video camera according to second image
Pixel coordinate.
7. localization method as described in claim 1, which is characterized in that the step e includes：
E1. the gaussian pyramid of the first object image and second target image is calculated；
E2. the gradient information of the gaussian pyramid of second target image is successively calculated；
E3. light stream matching is successively carried out to the first object image and second target image according to the gradient information,
Calculate Optic flow information.
8. localization method as claimed in claim 7, which is characterized in that the step e1 includes：
Convolution operation is carried out to the first object image and second target image using Gaussian kernel；
According to the first object image and second target image, the gaussian pyramid that height is 3 is established, is denoted as the respectively
One target image set A and the second target image set B, wherein,
The first object image collection A includes the first layer first object subgraph A that size is gradually reduced1, the first mesh of the second layer
Mark subgraph A2, third layer first object subgraph A3；
The second target image set B includes first layer the second target subgraph B that size is gradually reduced1, the second mesh of the second layer
Mark subgraph B2, third layer the second target subgraph B3。
9. localization method as claimed in claim 8, which is characterized in that the Gaussian kernel is [1,/16 1/4 3/8 1/4 1/
16]x[1/16 1/4 3/8 1/4 1/16]T。
10. localization method as claimed in claim 8, which is characterized in that
The first layer first object subgraph A1And first layer the second target subgraph B1For the image of 96*96 pixels；
The second layer first object subgraph A2And the second layer the second target subgraph B2For the figure of 48*48 pixels
Picture；
The third layer first object subgraph A3And third layer the second target subgraph B3For the image of 24*24 pixels.
11. localization method as claimed in claim 8, which is characterized in that the step e2 includes：
The gradient information of the second target image set B is successively calculated according to equation below：
Wherein,Represent i-th layer of second target image BiGradient information, gradxRepresent i-th layer of second target subgraph
As BiIn the gradient information of X-direction, gradyRepresent i-th layer of second target subgraph BiGradient information in the Y direction, i according to
It is secondary to take 3,2,1.
12. localization method as claimed in claim 11, which is characterized in that the step e3 includes：
The Optic flow information is calculated according to equation below
Wherein, dxRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in X-direction
Offset, dyRepresent i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi in the Y direction
Offset,Represent the light stream letter of i-th layer of first object subgraph Ai and i-th layer of second target subgraph Bi
Breath,
ΣNgyy、ΣNgyy、∑Ngxy、errxAnd erryIt is calculated respectively according to equation below：
∑NGxx=∑sNgradx*gradx；
∑NGyy=∑sNgrady*grady；
∑NGxy=∑sNgradx*grady；
errx=∑NDiff*gradx；
erry=∑NDiff*grady；
Wherein, N represents the neighborhood of characteristic point P, and characteristic point P chooses in each layer of the first object image collection A, Diff tables
Show the gray scale difference value of pixel in neighborhood N.
13. localization method as claimed in claim 12, which is characterized in that the neighborhood N is centered on characteristic point P, odd number
A pixel is the square area of the length of side.
14. localization method as claimed in claim 12, which is characterized in that i-th layer of first object subgraph Ai and i-th layer described
The Optic flow information of second target image Bi is according to i+1 layer first object subgraph Ai+1 and second target of i+1 layer
The Optic flow information of image Bi+1 calculates.
15. localization method as described in claim 1, which is characterized in that the step f includes：
N number of pixel is chosen in the first object image as the first pixel；
Chosen in second target image respectively N number of pixel corresponding with first pixel as the second pixel
Point；
The pixel coordinate of N number of first pixel is corrected using the Optic flow information, obtains the first pixel of N number of amendment；
The amendment homography matrix is calculated using the first pixel of the amendment and the pixel coordinate of second pixel.
16. localization method as described in claim 1, which is characterized in that the first object image has first object frame, institute
The boundary rectangle that first object frame is the first object image is stated, the step g includes：
According to the amendment homography matrix, the first object frame is mapped in second image, by the mapping mesh of acquisition
The boundary rectangle of frame is marked as the second target frame, using the image in the second target frame as the second target figure of the amendment
Picture.
17. such as claim 1 to 16 any one of them localization method, which is characterized in that the second target image of the amendment is used
In the image identification and graphical analysis of the target object.
18. a kind of camera system, which is characterized in that including：
First video camera, for obtaining the first image, described first image is the wide angle picture of a scene ken；
Second video camera, for obtaining the second image, second image is the partial enlarged view of the scene ken；And
Positioner, using such as claim 1 to 17 any one of them localization method, according to controlling described first image
Second video camera in second image to position the target object.
19. camera system as claimed in claim 18, which is characterized in that first video camera is gun shaped video camera, described
Second video camera is ball-shaped camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510711384.2A CN105335977B (en) | 2015-10-28 | 2015-10-28 | The localization method of camera system and target object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510711384.2A CN105335977B (en) | 2015-10-28 | 2015-10-28 | The localization method of camera system and target object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105335977A CN105335977A (en) | 2016-02-17 |
| CN105335977B true CN105335977B (en) | 2018-05-25 |
Family
ID=55286482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510711384.2A Active CN105335977B (en) | 2015-10-28 | 2015-10-28 | The localization method of camera system and target object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105335977B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107650122B (en) * | 2017-07-31 | 2019-11-05 | 宁夏巨能机器人股份有限公司 | A kind of robot hand positioning system and its localization method based on 3D visual identity |
| CN109242769B (en) * | 2018-12-13 | 2019-03-19 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
| CN109506658B (en) * | 2018-12-26 | 2021-06-08 | 广州市申迪计算机系统有限公司 | Robot autonomous positioning method and system |
| CN111698455B (en) * | 2019-03-13 | 2022-03-11 | 华为技术有限公司 | Method, device and medium for controlling linkage of ball machine and gun machine |
| CN111800604A (en) * | 2020-06-12 | 2020-10-20 | 深圳英飞拓科技股份有限公司 | Method and device for detecting human shape and human face data based on gun and ball linkage |
| CN111800605A (en) * | 2020-06-15 | 2020-10-20 | 深圳英飞拓科技股份有限公司 | Gun-ball linkage based vehicle shape and license plate transmission method, system and equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0714081A1 (en) * | 1994-11-22 | 1996-05-29 | Sensormatic Electronics Corporation | Video surveillance system |
| US6215519B1 (en) * | 1998-03-04 | 2001-04-10 | The Trustees Of Columbia University In The City Of New York | Combined wide angle and narrow angle imaging system and method for surveillance and monitoring |
| CN102148965A (en) * | 2011-05-09 | 2011-08-10 | 上海芯启电子科技有限公司 | Video monitoring system for multi-target tracking close-up shooting |
| CN102932598A (en) * | 2012-11-06 | 2013-02-13 | 苏州科达科技股份有限公司 | Method for intelligently tracking image on screen by camera |
| CN103024350A (en) * | 2012-11-13 | 2013-04-03 | 清华大学 | Master-slave tracking method for binocular PTZ (Pan-Tilt-Zoom) visual system and system applying same |
| CN103105858A (en) * | 2012-12-29 | 2013-05-15 | 上海安维尔信息科技有限公司 | Method capable of amplifying and tracking goal in master-slave mode between fixed camera and pan tilt zoom camera |
| CN103198487A (en) * | 2013-04-15 | 2013-07-10 | 厦门博聪信息技术有限公司 | Automatic calibration method for video monitoring system |
| CN104574425A (en) * | 2015-02-03 | 2015-04-29 | 中国人民解放军国防科学技术大学 | Calibration and linkage method for primary camera system and secondary camera system on basis of rotary model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8488001B2 (en) * | 2008-12-10 | 2013-07-16 | Honeywell International Inc. | Semi-automatic relative calibration method for master slave camera control |
-
2015
- 2015-10-28 CN CN201510711384.2A patent/CN105335977B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0714081A1 (en) * | 1994-11-22 | 1996-05-29 | Sensormatic Electronics Corporation | Video surveillance system |
| US6215519B1 (en) * | 1998-03-04 | 2001-04-10 | The Trustees Of Columbia University In The City Of New York | Combined wide angle and narrow angle imaging system and method for surveillance and monitoring |
| CN102148965A (en) * | 2011-05-09 | 2011-08-10 | 上海芯启电子科技有限公司 | Video monitoring system for multi-target tracking close-up shooting |
| CN102932598A (en) * | 2012-11-06 | 2013-02-13 | 苏州科达科技股份有限公司 | Method for intelligently tracking image on screen by camera |
| CN103024350A (en) * | 2012-11-13 | 2013-04-03 | 清华大学 | Master-slave tracking method for binocular PTZ (Pan-Tilt-Zoom) visual system and system applying same |
| CN103105858A (en) * | 2012-12-29 | 2013-05-15 | 上海安维尔信息科技有限公司 | Method capable of amplifying and tracking goal in master-slave mode between fixed camera and pan tilt zoom camera |
| CN103198487A (en) * | 2013-04-15 | 2013-07-10 | 厦门博聪信息技术有限公司 | Automatic calibration method for video monitoring system |
| CN104574425A (en) * | 2015-02-03 | 2015-04-29 | 中国人民解放军国防科学技术大学 | Calibration and linkage method for primary camera system and secondary camera system on basis of rotary model |
Non-Patent Citations (2)
| Title |
|---|
| Heterogeneous Fusion of Omnidirectional and PTZ Cameras for Multiple Object Tracking;Chung-Hao Chen 等;《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》;20080709;第18卷(第8期);1052-1063 * |
| 一种用于鱼眼PTZ主从监控系统的标定方法;石皓 等;《系统仿真学报》;20131008;第25卷(第10期);2412-2417 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105335977A (en) | 2016-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105335977B (en) | The localization method of camera system and target object | |
| CN110211043B (en) | Registration method based on grid optimization for panoramic image stitching | |
| CN106651942B (en) | Three-dimensional rotating detection and rotary shaft localization method based on characteristic point | |
| CN106803273B (en) | A kind of panoramic camera scaling method | |
| CN104484648B (en) | Robot variable visual angle obstacle detection method based on outline identification | |
| CN111750820B (en) | Image positioning method and system | |
| CN110176032B (en) | Three-dimensional reconstruction method and device | |
| CN105654476B (en) | Binocular calibration method based on Chaos particle swarm optimization algorithm | |
| CN105654547B (en) | Three-dimensional rebuilding method | |
| CN107767339B (en) | Binocular stereo image splicing method | |
| CN110298884A (en) | A kind of position and orientation estimation method suitable for monocular vision camera in dynamic environment | |
| CN107843251A (en) | The position and orientation estimation method of mobile robot | |
| CN110400278A (en) | A kind of full-automatic bearing calibration, device and the equipment of color of image and geometric distortion | |
| CN113191954B (en) | Panoramic image stitching method based on binocular camera | |
| WO2021035627A1 (en) | Depth map acquisition method and device, and computer storage medium | |
| CN111383270A (en) | Object positioning method and device, computer equipment and storage medium | |
| CN113763569A (en) | Image annotation method and device used in three-dimensional simulation and electronic equipment | |
| CN109003307B (en) | Underwater binocular vision measurement-based fishing mesh size design method | |
| CN110567441A (en) | Particle filter-based positioning method, positioning device, mapping and positioning method | |
| CN107038758A (en) | A kind of augmented reality three-dimensional registration method based on ORB operators | |
| Wan et al. | Drone image stitching using local mesh-based bundle adjustment and shape-preserving transform | |
| CN114998773A (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
| CN105513074A (en) | Badminton robot camera calibration method | |
| CN109448105B (en) | Three-dimensional human body skeleton generation method and system based on multi-depth image sensor | |
| WO2020133080A1 (en) | Object positioning method and apparatus, computer device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |