CN113125791B - Motion camera speed measuring method based on characteristic object and optical flow method - Google Patents
Motion camera speed measuring method based on characteristic object and optical flow method Download PDFInfo
- Publication number
- CN113125791B CN113125791B CN201911388507.8A CN201911388507A CN113125791B CN 113125791 B CN113125791 B CN 113125791B CN 201911388507 A CN201911388507 A CN 201911388507A CN 113125791 B CN113125791 B CN 113125791B
- Authority
- CN
- China
- Prior art keywords
- camera
- characteristic object
- speed
- video
- optical flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000003287 optical effect Effects 0.000 title claims abstract description 25
- 230000003068 static effect Effects 0.000 claims abstract description 7
- 238000012937 correction Methods 0.000 claims abstract description 6
- 238000000605 extraction Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 7
- 230000000007 visual effect Effects 0.000 description 6
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P3/00—Measuring linear or angular speed; Measuring differences of linear or angular speeds
- G01P3/36—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light
- G01P3/38—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light using photographic means
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Power Engineering (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
A speed measuring method of a moving camera based on a characteristic object and an optical flow method is used for correcting distortion of a plane video of a monocular camera, tracking a static characteristic object in the video by using the optical flow method, and obtaining the speed of the high-speed moving camera by combining information of the characteristic object. The invention provides a scheme for extracting speed information of a moving camera in a plane video of a common monocular lens, which realizes long-distance height measurement by using distortion correction and information of a characteristic object, and when the distance between the moving camera and the characteristic object is not more than 100m and the moving speed is not more than 300km/h, the error can be controlled within 1%.
Description
Technical Field
The invention belongs to the field of intelligent measurement, relates to a speed measuring method of a moving camera aiming at videos, and particularly relates to a speed measuring method of a moving camera based on a characteristic object and an optical flow method.
Background
With the rise of smartphones, a large number of mobile devices are provided with cameras so as to record videos at any time, so that a lot of video data for analysis are generated, for the common monocular plane videos, visual information analysis staff can only analyze the common monocular plane videos in a visual observation mode, the efficiency of information analysis is restricted, and the speed of extracting visual objects from video images is one of important contents of visual information analysis work.
Early object speed measurement mainly uses modes such as radar, laser, infrared rays and ultrasonic waves, the measurement methods have high requirements on equipment and are easily influenced by environment, and the equipment is usually in a static state, so that the speed measurement range has certain directivity, and the flexibility is insufficient. Also sometimes we are interested in the motion state of the device holder, which cannot be analyzed by the above method. In recent years, the development of computer vision theory and method has made it possible to detect the speed of a moving camera by video.
Currently, most research efforts focus on acquiring visual information using binocular or multi-vision systems or specially configured monocular vision systems, both monocular and binocular vision systems, requiring calibration of the camera's internal and external parameters. In addition, for the monocular vision system, in order to improve the speed measurement accuracy, the speed measurement line sensing device in a scene is relied on, so that the system can only work in specific scenes, such as highways, racetracks and the like, and the speed information of the system cannot be acquired. The binocular vision system can acquire the moving distance of an object through depth information generated by the binocular head, and can acquire the speed of the object through measuring the relative displacement distance of a static object, but the problems of high equipment cost and single video source exist. In many cases, therefore, most of the images or videos available to visual information analysts are also obtained with a common monocular camera.
The invention relates to a speed measuring method of a moving camera based on a characteristic object and an optical flow method, which provides a scheme for extracting speed information of the moving camera in a common monocular plane video, and realizes accurate acquisition of the speed information of the camera under the condition of high-speed movement of the camera.
Disclosure of Invention
The invention aims to solve the problems that: the method solves the problem of accurately extracting the speed information of the moving camera in a larger speed range when the camera is used for a common single-purpose plane video, and accurately measuring the speed of the camera under high-speed movement.
The technical scheme of the invention is as follows: a speed measuring method of a moving camera based on a characteristic object and an optical flow method is used for correcting distortion of a plane video of a monocular camera, tracking a static characteristic object in the video by using the optical flow method, and obtaining the speed of the high-speed moving camera by combining information of the characteristic object.
Preferably, the present invention comprises the steps of:
1) For an input video, acquiring the frame rate dt of the video;
2) For the input video extraction key frame, selecting two adjacent frames of video frames with characteristic objects, wherein the actual length L of the characteristic objects in the moving direction of the camera real It is known to correct the distortion of key points of a feature object; measuring the pixel length L of the corrected characteristic object in the camera motion direction pixel Tracking corrected key points of characteristic objects by using optical flow method, and obtaining the level and the pixel of the key pointsSpeeds u and v in the vertical direction;
3) Calculating the velocity v of a moving camera in adjacent frames p ：
4) Repeating the steps 2) and 3), and taking N groups of adjacent frames to obtain N groups of v p Calculate N groups v p As the average value of the velocity V of the moving camera camera 。
The invention has the following effective benefits: the prior art schemes can generate larger errors when the distance between the camera and the feature exceeds 2m or the movement speed exceeds 100 km/h. Because the object shot by the camera lens is distorted and the distortion is aggravated under the condition of high-speed movement, the invention realizes long-distance height measurement by using the distortion correction and the information of the characteristic object, and the error can be controlled within 1 percent when the distance between the object and the characteristic object is not more than 100m and the movement speed is not more than 300 km/h. The method has good universality and practicability.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is an effect show of optical flow tracking.
Detailed Description
The invention provides a motion camera speed measuring method based on a characteristic object and an optical flow method, which aims at a plane video of a monocular camera, firstly corrects distortion, then tracks a static characteristic object in the video by using the optical flow method, and combines information of the characteristic object to obtain the speed of the high-speed motion camera.
As shown in fig. 1, an implementation of the present invention is illustrated, including the following steps.
1) For an input video, the frame rate dt of the video is acquired using OpenCV.
2) For an input video extraction key frame, selecting two adjacent frames with characteristic objects, tracking key points of the characteristic objects by using an optical flow method, and acquiring speeds u and v of key point pixels in horizontal and vertical directions:
2.1 Extracting key frames from the input video by using OpenCV, selecting two adjacent frames with characteristic objects, and measuring the pixel length of the characteristic objects in the camera motion direction as L pixel The characteristic object selection criteria are:
2.1.1 The object plane is parallel to the camera motion plane, i.e. the perpendicular distance of the object from the camera motion plane is fixed.
2.1.2 Actual length L in camera movement direction real Known and common stationary objects such as garbage cans, stationary cars, etc.
2.2 Taking a pixel I (x, y, t) of the feature object as a key point at the time t, wherein x and y are respectively coordinates of the pixel point in a video frame, the pixel point is subjected to time dt in a next frame, dx and dy are respectively moved in the x and y directions, and because two pixels of a front frame and a rear frame are the same pixel, the relationship of the two pixels can be obtained:
I(x，y，t)＝I(x+dx，y+dy，t+dt)
2.3 Since the camera lens usually has distortion problem, the key points need to be corrected for distortion, and the camera internal parameters (f) can be obtained by calibrating the camera u ，f v ，u 0 ，v 0 ) And distortion coefficients (k 1, k2, k3, p1, p 2). Thus, for the keypoints (x, y), the keypoints (x ', y') in the camera coordinate system can be obtained as:
let r= (x' 2 +y′ 2 ) Camera coordinate systemThe point after the lower correction (x ", y") is:
x″＝x′*(1+k1*r+k2*r 2 +k3*r 3 )+2*p1*x′*y′+p2*(r+2*x′ 2 )
y″＝y′*(1+k1*r+k2*r2+k3*r 3 )+2*p2*x′*y′+p1*(r+2*y′ 2 )
the corrected points (x '", y'") are:
x″′＝x″*f u +u 0
y″′＝y″*f v +v 0
wherein f u And f v Is focal length u 0 And v 0 For the optical center, k1 is a 1-order radial distortion coefficient, k2 is a 2-order radial distortion coefficient, k3 is a 3-order radial distortion coefficient, p1 is a 1-order tangential distortion coefficient, and p2 is a 2-order tangential distortion coefficient.
The pixel relationship formula in step 2.2) therefore converts after distortion correction to:
I(x″′ 1 ，y″′ 1 ，t)＝I(x″′ 2 ，y″′ 2 ，t+dt)
2.4 Taylor expansion is carried out on the corrected pixel points in the step 2.3), and an optical flow equation can be obtained:
f x u+f y v+ft＝0
wherein f x And f y Is the gradient of the image, f t Is a gradient in time and is a function of the time,
2.5 Inverse solution of u, v) by the Lucas-Kanade method using the least squares method:
3) Calculating the velocity v of the moving camera in the adjacent frame p ：
4) Repeating the steps 2) and 3), taking as many frame pairs of N groups of adjacent video frames as possible, calculating the average value of the speeds as the speed V of the moving camera camera ：
Representing the velocity v of the motion camera resulting from the i-th set of adjacent video frames p ，i＝1,2,…N。
The invention carries out practical implementation test, the camera internal and external parameter information of the shot video is unknown and has no definite distance information, and the invention can better estimate the speed of the camera motion by tracking the static trolley appearing in the video by an optical flow method as shown in figure 2, and the method of the invention is used for measuring the camera motion speed of 304.56 km/h, has smaller error and has the error of less than 1% compared with the correct answer of 303 km/h.
Claims (4)
1. A speed measuring method of a moving camera based on a characteristic object and an optical flow method is characterized in that distortion correction is firstly carried out on a plane video of a monocular camera, then the static characteristic object in the video is tracked by the optical flow method, and the speed of the high-speed moving camera is obtained by combining the information of the characteristic object, and the method comprises the following steps:
1) For an input video, acquiring the frame rate dt of the video;
2) For the input video extraction key frame, selecting two adjacent frames of video frames with characteristic objects, wherein the actual length L of the characteristic objects in the moving direction of the camera real It is known to correct the distortion of key points of a feature object; measuring the pixel length L of the corrected characteristic object in the camera motion direction pixel Tracking corrected key points of the characteristic object by using an optical flow method, and acquiring speeds u and v of key point pixels in the horizontal and vertical directions;
in the step 2), after two adjacent frames of video frames are selected, one pixel I (x, y, t) of a characteristic object in a first frame is taken as a key point, wherein x and y are respectively coordinates of the pixel I (x, y, t) in the video frames, the pixel I (x, y, t) has a time dt in the next frame, dx and dy are respectively moved in the x and y directions, and the relationship of the pixels in the two frames is obtained:
I(x,y,t)＝I(x+dx,y+dy,t+dt)
correcting the distortion of the key points, and obtaining the camera internal parameters (f by calibrating the camera u ,f v ,u 0 ,v 0 ) And distortion coefficients (k 1, k2, k3, p1, p 2), for the key points (x, y), key points (x ', y') under the camera coordinate system are obtained as:
let r= (x) ′2 +y ′2 ) The corrected points (x ", y") in the camera coordinate system are:
x″＝x ′ *(1+k1*r+k2*r 2 +k3*r 3 )+2*p1*x ′ *y ′ +p2*(r+2*x ′2 )
y″＝y ′ *(1+k1*r+k2*r 2 +k3*r 3 )+2*p2*x ′ *y ′ +p1*(r+2*y ′2 )
the corrected points (x '", y'") are:
x″′＝x″*f u +u 0
y″′＝y″*f v +v 0
wherein f u And f v Is focal length u 0 And v 0 The optical center is an optical center, k1 is a 1-order radial distortion coefficient, k2 is a 2-order radial distortion coefficient, k3 is a 3-order radial distortion coefficient, p1 is a 1-order tangential distortion coefficient, and p2 is a 2-order tangential distortion coefficient;
the relationship between the pixels in the two frames after distortion correction is obtained is as follows:
I(x″′ 1 ,y″′ 1 ,t)＝I(x″′ 2 ,y″′ 2 ,t+dt)；
3) Calculating the velocity v of a moving camera in adjacent frames p ：
4) Repeating the steps 2) and 3), and taking N groups of adjacent frames to obtain N groups of v p Calculate N groups v p As the average value of the velocity V of the moving camera camera 。
2. The method for measuring the speed of a moving camera based on a characteristic object and an optical flow method according to claim 1, wherein in step 1), the frame rate of a video is obtained by using OpenCV.
3. The method for measuring speed of a moving camera based on a characteristic object and an optical flow method according to claim 1, wherein when two adjacent frames of video frames with the characteristic object are selected in step 2), the characteristic object selection criteria are as follows:
a) The object plane is parallel to the camera motion plane;
b) Actual length L in camera movement direction real Known stationary objects.
4. The method for measuring the speed of a moving camera based on a characteristic object and an optical flow method according to claim 1, wherein in the step 2), the speeds u and v of the key point pixels in the horizontal and vertical directions are obtained specifically as follows:
taylor expansion is carried out on the corrected pixel points, and an optical flow equation is obtained:
f x u+f y v+f t ＝0
wherein f x And f y Is the gradient of the image, f t Is a gradient in time and is a function of the time,
and (3) reversely solving u, v by using a least square method through a Lucas-Kanade method:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911388507.8A CN113125791B (en) | 2019-12-30 | 2019-12-30 | Motion camera speed measuring method based on characteristic object and optical flow method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911388507.8A CN113125791B (en) | 2019-12-30 | 2019-12-30 | Motion camera speed measuring method based on characteristic object and optical flow method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113125791A CN113125791A (en) | 2021-07-16 |
| CN113125791B true CN113125791B (en) | 2023-10-20 |
Family
ID=76768778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911388507.8A Active CN113125791B (en) | 2019-12-30 | 2019-12-30 | Motion camera speed measuring method based on characteristic object and optical flow method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113125791B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622895A (en) * | 2012-03-23 | 2012-08-01 | 长安大学 | Video-based vehicle speed detecting method |
| CN105989593A (en) * | 2015-02-12 | 2016-10-05 | 杭州海康威视系统技术有限公司 | Method and device for measuring speed of specific vehicle in video record |
| CN109410254A (en) * | 2018-11-05 | 2019-03-01 | 清华大学深圳研究生院 | A kind of method for tracking target modeled based on target and camera motion |
| CN110136168A (en) * | 2019-04-26 | 2019-08-16 | 北京航空航天大学 | A kind of more rotor-speed measurement methods based on Feature Points Matching and optical flow method |
| CN110319772A (en) * | 2019-07-12 | 2019-10-11 | 上海电力大学 | Visual large-span distance measurement method based on unmanned aerial vehicle |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9582722B2 (en) * | 2012-08-31 | 2017-02-28 | Xerox Corporation | Video-based vehicle speed estimation from motion vectors in video streams |
-
2019
- 2019-12-30 CN CN201911388507.8A patent/CN113125791B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622895A (en) * | 2012-03-23 | 2012-08-01 | 长安大学 | Video-based vehicle speed detecting method |
| CN105989593A (en) * | 2015-02-12 | 2016-10-05 | 杭州海康威视系统技术有限公司 | Method and device for measuring speed of specific vehicle in video record |
| CN109410254A (en) * | 2018-11-05 | 2019-03-01 | 清华大学深圳研究生院 | A kind of method for tracking target modeled based on target and camera motion |
| CN110136168A (en) * | 2019-04-26 | 2019-08-16 | 北京航空航天大学 | A kind of more rotor-speed measurement methods based on Feature Points Matching and optical flow method |
| CN110319772A (en) * | 2019-07-12 | 2019-10-11 | 上海电力大学 | Visual large-span distance measurement method based on unmanned aerial vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113125791A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105225482B (en) | Vehicle detecting system and method based on binocular stereo vision | |
| CN110807809B (en) | Light-weight monocular vision positioning method based on point-line characteristics and depth filter | |
| CN102646275B (en) | The method of virtual three-dimensional superposition is realized by tracking and location algorithm | |
| US8385595B2 (en) | Motion detection method, apparatus and system | |
| US8238610B2 (en) | Homography-based passive vehicle speed measuring | |
| CN108317953A (en) | A kind of binocular vision target surface 3D detection methods and system based on unmanned plane | |
| CN104575003B (en) | A kind of vehicle speed detection method based on traffic surveillance videos | |
| CN103411621A (en) | Indoor-mobile-robot-oriented optical flow field vision/inertial navigation system (INS) combined navigation method | |
| CN114037762B (en) | Real-time high-precision positioning method based on registration of image and high-precision map | |
| CN109859269B (en) | Shore-based video auxiliary positioning unmanned aerial vehicle large-range flow field measuring method and device | |
| Cvišić et al. | Recalibrating the KITTI dataset camera setup for improved odometry accuracy | |
| CN103093459A (en) | Assisting image matching method by means of airborne lidar point cloud data | |
| CN104200492A (en) | Automatic detecting and tracking method for aerial video target based on trajectory constraint | |
| CN113191239A (en) | Vehicle overall dimension dynamic detection system based on computer vision | |
| CN112990128A (en) | Multi-vehicle speed measuring method based on video tracking | |
| CN107945166B (en) | Binocular vision-based method for measuring three-dimensional vibration track of object to be measured | |
| CN113125791B (en) | Motion camera speed measuring method based on characteristic object and optical flow method | |
| CN104809720A (en) | Small cross view field-based double-camera target associating method | |
| CN116804553A (en) | Odometer system and method based on event camera/IMU/natural road sign | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera | |
| CN112985388B (en) | Combined navigation method and system based on large-displacement optical flow method | |
| CN113706599B (en) | Binocular depth estimation method based on pseudo label fusion | |
| CN107292932B (en) | Head-on video speed measurement method based on image expansion rate | |
| CN113192011A (en) | Traffic accident speed measurement method based on monitoring video image | |
| CN111415369A (en) | Target real-time speed measurement method based on road end monocular camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |