CN111780716A

CN111780716A - Monocular real-time distance measurement method based on target pixel area and aspect ratio

Info

Publication number: CN111780716A
Application number: CN202010629609.0A
Authority: CN
Inventors: 贾刚勇; 陈硕; 李尤慧子; 殷昱煜; 蒋从锋; 张纪林; 万健
Original assignee: Hangzhou Dianzi University
Current assignee: Hangzhou Dianzi University
Priority date: 2020-07-03
Filing date: 2020-07-03
Publication date: 2020-10-16

Abstract

The invention relates to a monocular real-time distance measuring method based on a target pixel area and an aspect ratio. The method comprises four parts: the ssd model is adopted to realize the detection processing of the target, and more accurate framing is realized; calibrating parameters of a camera, acquiring internal parameters and image resolution of the camera, and solving the ratio of pixel area to width to height of a target object based on the width and height of target detection; monocular distance measurement under different distances from multiple angles is realized through the relation between the pixel area and the distance of target detection and the characteristic of the aspect ratio of the specific target pixel; and due to the instability of target detection, Kalman filtering is introduced in the real-time ranging process, so that the stability of real-time ranging is improved. The invention provides a ranging method based on the pixel area and the aspect ratio of a target aiming at different imaging angles and instable real-time target detection, and introduces a Kalman filtering method, thereby improving the applicability of multi-angle ranging and improving the stability of real-time moving target ranging.

Description

Monocular real-time distance measurement method based on target pixel area and aspect ratio

Technical Field

The invention relates to the field of image processing and ranging, in particular to a monocular real-time ranging method based on a target pixel area and an aspect ratio.

Background

In places where manpower cannot directly work, such as dangerous places like high temperature and high fortune shooting operation and remote places like space universe, the image acquisition equipment is used for carrying out remote and non-contact distance measurement system, and the complex environments can be monitored and corresponding work instructions can be made. The research object and the related environment are connected through computer technology, and corresponding reference is provided for work of a user. At present, the application of visual ranging is mainly in the fields of robot arm grabbing, positioning, industrial robot detection, aerial surveying and mapping, intelligent traffic monitoring, reverse engineering, target identification, military application, medical imaging and the like.

The distance measurement technology is in a rapid development stage at home and abroad, and particularly under high standard requirements on real-time performance, stability and accuracy of a distance measurement system, the visual distance measurement technology is actively researched by related personnel at home and abroad. At present, various digital image fast processing algorithms are continuously proposed and improved on the aspect of image distance measurement. The distance measurement method mainly comprises the following steps: laser ranging, ultrasonic ranging, radar ranging, and computer vision ranging.

The laser ranging method is a high-precision ranging mode applied to specific occasions. Laser ranging requires high requirements for devices and signal processing techniques because of the short wavelength and fast speed of light. In practical application, the distance measurement principle is realized by mainly utilizing a counting principle and a phase principle, the working principle of distance measurement is simple, but the dependence on a counter is large, and errors are easily caused; the error can be conveniently controlled by using the phase principle for ranging, but the signal processing process is complex. Thus, laser ranging is limited in cost and reliability.

The ultrasonic ranging is a mature ranging method, and the method is low in cost and simple in working principle. Ultrasonic waves are active energy that must be attenuated during transmission to an extent proportional to the square of the propagation distance. That is, the farther the propagation distance is, the weaker the reflected sound wave signal is, and the larger the measurement error is. Therefore, the optimal range measurement range of the method is 5-10m, and the application range of the method is greatly reduced due to the limitation of the measurement range.

The radar ranging method has high precision and is slightly influenced by distance and weather conditions. The distance can be measured as long as the target to be measured can reflect radar waves. It should be noted that, when using radar ranging, the mutual interference between the ranging devices is very serious, so it cannot be used in multiple radar measurement environments at the same time. In addition, attention should be paid to electromagnetic interference between radar and other communication systems.

The computer vision distance measurement is that after the camera collects the image, the computer analyzes the image and calculates the distance between the camera and the measured target according to the relevant distance measurement principle. The method is a passive distance measuring method, and measuring equipment only needs to shoot an image containing a measured target instead of transmitting a detection signal to a measured object. Therefore, the computer machine vision distance measurement mainly has the advantages of low measurement cost, simple distance measurement principle, no influence of external environment on measurement, application in complex and harmful environments and the like.

The vision measuring method can be classified into a monocular vision measuring method, a binocular vision measuring method (stereoscopic vision), and a monocular vision measuring method (omnidirectional vision) according to the number of used vision imaging apparatuses. Due to the influence of practical factors such as an installation platform, a field, cost and the like, the practical application proportion of monocular vision is far larger than that of binocular and multi-view ranging. The monocular vision measuring method is a method for measuring information such as the geometric size of a target, the position and the posture of the target in space and the like by only utilizing one vision imaging device to acquire images, but the angle of the target is changed more in actual movement due to incomplete acquisition of monocular vision information. Therefore, it is desirable to accurately measure a distance to a target using a monocular camera, and to provide good results even when the angle between the target and the camera is changed.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a monocular real-time distance measuring method based on the target pixel area and the aspect ratio.

The invention carries out three-part optimization:

(1) representing distance relation between target and camera by target pixel area size

In the past, the relation between the distance and the target is generally expressed only by the width or height of the pixel of the target. The problem of width and height fluctuation in target detection is possibly caused, and the method of using the pixel width and height area to replace the pure pixel width or height is adopted, so that the stability of distance measurement is improved to a certain extent.

(2) Reducing the impact of view angle changes using target detection aspect ratios

When the visual angle of the camera and the target is changed, the pixel area of the front side is deduced according to the change of the aspect ratio, and the distance measurement effect is further achieved.

(3) Reducing real-time ranging fluctuations using kalman filtering

Kalman filtering does not require that both signal and noise are assumptions for a stationary process. For each time instant, the system disturbance and observation error (i.e. noise) can be estimated in an average sense by processing the observation signal containing noise, with some suitable assumptions about their statistical properties, to obtain the estimate of the true signal with the smallest error. Therefore, since the advent of kalman filter theory, it has been applied to many sectors such as communication systems, power systems, aerospace, environmental pollution control, industrial control, radar signal processing, and the like, and has achieved many successful results. In terms of image processing, for example, kalman filtering is applied to restore an image that is blurred due to some noise effect. After the noise is assumed to have certain statistical properties, a real image with the minimum mean square error can be obtained from the blurred image in a recursion mode by using a Kalman algorithm, so that the blurred image is restored. Because the target detection has fluctuation between each frame, the distance measurement error can be generated, and the Kalman filtering processing method is adopted, so that the detection error between the frames is reduced, and the overall stability of the real-time distance measurement system is improved.

The method comprises the following specific steps:

step 1: and the ssd model is adopted to realize the detection processing of the target, so that the accurate framing is realized.

Step 2: and calibrating parameters of the camera, acquiring internal parameters of the camera, and acquiring the pixel area and the aspect ratio of the front angle of the target based on the width and the height of the target detection.

And step 3: the monocular distance measurement under different distances from multiple angles is realized through the relation between the pixel area and the distance detected by the target and the characteristic of the aspect ratio of the target pixel.

And 4, step 4: kalman filtering is introduced into the real-time distance measurement, so that the error of target detection between frames is reduced, and the stability of the real-time distance measurement is improved.

The invention has the beneficial effects that: the invention provides a ranging method based on the pixel area and the aspect ratio of a target aiming at different imaging angles and instable real-time target detection, and introduces a Kalman filtering method, thereby improving the applicability of multi-angle ranging and improving the stability of real-time moving target ranging.

Drawings

FIG. 1 is a diagram of a pinhole imaging model architecture.

Fig. 2 is a schematic diagram of the width and height selection based on target detection.

FIG. 3 is a schematic diagram of the change in angle of a camera to a target.

Fig. 4 is a monocular real-time ranging flow chart based on target pixel area and aspect ratio.

Fig. 5 is a graph of the filtering effect of camera carrier stage velocity equal to the detected target velocity.

Fig. 6 is a diagram of the filtering effect when the speed of the camera carrier platform is less than the speed of the detected target.

Fig. 7 is a diagram of the filtering effect when the speed of the camera carrier platform is greater than the speed of the detected target.

Detailed Description

The invention will be further described with reference to the accompanying drawings and preferred embodiments.

Step 1: the target detection is realized by using an ssd model, and the initial part of the network model of the ssd is VGG-16 and is called as a basic network. After the VGG-16 network, the SSD adds the convolution characteristic maps with gradually decreased resolution, the characteristic maps have different feelings, so that the SSD can perform multi-scale target detection, namely, the high-resolution characteristic map is used for detecting small targets in an image, and the low-resolution characteristic map is used for detecting large targets in the image. This approach has two main advantages: firstly, a small target can be more accurately positioned on a high-resolution feature map through repositioning category (classification) and outline regression; secondly, the SSD does not need to perform region generation first, and the multi-scale processing does not increase the calculation amount of the basic network, so that the SSD is much Faster than the method such as fast R-CNN which needs region generation. The SSD model is trained on the basis of the PASCALVOC2012 data set, and the detection frame of the target is accurately determined in the process of testing the test set (only the detection frame with the confidence coefficient larger than 0.5 is displayed).

Step 2: as shown in fig. 1 and 2, the monocular camera pixel area based method implemented by the present invention includes the following steps:

s1, acquiring the internal parameters of the monocular camera;

the method mainly comprises the steps of calibrating a camera for the focal length f of the monocular camera (calibrating the camera by adopting a fixed target), and changing the distance u between the fixed target and the camera at equal intervals to obtain the horizontal focal length f of the monocular camera_xAnd a vertical focal length value f_y. In this embodiment, the monocular camera may be a common camera, a mobile phone, or other devices capable of taking pictures.

S2: shooting images of different distances u from the front angle of the target, wherein X is the actual width of the front of the target, Y is the actual height of the front of the target, and X_oPixel width, y, of the front of the object in the image_oThe pixel area s of the front angle of the target is the pixel height of the front of the target in the image_oThe relationship to the distance u of the camera is as follows:

pixel aspect ratio r of the front side of the object_oAs follows:

actual aspect ratio R of the target front angle, and R_oThe relationship with R is as follows:

R＝r_o

and step 3: as shown in FIG. 3, the multi-angle ranging algorithm based on the aspect ratio of the pixels of the monocular camera implemented by the present invention is as follows:

algorithm input B ═ x₁,y₁,x₂,y₂) Representing the upper left corner (x) of the object detection₁,y₁) And the lower right corner point (x)₂,y₂) R represents the actual aspect ratio of the front angle of the target, and the algorithm output u represents the actual distance between the target and the camera.

The algorithm mainly comprises the following contents: calculating the pixel width x ═ x of the target₂-x₁) And pixel height y ═ y (y)₂-y₁) Calculating the pixel width-to-height ratio r as x/y<(R- α) when the camera photographs the side view of the subject, y is constant as compared to the frontal view, and x is reduced as compared to the frontal view, and a width-to-height ratio y based on the frontal view is derived_cY and x_c＝y_cR; when the pixel aspect ratio r>(R + α) when the camera photographs the object with a top view angle, x being constant compared to the frontal view angle and y being reduced compared to the frontal view angle, and deriving a width-to-height ratio x based on the frontal view angle_cX and y_c＝x_cR; when in frontal view, x_cX and y_cY., finally, calculating the distance u according to a calculation formula of the pixel area and the distance, introducing α to improve the stability of the algorithm, ensuring that the operation of the first two steps is not frequently triggered when the aspect ratio is changed slightly, and x_cAnd y_cFor the pixel width and height for true range finding after algorithm processing, the pseudo code of the specific algorithm is as follows:

Algorithm 1 Ranging algorithm based on the ratio of width to height

Input:Object box B＝(x₁,y₁,x₂,y₂)

Ratio of width to height R

Output:Distance u

1:x＝x₂-x₁

2:y＝y₂-y₁

3:r＝x/y

4:if r<(R-a)then

y_c＝y

x_c＝y_c×R

5:else if r>(R+α)then

x_c＝x

y_c＝x_c/R

6:else

x_c＝x

y_c＝y

7:end if

and 4, step 4: in order to improve the stability of video real-time ranging, a Kalman filtering processing method is adopted, for the processing of a real-time video, the time interval of every two frames is small, and the motion of a target between adjacent frames can be considered to be slow and approximate to uniform motion, and the method is characterized in that:

x_k＝x_k-1+v_k-1Δt，v_k＝v_k-1where △ t is a two frame time interval.

The system is a linear dynamic model, and the system state equation is as follows:

X(k)＝AX(k-1)+G(k)

in the formula

X(k)＝(x₁,y₁,x₂,y₂)^T；

The period of two detections of Δ t video, g (k), represents the rate variation, i.e. gaussian white noise of the process.

The system measurements were:

Y(k)＝H_kX(k)+C(k)

h denotes a parameter of the measurement system, and c (k) denotes measurement noise. The target location and associated error covariance matrix P are predicted using a kalman filter:

where Q is the covariance of the process noise.

Combining the predicted values and the measured values, the optimized estimated value of the current state k can be obtained as follows:

and updating the error covariance of X (k | k) in the k state

P(k|k)＝P(k|k-1)-K_kH_kP(k|k-1)

In the formula, K_kIs a Kalman gain

In the formula, R_kRepresenting the covariance of the measurement noise, a lower value of the measurement covariance means a greater weighting on the current measurement value, at which point the sensitivity of the tracking system is higher.

The following are the detection effects of the present invention:

under the condition of image pixels 640 x 480, when the distance between the camera and the moving target is within the range of 275-700 cm, the measurement error is small and is generally less than 5%, and the target features with overlarge distance are not obviously detected, so that the detection is not accurate.

Numbering	Area of pixel	Actual distance (cm)	Calculating distance (cm)	Error (%)
					1	30854.77	275	278.7	1.35
2	26883.61	300	298.6	0.47
					3	20284.81	325	340.8	4.62
4	17884.31	350	366.1	4.60
					5	15963.73	375	387.5	3.33
6	15137.45	400	398.0	0.5
					7	13006.46	425	429.3	1.01
8	11080.78	450	465.1	3.36
					9	10291.51	475	482.7	1.62
10	9760.07	500	495.6	0.88
					11	9189.83	525	510.8	2.70
12	7877.28	550	551.7	0.31
					13	7159.28	575	578.7	0.64
14	6451.96	600	609.6	1.60
					15	6157.56	625	624.0	0.16
16	5807.32	650	642.5	1.15
					17	5232.53	675	676.9	0.28
18	4882.89	700	700.7	0.10

The distance between the camera and the target was kept at 2.6 m. The included angle between the optical axis of the camera and the horizontal plane is changed to 10-60 degrees. The multi-angle ranging error based on the aspect ratio is less than 5%.

Angle of rotation	Based on area only	Error (%)	Area and ratio	Error (%)
					10	268.4	3.23	267.9	3.04
20	268.0	3.08	265.4	2.08
					30	266.8	2.62	260.5	0.19
40	273.1	5.04	258.1	0.73
					50	288.7	11.04	267.9	3.04
60	306.2	17.77	270.4	4.00

In the case that the speed of the carrier platform is the same as that of the target, the speed of the carrier platform is lower than that of the target, and the speed of the carrier platform is higher than that of the target, the kalman filtering is introduced in combination with fig. 5, fig. 6, and fig. 7, after the kalman filtering, the stability of the real-time distance measurement is obviously improved, and in the case of three, the fluctuation of 25.221% is averagely reduced after the kalman filtering.

Claims

1. A monocular real-time distance measurement method based on target pixel area and aspect ratio is characterized by comprising the following steps:

the method comprises the following steps: the ssd model is adopted to realize the detection processing of the target, and more accurate framing is realized;

step two: calibrating the focal length of the camera, acquiring internal parameters of the camera, and acquiring the pixel area and the aspect ratio of the front angle of the target based on the width and the height of the target detection;

step three: the monocular distance measurement under different distances from multiple angles is realized by the relationship between the pixel area and the distance detected by a target and the characteristic of the aspect ratio of the target pixel, and specifically, the monocular distance measurement is as follows:

let the coordinate of the upper left corner point of the target be (x)₁,y₁) And the coordinate of the lower right corner point is (x)₂,y₂) R is the aspect ratio of the target front angle;

calculating the pixel width x ═ x of the target₂-x₁) And pixel height y ═ y (y)₂-y₁) Calculating the actual pixel aspect ratio r as x/y; when the pixel aspect ratio r<(R- α) when α is a threshold value set by experiment, the camera photographs the side view of the subject, y is constant as compared with the frontal view, and x is reduced as compared with the frontal view, and a width and a height y based on the frontal view are derived_cY and x_c＝y_c*R；

When the pixel aspect ratio r>(R + α), the camera shoots the subjectFrom the top view, x is constant compared to the frontal view and y is reduced compared to the frontal view, deriving x based on the width and height of the frontal view_cX and y_c＝x_c/R；

Threshold value set for alpha

When in frontal view, x_cX and y_c＝y；

Finally, calculating the distance u according to a calculation formula of the pixel area and the distance;

step four: kalman filtering is introduced into the real-time distance measurement, so that the error of target detection between frames is reduced, and the stability of the real-time distance measurement is improved.

2. The monocular real-time ranging method of claim 1, wherein: based on a pinhole imaging model, shooting a front angle image of a target by adopting a fixed target and a distance u between equidistant transformation and cameras to obtain a camera focal length f:

wherein f is_xIs the horizontal focal length of the monocular camera, f_yIs the vertical focal length value of the monocular camera, X is the actual width of the front of the target, Y is the actual height of the front of the target, X_oPixel width, y, of the front of the object in the image_oThe pixels in the image for the front of the object are high.

3. The monocular real-time ranging method of claim 2, wherein: the size of the object is represented by its pixel area, and the pixel area value s of the object is calculated using the following formula_o：