CN111780716A - A monocular real-time ranging method based on target pixel area and aspect ratio - Google Patents

Abstract

The invention relates to a monocular real-time ranging method based on target pixel area and aspect ratio. The method of the invention includes four parts: adopting the ssd model to realize the detection and processing of the target to achieve more accurate framing; calibrating the parameters of the camera to obtain the internal parameters and image resolution of the camera, and finding the target based on the detected width and height of the target. The pixel area and aspect ratio of the object; through the relationship between the pixel area and the distance detected by the target and the characteristics of the specific target pixel aspect ratio, monocular ranging at multiple angles and different distances is realized; due to the instability of target detection, in real time Kalman filter is introduced in the ranging process to stabilize the real-time ranging. Aiming at the difference of imaging at different angles and the instability of real-time target detection, the invention proposes a ranging method based on the target pixel area and aspect ratio, and introduces a Kalman filtering method to improve the applicability of multi-angle ranging, thereby improving the real-time Stability of moving target ranging.

Description

A monocular real-time ranging method based on target pixel area and aspect ratio

technical field

The present invention relates to the field of image processing and ranging, in particular to a monocular real-time ranging method based on target pixel area and aspect ratio.

Background technique

In places where manpower cannot directly work, such as dangerous places such as high temperature and high radiation operations, and distant places such as space and space, the use of image acquisition equipment for long-distance, non-contact ranging systems can monitor these complex environments and make corresponding work instructions. Through computer technology, a connection between the research object and the relevant environment is established, and the corresponding reference is provided for the user's work. At present, the applications of visual ranging are mainly in the fields of robot arm grasping, positioning, industrial robot detection, aerial surveying and mapping, intelligent monitoring of traffic, reverse engineering, target recognition, military application, medical imaging and other fields.

Ranging technology has been in a rapid development stage at home and abroad, especially under the high standard requirements for real-time, stability and accuracy of the ranging system, relevant personnel at home and abroad are actively studying the technology of visual ranging. At present, in image distance measurement, various digital image fast processing algorithms are continuously proposed and improved. Ranging methods mainly include: laser ranging, ultrasonic ranging, radar ranging and computer vision ranging.

Laser ranging method is a high-precision ranging method applied in specific occasions. Due to the short wavelength of light and the fast speed of light, laser ranging has high requirements on devices and signal processing technology. In practical applications, the counting principle and the phase principle are mainly used to realize distance measurement. The counting principle is simple in working principle, but it relies heavily on the counter, which is easy to cause errors. Using the phase principle to measure distance can easily control the error, but signal processing The process is complicated. Therefore, laser ranging is limited by cost and reliability.

Ultrasonic ranging is a relatively mature ranging method with low cost and simple working principle. Ultrasound is a kind of active energy, and it will be attenuated during the transmission process, and its attenuation degree is proportional to the square of the propagation distance. That is, the longer the propagation distance, the weaker the reflected sound wave signal, and the greater the measurement error. Therefore, the optimal distance measurement range of this method is 5-10m, and the limitation of measurement range greatly reduces its application.

Radar ranging is a high-precision ranging method that is very little affected by distance and climatic conditions. Distance can be measured as long as the target under test can reflect radar waves. It should be noted that when using radar ranging, because the mutual interference between ranging devices is very serious, it cannot be used in multiple radar measurement environments at the same time. In addition, electromagnetic interference between other communication systems of the radar should also be noted.

Computer vision ranging is a method that uses a computer to analyze and process the image after collecting the image by the camera, and obtain the distance between the camera and the measured target according to the relevant ranging principle. This method is a passive ranging method, and the measuring equipment only needs to take an image containing the measured object, instead of transmitting a detection signal to the measured object. Therefore, computer machine vision ranging mainly has the advantages of low measurement cost, simple ranging principle, measurement unaffected by external environment, and application in complex and harmful environments.

Depending on the number of visual imaging devices used, vision measurement methods can be divided into monocular vision measurement methods, binocular vision measurement methods (stereoscopic vision), and polyocular vision measurement methods (omnidirectional vision). Due to the actual factors such as installation platform, site and price, the actual application ratio of monocular vision is much larger than that of binocular and multi-eye ranging. The monocular vision measurement method is to use only one visual imaging device to collect images and measure the geometric size of the target, the position and attitude of the target in space, etc. However, due to the incomplete acquisition of monocular visual information, the angle of the target is There are many changes in the actual movement. Therefore, it has always been expected that the monocular camera can be used to measure the distance of the target more accurately, and it can also have good results under the condition that the angle between the target and the camera changes.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the present invention provides a monocular real-time ranging method based on target pixel area and aspect ratio.

The present invention carries out three-part optimization:

(1) The size of the target pixel area is used to represent the distance relationship between the target and the camera

In the past, ranging generally only relies on the pixel width or height of the target to express the relationship with the distance. There may be fluctuations in width and height in target detection. The pixel width and height area is used to replace the simple pixel width or height method, which improves the stability of ranging to a certain extent.

(2) Use the target detection aspect ratio to reduce the influence of viewing angle changes

Using a certain type of fixed target, the average aspect ratio and aspect ratio of the frontal angle detection are fixed, and the frontal viewing angle aspect ratio is used as the benchmark. When the viewing angle of the camera and the target is changed, it is derived according to the transformation of the aspect ratio. The pixel area on the front, and then achieve the effect of ranging.

(3) Kalman filtering is used to reduce the fluctuation of real-time ranging

Kalman filtering does not require the assumption that both signal and noise are stationary processes. For the system disturbance and observation error (that is, noise) at each moment, as long as some appropriate assumptions are made about their statistical properties, and by processing the observation signal containing noise, the error can be obtained in the average sense as The smallest estimate of the true signal. Therefore, since the advent of Kalman filter theory, it has been applied in communication systems, power systems, aerospace, environmental pollution control, industrial control, radar signal processing and many other departments, and achieved many successful application results. For example, in image processing, Kalman filtering is applied to restore blurred images due to some noise. After making some statistical assumptions about the noise, Kalman's algorithm can be used to get the real image with the smallest mean square error from the blurred image in a recursive way, so that the blurred image can be restored. Due to the fluctuation between each frame of target detection, ranging error will occur. Kalman filtering is used to reduce the detection error between frames and improve the overall stability of the real-time ranging system.

The concrete steps of the inventive method are:

Step 1: Use the ssd model to detect the target and achieve a more accurate framing.

Step 2: Perform parameter calibration on the camera, obtain the internal parameters of the camera, and obtain the pixel area and aspect ratio of the frontal angle of the target based on the width and height of the target detection.

Step 3: Through the relationship between the pixel area and the distance detected by the target and the characteristics of the target pixel aspect ratio, monocular ranging at different angles and distances is realized.

Step 4: Kalman filter is introduced in real-time ranging to reduce the error of target detection between each frame, thereby improving the stability of real-time ranging.

Beneficial effects of the present invention: Aiming at different imaging angles at different angles and the instability of real-time target detection, the present invention proposes a ranging method based on target pixel area and aspect ratio, and introduces a Kalman filtering method, which improves the applicability of multi-angle ranging Therefore, the stability of real-time moving target ranging is improved.

Description of drawings

Figure 1 is a structural diagram of the pinhole imaging model.

FIG. 2 is a schematic diagram of the selection of width and height based on target detection.

FIG. 3 is a schematic diagram of the angle change between the camera and the target.

Figure 4 is a flow chart of monocular real-time ranging based on target pixel area and aspect ratio.

Figure 5 is a filter effect diagram of the camera carrier platform speed equal to the detection target speed.

Figure 6 is a filter effect diagram of the camera carrier platform speed being less than the detection target speed.

Figure 7 is a filter effect diagram of the camera carrier platform speed being greater than the detection target speed.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and in conjunction with the preferred embodiments.

Step 1: Use the ssd model to achieve target detection. The initial part of the ssd network model is VGG-16, which is called the basic network. After the VGG-16 network, SSD adds convolutional feature maps with decreasing resolution layer by layer. These feature maps have different feelings, so SSD can perform multi-scale target detection, that is, use high-resolution feature maps to detect images. small objects, and use low-resolution feature maps to detect large objects in the image. This method has two main advantages: First, small objects can be more accurately located by relocating classification and outer border regression on high-resolution feature maps; second, SSD does not need to generate regions first, which This multi-scale processing does not increase the computational complexity of the underlying network, so it is much faster than methods such as Faster R-CNN that require region generation. The SSD model of this invention is trained on the basis of the PASCALVOC2012 data set. In the process of testing the test set (only the detection frame with a confidence greater than 0.5 is displayed), the detection frame of the target is relatively accurate.

Step 2: As shown in Figure 1 and Figure 2, the method based on the pixel area of a monocular camera implemented by the present invention includes the following steps:

S1: Get the internal parameters of the monocular camera;

This step is mainly to perform camera calibration on the focal length f of the monocular camera (using a fixed target to calibrate the camera), and transform the distance u between the fixed target and the camera at equal intervals to obtain the horizontal focal length f _x and vertical direction of the monocular camera. Directional focal length value f _y . In this embodiment, the monocular camera may be a device that can take pictures, such as a common camera, a mobile phone, and the like.

S2: Shoot images with different distances u from the front of the target, let X be the actual width of the front of the target, Y be the actual height of the front of the target, x _o is the pixel width of the front of the target in the image, and y _o is the width of the front of the target in the image. The relationship between the pixel height, the pixel area s _o of the frontal angle of the target and the distance u of the camera is as follows:

The pixel aspect ratio _ro of the front of the target is as follows:

The actual aspect ratio R of the frontal angle of the target, and the relationship between r _o and R are as follows:

R＝r_o

R=r _o

Step 3: As shown in FIG. 3, the multi-angle ranging algorithm based on the pixel aspect ratio of the monocular camera implemented by the present invention is as follows:

The algorithm input B=(x ₁ , y ₁ , x ₂ , y ₂ ) represents the upper left corner point (x ₁ , y ₁ ) and the lower right corner point (x ₂ , y ₂ ) of the target detection, and R represents the actual width of the frontal angle of the target High ratio, the algorithm output u represents the actual distance between the target and the camera.

The main content of the algorithm: calculate the pixel width x=(x ₂ -x ₁ ) and pixel height y=(y ₂ -y ₁ ) of the target, calculate the pixel aspect ratio r=x/y, when the pixel aspect ratio r<( R-α), the camera shoots the side view of the target, y is unchanged compared with the front view, x is reduced compared with the front view, and the aspect ratio based on the front view is derived y _c =y and x _c =y _c *R; when the pixel aspect ratio r>(R+α), the top-view angle of the camera shooting target, x is unchanged compared with the frontal angle of view, y is reduced compared with the frontal angle of view, it is derived Aspect ratio x _c =x and y _c =x _c /R based on frontal viewing angle; when in frontal viewing angle, x _c =x and _yc =y. Finally, the distance u is calculated according to the calculation formula of the pixel area and the distance. The introduction of α is to improve the stability of the algorithm and ensure that the operations of the first two steps will not be triggered frequently when the aspect ratio changes are small. x _c and y _c are the pixel width and height used for real ranging after the algorithm processing. The pseudo code of the specific algorithm is as follows:

Algorithm 1 Ranging algorithm based on the ratio of width to height

Input:Object box B＝(x ₁ ,y ₁ ,x ₂ ,y ₂ )

Ratio of width to height R

Output: Distance u

₁ :x= _x2 -x1

2: y=y ₂ -y ₁

3:r=x/y

4: if r<(R-a)then

y _c = y

x _c =y _c ×R

5: else if r>(R+α)then

x _c = x

y _c =x _c /R

6: else

x _c = x

y _c = y

7: end if

Step 4: In order to improve the stability of video real-time ranging, the Kalman filter processing method is adopted. For real-time video processing, the time interval between every two frames is relatively small, and it can be considered that the target moves slowly between adjacent frames, which is approximately a uniform speed. Motion, by the kinetic formula:

x _k =x _k-1 +v _k-1 Δt, v _k =v _k-1 where Δt is the time interval of two frames.

The system is a linear dynamic model, and the system state equation:

X(k)=AX(k-1)+G(k)

X(k)＝(x₁,y₁,x₂,y₂)^T；in the formula

X(k)=(x ₁ , y ₁ , x ₂ , y ₂ ) ^T ;

Δt is the period of two detections of the video, and G(k) represents the rate change, that is, the Gaussian white noise of the process.

The system measurements are:

Y(k)=H _k X(k)+C(k)

H represents the parameters of the measurement system, and C(k) represents the measurement noise. The target position and associated error covariance matrix P are predicted using a Kalman filter:

where Q is the covariance of the process noise.

Combining the predicted value and the measured value, the optimal estimate of the current state k can be obtained as:

and update the error covariance of X(k|k) in k states

P(k|k)=P(k|k-1)-K _k H _k P(k|k-1)

where K _k is the Kalman gain

In the formula, R _k represents the covariance of the measurement noise. When the value of the measurement covariance is lower, it means that there is a greater weight on the current measurement value, and the sensitivity of the tracking system is higher at this time.

The following is the detection effect of the present invention:

Under the condition of 640*480 image pixels, when the distance between the camera and the moving target is in the range of 275-700cm, the measurement error is very small, generally less than 5%, and the target feature is not obvious if the distance is too large, and the detection is inaccurate.

Numbering pixel area Actual distance (cm) Calculated distance (cm) error(%) 1 30854.77 275 278.7 1.35 2 26883.61 300 298.6 0.47 3 20284.81 325 340.8 4.62 4 17884.31 350 366.1 4.60 5 15963.73 375 387.5 3.33 6 15137.45 400 398.0 0.5 7 13006.46 425 429.3 1.01 8 11080.78 450 465.1 3.36 9 10291.51 475 482.7 1.62 10 9760.07 500 495.6 0.88 11 9189.83 525 510.8 2.70 12 7877.28 550 551.7 0.31 13 7159.28 575 578.7 0.64 14 6451.96 600 609.6 1.60 15 6157.56 625 624.0 0.16 16 5807.32 650 642.5 1.15 17 5232.53 675 676.9 0.28 18 4882.89 700 700.7 0.10

Keep the distance between the camera and the target at 2.6m. Change the angle between the optical axis of the camera and the horizontal plane by 10°-60°. The multi-angle ranging error based on aspect ratio is less than 5%.

angle based on area only error(%) Area and Proportion error(%) 10 268.4 3.23 267.9 3.04 20 268.0 3.08 265.4 2.08 30 266.8 2.62 260.5 0.19 40 273.1 5.04 258.1 0.73 50 288.7 11.04 267.9 3.04 60 306.2 17.77 270.4 4.00

Combined with Fig. 5, Fig. 6 and Fig. 7, Kalman filter is introduced in three cases when the speed of the carrier platform is the same as that of the target, the speed of the carrier platform is smaller than the target and the speed of the carrier platform is greater than the target. After Kalman filtering processing, the stability of real-time ranging is obviously improved In the three cases, the fluctuation is reduced by an average of 25.221% after filtering.

Claims (3)

1. A monocular real-time distance measurement method based on target pixel area and aspect ratio is characterized by comprising the following steps:

the method comprises the following steps: the ssd model is adopted to realize the detection processing of the target, and more accurate framing is realized;

step two: calibrating the focal length of the camera, acquiring internal parameters of the camera, and acquiring the pixel area and the aspect ratio of the front angle of the target based on the width and the height of the target detection;

step three: the monocular distance measurement under different distances from multiple angles is realized by the relationship between the pixel area and the distance detected by a target and the characteristic of the aspect ratio of the target pixel, and specifically, the monocular distance measurement is as follows:

let the coordinate of the upper left corner point of the target be (x)₁,y₁) And the coordinate of the lower right corner point is (x)₂,y₂) R is the aspect ratio of the target front angle;

calculating the pixel width x ═ x of the target₂-x₁) And pixel height y ═ y (y)₂-y₁) Calculating the actual pixel aspect ratio r as x/y; when the pixel aspect ratio r<(R- α) when α is a threshold value set by experiment, the camera photographs the side view of the subject, y is constant as compared with the frontal view, and x is reduced as compared with the frontal view, and a width and a height y based on the frontal view are derived_cY and x_c＝y_c*R；

When the pixel aspect ratio r>(R + α), the camera shoots the subjectFrom the top view, x is constant compared to the frontal view and y is reduced compared to the frontal view, deriving x based on the width and height of the frontal view_cX and y_c＝x_c/R；

Threshold value set for alpha

When in frontal view, x_cX and y_c＝y；

Finally, calculating the distance u according to a calculation formula of the pixel area and the distance;

step four: kalman filtering is introduced into the real-time distance measurement, so that the error of target detection between frames is reduced, and the stability of the real-time distance measurement is improved.

2. The monocular real-time ranging method of claim 1, wherein: based on a pinhole imaging model, shooting a front angle image of a target by adopting a fixed target and a distance u between equidistant transformation and cameras to obtain a camera focal length f:

wherein f is_xIs the horizontal focal length of the monocular camera, f_yIs the vertical focal length value of the monocular camera, X is the actual width of the front of the target, Y is the actual height of the front of the target, X_oPixel width, y, of the front of the object in the image_oThe pixels in the image for the front of the object are high.

3. The monocular real-time ranging method of claim 2, wherein: the size of the object is represented by its pixel area, and the pixel area value s of the object is calculated using the following formula_o：

Images