CN111083360B

CN111083360B - Camera monitoring equipment matching control method based on pixel coordinates

Info

Publication number: CN111083360B
Application number: CN201911250263.7A
Authority: CN
Inventors: 许乙付; 罗喜伶; 罗亨; 曾杰
Original assignee: Beihang University
Current assignee: Beihang University
Priority date: 2019-12-09
Filing date: 2019-12-09
Publication date: 2021-06-04
Anticipated expiration: 2039-12-09
Also published as: CN111083360A

Abstract

The invention discloses a pixel coordinate-based camera monitoring device matching control method, which relates to the field of camera monitoring. Aiming at the deficiency of the fuzzy one-way control of the camera monitoring equipment in the existing camera tracking system, the present invention improves the native PELCO control protocol, and realizes the quantitative binning control and periodic feedback of the zoom value of the zoom lens and the angle of view of the camera; The mathematical geometric model of calculation and correction further obtains the calculation formula of the set value of the camera angle of view, so that the proportion of target pixel size reaches the set value; the mathematical geometric model of target offset calculation and correction is constructed, and the rotation of the rotating pan/tilt is further obtained. The formula for calculating the value, so that the target returns to the center of the screen. Through effect experiments, the present invention can realize precise and stable control of the camera monitoring equipment, so that the size of the viewing angle of the equipment can be matched with the size of the target, and the direction of the viewing angle can be matched with the moving amount of the target.

Description

Camera monitoring equipment matching control method based on pixel coordinates

Technical Field

The invention relates to a matching control method of a camera monitoring device based on pixel coordinates, in particular to a method capable of enabling a target marked by the pixel coordinates to accurately return to the center of a picture and reach a set ratio.

Background

At present, a camera shooting tracking system usually follows the flow of image acquisition, target identification and equipment control, the target identification result is information such as target category and pixel coordinates, and the equipment control performs matching control on a zoom lens, a rotating holder and the like based on the information. When matching control is carried out, a control algorithm is needed to calculate a zoom value of the zoom lens and a rotation value of the rotating holder to be set according to the current zoom value of the zoom lens, the rotation value of the rotating holder and the pixel coordinate, wherein the former enables the proportion of the size of a target pixel to reach a set proportion, and the latter enables the target to return to the center of a picture. The method represented by a patent "an intrusion alert photoelectric monitoring system and method" (application number: 201910133621.X) usually indicates the process of acquisition-identification-control ", but lacks deep analysis of control steps, and especially lacks improved optimization of construction of a control geometric model and accurate control of a camera monitoring device.

In the aspect of camera monitoring equipment control, the most common protocol for controlling a zoom lens and a rotating pan-tilt is mainly PELCO (Pelco-Gogh) protocol, which comprises two types of PELCO-D and PELCO-P, but only provides a progress bar type fuzzy speed control instruction, has no feedback, and cannot meet the requirements of accurate and stable control. The technical threshold of the rotating tripod head is relatively low, and accurate stepping amount control can be realized by adopting a stepping motor for driving, so that the PELCO protocol only needs to be improved, and related contents are disclosed in a serial port communication protocol based on ball machine control (application number: 201610555808.5). The zoom lens is particularly referred to as a motorized zoom lens in engineering application, has higher technical threshold, and particularly is a long-focus motorized zoom lens which is often imported only, mainly takes the Japanese brand, and basically only provides a native PELCO protocol, so that the requirement of precise matching control cannot be met in the linkage with other systems.

Disclosure of Invention

In order to overcome the defect of fuzzy one-way control of the image pickup monitoring equipment in the existing image pickup tracking system, the invention provides the image pickup monitoring equipment matching control method based on the pixel coordinates.

The invention relates to a camera monitoring equipment matching control method based on pixel coordinates, which comprises the following steps:

s01: a camera acquires a collected image; calibrating a target and acquiring a pixel coordinate of the target;

s02: sequentially executing steps S03 and S04, or executing S04 and then executing S03; completing the matching control of the camera shooting monitoring equipment based on the pixel coordinates;

the S03 is as follows: according to the current value P of the target ratio₁Current value of horizontal angle of view theta of camera₁Setting the target ratio to a set value P₂Converting into a set value theta of horizontal visual angle of the camera₂；

According to the obtained horizontal visual angle set value theta of the camera₂Controlling the zoom lens of the camera to enable the proportion of the target in the picture to reach a set proportion;

the S04 is as follows: obtaining a target angle offset according to the central coordinate of the target and the central coordinate of the collected image; and adjusting the camera according to the target angle offset to return the target to the center of the picture.

As a preferred aspect of the present invention, the method for controlling a zoom lens of a camera specifically comprises:

the camera comprises a camera shooting tracking host, an external hardware module and a zoom lens;

uniformly dividing the focal length of the zoom lens into N gears, and recording delta t as the time required by adjacent gear change; then

T is the moving time of the focal length of the motorized zoom lens from the maximum value to the minimum value or from the minimum value to the maximum value, and a lens gear-visual angle comparison table is established;

according to the lens gear-visual angle comparison table, the camera shooting tracking host machine sets the horizontal visual angle of the camera to be a set value theta₂Converting into a gear setting gear n 2; the camera shooting tracking host sends gear setting information to the external hardware module, and the external hardware module compares the current gear to generate a focal length moving instruction to control the zoom lens and periodically feeds back the current gear to the camera shooting tracking host.

As a preferred scheme of the invention, the external hardware module receives a command from the camera tracking host, and updates the set gear n2 after verification;

every delta t time, the external hardware module judges the following steps: judging the current focal length control direction Dir, wherein a positive number indicates increasing, a negative number indicates decreasing, and 0 indicates stopping; calculating a gear potential difference delta n which is n2-n1, wherein the positive delta n represents that the focal length needs to be increased, the negative delta n represents that the focal length needs to be decreased, and 0 represents that the focal length value is just right;

and changing the current focal length control direction according to the values corresponding to the delta n and the Dir, and simultaneously returning the current gear value to the camera monitoring host.

The invention improves the native PELCO control protocol, realizes the quantitative stepping control and the periodic feedback of the zoom value of the zoom lens and the visual angle value of the camera; constructing a mathematical geometric model for calculating and correcting the target ratio, and further obtaining a calculation formula of a set value of a visual angle of the camera, so that the target pixel size ratio reaches the set value; and (3) constructing a mathematical geometric model for calculating and correcting the target offset, and further obtaining a calculation formula of the rotation value of the rotating holder, so that the target returns to the center of the picture. Through effect experiments, the invention can realize accurate and stable control of the camera monitoring equipment, so that the size of the visual angle of the equipment is self-matched with the size of a target, and the direction of the visual angle is self-matched with the movement amount of the target.

The method not only can realize the accurate control and feedback of the visual angle and orientation of the camera monitoring equipment, but also can realize the accurate matching calculation with the pixel coordinate, so that the target can return to the center of the picture and reach the set proportion.

Drawings

FIG. 1 is a schematic diagram of the connection of external hardware modules according to the present invention;

FIG. 2 is a logic diagram of the external hardware module according to the present invention;

FIG. 3 is a diagram of a camera imaging geometry model;

fig. 4 is a diagram of the effect of the present invention.

Detailed Description

The invention will be further illustrated and described with reference to specific embodiments. The technical features of the embodiments of the present invention can be combined correspondingly without mutual conflict.

Zoom lens control protocol improvement

In the prior art, the precise focal length control and feedback of the zoom lens cannot be realized through the native control protocol PELCO of the zoom lens, so that the protocol is improved and gear control is added in the embodiment.

The gear control method divides the focal length of the zoom lens into N gears approximately uniformly, the minimum value is recorded as 1 gear, and after the zoom lens is moved to the increasing direction for time delta t, the minimum value is recordedAnd 2, and the like until the maximum value is N gear. And delta T is the time required by adjacent gear shifting, and the time from the maximum value to the minimum value of the focal length of the motorized zoom lens (and conversely, the time is still consistent) is recorded as T, then

And establishing a lens gear-visual angle comparison table.

The external hardware module is schematically connected as shown in fig. 1, and is located between the camera tracking host and the zoom lens. The camera shooting tracking host sends a gear set value to the external hardware module through a user-defined protocol, the external hardware module generates a focal length moving instruction to control the zoom lens by comparing with the current gear, and the current gear is periodically fed back to the camera shooting tracking host.

Based on the PELCO-D protocol, a gear transmission/feedback command format is designed, as shown in Table 1, wherein byte 1 is a synchronization byte, byte 2 is a device number, byte 4 is information category information, byte 6 is gear information, and byte 7 is a check value.

TABLE 1 Camera tracking host gear position transmission/feedback command format

Command

Byte1

Byte2

Byte3

Byte4

Byte5

Byte6

Byte7

Gear setting

0xFF

Device numbering

0x00

0x4f

0x00

Gear position

Check Sum

Gear pass-back

0xFF

Device numbering

0x00

0x5d

0x00

Gear position

Check Sum

The internal logic of the external hardware module is shown in fig. 2, and the main part is serial port interrupt and timer interrupt. And the serial port interrupt receives a command from the camera tracking host, and the set gear n2 is updated after verification. Triggering once every delta t time by timed interruption, judging the current focal length control direction Dir after triggering, wherein a positive number indicates increasing, a negative number indicates decreasing, and 0 indicates stopping; if the value is not 0, the action is represented to have been executed for a delta t period, and corresponding addition and subtraction operations need to be carried out on the current state n 1; calculating a gear potential difference delta n which is n2-n1, wherein the positive delta n represents that the focal length needs to be increased, the negative delta n represents that the focal length needs to be decreased, and 0 represents that the focal length value is just right; the comprehensive judgment of the Δ n and the Dir has 9 cases, and the classified cases have 5 cases:

a) delta n is greater than 0, Dir is less than or equal to 0, the current focal length control direction is inconsistent with the gear change requirement, the gear requirement is increased, and an 'increase' instruction is sent;

b) delta n is less than 0, Dir is more than or equal to 0, the current focal length control direction is inconsistent with the gear change requirement, the gear requirement is reduced, and a reduction command is sent;

c) Δ n is 0, Dir! When the current gear is consistent with the set gear, and the focal length control direction is not stopped, a 'stop' instruction is sent;

d) delta n x Dir is more than 0, the current focal length control direction is consistent with the gear change requirement, the current change is kept, and no command is sent;

e) when Δ n is Dir 0, the focus control is finished and no command is sent.

And changing the current focal length control direction according to the corresponding value, and simultaneously returning the current gear value to the camera monitoring host, wherein the returning format is shown in a 'gear return' column in table 1.

Second, calculating and correcting target ratio

The camera imaging geometry model is shown in fig. 3, where the camera horizontal view angle is θ, the surveillance distance is L, the target diameter is D, and the surveillance range diameter is D. For convenience, the current value of a parameter is indexed by 1, e.g., θ₁And D₁While the parameter set point index is 2, e.g. θ₂And D₂。

The resolution of the acquired image is recorded as [ I ]_h,I_v](h is a horizontal axis, v is a vertical axis). The target pixel coordinate value is [ [ L ]_h,L_v]，[R_h,R_v]]Wherein the former is the coordinate value of the pixel at the upper left corner of the target calibration frame, and the latter is the value of the lower right corner.

The geometric relationship is described as shown in equation (2.1):

let the ratio of the target in the image be P, and similarly, let the current value of the target ratio be P₁The set value is set to P₂，P₁And P₂As shown in equation (2.2):

by eliminating the monitoring distance L and the monitoring range diameter D in the formula (2.1) and the formula (2.2), the formula (2-3) can be obtained:

wherein the target ratio current value P₁As shown in (2.4):

thus, according to the current value P of the target ratio₁The current value theta of the horizontal visual angle of the camera is the target ratio set value P₂Converting into a set value theta of horizontal visual angle of the camera₂And then, the visual angle control is realized according to the improved control protocol of the motorized zoom lens, so that the target ratio is proper. Target ratio set point P₂The effect is better when the concentration is set to be 8-13%.

Third, target offset calculation and correction

The resolution of the acquired image is recorded as [ I ]_h,I_v](h is a horizontal axis, v is a vertical axis). The target pixel coordinate value is [ [ L ]_h,L_v]，[R_h,R_v]]Wherein the former is the coordinate value of the pixel at the upper left corner of the target calibration frame, and the latter is the value of the lower right corner. The imaging visual angle of the camera is [ theta ]_h,θ_v]。

Target center coordinate [ C ]_h,C_v]As shown in equation (3.1):

the image center coordinates are

The target pixel shift amount is the difference between the target center and the image center coordinate

Then, through the geometric relation, the target angle offset [ O ] can be deduced_h,O_v]As shown in equation (3.2):

θ_vcan be expressed as formula (3.3):

let theta equal to theta_hFormula (3.4) can be simplified from formula (3.1) to formula (3.3):

rotation angle of the rotary head, i.e. target angle offset [ O ]_h,O_v]The parameter involved includes the image resolution [ I ] as shown in equation (3.4)_h,I_v]Target pixel coordinate value [ [ L ]_h,L_v]，[R_h,R_v]]And a camera horizontal imaging view angle theta. The current rotating cradle head often supports angle accurate control, so the current rotating cradle head can be directly connected with a camera tracking host or forwarded through an external hardware module.

The technical scheme of the invention is used for tracking and controlling the actual unmanned aerial vehicle, and the effect is shown in figure 4.

The method comprises the steps of returning the target to the center of the picture and enabling the proportion of the target in the picture to reach the set proportion, wherein the execution sequence between the two steps is not required. In the embodiment shown in fig. 4, the step of returning the object to the center of the screen is performed first. In fig. 4, lines a and b are visible and infrared tracking for a four-axis drone, and lines c and d are visible and infrared tracking for a six-axis drone. The 1 st column completes the identification of the unmanned aerial vehicle, calibrates the unmanned aerial vehicle and acquires the pixel coordinates of the unmanned aerial vehicle; column 2, completing the matching control of the rotating pan-tilt, and returning the target to the center of the picture; column 3 completes zoom camera control to make the ratio of the target in the frame reach the set ratio.

The effect graph shows that the invention can realize the quantitative stepping control and the periodic feedback of the zoom value of the zoom lens and the visual angle value of the camera by improving the native PELCO control protocol, can realize the accurate and stable control of the camera monitoring equipment, and can ensure that the visual angle of the equipment is self-matched with the target size and the visual angle direction is self-matched with the target movement amount.

The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. a camera monitoring device matching control method based on pixel coordinates, is characterized in that comprising the steps:

S01: The camera obtains the captured image; calibrates the target and obtains its pixel coordinates;

S02: execute steps S03 and S04 in sequence, or execute S04 first and then execute S03; complete the pixel coordinate-based camera monitoring device matching control;

The described S03 is: according to the current value P ₁ of the target ratio and the current value θ ₁ of the horizontal viewing angle of the camera, converting the set value P ₂ of the target ratio into the set value θ ₂ of the horizontal viewing angle of the camera;

Control the zoom lens of the camera according to the obtained set value θ ₂ of the horizontal angle of view of the camera, so that the proportion of the target in the picture reaches the set ratio;

The method for controlling the zoom lens of the camera is specifically:

The camera includes a camera tracking host, an external hardware module and a zoom lens;

The focal length of the zoom lens is evenly divided into N gears, and Δt is the time required for the adjacent gears to change; then

T is the moving time of the focal length of the electric zoom lens from the maximum value to the minimum value or from the minimum value to the maximum value, and establishes a lens gear-viewing angle comparison table;

According to the lens gear-viewing angle comparison table, the camera tracking host converts the camera horizontal viewing angle setting value θ2 into the gear setting gear _n2 ; the camera tracking host sends gear setting information to the external hardware module, and the external hardware module compares The current gear generates a focal length movement command to control the zoom lens, and periodically feeds back the current gear to the camera tracking host;

The step S04 is: obtaining the target angle offset according to the center coordinates of the target and the center coordinates of the collected image; adjusting the camera according to the target angle offset to make the target return to the center of the screen.

2. the camera monitoring device matching control method based on pixel coordinates according to claim 1, is characterized in that the command format that camera tracking host communicates with external hardware module is:

Order byte 1 byte 2 byte 3 byte 4 byte 5 byte 6 byte 7 Gear setting 0xFF device ID 0x00 0x4f 0x00 gear Check Sum Gear return 0xFF device ID 0x00 0x5d 0x00 gear Check Sum

Among them, byte 1 is the synchronization byte, byte 2 is the device number, byte 4 is the information type information, byte 6 is the gear information, and byte 7 is the check value Check Sum.

3. The camera monitoring device matching control method based on pixel coordinates according to claim 1, is characterized in that the described external hardware module receives the order from the camera tracking host, and the set position n2 is updated after the verification;

Every Δt time, the external hardware module makes the following judgment: Judging the current focal length control direction Dir, positive number means increase, negative number means decrease, 0 means stop; calculate the gear difference Δn=n2-n1, Δn is positive means the focal length Need to increase, negative means the focal length needs to be reduced, 0 means the focal length value is just right; n1 is the current gear;

Change the current focus control direction according to the values corresponding to Δn and Dir, and return the current gear value to the camera monitoring host at the same time.

4. The pixel coordinate-based camera monitoring device matching control method according to claim 3, wherein the changing the current focus control direction according to the corresponding values of Δn and Dir is specifically:

a) Δn>0, Dir≤0, the current focal length control direction is inconsistent with the gear shift requirement, and the gear requirement becomes larger, the "increase" command should be sent;

b) Δn<0, Dir≥0, the current focal length control direction is inconsistent with the gear shift requirement, and the gear position requirement becomes smaller, a "decrease" command should be sent;

c) Δn=0, Dir! =0, the current gear is consistent with the set gear, and the focus control direction is not stop, and the "stop" command should be sent;

d) Δn*Dir>0, the current focal length control direction is consistent with the gear change requirements, the current change should be maintained, and no command should be sent;

e) Δn=Dir=0, the focus control ends, and no command is sent.

5. The pixel coordinate-based camera monitoring device matching control method according to claim 1, wherein the method for obtaining the target angle offset is:

The resolution of the collected image is recorded as [I _h , I _v ], where h is the horizontal axis and v is the vertical axis; the target pixel coordinate values are [[L _h , L _v ], [R _h , R _v ]], where [ L _h , L _v ] is the pixel coordinate value of the upper left corner of the target calibration frame, [R _h , R _v ] is the lower right corner value; the camera imaging angle is [θ _h , θ _v ],

The target center coordinates [C _h , C _v ] are shown in formula (3.1):

The coordinates of the center of the image are

The target pixel offset is the difference between the target center and the image center coordinates

Through the geometric relationship, the target angle offset [O _h ,O _v ] is derived, as shown in formula (3.2):

θ _v is expressed as Equation (3.3):

The camera horizontal imaging angle θ _h is recorded as θ, and formula (3.4) can be obtained by simplifying formula (3.1) to formula (3.3):