WO2022057800A1

WO2022057800A1 - Gimbal camera, gimbal camera tracking control method and apparatus, and device

Info

Publication number: WO2022057800A1
Application number: PCT/CN2021/118345
Authority: WO
Inventors: 李林芝; 郭一民
Original assignee: 华为技术有限公司
Priority date: 2020-09-16
Filing date: 2021-09-14
Publication date: 2022-03-24

Abstract

Provided are a gimbal camera, a gimbal camera tracking control method and apparatus, and a device. When a target object satisfies a preset rule or a target object tracking instruction indicated by a user is received, an error value between the position of the target object in the current frame image and a target position of the current frame image is analyzed to obtain a change in a motion posture of the target object; and an error distance variation value between the target object in the current frame image and the target object in any of frame images before the current frame image is analyzed to obtain a relative spatial position of the target object, so as to analyze the current speed change status of the target object in multiple dimensions and calculate spatial positions of the target object and a holder of a gimbal camera. By accurately adjusting a photographing angle of a camera body, the position of the target object in a subsequent frame image is corrected in time, so that the target object is always kept at a target position of the subsequent frame image, thus solving the problem of target object tracking loss due to a violent motion change of the target object.

Description

PTZ camera, tracking control method, device and device for PTZ camera

This application claims the priority of the Chinese patent application with the application number 202010974180.9 and the application title "An intelligent control method for automatic tracking of a spherical camera", which was submitted to the China Patent Office on September 16, 2020. This application claims to be filed on November 2020. The priority of the Chinese patent application filed with the Chinese Patent Office on March 06, the application number is 202011232127.8, and the application name is "PTZ camera, tracking control method, device and equipment for PTZ camera", the entire content of which is incorporated by reference in this application middle.

technical field

The present application relates to the technical field of video surveillance, and in particular, to a pan-tilt camera and a tracking control method, device and device for a pan-tilt camera.

Background technique

The PTZ camera is a camera that supports adjusting the shooting direction and the focal length of the lens. The PTZ camera observes the surrounding objects in all directions, and the user's field of view is not limited by factors such as the installation position, installation angle and lens angle of the PTZ camera.

Therefore, in security monitoring application scenarios, PTZ cameras are often used to automatically track moving target objects (such as motor vehicles, non-motor vehicles, pedestrians, etc.), so that the target object remains in the set area of the displayed image, such as the central area. .

Usually, the movement of the target object will cause the target object not to be in the set area of the displayed image, resulting in output errors. The PTZ in the PTZ camera needs an automatic control method to eliminate this error. At present, a proportional integral derivative (PID) controller can be used in the gimbal, and a certain linear proportional relationship is formed between the output signal of the controller and the output error signal, and the steady-state error can be eliminated.

However, the change of the moving speed and running direction of the target object is cumbersome and unruly, causing the target object to change non-linearly, which is essentially different from the linear change in the aforementioned automatic control method to eliminate errors, which is easy to lead to the aforementioned automatic control method. The error elimination effect becomes poor, and even the target object is lost.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a pan-tilt camera, a tracking control method, device, and device for a pan-tilt camera, which can solve the problem of tracking and losing the target object due to the drastic changes in the motion of the target object, and realize the accurate tracking of the target with drastic changes in motion. object, so that the target object can always be displayed at the target position in subsequent frame images.

In a first aspect, an embodiment of the present application provides a pan-tilt camera, including a camera body, a pan-tilt, and a pan-tilt controller.

When the target object conforms to the preset rules or receives an instruction to track the target object instructed by the user, the PTZ controller is used to obtain the first frame of images and the second frame of images collected by the camera body. The first frame of images is collected late at the acquisition moment of the second frame of image. The pan-tilt controller is further configured to determine a first distance difference between the position of the target object in the first frame of image and the target position of the first frame of image. The pan-tilt controller is further configured to determine the distance change value between the position of the target object in the first frame image and the position of the target object in the second frame image. The PTZ controller is also used to adjust the shooting angle of the camera body based on the first distance difference value and the distance change value, so that the target object is displayed at the target position of the third frame image collected by the camera body, and the third frame image is collected. The time is later than the acquisition time of the first frame image.

Wherein, the first distance difference value can represent the distance error that the position of the target object displayed in the first frame image deviates from the target position of the first frame image, that is, the first distance difference value can indicate the movement posture change of the target object.

The distance change value can represent the position change rate of the target object from the second frame image to the first frame image, that is, the distance change value can indicate the relative spatial position of the target image.

Through the PTZ camera of the first aspect, the error value between the position of the target object in the current frame image and the target position of the current frame image and the difference between the target object in the current frame image and any frame image before the current frame image are analyzed The change value of the error distance between the two can accurately obtain the information such as the movement posture change and relative spatial position of the target object, so as to analyze the current speed change of the target object in multiple dimensions. Therefore, the spatial position between the target object and the PTZ camera is calculated, and the shooting angle of the camera body is precisely adjusted, so as to correct the position of the target object in the subsequent frame images in time, so that the target object is always kept at the target of the subsequent frame images. Position, solves the situation that the tracking of the target object is lost due to the drastic changes in the motion of the target object, improves the accuracy of tracking the target object with drastic changes in motion, and improves the tracking ability of the target object with drastic changes in motion.

In a possible design, the gimbal controller is specifically configured to obtain a control quantity parameter based on the first distance difference value and the distance change value, and the control quantity parameter includes the distance change value of the gimbal in the horizontal direction and the change value of the gimbal in the horizontal direction. The distance change value in the vertical direction; control the first motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the horizontal direction, and control the second motor in the gimbal to change according to the distance of the gimbal in the vertical direction value to rotate the gimbal to adjust the shooting angle of the camera body.

As a result, the pan-tilt server changes the rotation angle of the pan-tilt in the horizontal and vertical directions, and can precisely adjust the posture of the pan-tilt, thereby adjusting the shooting angle of the camera body, so that the target object can be displayed in the subsequent frame images. target location.

In a possible design, the control quantity parameter further includes a change value of the focal length of the camera body in the direction of the optical axis. The pan-tilt controller is also used to adjust the focal length of the camera body according to the focal length change value of the camera body in the optical axis direction, so that the target object is displayed at the target position of the third frame image.

Therefore, in addition to the distance change value of the gimbal in the horizontal direction and the distance change value of the gimbal in the vertical direction, the gimbal server can also learn the current movement line of the target object based on the first distance difference and the distance change value. The magnification of the angular velocity change of the speed on the spherical surface of the camera body, and the focal length change value of the camera body in the direction of the optical axis is obtained. Therefore, the PTZ server can adjust the shooting angle of the camera body and adjust the focal length of the camera in time, so that the target object can be displayed in the target position of the subsequent frame image, which improves the accuracy of tracking the target object that moves rapidly at a short distance, and improves the accuracy of tracking the target object in close range. The ability to track distances to fast-moving objects.

In a possible design, the pan-tilt controller is specifically used to perform fuzzy processing on the first distance difference value and the distance change value to obtain the proportional-integral-derivative PID control coefficient; based on the proportional-integral-derivative PID control coefficient, obtain the control parameter parameter .

In a possible design, the pan-tilt controller is specifically used to perform fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID correction coefficient; The product of the initial coefficients is determined as the proportional-integral-derivative PID control coefficient; based on the proportional-integral-derivative PID control coefficient, the control quantity parameter is obtained.

Based on the improved proportional-integral-derivative PID control coefficients in the above two methods, the proportional-integral-derivative PID server in the PTZ server can obtain more accurate control parameters, reducing the position of the target object in the subsequent frame image and the subsequent frame image. The position of the target object in the subsequent frame image is adjusted in time, so that the target object is displayed in the target position of the subsequent frame image, and the PTZ camera can track the target object more accurately.

In a possible design, the gimbal controller is also used to obtain the depression angle of the gimbal in the vertical direction at the acquisition moment of the first frame of image; since the depression angle of the gimbal in the vertical direction increases, the target can be determined The subject is rapidly approaching the gimbal camera. Therefore, when the depression angle is greater than or equal to the preset angle, the pan-tilt server can determine that the moving linear velocity of the target object increases, which means that the target object moves rapidly. Therefore, the PTZ server can increase at least one of the distance change value of the PTZ in the horizontal direction and the distance change value of the PTZ in the vertical direction, so that the control parameter becomes larger, and the increased control value is obtained. parameter; determine the increased control value parameter as the control value parameter. Therefore, the pan-tilt controller can control the distance of the pan-tilt to change in unit time to increase based on the control parameter parameters, so that the pan-tilt can track the fast-moving target object in time.

In a possible design, the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.

In a possible design, the pan-tilt controller is also used to determine the area change value of the target object in the first frame image and the second frame image. Since the proportion of the area of the target object in the image is increasing rapidly, it can be determined that the target object has a tendency to approach the PTZ camera quickly or deviate from the target position quickly. If the area of the target object in the image is small or even negatively increases, it can be determined that the target object has a tendency to quickly move away from the PTZ camera. Therefore, when the area change value is greater than the first threshold, or, when the area change value is less than the second threshold and the absolute value of the area change value is greater than the third threshold, the pan-tilt controller will determine the distance change value of the pan-tilt in the horizontal direction, the cloud Adjust at least one value of the distance change value of the stage in the vertical direction and the focal length change value of the camera body in the optical axis direction, so that the control amount parameter changes, and the adjusted control amount parameter is obtained; Determined as the control parameter. In this way, the pan-tilt controller can control the pan-tilt change in unit time based on the control parameter parameters, and can track the fast-moving target object in time.

In a possible design, the pan-tilt controller is also used to acquire the focal length of the camera body at the acquisition moment of the first frame of image.

Since the PTZ server can determine the focal plane where the target object is located, when the focal length of the camera body is large, the distance between the target object and the PTZ camera is far, and the rotation range of the PTZ is small, that is, the required control parameters need to be reduced. . Therefore, when the focal length of the camera body is greater than the fourth threshold, the pan/tilt controller changes the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the focal length of the camera body in the direction of the optical axis. At least one of the values is reduced so that the control quantity parameter becomes smaller, and the reduced control quantity parameter is obtained; the reduced control quantity parameter is determined as the control quantity parameter. Therefore, the pan-tilt controller can control the distance that the pan-tilt changes in unit time to become smaller based on the control parameter parameters, so that the pan-tilt can track the fast-moving target object in time.

Or, because the PTZ server can determine the focal plane of the target object, when the focal length of the camera body is small, the distance between the target object and the PTZ camera is short, and the rotation range of the PTZ is large, that is, the required control parameters need to be adjusted. Big. Therefore, when the focal length of the camera body is less than the fifth threshold, the pan-tilt controller changes the distance change value of the pan-tilt head in the horizontal direction, the distance change value of the pan-tilt head in the vertical direction, and the focal length change of the camera body in the optical axis direction. At least one of the values is increased so that the control variable parameter becomes larger, and the increased control variable parameter is obtained; the increased control variable parameter is determined as the control variable parameter. Therefore, the pan-tilt controller can control the distance of the pan-tilt to change in unit time to increase based on the control parameter parameters, so that the pan-tilt can track the fast-moving target object in time.

Therefore, the PTZ server can evaluate the current motion of the target object by reading parameters such as the depression angle of the PTZ in the vertical direction, the area change value of the target object in the first frame image and the second frame image, and the focal length of the camera body. The linear velocity is the magnification of the angular velocity change on the spherical surface of the camera body, which realizes the timely adjustment of the shooting angle of the camera body and the focal length of the camera body, so that the target object can display the target position in the subsequent frame images, so that the PTZ camera can be more Accurately track the target object that moves quickly at a short distance, which improves the tracking ability and accuracy of the target object that moves quickly at a short distance.

In a possible design, the pan-tilt controller is specifically configured to determine a second distance difference between the position of the target object in the second frame image and the target position of the second frame image; based on the first distance difference and the second distance difference to determine the distance change value.

In a possible design, the target position is the central area of any frame of images.

In a second aspect, an embodiment of the present application provides a tracking control method for a pan-tilt camera, which starts the method when a target object complies with a preset rule or receives an instruction to track the target object instructed by a user. The method includes: obtaining a first frame of images and a second frame of images collected by a camera body, where the collection time of the first frame image is later than the collection time of the second frame image. A first distance difference between the position of the target object in the first frame image and the target position of the first frame image is determined. A distance change value between the position of the target object in the first frame image and the position of the target object in the second frame image is determined. Based on the first distance difference value and the distance change value, adjust the shooting angle of the camera body so that the target object is displayed at the target position of the third frame image collected by the camera body, and the collection time of the third frame image is later than that of the first frame image. Collection time.

In a possible design, adjusting the shooting angle of the camera body based on the first distance difference value and the distance change value includes: obtaining a control amount parameter based on the first distance difference value and the distance change value, and the control amount parameter includes a pan/tilt head The distance change value in the horizontal direction and the distance change value of the gimbal in the vertical direction. Control the first motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the horizontal direction, and control the second motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the vertical direction, so as to Adjust the shooting angle of the camera body.

In a possible design, the control quantity parameter further includes a change value of the focal length of the camera body in the direction of the optical axis. The method further includes: adjusting the focal length of the camera body according to the focal length change value of the camera body in the direction of the optical axis, so that the target object is displayed at the target position of the third frame image.

In a possible design, obtaining the control parameter parameters based on the first distance difference and the distance change value includes: performing fuzzy processing on the first distance difference and the distance change value to obtain the proportional-integral-derivative PID control coefficient. Based on the proportional-integral-derivative PID control coefficients, the control parameters are obtained.

In a possible design, obtaining the control parameter parameters based on the first distance difference value and the distance change value includes: performing fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID correction coefficient. The product of the proportional integral derivative PID correction coefficient and the proportional integral derivative PID initial coefficient is determined as the proportional integral derivative PID control coefficient. Based on the proportional-integral-derivative PID control coefficients, the control parameters are obtained.

In a possible design, the method further includes: at the acquisition moment of the first frame of image, acquiring the depression angle of the pan/tilt head in the vertical direction. When the depression angle is greater than or equal to the preset angle, at least one of the distance change value of the gimbal in the horizontal direction and the distance change value of the gimbal in the vertical direction is increased to obtain the increased control parameter; The larger control quantity parameter is determined as the control quantity parameter.

In a possible design, the method further includes: determining area change values of the target object in the first frame image and the second frame image. When the area change value is greater than the first threshold, or, when the area change value is smaller than the second threshold and the absolute value of the area change value is greater than the third threshold, the distance change value of the gimbal in the horizontal direction and the distance change of the gimbal in the vertical direction Adjust at least one of the distance change value and the focal length change value of the camera body in the direction of the optical axis to obtain the adjusted control amount parameter; the adjusted control amount parameter is determined as the control amount parameter.

In a possible design, the method further includes: at the acquisition moment of the first frame of image, acquiring the focal length of the camera body. When the focal length of the camera body is greater than the fourth threshold, change at least one of the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the focal length change value of the camera body in the optical axis direction Adjust it to a smaller value to obtain the reduced control quantity parameter; determine the reduced control quantity parameter as the control quantity parameter.

In a possible design, the method further includes: at the acquisition moment of the first frame of image, acquiring the focal length of the camera body. When the focal length of the camera body is less than the fifth threshold, at least one of the distance change value of the pan/tilt head in the horizontal direction, the distance change value of the pan/tilt head in the vertical direction, and the focal length change value of the camera body in the optical axis direction Increase the control value to obtain the control value parameter after the increase; determine the control value parameter after the increase as the control value parameter.

In a possible design, determining the distance change value between the position of the target object in the first frame image and the position of the target object in the second frame image includes: determining the position of the target object in the second frame image A second distance difference from the target position of the second frame image. A distance change value is determined based on the first distance difference value and the second distance difference value.

For the tracking control method of the pan-tilt camera provided in the second aspect and the possible designs of the second aspect, the beneficial effects of the tracking control method can refer to the beneficial effects brought by the first aspect and the possible implementations of the first aspect. The effect will not be repeated here.

In a third aspect, an embodiment of the present application provides a tracking control device for a pan-tilt camera, including: an acquisition module configured to acquire information about the camera body when the target object conforms to a preset rule or receives an instruction to track the target object instructed by a user. For the first frame image and the second frame image collected, the collection time of the first frame image is later than the collection time of the second frame image. The determining module is configured to determine the first distance difference between the position of the target object in the first frame image and the target position of the first frame image. The determining module is further configured to determine a distance change value between the position of the target object in the first frame of image and the position of the target object in the second frame of image. The adjustment module is used to adjust the shooting angle of the camera body based on the first distance difference value and the distance change value, so that the target object is displayed at the target position of the third frame image collected by the camera body, and the collection time of the third frame image is later than The acquisition moment of the first frame of image.

In a possible design, the adjustment module is specifically configured to obtain a control amount parameter based on the first distance difference value and the distance change value, and the control amount parameter includes the distance change value of the pan/tilt in the horizontal direction and the pan/tilt in the vertical direction The distance change value on the PTZ; control the first motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the horizontal direction, and control the second motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the vertical direction. The gimbal rotates to adjust the shooting angle of the camera body.

In a possible design, the control quantity parameter further includes a change value of the focal length of the camera body in the direction of the optical axis. The adjustment module is further configured to adjust the focal length of the camera body according to the focal length change value of the camera body in the direction of the optical axis, so that the target object is displayed at the target position of the third frame image.

In a possible design, the adjustment module is specifically configured to perform fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID control coefficient; based on the proportional-integral-derivative PID control coefficient, obtain a control parameter parameter. Or, an adjustment module, which is specifically used to perform fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID correction coefficient; and determine the product of the proportional-integral-derivative PID correction coefficient and the proportional-integral-derivative PID initial coefficient as proportional-integral Differential PID control coefficient; based on the proportional integral derivative PID control coefficient, the control parameter is obtained.

In a possible design, the device further includes: a first acquisition module, configured to acquire the depression angle of the pan/tilt head in the vertical direction at the acquisition moment of the first frame of image. The adjustment module is also used to increase at least one of the distance change value of the gimbal in the horizontal direction and the distance change value of the gimbal in the vertical direction when the depression angle is greater than or equal to the preset angle, to obtain the increased value. Control quantity parameter; determine the enlarged control quantity parameter as the control quantity parameter.

In a possible design, the determining module is further configured to determine the area change value of the target object in the first frame image and the second frame image. The adjustment module is also used for adjusting the distance change value of the pan/tilt in the horizontal direction, the cloud value when the area change value is greater than the first threshold value, or when the area change value is smaller than the second threshold value and the absolute value of the area change value is greater than the third threshold value. Adjust at least one of the distance change value of the stage in the vertical direction and the focal length change value of the camera body in the optical axis direction to obtain the adjusted control quantity parameter; the adjusted control quantity parameter is determined as the control quantity parameter.

In a possible design, the device further includes: a second acquisition module, configured to acquire the focal length of the camera body at the acquisition moment of the first frame of image.

The adjustment module is also used to change the distance change value of the gimbal in the horizontal direction, the distance change value of the gimbal in the vertical direction, and the focal length change of the camera body in the direction of the optical axis when the focal length of the camera body is greater than the fourth threshold At least one of the values is reduced to obtain a reduced control quantity parameter; the reduced control quantity parameter is determined as a controlled quantity parameter.

Or, the adjustment module is also used to adjust the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the distance change of the camera body in the optical axis direction when the focal length of the camera body is smaller than the fifth threshold. At least one of the focal length change values is increased to obtain an increased control amount parameter; the increased control amount parameter is determined as a control amount parameter.

In a possible design, the determining module is specifically configured to determine a second distance difference between the position of the target object in the second frame image and the target position of the second frame image; based on the first distance difference and the first distance difference Two distance difference, determine the distance change value.

For the tracking control device of the pan-tilt camera provided in the third aspect and the possible designs of the third aspect, the beneficial effects of the tracking control device can refer to the beneficial effects brought by the possible implementations of the second aspect and the second aspect. The effect will not be repeated here.

In a fourth aspect, the present application provides an electronic device, comprising: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory to enable the electronic device to execute the second aspect and any possibility of the second aspect The tracking control method of the PTZ camera in the design.

In a fifth aspect, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen, and a sensor; the chip system includes: a processor; when the processor executes the computer instructions stored in the memory, the electronic device executes the first The tracking control method of the PTZ camera in the second aspect and any possible design of the second aspect.

In a sixth aspect, the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by the processor to enable the electronic device to implement the second aspect and the PTZ in any possible design of the second aspect. Camera tracking control method.

In a seventh aspect, the present application provides a computer program product, comprising: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and at least one processor Executing the execution instruction enables the electronic device to implement the second aspect and the tracking control method for the pan-tilt camera in any possible design of the second aspect.

Description of drawings

1A-1B are schematic structural diagrams of a pan-tilt camera provided by an embodiment of the application;

2 is a schematic flowchart of a tracking control method for a PTZ camera provided by an embodiment of the present application;

3 is a schematic diagram of a proportional-integral-derivative PID controller using fuzzy control provided by an embodiment of the present application;

4 is a schematic diagram of a scene of a tracking control method for a PTZ camera provided by an embodiment of the present application;

FIG. 5 is a schematic scene diagram of a tracking control method for a PTZ camera provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a tracking control device for a pan-tilt camera according to an embodiment of the present application.

detailed description

The present application provides a pan-tilt camera, a tracking control method for a pan-tilt camera, a tracking control device for a pan-tilt camera, an electronic device, a computer storage medium, a computer program product, and a chip system, which can accurately know the movement posture change of a target object, relative Spatial position and other information, multi-dimensional analysis of the speed change of the target object, so as to solve the spatial position between the target object and the PTZ camera, and accurately adjust the shooting angle of the camera body, so as to correct the target object in subsequent frame images The position of the target object is always kept at the target position of the subsequent frame image, which solves the situation that the target object tracking is lost due to the drastic changes in the motion of the target object, improves the tracking accuracy of the target object with drastic changes in motion, and improves the accuracy of the target object. The ability to track objects with drastic changes in motion.

Please refer to FIGS. 1A-1B. FIGS. 1A-1B are schematic structural diagrams of a pan-tilt camera according to an embodiment of the present application. As shown in FIGS. 1A-1B , the pan-tilt camera in this embodiment of the present application may include: a camera body 101 , a pan-tilt 102 , and a pan-tilt controller 103 .

The camera body 101 and the pan-tilt controller 103 are communicatively connected, and the pan-tilt controller 103 and the pan-tilt 102 are electrically connected. The camera may be a dome camera (spherical camera) or a camera connected to the external pan/tilt 102 . The pan-tilt controller 103 may be integrated in the camera body 101 , or may be provided separately from the camera body 101 . The camera body 101 and the pan/tilt head 102 may be integrally provided, or may be provided separately. The embodiments of the present application do not limit the foregoing contents.

The camera body 101 includes a lens, a sensor and a processor.

The function of the lens is to present the light image of the observed target on the sensor of the camera. The lens combines various optical parts (reflectors, transmission mirrors, prisms) of different shapes and different media (plastic, glass or crystal) in a certain way, so that after the light is transmitted or reflected by these optical parts, according to people's expectations. It is necessary to change the transmission direction of the light to be received by the receiving device to complete the optical imaging process of the object. Generally speaking, each lens is composed of multiple groups of lenses with different curved curvatures combined at different intervals. The focal length of the lens is determined by the selection of indicators such as spacing, lens curvature, and light transmittance. The main parameters of the lens include: effective focal length, aperture, maximum image plane, field of view, distortion, relative illumination, etc. The value of each index determines the overall performance of the lens.

A sensor (also known as an image sensor) is a device that converts an optical image into an electronic signal, and is widely used in digital cameras and other electronic and optical devices. Common sensors include: charge-coupled device (CCD) and complementary metal oxide semiconductor (complementary MOS, CMOS). Both CCD and CMOS have a large number (eg, tens of millions) of photodiodes, each photodiode is called a photosensitive cell, and each photosensitive cell corresponds to a pixel. During exposure, the photodiode converts the light signal into an electrical signal containing brightness (or brightness and color) after receiving light, and the image is reconstructed accordingly. Bayer array is a common image sensor technology that can be used in CCD and CMOS. Bayer array uses Bayer color filter to make different pixels only sensitive to one of the three primary colors of red, blue and green. These pixels are interleaved and then interpolated by demosaicing to restore the original image. Bayer arrays can be applied to CCD or CMOS, and sensors using Bayer arrays are also called Bayer sensors. In addition to the Bayer sensor, there are also sensor technologies such as X3 (developed by Foveon). X3 technology uses three layers of photosensitive elements, each layer records one of the color channels of RGB, so it can capture all colors on one pixel. Image sensor.

A processor (also known as an image processor) is used to perform operations such as digital signal processing, image signal processing (ISP), and encoding.

The pan/tilt 102 is a supporting device for installing and fixing the camera body 101 . With the rotation of the pan/tilt 102, the camera body 101 can be driven to rotate, thereby changing the rotation angle of the camera body 101, so that the camera body 101 can automatically scan a predetermined area. The gimbal 102 can be divided into a horizontal rotating gimbal that can only rotate left and right, and an omnidirectional gimbal that can rotate left and right as well as up and down. The pan/tilt head 102 may be a mounting platform composed of a first motor and a second motor. Wherein, the first motor can be used to realize the rotation of the pan-tilt 102 in the horizontal direction, and the second motor can be used to realize the rotation of the pan-tilt 102 in the vertical direction. And the first motor and the second motor may be AC motors or DC motors.

For convenience of description, FIG. 1B shows that the pan/tilt head 102 can rotate in the horizontal direction along the P-axis, and can also rotate in the vertical direction along the T-axis. Among them, the P axis (pan), called the horizontal rotation axis, is perpendicular to the horizontal plane. The T-axis (tilt), called the vertical axis of rotation, is parallel to the horizontal plane.

The pan-tilt controller 103 is configured to output control parameters to the pan-tilt 102 based on the tracking control method provided by the embodiment of the present application, so that the motor in the pan-tilt 102 controls the pan-tilt 102 to rotate based on the control-level parameters, so as to adjust the camera body Rotation angle of 101. And the pan-tilt controller 103 is also used for adjusting the focal length of the camera body 101 based on the control parameter. Wherein, the present application does not limit parameters such as the quantity and model of the PTZ controller 103 .

When the camera body 101 monitors the predetermined area in real time, the camera body 101 can transmit the collected images to the pan-tilt controller 103 so that the pan-tilt controller 103 can identify the target object in each frame of image. There may be one or more target objects, which is not limited in this embodiment of the present application.

In some embodiments, an algorithm module may be set in the pan-tilt controller 103, so that the pan-tilt controller 103 determines whether the target object conforms to the preset rule through the algorithm module.

The preset rules may include but are not limited to rules such as violating traffic rules, breaking into a private residence, or matching images of criminal suspects.

Therefore, when the target object complies with the preset rules, the pan-tilt controller 103 can trigger to execute the tracking control method provided by the embodiment of the present application, so as to correct the position of the target object in the subsequent frame images.

In other embodiments, the pan-tilt controller 103 may be connected to the terminal device in communication, and the pan-tilt controller 103 may receive an instruction to track the target object instructed by the user from the terminal device.

Alternatively, the pan-tilt controller 103 may be communicatively connected to the server, and the server may be communicatively connected to the terminal device, and the pan-tilt controller 103 may receive, through the server, an instruction to track the target object indicated by the user and sent by the terminal device.

Wherein, the present application does not limit the specific implementation manner of the instruction. For example, the user designates the target object as the target A, or the user designates the target object as the target appearing in each frame of the image at the target moment.

Therefore, when receiving the instruction to track the target object indicated by the user, the pan-tilt controller 103 can trigger to execute the tracking control method provided by the embodiment of the present application to correct the position of the target object in the subsequent frame images.

Please refer to FIG. 2 , which is a schematic flowchart of a tracking control method for a pan-tilt camera according to an embodiment of the present application. As shown in FIG. 2 , the PTZ server 103 is used as the main body of execution, and the current frame image is taken as an example of the first frame image, any frame image before the current frame image is taken as an example of the second frame image, and any frame image after the current frame image is taken as an example. One frame of image is illustrated by taking the third frame of image as an example. The tracking control method of the pan-tilt camera according to the embodiment of the present application may include:

S101. Obtain a first frame of image and a second frame of image collected by the camera body.

S102. Determine a first distance difference between the position of the target object in the first frame image and the target position of the first frame image.

Based on the foregoing description, the PTZ server 103 has learned the target object and the position of the target object in the subsequent frame images that needs to be corrected. Thus, the pan-tilt server 103 can determine the position of the target object in the first frame of image, and the target position of the first frame of image (ie, the position where the target object is correctly displayed in the first frame of image).

Wherein, the present application does not limit the specific position of the target position in the first frame of image. For example, the target position is the central area of the first frame image.

Therefore, the pan-tilt server 103 can obtain the first distance difference based on the position of the target object in the first frame of image and the target position of the first frame of image.

Wherein, the present application does not limit the implementation manner in which the PTZ server 103 obtains the first distance difference.

In some embodiments, the PTZ server 103 may equate the position of the target object in the first frame image with the center coordinates of the target object in the first frame image. Likewise, the pan-tilt server 103 may equate the target position of the first frame image with the center coordinates of the target position. Therefore, the pan-tilt server 103 can obtain the first distance difference by taking the difference between the aforementioned two center coordinates.

In other embodiments, the PTZ server 103 may be provided with a first initial model. The first initial model is a model of the distance difference between two irregular images. The pan-tilt server 103 may obtain a plurality of boundary coordinates from the position of the target object in the first frame image. Similarly, the pan-tilt server 103 can also take a plurality of boundary coordinates from the target position of the first frame of image. Therefore, the pan-tilt server 103 inputs the aforementioned two sets of boundary coordinates into the first initial model, and trains the first initial model to obtain the first distance difference.

It should be noted that the embodiments of the present application include but are not limited to the above two implementation manners.

S103: Determine a distance change value between the position of the target object in the first frame image and the position of the target object in the second frame image.

The PTZ server 103 may determine the position of the target object in the first frame of image and the position of the target object in the second frame of image. Wherein, the acquisition time of the first frame of image is later than the acquisition time of the second frame of image. For example, the second frame of image may be the previous frame of the first frame of image, or may be the Nth frame of image before the first frame of image, where N is greater than or equal to 2, which is not limited in this embodiment of the present application.

Therefore, the pan-tilt server 103 can obtain the distance change value based on the position of the target object in the first frame of image and the position of the target object in the second frame of image.

Wherein, the embodiment of the present application does not limit the specific implementation manner in which the pan-tilt server 103 obtains the distance change value.

In some embodiments, the pan-tilt server 103 determines the second distance between the position of the target object in the second frame image and the target position of the second frame image in the same manner as the first distance difference value obtained in step S101 difference. Therefore, the pan-tilt server 103 may use the form of the difference or quotient of the first distance difference and the second distance difference as the distance change value. Wherein, the target position of the first frame image is the same as the target position of the second frame image.

In other embodiments, the PTZ server 103 may be provided with a second initial model. Wherein, the second initial model is a model of the distance change value of the same target in the two images. The pan-tilt server 103 may obtain a plurality of boundary coordinates from the position of the target object in the first frame image. Similarly, the pan-tilt server 103 may also obtain a plurality of boundary coordinates from the position of the target object in the second frame image. Therefore, the PTZ server 103 inputs the aforementioned two sets of boundary coordinates into the second initial model, and trains the second initial model to obtain the distance change value.

It should be noted that the embodiments of the present application include but are not limited to the above two implementation manners. Moreover, steps S102 and S103 have no sequential order, and steps S102 and S103 may be performed simultaneously or sequentially.

S104. Based on the first distance difference value and the distance change value, adjust the shooting angle of the camera body, so that the target object is displayed at the target position of the third frame image.

On the one hand, the pan-tilt server 103 can know the distance error of the target object deviating from the target position of the first frame image based on the first distance difference value, so as to know the movement posture change of the target object.

On the other hand, the pan-tilt server 103 can know the position change rate of the target object from the second frame image to the first frame image based on the distance change value, so as to know the relative spatial position of the target object moving from the second frame image to the first frame image .

Therefore, based on the information such as the movement posture change and relative spatial position of the target object mentioned in the above two aspects, the PTZ server 103 can analyze the speed change of the target object in a multi-dimensional manner, which is beneficial to solve the problem between the target object and the PTZ camera. In order to precisely adjust the shooting angle of the camera body, to adjust the position of the target object in the third frame image, so that the target object can be displayed in the target position of the third frame image.

Wherein, the acquisition time of the third frame of image is later than the acquisition time of the first frame of image. For example, the third frame of image may be the next frame of the first frame of image, or may be the M-th frame of image after the first frame of image, where M is greater than or equal to 2, which is not limited in this embodiment of the present application.

The tracking control method of the PTZ camera provided by the present application, by analyzing the error value between the position of the target object in the current frame image and the target position of the current frame image and any one of the target object before the current frame image and the current frame image. The change value of the error distance between the frame images can accurately know the movement posture change, relative spatial position and other information of the target object, so as to analyze the current speed change of the target object in multiple dimensions. Therefore, the spatial position between the target object and the PTZ camera is calculated, and the shooting angle of the camera body is precisely adjusted, so as to correct the position of the target object in the subsequent frame images in time, so that the target object is always kept at the target of the subsequent frame images. Position, solves the situation that the tracking of the target object is lost due to the drastic changes in the motion of the target object, improves the accuracy of tracking the target object with drastic changes in motion, and improves the tracking ability of the target object with drastic changes in motion.

In the embodiment of the present application, the pan-tilt server 103 can obtain the control amount parameter based on the first distance difference value and the distance change value.

Those skilled in the art can understand that the pan/tilt 102 is a supporting device for installing and fixing the camera body 101 . Therefore, the rotation of the pan/tilt head 102 in the horizontal direction and the vertical direction can drive the camera body 101 to rotate in any direction within the mechanical movement range, so that the camera body 101 can monitor the predetermined area in real time.

Based on the above description, the control amount parameter can be used to adjust the posture of the gimbal 102 , so that the adjustment of the shooting angle of the camera body 101 can change with the change of the posture of the gimbal 102 .

Hereinafter, with reference to several embodiments, the specific implementation manner of obtaining the control parameter parameters by the PTZ server 103 is described.

In some embodiments, the PTZ server 103 obtains the proportional-integral-derivative PID control coefficient based on the first distance difference value and the distance change value. The PTZ server 103 can use a PID controller with proportional integral derivative PID control coefficients to perform fuzzy processing on the attitude of the PTZ 102 to obtain control parameters. Considering that the gimbal 102 can rotate in the horizontal direction and the vertical direction, the gimbal server 103 can decompose the control quantity parameter orthogonally along the P-axis and the T-axis into the components in the horizontal direction and the vertical direction, that is, The distance change value of the gimbal 102 in the horizontal direction and the distance change value of the gimbal 102 in the vertical direction.

In other embodiments, considering that the gimbal 102 can rotate in the horizontal direction and the vertical direction, the gimbal server 103 can decompose the first distance difference orthogonally along the P-axis and the T-axis into the horizontal direction and the vertical direction. The components of the two directions, and the orthogonal decomposition of the distance change value into the components of the two directions of the horizontal direction and the vertical direction.

The pan-tilt server 103 obtains the proportional-integral-derivative PID control coefficient in the horizontal direction based on the component of the first distance difference in the horizontal direction and the component of the distance change value in the horizontal direction. The pan-tilt server 103 can use a PID controller with proportional-integral-derivative PID control coefficients in the horizontal direction to perform fuzzy processing on the posture of the pan-tilt 102 to obtain the distance change value of the pan-tilt 102 in the horizontal direction.

The PTZ server 103 obtains the proportional-integral-derivative PID control coefficient in the vertical direction based on the vertical component of the first distance difference value and the vertical component of the distance change value. The PTZ server 103 can use a PID controller with proportional integral derivative PID control coefficients in the vertical direction to perform fuzzy processing on the attitude of the PTZ 102 to obtain the distance change value of the PTZ 102 in the vertical direction.

Therefore, based on the above two methods, the pan-tilt controller 103 can output the distance change value of the pan-tilt 102 in the horizontal direction to the first motor in the pan-tilt 102, so that the first motor in the pan-tilt 102 is in accordance with the position of the pan-tilt 102. The distance change value in the horizontal direction determines the speed change value of the first motor. The first motor in the gimbal 102 rotates the gimbal 102 based on the change value of the rotational speed of the first motor, and changes the rotation angle of the gimbal 102 in the horizontal direction, so that the shooting angle of the camera body 101 is adjusted according to the rotation of the gimbal 102 change by rotation.

In addition, the gimbal controller 103 can output the distance change value of the gimbal 102 in the vertical direction to the second motor in the gimbal 102 , so that the second motor in the gimbal 102 changes according to the distance of the gimbal 102 in the vertical direction value to determine the rotational speed change value of the second motor. The second motor in the gimbal 102 rotates the gimbal 102 based on the rotational speed change value of the second motor to change the rotation angle of the gimbal 102 in the vertical direction.

The embodiments of the present application do not limit parameters such as specific types, parameters, and models of the first motor and the second motor.

In this way, the pan-tilt server 103 can accurately adjust the posture of the pan-tilt 102 in the horizontal and vertical directions, so that the shooting angle of the camera body 101 is adjusted as the pan-tilt 102 rotates. In addition, the pan-tilt server 103 can also precisely adjust the posture of the pan-tilt 102 in the horizontal direction or the vertical direction, and it is not necessary to adjust the posture of the pan-tilt 102 in both directions.

In addition to the shooting angle of the camera body 101 , the pan-tilt server 103 can also change the focal length of the camera body 101 . That is, when the focal length of the camera body 101 changes, the position of the target object captured by the camera body 101 in the current frame image will change. Among them, the focal length of the camera body 101 refers to Z zoom, which is called zoom zoom. Zooming: changing the distance between the lens and the imaging surface to achieve the target object with a clear image, that is, the focal length of the objective lens remains unchanged, and the focal length of the eyepiece is changed to achieve the purpose of zooming. Zoom: Change the focal length of the lens, that is, change the angle of view, that is, the focal length of the eyepiece remains unchanged, and the focal length of the objective lens is changed to achieve the purpose of zooming.

Based on the foregoing description, the control parameter parameters can also be used to adjust the posture of the pan/tilt head 102 and the focal length of the camera body 101 , so that both the shooting angle of the camera body 101 and the focal length of the camera body 101 can be changed.

In some embodiments, in addition to the distance change value of the gimbal 102 in the horizontal direction and the distance change value of the gimbal 102 in the vertical direction, the control parameter may further include: the focal length of the camera body 101 in the optical axis direction change value.

The specific implementation of the distance change value of the PTZ server 103 in the horizontal direction and the distance change value of the PTZ 102 in the vertical direction can be found in the foregoing description, and will not be repeated here.

Thus, based on the first distance difference value and the distance change value, the pan-tilt server 103 can obtain the magnification of the angular velocity change of the current linear velocity of the target object on the spherical surface of the camera body 101, and obtain the focal length of the camera body 101 in the direction of the optical axis change value. Therefore, the pan-tilt controller 103 can precisely adjust the focal length of the camera body 101 according to the change value of the focal length of the camera body 101 in the optical axis direction, so that the target object is displayed at the target position of the third frame image, which improves the tracking speed at close range. The accuracy of moving target objects improves the tracking ability of fast moving target objects at close range.

In this embodiment of the present application, the PTZ server 103 is provided with a proportional-integral-derivative PID controller that adopts fuzzy control as exemplarily shown in FIG. 3 .

Below, with reference to several embodiments, the working principle of the fuzzy control of the proportional-integral-derivative PID controller is introduced.

In some embodiments, the PTZ server 103 inputs the first distance difference value and the distance change value into the proportional-integral-derivative PID controller. The proportional-integral-derivative PID controller calculates the first distance difference and the distance change value through the fuzzy algorithm, and can obtain the proportional-integral-derivative PID correction coefficients, namely Ka, Kb, Kc in Fig. 3 .

Therefore, the proportional-integral-derivative PID controller determines the product of the proportional-integral-derivative PID correction coefficient and the proportional-integral-derivative PID initial coefficient (that is, Kp, Ki, Kd in Fig. 3) as the proportional-integral-derivative PID control coefficient, that is, in Fig. 3 Ka*Kp, Kb*Ki, Kc*Kd. Among them, the proportional integral derivative PID control coefficient can affect the size of each control factor in the proportional integral derivative PID controller, and the proportional integral derivative PID initial coefficient is the control parameter initially set by the proportional integral derivative PID controller. The proportional-integral-derivative PID controller can obtain the control parameters based on the proportional-integral-derivative PID control coefficients.

In other embodiments, the PTZ server 103 inputs the first distance difference value and the distance change value into the proportional-integral-derivative PID controller. The proportional-integral-derivative PID controller calculates the first distance difference and the distance change value through the fuzzy algorithm, and can obtain the proportional-integral-derivative PID control coefficients, namely Ka*Kp, Kb*Ki, Kc*Kd in Figure 3.

Therefore, the proportional-integral-derivative PID controller can obtain the control quantity parameter based on the proportional-integral-derivative PID control coefficient.

Based on the improved proportional-integral-derivative PID control coefficients in the aforementioned two methods, the proportional-integral-derivative PID server in the PTZ server 103 can obtain more accurate control parameters, reducing the position of the target object in the subsequent frame images and the subsequent frame images. The position of the target object in the subsequent frame image is adjusted in time, so that the target object is displayed in the target position of the subsequent frame image, and the PTZ camera can track the target object more accurately.

Among them, the proportional-integral-derivative PID controller can adopt various methods, and the first distance difference value and the distance change value are calculated by the fuzzy algorithm, and the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient can be obtained.

In some embodiments, the proportional-integral-derivative PID controller can obtain the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient based on the first distance difference and the distance change value by looking up a table, as shown in FIG. 3 . Any of Ka, Kb, Kc, Ka*Kp, Kb*Ki, Kc*Kd.

Assuming that the variation range of the first distance difference value is [-100, 100], the variation range of the distance variation value is

[-200, 200], the control degree corresponding to the first distance difference value is shown in Table 1, and the control degree corresponding to the distance change value is shown in Table 2.

Table 1 Control degree corresponding to the first distance difference

[-100,-50)[-100,-50)	[-50,-10)[-50,-10)	[-10,10)[-10,10)	[10,50)[10,50)	[50,100][50,100]
误差反向过大Error reverse is too large	误差反向略大The error is slightly larger	误差较小less error	误差正向略大The error is slightly larger	误差正向过大Error is too large

Table 2 Control degree corresponding to distance change value

[-200,-50)[-200,-50)	[-50,-5)[-50,-5)	[-5,5)[-5,5)	[5,50)[5,50)	[50,200][50,200]
反向增长过快Reverse growth too fast	反向增长略快Reverse growth slightly faster	基本持平Flat	正向增长略快Positive growth slightly faster	正向增长过快Positive growth too fast

Based on Table 1 and Table 2, the PTZ server 103 can be divided into five control strength intervals, which can form a 5×5 fuzzy table, as shown in Table 3.

Table 3 Fuzzy table

In Table 3, the symbol "+" and the symbol "-" in the fuzzy table represent the control strength, respectively. Among them, one "+" means to increase the control strength, multiple "+" means that the control strength needs to continue to increase, one "-" means to weaken the control strength, and multiple "-" means that the control strength needs to continue to be weakened.

To sum up, based on Table 3, the proportional-integral-derivative PID controller can obtain a strategy for adjusting the control variable parameters, that is, when the first distance difference is large and the distance change value is large, the PTZ server 103 can increase the control variable parameters. Control strength. When the first distance difference is large and the distance change value is small, the PTZ server 103 can weaken the control strength of the control quantity parameter.

Therefore, the proportional-integral-derivative PID controller can obtain the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient based on the above strategy.

In other embodiments, the proportional-integral-derivative PID controller can obtain the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient based on the first distance difference value and the distance change value by means of a formula, as shown in FIG. 3 Any of Ka, Kb, Kc, Ka*Kp, Kb*Ki, Kc*Kd.

For example, the PTZ server 103 can substitute the first distance difference value and the distance change value into Formula 1 to obtain the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient:

Among them, a is the first distance difference, and the variation range of the first distance difference is [-A, A]. b is the distance change value, and the change range of the distance change value is [-2A, 2A]. m is the proportional-integral-derivative PID correction coefficient or the proportional-integral-derivative PID control coefficient.

It should be noted that the embodiments of the present application include, but are not limited to, methods of looking up tables or formulas to obtain proportional-integral-derivative PID correction coefficients or proportional-integral-derivative PID control coefficients.

In this embodiment of the present application, in addition to the first distance difference value and the distance change value, the PTZ server 103 can also adjust the shooting angle of the camera body 101 through other parameters, or adjust the shooting angle of the camera body 101 and the camera body 101 , so that the target object is displayed at the target position of the subsequent frame image.

Those skilled in the art can understand that as the depression angle of the gimbal 102 in the vertical direction increases, it can be determined that the target object is rapidly approaching the gimbal camera body. At this time, the current linear velocity of the movement of the target object is mapped to the rotational angular velocity of the gimbal 102 will increase significantly.

Please refer to FIG. 4 , which is a schematic diagram of a scene of a tracking control method for a pan-tilt camera provided by an embodiment of the present application. As shown in FIG. 4 , the pan/tilt 102 is located at the observation point A, the height of the observation point A from the horizontal plane aa is h, and the distance between the target object (illustrated by taking a car as an example in FIG. 4 ) and the observation point A in the horizontal direction is s. The line connecting the observation point A and the target object is the line of sight, the angle between the line of sight and the vertical direction of the observation point A is θ, and θ and the depression angle of the gimbal 102 in the vertical direction are complementary angles to each other.

Assuming that the target object approaches the observation point A at a constant speed along the horizontal plane aa, the sight of the gimbal 102 rotates with the movement of the target object, so the change of θ corresponds to the change of the depression angle of the gimbal 102 in the vertical direction.

Among them, θ can be obtained by formula 2:

Wherein, the relationship between the rotational angular velocity of the gimbal 102 and the linear velocity of the target object can be obtained by formula 3:

Among them, v _θ is the rotational angular velocity of the gimbal 102 , and v _s is the linear velocity of the target object.

Furthermore, as the target object approaches the observation point A, s gradually decreases,

factor gradually increases. Therefore, the current linear velocity of the target object is mapped to the rotational angular velocity of the pan/tilt head 102 , which will increase significantly.

Based on the above description, the pan-tilt controller 103 can acquire the depression angle of the pan-tilt 102 in the vertical direction at the acquisition moment of the first frame of image. When the depression angle is greater than or equal to the preset angle, the gimbal server 103 may increase at least one of the distance change value of the gimbal 102 in the horizontal direction and the distance change value of the gimbal 102 in the vertical direction according to a certain ratio, Make the control quantity parameter larger, and obtain the enlarged control quantity parameter. Therefore, the pan-tilt server 103 adjusts the posture of the pan-tilt 102 by using the increased control quantity parameter as the final control parameter, so that the distance that the pan-tilt 102 changes per unit time becomes larger, and the target that moves quickly at a short distance can be tracked in time. object.

Wherein, the embodiment of the present application does not limit the size of the preset angle. For example, the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.

Those skilled in the art can understand that when the area ratio of the target object in the image is rapidly increasing, the pan-tilt server 103 may determine that the target object has a tendency to rapidly approach the pan-tilt camera or rapidly deviate from the target position. In addition, under this trend, the pan-tilt server 103 can adjust the posture of the pan-tilt 102 in time in combination with the distance change value, so that the shooting angle of the camera body 101 changes in time. When the area ratio of the target object in the image is small or even negatively increases, the PTZ server 103 may determine that the target object tends to move away from the PTZ camera quickly.

Based on the above description, the pan-tilt controller 103 can determine the area ratio value of the target object in the first frame of image, and the area ratio value of the target object in the second frame image. Therefore, the pan-tilt controller 103 takes the form of the difference or quotient of the aforementioned two area ratios as the area change value of the target object in the first frame image and the second frame image.

When the area change value is greater than the first threshold, or, when the area change value is smaller than the second threshold and the absolute value of the area change value is greater than the third threshold, the pan/tilt controller 103 may, according to a certain proportion, measure the horizontal direction of the pan/tilt 102 in the horizontal direction. Adjust at least one of the distance change value, the distance change value of the gimbal 102 in the vertical direction, and the focal length change value of the camera body 101 in the direction of the optical axis, so that the control amount parameter changes, and the adjusted control amount parameter is obtained. . Therefore, the pan-tilt server 103 uses the adjusted control amount parameter as the final control amount parameter to adjust the shooting angle of the camera body 101 and the focal length of the camera body 101, so that the distance changed by the pan-tilt 102 per unit time can be tracked to the close range in time A fast-moving target object.

The embodiments of the present application do not limit the sizes of the first threshold, the second threshold, and the third threshold.

Those skilled in the art can understand that under the condition of the same angular velocity of the gimbal 102, the larger the Z zoom, the larger the focal length of the camera body 101, and the larger the linear velocity of the focal plane of the picture. In the case of the same linear velocity of the pan/tilt head 102, the larger the Z zoom, the larger the focal length of the camera body 101, and the smaller the angular velocity of the focal plane of the screen. And the Z magnification of the camera body 101 corresponds to the linear velocity of the focal plane of the screen. Therefore, the distance of the target object affects the size of the Z zoom of the camera body 101 . In general, the farther the distance is, the larger the Z zoom will be.

Please refer to FIG. 5 , which is a schematic diagram of a scene of a tracking control method for a pan-tilt camera provided by an embodiment of the present application. As shown in Figure 5, the PTZ camera is located at observation point B.

Suppose two target objects (cars A1 and A2 are used as examples in FIG. 5 ): one target object A1 needs the Z zoom of the camera body 101 to be z times, and the other target object A2 needs the Z zoom of the camera body 101 For 20z times, the distance ratio between the two target objects and the observation point A is about 1:20.

When the two target objects move the same distance laterally to the gimbal camera along the same bb direction, the rotation range of the gimbal 102 corresponding to the camera body 101 with a large Z zoom is smaller, and the camera body 101 with a small Z zoom corresponds to The rotation range of the gimbal 102 is relatively large.

Therefore, the pan-tilt server 103 can determine the focal plane where the target object is located. When the Z zoom of the camera body 101 is large, the distance between the target object and the pan-tilt camera is long, and the rotation of the pan-tilt 102 is small, that is, the Z zoom The control quantity parameter required by the pan/tilt 102 corresponding to the camera body 101 with a larger magnification is smaller than that required by the pan/tilt 102 corresponding to the camera body 101 with a smaller Z zoom.

The pan-tilt server 103 can determine the focal plane where the target object is located. When the Z zoom of the camera body 101 is small, the distance between the target object and the pan-tilt camera is short, and the rotation of the pan-tilt 102 is large, that is, the Z zoom is small. The control quantity parameter required by the pan/tilt 102 corresponding to the camera body 101 of 100 is larger than that required by the pan/tilt 102 corresponding to the camera body 101 with a larger Z zoom.

Based on the above description, the pan-tilt controller 103 acquires the focal length of the camera body 101 at the acquisition moment of the first frame of image.

When the focal length of the camera body 101 is greater than the fourth threshold, the pan/tilt controller 103 may change the distance change value of the pan/tilt head 102 in the horizontal direction, the distance change value of the pan/tilt head 102 in the vertical direction, and the camera body 101 according to a certain ratio. At least one of the focal length change values in the direction of the optical axis is reduced, so that the control quantity parameter becomes smaller, and the reduced control quantity parameter is obtained. Therefore, the pan/tilt server 103 uses the reduced control amount parameter as the final control amount parameter to adjust the shooting angle of the camera body 101 and/or the focal length of the camera body 101, so that the distance changed by the pan/tilt 102 per unit time becomes smaller and more /or the focal length of the camera body 101 is reduced, so that a target object that moves rapidly at a short distance can be tracked in time.

When the focal length of the camera body 101 is less than the fifth threshold, the pan/tilt controller 103 may change the distance change value of the pan/tilt head 102 in the horizontal direction, the distance change value of the pan/tilt head 102 in the vertical direction, and the camera body 101 according to a certain ratio. At least one of the focal length change values in the direction of the optical axis is increased, so that the control amount parameter is increased, and the increased control amount parameter is obtained. Therefore, the pan-tilt server 103 uses the increased control amount parameter as the final control amount parameter to adjust the shooting angle of the camera body 101 and/or the focal length of the camera body 101, so that the distance changed by the pan-tilt 102 per unit time becomes larger and/or Or the focal length of the camera body 101 becomes larger, so that a target object that moves rapidly at a short distance can be tracked in time.

The size of the fourth threshold and the fifth threshold is not limited in this embodiment of the present application.

To sum up, in the embodiment of the present application, the PTZ server 103 reads the depression angle of the PTZ 102 in the vertical direction, the area change value of the target object in the first frame image and the second frame image, and the focal length of the camera body 101, etc. parameter, can evaluate the current moving linear velocity of the target object on the spherical surface of the camera body 101 angular velocity change magnification, realize the adjustment of the shooting angle of the camera body 101 and the focal length of the camera body 101, so that the PTZ camera can track more accurately The target object that moves quickly at a short distance improves the tracking ability and accuracy of the target object that moves quickly at a short distance.

Exemplarily, the embodiments of the present application further provide a tracking control device for a pan-tilt camera.

Please refer to FIG. 6. FIG. 6 is a schematic structural diagram of a tracking control device for a pan-tilt camera according to an embodiment of the present application. As shown in FIG. 6 , the tracking control device of the PTZ camera according to the embodiment of the present application may include:

The acquisition module 11 is configured to acquire the first frame of images and the second frame of images collected by the camera body, and the collection time of the first frame of images is later than the collection time of the second frame of images.

The determining module 12 is used to determine the first position between the position of the target object in the first frame image and the target position of the first frame image when the target object conforms to the preset rules or receives an instruction to track the target object indicated by the user. distance difference.

The determining module 12 is further configured to determine a distance change value between the position of the target object in the first frame of image and the position of the target object in the second frame of image.

The adjustment module 13 is used to adjust the shooting angle of the camera body based on the first distance difference value and the distance change value, so that the target object is displayed at the target position of the third frame image collected by the camera body, and the third frame image is collected later. at the acquisition moment of the first frame of image.

In some embodiments, the adjustment module 13 is specifically configured to obtain a control amount parameter based on the first distance difference value and the distance change value, where the control amount parameter includes the distance change value of the pan/tilt in the horizontal direction and the pan/tilt in the vertical direction the distance change value; control the first motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the horizontal direction, and control the second motor in the gimbal to rotate the gimbal according to the distance change value of the gimbal in the vertical direction The stage is rotated to adjust the shooting angle of the camera body.

In some embodiments, the control quantity parameter further includes a focal length change value of the camera body in the direction of the optical axis. The adjustment module 13 is further configured to adjust the focal length of the camera body according to the focal length change value of the camera body in the optical axis direction, so that the target object is displayed at the target position of the third frame image.

In some embodiments, the adjustment module 13 is specifically configured to perform fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID control coefficient; and based on the proportional-integral-derivative PID control coefficient, obtain a control parameter parameter.

Or, the adjustment module 13 is specifically configured to perform fuzzy processing on the first distance difference value and the distance change value to obtain a proportional-integral-derivative PID correction coefficient; the product of the proportional-integral-derivative PID correction coefficient and the proportional-integral-derivative PID initial coefficient is determined as a proportional Integral-derivative PID control coefficient; based on the proportional-integral-derivative PID control coefficient, the control parameter is obtained.

In some embodiments, the acquisition module 11 is further configured to acquire the depression angle of the pan-tilt head in the vertical direction at the acquisition moment of the first frame of image. The adjustment module 13 is also used to increase at least one value of the distance change value of the gimbal in the horizontal direction and the distance change value of the gimbal in the vertical direction when the depression angle is greater than or equal to the preset angle, so as to obtain the increased value. Control quantity parameter; determine the enlarged control quantity parameter as the control quantity parameter.

In some embodiments, the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.

In some embodiments, the determining module 12 is further configured to determine the area change value of the target object in the first frame image and the second frame image. The adjustment module 13 is also used for adjusting the distance change value of the pan/tilt head in the horizontal direction, Adjust at least one of the distance change value of the gimbal in the vertical direction and the focal length change value of the camera body in the optical axis direction to obtain the adjusted control amount parameter; the adjusted control amount parameter is determined as the control amount parameter.

In some embodiments, the acquiring module 11 is configured to acquire the focal length of the camera body at the acquisition moment of the first frame of image. The adjustment module 13 is also used to adjust the distance change value of the pan/tilt in the horizontal direction, the distance variation value of the pan/tilt in the vertical direction, and the focal length of the camera body in the direction of the optical axis when the focal length of the camera body is greater than the fourth threshold At least one value of the change value is reduced to obtain a reduced control variable parameter; the reduced control variable parameter is determined as a controlled variable parameter.

Alternatively, the adjustment module 13 is further configured to adjust the distance change value of the pan/tilt in the horizontal direction, the distance variation value of the pan/tilt in the vertical direction, and the camera body in the optical axis direction when the focal length of the camera body is smaller than the fifth threshold. At least one value of the focal length change value of , is increased to obtain the increased control amount parameter; the increased control amount parameter is determined as the control amount parameter.

In some embodiments, the determining module 12 is specifically configured to determine a second distance difference between the position of the target object in the second frame image and the target position of the second frame image; based on the first distance difference and the second distance difference Distance difference, to determine the distance change value.

In some embodiments, the target position is the center area of any frame of images.

The tracking control device for a pan-tilt camera provided in the embodiments of the present application can execute the above method embodiments, and the specific implementation principles and technical effects thereof can be found in the above method embodiments, which are not repeated in the embodiments of the present application.

Exemplarily, the present application provides an electronic device, including: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory to enable the electronic device to execute the tracking control of the pan-tilt camera in the foregoing embodiment. method.

Exemplarily, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen and a sensor; the chip system includes: a processor; when the processor executes the computer instructions stored in the memory, the electronic device executes the foregoing The tracking control method of the PTZ camera in the embodiment.

Exemplarily, the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by the processor to cause the electronic device to implement the tracking control method of the pan-tilt camera in the foregoing embodiment.

Exemplarily, the present application provides a computer program product, comprising: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and at least one processor Executing the execution instruction enables the electronic device to implement the tracking control method for the pan-tilt camera in the foregoing embodiment.

In the above embodiments, all or part of the functions may be implemented by software, hardware, or a combination of software and hardware. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium. A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media. Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented. The process can be completed by instructing the relevant hardware by a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed , which may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random storage memory RAM, magnetic disk or optical disk and other mediums that can store program codes.

Claims

A pan-tilt camera, comprising: a camera body, a pan-tilt and a pan-tilt controller;

When the target object complies with the preset rules or receives an instruction to track the target object instructed by the user,

the pan-tilt controller, configured to obtain the first frame of images and the second frame of images collected by the camera body, where the collection time of the first frame of images is later than the collection time of the second frame of images;

The pan-tilt controller is further configured to determine a first distance difference between the position of the target object in the first frame of image and the target position of the first frame of image;

The pan-tilt controller is further configured to determine a distance change value between the position of the target object in the first frame of image and the position of the target object in the second frame of image;

The pan-tilt controller is further configured to adjust the shooting angle of the camera body based on the first distance difference value and the distance change value, so that the target object is displayed on the third image collected by the camera body. The target position of the frame image, the acquisition time of the third frame image is later than the acquisition time of the first frame image.
The PTZ camera according to claim 1, wherein,

The pan/tilt controller is specifically configured to obtain a control amount parameter based on the first distance difference value and the distance change value, where the control amount parameter includes the distance change value of the pan/tilt in the horizontal direction and the distance change value. the distance change value of the pan/tilt in the vertical direction; control the first motor in the pan/tilt to rotate the pan/tilt according to the distance change value of the pan/tilt in the horizontal direction, and control the first motor in the pan/tilt to rotate the pan/tilt according to the distance change value of the pan/tilt in the horizontal direction The second motor rotates the pan/tilt according to the distance change value of the pan/tilt in the vertical direction, so as to adjust the shooting angle of the camera body.
The pan-tilt camera according to claim 2, wherein the control parameter further comprises a change value of the focal length of the camera body in the direction of the optical axis;

The pan-tilt controller is further configured to adjust the focal length of the camera body according to the focal length change value of the camera body in the optical axis direction, so that the target object is displayed on the target of the third frame image Location.
The pan-tilt camera according to any one of claims 1-3, wherein the pan-tilt controller is further configured to acquire the vertical direction of the pan-tilt at the acquisition moment of the first frame of image When the depression angle is greater than or equal to the preset angle, at least one of the distance change value of the head in the horizontal direction and the distance change value of the head in the vertical direction is increased to obtain the adjustment The increased control amount parameter; the increased control amount parameter is determined as the control amount parameter.
The pan-tilt camera according to claim 4, wherein the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.
The pan-tilt camera according to any one of claims 1-5, wherein the pan-tilt controller is further configured to acquire the focal length of the camera body at the acquisition moment of the first frame of image; When the focal length of the camera body is greater than the fourth threshold, the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the focal length of the camera body in the optical axis direction are calculated. At least one of the changed values is reduced to obtain a reduced control amount parameter; the reduced control amount parameter is determined as the control amount parameter; or, when the focal length of the camera body is less than a fifth threshold When , at least one of the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the focal length variation value of the camera body in the optical axis direction is increased, Obtain the increased control amount parameter; determine the increased control amount parameter as the control amount parameter.
The pan-tilt camera according to any one of claims 1-6, wherein the pan-tilt controller is specifically configured to determine the position of the target object in the second frame of image and the second frame of image a second distance difference value between the target positions of the frame images; the distance change value is determined based on the first distance difference value and the second distance difference value.
The pan-tilt camera according to any one of claims 1-7, wherein the target position is a central area of any frame of images.
A tracking control method for a PTZ camera, characterized in that:

The method is started when the target object conforms to the preset rule or an instruction to track the target object instructed by the user is received;

The method includes:

obtaining a first frame of images and a second frame of images collected by the camera body, where the collection time of the first frame of images is later than the collection time of the second frame of images;

determining a first distance difference between the position of the target object in the first frame of image and the target position of the first frame of image;

determining the distance change value between the position of the target object in the first frame image and the position of the target object in the second frame image;

Based on the first distance difference value and the distance change value, the shooting angle of the camera body is adjusted, so that the target object is displayed at the target position of the third frame image collected by the camera body, and the third frame image is collected by the camera body. The acquisition time of the frame image is later than the acquisition time of the first frame image.
The method according to claim 9, wherein the adjusting the shooting angle of the camera body based on the first distance difference value and the distance change value comprises:

Based on the first distance difference value and the distance change value, a control amount parameter is obtained, and the control amount parameter includes the distance change value of the pan/tilt in the horizontal direction and the distance change of the pan/tilt in the vertical direction value;

Control the first motor in the pan/tilt to rotate the pan/tilt according to the distance change value of the pan/tilt in the horizontal direction, and control the second motor in the pan/tilt to rotate the pan/tilt according to the vertical direction of the pan/tilt. The pan/tilt is rotated by the distance change value to adjust the shooting angle of the camera body.
The method according to claim 10, wherein the control quantity parameter further comprises a focal length change value of the camera body in the direction of the optical axis; the method further comprises:

The focal length of the camera body is adjusted according to the focal length change value of the camera body in the optical axis direction, so that the target object is displayed at the target position of the third frame image.
The method according to any one of claims 9-11, wherein the method further comprises:

At the acquisition moment of the first frame of image, acquiring the depression angle of the gimbal in the vertical direction;

When the depression angle is greater than or equal to a preset angle, at least one of the distance change value of the pan/tilt in the horizontal direction and the distance variation value of the pan/tilt in the vertical direction is increased to obtain the increased control quantity parameter; determine the enlarged control quantity parameter as the control quantity parameter.
The method according to claim 12, wherein the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.
The method according to any one of claims 9-13, wherein the method further comprises:

Obtain the focal length of the camera body at the acquisition moment of the first frame of image;

When the focal length of the camera body is greater than the fourth threshold, the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the distance change value of the camera body in the optical axis direction are calculated. At least one of the focal length change values is reduced to obtain a reduced control amount parameter; the reduced control amount parameter is determined as the control amount parameter;

or,

When the focal length of the camera body is smaller than the fifth threshold, the distance change value of the pan/tilt in the horizontal direction, the distance change value of the pan/tilt in the vertical direction, and the distance change value of the camera body in the optical axis direction are calculated. At least one of the focal length change values is increased to obtain an increased control amount parameter; the increased control amount parameter is determined as the control amount parameter.
The method according to any one of claims 9-14, wherein the determining the position of the target object in the first frame of image and the position of the target object in the second frame of image is between The distance change value of , including:

determining a second distance difference between the position of the target object in the second frame image and the target position of the second frame image;

The distance change value is determined based on the first distance difference value and the second distance difference value.
The method according to any one of claims 9-15, wherein the target position is a central area of any frame of images.
A tracking control device for a pan-tilt camera, characterized in that it includes:

The acquisition module is used to acquire the first frame of images and the second frame of images collected by the camera body when the target object conforms to the preset rules or receives an instruction to track the target object instructed by the user, and the collection moment of the first frame of image later than the acquisition time of the second frame of images;

a determining module, configured to determine the first distance difference between the position of the target object in the first frame image and the target position of the first frame image;

a determining module, further configured to determine a distance change value between the position of the target object in the first frame of image and the position of the target object in the second frame of image;

an adjustment module, configured to adjust the shooting angle of the camera body based on the first distance difference value and the distance change value, so that the target object is displayed at the target position of the third frame image collected by the camera body , the acquisition time of the third frame of image is later than the acquisition time of the first frame of image.
The device according to claim 17, wherein the adjustment module is specifically configured to obtain a control amount parameter based on the first distance difference value and the distance change value, and the control amount parameter includes the cloud The distance change value of the head in the horizontal direction and the distance change value of the head in the vertical direction; control the first motor in the head to the head according to the change value of the distance of the head in the horizontal direction. Rotating, and controlling the second motor in the pan/tilt to rotate the pan/tilt according to the distance change value of the pan/tilt in the vertical direction, so as to adjust the shooting angle of the camera body.
The device according to claim 18, wherein the control quantity parameter further comprises a change value of the focal length of the camera body in the direction of the optical axis;

The adjustment module is further configured to adjust the focal length of the camera body according to the focal length change value of the camera body in the optical axis direction, so that the target object is displayed at the target position of the third frame image.
The device according to any one of claims 17-19, characterized in that,

The acquisition module is further configured to acquire the depression angle of the pan/tilt in the vertical direction at the acquisition moment of the first frame of image;

The adjustment module is further configured to, when the depression angle is greater than or equal to a preset angle, change at least one value of the distance change value of the pan/tilt in the horizontal direction and the distance variation value of the pan/tilt in the vertical direction. Increase the control amount to obtain the increased control amount parameter; determine the increased control amount parameter as the control amount parameter.
The device according to claim 20, wherein the value range of the preset angle is greater than or equal to 20° and less than or equal to 90°.
The device according to any one of claims 17-21, characterized in that,

The acquisition module is further configured to acquire the focal length of the camera body at the acquisition moment of the first frame of image;

The adjustment module is further configured to adjust the distance change value of the pan/tilt in the horizontal direction, the distance variation value of the pan/tilt in the vertical direction, and the At least one of the focal length variation values of the camera body in the direction of the optical axis is reduced to obtain the reduced control amount parameter; the reduced control amount parameter is determined as the control amount parameter;

or,

The adjustment module is further configured to change the distance change value of the pan/tilt in the horizontal direction, the distance variation value of the pan/tilt in the vertical direction, and the At least one of the focal length change values of the camera body in the optical axis direction is increased to obtain the increased control amount parameter; the increased control amount parameter is determined as the control amount parameter.
The device according to any one of claims 17-22, wherein the determining module is specifically configured to determine the position of the target object in the second frame of image and the target of the second frame of image a second distance difference between locations; and the distance change value is determined based on the first distance difference and the second distance difference.
The device according to any one of claims 17-23, wherein the target position is a central area of any frame of images.
An electronic device comprising: a display screen; one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; characterized in that, the The one or more processors, when executing the one or more computer programs, cause the electronic device to implement the method of any of claims 9-16.
A computer storage medium, characterized by comprising computer instructions, which, when the computer instructions are executed on an electronic device, cause the electronic device to perform the method according to any one of claims 9-16.
A computer program product, characterized in that, when the computer program product runs on a computer, the computer is caused to execute the method according to any one of claims 9-16.