WO2022022118A1

WO2022022118A1 - Dome camera control method, apparatus, and device

Info

Publication number: WO2022022118A1
Application number: PCT/CN2021/099895
Authority: WO
Inventors: 仇悦
Original assignee: 华为技术有限公司
Priority date: 2020-07-31
Filing date: 2021-06-12
Publication date: 2022-02-03
Also published as: CN114071069A

Abstract

A dome camera control method, applicable in a processing device. The method comprises: when a target subject to be tracked is determined, periodically transmitting a control instruction to a dome camera, controlling via the control instruction a stepping motor of a gimbal in the dome camera to drive a camera mounted on the gimbal to rotate to a monitoring area in which the target subject is located, thus allowing a processing device to track the target subject on the basis of images received from the dome camera. By presetting the period for transmitting the control instruction, the processing device periodically transmits the control instruction to the rotatable, variable-magnification dome camera; this effectively overcomes the current shortcoming in which when controlling the rotation of a camera of a dome camera, the rotation of the dome camera is insufficiently smooth and steady due to the tendency for gimble response to lag significantly behind changes in the location of a target subject in the actual picture. The invention thus prevents lag in the dome camera rotation process, and thereby achieves the smooth and steady tracking of the target subject, and enhances use experience of the dome camera.

Description

Ball machine control method, device and equipment

technical field

The present application relates to the technical field of control, and in particular, to a ball machine control method, device and device.

Background technique

The dome camera is a commonly used equipment in the field of intelligent security video surveillance, including a camera and a pan/tilt for supporting the camera. The stepping motor in the pan/tilt drives the camera installed on the pan/tilt to rotate to realize the change of the shooting picture. Because the camera in the dome camera can be rotated and the magnification can be changed, it is widely used in the target tracking scene in security monitoring.

At present, when the dome camera is tracking the target object, it usually first determines the square frame where the target object is located based on the collected image, and then calculates the angle deviation of the dome camera according to the pixel deviation between the center point of the square frame and the center point of the image. Then, the camera in the dome camera is driven by the PTZ to rotate. However, because the above control process is to determine the angle deviation, then send commands to control the rotation of the gimbal. In the specific implementation process, the response of the gimbal tends to lag significantly behind the change of the position of the target object in the actual screen, and the rotation and magnification of the dome camera will change. The camera shakes and the captured image is blurred or lagged. It is impossible to control the rotation of the dome camera and capture the image smoothly and stably, which affects the monitoring quality. Therefore, how to provide a smooth and stable ball machine control method has become an urgent technical problem to be solved.

SUMMARY OF THE INVENTION

The present disclosure provides a method, device and equipment for controlling a ball machine, so that the ball machine can achieve smooth and stable tracking of a target object and improve the use experience of the ball machine.

In a first aspect, a ball machine control method is provided, and the method is applied to a processing device. The method may include: after the processing device determines the target object to be tracked, periodically sending control instructions to the dome camera, and controlling the stepping motor in the dome camera through the control instructions to drive the camera installed on the dome camera to rotate to the location where the target object is located. The area is monitored so that the processing device can track the target object based on the images received from the dome. In this way, considering that the response of the gimbal tends to lag significantly behind the change of the position of the target object in the actual picture, the rotation of the dome camera and the change of the magnification will cause the camera to shake and cause the problem of blurring or lag in the captured image. The cycle of the command, the processing device periodically sends control commands to the dome camera with rotatable and variable magnification, which effectively overcomes the shortcomings of the current lack of smoothness and stability when controlling the rotation of the camera of the dome camera, so that the rotation process of the dome camera is no longer stuck. to achieve smooth and stable tracking of the target object, thereby improving the use experience of the dome camera. Among them, considering that if there is a backlog of unexecuted control commands in the dome camera, it will cause the stepper motor to be unable to make tracking adjustments in a timely and effective manner according to the actual situation of the target object in the dome camera screen. In order to overcome this problem, in this method , the preset cycle of sending control commands to the dome camera in the processing device is greater than the excitation response delay of the stepping motor, so that it can ensure that there is no backlog of unexecuted control commands in the dome camera, so that the stepping motor can follow the target object in the ball machine. The actual situation in the screen of the machine can track the target object in a timely and effective manner.

In a possible implementation manner, before the periodic sending of the control instruction to the dome camera, the method may further include: determining the control instruction according to the tracking result of the target object in the image, where the tracking result includes a normal state and an abnormal state. In this way, the corresponding control commands can be effectively generated according to the actual tracking results, which provides a reliable data basis for periodically sending control commands to the dome camera, and makes high-quality tracking of the target object possible.

In another possible implementation manner, when the tracking result of the target object is in a normal state, the control command sent to the dome machine is specifically a first control command, and the first control command is used to instruct the dome machine to continue tracking the target object. Wherein, the first control instruction is a movement instruction. In this example, periodically sending the control command to the dome machine may specifically be sending the newly generated first control command to the dome machine in the same cycle. For example, assuming that when cycle 1 is reached, there are no unsent control commands in the processing device, and the dome camera is always in normal state 1 during this cycle 1, and the movement command 1, movement command 2 and movement command 3 are generated in the order of time. Then, when entering the next cycle 2 from cycle 1, the processing device sends the newly generated movement command 3 to the dome camera, and the movement command 1 and movement command 2 can be discarded by the processing device.

In another possible implementation manner, when the tracking result of the target object is in an abnormal state, the control command sent to the dome machine is specifically a second control command, and the second command is used to instruct the dome machine to execute the control command when the dome machine is in an abnormal state. Ball machine control. Wherein, the second control instruction includes at least one of the following instructions: a stop instruction, a magnification reduction instruction, or a return to a preset point instruction. In this example, if the processing device saves the control command through the command queue, the periodic sending of the control command to the dome machine may specifically be directed to the dome machine to send the earliest generated second control command stored in the command queue. For example, assuming that when cycle 3 is reached, there are no unsent control commands in the processing device, and the dome camera is always in an abnormal state during cycle 3, and the stop command 1, stop command 2 and magnification reduction command 1 are generated in the order of time. Then, the command queue includes stop command 1 , stop command 2 and magnification reduction command 1 . When entering the next cycle 4 from cycle 3, the processing device sends the earliest generated stop command 1 to the dome camera, and the command queue includes stop command 2 and reduce magnification command 1; when entering the next cycle 5 from cycle 4, The stop command 2 generated after the processing device sends the stop command 1 to the dome camera, and the command queue includes the lowering magnification command 1; when entering the next cycle 6 from cycle 5, the processing device sends the stop command 2 to the dome camera. Override command 1.

In another possible implementation manner, when the tracking result of the target object is in a normal state, determining the control instruction according to the tracking result of the target object in the image may specifically refer to: determining the first control instruction according to a preset rule, wherein , the first control command may include a first adjustment speed in the first direction, a second adjustment speed in the second direction, and a target magnification adjustment speed.

In another possible implementation manner, the preset rule may include determining the first control instruction according to the magnification of the camera in the dome camera and a proportional integral derivative (proportional integral derivative, PID) parameter group. Specifically, the processing device includes multiple sets of correspondences between magnifications and PID parameter groups, and the process of determining the first control command according to the magnification of the camera in the dome camera and the PID parameter group may include: according to the current magnification of the camera in the dome camera , determine the PID parameter group corresponding to the magnification; then, according to the PID parameter group, determine the first adjustment speed and the second adjustment speed; determine the first control command based on the first adjustment speed and the second adjustment speed. Wherein, determining the first adjustment speed and the second adjustment speed according to the PID parameter group, for example, may include: determining the first adjustment speed according to the PID parameter and the first deviation in the first direction (such as the x direction), where the first deviation is The distance in the first direction between the center point of the image and the center point of the area where the target object is located in the image; the second adjustment speed is determined according to the PID parameters and the second deviation in the second direction (such as the y direction). The second deviation is the distance in the second direction between the center point of the image and the center point of the area where the target object is located in the image. It should be noted that the area where the target object is located in the image refers to an area in the image that can completely include the target object, for example, a rectangular area (also called a square frame) that completely includes the target object. In this way, the corresponding relationship between the magnification and the PID parameter group is preset in the processing device. When the adjustment speed of the rotating stepper motor needs to be calculated, the corresponding PID parameter group can be determined directly according to the current magnification of the camera in the dome camera. The adjustment speed of the stepping motor suitable for the current magnification is obtained by calculation, so that the generated first control command can be used for tracking control of the target object more effectively.

It should be noted that, before determining the first adjustment speed according to the PID parameters and the first deviation in the first direction, the method may further include: if the first deviation is smaller than the preset first deviation threshold, then setting the first deviation and, before determining the second adjustment speed according to the PID parameter and the second deviation in the second direction, the method may further include: if the second deviation is less than a preset second deviation threshold, then, setting the second deviation Zero. Wherein, the first deviation threshold and the second deviation threshold are determined based on a preset central area of the image. In this way, by judging whether the target object is in the center area of the image, if the target object is located in the center of the image or the target object is not far from the center of the image, the deviation in the corresponding direction is set to zero, so that the PID is calculated and output in the corresponding direction The adjustment speed of the stepper motor is more accurate to avoid the vibration of the PID due to the too small input value in the PID calculation when the adjustment range is small and the effect is not obvious, which makes the stepper motor shake.

It should be noted that, after obtaining the first adjustment speed and the second adjustment speed, and before generating the first control command, a dead zone judgment can also be performed, specifically: judging whether the first adjustment speed is within the dead zone range, and if so, Then, set the first adjustment speed to zero; determine whether the second adjustment speed is within the dead zone range, and if so, set the second adjustment speed to zero. The dead zone range is determined according to the adjustment accuracy of the stepping motor. For example, if the minimum rotation gear of the stepping motor is 0.1, then the dead zone range can be -0.1-0.1. In this way, the adjustment speed of the stepper motor calculated by the PID is effectively prevented from being smaller than the minimum rotation gear of the stepper motor, which causes the stepper motor to rotate too far, which makes the rotation process of the stepper motor shake back and forth and cannot be stabilized. The machine can track the target object smoothly and stably.

In another possible implementation manner, the preset rule may further include determining the first control instruction according to the proportion of the target object in the image in the preset direction. Specifically, the process of determining the first control instruction according to the proportion of the target object in the image in the preset direction may include: obtaining the proportion of the target object in the image in the preset direction; then, according to the proportion and the preset proportion threshold, determining The target magnification adjustment speed, the target magnification adjustment speed is the preset magnification adjustment speed or the opposite number of the preset magnification adjustment speed; then, the first control command can be determined according to the target magnification adjustment speed. For example, assuming that the preset magnification adjustment command is greater than zero, when the ratio is smaller than the preset first ratio threshold, the preset magnification adjustment speed itself is determined as the target magnification adjustment speed, so as to control the camera to increase the camera's zoom rate according to the target magnification adjustment speed. magnification; when the ratio is greater than the preset second ratio threshold, the opposite number of the preset magnification adjustment speed is determined as the target magnification adjustment speed, so as to control the camera to reduce the camera's magnification according to the target magnification adjustment speed, wherein the second ratio threshold is greater than The first scale threshold. It should be noted that the preset direction can be determined based on different target objects. For example, when the target object is a slender shape (such as a person), the height of the target object in the image is determined as the preset direction; for another example, When the target object is a short and wide shape (such as a vehicle), the width of the target object in the image is determined as the preset direction. In this way, through a reasonable preset magnification adjustment speed, adjust the magnification of the camera according to the preset magnification adjustment speed or its opposite, so as to avoid blurred images collected by the dome camera when the magnification is changed due to the excessively fast magnification adjustment speed. The problem that seriously affects the tracking of the target object improves the monitoring quality of the camera.

In another possible implementation manner, in the case where the tracking result of the target object is an abnormal state, determining the control instruction according to the tracking result of the target object in the image may specifically refer to: determining the second control instruction according to a preset rule, the The preset rule may include determining the second control instruction according to the type of abnormal state. The types of the abnormal state may include, but are not limited to: a first abnormal state, a second abnormal state, a third abnormal state, and a fourth abnormal state.

In another possible implementation manner, when the dome camera is in a normal state and the captured image does not include the target object, it is determined that the dome camera enters the first abnormal state (also referred to as a possible loss state), and indicates that the camera is in the PTZ The stepper motor stopped working. That is, when the dome machine changes from the normal state to the first abnormal state within one cycle of sending the control command to the dome machine, the control command sent to the PTZ may be, for example, the first stop command.

In another possible implementation manner, when the dome camera is in the first abnormal state and the target object is not included in the multi-frame images collected at the first preset time, it is determined that the dome camera enters the second abnormal state (also It can be called as determining the lost state), and instruct the camera to reduce the magnification to expand the monitoring range. That is, when the dome machine changes from the first abnormal state to the second abnormal state within one cycle of sending the control command to the dome machine, if there is still a third stop command to be sent, the control command sent to the PTZ is the first Three stop commands; if there is no third stop command to be sent, the control command sent to the PTZ is the first magnification reduction command.

In another possible implementation manner, when the dome camera is in the fourth abnormal state and the image collected by the dome camera within the third preset time does not include the target object, it is determined that the dome camera has entered the second abnormal state, and an indication The camera reduces the magnification to expand the monitoring range. That is, when the dome machine changes from the fourth abnormal state to the second abnormal state within one cycle of sending the control command to the dome machine, if there is still a sixth stop command to be sent, the control command sent to the PTZ is the first Six stop instructions; if there is no sixth stop instruction to be sent, the control instruction sent to the PTZ is the fourth reduction magnification instruction.

In another possible implementation manner, when the dome camera is in the second abnormal state and no target object is included in the multi-frame images collected at the second preset time, it is determined that the dome camera enters the third abnormal state (also It can be called idle state), and instructs the stepper motor to drive the camera to return to the preset point and exit the tracking of the target object. That is, when the dome camera changes from the second abnormal state to the third abnormal state within one cycle of sending the control command to the dome camera, if there is still a second lower magnification command to be sent, the control command sent to the PTZ is: The second magnification reduction command; if there is no second magnification reduction command to be sent, the control command sent to the PTZ is the first return to the preset point command.

In another possible implementation manner, when the dome camera is in the second abnormal state and the collected image includes the target object, it is determined that the dome camera enters the fourth abnormal state (also referred to as a possible recovery state), And instruct the stepper motor in the gimbal to stop working. That is, within one cycle of sending the second control command to the dome camera, when the dome camera changes from the second abnormal state to the fourth abnormal state, if there is still a third command to reduce the magnification to be sent, send it to the PTZ. The control command is the third magnification reduction command; if there is no third magnification reduction command to be sent, the control command sent to the PTZ is the fourth stop command.

In another possible implementation manner, when the dome camera is in the fourth abnormal state and the multi-frame images collected by the dome camera at the third preset time all include the target object, it is determined that the dome camera has entered a normal state. That is, when the dome machine changes from the fourth abnormal state to the normal state within one cycle of sending the control command to the dome machine, if there is a fifth stop command to be sent, the control command sent to the PTZ is the first Five stop instructions; if there is no fifth stop instruction to be sent, the control instruction sent to the PTZ is the newly generated movement instruction.

In another possible implementation manner, when the dome machine is in the first abnormal state and the image collected by the dome machine within the first preset time includes the target object, it is determined that the dome machine enters the normal state. That is, when the dome camera changes from the first abnormal state to the normal state within one cycle of sending the control command to the dome camera, if there is a second stop command to be sent, the control command sent to the PTZ is the second stop command. instruction; if there is no second stop instruction to be sent, the control instruction sent to the PTZ is the first movement instruction.

It can be seen that by presetting the subdivided state flow logic in the processing device, a more efficient and orderly control of the dome camera is realized, so that the dome camera can complete the tracking of the target object with high quality.

In another possible implementation manner, the method may further include: when the dome camera has acquired the key information of the target object, determining that the dome camera has entered the fifth abnormal state, and instructing the stepper motor to drive the camera to return to the preset Click to exit the tracking of the target object. The key information may be identification information used to indicate the target object, such as a frontal face image of the target person, or a license plate image of the target vehicle. In this way, the processing device can send the image including the key information of the target person to the central device, so that the central device can analyze and summarize the key information of the target person in the image, so that the tracking of the target object can achieve more important value.

In another possible implementation manner, the instructions generated by the processing device may also be stored through the instruction queue. In specific implementation, periodically sending control commands to the dome camera may include, for example: at the first moment, sending the first control command in the first command queue to the gimbal; at the second moment, sending the second command in the second command queue to the gimbal. Control instruction. Among them, the time from the first time to the second time is one cycle of the ball machine sending the control command, the control command sent at the first time is the first control command, and the control command sent at the second time is the second control command ; If all the control instructions generated at the first moment only include movement instructions, then, only the first control instructions included in the first instruction queue, the first control instructions are the latest generated movement instructions before the first moment; if the deadline All control commands generated at the first moment include at least one of a stop command, a magnification reduction command, or a return to the preset point command, then the first command queue includes all stop commands, reduce For a magnification command or a return to a preset point command, the first command queue does not include a movement command, and the first control command is the command whose generation time is the longest from the first moment in the first command queue. As an example, if the first control command is the first movement command, between the first moment and the second moment, the method may further include: at a third moment, generating a fifth stop command, and overwriting the fifth stop command For the first move instruction in the instruction queue, obtain the updated first instruction queue, and the updated first instruction queue includes the first stop instruction; at the fourth moment, generate the second move instruction, according to the updated first instruction queue Discard the second movement instruction; at the fifth moment, generate a second return preset point instruction, and add the second return preset point instruction to the updated fifth stop instruction of the first instruction queue, and obtain the second instruction queue , the second instruction queue includes a fifth stop instruction and a second return to the preset point instruction. Then, in this example, sending the second control instruction in the second instruction queue to the PTZ includes: sending a fifth stop instruction to the PTZ to control the stepping motor in the PTZ to stop working. Then, after the second moment, the method may further include: at the sixth moment, sending a second return to the preset point instruction to the PTZ, so as to control the stepping motor in the PTZ to drive the dome camera to return to the preset point, wherein the second The time from the moment to the sixth moment is a cycle. It can be seen that by storing the control commands to be sent to the dome machine in the form of an instruction queue, it can ensure that the control commands are sent to the dome machine in an orderly manner so as to effectively control the dome machine.

In a second aspect, the present application provides a ball camera control device, the device including each module for executing the ball camera control method in the first aspect or any possible implementation manner of the first aspect.

In a third aspect, a ball machine control device is also provided, and the ball machine control device includes a processor and a memory. Wherein, the memory is used to store computer instructions; the processor is used to execute the operation steps of the ball camera control method in the first aspect or any possible implementation manner of the first aspect according to the computer instructions.

In a fourth aspect, a monitoring system is also provided. The monitoring system includes a first device, a second device, and a processing device, and the processing device communicates with the first device and the second device, respectively. Wherein, the first device is configured to acquire an image within a fixed monitoring range, and send the first image including the target object to the processing device; the processing device is configured to execute the first aspect or any of the first aspect based on the first image. The operation steps of the dome camera control method in a possible implementation manner include sending a control instruction to a second device; the second device is configured to complete the tracking of the target object based on the control instruction sent by the processing device.

Wherein, the second device is a ball machine. The first device may be a bolt action.

In a possible implementation manner, the monitoring system may further include a central device, a processing device, which is further configured to acquire key information of the target object, and send the key information to the central device; the central device is used to monitor the key information analysis and processing.

In a fifth aspect, the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, when the computer-readable storage medium runs on a computer, the computer performs the operation steps of the methods of the above aspects.

In a sixth aspect, the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the operation steps of the methods of the above aspects.

On the basis of the implementation manners provided by the above aspects, the present application may further combine to provide more implementation manners.

Description of drawings

FIG. 1 is a schematic diagram of the logical architecture of a monitoring system 10 suitable for ball machine control provided in this embodiment;

FIG. 2 is a schematic diagram of a monitoring process of a target person by a monitoring system 10 provided in this embodiment;

FIG. 3 is a flowchart interaction diagram of a ball machine control method provided in this embodiment;

FIG. 4 is a schematic diagram of state flow of a ball machine provided in this embodiment;

FIG. 5 is a schematic diagram of an instruction queue for saving control instructions according to the present embodiment;

FIG. 6 is a schematic diagram of a center area judgment provided by the present embodiment;

Fig. 7 is a kind of schematic flow chart of calculating moving speed based on PID provided by this embodiment;

FIG. 8 is a schematic structural diagram of a ball machine control device provided in this embodiment;

FIG. 9 is a schematic structural diagram of a ball machine control device provided in this embodiment.

detailed description

The technical solutions of the present application will be described below with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of the logical architecture of a monitoring system 10 suitable for dome camera control in this embodiment. As shown in FIG. 1 , the monitoring system 10 includes: a processing device 100 , at least one bolt 200 , at least one dome 300 and a central device 400. The processing device 100 can communicate with at least one trigger 200, at least one dome 300, and the central device 400, respectively, receive data signals corresponding to images collected by the trigger 200 or the dome 300, and send control signals to the dome 300, Realize the control of at least one ball machine 300, and send the monitoring results obtained by the trigger 200 or the ball machine 300 to the central device 400, and the central device 400 analyzes and summarizes the monitoring results. Wherein, the monitoring result includes key information of the target object, such as: the frontal face image of the target person, and another example: the license plate image of the target vehicle. The processing device 100 may include at least a communication module 110 and a processing module 120. The communication module 110 is used to implement communication with other devices (such as the camera 200, the ball camera 300 and the central device 400), and the processing module 120 is used to communicate with other devices according to the central The signals sent by the device 400 , the trigger 200 and/or the dome machine 300 are processed to obtain control signals suitable for the dome machine 300 . The bolt 200 may at least include: a communication module 210 , a processing module 220 and a camera 230 . The communication module 210 is used to implement communication with other devices (such as the processing device 100 ); the processing module 220 is used to process the data signals output by the camera 230 (such as decoding, unpacking, compression, conversion, etc.); the camera 230 is used to capture the image of the corresponding area of the bolt 200 . The dome camera 300 may at least include: a communication module 310 , a processing module 320 , a camera 330 and a pan/tilt 340 . The communication module 310 is used to implement communication with other devices (such as the processing device 100 ); The data signal and the control signal sent by the processing device 100 are processed; the pan/tilt 340 is used to support the camera 330, the rotation of the stepping motor 341 in the pan/tilt 340 can drive the camera 330 to rotate, and the camera 330 also has the function of changing the magnification, so, The camera 330 is used to photograph objects of different distances from different angles. The central device 400 is configured to communicate with multiple processing devices 100 , analyze and summarize the monitoring results sent by the multiple processing devices 400 , and monitor and manage the areas covered by the multiple processing devices 100 .

The processing device 100 can be a device with computing capabilities, for example, it can be a street cabinet (also called an edge station) corresponding to at least one trigger 200 and at least one dome 300, and the trigger 200 refers to the monitoring area that cannot be switched. The dome camera 300 refers to a monitoring device that can switch the monitoring area and can change the magnification, and the central device 400 can be a server, such as a cloud server. According to actual deployment requirements, the processing device 100 may control and manage monitoring devices on multiple adjacent monitoring poles, and may also control and manage some or all monitoring devices on one monitoring pole. Each monitoring rod may include one trigger 200 and one ball machine 300, or multiple triggers 200 and one ball machine 300, and may also include multiple triggers 200 and multiple ball machines 300, which is not correct in this application. The number of triggers 200 and the number of ball machines 300 included on each monitoring rod are specifically limited.

Optionally, in the monitoring system 10, the processing device 100 can also be integrated into any one of the bolts 200 or the dome 300 managed by it. In the processing module 320 of the camera 300; for another example, the processing device 100 can be integrated in the gun camera 200 or the dome camera 300 as an independent module, and the independent module is only used to realize the function of the processing device 100. The specific implementation form is in This application is not limited. For the convenience of description, this application describes the processing device 100 as a separate entity device from the trigger 200 and the dome camera 300.

For example: in the monitoring system 10, it is assumed that a target person needs to be searched, and the distinctive feature of the target person is wearing a red shirt, then, the monitoring process of the target person by the monitoring system 10 is shown in FIG. 2, which may include:

S11, the central device 400 generates and sends a task instruction to the multiple processing devices 100 connected to it, where the task instruction includes feature information of the target person, and the feature information is used to indicate the feature of the target person. For example, the feature information can indicate the target person. The character is wearing a red shirt.

S12, the processing device 100 processes the received task instruction, generates a request instruction that can be recognized by the trigger 200 and the ball camera 300, and sends the request instruction to at least one trigger 200 and at least one ball camera 300 controlled by it.

S13, the processing device 100 receives the images collected by the trigger 200 and the dome camera 300, and performs target detection and feature matching. If the detected object and the target object are successfully matched, it is determined that the target object is detected.

Wherein, if it is first detected that the image of the target object comes from the trigger 200, the detection process may include: the trigger 200 sends the image 1 of the target object to the processing device 100, and the processing device 100 detects at least the image 1 through the target detection algorithm. After one object, perform feature matching on the detected object and the target object, and take the successfully matched object as the target object; The image including the target object obtained after matting. In this case, the processing device 100 generates based on the square frame 1 and sends the return to the preset point command 1 to the dome camera 300. The return to the preset point command 1 is used to instruct the dome camera 300 at the initial position to turn to the monitoring range of the trigger 200. The Returning to the preset point command 1 can carry the identification of the trigger 200 (ie, the identification of the preset point).

If it is first detected that the image of the target object comes from the dome camera 300, the detection process may include: the dome camera 300 sends the image 1' of the target object to the processing device 100, and the processing device 100 detects at least the image 1' through the target detection algorithm. After one object, feature matching is performed between the detected object and the target object, and the successfully matched object is used as the target object; the image 1' is processed to obtain a square frame 1' corresponding to the target object in the image 1'. The square frame 1' refers to the image including the target object obtained after matting from the image 1.

It should be noted that, to cut out the image, specifically, the center of the target object in the image can be used as the origin, and the preset length is used as the radius to obtain the circular area including the target object in the image. The center of the target object is the intersection of the diagonal lines, resulting in a rectangular area including the target object (ie, the square box mentioned in other parts). Wherein, the setting of the radius of the circular area or the length and width of the rectangular area needs to make the obtained area completely include the target object. For the convenience of description, in this embodiment, a square frame obtained by cutout is taken as an example for description.

S14, the dome camera 300 collects the image 2 and sends it to the processing device 100;

S15, the processing device 100 matches the image 2 with the square frame 1 (or the square frame 1'), and if the matching is completed, generates a square frame 2 including the target object in the image 2, and triggers the ball camera 300 to track the target object.

S16, the processing device 100 executes the following S17-S23 for each image 2 collected by the dome camera 300, until the image 2 including the key information of the target person acquired by the processing device 100 is received, the operation times out, or the state 0 becomes an abnormal state In the idle state, the following S25 is performed.

S17, the processing device 100 obtains the status flag bit of whether the tracking is successful or not according to the image 2, the target detection algorithm and the preset tracking algorithm. When the status flag indicates that the tracking is successful, the relevant parameters of the square frame 2 corresponding to the target object in the image 2 can also be obtained, such as (x, y, w, h) corresponding to the square frame 2, where (x, y) is the coordinate value of the center point of the square frame 2 in the image 2, w and h are the width and height of the square frame 2, respectively;

S18, the processing device 100 determines the state 0 to which the target object is currently being tracked according to the change of the state flag bit. The states may include, for example, a normal state and an abnormal state, wherein the abnormal state includes information used to indicate the tracking result of the target object. For example, abnormal states include a possible lost state, a certain lost state, a possible retrieved state, and an idle state.

S19, the processing device 100 determines whether the state 0 is a normal state, and if so, executes S20, otherwise, executes S22;

S20, the processing device 100 determines the PID parameters corresponding to the current magnification 1: KP1, KI1 and KD1 according to the corresponding relationship between the preset magnification and proportional integral derivative (PID) parameters of the ball camera 300, and according to the following Formula (1) and formula (2) calculate the moving speed Vpan on the x direction and the moving speed Vtilt on the y direction; And, according to the ratio of h accounting for the total height of the image 2, determine the direction of the preset magnification adjustment speed Vzoom;

Among them, Δx(t) and Δy(t) are the deviation values between the center point (x0, y0) of the image 2 collected at time t and the center point (x, y) of the square frame 2 . It should be noted that the calculated V _pan and V _tilt are continuous values corresponding to the fixed rotational gears of the stepping motor 341 . For example, the rotational speed of the stepping motor 341 ranges from -1.0 to 1.0 (positive and negative values represent The rotation direction of the stepper motor 341, the absolute value represents the rotation speed), and every 0.1 is divided into 1 rotation gear. Then, the stepper motor 341 includes 10 rotation gears in two directions respectively. The calculated V _pan and The value of _Vtilt ranges from -1.0 to 1.0. If V _pan =0.17, then the stepper motor 341 can rotate according to the rotation gear of 0.1 or 0.2. It should be noted that, if the calculated value ranges of V _pan and V _tilt are not in the range of -1.0 to 1.0, then it can be executed according to the maximum rotation gear. For example, if V _pan =1.2 is calculated, then the stepping motor 341 It can be rotated according to the rotation gear of 1.0.

S21, the processing device 100 generates a movement instruction 1, which includes (V _pan , V _tilt , V _zoom ) and includes a forced execution flag. For example, the value of the forced execution flag may be 0, which is used to indicate Move instruction 1 is not a mandatory operation, and can be discarded or overwritten with move instructions in the instruction queue;

S22 , the processing device 100 generates a stop command 1 , a magnification reduction command 1 or a return to preset point command 2 based on the specific abnormal state. The stop command 1, the magnification reduction command 1 and the return to preset point command 2 all include mandatory execution flags. For example, the value of the mandatory execution flag of the stop command 1, the magnification reduction command 1 and the return to preset command 2 can be set to 1. , used to indicate that these instructions are operations that need to be enforced and cannot be discarded;

S23, the processing device 100 periodically sends a control command to the ball camera 300 to realize the tracking of the target object, and the period is greater than the excitation response time delay T1 of the stepping motor 341;

S24, the dome camera 300 controls the stepping motor 341 and/or the camera 330 based on the received control instruction to track the target object.

S25, the processing device 100 generates and sends an instruction to return to the preset point to the dome machine 300 to control the dome machine 300 to return to the initial position, wherein the key information includes the identification information of the target object, for example, when the target object is the target person, the key information can be Including the frontal face image of the target person;

S26 , the processing device 100 sends the image 2 including the key information of the target person to the central device 300 , and the central device 300 analyzes and summarizes the key information of the target person in the image 2 .

It should be noted that the specific manner of triggering the dome camera 300 to start tracking the target object may include:

In the first case, it may be that the processing device 100 detects the target object from the images sent by other monitoring devices (for example, the trigger 200), and then the processing device 100 triggers the dome camera 300 to start tracking the target object;

In the second case, the processing device 100 may also detect the target object from the image sent by the dome camera 300, and then the processing device 100 triggers the dome camera 300 to start tracking the target object.

The manner of triggering the ball machine 300 to track the target object is not specifically limited in this application. For the convenience of description, the following description takes the triggering manner of the first case as an example.

It should be noted that the system architecture shown in FIG. 1 is only an example of the system architecture provided by the ball machine control method provided in this application, and the monitoring process of the target person by the monitoring system shown in FIG. This is a scenario example provided to better illustrate the overall flow of the ball machine control method provided by the present application, and does not constitute a limitation to the embodiments of the present application.

Based on the above system architecture, the present application provides a dome camera control method. When the processing device acquires the first image of the target object to be tracked, it can be based on the preset association relationship between the first device and the second device (ie, the dome camera). , control the stepping motor of the pan/tilt in the second device to work to drive the second device to rotate to the monitoring area corresponding to the first device, so that the camera in the second device can capture the second image including the target object and Send to the processing device, the processing device can generate control instructions for the dome camera based on the second image sent by the second device, and periodically send control instructions to the PTZ to control the stepping motor in the PTZ Work to achieve the tracking of the target object. It can be seen that this method considers that the rotation of the camera in the dome camera is driven by the rotation of the stepper motor, the stepper motor has a certain excitation response time delay, and the process of capturing images by the camera in the dome camera is discrete. The cycle of sending control commands is preset, and the processing device periodically sends control commands to the dome camera with rotatable and variable magnification, which effectively overcomes the shortcomings of the current lack of smoothness and stability when controlling the rotation of the dome camera. The process is no longer stuck, and smooth and stable tracking of the target object is achieved, thereby improving the use experience of the dome camera.

As a possible embodiment, the camera of the dome camera that is triggered to track the target object usually collects images at equal time intervals, and sends the images to the processing device. The equal time interval is related to the frequency at which the camera collects images. For example, if the frequency at which the camera collects images is 25 frames per second, the time interval at which the camera collects images is 40 milliseconds. In one case, every time the camera collects a frame of image, the image will be sent to the processing device, and the processing device will generate a corresponding control command through the processing of algorithms such as target detection algorithm and tracking algorithm. In another case, the processing device can also perform frame skipping processing on the images captured by the camera. For example, every time the camera captures one frame of image, the image will be sent to the processing device, and the processing device will send the image to the processing device every time it receives five frames of images. The fifth frame of image is processed to generate a corresponding control command; for another example: every five frames of images are collected by the camera, the fifth frame of image is sent to the processing device, and the processing device processes each received frame of image and generates a corresponding one. Control instruction.

As another possible embodiment, the processing device periodically sends control commands to the dome camera. In a cycle of sending commands, if the processing device has no backlog of mandatory execution commands (for example: stop command, reduce magnification command, or return to the preset command) Positioning command), and all the movement commands are generated in this cycle, then only the latest generated movement command will be kept, and when the command is sent, the latest generated movement command will be sent to the PTZ of the dome camera, so that the dome camera can follow the The position of the target object in the newly collected image can be adjusted adaptively to the camera of the dome camera to achieve fast and effective tracking of the target object. If there is an unsent mandatory execution instruction in the processing device or a mandatory execution instruction is generated in this cycle, all the mandatory execution instructions need to be saved according to the generation time, and the movement instructions generated in this cycle are overwritten or Discard, wait until the moment of sending the command, and send the backlog of the earliest generated mandatory command to the dome camera's PTZ, so that the dome camera can adjust the camera of the dome camera in an orderly manner according to the abnormal order, so that the dome camera can adjust the target object. The tracking can return to the normal state as much as possible. It should be noted that the cycle of sending the control command needs to be greater than the excitation response delay of the stepping motor (that is, the time interval between the excitation of the input stepping motor and the output response of the stepping motor), which is different from the frequency at which the camera captures images. It doesn't matter.

Next, taking the monitoring system 10 shown in FIG. 1 as an example, the ball camera control method provided by the present application is described in detail with reference to FIG. 3 , and the method is applied to the processing device 100 in the monitoring system 10 . As shown in Figure 3, the method includes:

S301: Acquire a first image sent by the trigger 200 in the monitoring system 10, where the first image includes the target object to be tracked.

It should be noted that S301 is only an exemplary description, and in specific implementation, the processing device 100 only needs to acquire the target object to be tracked, thereby triggering the dome camera 300 to track the target object. The method of acquiring the target object may be acquired based on the first image collected by the bolt 200 in the monitoring system, or acquired from the first video, or may be triggered based on the user's manual operation, which is not specifically limited in this embodiment. . In the following, for the convenience of description, the acquisition of the target object from the first image is taken as an example for description.

The bolt 200 may refer to any monitoring device in the monitoring system 10 that can communicate with the processing device 100 . In this embodiment, considering that the trigger can clearly monitor a large range, the combination of the trigger 200 and the ball camera 300 is selected to provide an intelligent and comprehensive monitoring service.

During specific implementation, when the monitoring system 10 needs to retrieve a target object with certain or certain characteristics from monitoring according to actual needs, the processing device 100 in the monitoring system 10 can obtain a task instruction, which includes Feature information of the target object. Feature information refers to the features of the target object that can be significantly distinguished from other objects. For example, if the target object is a person, the feature information of the target object can be the person's clothing features, such as wearing a red shirt; For example, if the target object is a vehicle, the feature information of the target object may be the model, color and other features of the vehicle, for example, a small black car.

As an example, the task instruction can be triggered and generated on the central device 400 of the monitoring system 10 and sent to the processing device 100 by the central device 400. Then, the processing device 100 can translate the task into the ability of the bolt 200 and the dome 300 to be able to After identifying the request command, the request command is sent to the trigger 200 and the ball camera 300 controlled by the processing device 100, and the request command includes the feature information of the target object. This example is suitable for situations where the possible range of activity of the target object is large or the possible range of activity of the target object is uncertain. The central device 400 can issue task instructions to some or all of the processing devices 100 in the monitoring system 10 in batches, so that the monitoring The mass monitoring devices in the system 10 can search and monitor the target objects in most or all of the monitoring areas covered by the monitoring system 10, without the need for separate configuration and triggering on each processing device 100, saving manpower and material resources to a certain extent, Improve the efficiency of monitoring services.

As another example, the task instruction may also be a request instruction that is directly generated on the processing device 100 of the monitoring system 10 and can be recognized by the trigger 200 and the dome camera 300. Then, the processing device 100 can send the request instruction to the processing device 100. The trigger 200 and the dome 300 responsible for control include the characteristic information of the target object in the request command. This example is applicable to the situation where the possible range of activities of the target object has been determined, and a request instruction can be configured on the processing device 100 responsible for the monitoring device within the range of activities, so that the corresponding monitoring device can quickly monitor the target in a small coverage area. To search and monitor objects, there is no need for the central device 400 to issue a task request to the corresponding processing device 100 or for the processing device 100 to process the task request, which saves network resources in the monitoring system 10 and improves the efficiency of monitoring services.

After the trigger 200 sends the first image to the processing device 100, the processing process of the processing device 100 may include: the processing device 100 performs target detection on the first image, and detects at least one object; the processing device 100 extracts the at least one object The feature information of the object, and the feature information of the at least one object and the feature information of the target object are matched, if the matching is successful, the object corresponding to the feature information is determined as the target object; The detection frame that completely encloses the target object is used as a square frame including the target object in the first image, and the square frame can be used as basic data for subsequent tracking of the target object.

It should be noted that, after S301, the processing device 100 performs matting processing on the acquired first image to obtain a square frame including the target object. The square box is an image.

It should be noted that each monitoring device (including the trigger 200 and the dome camera 300 ) that has received the request instruction sends the images collected by itself to the processing device 100 for target detection and feature matching.

S302 , send a control command 1 to the dome camera 300 , and the control command 1 is used to control the stepping motor 341 of the pan/tilt 340 in the dome camera 300 to work, so as to drive the dome camera 300 to rotate to the monitoring area corresponding to the trigger 200 .

Considering that the camera 330 of the dome camera 300 can adjust the shooting angle and magnification, it is suitable for the tracking of the target object in the active state. Therefore, after the processing device 100 obtains the square frame including the target object in the first image, the processing device 100 can Generate and send the control command 1 to the dome camera 300 to control the stepper motor 341 of the gimbal 340 in the dome camera 300 to work, and drive the camera 330 to rotate through the stepper motor 341, so that the camera 330 faces the monitoring area corresponding to the gun 200. Wherein, the camera 330 faces the monitoring area corresponding to the gun 200, which may mean that the monitoring area of the camera 330 completely covers the monitoring range of the gun 200.

As an example, S302 may specifically include: after the processing device 100 obtains the square frame including the target object in the first image, it can determine that the target object is within the monitoring range of the camera 200, so as to generate and send a return preview to the camera 300. Set point instruction 1, the return preset point instruction 1 includes the logo of the trigger 200, which is used to instruct the stepping motor 341 of the pan/tilt 340 in the dome camera 300 to rotate to the monitoring range of the trigger 200 corresponding to the logo. In one case, the dome camera 300 can record the position of the trigger 200, then, the dome camera 300 rotates to the position of the trigger 200 according to the return to preset point command 1, and then stops rotating. In another case, the positional relationship between the dome machine 300 and the trigger 200 is stored in the dome machine 300. Then, the dome machine 300 returns to the preset point command 1 according to the identification of the trigger 200 and the locally saved trigger 200 and trigger 200. The positional relationship of the dome camera 300 determines the rotation parameters that need to be rotated for the dome camera 300 to turn to the coverage area of the trigger 200, so as to rotate the monitoring area of the steering trigger 200 based on the rotation parameters.

As another example, in S302 , the dome camera 300 may also be rotated to the monitoring area of the trigger 200 based on the preset positional relationship between the trigger 200 and the dome 300 . For example, the processing device 100 may be preset with the positional relationship between all monitoring devices it is responsible for, including the positional relationship between the trigger 200 and the dome camera 300, then, S302 may specifically include: after the processing device 100 obtains the first image, It can be determined that the target object is within the monitoring range of the trigger 200, thus, based on the positional relationship between the trigger 200 and the dome 300, determine the rotation parameters that the dome 300 needs to rotate to turn to the coverage of the trigger 200; then, the processing device 100 The movement command 1 can be generated and sent to the dome camera 300 according to the determined rotation parameters, and the movement command 1 includes the rotation parameters to instruct the stepping motor 341 of the pan/tilt 340 in the dome camera 300 to work according to the rotation parameters; in this way, The pan/tilt 340 in the dome camera 300 parses the control command 1 to obtain rotation parameters, and the stepping motor 341 of the pan/tilt 340 rotates based on the rotation parameters to drive the camera 330 to turn to the monitoring range of the gun 200 .

The above-mentioned rotation parameters can be the angle and direction of rotation, for example, rotate 90 degrees counterclockwise; or, the rotation parameters can also be the rotation speed and rotation time, for example, rotate at a rotation gear of 0.1 for 9 seconds.

It can be seen that when the dome camera 300 rotates to the monitoring area corresponding to the trigger 200, an image including the target object can be collected, which provides a prerequisite for tracking the target object and obtaining key information of the target object.

S303 , based on the second image collected by the camera 330 in the dome camera 300 , periodically send a control command to the pan-tilt 340 , where the control command is used to control the stepping motor 341 in the pan-tilt 340 to work to track the target object.

It should be understood that when the dome camera 300 turns to the monitoring area of the gun 200 , the camera 330 of the dome 300 can photograph the target object in the monitoring area of the gun 200 . At this time, the processing device 100 performs target detection and feature matching on the first frame of the second image sent by the dome camera 300, and when it is determined that the second image includes a target object, the dome camera 300 is triggered to track the target object. The feature matching performed by the processing device 100 on the first frame of the second image may be to match the second image with the first image (ie, the square frame including the target object in the image collected by the gun machine 200 ). If the ball camera 300 starts to track the target object, the ball camera 300 can send the second image of each frame to be processed to the processing device 100, and the processing device 100 executes S303 to control the ball camera 300 to track the target object .

During specific implementation, the processing device 100 includes a complete ball machine control flow, and for details of the flow, please refer to the relevant descriptions of S11 to S26 corresponding to FIG. 2 .

It should be noted that, in one case, the frequency of processing the second image in the processing device 100 may be consistent with the frequency of collecting the second image by the camera 330, for example, the camera 330 collects a second image every 40 milliseconds, and After the collected second image is sent to the processing device 100, the processing device 100 processes the received second image every 40 milliseconds. Or, in another case, the frequency of processing the second image in the processing device 100 may also have a certain multiple relationship with the frequency at which the camera 330 collects the second image, for example, the camera 330 collects one frame of the second image every 40 milliseconds , and send the collected second image to the processing device 100, and the processing device 100 processes the latest received second image every 200 milliseconds; or, the camera 330 collects a second image every 40 milliseconds, but every 200 milliseconds In 200 milliseconds, the newly collected frame of the second image is sent to the processing device 100, and the processing device 100 processes the received second image every 200 milliseconds.

It should be noted that the period for sending the control command 2 is preset in the processing device 100 , and the period may be greater than the excitation response time delay of the stepping motor 341 .

When specifically implemented, the implementation process of S303 includes: S3031, the processing device 100 processes the second image according to the target detection algorithm and the tracking algorithm, and determines whether the second image includes the target object, and if it does, it is determined that it is currently in a normal state, if If not included, it is determined that it is currently in an abnormal state; S3032, the processing device 100 generates a corresponding control instruction 2 according to the current state; S3033, the processing device 100 periodically sends the control instruction 2 to the ball machine 300 according to a preset rule. Wherein, the second image includes the target object, which may include that the confidence of the square frame including the target object in the second image is greater than a preset confidence threshold, and the second image is incompletely detected in the second image.

In this embodiment, in order to achieve more efficient and orderly control of the dome machine, the processing device 100 in S3031 subdivides the current state of the dome machine 300 into an initial state, a normal state, an abnormal state and an idle state, wherein the abnormal state The state is divided into: possible loss state, definite loss state and possible recovery state according to the actual tracking situation.

It can be visually understood that the processing device 100 has a state flow diagram as shown in FIG. 4 . Referring to FIG. 4 , when the processing device 100 is triggered to track the target object, the corresponding state is the initial state 0. At this time, if the processing device If the second image 1 received by the processing device 100 includes the target object, it is determined that the current corresponding state becomes the normal state 1; if the second image 1 received by the processing device 100 does not include the target object, then it is determined that the current corresponding state becomes the normal state 1. State 2 may be lost. After entering the normal state 1, if the second image 2 received by the processing device 100 includes the target object, it is determined that the current corresponding state remains the normal state 1; once the second image 2 received by the processing device 100 does not include the target object, then , it is determined that the current corresponding state becomes the possible loss state 2. After entering the possible loss state 2, once the processing device 100 receives the second image 3 including the target object within the preset time t1, it is determined that the current corresponding state becomes the normal state 1; If any frame of the second image 3 is received within a period of time that does not include the target object, it is determined that the current corresponding state remains in the possible loss state 2; if the processing device 100 reaches the preset time t1, the received second image 3 does not include the target object object, then it is determined that the current corresponding state becomes the determined loss state 3. After entering the determined loss state 3, once the processing device 100 receives the second image 4 including the target object within the preset time t2, it is determined that the current corresponding state becomes possible to retrieve the state 4; if the processing device 100 is in the preset time t2 If any frame of the second image 4 is received within the time t2 and does not include the target object, then it is determined that the current corresponding state remains the determined loss state 3; if the processing device 100 reaches the preset time t2, neither of the received second images 4 If the target object is included, it is determined that the current corresponding state becomes the determined idle state 5 . After entering the possible retrieval state 4, once the processing device 100 receives the second image 5 that does not include the target object within the preset time t3, it is determined that the current corresponding state becomes the determined loss state 3; Assuming that the target object is included in any frame of the second image 5 received within the time t3, it is determined that the current corresponding state remains the possible retrieval state 4; if the processing device 100 reaches the preset time t3, the received second image 5 is all in the state 4. If the target object is included, it is determined that the current corresponding state becomes the normal state 1. From the determined loss state 3 to the idle state 5, it can be considered that the tracking of the target object has completely failed. At this time, the processing device 100 can send an exit instruction to the dome camera 300 to instruct the dome camera 300 to stop tracking the target object; and, processing The device 100 may also send a tracking failure notification to the central device 400 to inform the central device 400 that the tracking of the target object fails.

If it is determined that the ball machine 300 is in the normal state 1, the processing device 100 can calculate the adjustment speed according to the method shown in FIG. 7 below. The control command 2 generated in S3032 is a movement command generated based on the adjustment speed, and the movement command includes the adjustment speed. speed. Compared with adjusting the angle included in the moving command, adjusting the monitoring image of the camera at the speed makes the rotation of the stepping motor 341 smoother, thereby ensuring that the tracking of the target object can be more detailed and smooth.

If it is determined that the ball machine 300 is in an abnormal state, the control command 2 generated by the processing device 100 in S3032 may be a stop command, a magnification reduction command, or a return to a preset command. As an example, when the dome camera 300 enters the possible loss state 2, the processing device 100 may generate a stop instruction 1, which is used to instruct the stepper motor 341 in the gimbal 340 to stop working, and instruct the camera 330 to stop changing magnification. As another example, when the dome camera 300 enters the determined loss state 3, the processing device 100 may generate a magnification reduction instruction, which is used to instruct the camera 330 to reduce the magnification to expand the monitoring range and increase the probability of finding the target object. As yet another example, when the dome camera 300 enters the idle state 5, the processing device 100 may generate a return to the preset point instruction, and the return to the preset point instruction is used to instruct the stepping motor 341 to drive the camera 330 to return to the preset point, and to exit the target object tracking. As another example, when the dome camera 300 enters the possible recovery state 4, the processing device 100 may generate a stop instruction 2, which is used to instruct the stepper motor 341 in the gimbal 340 to stop working, and instruct the camera 330 Stop changing the magnification.

The preset rules for sending control instruction 2 in S3033 can be implemented by adding mandatory execution flags to various instructions in the dome communication protocol. Considering that the movement command is time-sensitive, and it takes time to send, process and execute the command, and the stepper motor also has an excitation response delay, so the movement command does not have to be executed every time, only the distance is sent when the command is sent. The movement command generated at the most recent time can achieve the best adjustment effect for the ball machine 300, so as to ensure that the target object is not easily lost. Based on this, the forced flag bit of the move instruction can be set to 0, indicating that the move instruction is not a necessary instruction for forced execution. However, the stop command, the magnification reduction command and the return to the preset point command are the commands that must be executed by the dome camera 300 to handle exceptions. If they are not executed, the exception may not be handled correctly, resulting in a complete failure to track the target object. Therefore, the stop command, The mandatory flag bit of the lower magnification command and the command of returning to the preset point can be set to 1, indicating that these commands are necessary commands that need to be forced to be executed. It should be noted that, in the preset rule, if the processing device 100 saves a non-mandatory execution instruction that is not executed, then the subsequently generated non-mandatory execution instruction or mandatory execution instruction can overwrite the previously generated non-mandatory execution instruction; if The processing device 100 saves the mandatory execution instructions that are not executed, then the non-mandatory execution instructions generated subsequently are discarded, the mandatory execution instructions generated subsequently are also saved, and the two mandatory execution instructions are periodically sent in the order in which they were generated. Give the ball machine 300.

For example, suppose that when cycle 1 is reached, there are no unsent control commands in the processing device 100, and the dome camera 300 is always in the normal state 1 during the cycle 1, and the movement command 1, the movement command 2 and the movement command are generated in the order of time. 3. Then, when the cycle 1 ends and the next cycle 2 starts, the processing device 100 sends the newly generated movement command 3 to the ball machine 300, and the movement command 1 and the movement command 2 can be discarded by the processing device 100. For another example, suppose that when cycle 3 is reached, there are no unsent control commands in the processing device 100, and the dome camera 300 is always in an abnormal state during cycle 3, and the stop command 1, the stop command 2 and the reduction ratio are sequentially generated in the order of time. Command 1, then, when the cycle 3 ends and the next cycle 4 starts, the processing device 100 sends the earliest generated stop command 1 to the dome camera 300, the move command 2 and the magnification reduction command 1 can be saved by the processing device 100 ; When entering the next cycle 5 from cycle 4, the processing device 100 sends the stop command 2 generated after the stop command 1 to the dome camera 300; when entering the next cycle 6 from cycle 5, the processing device 100 sends the dome camera 300 Decrease magnification command 1 generated after stop command 2 is sent.

In a preset cycle of sending the control command 2, in one case, if the dome camera 300 is in the normal state 1 and there is no unsent stop command, magnification reduction command and return to the preset point command in the processing device 100, then , the processing device 100 may generate multiple movement commands, but only saves the latest movement command as the control command 2 to be sent. When the time to send the control command 2 arrives, the processing device 100 sends the latest generated movement to the dome camera 300 instruction. In another case, if the dome camera 300 changes from the normal state 1 to the possible loss state 2, then the processing device 100 can generate at least one stop command, each stop command is stored in the processing device 100, and when the control command is sent At the time of 2, the processing device 100 sends the stop command 1 that is generated earliest in the cycle to the ball machine 300. In another case, if the dome camera 300 changes from the possible loss state 2 to the definite loss state 3, then the processing device 100 may generate at least one magnification reduction command, and each reduction magnification command is stored in the processing device 100. When the control command 2 is sent, if there is a stop command 2 to be sent, the stop command 2 is sent to the pan/tilt 340; if there is no stop command to be sent, a magnification reduction command 1 is sent to the pan/tilt 340. In another case, if the dome machine 300 changes from the determined loss state 3 to the idle state 5, then the processing device 100 may generate at least one return to preset point instruction, and each return to preset point instruction is stored in the processing device 100 , when the time to send the control command 2 arrives, if there is still a lower magnification command 2 to be sent, then send the lower magnification command 2 to the pan/tilt 340; if there is no lower magnification command to be sent, then send the Return to preset point command 1. In another case, if the dome camera 300 changes from the determined loss state 3 to the possible recovery state 4, then the processing device 100 may generate at least one stop command, and each stop command is stored in the processing device 100. At the time of the control command 2, if there is still a magnification reduction command 3 to be sent, send the magnification reduction command 3 to the pan/tilt 340; if there is no magnification reduction command to be sent, then send a stop command 3 to the pan/tilt 340. In another case, if the dome camera 300 changes from the possible recovery state 4 to the normal state 1, then the processing device 100 may generate at least one movement command, and if there is still a stop command 4 to be sent, then all the generated movements are lost. command, when the time to send the control command 2 is reached, send the stop command 4 to the PTZ 340; if there is no stop command to be sent, only the latest generated movement command is saved, and when the time to send the control command 2 is reached, it will be sent to the PTZ 340. The pan/tilt 340 sends the newly generated movement command.

If the processing device 100 determines that the second image sent by the dome camera 300 includes the key information of the target object, the processing device 340 can also transfer the tracking state flow to the idle state 5 shown in FIG. 4 . At this time, the processing device 100 A return to preset point instruction can be sent to the dome camera 300 to control the stepping motor 341 to drive the camera 330 to return to the preset point and to exit the tracking of the target object. The instruction for returning to the preset point includes a preset point identifier, and the preset point identifier is used to uniquely identify the preset point. In one case, the related information of at least one preset point (including the preset point identifier) can be preset and recorded in the dome camera 300. When the dome camera 300 receives the instruction to return to the preset point, it can parse the return preset point. The instruction acquires the preset point identifier, so as to determine the preset point to be returned based on the preset point identifier, so as to return to the corresponding preset point based on the relevant information of the preset point, and complete the instruction of returning to the preset point. In another case, the processing device 300 presets and records the relevant information of at least one preset point (including the preset point identifier and position) for the dome machine, and the processing device 100 generates and sends the return preset point of the dome machine 300 The command includes the position of the preset point. When the dome camera 300 receives the instruction of returning to the preset point, it can parse the instruction of returning to the preset point to obtain the position of the preset point, so as to return to the corresponding preset point based on the position of the preset point. Complete the command of returning to the preset point.

The local storage manner of the generated control instruction 2 to be sent in the processing device 100 is not specifically limited in this embodiment.

As an example, at least one control instruction 2 may be stored in the processing device 100 in the form of an instruction queue. When the instruction queue of the processing device 100 is empty and the move instruction is continuously generated, the move instruction generated before can be overwritten with the move instruction generated later to ensure that only the move instruction generated newly is stored in the instruction queue. When the command queue of the processing device 100 stores a move command, and generates a stop command, a magnification reduction command, or a return to preset command, the stop command, magnification reduction command, or return to preset command generated later can be used to overwrite the previously generated movement command. command to ensure that only stop commands, lower magnification commands or return to preset commands are stored in the command queue. When the instruction queue of the processing device 100 stores a stop instruction, a magnification reduction instruction, or a return to a preset point instruction, and a move instruction is generated, the move instruction can be discarded. When the command queue of the processing device 100 includes a stop command, a magnification reduction command, or a return to preset command, and a stop command, a magnification reduction command, or a return to preset command is generated, the newly generated stop command, magnification reduction command may be Or the command to return to the preset point is also saved, and then periodically sent to the ball machine 300 according to the generation sequence.

For example: Assuming that the preset cycle of the processing device 100 for sending control commands is 120 milliseconds, as shown in FIG. 5 , the specific situation of the command queue is: t=10 milliseconds, the processing device 100 generates a move command as Move 1, the The instruction queue only includes Move 1, and the processing device 100 sends the Move 1 in the instruction queue to the PTZ 340 of the ball camera 300, and the PTZ 340 controls the rotation of the stepping motor and/or the zoom ratio according to the parameters in the Move 1; the first In a cycle, t=50 milliseconds, the processing device 100 generates Move 2, the instruction queue only includes Move 2, t=90 milliseconds, the processing device 100 generates Move 3 and discards Move 2, the instruction queue only includes Move 3, t= 130 milliseconds, the processing device 100 generates a stop instruction Stop 1 and discards Move 3. At this time, the instruction queue only includes Stop 1, and the processing device 100 sends Stop 1 to the PTZ 340, and the PTZ 340 controls the stepping motor 341 according to the Stop 1. Stop the rotation and the zoom ratio; in the second cycle, t=170 milliseconds, the processing device 100 generates Stop 2, the instruction queue only includes Stop 2, and when t=210 milliseconds, the processing device 100 generates Move 4 and discards Move 4, the The instruction queue only includes Stop 2, t=250 milliseconds, the processing device 100 generates a return preset point instruction GotoPreset 1, the instruction queue includes Stop 2 and GotoPreset 1, the processing device 100 sends Stop 2 to the PTZ 340, and the PTZ 340 follows this Stop 2 controls the stepping motor 341 to stop rotating and changing the magnification; in the third cycle, t=290 milliseconds, the processing device 100 generates Move 5 and discards Move 5, the instruction queue includes GotoPreset 1, and when t=330 milliseconds, the processing device 100 generates Move 6 and discards Move 6, the instruction queue includes GotoPreset 1, t=370 milliseconds, the processing device 100 generates Move 7 and discards Move 7, the instruction queue includes GotoPreset 1, the processing device 100 sends GotoPreset 1 to the PTZ 340, The PTZ 340 controls the stepping motor 341 to rotate to the preset point according to the GotoPreset 1.

Wherein, as shown in FIG. 5 , the movement instruction Move may include a first adjustment speed V _pan in the first direction, a second adjustment speed V _tilt in the second direction, a first magnification adjustment speed V _zoom and a forced flag F, wherein , the calculation methods of the values of V _pan in the first direction and V _pan in the second direction refer to the relevant description in the embodiment shown in Figure 7 below; the value of V _zoom is the preset magnification adjustment speed, and the direction is based on The actual state of the target object in the second image is determined; the value of the mandatory flag bit F is 0. The stop command Stop can include the stop rotation indicator bit T, the stop zoom ratio indicator bit M, and the mandatory flag bit F. Usually, the stop rotation indicator bit T takes the value of 1, and the stop zoom indicator bit M takes the value of 1, and the mandatory value is 1. The value of the flag bit F is 1. In this case, the Stop is used to instruct the dome camera 300 to stop rotating and stop changing the magnification, and the Stop is an instruction that needs to be enforced. The preset point return command GotoPreset can include the preset point identifier D and the mandatory flag bit F. Usually, the preset point identifier D is used to uniquely identify a preset point of the dome camera 300, and the value of the mandatory flag bit F is 1. In this case, the GotoPreset is used to instruct the dome camera 300 to return to the preset point corresponding to the identifier D, for example, the preset point named home, and the GotoPreset is an instruction that needs to be enforced. In addition, when the processing device 100 determines that the dome camera 300 is in the lost state 3, it can also generate a magnification reduction command ZoomOut (also referred to as a zoom-out command), and the ZoomOut can include a stop rotation indication bit T, a stop zoom ratio indication bit M and The mandatory flag bit F, usually, the value of the stop rotation indicator bit T is 1, the value of the stop zoom indicator bit M is 0, and the value of the mandatory flag bit F is 1. In this case, the ZoomOut is used to indicate the ball The machine 300 stops rotating and reduces the magnification to expand the monitoring range, so as to search for the target object from a larger monitoring area and increase the probability of retrieving the target object. ZoomOut is an instruction that needs to be enforced. It should be noted that ZoomOut and Stop can be two different types of instructions, or two instructions with different functions under the same type of instruction, which are distinguished based on the value of the stop zoom indicator bit M, and are implemented in this application. Examples are not specifically limited.

In this embodiment, the above-mentioned PID algorithm calculates the parameters V _pan and V _tilt in the movement command Move. In order to more accurately calculate the parameters V _pan and V _tilt that make the tracking target object have the best effect, in the PID calculation provided by this embodiment, prominent contribution points include:

On the one hand, the corresponding relationship between the magnification and the PID parameter group {K _P , K _I and K _D } is preset in the processing device 100 , for example, the magnification change of the camera 330 in the dome camera 300 is divided into 1.0～30.0 In this case, 10 sets of corresponding relationships are preset: 1.0~2.9 correspond to PID parameter group 1 {K _P1 , K _I1 and K _D1 }, 3.0~5.9 correspond to PID parameter group 2 {K _P2 , K _I2 and K _D2 },... , 24.0~26.9 correspond to PID parameter group 9 {K _P9 , K _I9 and K _D9 } and 27.0~30.0 correspond to PID parameter group 10 {K _P10 , K _I10 and K _D10 }. In this way, when V _pan and V _tilt need to be calculated, the corresponding PID parameter group can be determined directly according to the current magnification of the camera 330 in the dome camera 300, and V _pan and V can be calculated based on the above formulas (1) and (2). _tilt .

On the other hand, considering that the adjustment speeds V _pan and V _tilt during the movement command Move executed by the stepping motor 341 are discrete, while the adjustment speed of the PID output is continuous, the processing device 100 calculates V based on the PID Before _pan and V _tilt , it is also possible to judge whether the target object is in the center area of the second image, and if so, set the deviation between the center point of the second image at the current moment and the center point of the target object in the second image. PID calculation is performed after zero; if not, PID calculation is directly performed according to the deviation between the center point of the second image at the current moment and the center point of the target object in the second image. Wherein, the center area of the second image is preset by the processing device 100, and preset: the deviation between the center point of the second image and the center point of the target object in the second image Δx≤Δx _threshold and Δy≤Δ y _threshold , as shown in Figure 6, the coordinates of the center point of the second image are (x0, y0), the center point of the square frame including the target object in the second image is (x, y), and the deviation is (△x, Δy), the preset length of the central region in the x direction is the Δx _threshold , and the length in the y direction is the Δy _threshold , where Δx=|x-x0|, Δy=|y-y0|. At time t, if △x(t)≤△x _threshold and △y(t)>△y _threshold , then, after setting △y(t) in formula (2) to zero, calculate V _tilt according to formula (2) , do not perform instruction processing on △x(t), directly bring it into formula (1) to calculate V _pan ; if △x(t)>△x _threshold and △y(t)≤△y _threshold , then formula (1) After setting △x(t) to zero, calculate V _pan according to formula (1), without performing instruction processing on △y(t), directly bring it into formula (2) to calculate V _tilt ; if △x(t)≤△x _threshold And △y(t)≤△y _threshold , then, after setting △x(t) in formula (1) to zero, calculate V _pan according to formula (1), and set △y(t) in formula (2) to After zero, calculate V _tilt according to formula (2); if △x(t)>△x _threshold and △y(t)>△y _threshold , then do not perform instruction processing on △x(t), and directly bring it into formula (1) ) to calculate V _pan , without performing instruction processing on △y(t), directly bring it into formula (2) to calculate V _tilt . In this way, for the case where the target object is located in the center of the second image or the target object is not far from the center of the second image, by setting the deviation in the corresponding direction to zero, the PID outputs a more accurate adjustment speed in the corresponding direction.

On the other hand, considering that the stepper motor 341 rotates according to different rotation gears, and usually has the smallest gear (eg, 0.1 gear), then, in order to avoid that the V _pan and V _tilt calculated by the PID are too small (for example, less than 0.1 ) causes the stepping motor 341 to rotate too far, so that the rotation process of the stepping motor ₃₄₁ _shakes _back and forth and cannot be _stabilized . The dead zone is judged separately. When it is in the dead zone, the adjustment speed is set to zero (that is, no adjustment is performed in this direction). For example, the processing device 100 can preset the dead zone to be -ζ˜ζ, usually the value of ζ is the minimum rotation gear of the stepper motor 341, for example, ζ=0.1, then, if V _pan =0.07, then, because -0.1≤0.07≤0.1, then, set V _pan =0; if _Vtilt =-0.05, then, since -0.1≤-0.05≤0.1, then set _Vtilt =0.

Next, the specific process of calculating the parameters V _pan and V _tilt in the movement command Move by using the PID algorithm in the present embodiment is further described in conjunction with FIG. 7 as follows:

S31, obtaining the center point (x, y) of the square frame including the target object in the second image;

S32, calculate the deviation (Δx, Δy) between (x, y) and the center point (x0, y0) of the second image;

S33, respectively determine whether △x is less than or equal to the △x _{threshold value} , and whether △y is less than or equal to the △y _{threshold value} , if yes, then perform S34 after setting the corresponding deviation to zero, if not, directly perform S34;

S34, obtain the current magnification n of the ball camera 300;

S35, according to the preset magnification and the corresponding relationship between the PID parameter groups {K _P , K _I and K _D }, determine the PID parameter group n {K _Pn , K _In and K _Dn } corresponding to the magnification n;

S36, calculate V _pan and V _tilt according to the PID parameter groups n{K _Pn , K _In and K _Dn }, Δx, Δy and formula (1) and formula (2);

S37, respectively determine whether V _pan and V _tilt are located in the preset dead zone, if so, set V _pan and/or V _tilt located in the dead zone to zero and then execute S38, if otherwise, directly execute S38;

S38 , generate a move instruction Move according to V _pan and V _tilt , where Move includes V _pan and V _tilt .

It should be noted that the central area judgment (i.e. S33) and the dead zone judgment (i.e. S37) are both optional steps to improve the monitoring effect, and whether to perform is not specifically limited in the embodiments of the present application.

Although adjusting the magnification of the camera 330 can adjust the size of the monitoring range, considering that the magnification adjustment speed is too fast, the second image collected by the dome camera 330 when the magnification is changed will be blurred, which will seriously affect the tracking of the target object. Then, in this embodiment, the magnification adjustment speed is determined by using a preset value instead of using the PID algorithm for calculation, and the preset value is used as the magnification adjustment speed to adjust the magnification, which can ensure that the second data collected by the camera 330 during the magnification adjustment process can be ensured. The image is also sharp and does not affect the tracking of the target object. In specific implementation, assuming that the preset value is a (a>0), the method for determining the magnification adjustment speed V _zoom may include: S41, obtaining the proportion of the target object in the second image in the second image; S42, judging the proportion and the preset Set the size relationship between the ratio threshold 1 and the ratio threshold 2, if the ratio is less than the ratio threshold 1, then execute S43, if the ratio is greater than or equal to the ratio threshold 1 and less than or equal to the ratio threshold 2, then execute S44, if the ratio is greater than or equal to the ratio threshold 2, then execute S44 If the scale threshold value is 2, execute S45, wherein the scale threshold value 1 is smaller than the scale threshold value 2; S43, determine V _zoom as a, so as to control the camera 330 to increase the magnification of the camera 330 according to a; S44, determine V _zoom as 0, so as to control the wrong Adjust the magnification of the camera 330; S45, determine V _zoom to be -a, so as to control the camera 330 to reduce the magnification of the camera 330 according to a. It should be noted that, the larger the magnification of the camera 330, the smaller the monitoring range, and the larger the proportion of the target object in the second image; conversely, the smaller the magnification of the camera 330, the larger the monitoring range, and the larger the target object. The larger the proportion in the second image.

The processing device 100 may determine a reference factor for magnification adjustment according to different target objects. For example, when the target object is an elongated shape (such as a person), the height h of the target object in the second image is determined as the magnification. The reference factor for adjustment; for another example, when the target object is a short and wide shape (such as a vehicle), the width w of the target object in the second image is determined as the reference factor for magnification adjustment; for another example, when the target object is short and wide If the shape is wide (such as a vehicle), but the width is approximately equal to the width of other objects, it is determined that the height h and width w of the target object in the second image are taken together as a reference factor for magnification adjustment.

The target object is a person, the height h is used as the reference factor for magnification adjustment, the preset value a=1.5/sec, and the preset scale threshold value 1 and scale threshold value 2, the scale threshold value 1 is 1/3, and the scale threshold value is 2 bits 2/3 For example, if at time t1, the processing device 100 determines that the ratio of the height of the person to the height of the second image 1 is 1/5, then, since 1/5 is less than 1/3, the generated movement instruction Move includes the magnification ratio The adjustment speed V _zoom is 1.5/second; if at time t2, the processing device 100 determines that the ratio of the height of the person to the height of the second image 2 is 1/2, then, since 1/2 is less than 2/3 and greater than 1/ 3. The magnification adjustment speed V _zoom included in the generated movement instruction Move is 0; if at time t3, the processing device 100 determines that the ratio of the height of the person to the height of the second image 3 is 5/6, then, since 5/ 6 is greater than 2/3, the magnification adjustment speed V _zoom included in the generated movement command Move is -1.5/sec.

It can be seen that with the method provided in this embodiment, the processing device 100 can control the stepping motor 341 of the pan/tilt 340 in the dome camera 300 to work when acquiring the first image including the target object to be tracked sent by the trigger 200 in the monitoring system , in order to drive the dome camera 300 to rotate to the monitoring area corresponding to the bolt 200, so that the camera in the dome camera 300 can collect the second image including the target object and send it to the processing device 100. The processing device 100 can be based on the dome camera The second image sent by 300 generates a control command for the dome camera 300, and periodically sends control commands to the PTZ 340 to control the stepping motor 341 in the PTZ 340 to work, so as to track the target object . Since this method considers that the rotation of the camera 330 in the dome camera 300 is driven by the rotation of the stepper motor 341, the stepper motor 341 has a certain excitation response time delay, and the process of capturing images by the camera 330 in the dome camera 300 is discrete The processing device 100 periodically sends control commands to the dome camera 300 with rotatable and variable magnification by presetting the period for sending control commands, which effectively overcomes the lack of smooth rotation of the camera 330 currently controlling the dome camera 300. The shortcoming of stability makes the rotation process of the dome machine 300 no longer stuck, and realizes smooth and stable tracking of the target object, thereby improving the use experience of the dome machine 300 .

The ball machine control method provided by this embodiment is described in detail above with reference to FIG. 2 and FIG. 3 , and the ball machine control device and equipment provided according to this embodiment will be described below with reference to FIG. 8 and FIG. 9 .

FIG. 8 provides a ball camera control apparatus 800 provided in this embodiment, the ball camera control apparatus 800 is applied to a processing device, and the ball camera control apparatus 800 includes: a determination unit 801, a transmission unit 802, and a tracking unit 803;

A determination unit 801, used for determining the target object to be tracked;

The sending unit 802 is used to periodically send a control command to the dome camera, the control command is used to control the stepping motor of the pan/tilt in the dome camera to drive the camera in the dome camera to rotate to the monitoring area where the target object is located;

The tracking unit 803 is used for tracking the target object based on the image collected by the camera in the dome camera.

Among them, the period of sending control commands to the dome camera is greater than the excitation response time delay of the stepping motor.

Optionally, the determining unit 801 is further configured to determine the control instruction according to the tracking result of the target object in the image before periodically sending the control instruction to the dome camera, where the tracking result includes a normal state and an abnormal state.

As an example, when the tracking result of the target object is in a normal state, the control instruction is a first control instruction, and the first control instruction is used to instruct the dome camera to continue tracking the target object.

Optionally, the sending unit 802 is further configured to: send the newly generated first control instruction in the same cycle to the ball machine.

Optionally, the determining unit 801 is further configured to determine the first control instruction according to a preset rule, and the preset rule includes determining the first control instruction according to the magnification of the camera in the dome camera and the proportional-integral-derivative PID parameter group.

Optionally, the determining unit 801 includes: a first determining subunit, a second determining subunit and a third determining subunit. Among them, the first determination subunit is used to determine the PID parameter group corresponding to the magnification according to the current magnification of the camera in the dome camera; the second determination subunit is used to determine the first adjustment speed and the second adjustment speed according to the PID parameter group ; a third determination sub-unit for determining the first control command according to the first adjustment speed and the second adjustment speed.

Optionally, the second determination subunit is further configured to determine the first adjustment speed according to the PID parameter and the first deviation in the first direction, wherein the first deviation is the center point of the image and the area where the target object is located in the image. The distance between the center points in the first direction.

Optionally, the determining unit 801 further includes a first zero-setting sub-unit, where the first zero-setting sub-unit is configured to, before determining the first adjustment speed according to the PID parameters and the first deviation in the first direction, if the first If the deviation is smaller than the preset first deviation threshold, the first deviation is set to zero, and the first deviation threshold is determined based on the preset central area of the image.

Optionally, the determining unit 801 further includes a second zero-setting subunit, the second zero-setting subunit is configured to set the first adjustment speed to zero if the first adjustment speed is within the dead zone range, and the dead zone The range is determined according to the adjustment accuracy of the stepper motor.

Optionally, the preset rule may further include determining the first control instruction according to the proportion of the target object in the image in the preset direction.

Optionally, the determining unit 801 further includes: an acquiring subunit, a fourth determining subunit, and a fifth determining subunit. Among them, the acquisition subunit is used to acquire the proportion of the target object in the image in the preset direction; the fourth determination subunit is used to determine the target magnification adjustment speed according to the proportion and the preset proportion threshold, and the target magnification adjustment speed is preset The magnification adjustment speed or the opposite number of the preset magnification adjustment speed; the fifth determination subunit is used for determining the first control command according to the target magnification adjustment speed.

As another example, when the tracking result of the target object is in an abnormal state, the control instruction is a second control instruction, and the second instruction is used to instruct the dome camera to control the dome camera in an abnormal state.

Optionally, the determining unit 801 is further configured to determine a second control instruction according to a preset rule, the preset rule includes determining a second control instruction according to the type of abnormal state, and the second control instruction includes at least one of the following instructions: stop command, reduce magnification command or return to preset point command.

Optionally, the sending unit 802 is further configured to send the earliest generated second control instruction stored in the instruction queue to the ball machine.

In some possible implementations, the abnormal state includes a first abnormal state, a second abnormal state, a third abnormal state and a fourth abnormal state, and the apparatus 800 further includes: a first processing unit to a fifth processing unit. Among them, the first processing unit is used to determine that the dome camera has entered the first abnormal state when the dome camera is in a normal state and the collected image does not include the target object, and instruct the stepper motor in the gimbal to stop working; the second process The unit is used when the dome camera is in the first abnormal state and the multi-frame images collected at the first preset time do not include the target object, or when the dome camera is in the first abnormal state and the dome camera is in the first preset time The images collected within the time include the target object, determine that the dome camera has entered the second abnormal state, and instruct the camera to reduce the magnification to expand the monitoring range; the third processing unit is used for when the dome camera is in the second abnormal state and in the second preset state. When the target object is not included in the multi-frame images collected at the time, it is determined that the dome camera enters the third abnormal state, and the stepper motor is instructed to drive the camera to return to the preset point and exit the tracking of the target object; the fourth processing unit is used for When the dome camera is in the second abnormal state and the collected image includes the target object, it is determined that the dome camera enters the fourth abnormal state, and the stepping motor in the gimbal is instructed to stop working; the fifth processing unit is used for when the dome camera is in the fourth abnormal state. The fourth abnormal state and the multi-frame images collected by the dome camera within the third preset time all include the target object, or, when the dome camera is in the first abnormal state and the images collected by the dome camera within the first preset time include the target object. The target object, confirm that the dome camera enters the normal state.

Wherein, the sending unit 802 is used to send the first stop instruction to the PTZ when the dome machine changes from the normal state to the first abnormal state within one cycle of sending the control command to the dome machine; or, the sending unit 802 is used to When the dome camera changes from the first abnormal state to the normal state within one cycle of sending the control command to the dome camera, if there is a second stop command to be sent, it will send the second stop command to the PTZ; if there is no second stop command to be sent If the second stop command is given, then send the first movement command to the PTZ; or, the sending unit 802 is used to change the dome machine from the first abnormal state to the second abnormal state within one cycle of sending the control command to the dome machine When there is still a third stop command to be sent, send a third stop command to the PTZ; if there is no third stop command to be sent, send a first lower magnification command to the PTZ; or, the sending unit 802 , used to send the second reduction ratio command to the PTZ when the dome camera changes from the second abnormal state to the third abnormal state within one cycle of sending the control command to the dome camera, if there is still a second reduction ratio command to be sent. magnification command; if there is no second magnification reduction command to be sent, send the first return preset point command to the pan/tilt; When changing from the second abnormal state to the fourth abnormal state, if there is still a third command to reduce the magnification to be sent, send the third command to reduce the magnification to the PTZ; if there is no third command to reduce the magnification to be sent, send the command to The PTZ sends the fourth stop command; or, the sending unit 802 is used for when the dome machine changes from the fourth abnormal state to the normal state within one cycle of sending the control command to the dome machine, if there is still a fifth stop command to be sent , then, send the fifth stop instruction to the PTZ; if there is no fifth stop instruction to be sent, then send the newly generated movement instruction to the PTZ; or, the sending unit 802 is used to send the control instruction to the ball machine In one cycle, when the dome camera changes from the fourth abnormal state to the second abnormal state, if there is a sixth stop command to be sent, it will send the sixth stop command to the PTZ; if there is no sixth stop command to be sent, Then, send the fourth reduction magnification command to the PTZ.

In some possible implementations, the apparatus 800 may further include: a sixth processing unit. The sixth processing unit is used for determining that the dome camera has entered the fifth abnormal state when the dome camera has acquired the key information of the target object, and instructing the stepper motor to drive the camera to return to the preset point and exit the tracking of the target object. As an example, the sending unit 802 may include: a first sending subunit and a second sending subunit. The first sending subunit is used to send the first control command in the first command queue to the PTZ at the first moment; the second sending subunit is used to send the second command to the PTZ at the second moment The second control instruction in the queue. Among them, the time from the first time to the second time is one cycle of the ball machine sending the control command, the control command sent at the first time is the first control command, and the control command sent at the second time is the second control command ; If all the control instructions generated at the first moment only include movement instructions, then, the first control instructions only included in the first instruction queue, the first control instructions are the latest generated movement instructions before the first moment; All the control commands generated at a moment include at least one of a stop command, a command to reduce the magnification, or a command to return to a preset point, then the first command queue includes all stop commands, a command to reduce the magnification, and all the stop commands, the magnification decrease, and the commands generated by the first moment in sequence according to the generation sequence. An instruction or a return to a preset point instruction, the first instruction queue does not include a move instruction, and the first control instruction is an instruction in the first instruction queue whose generation time is the longest from the first time.

As an example, if the first control instruction is the first movement instruction, between the first moment and the second moment, the apparatus 800 may further include: a first queue updating unit to a third queue updating unit. The first queue updating unit is configured to generate a fifth stop instruction at the third moment, and overlay the fifth stop instruction over the first movement instruction in the first instruction queue to obtain the updated first instruction queue, and after the update The first instruction queue includes a first stop instruction; a second queue updating unit is used to generate a second moving instruction at the fourth moment, and discard the second moving instruction according to the updated first instruction queue; the third queue updating unit , used to generate a second return preset point instruction at the fifth moment, and add the second return preset point instruction to the updated fifth stop instruction of the first instruction queue to obtain the second instruction queue, the second instruction The instruction queue includes a fifth stop instruction and a second return to the preset point instruction. In this example, the second sending subunit is configured to send a fifth stop instruction to the PTZ, so as to control the stepping motor in the PTZ to stop working. In this example, the sending unit 802 is further configured to send the second return preset point instruction to the gimbal at the sixth time after the second time, so as to control the stepping motor in the gimbal to drive the dome camera to return to the preset point, wherein , the time from the second moment to the sixth moment is one cycle.

It should be understood that the apparatus 800 in this embodiment of the present application may be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD may be a complex program logic device (complex programmable logical device, CPLD), field-programmable gate array (field-programmable gate array, FPGA), general array logic (generic array logic, GAL) or any combination thereof. When the ball machine control method shown in FIG. 3 can also be implemented by software, the device 800 and its respective units can also be software modules.

The dome machine control device 800 according to this embodiment may correspond to executing the method described in this embodiment, and the above-mentioned and other operations and/or functions of each unit in the dome machine control device 800 are respectively for the purpose of realizing the method in FIG. 3 . For the sake of brevity, the corresponding process is not repeated here.

FIG. 9 is a schematic diagram of a ball machine control device 900 provided in this embodiment. As shown in the figure, the ball machine control device 900 includes a processor 901 , a memory 902 , a communication interface 903 and a memory unit 904 . The processor 901, the memory 902, the communication interface 903, and the memory unit 904 communicate through the bus 905, and can also communicate through other means such as wireless transmission. The memory 902 is used for storing instructions, and the processor 901 is used for executing the instructions stored in the memory 902 . The memory 902 stores program codes, and the processor 901 can call the program codes stored in the memory 902 to perform the following operations:

Determine the target object to be tracked;

Periodically send control commands to the dome camera, where the control commands are used to control the stepping motor of the pan/tilt in the dome camera to drive the camera in the dome camera to rotate to the monitoring area where the target object is located;

The target object is tracked based on images collected by a camera in the dome camera.

It should be understood that in this embodiment of the present application, the processor 901 may be a CPU, and the processor 901 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or any conventional processor or the like.

The memory 902 , which may include read-only memory and random access memory, provides instructions and data to the processor 901 . Memory 902 may also include non-volatile random access memory. For example, memory 902 may also store device type information.

The memory 902 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of illustration and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), Double data rate synchronous dynamic random access memory (double data date SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) and direct Memory bus random access memory (direct rambus RAM, DR RAM).

In addition to the data bus, the bus 905 may also include a power bus, a control bus, a status signal bus, and the like. However, for the sake of clarity, the various buses are labeled as bus 905 in the figure.

It should be understood that the ball machine control device 900 according to the embodiment of the present application may correspond to the ball machine control device 800 in the embodiment of the present application, and may correspond to the corresponding subject in executing the method shown in FIG. 3 according to the embodiment of the present application, In addition, the above and other operations and/or functions of each module in the ball machine control device 900 are to implement the corresponding flow of each method in FIG. 3 , and are not repeated here for brevity.

As another possible embodiment, the present application further provides a monitoring system, where the monitoring system includes a first device, a second device, and a processing device, and the processing device communicates with the first device and the second device respectively. Wherein, the first device is used to acquire an image within a fixed monitoring range, and send the first image including the target object to the processing device; the processing device is used to execute each operation step of the above-mentioned method as shown in FIG. The second device sends a control instruction to control the second device, which is not repeated here for brevity; the second device is used to complete the tracking of the target object based on the control instruction sent by the processing device.

Wherein, the second device is a ball machine. The first device may be a bolt action. The specific structure can be found in Figure 1.

As an example, the monitoring system may further include a central device and a processing device, which are further used to acquire key information of the target object and send the key information to the central device; the central device is used to analyze and process the key information.

The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that contains one or more sets of available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media. The semiconductor medium may be a solid state drive (SSD).

The above descriptions are merely specific embodiments of the present application. Those skilled in the art can think of changes or substitutions based on the specific embodiments provided in the present application, which should all fall within the protection scope of the present application.

Claims

A ball machine control method, comprising:

Determine the target object to be tracked;

Periodically send control commands to the dome camera, where the control commands are used to control the stepping motor of the pan/tilt in the dome camera to drive the camera in the dome camera to rotate to the monitoring area where the target object is located;

The target object is tracked based on images collected by a camera in the dome camera.
The method according to claim 1, wherein the period of sending the control command to the dome camera is greater than the excitation response delay of the stepper motor.
The method according to claim 1 or 2, wherein before the periodic sending of control instructions to the ball machine, the method further comprises:

The control instruction is determined according to the tracking result of the target object in the image, and the tracking result includes a normal state and an abnormal state.
The method according to claim 3, wherein the method further comprises:

When the tracking result of the target object is in a normal state, the control instruction is a first control instruction, and the first control instruction is used to instruct the dome camera to continue tracking the target object.
The method according to claim 4, wherein the periodically sending a control command to the ball machine comprises:

Send the newly generated first control instruction in the same cycle to the ball machine.
The method according to claim 4 or 5, wherein the determining the control instruction according to the tracking result of the target object in the image comprises:

The first control instruction is determined according to a preset rule, and the preset rule includes determining the first control instruction according to the magnification of the camera in the dome camera and the proportional-integral-derivative PID parameter group.
The method according to claim 6, wherein the determining the first control command according to the magnification of the camera in the dome camera and the proportional-integral-derivative PID parameter group comprises:

According to the current magnification of the camera in the dome camera, determine the PID parameter group corresponding to the magnification;

According to the PID parameter group, determine the first adjustment speed and the second adjustment speed;

The first control command is determined according to the first adjustment speed and the second adjustment speed.
The method according to claim 7, wherein the determining the first adjustment speed according to the PID parameter group comprises:

The first adjustment speed is determined according to the PID parameters and the first deviation in the first direction, wherein the first deviation is the center point of the image and the center of the area where the target object is located in the image The distance between points in the first direction.
The method according to claim 8, wherein before the determining the first adjustment speed according to the PID parameter and the first deviation in the first direction, the method further comprises:

If the first deviation is smaller than a preset first deviation threshold, the first deviation is set to zero, and the first deviation threshold is determined based on a preset central area of the image.
The method according to any one of claims 7-9, wherein the method further comprises:

If the first adjustment speed is within the dead zone range, the first adjustment speed is set to zero, and the dead zone range is determined according to the adjustment precision of the stepper motor.
The method according to any one of claims 6-10, wherein the preset rule further comprises determining the first control instruction according to the proportion of the target object in the image in a preset direction.
The method according to claim 11, wherein, determining the first control instruction according to the proportion of the target object in the image in a preset direction, comprising:

obtaining the proportion of the target object in the image in the preset direction;

determining a target magnification adjustment speed according to the ratio and a preset ratio threshold, and the target magnification adjustment speed is a preset magnification adjustment speed or an opposite number of the preset magnification adjustment speed;

The first control command is determined according to the target magnification adjustment speed.
The method according to claim 3, wherein the method further comprises:

When the tracking result of the target object is in an abnormal state, the control instruction is a second control instruction, and the second instruction is used to instruct the dome camera to control the dome camera in the abnormal state.
The method according to claim 13, wherein the determining the control instruction according to the tracking result of the target object in the image comprises:

The second control instruction is determined according to a preset rule, the preset rule includes determining the second control instruction according to the type of the abnormal state, and the second control instruction includes at least one of the following instructions: a stop instruction , reduce the magnification command or return to the preset point command.
The method according to claim 14, wherein the periodically sending a control command to the ball machine comprises:

Send the earliest generated second control instruction stored in the instruction queue to the ball machine.
A ball machine control device, characterized in that it includes:

a determination unit for determining the target object to be tracked;

The sending unit is used to periodically send control instructions to the dome camera, and the control instructions are used to control the stepping motor of the pan/tilt in the dome camera to drive the camera in the dome camera to rotate to the monitoring area where the target object is located;

A tracking unit, configured to track the target object based on the image collected by the camera in the dome camera.
The device according to claim 16, wherein the period of sending the control command to the dome camera is greater than the excitation response delay of the stepping motor.
The device according to claim 16 or 17, characterized in that:

The determining unit is further configured to determine the control instruction according to the tracking result of the target object in the image before the periodic sending of the control instruction to the dome camera, where the tracking result includes a normal state and an abnormal state.
The device according to any one of claims 16 to 18, wherein the sending unit is further configured to:

Send the newly generated control command in the same cycle to the ball machine.
A device controlled by a ball machine, characterized in that it includes a processor and a memory; the memory is used to store computer instructions; the processor is used to execute any one of claims 1-15 according to the computer instructions The operation steps of the method described in the item.