EP4548301A1

EP4548301A1 - Methods, systems, and computer-readable media for target tracking

Info

Publication number: EP4548301A1
Application number: EP23879101.6A
Authority: EP
Inventors: Siming WU; Deqiang CHENG
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2022-10-20
Filing date: 2023-10-17
Publication date: 2025-05-07
Also published as: CN115760910B; WO2024083113A1; CN115760910A; EP4548301A4

Abstract

The present disclosure provide methods, systems, and non-transitory computer readable media for target tracking. The method may include determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object（210）; determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object（220）; determining, based on the first motion trajectory, a predicted position associated with the target object（230）; in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object（240）.

Description

METHODS, SYSTEMS, AND COMPUTER-READABLE MEDIA FOR TARGET TRACKING

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to Chinese Patent Application No. 202211290950.3, filed on October 20, 2022, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the field of security monitoring technology, in particular, to methods, systems, and computer-readable media for target tracking.

BACKGROUND

The coordinated monitoring method using bullet cameras and dome cameras refers to monitoring an area by using a wide-angle bullet camera, identifying objects of interest in the monitored area (e.g., pedestrians, vehicles, non-motor vehicles) , and controlling the dome camera to identify, track, and enlarge the objects of interest in the monitored area. In the actual monitoring scene, the features of a target may be different in images acquired by the bullet camera and the dome camera, or the target may be blocked in the images, which may affect the accuracy of the target tracking to a certain extent.
Therefore, it is desirable to provide a method and a system for target tracking, which can improve the accuracy of target tracking in the situation that the features of the target are different in images acquired by the bullet camera and the dome camera, or the target is blocked in the images acquired by the bullet camera and the dome camera.

SUMMARY

An aspect of the present disclosure provides a method for target tracking. The method may include: determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object; determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple second historical images may be acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera; determining, based on the first motion trajectory, a predicted position associated with the target object; and in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object, wherein the actual position associated with the target object may be determined based on a current image acquired by the bullet camera.
In some embodiments, determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object may include: for each of the multiple first historical images, determining a first historical position of the target object in the first historical image; and converting the first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on a correlation between the bullet camera and the dome camera.
In some embodiments, determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object may include: for each of the multiple second historical images, determining a second historical position of the target object in the second historical image; converting the second historical position to a third historical position of the target object in a preset coordinate system based on one of the plurality of historical monitoring positions of the dome camera corresponding to the second historical image; and generating the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images.
In some embodiments, the predicted position associated with the target object may include a predicted monitoring position of the dome camera and/or a predicted first position of the target object in the current image acquired by the bullet camera; and the actual position associated with the target object may include an actual monitoring position of the dome camera and/or an actual first position of the target object in the current image acquired by the bullet camera.
In some embodiments, the predicted position associated with the target object may include the predicted monitoring position of the dome camera, and determining, based on the first motion trajectory, a predicted position associated with target object may include: determining a predicted second position of the target object in the preset coordinate system based on the first motion trajectory; and converting the predicted second position to the predicted monitoring position of the dome camera.
In some embodiments, the actual position associated with the target object may include the actual monitoring position of the dome camera, and the method may further include: acquiring the actual monitoring position of the dome camera by: determining the actual first position of the target object in the current image acquired by the bullet camera; and converting the actual first position to the actual monitoring position of the dome camera based on a correlation between the bullet camera and the dome camera.
In some embodiments, the actual position associated with the target object may include the actual first position, the predicted position associated with the target object includes the predicted first position, and determining, based on the first motion trajectory, the predicted position associated with the target object may include: determining, based on the first motion trajectory, a predicted second position of the target object in the preset coordinate system; converting the predicted second position to the predicted monitoring position of the dome camera; and converting the predicted monitoring position of the dome camera to the predicted first position.
In some embodiments, in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object may include: determining a second difference, wherein the second difference includes a trajectory difference and/or an image feature difference; obtaining a fused difference by fusing the second difference and the first difference; and in response to determining that the fused difference satisfies the preset condition, controlling the dome camera to track the target object.
In some embodiments, the second difference may include the trajectory difference, and determining the second difference may include: determining, based on the multiple first historical images, a plurality of first historical positions of the target object; generating, based on the plurality of first historical positions, a second motion trajectory of the target object; and determining, based on the first motion trajectory and the second motion trajectory, the trajectory difference.
In some embodiments, the second difference may include the image feature difference, and determining the second difference may include: acquiring, based on the actual first position, a first comparison image from the current image; acquiring, based on the predicted first position, a second comparison image from the current image; and determining, based on the first comparison image and the second comparison image, the image feature difference.
Another aspect of the present disclosure provides a system for target tracking. The system may include: at least one storage device including a set of instructions; and at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to: determine, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object; determine, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple historical images may be acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera; determine, based on the first motion trajectory, a predicted position associated with the target object; and in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, control the dome camera to track the target object, wherein the actual position associated with the target object may be determined based on a current image acquired by the bullet camera.
Another aspect of the present disclosure provides a non-transitory computer-readable medium, comprising a set of instructions, wherein when executed by a computer, the set of instructions may direct the computer to perform a method. The method may include: determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object; determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple second historical images may be acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera; determining, based on the first motion trajectory, a predicted position associated with the target object; and in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object, wherein the actual position associated with the target object may be determined based on a current image acquired by the bullet camera.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further illustrated in terms of exemplary embodiments, and these exemplary embodiments are described in detail regarding the drawings. These embodiments are not restrictive. In these embodiments, the same number indicates the same structure, wherein:
FIG. 1 is a schematic diagram illustrating an exemplary application scenario of a system for target tracking according to some embodiments of the present disclosure;
FIG. 2 is a flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure;
FIG. 3 is a flowchart illustrating an exemplary process of determining a first difference according to some embodiments of the present disclosure;
FIG. 4 is a flowchart illustrating an exemplary process of determining a predicted first position according to some embodiments of the present disclosure;
FIG. 5 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure;
FIG. 6 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure;
FIG. 7 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure;
FIG. 8 is a schematic diagram illustrating an exemplary device for target tracking according to some embodiments of the present disclosure;
FIG. 9 is a schematic diagram illustrating an exemplary terminal device according to some embodiments of the present disclosure; and
FIG. 10 is a schematic diagram illustrating an exemplary computer-readable medium according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In order to illustrate the technical solutions related to the embodiments of the present disclosure, a brief introduction of the drawings referred to in the description of the embodiments is provided below. Obviously, the drawings described below are only some examples or embodiments of the present disclosure. Those having ordinary skills in the art, without further creative efforts, may apply the present disclosure to other similar scenarios according to these drawings. Unless stated otherwise or obvious from the context, the same reference numeral in the drawings refers to the same structure and operation.
It will be understood that the terms “system, ” “device, ” “unit, ” and/or “module” used herein are one method to distinguish different components, elements, parts, sections, or assemblies of different levels in ascending order. However, the terms may be displaced by other expressions if they may achieve the same purpose.
As shown in the present disclosure and claims, unless the context clearly indicates exceptions, the words “a, ” “an, ” “one, ” and/or “the” do not specifically refer to the singular, but may also include the plural. The terms "including" and "comprising" only suggest that the steps and elements that have been clearly identified are included, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.
The flowcharts used in the present disclosure may illustrate operations executed by the system according to embodiments in the present disclosure. It should be understood that a previous operation or a subsequent operation of the flowcharts may not be accurately implemented in order. Conversely, various operations may be performed in inverted order, or simultaneously. Moreover, other operations may be added to the flowcharts, and one or more operations may be removed from the flowcharts.
FIG. 1 is a schematic diagram illustrating an exemplary application scenario of a system for target tracking according to some embodiments of the present disclosure.
As shown in FIG. 1, the application scenario 100 of the system for target tracking may include a processor 110, a network 120, a storage device 130, a bullet camera 140, and a dome camera 150.
The processor 110 may process data and/or information acquired by the storage device 130, the bullet camera 140, and/or the dome camera 150. For example, the processor 110 may acquire multiple first historical images from the storage device 130 and/or the bullet camera 140, acquire multiple second historical images from the storage device 130 and/or the dome camera 150. As another example, the processor 110 may determine a plurality of historical monitoring positions of the dome camera 150 with respect to a target object based on the multiple first historical images acquired by the bullet camera 140. As another example, the processor 110 may determine a first motion trajectory of the target object based on the multiple second historical images acquired by the dome camera 150; determine a predicted position associated with the target object based on the first motion trajectory; and in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, control the dome camera 150 to track the target object.
In some embodiments, the processor 110 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 110 may be local or remote. For example, the processor 110 may access information and/or data from storage device 130, the bullet camera 140, and the dome camera 150 via the network 120. As another example, the processor 110 may be directly connected with the storage device 130, the bullet camera 140, and the dome camera 150 to access information and/or data.
In some embodiments, the processor 110 may be integrated into the bullet camera 140 or the dome camera 150. In some embodiments, the processor 110 may be directly or indirectly connected with the bullet camera 140 or the dome camera 150 to realize the methods and/or functions described in the present disclosure.
The network 120 may include any suitable network capable of facilitating the exchange of information and/or data. In some embodiments, at least one component (e.g., the processor 110, the storage device 130, the bullet camera 140, or the dome camera 150) of the application scenario 100 of the system for target tracking may exchange information and/or data with at least one other component of the application scenario 100 of the system for target tracking via the network 120.
The storage device 130 may store data, instructions, and/or any other information. In some embodiments, the storage device 130 may store the data and/or information (e.g., a first historical image, a second historical image, etc. ) acquired by the processor 110, the bullet camera 140, and the dome camera 150, and may also store results (e.g., the plurality of historical monitoring positions of the dome camera 150 with respect to a target object, the first motion trajectory, etc. ) generated by the processor 110. In some embodiments, the storage device 130 may store data and/or instructions used by the processor 110 to execute or to complete the exemplary method described in the present disclosure. In some embodiments, the storage device 130 may be a portion of the processor 110, the bullet camera 140, and/or the dome camera 150.
The bullet camera 140 may refer to a wide-angle camera arranged at a fixed position in the system for target tracking. The bullet camera may be a kind of camera used for monitoring, which may be provided with a C/CS interface of lens used to connect to a lens. A field of vision of the bullet camera may be relatively wider than the dome camera 150.
The dome camera 150 may refer to a camera that can identify and track the target object in the system for target tracking and implement a close-up enlargement function. The dome camera 150 may be a camera that can be zoomed, thus more details of the object may be observed by zooming. A shape of the dome camera may be globular or have other shapes. In some embodiments, the dome camera 150 may be equipped with an integrated camera, a PTZ structure, a decoder, or the like. The dome camera 150 may rotate up, down, left, or right under the control of a control end (i.e., the processor 110) to monitor the target object.
In some embodiments, the bullet camera 140 and the dome camera 150 may operate coordinately to track the target object. Based on a cooperation between the bullet camera 140 and the dome camera 150, a field of vision may be wider and the more details of the object may be observed, which may achieve effective target tracking. The coordinated monitoring method of the bullet camera and the dome camera may need to obtain a correlation among the bullet camera, the dome camera, and the scene environment. The bullet camera may recognize and track the target object in an image acquired by the bullet camera, determine a position of the target object in the image, and send a corresponding control instruction to the dome camera based on a correlation between the dome camera and the bullet camera, to control the motion of the dome camera to track the target object.
In some embodiments, the system for target tracking may include a monitoring position determination module, a motion trajectory determination module, a target position determination module, and a target tracking control module.
In some embodiments, the monitoring position determination module may be used to determine a plurality of historical monitoring positions of the dome camera with respect to a target object based on multiple first historical images acquired by the bullet camera.
In some embodiments, in order to determine the plurality of historical monitoring positions of the dome camera, the monitoring position determination module may be further used to determine a first historical position of the target object in a first historical image for each of the multiple first historical images; and convert the first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on a correlation between the bullet camera and the dome camera.
More descriptions of the first historical images, the target object, the historical monitoring positions of the dome camera, and the first historical position may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the motion trajectory determination module may be used to determine the first motion trajectory of the target object based on multiple second historical images, wherein the multiple second historical images may be acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera.
More descriptions of the second historical images and the first motion trajectory may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the target position determination module may be used to determine a predicted position associated with the target object based on the first motion trajectory.
In some embodiments, in order to determine the first motion trajectory, the motion trajectory determination module may be further used to determine a second historical position of the target object in the second historical image for each of the multiple second historical images; convert the second historical position to a third historical position of the target object in a preset coordinate system based on one of the plurality of historical monitoring positions of the dome camera corresponding to the second historical image; and generate the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images.
In some embodiments, the predicted position associated with the target object may include a predicted monitoring position of the dome camera and/or a predicted first position of the target object in the current image acquired by the bullet camera. The actual position associated with the target object may include an actual monitoring position of the dome camera and/or an actual first position of the target object in the current image acquired by the bullet camera.
In some embodiments, the predicted position associated with the target object may include a predicted monitoring position of the dome camera. In order to determine the predicted position associated with the target object, the target position determination module may be further used to determine a predicted second position of the target object in the preset coordinate system based on the first motion trajectory; and convert the predicted second position to the predicted monitoring position of the dome camera.
In some embodiments, the actual position associated with the target object may include the actual monitoring position of the dome camera, and the target position determination module may be further used to acquire the actual monitoring position of the dome camera. In order to acquire the actual monitoring position of the dome camera, the target position determination module may be further used to determine the actual first position of the target object in the current image acquired by the bullet camera; and convert the actual first position to the actual monitoring position of the dome camera based on a correlation between the bullet camera and the dome camera.
In some embodiments, the actual position associated with the target object may include the actual first position, and the predicted position associated with the target object may include the predicted first position. In order to determine the predicted position associated with the target object, the target position determination module may be further used to determine a predicted second position of the target object in the preset coordinate system based on the first motion trajectory; convert the predicted second position to the predicted monitoring position of the dome camera; and convert the predicted monitoring position of the dome camera to the predicted first position.
More descriptions of the predicted position associated with the target object, the actual position associated with the target object, the actual first position associated with the target object, the first historical positions, the correlation between the bullet camera and the dome camera, the second historical positions, the third historical position, the predicted monitoring position of the dome camera, the actual monitoring position of the dome camera, the predicted first position may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions. More descriptions of the predicted second position may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions.
In some embodiments, the target tracking control module may be used to control the dome camera to track the target object in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, where the actual position associated with the target object may be determined based on a current image acquired by the bullet camera.
In some embodiments, in order to control the dome camera to track the target object, the target tracking control module may be further used to determine a second difference, wherein the second difference may include a trajectory difference and/or an image feature difference; obtain a fused difference by fusing the second difference and the first difference; in response to determining that the fused difference satisfies the preset condition, control the dome camera to track the target object.
In some embodiments, the second difference may include the trajectory difference. In order to determine the second difference, the target tracking control module may be further used to determine a plurality of first historical positions of the target object based on the multiple first historical images; generate a second motion trajectory of the target object based on the plurality of first historical positions; and determine the trajectory difference based on the first motion trajectory and the second motion trajectory.
In some embodiments, the second difference may include the image feature difference. In order to determine the second difference, the target tracking control module may be further used to acquire a first comparison image from the current image based on the actual first position; acquire a second comparison image from the current image based on the predicted first position; and determine the image feature difference based on the first comparison image and the second comparison image.
More descriptions of the second difference, the trajectory difference, the image feature difference, the fused difference, the second motion trajectory, the first comparison image, and the second comparison image may be found elsewhere in the present disclosure, for example, FIG. 4 and the relevant descriptions.
It should be noted that the application scenario 100 of the system for target tracking is merely provided for illustration purposes and is not intended to limit the scope of the present disclosure. For those skilled in the art, many modifications or changes may be made according to the description of the present disclosure. For example, the application scenario 100 of the system for target tracking may also include cloud storage. As another example, the application scenario 100 of the system for target tracking may be implemented on other devices to achieve similar or different functions. However, those changes and modifications may not deviate from the scope of the present disclosure.
FIG. 2 is a flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure. As shown in FIG. 2, process 200 may include the following operations:
In 210, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object may be determined. In some embodiments, operation 210 may be performed by a processor (e.g., the monitoring position determination module) . More information of the bullet camera may be found elsewhere in the present disclosure, for example, FIG. 1 and the relevant descriptions.
A first historical image may refer to a monitoring image captured by the bullet camera at a historical moment. The multiple first historical images (e.g., k-1 first historical images) may refer to monitoring images captured by the bullet camera at multiple historical moments (e.g., k-1 historical moments) . A value of k may be a natural number greater than or equal to 1. The multiple first historical images may be captured sequentially. For example, the multiple first historical images may be images captured by the bullet camera in a continuous time period. For example, the multiple first historical images 1 to k-1 may be ordered in chronological order. In some embodiments, the processor may acquire the multiple first historical images in various ways. For example, the processor may acquire the multiple first historical images from the bullet camera or a storage device via the network.
In some embodiments, the processor may determine a count of frames of the multiple first historical images or multiple second historical images by identifying a monitoring environment. For example, the more complex the monitoring environment is, the larger the count of frames is. For example, the larger the traffic flow in the monitored area is, the larger the count of frames is. The processor may recognize objects in the multiple first or second historical images through image recognition technology. The objects may include pedestrians, vehicles, non-motor vehicles, trees, etc., in the monitoring image. The more objects in the image recognized by the processor, the more complex the monitoring environment is, and the larger the count of frames is.
The target object may refer to any object of interest in a monitoring image. For example, the target object may include at least one of the pedestrians, vehicles, non-motor vehicles, etc. in the monitoring image. A count of target objects in the monitoring image may be 1 or more. The count of target objects may be determined based on actual requirements.
In some embodiments, the processor may determine the target object in a variety of ways. For example, the processor may determine the target object based on a sensitive area. For example, the processor may determine the sensitive area in the screen of the bullet camera. When a moving object enters the sensitive area, the system for target tracking may trigger an alarm to determine the moving object as the target object. As another example, the target object may be manually determined.
In some embodiments, features of the target object may be represented by image features of the target object, or by attributes (such as color, size, etc. ) of the target object. More descriptions of the image features of the target object may be found elsewhere in the following descriptions of the present disclosure.
A historical monitoring position of the dome camera may refer to a monitoring position of the dome camera with respect to the target object monitored by the dome camera at a historical moment. The historical monitoring position of the dome camera may be represented by pan tilt zoom (PTZ) coordinates. The P in the PTZ coordinates may represent a horizontal motion of the dome camera (i.e., a rotation of the dome camera) , the T may represent a vertical motion (i.e., bowing or raising of the dome camera) , and the Z may represent a zoom (i.e., adjusting a focal length of the dome camera) .
There may be a correlation between the plurality of historical monitoring positions of the dome camera and the multiple first historical images. For a certain frame of the multiple first historical images, the processor may determine a corresponding historical monitoring position of the dome camera based on the frame of the first historical image. The processor may control the dome camera based on the plurality of historical monitoring positions of the dome camera, thus the target object to be captured by the dome camera may be located in a central area of the screen of the dome camera.
In some embodiments, the processor may determine the first historical position of the target object in a first historical image of the bullet camera; and convert the first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on a correlation between the bullet camera and the dome camera.
The first historical position of the target object may refer to a position of the target object in one of the plurality of first historical images of the bullet camera. The first historical position of the target object may be represented in many ways. For example, the first historical position of the target object may be represented by pixel coordinates of the target object in the first historical image of the bullet camera.
In some embodiments, the processor may determine the first historical position of the target object in the first historical image of the bullet camera based on image features of the target object. For example, the processor may determine the first historical position by processing the image features of the target object and the first historical image of the bullet camera through a target tracking algorithm. Exemplary target tracking algorithms may include a mean shift method, a particle filter method, a single target tracking algorithm (SiamRPN) , etc.
The image features of the target object may include an image area (also referred to as a “target box” ) where the target object is located in a monitoring image and/or features of the image area. For example, the target box may be processed by a neural network model (e.g., a pre-trained convolution neural network model) , and the features of the target box may be extracted as the image features of the target object.
In some embodiments, the image features of the target object may be determined by a first frame of first historical image in the multiple first historical images. The first frame of first historical image refers to the historical image that is captured at the earliest time among the multiple first historical images. For example, the processor may determine a first position of the target object in the first frame of first historical image, determine a target box of the target object based on the first position, and further determine the image features of the target object. The processor may determine the first position of the target object in the first frame of first historical image in various ways, such as based on the sensitive area, by manual determination, image recognition, or the like.
In some embodiments, for a second frame of first historical image to a (k-1) -th frame of first historical image, the processor may determine the first historical position of the target object based on the target tracking algorithm.
The correlation between the bullet camera and the dome camera may be a corresponding relationship between a position of the bullet camera and a position of the dome camera when monitoring a certain target object. The position of the bullet camera may be represented by a position of the target object in a monitoring image captured by the bullet camera. The position of the dome camera may be represented by a monitoring position (e.g., PTZ coordinates) of the dome camera when the dome camera monitors the target object. When the dome camera is at the monitoring position, the target object may be in the central area of the screen of the dome camera. In some embodiments, the position of the bullet camera and the position of the dome camera may be represented by coordinates, and the correlation between the bullet camera and the dome camera may include a coordinate mapping relationship between the coordinate corresponding to the bullet camera and the coordinate corresponding to the dome camera. For example, the correlation between the bullet camera and the dome camera may be represented by a transformation matrix. By processing the position of the bullet camera and the transformation matrix, the corresponding position of the dome camera may be obtained.
In some embodiments, the correlation between the bullet camera and the dome camera may include a relationship used to convert the position of the bullet camera to the position of the dome camera, and a relationship used to convert the position of the dome camera to the position of the bullet camera.
In some embodiments, the processor may determine the correlation between the bullet camera and the dome camera based on a calibration algorithm. For example, the processor may obtain a monitoring image acquired by the bullet camera and a monitoring image acquired by the dome camera respectively. The processor may select a fixed target (e.g., a building, a stationary vehicle, etc. ) from the two monitoring images as the comparison target and determine a coordinate mapping relationship based on the fixed target.
In some embodiments, there may be a one-to-one corresponding relationship between the plurality of first historical positions and the plurality of historical monitoring positions of the dome camera. In some embodiments, the processor may convert a first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on the correlation between the bullet camera and the dome camera. For example, the processor may directly convert the first historical position based on the correlation between the bullet camera and the dome camera. As another example, the bullet camera may send the first historical position of the target object to the dome camera, and the dome camera may convert the first historical position to a historical monitoring position of the dome camera based on the correlation between the bullet camera and the dome camera.
In 220, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object may be determined. In some embodiments, operation 220 may be performed by a processor (e.g., the motion path determination module) .
A second historical image may refer to a monitoring image captured by the dome camera at a historical moment. The multiple second historical images (e.g., k-1 frames of second historical images) may refer to monitoring images taken by the dome camera at multiple historical moments. In some embodiments, the processor may control the dome camera based on the plurality of historical monitoring positions of the dome camera to capture the multiple second historical images. For example, for the first frame of second historical image, a corresponding historical monitoring position of the dome camera may be determined based on the first frame of first historical image, and the dome camera may be controlled based on the historical monitoring position of the dome camera to capture the corresponding first frame of second historical image. For a second frame of second historical image to a (k-1) -th frame of second historical image, historical monitoring positions of the dome camera may be determined based on the second frame of first historical image to the (k-1) -th frame of first historical image respectively, and the dome camera may be controlled based on historical monitoring positions of the dome camera to capture the corresponding second frame of second historical image to the (k-1) -th frame of second historical image.
The first motion trajectory may refer to a motion trajectory of the target object in images acquired by the dome camera.
In some embodiments, for each of the multiple second historical images, the processor may determine a second historical position of the target object in the second historical image, and convert the second historical position to a third historical position of the target object in a preset coordinate system based on the historical monitoring position of the dome camera corresponding to the second historical image. The processor may generate the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images.
The second historical position may refer to a position of the target object in a second historical image. For example, the second historical position of the target object may be represented by pixel coordinates (two-dimensional coordinates) of the target object in the second historical image.
In some embodiments, the processor may determine the second historical position of the target object in the second historical image based on the image features of the target object. For example, the processor may determine the second historical position by processing the image features of the target object based on the target tracking algorithm. The determination of the second historical position may be performed in a similar manner as the determination of the first historical position.
The image features of the target object used to determine the second historical position may be determined based on the first frame of first historical image as described above, or by the first frame of second historical image. For example, the target box at the central area of first frame of second historical image may be used to determine the image features of a new target object, and determine second historical positions of the subsequent frames.
The third historical position may refer to a position of the target object in the preset coordinate system. The preset coordinate system may include a 2D coordinate system, a 3D coordinate system, or the like.
If the target object moves, a position of the target object in the screen of the bullet camera may change, and accordingly, a monitoring position (i.e., the PTZ coordinates) of the dome camera may also change, so that the target object may be located in the central area of the second historical images represented in different coordinate systems. In order to make the target object located in the central area of the second historical images, when the monitoring position of the dome camera is changed, the coordinate system indicating the position of the target object in the image acquired by the dome camera may also be changed. By normalizing different coordinate systems (i.e., determining the preset coordinate system) , the position of the target object in the same coordinate system at different historical moments may be determined.
In some embodiments, the preset coordinate system may be determined based on any frames of the multiple second historical images. For example, the processor may designate a coordinate system representing the position of the target object in a second historical image (referred to as “a coordinate system corresponding to the second historical image” ) as the preset coordinate system, at this time, the preset coordinate system may correspond to the monitoring position of the dome camera where the second historical image is acquired. In the coordinate system corresponding to the second historical image, the target object may be located in the center area of the second historical image. In some embodiments, the preset coordinate system may be determined based on the last frame of the multiple second historical images. For example, the processor may designate the coordinate system corresponding to the (k-1) -th frame of second historical image as the preset coordinate system. It should be understood that taking the coordinate system corresponding to the (k-1) -th frame of second historical image as the preset coordinate system means that the positions of target objects at all historical moments are represented in the (k-1) -th frame of second historical image. That is, the third historical position may be a position of the target object in the (k-1) -th frame of second historical image.
In some embodiments, the processor may convert the second historical position of a second historical image to a third historical position of the target object in a preset coordinate system based on the historical monitoring position of the dome camera corresponding to the second historical image. For example, based on a difference between a historical monitoring position of the dome camera corresponding to the second historical image and a historical monitoring position of the dome camera corresponding to the preset coordinate system, the second historical position may be converted to the third historical position of the target object represented under the preset coordinate system by coordinate transformation. For example, the preset coordinate system may correspond to the monitoring position of the dome camera where the (k-1) -th frame of second historical image is acquired. The processor may convert second historical positions of the target object in the first frame to (k-2) -th frame of second historical images to the third historical positions under the preset coordinate system. The first frame to (k-2) -th frame of second historical images may correspond to positions (i.e., (k-1) positions) under the preset coordinate system, and the (k-1) positions may be used to generate the first motion trajectory.
In some embodiments, the processor may generate the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images arranged in chronological order.
In 230, based on the first motion trajectory, a predicted position associated with the target object may be determined. In some embodiments, operation 230 may be performed by a processor (e.g., the target location determination module) .
The predicted position associated with the target object may refer to a position associated with the target object at the current time. In some embodiments, the predicted position associated with the target object may include a predicted monitoring position of the dome camera and/or a predicted first position of the target object in the current image acquired by the bullet camera.
The predicted monitoring position of the dome camera may refer to a predicted possible position (e.g., PTZ coordinate) of the dome camera to monitor the target object. More description of the predicted monitoring position of the dome camera may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions.
The predicted first position of the target object may be a predicted current possible position of the target object in an image acquired by the bullet camera (e.g., the multiple first historical images) . It should be understood that the bullet camera may not move, indicating that the coordinate system of the position of the target object in the screen of the bullet camera may be the same. More descriptions of the predicted first position may be found elsewhere in the present disclosure, for example, FIG. 4 and the relevant descriptions.
In some embodiments, the predicted position associated with the target object may include the predicted monitoring position of the dome camera. The processor may determine a predicted second position of the target object in the preset coordinate system based on the first motion trajectory, and convert the predicted second position to the predicted monitoring position of the dome camera. More description of the predicted monitoring position of the dome camera may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions.
In some embodiments, if the predicted position associated with the target object includes the predicted first position, the predicted monitoring position may be converted to the predicted first position based on the correlation between the bullet camera and the dome camera.
In 240, in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, the dome camera may be controlled to track the target object. In some embodiments, operation 240 may be performed by a processor (e.g., the target tracking control module) .
A current image may refer to a monitoring image acquired by the bullet camera at the current time. For example, the bullet camera may have taken 1～ (k-1) frames of monitoring images, and the current image may be the k-th frame of monitoring image acquired by the bullet camera at present. In some embodiments, the processor may acquire the current image based on the bullet camera.
The actual position associated with the target object may refer to a current actual position associated with the target object. In some embodiments, the actual position associated with the target object may include an actual monitoring position of the dome camera and/or an actual first position of the target object in the current image acquired by the bullet camera.
The actual first position of the target object may be an actual position of the target object in the current image. In some embodiments, the processor may automatically track the target object in the current image through the target tracking algorithm based on image features of the current image and the target object, and mark the target object using a target box. More descriptions of the actual first position may be found elsewhere in the present disclosure, for example, FIG. 4 and the relevant descriptions.
The actual monitoring position of the dome camera may refer to an actual position (e.g., PTZ coordinates) of the current dome camera to monitor the target object. In some embodiments, the processor may convert the actual first position to the actual monitoring position of the dome camera based on a correlation between the bullet camera and the dome camera. More description of the actual monitoring position of the dome camera may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions.
The first difference may refer to a difference between the predicted position associated with the target object and an actual position associated with the target object. The first difference may be a distance difference between the predicted position associated with the target object and an actual position associated with the target object, a position coordinate difference between the predicted position associated with the target object and the actual position associated with the target object, etc. The first difference may be determined in many ways. In some embodiments, the first difference between the predicted position associated with the target object and an actual position associated with the target object may be determined based on a difference between the predicted monitoring position of the dome camera and the actual monitoring position of the dome camera. In some embodiments, the first difference between the predicted position associated with the target object and an actual position associated with the target object may be determined based on a difference between the predicted first position of the target object and the actual first position of the target object.
The preset condition may refer to a condition that the first difference between the predicted position associated with the target object and the actual position associated with the target object needs to satisfy. In some embodiments, the preset condition may include that the first difference is less than a difference threshold. For example, the preset condition is that the first difference is less than the difference threshold.
The difference threshold may refer to a threshold condition that the first difference or a second difference needs to meet. For example, the difference threshold may be that a distance between two PTZ coordinates is less than 0.2°. More description of the first difference may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions. In some embodiments, the difference threshold may be determined based on calculation requirements or set by the system default. More descriptions of the second difference may be found elsewhere in the present disclosure, for example, FIG. 4 and the relevant description.
In some embodiments, the processor may adjust the difference threshold. In some embodiments, the processor may adjust the difference threshold based on a confidence of the predicted second position output from a position prediction model. In some embodiments, the processor may determine whether the confidence is lower than a preset threshold, and adjust the difference threshold in response to determining that the confidence is lower than the preset threshold. The preset threshold may refer to a threshold condition that needs to be met by the preset confidence. For example, the difference threshold is positively correlated with the confidence. The lower the confidence is, the smaller the difference threshold is. Merely by way of example, the processor may determine corresponding difference thresholds for different confidences in advance and store the corresponding difference thresholds. When the confidence is determined and the confidence is lower than the preset threshold, the difference threshold corresponding to the confidence may be directly obtained as the adjusted difference threshold.
More descriptions of the position prediction model, the predicted second position and the confidence may be found elsewhere in the present disclosure, for example, FIG. 3 and the relevant descriptions.
In some embodiments of the present disclosure, the difference threshold may be adjusted by determining the confidence of the output of the position prediction model, and so as to adjust a tracking strictness of the dome camera with respect to the target object, which may further improve the accuracy of target tracking.
In some embodiments, when the first difference between the predicted position associated with the target object and the actual position associated with the target object satisfies the preset condition, the processor may control the dome camera to automatically track the target object.
In some embodiments, when the first difference between the predicted position associated with the target object and the actual position associated with the target object does not satisfy the preset condition, the processor may repeat the operations 220 to 230 to obtain new first historical images and new second historical images for judgment, until the newly determined first difference satisfies the preset condition, and the dome camera may be controlled to track the target object.
In some embodiments of the present disclosure, the processor may determine the second difference, wherein the second difference may include a trajectory difference and/or an image feature difference. The processor may fuse the second difference and the first difference, and control the dome camera to track the target object in response to determining that the fused difference satisfies the preset condition. If the preset condition is not satisfied, new first historical images and new second historical images may be acquired, a new second difference may be determined, and whether to track the target object may further be determined. More description of the second difference may be found elsewhere in the present disclosure, for example, FIG. 4 and the relevant descriptions.
In some embodiments, the processor may determine whether to track the target object using the dome camera based on a tracking prediction model.
In some embodiments, an input of the tracking prediction model may include a target box of the target object, the multiple first historical images, the multiple second historical images, and the current image, and an output of the tracking prediction model may be a determination about whether to track the target object using the dome camera.
In some embodiments, the tracking prediction model may be a multi-layer structure. The tracking prediction model may include a target feature extraction layer, a motion trajectory extraction layer, a prediction layer, and a comparison layer. An input of the target feature extraction layer may include the target box of the target object, and an output of the target feature extraction layer may include the image features of the target object. An input of the motion trajectory prediction layer may include the image features of the target object output by the target feature extraction layer, the multiple first historical images and the multiple second historical images, and an output of the motion trajectory prediction layer may be a motion trajectory feature of the target object. An input of the prediction layer may be the motion trajectory feature output by the motion trajectory prediction layer, and an output of the prediction layer may be the prediction position associated with the target object. An input of the comparison layer may include the prediction position associated with the target object output by the prediction layer, the current image, and the image features of the target object, and an output of the comparison layer may be a determination about whether to track the target object using the dome camera.
The tracking prediction model may be a machine learning model. In some embodiments, the target feature extraction layer may be a convolutional neural network (CNN) model. The motion trajectory extraction layer may be a recurrent neural network (RNN) model, a long short-term memory (LSTM) model, or other sequence models. The prediction layer and comparison layer may be a deep neural network (DNN) model, a neural network (NN) model, or other models.
The target feature extraction layer, the motion trajectory extraction layer, the prediction layer, and comparison layer in the tracking prediction model may be obtained through joint training of multiple groups of first training samples with first labels. In some embodiments, a first training sample of the joint training may include multiple sample first historical images, multiple sample second historical images, a sample current image, and a sample box of a sample object. A first label of the first training sample may be an actual result of whether the sample object is tracked by the dome camera. The first training samples and corresponding first labels may be obtained based on historical data.
The sample box may be input into the target feature extraction layer to obtain the image features of the sample object output by the target feature extraction layer. The image features of the sample object may be used as training sample data, and may be input into the motion track extraction layer with the multiple sample first historical images and the multiple sample second historical images to obtain the trajectory feature of the sample object output by the motion track extraction layer. The trajectory feature of the sample object may be input into the prediction layer as training sample data to obtain the prediction position associated with the sample object output by the prediction layer. The predicted position associated with the sample object may be taken as the training sample data, and may be input into the comparison layer with the sample current image and the image features of the sample object to obtain the result of whether to track the sample object using the dome camera. A loss function may be constructed based on the actual results of whether the sample object is tracked by the dome camera and results of whether the sample object is tracked by the dome camera output by the comparison layer. Parameters of the target feature extraction layer, the motion trajectory extraction layer, the prediction layer, and the comparison layer may be synchronously updated based on the loss function. Through parameter updating, a trained target feature extraction layer, a trained motion trajectory extraction layer, a trained prediction layer, and a trained comparison layer may be obtained.
In some embodiments of the present disclosure, determining whether to track the target object using the dome camera based on the tracking prediction model with the multi-layer structure may further improve the efficiency of the determination and the accuracy of the determination result, and further realize the accurate and autonomous tracking for the target object using the dome camera, and the accuracy of target tracking may be improved.
In some embodiments of the present disclosure, by determining the predicted position associated with the target object, a monitoring position of the dome camera with respect to the target object may still be temporarily determined based on the predicted position associated with the target object, and the dome camera can monitor the target object even if the target object is blocked. Moreover, through the coordination between the bullet camera and the dome camera, even if there is a difference between the target features in the different images acquired by the bullet camera and the dome camera, or the target object is blocked, whether the dome camera should track the target object can be determined based on the comparison of the predicted position associated with the target object and the actual position associated with the target object, accurate and autonomous tracking for the target object by the dome camera may be implemented, and the accuracy of the target tracking may be improved.
FIG. 3 is a flowchart illustrating an exemplary process of determining a first difference according to some embodiments of the present disclosure. In some embodiments, process 300 may be performed by a processor. As shown in FIG. 3, process 300 may include the following operations:
In some embodiments, the predicted position associated with the target object may include a predicted monitoring position of the dome camera, and the processor may obtain the predicted monitoring position of the dome camera based on operations 310 and 320.
In 310, a predicted second position of the target object in the preset coordinate system may be determined based on the first motion trajectory. In some embodiments, operation 310 may be performed by the processor.
For example, assuming that the first motion trajectory includes 1～ (k-1) third historical positions of 1～ (k-1) frames of monitoring images of the dome camera in the (k-1) -th frame of monitoring image of the dome camera, the predicted monitoring position of the dome camera may be PTZ coordinates that corresponds to the predicted k-th monitoring image of the dome camera. It is equivalent to assuming that when the dome camera monitors the target object by determining the predicted monitoring position of the dome camera, the target object is in the center of the screen of the dome camera. More descriptions of the predicted monitoring position of the dome camera may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
The predicted second position may refer to a predicted current position of the target object in the preset coordinate system. In some embodiments, taking a coordinate system corresponding to a second historical image as the preset coordinate system, the predicted second position may be the predicted current position information of the target object on a corresponding second historical image acquired by the dome camera. For example, the coordinate system corresponding to a (k-1) -th frame of second historical image is designated as the preset coordinate system, and the predicted second position is the predicted position information of the target object on a (k-1) -th frame of second historical image acquired by the dome camera.
In some embodiments, the processor may determine the predicted second position of the target object in the preset coordinate system based on the first motion trajectory by a dynamic trajectory prediction algorithm (e.g., a Kalman filtering method, or a velocity averaging method) . In some embodiments, the processor may also use other methods to determine the predicted second position of the target object in the preset coordinate system based on the first motion trajectory, which may not be limited in the embodiments of the present disclosure.
In some embodiments, the processor may determine event information based on the multiple first historical images; judge whether the event information satisfies a preset event requirement; in response to determining that the event information satisfies the preset event requirement, the event information and the first motion trajectory may be processed based on a position prediction model to determine the predicted second position, wherein the position prediction model is a machine learning model.
The event information may indicate whether there are abnormal events in a process of target tracking by the bullet camera and the dome camera, and an abnormal degree of an abnormal event. For example, the event information may indicate whether the target object is blocked when being captured by the bullet camera, and a block degree of the target object (such as 20%, 30%, 50%, etc. ) . As another example, the event information may include whether there is a traffic accident when monitoring the target object, a severity of the traffic accident, whether the road is congested, a degree of road congestion, whether the road is complex, a degree of the road complexity, or the like, or any combination thereof.
In some embodiments, the processor may acquire the multiple first historical images, process the multiple first historical images based on the image recognition technology, and determine the event information.
The preset event requirement may refer to a preset restriction requirement or a judgment condition for event information. For example, the preset event requirement may include that the target object is blocked, and the block degree of the target object is greater than 30%. As another example, the preset event requirement may include that the road is congested, the degree of road congestion degree is greater than 40%, etc. In some embodiments, the preset event requirement may be set manually or according a default setting based on the actual requirements of target tracking.
In some embodiments, the processor may compare the event information with the preset event requirement to determine whether the event information satisfies the preset event requirement. For example, if the preset event requirement is that the block degree of the target object is greater than 30%, and the block degree of the target object in the event information is 50%, the event information may be determined as satisfying the preset event requirement.
In some embodiments, in response to determining that the event information satisfies the preset event requirement, the processor may process the event information and the first motion trajectory based on the position prediction model to determine the predicted second position.
In some embodiments, the position prediction model may be a machine learning model, for example, a DNN model or other customized networks, or any combination thereof.
In some embodiments, an output of the position prediction model may include a confidence of the predicted second position. The confidence may refer to a reliability degree of the predicted second position determined by the position prediction model. The confidence may be expressed by a numerical value (e.g., percentages) , such as 80%, 85%, 90%, etc. The greater the value is, the greater the confidence of the predicted second position.
In some embodiments, the position prediction model may be obtained based on model training. In some embodiments, a second training sample of the position prediction model may include sample event information and a sample first motion trajectory. A second label corresponding to the second training sample may include an actual second position. The second training sample and the second label can be obtained from historical data.
In some embodiments, the processor may input the sample event information and the sample first motion trajectory into an initial position prediction model to obtain an initial predicted second position. A loss function may be constructed based on the initial predicted second position and the second label. The loss function may be used to update parameters of the initial position prediction model. By updating the parameters, the trained position prediction model may be obtained.
In some embodiments, the processor may use the trained position prediction model as the prediction layer of the tracking prediction model to reduce the training data requirement of the tracking prediction model.
In some embodiments of the present disclosure, by determining the predicted second position using the position prediction model if the event information satisfies the preset event requirement, a problem of inaccurate prediction when abnormal events (e.g., the target object being blocked, etc. ) occur in the process of target tracking may be avoided, and the accuracy of determining the predicted second position may be improved.
In 320, the predicted second position may be converted to the predicted monitoring position of the dome camera. In some embodiments, operation 320 may be performed by the processor.
In some embodiments, the processor may convert the predicted second position to the predicted monitoring position of the dome camera based on a coordinate conversion relationship of the dome camera. The coordinate conversion relationship of the dome camera may refer to a conversion relationship (such as a conversion matrix) for converting the predicted second position to the predicted monitoring position of the dome camera in the preset coordinate system. The coordinate conversion relationship of the dome camera may be determined in a variety of ways. For example, the coordinate conversion relationship of the dome camera may be determined based on at least two pairs of second positions and corresponding monitoring positions of the dome camera.
In 330, the actual monitoring position of the dome camera may be acquired. In some embodiments, operation 330 may be performed by the processor.
In some embodiments, the actual position associated with the target object may include the actual monitoring position of the dome camera, and the processor may obtain the actual monitoring position of the dome camera. In some embodiments, the processor may determine an actual first position of the target object in the current image based on the current image. Based on the correlation between the dome camera and the bullet camera, the actual first position may be converted to the actual monitoring position of the dome camera. More descriptions of the actual monitoring position of the dome camera, the current image, and the actual first position may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the processor may designate the actual position associated with the target object in the current first image as the actual first position based on the target recognition method and a current monitoring image of the bullet camera.
In some embodiments, the processor may convert the actual first position to the actual monitoring position of the dome camera based on the correlation between the bullet camera and the dome camera. The conversion method may be similar to the method for converting the predicted first position to the predicted monitoring position of the dome camera. More information of the correlation between the bullet camera and the dome camera may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In 340, a first difference may be determined based on a difference between the predicted monitoring position of the dome camera and an actual monitoring position of the dome camera.
For example, the processor may designate a coordinate difference between the PTZ coordinates corresponding to the predicted monitoring position of the dome camera and the PTZ coordinates corresponding to the actual monitoring position of the dome camera as the first difference.
In some embodiments, the processor may judge whether the predicted monitoring position of the dome camera and the actual monitoring position of the dome camera satisfy the preset condition. If the preset condition is satisfied, the dome camera may be controlled to track the target object. More descriptions may be found elsewhere in the present disclosure, for example, FIG. 5 and the relevant descriptions.
FIG. 4 is a flowchart illustrating an exemplary process of determining a predicted first position according to some embodiments of the present disclosure. In some embodiments, process 400 may be performed by a processor. As shown in FIG. 4, process 400 may include the following operations:
In some embodiments, the actual position associated with the target object may include an actual first position of the target object in the current image acquired by the bullet camera, and the predicted position associated with the target object may include a predicted first position of the target object in the current image acquired by the bullet camera. More descriptions of the predicted first position and the actual first position may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In 410, a predicted second position of the target object in the preset coordinate system may be determined based on the first motion trajectory. In some embodiments, operation 410 may be performed by the processor.
Operation 410 is similar to operation 310, more descriptions of which may be found elsewhere in the present disclosure, for example, operation 310 and the relevant descriptions.
In 420, the predicted second position may be converted to the predicted monitoring position of the dome camera. In some embodiments, operation 420 may be performed by the processor.
Operation 420 is similar to operation 320, more descriptions of which may be found elsewhere in the present disclosure, for example, operation 320 and the relevant descriptions.
In 430, the predicted monitoring position of the dome camera may be converted to the predicted first position. In some embodiments, operation 430 may be performed by the processor.
In some embodiments, the processor may convert the predicted monitoring position of the dome camera to the predicted first position based on the correlation between the bullet camera and the dome camera. More descriptions of the correlation between the bullet camera and the dome camera may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, when the first difference between the actual first position of the target object and the predicted first position of the target object satisfies the preset condition, the processor may control the dome camera to automatically track the target object. In some embodiments, when the first difference between the actual first position of the target object and the predicted first position of the target object does not satisfy the preset condition, the processor may repeat operations 310-330 and operations 410-420 to acquire a new actual first position and a new predicted first position until the new first difference satisfies the preset condition, and the dome camera may be controlled to track the target object. More descriptions of the first difference and preset condition may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the processor may determine the second difference, wherein the second difference may include a trajectory difference and/or an image feature difference. A fused difference may be obtained by fusing the second difference and the first difference, and in response to determining that the fused difference satisfies the preset condition, the dome camera may be controlled to track the target object.
The second difference may refer to a difference of data related to the target object obtained during the process of target tracking. For example, the second difference may include the trajectory difference and/or the image feature difference. For example, the second difference may be expressed by a difference score. The greater the difference is, the lower the difference score is.
The trajectory difference may refer to a difference between the first motion trajectory and a second motion trajectory.
The second motion trajectory may be a motion trajectory generated based on multiple first historical positions (e.g., (k-1) first historical positions) . For example, the second motion trajectory is the motion trajectory of the target object in the screen of the bullet at (k-1) historical times. If the coordinate system corresponding to the (k-1) -th frame of second historical image is used as the preset coordinate system, the first motion trajectory is the motion trajectory of the target object at (k-1) historical times in the (k-1) -th frame of second historical image. More descriptions of the first motion trajectory may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the processor may determine a plurality of first historical positions based on the multiple first historical images. The second motion trajectory may be generated based on the plurality of first historical positions. For example, the plurality of first historical positions may be connected to generate the second motion trajectory.
More description of determining the plurality of first historical positions may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the processor may compare the first motion trajectory with the second motion trajectory to determine the trajectory difference. For example, the processor may determine the distance difference between each position in the first motion trajectory and a corresponding position in the second motion trajectory, determine an average distance difference, and designate the average distance difference as the trajectory difference.
In some embodiments of the present disclosure, the second difference may be determined by comparing the first and second motion trajectories, which may make the methods of determining the second difference diverse and improve the accuracy of determining the second difference.
The image feature difference may refer to a difference between a first comparison image and a second comparison image.
In some embodiments, the processor may obtain the first comparison image from the current image based on the actual first position. The second comparison image may be obtained from the current image based on the predicted first position.
The first comparison image may refer to a comparison image obtained based on the actual first position. In some embodiments, the processor may extract the first comparison image at a corresponding position on the current image based on the actual first position. For example, the processor may determine a box in the current image based on the actual first position, wherein the center of the box is located at the actual first position, and each edge of the box is located at a preset distance from the actual first position. The region in the box may be determined as the first comparison image.
The second comparison image may refer to a comparison image obtained based on the predicted first position. In some embodiments, the processor may extract the second comparison image at a corresponding position on the current image based on the predicted first position. For example, the processor may determine a box in the current image based on the predicted first position.
In some embodiments, the processor may compare the first comparison image and the second comparison image to determine the image feature difference. For example, the processor may determine an image feature of the first comparison image and an image feature of the second comparison image by a hash algorithm, a computed histogram, an image feature extraction model (e.g., a CNN model) , or other methods. The processor may calculate a difference between the image feature of the first comparison image and the image feature of the second comparison image as the image feature difference.
In some embodiments of the present disclosure, various methods may be used to determine the second difference, and the accuracy of determining the second difference may be improved by determining the image feature difference.
The fused difference may refer to a difference determined by fusing the first difference and the second difference. In some embodiments, the processor may fuse the first difference and the second difference in a variety of ways, such as averaging, which may not be limited in the present disclosure.
In some embodiments, the fusion of the first difference and the second difference may include a weighting operation and a summing operation, where a second weight of the second difference and a first weight of the first difference may be determined based on the complexity of the monitoring scene.
The complexity of the monitoring scene may include the severity of the monitoring environment, the complexity of the target in the monitoring scene, and the block degree of the target object. The severity of the monitoring environment may include whether the monitoring environment is open, whether there are traffic accidents, whether the road is congested, whether the road is complex, whether the weather is bad, the visibility of the monitoring environment, etc. The complexity of the target in the monitoring scene may be associated with the count of moving targets in the monitoring scene. The greater the count of moving targets is, the more complex of the target is.
In some embodiments, the processor may determine the second weight of the second difference and the first weight of the first difference based on the complexity of the monitoring scene. The higher the complexity of the monitoring scene is, the greater the first weight of the first difference is, and the smaller the second weight of the second difference is. For example, the higher the block degree of the target object, the greater the first weight of the first difference is, and the smaller the second weight of the second difference is. In some embodiments, the block degree of the target object may be determined by the image recognition technology based on an image feature of the target object in a first frame of image acquired by the bullet camera and an image feature of the target object in a k-th frame of image acquired by the bullet camera. For example, the image recognition technology may determine a coincidence degree of the image feature of the target object in the first frame of image acquired by the bullet camera and the image feature of the target object in the k-th frame of image acquired by the bullet camera. The lower the coincidence degree is, the higher the block degree of the target object.
In some embodiments of the present disclosure, the accuracy of determining the fused difference may be improved by determining weights of the first and second differences of fusion based on the complexity of the monitoring scenes, the impact of the different complexity of the monitoring scenes on the accuracy of the acquired monitoring data (e.g., the monitoring images) may be taken into account, so that the dome camera may be controlled accurately to track the target object based on the fused difference.
In some embodiments, the processor may compare the fused difference with the preset condition to determine whether the fused difference satisfies the preset condition. When the fused difference satisfies the preset condition, the processor may control the dome camera to track the target. In some embodiments, when the fused difference does not satisfy the preset condition, the processor may repeatedly obtain a new first difference and a new second difference, until a new fused difference satisfies the preset condition, the dome camera may be controlled to track the target.
In some embodiments of the present disclosure, the fused difference may be obtained by determining the second difference, and then the second difference and the first difference may be fused. If the fused difference satisfies the preset condition, the dome camera may be controlled to track the target object, which may improve the target tracking accuracy.
FIG. 5 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure. In some embodiments, process 500 may be performed by a processor. As shown in FIG. 5, process 500 may include the following operations:
In 510, multiple historical monitoring images of the bullet camera may be acquired, the multiple historical monitoring images of the bullet camera may be identified, and a monitoring position of the bullet camera with respect to the target object may be obtained. In some embodiments, a historical monitoring image of the bullet camera may also be referred to as a first historical image, and a monitoring position of the bullet camera may also be referred to as a first historical position, more descriptions of which may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiment, the processor may obtain the multiple historical monitoring images of the bullet camera, and extract a target feature and target position information of the target object based on the multiple historical monitoring images of the bullet camera.
Optionally, the target object may include but be not be limited to one or more of vehicles, pedestrians, or non-motor vehicles.
Optionally, the target feature may be a convolution feature obtained by inputting a corresponding image region of the target object into a pre-trained convolution neural network. In some embodiments, the target feature may also be referred to as an image feature of the target object, more descriptions of which may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In one embodiment, the bullet camera may acquire the second frame to the k-th frame of the multiple historical monitoring images, and extract the target feature and the target position information from these historical monitoring images.
Optionally, the target position information may be pixel position information of the target object in the historical monitoring images of the bullet camera.
Before performing the operation 510, the method for target tracking shown in some embodiments of the present disclosure may also include the following operations: calibrating the bullet camera and the dome camera and obtaining a correlation between the bullet camera and the dome camera.
In one embodiment, a device for target tracking may calibrate the bullet camera and the dome camera respectively, and obtain internal parameters, external parameters, and distortion coefficients of the bullet camera and the dome camera respectively, to establish a correlation between an actual position of the target object in the real environment, and the pixel position of the target object in the monitoring image obtained by the bullet camera and the dome camera.
Optionally, the bullet camera and the dome camera may be calibrated based on a checkerboard.
In some embodiment, the device for target tracking may obtain monitoring images of the bullet camera and the dome camera respectively, select one or more fixed target objects in the monitoring image of the bullet camera and the monitoring image of the dome camera as comparison targets, construct a coordinate conversion algorithm, and obtain the correlation between the bullet camera and the dome camera.
Optionally, the comparison targets may be buildings in the monitoring scene.
Optionally, the comparison targets may be stationary vehicles in the monitoring scene.
In 520, the monitoring position of the bullet camera with respect to the target object may be mapped to multiple historical monitoring images of the dome camera to obtain a monitoring position of the dome camera with respect to the target object. In some embodiments, a historical monitoring image of the dome camera may also be referred to as a second historical image, and the monitoring position of the dome camera may also be referred to as a second historical position, more descriptions of which may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, when the target object is identified from both the historical monitoring images of the bullet camera and the historical monitoring images of the dome camera, the operation of mapping the monitoring position of the bullet camera with respect to the target object to the multiple historical monitoring images of the dome camera may be performed.
In some embodiments, when the target object fails to be obtained from at least one of the historical monitoring images of the bullet camera and the historical monitoring images of the dome camera, the operation of mapping the monitoring position of the bullet camera with respect to the target object to the multiple historical monitoring images of the dome camera may not be performed, and the operation 510 may be performed again.
The operation of mapping the monitoring position of the bullet camera with respect to the target object to the multiple historical monitoring images of the dome camera may include:
Based on the correlation between the bullet camera and the dome camera, the monitoring position of the bullet camera with respect to the target object may be mapped to the multiple historical monitoring images of the dome camera.
The correlation between the bullet camera and the dome camera may be determined from calibration parameters of the bullet camera and the dome camera. Specifically, a coordinate position of a calibration object in a calibration image of the bullet camera and a coordinate position of the calibration object in a calibration image of the dome camera may be determined based on the same calibration object in the real environment, and then the correlation between the bullet camera and the dome camera may be determined based on the two coordinate positions.
Specifically, the device for target tracking may obtain the target position information of the target object in the second frame to the k-th frame of historical monitoring images collected by the bullet camera, and convert the (k-1) historical monitoring positions of the bullet camera with respect to the target object to the (k-1) monitoring positions of the dome camera with respect to the target object based on the correlation between the bullet camera and the dome camera.
Through the above operations, the target position information obtained by the bullet camera may be mapped to the monitoring position of the dome camera with respect to the target object.
In 530, based on the monitoring position of the dome camera with respect to the target object, a motion trajectory of the target object with respect to the dome camera may be obtained, and a first current monitoring position of the dome camera with respect to the target object may be determined based on the motion trajectory of the target object with respect to the dome camera. In some embodiments, the motion trajectory of the target object with respect to the dome camera may also be referred to as the first motion trajectory, and the first current monitoring position of the dome camera may also be referred to as the predicted monitoring position of the dome camera, more descriptions of which may be found elsewhere in the present disclosure, for example, FIG. 2 and FIG. 3, and the relevant descriptions.
The motion trajectory of the target object with respect to the dome camera may include the monitoring positions of the dome camera with respect to the target object in the multiple historical monitoring images of the dome camera. The way to obtain the motion trajectory of the target object with respect to the dome camera may include mapping the monitoring position of the bullet camera on a historical monitoring image of the bullet camera to a historical monitoring image of the dome camera one by one, and then generating the motion trajectory of the target object with respect to the dome camera based on the monitoring positions of the dome camera on the multiple monitoring images of the dome camera in chronological order. The way to obtain the motion trajectory of the target object with respect to the dome camera may also include generating the motion trajectory of the bullet camera with respect to the target object based on the monitoring positions on the multiple historical monitoring images of the bullet camera in the chronological order and then map the motion trajectory of the bullet camera with respect to the target object to the historical monitoring image of the dome camera to generate the motion trajectory of the target object with respect to the dome camera.
The method of determining the first current monitoring position of the dome camera with respect to the target object based on the motion trajectory of the target object with respect to the dome camera may include the following operations:
In response to a preset target tracking algorithm, positions of the target object on the multiple historical monitoring images of the dome camera may be tracked based on the motion trajectory of the target object with respect to the dome camera, and the first current monitoring position of the dome camera with respect to the target object may be determined on a current monitoring image of the dome camera.
In some embodiments, the preset target tracking algorithm may be a single target tracking algorithm. Through the preset target tracking algorithm, the position information of the target object in multiple historical monitoring images of the dome camera may be obtained, and the motion trajectory of the target object with respect to the dome camera may be obtained.
Optionally, the single target tracking algorithm may include a target tracking algorithm based on a model matching, such as a mean shift method, or a particle filter method.
Optionally, the single target tracking algorithm may include a target tracking algorithm based on deep learning, such as a target tracking algorithm based on CNN, a target tracking algorithm based on a twin neural network and Hungarian algorithm, or a target tracking algorithm combining twin neural network and regional recommendation network.
In one embodiment, the device for target tracking may predict the position information of the target object (i.e., the first current monitoring position of the dome camera) in (k+1) frames of historical monitoring images of the dome camera based on the motion trajectory of the dome camera.
Optionally, the method of determining the position information of the target object in the (k+1) frames of historical monitoring images of the dome camera may be a trajectory prediction method such as the Kalman filtering method, or a velocity averaging method.
Optionally, the position information of the target object may include a pixel value in the center of the target box of the target object.
In 540, a current monitoring image of the dome camera may be obtained, and a second current monitoring position of the dome camera with respect to the target object may be obtained based on the current monitoring image of the dome camera. In some embodiments, the second current monitoring position of the dome camera may also be referred to as the actual monitoring position of the dome camera, and the current monitoring image can also be referred to as the current image, more descriptions may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant descriptions.
In some embodiments, the device for target tracking may convert a (k+1) -th frame of current monitoring image of the bullet camera to the second current monitoring position of the dome camera based on the correlation between the bullet camera and the dome camera.
In some embodiments, the specific implementation details of operation 540 may be found in FIG. 6 and the relevant descriptions.
In 550, the dome camera may be controlled to track the target object based on the first current monitoring position of the dome camera and the second current monitoring position of the dome camera.
Specifically, the device for target tracking may control whether the dome camera adjusts a PTZ of the dome camera to track the target object by comparing the information of the first current monitoring position of the dome camera and the second current monitoring position of the dome camera, and obtain the current monitoring image of the dome camera with respect to the target object in real-time.
In response to a position difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is less than a preset threshold, the dome camera may be controlled to track the target object.
In some embodiments, if the position difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is less than the preset threshold, the handover between the bullet camera and the dome camera may be realized, and the image feature and motion trajectory information of the target object may be updated, and the dome camera may be controlled to track the target object.
Optionally, the position difference may be a Manhattan distance between the center of the target box in the first current monitoring position of the dome camera and the center of the target box in the second current monitoring position of the dome camera.
Optionally, the position difference may be a Euclidean distance between the center of the target box in the first current monitoring position of the dome camera and the center of the target box in the second current monitoring position of the dome camera.
In response to determining that the position difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is less than the preset threshold, the motion trajectory of the dome camera may be updated based on the second current monitoring position of the dome camera, and the dome camera may be controlled to track the target object based on the updated motion trajectory of the dome camera.
In some embodiments, if the position difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is less than the preset threshold, the device for target tracking may re-control the dome camera to track the target object based on the second current monitoring position of the dome camera as the updated motion trajectory of the dome camera.
FIG. 6 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure. In some embodiments, process 600 may be performed by a processor. As shown in FIG. 6, process 600 may include the following operations:
In 610, a current monitoring position of the dome camera with respect to the target object in the current monitoring image of the dome camera may be obtained. In some embodiments, the current monitoring position of the dome camera with respect to the target object may also be referred to as the actual first position of the target object. More detailed descriptions may be found elsewhere in the present disclosure, for example, FIG. 2 and the relevant description.
Specifically, the bullet camera may follow the target object and obtain position information of the target object in the (k+1) -th frame of monitoring image of the bullet camera.
In 620, the current monitoring position of the dome camera with respect to the target object may be mapped on a current monitoring image of the dome camera, and a second current monitoring position of the dome camera with respect to the target object may be obtained.
Specifically, the device for target tracking may convert a value of a pixel position of the target object in the current monitoring image of the bullet camera into the position information of the target object in the (k+1) -th frame of monitoring image of the bullet camera (i.e., the second current monitoring position of the dome camera) based on a correlation between the bullet camera and the dome camera.
Optionally, the position information of the target object may include a pixel value in the center of a target box of the target object.
In some embodiments, the operations in process 600 may be similar to the operations of determining the actual first position and determining the actual monitoring image of the dome camera based on the actual first position shown in FIG. 3, which may refer to FIG. 3 and the relevant descriptions.
FIG. 7 is another flowchart illustrating an exemplary method for target tracking according to some embodiments of the present disclosure. In some embodiments, process 700 may be performed by a processor. As shown in FIG. 7, process 700 may include the following operations:
In 710, a first frame of monitoring image of the bullet camera may be obtained, and a target feature and target position information of the target object may be obtained.
In some embodiments, the first frame of monitoring image of the bullet camera may be obtained, and the target feature and the target position information of the target object may be extracted based on the first frame of monitoring image of the bullet camera.
Optionally, the target object may include, but may not be limited to, one or more of vehicles, pedestrians, and non-motor vehicles.
Optionally, the target feature may be a convolution feature obtained by inputting a corresponding image area of the target object into a pre-trained convolution neural network.
Optionally, the position information of the target object may be position information of the target object in an image acquired by the bullet camera.
Before performing operation 710, the following operations may also be performed: calibrating the bullet camera and the dome camera, and obtaining the correlation between the bullet camera and dome camera.
In some embodiment, the device for target tracking may calibrate the bullet camera and dome camera respectively, and obtain internal parameters, external parameters, and distortion coefficients of the bullet camera and dome camera respectively, thus establishing correlations between an actual position of the target in the real environment and a pixel position of the target object in the image acquired by the bullet camera or the dome camera.
Optionally, the bullet camera and the dome camera may be calibrated based on a checkerboard.
In some embodiment, the bullet camera and the dome camera may obtain corresponding monitoring images respectively, select one or more fixed target objects in the monitoring image of the bullet camera and the monitoring image of the dome camera as comparison targets, construct a coordinate conversion algorithm, and obtain the correlation between the bullet camera and the dome camera.
In 720, a second frame to a (k-1) -th frame of monitoring images of the bullet camera in multiple consecutive moments may be obtained, the target position information of the target object may be obtained based on a target tracking algorithm, and a corresponding monitoring position of the dome camera with respect to the target object may be obtained based on target position information in different frames of monitoring images of the bullet camera through a coordinate conversion with a correlation between the bullet camera and the dome camera.
In some embodiments, the device for target tracking may obtain the target position information of the target object in continuous monitoring images from the second frame to the (k-1) -th frame acquired by the bullet camera, and convert monitoring positions of the bullet camera with respect to the target object in the (k-2) frames of the continuous monitoring images acquired by the bullet camera to monitoring positions of the dome camera with respect to the target object in (k-2) frames of the continuous monitoring images acquired by the dome camera based on the correlation between the bullet camera and the dome camera.
After performing the above operations, monitoring position information of the bullet camera with respect to the target object may be mapped to monitoring position information of the dome camera with respect to the target object.
In 730, based on a coordinate conversion relationship of the dome camera, coordinates of the target object in a screen of the dome camera may be converted to coordinates under the same coordinate system based on monitoring position information of the dome camera, and motion trajectory information of the target object with respect to the dome camera may be updated.
Because PTZ coordinates information of the target object may be different under different monitoring positions of the dome camera with respect to the target object, the PTZ coordinates information of the target object may be converted into coordinate information of the dome camera based on the coordinate conversion relationship of the dome camera.
Specifically, under different monitoring positions of the dome camera, monitoring directions of the dome camera may change, resulting in non-uniform coordinate systems of multiple monitoring images of the dome camera. At this time, the device for target tracking may unify the coordinates of the target object in the multiple monitoring image of the dome camera on the same coordinate system based on the coordinate conversion relationship of the dome camera, which may be convenient for generating the motion trajectory of the target object with respect to the dome camera.
In some embodiments, the motion trajectory information of the target object may be obtained based on a target tracking algorithm.
Optionally, a single target tracking algorithm may include a target tracking algorithm based on model matching, such as a mean shift method, or a particle filter method.
Optionally, the single target tracking algorithm may include a target tracking algorithm based on deep learning, such as a target tracking algorithm based on CNN, a target tracking algorithm based on a twin neural network and Hungarian algorithm, or a target tracking algorithm combining twin neural network and regional recommendation network.
In 740, based on the motion trajectory information, target position information corresponding to a k-th frame of monitoring image of the bullet camera may be determined, and a first current monitoring position of the dome camera may be obtained based on the coordinate conversion relationship of the dome camera.
In some embodiment, the device for target tracking may determine the target position information corresponding to a k-th frame of monitoring image of the bullet camera (i.e., the first current monitoring position of the dome camera) based on the motion trajectory of the target object obtained from the second frame to the (k-1) -th frame of monitoring images of the dome camera.
Optionally, the method for determining the target position information corresponding to a k-th frame of monitoring image of the bullet camera may be a trajectory prediction method such as a Kalman filter method, or a velocity averaging method.
Optionally, the target position information of the target object may include a value of a pixel in the center of the target box of the target object.
In 750, a k-th frame of monitoring image of the bullet camera may be obtained, the target position information of the target object may be obtained, and a second current monitoring position of the dome camera may be obtained based on a correlation between the bullet camera and the dome camera.
In some embodiments, the bullet camera may follow the target object to obtain the target position information of the target object corresponding to the k-th frame of monitoring image of the bullet camera.
The device for target tracking may convert a value of a pixel position of the target object in the current monitoring image (i.e., the k-th frame of monitoring image) of the bullet camera into the target position information of the target object corresponding to a k-th frame of monitoring image of the dome camera (i.e., the second current monitoring position of the dome camera) based on the correlation between the bullet camera and the dome camera.
Optionally, the target position information of the target object may include a value of a pixel in the center of the target box of the target object.
In 760, whether a difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is within a preset threshold may be determined.
In some embodiments, if the difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is within the preset threshold, operation 780 may be performed next. If the difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera is not within the preset threshold, operation 770 may be performed next.
Optionally, the difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera may be a Manhattan distance between a center of a target box of the target object in the first current monitoring position of the dome camera and a center of a target box of the target object in the second current monitoring position of the dome camera.
Optionally, the difference between the first current monitoring position of the dome camera and the second current monitoring position of the dome camera may be a European distance between a center of a target box of the target object in the first current monitoring position of the dome camera and a center of a target box of the target object in the second current monitoring position of the dome camera.
In 770, k may be assigned as a value of (k+1) .
In some embodiments, the device for target tracking may update the running trajectory of the dome camera based on the second current monitoring position of the dome camera, and control the dome camera to track the target object based on the updated running trajectory of the dome camera.
Operation 730 may be performed.
In 780, the target object may be autonomously tracked by the dome camera.
In some embodiments, the device for target tracking may perform a handover between the bullet camera and the dome camera based on the above process for target confirmation, update the image features and motion trajectory information of the target object, and control the dome camera to track the target object autonomously.
The above embodiments are merely common cases of the present disclosure, and do not limit the technical scope of the present disclosure. Therefore, any modifications, changes, or modifications made to the above contents according to the essence of the solution of the present disclosure may still be within the scope of the technical solution of the present disclosure.
FIG. 8 is a schematic diagram illustrating an exemplary device for target tracking according to some embodiments of the present disclosure. As shown in FIG. 8, the device 800 for target tracking shown in some embodiments of the present disclosure may include an identification module 810, a mapping module 820, a prediction module 830, a monitoring module 840, and a tracking module 850.
The identification module 810 may be used to obtain multiple continuous historical monitoring images acquired by the bullet camera, identify the multiple continuous historical monitoring images acquired by the bullet camera, and obtain a monitoring position of the dome camera with respect to the target object.
The mapping module 820 may be used to map the monitoring position of the dome camera with respect to the target object to multiple continuous historical monitoring images acquired by the dome camera to obtain a monitoring position of the dome camera with respect to the target object.
The prediction module 830 may be used to obtain a running trajectory of the dome camera with respect to the target object based on the monitoring position of the dome camera with respect to the target object, and determine a first current monitoring position of the dome camera with respect to the target object based on the running trajectory of the dome camera with respect to the target object.
The monitoring module 840 may be used to obtain a current monitoring image of the bullet camera and obtain a second current monitoring position of the dome camera based on the current monitoring image of the bullet camera.
The tracking module 850 may be used to control the dome camera to track the target object based on the first current monitoring position and the second current monitoring position of the dome camera.
FIG. 9 is a schematic diagram illustrating an exemplary terminal device according to some embodiments of the present disclosure. As shown in Figure 9, the terminal device 900 shown in some embodiments of the present disclosure may include a processor 910, and a storage device 920.
The processor 910 and the storage device 920 may be connected to a bus, respectively. The storage device 920 may store program data. The processor 910 may be used to execute program data to achieve the method for target tracking described in the above embodiments of the present disclosure.
In some embodiments of the present disclosure, the processor 910 may also be referred to as a central processing unit (CPU) . The processor 910 may be an integrated circuit chip with signal processing capability. The processor 910 may also be a general-purpose processor, a digital signal processor (DSP) , an application-specific integrated circuit (ASIC) , a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor, or the processor 910 may also be any conventional processor or the like.
FIG. 10 is a schematic diagram illustrating an exemplary computer-readable medium according to some embodiments of the present disclosure.
In some embodiments of the present disclosure, a computer-readable medium may be provided. As shown in FIG. 10, the computer-readable medium 1000 may store program data 1010, which may be used to realize the method for target tracking described in the above embodiments when executed by the processor.
When the embodiments of the present disclosure are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable medium. Based on this understanding, the technical solution shown in some embodiments of the present disclosure may be embodied in the form of software products, and may be stored in a storage medium, which may include several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc. ) or a processor to perform all or a portion of the operations of the method described in the various embodiments of the present disclosure. The aforementioned medium may include a USB flash disk, a removable hard disk, a read-only memory (ROM) , a random access memory (RAM) , a magnetic disk or optical disk, or other media that can store program code.
Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure and are within the spirit and scope of the exemplary embodiments of this disclosure.
Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and/or “some embodiments” mean that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of the present disclosure are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the present disclosure.
Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software-only solution, e.g., an installation on an existing server or mobile device.
Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various inventive embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, inventive embodiments lie in less than all features of a single foregoing disclosed embodiment.
In some embodiments, the numbers expressing quantities, properties, and so forth, used to describe and claim certain embodiments of the application are to be understood as being modified in some instances by the term “about, ” “approximate, ” or “substantially. ” For example, “about, ” “approximate, ” or “substantially” may indicate ±20%variation of the value it describes, unless otherwise stated. Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the application are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable.
Each of the patents, patent applications, publications of patent applications, and other material, such as articles, books, specifications, publications, documents, things, and/or the like, referenced herein is hereby incorporated herein by this reference in its entirety for all purposes, excepting any prosecution file history associated with same, any of same that is inconsistent with or in conflict with the present document, or any of same that may have a limiting effect as to the broadest scope of the claims now or later associated with the present document. By way of example, should there be any inconsistency or conflict between the description, definition, and/or the use of a term associated with any of the incorporated material and that associated with the present document, the description, definition, and/or the use of the term in the present document shall prevail.
In closing, it is to be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of the application. Other modifications that may be employed may be within the scope of the application. Thus, by way of example, but not of limitation, alternative configurations of the embodiments of the application may be utilized in accordance with the teachings herein. Accordingly, embodiments of the present application are not limited to that precisely as shown and described.

Claims

A method for target tracking, the method comprising:

determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object;

determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple second historical images are acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera;

determining, based on the first motion trajectory, a predicted position associated with the target object; and

in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object, wherein the actual position associated with the target object is determined based on a current image acquired by the bullet camera.
The method of claim 1, wherein determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object includes:

for each of the multiple first historical images,

determining a first historical position of the target object in the first historical image; and

converting the first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on a correlation between the bullet camera and the dome camera.
The method of claim 1, wherein determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object includes:

for each of the multiple second historical images,

determining a second historical position of the target object in the second historical image;

converting the second historical position to a third historical position of the target object in a preset coordinate system based on one of the plurality of historical monitoring positions of the dome camera corresponding to the second historical image; and

generating the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images.
The method of claim 3, wherein the predicted position associated with the target object includes a predicted monitoring position of the dome camera and/or a predicted first position of the target object in the current image acquired by the bullet camera; and

the actual position associated with the target object includes an actual monitoring position of the dome camera and/or an actual first position of the target object in the current image acquired by the bullet camera.
The method of claim 4, wherein the predicted position associated with the target object includes the predicted monitoring position of the dome camera, and determining, based on the first motion trajectory, a predicted position associated with target object includes:

determining a predicted second position of the target object in the preset coordinate system based on the first motion trajectory; and

converting the predicted second position to the predicted monitoring position of the dome camera.
The method of claim 5, wherein the actual position associated with the target object includes the actual monitoring position of the dome camera, and the method further includes:

acquiring the actual monitoring position of the dome camera by:

determining the actual first position of the target object in the current image acquired by the bullet camera; and

converting the actual first position to the actual monitoring position of the dome camera based on a correlation between the bullet camera and the dome camera.
The method of claim 4, wherein the actual position associated with the target object includes the actual first position, the predicted position associated with the target object includes the predicted first position, and determining, based on the first motion trajectory, the predicted position associated with the target object includes:

determining, based on the first motion trajectory, a predicted second position of the target object in the preset coordinate system;

converting the predicted second position to the predicted monitoring position of the dome camera; and

converting the predicted monitoring position of the dome camera to the predicted first position.
The method of claim 7, wherein in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object includes:

determining a second difference, wherein the second difference includes a trajectory difference and/or an image feature difference;

obtaining a fused difference by fusing the second difference and the first difference; and

in response to determining that the fused difference satisfies the preset condition, controlling the dome camera to track the target object.
The method of claim 8, wherein the second difference includes the trajectory difference, and determining the second difference includes:

determining, based on the multiple first historical images, a plurality of first historical positions of the target object;

generating, based on the plurality of first historical positions, a second motion trajectory of the target object; and

determining, based on the first motion trajectory and the second motion trajectory, the trajectory difference.
The method of claim 8, wherein the second difference includes the image feature difference, and determining the second difference includes:

acquiring, based on the actual first position, a first comparison image from the current image;

acquiring, based on the predicted first position, a second comparison image from the current image; and

determining, based on the first comparison image and the second comparison image, the image feature difference.
A system for target tracking, comprising:

at least one storage device including a set of instructions; and

at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to:

determine, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object;

determine, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple historical images are acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera;

determine, based on the first motion trajectory, a predicted position associated with the target object; and

in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, control the dome camera to track the target object, wherein the actual position associated with the target object is determined based on a current image acquired by the bullet camera.
The system of claim 11, wherein to determine, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object, the at least one processor is further configured to:

for each of the multiple first historical images,

determine a first historical position of the target object in the first historical image; and

convert the first historical position to one of the plurality of historical monitoring positions of the dome camera with respect to the target object based on a correlation between the bullet camera and the dome camera.
The system of claim 11, wherein to determine, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, the at least one processor is further configured to:

for each of the multiple second historical images,

determine a second historical position of the target object in the second historical image acquired by the dome camera;

converting the second historical position to a third historical position of the target object in a preset coordinate system based on one of the plurality of historical monitoring positions of the dome camera corresponding to the second historical image; and

generate the first motion trajectory of the target object based on a plurality of third historical positions in the preset coordinate system corresponding to the multiple second historical images.
The system of claim 13, wherein the predicted position associated with the target object includes a predicted monitoring position of the dome camera and/or a predicted first position of the target object in the current image acquired by the bullet camera; and

the actual position associated with the target object includes an actual monitoring position of the dome camera and/or an actual first position of the target object in the current image acquired by the bullet camera.
The system of claim 14, wherein the predicted position associated with the target object includes the predicted monitoring position of the dome camera, and to determine, based on the first motion trajectory, a predicted position associated with the target object the target object, the at least one processor is further configured to:

determine a predicted second position of the target object in the preset coordinate system based on the first motion trajectory; and

convert the predicted second position to the predicted monitoring position of the dome camera.
The system of claim 15, wherein the actual position associated with the target object includes the actual monitoring position of the dome camera, and the at least one processor is further configured to:

acquire the actual monitoring position of the dome camera, wherein to acquire the actual monitoring position of the dome camera, the at least one processor is further configured to:

determine the actual first position of the target object in the current image acquired by the bullet camera; and

convert the actual first position to the actual monitoring position of the dome camera based on a correlation between the bullet camera and the dome camera.
The system of claim 14, wherein the actual position associated with the target object includes the actual first position, the predicted position associated with the target object includes the predicted first position, and to determine, based on the first motion trajectory, the predicted position associated with the target object, the at least one processor is further configured to:

determine, based on the first motion trajectory, a predicted second position of the target object in the preset coordinate system;

convert the predicted second position to the predicted monitoring position of the dome camera; and

convert the predicted monitoring position of the dome camera to the predicted first position.
The system of claim 17, wherein in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, to control the dome camera to track the target object, the at least one processor is further configured to:

determine a second difference, wherein the second difference includes a trajectory difference and/or an image feature difference;

obtain a fused difference by fusing the second difference and the first difference; and

in response to determining that the fused difference satisfies the preset condition, control the dome camera to track the target object.
The system of claim 18, wherein the second difference includes the trajectory difference, and to determine the second difference, the at least one processor is further configured to:

determine, based on the multiple first historical images, a plurality of first historical positions of the target object;

generate, based on the plurality of first historical positions, a second motion trajectory of the target object; and

determine, based on the first motion trajectory and the second motion trajectory, the trajectory difference.
The system of claim 18, wherein the second difference includes the image feature difference, and to determine the second difference, the at least one processor is further configured to:

acquire, based on the actual first position, a first comparison image from the current image;

acquire, based on the predicted first position, a second comparison image from the current image; and

determine, based on the first comparison image and the second comparison image, the image feature difference.
A non-transitory computer readable medium, comprising a set of instructions, wherein when executed by a computer, the set of instructions direct the computer to perform a method, the method comprising:

determining, based on multiple first historical images acquired by a bullet camera, a plurality of historical monitoring positions of a dome camera with respect to a target object;

determining, based on multiple second historical images acquired by the dome camera, a first motion trajectory of the target object, wherein the multiple historical images are acquired by the dome camera based on the plurality of historical monitoring positions of the dome camera;

determining, based on the first motion trajectory, a predicted position associated with the target object; and

in response to determining that a first difference between the predicted position associated with the target object and an actual position associated with the target object satisfies a preset condition, controlling the dome camera to track the target object, wherein the actual position associated with the target object is determined based on a current image acquired by the bullet camera.