WO2020108037A1 - 视频跟踪的方法、装置、设备和存储介质 - Google Patents

视频跟踪的方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2020108037A1
WO2020108037A1 PCT/CN2019/106852 CN2019106852W WO2020108037A1 WO 2020108037 A1 WO2020108037 A1 WO 2020108037A1 CN 2019106852 W CN2019106852 W CN 2019106852W WO 2020108037 A1 WO2020108037 A1 WO 2020108037A1
Authority
WO
WIPO (PCT)
Prior art keywords
tracking
video frame
feature point
next video
associated feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/106852
Other languages
English (en)
French (fr)
Inventor
刘运
周卫
梁柱锦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Baiguoyuan Information Technology Co Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Publication of WO2020108037A1 publication Critical patent/WO2020108037A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • Embodiments of the present application relate to the field of data processing, for example, to a video tracking method, device, device, and storage medium.
  • Lucas–Kanade (LK) optical flow tracking algorithms are used to track the target object in the video picture. Taking a single key point in the target object as an example, you need to obtain the position of the key point in the current video frame first LK optical flow tracking algorithm is used to track the position of this key point in subsequent video frames.
  • the pixel coordinates of the key point in the current video frame are not integer pixel coordinates, you first need to calculate the pixel coordinates of the key point in the current video frame through a linear interpolation algorithm, and because there may be multiple target objects in the current video frame
  • the target tracking operation in the LK optical flow tracking algorithm is complicated, the cost of linear interpolation operation is large, and the high real-time requirements also lead to a low target tracking success rate.
  • the embodiments of the present application provide a method, device, device and storage medium for video tracking, which solves the problems of high linear interpolation operation cost and low tracking success rate in the tracking algorithm, and reduces the amount of calculation in video tracking. Improve the accuracy of video tracking.
  • An embodiment of the present application provides a video tracking method.
  • the method includes:
  • An embodiment of the present application provides a video tracking device, which includes:
  • An associated point acquisition module configured to acquire an associated feature point of a target feature point in the current video frame, the associated feature point being a pixel point having integer pixel coordinates adjacent to the target feature point;
  • An associated position determination module configured to determine the tracking position of the associated feature point in the next video frame
  • the target position determining module is configured to determine the target feature point based on the difference between the position of the associated feature point and the target feature point in the current video frame and the tracking position of the associated feature point in the next video frame The tracking position in the next video frame.
  • An embodiment of the present invention provides a device, which includes:
  • One or more processors are One or more processors;
  • Storage device set to store one or more programs
  • the one or more processors implement the video tracking method described in any embodiment of the present application.
  • An embodiment of the present application provides a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, the method for video tracking described in any embodiment of the present application is implemented.
  • FIG. 1A is a flowchart of a video tracking method according to Embodiment 1 of the present application.
  • FIG. 1B is a schematic diagram of a corresponding position between a target feature point and an associated feature point according to Embodiment 1 of the present application;
  • FIG. 2A is a flowchart of another video tracking method according to Embodiment 2 of the present application.
  • FIG. 2B is a schematic diagram of an iterative process for determining a tracking position of associated feature points in a next video frame according to Embodiment 2 of the present application;
  • FIG. 3 is a flowchart of another video tracking method provided in Embodiment 3 of the present application.
  • FIG. 4 is a schematic structural diagram of a video tracking device according to Embodiment 4 of the present application.
  • FIG. 5 is a schematic structural diagram of a device according to Embodiment 5 of the present application.
  • the pixel coordinates of the at least one target feature point in the current video frame need to be determined first, so as to obtain the pixel grayscale of the at least one target feature point.
  • the target feature points of the coordinates need to use multiple linear interpolation algorithms in the current video frame to determine the pixel coordinates of multiple pixel points adjacent to each target feature point, there is a large calculation overhead, and the pixels of integer pixel coordinates
  • the point position can be easily obtained without using a linear interpolation algorithm, which can greatly reduce the amount of calculation in the linear interpolation process; therefore, the embodiments of the present application are mainly directed to the target feature point of the non-integer pixel coordinates in the current video frame, which will be close to the target feature point
  • the associated feature point with integer pixel coordinates is used as an intermediate quantity to determine the tracking position of the associated feature point in the next video frame, and then the target feature point is determined according to the position difference between the target feature point and the associated feature point in the current
  • FIG. 1A is a flowchart of a video tracking method according to Embodiment 1 of the present application.
  • This embodiment can be applied to any mobile terminal configured with a video playback application and requiring position tracking of a target object in a video screen .
  • the technical solutions of the embodiments of the present application are applicable to the case of how to accurately track a target object in a video picture.
  • the video tracking method provided in this embodiment may be executed by the video tracking device provided in the embodiment of the present application.
  • the device may be implemented by software and/or hardware, and integrated in the device that executes the method.
  • the device may be a mobile terminal such as a smartphone, tablet computer, or notebook that needs to track the position of the target object in the video screen.
  • the method may include S110-S130.
  • the associated feature point is a pixel point having integer pixel coordinates adjacent to the target feature point.
  • the video is composed of multiple time-varying video frames.
  • the current video frame may be the first video frame in the video or a time-varying subsequent video frame;
  • the characteristic point is that the current video frame has a distinctive Characteristics and can effectively reflect the essential characteristics of the object and represent the pixel points of the corresponding target part in the video frame
  • the target feature point is a key feature point that can represent at least one key part of the target object that needs to be tracked in the current video frame;
  • the target feature point can be obtained by using a target positioning algorithm
  • the target feature point can be obtained by using the The video tracking method tracks the position of the corresponding feature point in the previous video frame.
  • the position of the target feature point in the current video frame may be a non-integer pixel coordinate
  • a linear interpolation algorithm needs to be used to determine the adjacent pixel point
  • the pixel coordinates of will have a large computational overhead; therefore, a concept of associated feature points is introduced, as shown in FIG. 1B.
  • point q s is the target feature point in the current video frame
  • the neighboring point q s will have Point with integer pixel coordinates
  • the associated feature points since the time variation between two adjacent video frames is very small, it can be set that the deformation in the set area centered on the target feature point in two consecutive adjacent video frames is very small, which is different from the target
  • the associated feature points adjacent to the feature points are also in the set area, so when judging the displacement of the target feature point moving from the current video frame to the next video frame, the displacement and the associated feature points can be set reasonably
  • the displacement of the target feature point is approximately equal, and the displacement of the target feature point can be used as the displacement of the target feature point by calculating the displacement of the associated feature point when moving from the current video frame to the next video frame.
  • the associated feature point is the position closest to the target feature point in the current video frame Of pixels with integer pixel coordinates.
  • the target feature points of the target object in multiple video frames need to be determined in sequence, if the current video frame is the first video frame in the video, Then, the target feature point in the current video frame can be obtained by using a target positioning algorithm; if the current video frame is a time-varying subsequent video frame in the video, the target in the previous video frame can be targeted by the video tracking method in this embodiment
  • the feature points are tracked to obtain the tracking position of the target feature point in the current video frame.
  • the video tracking algorithm is directly used to determine the tracking position of the target feature point in the next video frame; if the tracking position of the target feature point is not an integer pixel coordinate, the position pixel of the target feature point is also required Coordinates, select a pixel point with integer pixel coordinates that is closest to the target feature point in the current video frame, use the selected pixel point as the associated feature point of the target feature point, and then use the associated feature point in the current video.
  • the movement of the frame to the next video frame is used as a reference to track the target feature point.
  • S120 Determine the tracking position of the associated feature point in the next video frame.
  • a video tracking algorithm may be used to track the position of the associated feature point in the next video frame.
  • Video tracking algorithms mainly include: Centroid tracking algorithm, Multi Target Tracking (MTT) algorithm, Correlation tracking algorithm, Edge tracking algorithm, Phase Correlation tracking algorithm, scene locking (Scene Lock) algorithm, combined tracking algorithm and LK optical flow tracking algorithm, etc.
  • the moving speed determined in the LK optical flow tracking algorithm is:
  • I x (q n ) is the partial derivative of the pixel gray level at the pixel point q n to the pixel variable x
  • I y (q n ) is the partial derivative of the pixel gray level at the pixel point q n to the pixel variable y
  • I t (q n ) is the partial derivative of the gray level of the pixel at the pixel point q n to the time variable t
  • q 1 to q n are the multiple pixels in the set area near the target feature point q s in the current video frame.
  • the nearby multiple pixel points q 1 to q n are also not integer pixel coordinates, and each is determined according to formula (2)
  • Formula (2) determines the partial derivatives of the pixel gray values of multiple pixels near the target feature point in formula (1) to multiple variables, and the target feature point is obtained according to the multiple partial derivatives in formula (1)
  • the LK optical flow tracking algorithm needs to determine the pixel coordinates of multiple pixel points q1 to qn near each target feature point in the current video frame through multiple linear interpolation algorithms, so as to obtain the corresponding pixel points required in formula (2)
  • the gray value of the pixel the use of multiple linear interpolation algorithms in the video tracking process brings a large calculation overhead; at the same time, in order to solve the problem of low accuracy and large error of the target feature point obtained after one optical flow calculation, it also
  • the above process needs to be performed multiple times for iterative processing to continuously approximate the actual position of the target feature point in the next video frame, and each iteration process needs to use a linear interpolation algorithm multiple times to obtain the pixel coordinates of the target feature point in the current video frame,
  • the iterative operation is extremely cumbersome, which brings greater computational overhead, greatly reduces the accuracy of video tracking, and easily causes jitter in the tracking result.
  • the target feature point in the above process is directly replaced with the associated feature point, and the location of the associated feature point in the next video frame is performed. track.
  • the points q 1 to q n in the are the associated feature points in the current video frame Multiple pixels in the nearby setting area; due to associated feature points Is a pixel point with integer pixel coordinates, then multiple nearby pixel points q 1 to q n also have integer pixel coordinates, without using a linear interpolation algorithm to calculate the pixel coordinates of multiple pixel points q 1 to q n in the current video frame , Greatly reducing the computational overhead caused by linear interpolation; according to formula (2), it can also be determined that each iteration, the gray scale of multiple pixel points q 1 to q n in A of formula (1) to the pixel coordinate variable at each iteration
  • the partial derivatives of x and y are all fixed values and do not need to be calculated again.
  • multiple pixel points q 1 to q n are The grayscale of the pixels in the current video frame is a fixed value, and there is no need to calculate again. It only needs to recalculate the grayscale of the pixels q 1 to q n in the next video frame for each pixel in the set area during each iteration. Therefore, after the final iteration, the tracking position of the associated feature point in the next video frame is obtained, which greatly reduces the linear interpolation overhead in the video tracking process, reduces the calculation amount in the iteration process, and improves the tracking rate. On the basis, the accuracy of video tracking is improved.
  • S130 Determine the tracking position of the target feature point in the next video frame according to the position difference between the associated feature point and the target feature point in the current video frame, and the tracking position of the associated feature point in the next video frame.
  • the time variation between two adjacent video frames is very small, it can be set that the deformation of the setting area centered on the target feature point in two consecutive adjacent video frames is very small, which is different from the target feature
  • the related feature points adjacent to the point are also in the set area, and the deformation of the set area is small to ensure that the position difference between the target feature point and the related feature point in the current video frame, and the target feature point and the related feature point in the next video
  • the position difference in the frame remains almost the same, so the position difference between the target feature point and the associated feature point in the current video frame can be reasonably used as the target object, the target feature point and the associated feature point in the next video frame Poor location.
  • the position difference between the target feature points and the associated feature points in the current video frame needs to be determined, and according to the position difference and the determined associated features
  • the tracking position of the point in the next video frame directly determines the tracking position of the target feature point in the next video frame, thereby realizing the tracking of the target feature point in the video.
  • the technical solution provided in this embodiment uses the associated feature point with integer pixel coordinates adjacent to the target feature point in the current video frame as an intermediate reference quantity to determine the tracking position of the associated feature point in the next video frame, and then passes the current video The position difference between the two in the frame to determine the tracking position of the target feature point in the next video frame, without using multiple linear interpolation algorithms to obtain the pixel coordinates of each target feature point in the current video frame, which accelerates the tracking rate and meets the video tracking.
  • the real-time requirement of the solution solves the problems of high cost of linear interpolation operation and low tracking success rate in the tracking algorithm, reduces the amount of calculation in video tracking, and improves the accuracy of video tracking.
  • FIG. 2A is a flowchart of another video tracking method according to Embodiment 2 of the present application. This embodiment is described based on the above embodiment. As shown in FIG. 2B, in this embodiment, an iterative process for determining the tracking position of the associated feature point in the next video frame is described.
  • S210-S270 may be included in this embodiment.
  • S220 Determine the associated feature point in the next video frame according to the pixel gray of multiple pixels in the first tracking area in the current video frame and the pixel gray of multiple pixels in the second tracking area in the next video frame The initial tracking position in.
  • the first tracking area is the area centered on the associated feature point in the current video frame
  • the second tracking area is the area corresponding to the first tracking area in the next video frame.
  • first determine the first tracking area in the current video frame as shown in FIG. 2B, when the associated feature point is obtained in the current video frame, the associated feature point is used as the center to determine that it can contain a certain pixel range
  • the rectangular frame of the pixel points is used as the first tracking area, and a plurality of pixel points in the first tracking area are separated from the associated feature point by a certain integer pixel coordinate.
  • the associated feature point is set to Then the multiple pixels near the associated feature point, that is, the multiple pixels in the first tracking area are In this formula, m ⁇ [-L, L], n ⁇ [-L, L], and m, n and L are integers, L can represent the size of the set first tracking area; the first tracking area is In the current video frame, As the center, a square area with a side length of 2*L+1.
  • the initial tracking position of the associated feature point in the next video frame when calculating the initial tracking position of the associated feature point in the next video frame according to the formula (1) and formula (2) in the above-mentioned first embodiment, it is first necessary to calculate the first tracking area in the current video frame
  • the pixel gray scale of a plurality of pixels determining the partial derivatives of the plurality of pixels in A in formula (1) to the pixel coordinate variables x, y; and determining the formula by pre-determining the second tracking area in the next video frame (1) Part b of the pixel gray scale of multiple pixels in the tracking area in different video frames in time frame t to obtain the moving speed of the associated feature point from the current video frame to the next video frame , So as to determine the initial tracking position of the associated feature point in the next video frame according to the time difference between two adjacent frames.
  • the second tracking area in the next video frame may be set to the area corresponding to the first tracking area in the current video frame in the next video frame. In order to re-determine the corresponding second tracking area in the next video frame according to the obtained initial tracking position of the associated feature point in the next video frame, to achieve the convergence of the LK optical flow tracking algorithm in the iterative process.
  • S230 Determine the current displacement of the associated feature point according to the preset position of the associated feature point in the next video frame and the initial tracking position.
  • the position of the associated feature point in the current video frame can be directly used as the next video frame
  • the preset position of can also calculate the moving speed of the associated feature point in the first few video frames, and obtain an estimated position based on the moving speed and the time difference between the two adjacent frames, and the position in the current video frame, As the preset position in the next video frame; however, when the target object has a large motion range or irregular motion, the preset position determined by the above two methods may be related to the actual position of the associated feature point in the next video frame The difference is far away, which leads to tracking failure or requires multiple iterations to converge, which increases the calculation overhead.
  • the video tracking method in this embodiment can also be combined with a target positioning algorithm, and the preset position of the associated feature point in the next video frame can be determined by the target positioning algorithm, that is, the target positioning algorithm is used in the next video frame to obtain the association
  • the target positioning algorithm that is, the target positioning algorithm is used in the next video frame to obtain the association
  • the preset position of the feature point in the next video frame during the first iteration, the determined preset position can better fall near the true position of the associated feature point in the next video frame, and iterate on this basis, in turn Approaching the actual position of the associated feature point in the next video frame can enable the iterative process to converge faster and improve the video tracking rate.
  • the initial tracking position determined last time is sequentially used as the new preset position, and compared with the recovered initial tracking position to determine whether convergence has been reached.
  • the preset threshold is a preset target value that can ensure that the obtained initial tracking position is closest to the actual position of the associated feature point in the next video frame.
  • the current displacement exceeds the preset threshold it means that the current iteration has not converged, and there is a certain gap between the initial tracking position and the real position; when the current displacement does not exceed the preset threshold, it means that the iteration has converged.
  • the initial tracking position has been very close to the true position of the associated feature point in the next video frame, and there is no need to iterate again.
  • the current displacement exceeds the preset threshold, it means that the iteration has not yet converged, and the initial tracking position obtained this time still has a certain gap with the real position. It is necessary to continue to re-determine the new initial tracking in the next video frame Position; take the initial tracking position obtained this time as the new preset position in the next iteration, and take the initial tracking position obtained this time as the center and the preset 2*L+1 as the side length in the next video again Re-determine a new second tracking area in the frame, return to S220, continue according to the pixel grayscale of multiple pixels in the first tracking area in the current video frame, and the new second tracking area in the next video frame Pixel gray of multiple pixels, re-determine the new initial tracking position of the associated feature point in the next video frame, and perform another iterative process.
  • the initial tracking position obtained during multiple iterations is To (n is the number of iterations)
  • the initial tracking position obtained from the previous iteration in turn As the center, with 2*L+1 as the side length, re-determine the new second tracking area in the next video frame, and calculate the associated feature points again New initial tracking position in the next video frame
  • the center point in the new second tracking area that is, the initial tracking position obtained last time may be non-integer pixel coordinates, so in determining the pixels of multiple pixels in the new second tracking area in the next video frame
  • For grayscale first of all, it is necessary to determine the pixel coordinates of multiple pixels in the new second tracking area through a linear interpolation algorithm, so as to obtain the pixel grayscale of multiple pixels.
  • the new initial tracking position is continuously compared with the new preset position to obtain the current displacement in this iteration, and it is judged again whether the current displacement still exceeds the preset Threshold; loop in turn until the current displacement does not exceed the preset threshold, completing the iterative process.
  • the initial tracking position obtained in this iteration can be used as the final tracking position of the associated feature point in the next video frame.
  • S270 Determine the tracking position of the target feature point in the next video frame according to the position difference between the associated feature point and the target feature point in the current video frame, and the tracking position of the associated feature point in the next video frame.
  • FIG. 3 is a flowchart of another video tracking method provided in Embodiment 3 of the present application. This embodiment is based on the above embodiment. In the process of tracking the target feature points in the current video frame in this embodiment, the tracking confidence of the current video tracking can be determined without giving an additional calculation amount, and a relatively accurate tracking effect evaluation can be given .
  • this embodiment may include 310-S360.
  • S320 Determine the associated feature point in the next video frame according to the pixel gray of multiple pixels in the first tracking area in the current video frame and the pixel gray of multiple pixels in the second tracking area in the next video frame The initial tracking position in.
  • S330 Determine the current displacement of the associated feature point according to the preset position of the associated feature point in the next video frame and the initial tracking position.
  • S350 Determine the tracking position of the target feature point in the next video frame according to the position difference between the associated feature point and the target feature point in the current video frame, and the tracking position of the associated feature point in the next video frame.
  • S360 Determine the tracking confidence of the current video tracking according to the pixel gray levels of multiple pixels in the first tracking area in the current video frame and the pixel gray levels of multiple pixels in the second tracking area in the next video frame .
  • the pixel grayscale of multiple pixels in the first tracking area in the current video frame is determined during each iteration, And the pixel grayscale of multiple pixels in the second tracking area in the next video frame, to determine the partial derivative of the pixel grayscale of multiple pixels in each iteration process to the time variable t, which is one of the embodiments of the present application
  • the partial derivative values of multiple terms in b of formula (1) mentioned are The partial derivative can represent the difference in the grayscale of the pixels on the same pixel in the tracking area of different video frames, that is, to determine the similarity of the video images in the tracking area of different video frames.
  • the pixel grayscale of multiple pixels in the second tracking area of Zhongxin that is, to determine the partial derivative of the pixel grayscale of multiple pixels in the tracking area to the time variable t during the last iteration, to determine the current video tracking Confidence in tracking.
  • the video images in the first tracking area in the frame are not relevant, that is, when the video images in the second tracking area and the first tracking area have a large difference, var(I t (q)) will be very large, and the Score tends to 0 ;
  • the tracking confidence in this embodiment can more accurately reflect the similarity of the tracking areas in different video frames, and because the LK optical flow tracking algorithm itself
  • the target positioning algorithm can be directly used in the video frame to recalculate the position of the target feature point.
  • the video tracking method in this embodiment It can be used in combination with a specific target positioning algorithm to suppress the jitter of the positioning result of the target positioning algorithm in consecutive frames of the video, and the calculation amount is low, the resource consumption is small, and the stability of the positioning result of the target object in the video is greatly improved.
  • the technical solution provided by this embodiment on the basis of tracking the target feature points in the current video frame, by analyzing the intermediate calculation results existing in the tracking process, to determine the tracking confidence of this video tracking, without additional increase Calculate the cost and objectively evaluate the effectiveness of this video tracking to effectively judge the accuracy of this video tracking.
  • FIG. 4 is a schematic structural diagram of a video tracking device according to Embodiment 4 of the present application. As shown in FIG. 4, the device may include:
  • the related point acquiring module 410 is configured to acquire related feature points of a target feature point in the current video frame, and the related feature points are pixel points with integer pixel coordinates adjacent to the target feature point.
  • the associated position determination module 420 is configured to determine the tracking position of the associated feature point in the next video frame.
  • the target position determining module 430 is set to determine the tracking position of the target feature point in the next video frame based on the difference in position between the associated feature point and the target feature point in the current video frame, and the tracking position of the associated feature point in the next video frame .
  • the technical solution provided in this embodiment uses the associated feature point with integer pixel coordinates adjacent to the target feature point in the current video frame as an intermediate reference quantity to determine the tracking position of the associated feature point in the next video frame, and then passes the current video The position difference between the two in the frame to determine the tracking position of the target feature point in the next video frame, without using multiple linear interpolation algorithms to obtain the pixel coordinates of each target feature point in the current video frame, which accelerates the tracking rate and meets the video tracking.
  • the real-time requirement of the solution solves the problems of high cost of linear interpolation operation and low tracking success rate in the tracking algorithm, reduces the amount of calculation in video tracking, and improves the accuracy of video tracking.
  • the associated position determination module 420 is set to: according to the pixel grayscale of a plurality of pixels in the first tracking area in the current video frame and a plurality of pixels in the second tracking area in the next video frame The grayscale of the pixel, to determine the initial tracking position of the associated feature point in the next video frame, where the first tracking area is the area centered on the associated feature point in the current video frame, and the second tracking area is the next video frame The area corresponding to the first tracking area; the current displacement of the associated feature point is determined according to the preset position of the associated feature point in the next video frame and the initial tracking position; when the current displacement exceeds the preset threshold, the initial tracking position is used as A new preset position, centered on the initial tracking position, determine a new second tracking area in the next video frame, and return to execute the pixel according to the multiple pixels in the first tracking area in the current video frame Grayscale, and the pixel grayscale of multiple pixels in the second tracking area in the next video frame, to determine the initial tracking position of the associated feature point
  • the pixel coordinates of the initial tracking position are non-integer pixel coordinates
  • the pixel coordinates of multiple pixels in the second tracking area in the next video frame may be determined by a linear interpolation algorithm.
  • the preset position of the associated feature point in the next video frame may be determined by a target positioning algorithm.
  • the video tracking device may further include: a confidence determination module, configured to use the new initial tracking position as the associated feature point tracking position in the next video frame, according to the first tracking in the current video frame
  • a confidence determination module configured to use the new initial tracking position as the associated feature point tracking position in the next video frame, according to the first tracking in the current video frame
  • the pixel gray levels of multiple pixels in the area and the pixel gray levels of multiple pixels in the second tracking area in the next video frame determine the tracking confidence of this video tracking.
  • the video tracking device provided in this embodiment can be applied to the video tracking method provided in any of the foregoing embodiments, and has corresponding functions and beneficial effects.
  • the device includes a processor 50, a storage device 51, a communication device 52, an input device 53, and an output device 54; the processor in the device
  • the number of 50 may be one or more, and one processor 50 is taken as an example in FIG. 5; the processor 50, the storage device 51, the communication device 52, the input device 53, and the output device 54 in the device may be connected through a bus or other means , Figure 5 takes the connection through the bus as an example.
  • the storage device 51 as a computer-readable storage medium may be configured to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the video tracking method described in any embodiment of the present application.
  • the processor 50 executes at least one functional application of the device and data processing by running software programs, instructions, and modules stored in the storage device 51, that is, implementing the above-mentioned video tracking method.
  • the storage device 51 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system and at least one function required application program; the storage data area may store data created according to the use of the terminal, etc. .
  • the storage device 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the storage device 51 may include memories remotely provided with respect to the processor 50, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.
  • the communication device 52 may be configured to implement network connection or mobile data connection between devices.
  • the input device 53 may be configured to receive input numeric or character information, and generate key signal input related to user settings and function control of the device.
  • the output device 54 may include a display device such as a display screen.
  • a device provided in this embodiment may be configured to perform the video tracking method provided in any of the above embodiments, and has corresponding functions and beneficial effects.
  • Embodiment 6 of the present application further provides a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, the video tracking method in any of the foregoing embodiments may be implemented.
  • the method may include: acquiring an associated feature point of a target feature point in the current video frame, the associated feature point being a pixel point having integer pixel coordinates adjacent to the target feature point; determining a tracking position of the associated feature point in the next video frame; The tracking position of the target feature point in the next video frame is determined according to the difference in position between the associated feature point and the target feature point in the current video frame, and the tracking position of the associated feature point in the next video frame.
  • a storage medium containing computer-executable instructions provided by an embodiment of the present application is not limited to the method operations described above, and can also perform relevant methods in the video tracking method provided by any embodiment of the present application operating.
  • the present application can be implemented by software and general hardware, or by hardware.
  • the technical solution of the present application can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (Read-Only Memory, ROM), random access memory ( Random Access (RAM), flash memory (FLASH), hard disk or optical disc, etc., including multiple instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform any of the embodiments described in this application method.
  • a computer-readable storage medium such as a computer floppy disk, read-only memory (Read-Only Memory, ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disc, etc.
  • the included units and modules are only divided according to functional logic, but it is not limited to the above division, as long as the corresponding functions can be realized; in addition, the name of each functional unit It is only for the purpose of distinguishing each other, and is not intended to limit the scope of protection of this application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

本文公开了视频跟踪的方法、装置、设备和存储介质。该方法包括：获取当前视频帧中目标特征点的关联特征点，该关联特征点为与目标特征点邻近的具有整数像素坐标的像素点；确定关联特征点在下一视频帧中的跟踪位置；根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。

Description

视频跟踪的方法、装置、设备和存储介质
本申请要求在2018年11月29日提交中国专利局、申请号为201811447836.0的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及数据处理领域，例如涉及一种视频跟踪的方法、装置、设备和存储介质。
背景技术
在网络直播或短视频录制等产品中，较多特效功能需要对视频画面中的目标物体进行定位，例如在视频中逐帧对多个目标物体进行关键点定位，由于连续多帧中的同一目标物体的定位可能存在误差，会造成在视觉上，定位出来的目标位置会随着时间做微小的抖动，影响用户体验。
行业内大多采用Lucas–Kanade(LK)光流跟踪算法对视频画面中的目标物体进行跟踪，以目标物体中的单一关键点为例，首先需要获取该关键点在当前视频帧中的位置，然后采用LK光流跟踪算法对该关键点在后续视频帧中的位置进行跟踪。
若该关键点在当前视频帧中的像素坐标不是整数的像素坐标，则首先需要通过线性插值算法计算当前视频帧中该关键点的像素坐标，且由于当前视频帧中目标物体中可能存在多个关键点，则需要通过多次线性插值算法确定每一个关键点的像素坐标，从而对当前视频帧中的每一关键点都进行一次跟踪；同时，为了解决经过一次光流计算得到的关键点位置精度较低、误差较大的问题，还需要采用多次线性插值算法对每一关键点进行迭代处理，不断逼近该关键点在下一视频帧中的实际位置。
因此LK光流跟踪算法中的目标跟踪操作复杂，线性插值运算的开销较大，实时性的高要求也导致目标跟踪成功率较低。
发明内容
本申请实施例提供了一种视频跟踪的方法、装置、设备和存储介质，解决了跟踪算法中线性插值运算的开销较大以及跟踪成功率较低的问题，降低了视频跟踪中的运算量，提高了视频跟踪的准确性。
本申请实施例提供了一种视频跟踪的方法，该方法包括：
获取当前视频帧中目标特征点的关联特征点，所述关联特征点为与所述目标特征点邻近的具有整数像素坐标的像素点；
确定所述关联特征点在下一视频帧中的跟踪位置；
根据所述关联特征点与所述目标特征点在当前视频帧中的位置差，以及所述关联特征点在下一视频帧中的跟踪位置，确定所述目标特征点在所述下一视频帧中的跟踪位置。
本申请实施例提供了一种视频跟踪的装置，该装置包括：
关联点获取模块，设置为获取当前视频帧中目标特征点的关联特征点，所述关联特征点为与所述目标特征点邻近的具有整数像素坐标的像素点；
关联位置确定模块，设置为确定所述关联特征点在下一视频帧中的跟踪位置；
目标位置确定模块，设置为根据所述关联特征点与所述目标特征点在当前视频帧中的位置差，以及所述关联特征点在下一视频帧中的跟踪位置，确定所述目标特征点在所述下一视频帧中的跟踪位置。
本发明实施例提供了一种设备，该设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本申请任意实施例所述的视频跟踪的方法。
本申请实施例提供了一种计算机可读存储介质，存储有计算机程序，该程序被处理器执行时实现本申请任意实施例所述的视频跟踪的方法。
附图说明
图1A为本申请实施例一提供的一种视频跟踪的方法的流程图；
图1B为本申请实施例一提供的一种目标特征点与关联特征点的对应位置的示意图；
图2A为本申请实施例二提供的另一种视频跟踪的方法的流程图；
图2B为本申请实施例二提供的一种确定关联特征点在下一视频帧中的跟踪位置的迭代过程的示意图；
图3为本申请实施例三提供的另一种视频跟踪的方法的流程图；
图4为本申请实施例四提供的一种视频跟踪的装置的结构示意图；
图5为本申请实施例五提供的一种设备的结构示意图。
具体实施方式
下面结合附图和实施例对本申请进行说明。本文所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
由于对当前视频帧中的至少一个目标特征点进行跟踪，首先需要确定该至少一个目标特征点在当前视频帧中的像素坐标，从而获得该至少一目标特征点的像素灰度，对于非整数像素坐标的目标特征点，需要在当前视频帧中采用多次线性插值算法来确定与每个目标特征点相邻的多个像素点的像素坐标，存在较大的运算开销，而整数像素坐标的像素点位置无需采用线性插值算法便可以轻易得到，可以极大降低线性插值过程中的运算量；因此本申请实施例主要针对当前视频帧中非整数像素坐标的目标特征点，将与目标特征点临近的具有整数像素坐标的关联特征点作为中间量，来确定该关联特征点在下一视频帧中的跟踪位置，进而根据目标特征点和关联特征点在当前视频帧中的位置差，确定目标特征点在下一视频帧中的跟踪位置，无需采用线性插值算法，极大降低了视频跟踪过程中的运算量，加快了视频跟踪速率，满足视频跟踪的实时性要求，提高了视频跟踪的准确性。
实施例一
图1A为本申请实施例一提供的一种视频跟踪的方法的流程图，本实施例可应用于任意一种配置有视频播放应用，且对视频画面中目标物体有位置跟踪需求的移动终端上。本申请实施例的技术方案适用于如何对视频画面中的目标物体进行准确跟踪的情况中。本实施例提供的一种视频跟踪的方法可以由本申请实施例提供的视频跟踪的装置来执行，该装置可以通过软件和/或硬件的方式来实现，并集成在执行本方法的设备中，该设备可以是需要对视频画面中目标物体的位置进行跟踪的智能手机、平板电脑、笔记本等移动终端。
参考图1A，该方法可以包括S110-S130。
S110，获取当前视频帧中目标特征点的关联特征点。
本实施例中，关联特征点为与目标特征点邻近的具有整数像素坐标的像素点。一实施例中，视频是由多个时变的视频帧构成，当前视频帧可以是该视频中的首个视频帧，也可以是时变的后续视频帧；特征点为当前视频帧中具有鲜明特性并能够有效反映物体本质特征、表示视频帧中对应目标部位的像素点， 目标特征点是在当前视频帧中能够表示出需要进行位置跟踪的目标物体的至少一个关键部位的关键特征点；在当前视频帧为视频中的首个视频帧时，该目标特征点可以通过采用目标定位算法得到，在当前视频帧为时变的后续视频帧时，该目标特征点可以通过采用本实施例中的视频跟踪的方法对上一视频帧中的对应特征点的位置进行跟踪得到。
一实施例中，由于目标特征点在当前视频帧中的位置可能是非整数的像素坐标，如果直接对该目标特征点在下一视频帧中的位置进行跟踪，需要采用线性插值算法确定相邻像素点的像素坐标，会存在较大的运算开销；因此引入一个关联特征点的概念，如图1B所示，若点q s为当前视频帧中的目标特征点，则将与点q s邻近的具有整数像素坐标的点
Figure PCTCN2019106852-appb-000001
作为关联特征点，由于相邻两个视频帧之间的时变非常小，可以设定连续相邻的两个视频帧中以目标特征点为中心的设定区域内的形变很小，与目标特征点邻近的关联特征点也在该设定区域内，因此在对从当前视频帧移动到下一视频帧的该目标特征点的位移进行判断时，可以合理地设定关联特征点的位移与目标特征点的位移近似相等，可以通过计算该关联特征点从当前视频帧移动到下一视频帧时的位移，作为该目标特征点的位移。同时，为了降低跟踪误差，需要尽量保证关联特征点与目标特征点在当前视频帧和下一视频帧中的相对位置不变，因此关联特征点为当前视频帧中与目标特征点的位置最为邻近的具有整数像素坐标的像素点。
一实施例中，在对一个目标物体在视频中的位置进行跟踪时，首先需要依次确定该目标物体在多个视频帧中的目标特征点，若当前视频帧为视频中的首个视频帧，则可以采用目标定位算法获取当前视频帧中的目标特征点；若当前视频帧为视频中时变的后续视频帧，则可以通过本实施例中的视频跟踪的方法对上一视频帧中的目标特征点进行跟踪，得到该目标特征点在当前视频帧中的跟踪位置。在得到当前视频帧中的目标特征点后，为了降低跟踪过程中的运算量，提高对应的跟踪效率，还需要判断该目标特征点的跟踪位置是否为整数像素坐标；若该目标特征点的跟踪位置为整数像素坐标，则直接采用视频跟踪算法确定该目标特征点在下一视频帧的跟踪位置；若该目标特征点的跟踪位置不为整数像素坐标，则还需要根据该目标特征点的位置像素坐标，在当前视频帧中选取一个与该目标特征点最为邻近的具有整数像素坐标的像素点，将该选取的像素点作为该目标特征点的关联特征点，后续将该关联特征点在当前视频帧到下一视频帧中的的移动作为参考，对该目标特征点进行跟踪。
S120，确定关联特征点在下一视频帧中的跟踪位置。
一实施例中，在当前视频帧中得到目标特征点的关联特征点后，可以采用视频跟踪算法对该关联特征点在下一视频帧中的位置进行跟踪。视频跟踪算法 主要包括：质心跟踪(Centroid)算法、多目标跟踪(Multi Target Tracking，MTT)算法、相关(Correlation)跟踪算法、边缘(Edge)跟踪算法、相位相关(Phase Correlation)跟踪算法、场景锁定(Scene Lock)算法、组合(Combined)跟踪算法以及LK光流跟踪算法等。
由于本实施例中采用LK光流跟踪算法对视频中的目标物体进行跟踪，若直接采用LK光流跟踪算法对目标特征点进行跟踪，则当前视频帧中多个目标特征点的像素灰度是关于像素坐标变量x、y和时间变量t的函数，若给定t 0时刻对应的当前视频帧中的目标特征点q s(x s,y s)，在经过一段很短的时间Δt后，当前视频帧中的目标特征点q s(x s,y s)，会移动到(t 0+Δt)时刻对应的下一视频帧中的点q d(x d,y d)处，移动速度为v＝(q d-q s)/Δt。LK光流跟踪算法中确定的移动速度为：
v＝(A TA)- 1A Tb    (1)
公式(1)中，
Figure PCTCN2019106852-appb-000002
I x(q n)是像素点q n处的像素灰度对像素变量x的偏导，I y(q n)是像素点q n处的像素灰度对像素变量y的偏导，I t(q n)是像素点q n处的像素灰度对时间变量t的偏导，q 1到q n是在当前视频帧中目标特征点q s附近设定区域内的多个像素点。
对于像素灰度的偏导有：
Figure PCTCN2019106852-appb-000003
若目标特征点q s的像素坐标q s(x s,y s)不是整数像素坐标，那么附近的多个像素点q 1到q n也不是整数像素坐标，在根据公式(2)确定每个像素点处的偏导前，首先需要通过多次线性插值算法计算在当前视频帧中该目标特征点附近的多个像素点的像素坐标，从而得到多个像素点的像素灰度值，然后根据公式(2)确定出公式(1)中该目标特征点附近的多个像素点的像素灰度对多个变量的偏导，并根据公式(1)中的多个偏导得到该目标特征点的移动速度，本实施例中设定相邻两帧的时间差Δt＝1，则根据当前视频帧中的该目标特征点q s(x s,y s)与得到的移动速度可以确定该关键点在下一视频帧中的位置。
LK光流跟踪算法中需要通过多次线性插值算法分别确定当前视频帧中每一个目标特征点附近的多个像素点q1到qn的像素坐标，从而得到公式(2)中所 需的对应像素点的像素灰度值，视频跟踪过程中采用多次线性插值算法带来了较大的运算开销；同时为了解决经过一次光流计算得到的目标特征点位置精度较低、误差较大的问题，还需要多次执行上述过程进行迭代处理，不断逼近目标特征点在下一视频帧中的实际位置，而每次迭代过程均需要多次采用线性插值算法得到目标特征点在当前视频帧中的像素坐标，迭代操作极其繁琐，带来了更大的运算开销，极大降低了视频跟踪的准确性，极易引起跟踪结果的抖动。
因此，本实施例中在当前视频帧中获取到该目标特征点的关联特征点时，直接将上述过程中的目标特征点替换为关联特征点，对关联特征点在下一视频帧中的位置进行跟踪。公式(1)中
Figure PCTCN2019106852-appb-000004
中的点q 1到q n则为当前视频帧中关联特征点
Figure PCTCN2019106852-appb-000005
附近设定区域内的多个像素点；由于关联特征点
Figure PCTCN2019106852-appb-000006
为具有整数像素坐标的像素点，那么附近的多个像素点q 1到q n也具有整数像素坐标，无需采用线性插值算法计算多个像素点q 1到q n在当前视频帧中的像素坐标，极大降低了线性插值带来的运算开销；根据公式(2)也可以确定每次迭代时，公式(1)的A中多个像素点q 1到q n的像素灰度对像素坐标变量x、y的偏导均为定值，无需再次计算，b中计算多个像素点q 1到q n的像素灰度对时间变量t的偏导时，多个像素点q 1到q n在当前视频帧中的像素灰度为定值，无需再次计算，仅需要在每次迭代过程中重新计算设定区域内的多个像素点q 1到q n在下一视频帧中的像素灰度，从而在最终迭代后得到关联特征点在下一视频帧中的跟踪位置，极大降低了视频跟踪过程中的线性插值开销，减少了迭代过程中的运算量，提高了跟踪速率，在满足实时性的基础上，提高了视频跟踪的准确性。
S130，根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。
本实施例中，由于相邻两个视频帧之间的时变非常小，可以设定连续相邻的两个视频帧中以目标特征点为中心的设定区域的形变很小，与目标特征点邻近的关联特征点也在该设定区域内，设定区域的形变很小可以保证目标特征点和关联特征点在当前视频帧中的位置差，与目标特征点和关联特征点在下一视频帧中的位置差几乎保持不变，因此可以合理地将目标特征点和关联特征点在当前视频帧中的位置差，作为目标物体移动后，目标特征点和关联特征点在下一视频帧中的位置差。一实施例中，在获取当前视频帧中目标特征点的关联特征点后，还需要确定目标特征点与关联特征点在当前视频帧中的位置差，并根 据该位置差，以及确定的关联特征点在下一视频帧中的跟踪位置，直接确定目标特征点在下一视频帧中的跟踪位置，从而实现视频中对目标特征点的位置跟踪。
本实施例提供的技术方案，将当前视频帧中与目标特征点临近的具有整数像素坐标的关联特征点作为中间参考量，确定该关联特征点在下一视频帧中的跟踪位置，进而通过当前视频帧中两者的位置差，确定下一视频帧中目标特征点的跟踪位置，无需通过多次线性插值算法获取当前视频帧中每个目标特征点的像素坐标，加快了跟踪速率，满足视频跟踪的实时性要求，解决了跟踪算法中线性插值运算的开销较大以及跟踪成功率较低的问题，降低了视频跟踪中的运算量，提高了视频跟踪的准确性。
实施例二
图2A为本申请实施例二提供的另一种视频跟踪的方法的流程图。本实施例以上述实施例为基础进行说明。如图2B所示，本实施例中对于确定关联特征点在下一视频帧中的跟踪位置的迭代过程进行说明。
如图2A所示，本实施例中可以包括S210-S270。
S210，获取当前视频帧中目标特征点的关联特征点，该关联特征点为与目标特征点邻近的具有整数像素坐标的像素点。
S220，根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定关联特征点在下一视频帧中的初始跟踪位置。
本实施例中，第一跟踪区域为当前视频帧中以关联特征点为中心的区域，第二跟踪区域内为下一视频帧中与第一跟踪区域对应的区域。一实施例中，由于相邻两个视频帧之间的时变非常小，可以设定连续相邻的两个视频帧中以目标特征点为中心的设定区域的形变很小，与目标特征点邻近的关联特征点也在该设定区域内，因此在对关联特征点进行跟踪时，需要设定该关联特征点的跟踪区域。
一实施例中，首先确定当前视频帧中的第一跟踪区域，如图2B所示，在当前视频帧中获取到关联特征点时，以该关联特征点为中心，确定能够包含一定像素范围内的像素点的矩形框作为第一跟踪区域，该第一跟踪区域内的多个像素点均与关联特征点相隔一定的整数像素坐标。本实施例中，设定关联特征点为
Figure PCTCN2019106852-appb-000007
则该关联特征点附近的多个像素点，也就是第一跟踪区域内的多个像素点为
Figure PCTCN2019106852-appb-000008
该公式中，m∈[-L,L]，n∈[-L,L]，且m、n 及L均为整数，L可以表示设定的第一跟踪区域的大小；第一跟踪区域为当前视频帧中以
Figure PCTCN2019106852-appb-000009
为中心，边长为2*L+1的正方形区域。
一实施例中，在根据上述实施例一中的公式(1)和公式(2)计算该关联特征点在下一视频帧中的初始跟踪位置时，首先需要计算当前视频帧中第一跟踪区域内多个像素点的像素灰度，确定公式(1)中的A中的多个像素点对像素坐标变量x、y的偏导；以及通过在下一视频帧中预先确定第二跟踪区域，确定公式(1)中的b中在不同视频帧中的跟踪区域内多个像素点的像素灰度对时间变量t的偏导，得到该关联特征点从当前视频帧移动到下一视频帧的移动速度，从而根据相邻两帧的时间差确定关联特征点在下一视频帧中的初始跟踪位置。
本实施例中在首次对关联特征点进行跟踪时，可以设定下一视频帧中的第二跟踪区域为在下一视频帧中与当前视频帧中第一跟踪区域对应的区域，在后续迭代过程中，依次根据得到的关联特征点在下一视频帧中的初始跟踪位置重新确定下一视频帧中对应的第二跟踪区域，以实现LK光流跟踪算法在迭代过程中的收敛。
S230，根据关联特征点在下一视频帧中的预设位置以及初始跟踪位置确定关联特征点的当前位移。
本实施例中，对关联特征点在下一视频帧中的跟踪位置进行迭代的过程中，需要预先在下一视频帧中预估一个关联特征点的大致位置作为预设位置，在首次得到该关联特征点的初始跟踪位置后，通过将该初始跟踪位置与该预设位置进行比对，判断两者之间存在的当前位移，根据该当前位移确定本次得到的初始跟踪位置是否收敛，以得到准确的跟踪位置。
一实施例中，在首次进行关联特征点的跟踪时，对于关联特征点在下一视频帧中预设位置的确定，可以直接将该关联特征点在当前视频帧中的位置作为下一视频帧中的预设位置；也可以计算前几个视频帧中该关联特征点的移动速度，根据该移动速度和相邻两帧之间的时间差，以及在当前视频帧中的位置得到一个预估位置，作为下一视频帧中的预设位置；但是当目标物体运动幅度较大，或者进行不规则运动时，上述两种方法确定的预设位置可能和该关联特征点在下一视频帧中的实际位置相差较远，导致跟踪失败或者需要较多次迭代才收敛，增大了计算开销。因此，本实施例中视频跟踪的方法还可以结合目标定位算法，该关联特征点在下一视频帧中的预设位置可以通过目标定位算法确定，也就是在下一视频帧中采用目标定位算法得到关联特征点在进行首次迭代时的在下一视频帧中的预设位置，确定的预设位置能较好的落在关联特征点在下一视频帧中的真实位置附近，在此基础上进行迭代，依次逼近关联特征点在下一视频帧中的实际位置，能够使迭代过程较快达到收敛，提高视频跟踪速率。
在后续的迭代过程中，依次将上一次确定的初始跟踪位置作为新的预设位置，与重新得到的初始跟踪位置进行比对，判断是否达到收敛。
S240，判断当前位移是否超出预设阈值，若当前位移超出预设阈值，执行S250；若当前位移未超出预设阈值，执行S260。
本实施例中，在每次迭代过程中根据预设位置和初始跟踪位置得到本次迭代的当前位移时，需要判断该当前位移是否超出对应的预设阈值，以确定本次迭代是否收敛。该预设阈值为预先设定的能够保证得到的初始跟踪位置与关联特征点在下一视频帧中的实际位置最为接近的目标值。在当前位移超出预设阈值时，说明本次迭代还未收敛，本次的初始跟踪位置与真实位置存在一定差距；在当前位移未超出预设阈值时，说明本次迭代已经收敛，本次的初始跟踪位置已经非常逼近该关联特征点在下一视频帧中的真实位置，无需再次进行迭代。
S250，将初始跟踪位置作为新的预设位置，并以初始跟踪位置为中心，在下一视频帧中确定新的第二跟踪区域，返回执行S220，得到新的初始跟踪位置。
一实施例中，在当前位移超出预设阈值时，说明本次迭代还未收敛，本次得到的初始跟踪位置与真实位置还存在一定差距，需要继续在下一视频帧中重新确定新的初始跟踪位置；将本次得到的初始跟踪位置作为下一次迭代过程中新的预设位置，并以本次得到的初始跟踪位置为中心，以预设的2*L+1为边长再次在下一视频帧中重新确定一个新的第二跟踪区域，返回执行S220，继续根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中新的第二跟踪区域内的多个像素点的像素灰度，重新确定该关联特征点在下一视频帧中新的初始跟踪位置，进行再一次的迭代处理。如图2B所示，若多次迭代过程中得到的初始跟踪位置为
Figure PCTCN2019106852-appb-000010
到
Figure PCTCN2019106852-appb-000011
(n为迭代次数)，后续每次迭代过程中，依次以上一次迭代得到的初始跟踪位置
Figure PCTCN2019106852-appb-000012
为中心，以2*L+1为边长，在下一视频帧中重新确定新的第二跟踪区域，再次计算关联特征点
Figure PCTCN2019106852-appb-000013
在下一视频帧中新的初始跟踪位置
Figure PCTCN2019106852-appb-000014
由于新的第二跟踪区域内的中心点，也就是上一次得到的初始跟踪位置可能为非整数像素坐标，因此在确定下一视频帧中新的第二跟踪区域内的多个像素点的像素灰度时，首先需要通过线性插值算法在下一视频帧中确定新的第二跟踪区域内多个像素点的像素坐标，从而得到多个像素点的像素灰度。
一实施例中，在得到新的初始跟踪位置后，继续将该新的初始跟踪位置与新的预设位置进行比对，得到本次迭代中的当前位移，再次判断当前位移是否还是超出预设阈值；依次循环，直至当前位移未超出该预设阈值，完成迭代过程。
S260，将新的初始跟踪位置作为关联特征点在下一视频帧中的跟踪位置。
一实施例中，在当前位移未超出预设阈值时，说明本次迭代已经收敛，本次的初始跟踪位置已经非常逼近该关联特征点在下一视频帧中的真实位置，无需再次进行迭代；因此，可以将本次迭代中重新得到的初始跟踪位置作为最终确定的该关联特征点在下一视频帧中的跟踪位置。
S270，根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。
本实施例提供的技术方案，通过确定关联特征点在下一视频帧中的跟踪位置，而关联特征点具有整数像素坐标，可以保证参与迭代过程的当前视频帧中第一跟踪区域内的多个像素点均具有整数像素坐标，无需通过多次线性插值算法获取当前视频帧中第一跟踪区域内多个像素点的像素坐标，加快了跟踪速率，满足视频跟踪的实时性要求，降低了视频跟踪中的运算量，提高了视频跟踪的准确性。
实施例三
图3为本申请实施例三提供的另一种视频跟踪的方法的流程图。本实施例以上述实施例为基础。本实施例中在对当前视频帧中的目标特征点进行跟踪的过程中，还可以在不额外增加计算量的前提下，确定本次视频跟踪的跟踪置信度，给出相对准确的跟踪效果评估。
如图3所示，本实施例中可以包括310-S360。
S310，获取当前视频帧中目标特征点的关联特征点，该关联特征点为与目标特征点邻近的具有整数像素坐标的像素点。
S320，根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定关联特征点在下一视频帧中的初始跟踪位置。
S330，根据关联特征点在下一视频帧中的预设位置以及初始跟踪位置确定关联特征点的当前位移。
S340，在所当前位移超出预设阈的情况下，将初始跟踪位置作为新的预设位置，并以初始跟踪位置为中心，在下一视频帧中确定新的第二跟踪区域，得到新的初始跟踪位置，直至当前位移未超出预设阈值，将新的初始跟踪位置作为关联特征点在下一视频帧中的跟踪位置。
S350，根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。
S360，根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内多个像素点的像素灰度，确定本次视频跟踪的跟踪置信度。
一实施例中，本实施例中在确定关联特征点在下一视频帧中的跟踪位置时，在每次迭代过程中会根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内多个像素点的像素灰度，确定每次迭代过程中多个像素点的像素灰度对时间变量t的偏导，也就是本申请实施例一种提及的公式(1)的b中的多个项偏导值，为
Figure PCTCN2019106852-appb-000015
该偏导可以表示在不同视频帧的跟踪区域内的同一像素点上的像素灰度的差值大小，也就是判断不同视频帧的跟踪区域内视频画面的相似程度，本实施例中在将新的初始跟踪位置作为关联特征点在下一视频帧中的跟踪位置之后，可以通过判断当前视频帧中第一跟踪区域中多个像素点的像素灰度与最后一次迭代过程中确定的下一视频帧中新的第二跟踪区域中多个像素点的像素灰度，也就是确定最后一次迭代过程中跟踪区域内多个像素点的像素灰度对时间变量t的偏导，来确定本次视频跟踪的跟踪置信度。
一实施例中，本实施例中确定的跟踪置信度为：Score＝exp(-var(I t(q))/255)；该公式中，Score为本次视频跟踪的跟踪置信度，var(I t(q))为对多个像素点的像素灰度对时间变量t的偏导求取的方差；当最后一次迭代中在下一视频帧中确定的新的第二跟踪区域和当前视频帧中第一跟踪区域内的视频画面完全一致时，var(I t(q))＝0，Score＝1，而当最后一次迭代中在下一视频帧中确定的新的第二跟踪区域和当前视频帧中第一跟踪区域内的视频画面不相关，也就是第二跟踪区域和第一跟踪区域内的视频画面差异较大时，会使var(I t(q))很大，Score趋向于0；本实施例中的跟踪置信度能够较为准确的反映不同视频帧中跟踪区域的相似程度，且由于LK光流跟踪算法本身就需要计算I t(q)，计算跟踪置信度的操作只在最后一次迭代时进行，保证增加的运算开销非常小，因此本实施例中的跟踪置信度能够有效的判断出本次视频跟踪的准确性。
此外，若根据本次视频跟踪的跟踪置信度判断本次未跟踪上对应的目标时，可以直接在该视频帧中采用目标定位算法重新计算目标特征点的位置，本实施例中视频跟踪的方法可以和特定的目标定位算法融合使用，抑制目标定位算法在视频连续帧中定位结果的抖动，且计算量低，资源占用少，大幅改善视频中 对目标物体定位结果的稳定性。
本实施例提供的技术方案，在对当前视频帧中的目标特征点进行跟踪的基础上，通过对跟踪过程中存在的中间计算结果进行分析，确定本次视频跟踪的跟踪置信度，不额外增加计算开销，并对本次视频跟踪的效果进行客观评价，有效判断本次视频跟踪的准确性。
实施例四
图4为本申请实施例四提供的一种视频跟踪的装置的结构示意图，如图4所示，该装置可以包括：
关联点获取模块410，设置为获取当前视频帧中目标特征点的关联特征点，该关联特征点为与目标特征点邻近的具有整数像素坐标的像素点。
关联位置确定模块420，设置为确定关联特征点在下一视频帧中的跟踪位置。
目标位置确定模块430，设置为根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。
本实施例提供的技术方案，将当前视频帧中与目标特征点临近的具有整数像素坐标的关联特征点作为中间参考量，确定该关联特征点在下一视频帧中的跟踪位置，进而通过当前视频帧中两者的位置差，确定下一视频帧中目标特征点的跟踪位置，无需通过多次线性插值算法获取当前视频帧中每个目标特征点的像素坐标，加快了跟踪速率，满足视频跟踪的实时性要求，解决了跟踪算法中线性插值运算的开销较大以及跟踪成功率较低的问题，降低了视频跟踪中的运算量，提高了视频跟踪的准确性。
一实施例中，关联位置确定模块420，是设置为：根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定关联特征点在下一视频帧中的初始跟踪位置，其中，第一跟踪区域为当前视频帧中以关联特征点为中心的区域，第二跟踪区域内为下一视频帧中与第一跟踪区域对应的区域；根据关联特征点在下一视频帧中的预设位置以及初始跟踪位置确定关联特征点的当前位移；在当前位移超出预设阈值的情况下，将初始跟踪位置作为新的预设位置，并以初始跟踪位置为中心，在下一视频帧中确定新的第二跟踪区域，并返回执行所述根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定所述关联特征点在所述下一视频帧中的初始跟 踪位置，得到新的初始跟踪位置，直至当前位移未超出预设阈值，将新的初始跟踪位置作为关联特征点在下一视频帧中的跟踪位置。
一实施例中，在初始跟踪位置的像素坐标为非整数像素坐标的情况下，下一视频帧中第二跟踪区域内多个像素点的像素坐标可以通过线性插值算法确定。
一实施例中，关联特征点在下一视频帧中的预设位置可以通过目标定位算法确定。
一实施例中，视频跟踪的装置，还可以包括：置信度确定模块，设置为在将新的初始跟踪位置作为关联特征点在下一视频帧中的跟踪位置之后，根据当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内多个像素点的像素灰度，确定本次视频跟踪的跟踪置信度。
本实施例提供的视频跟踪的装置可适用于上述任意实施例提供的视频跟踪的方法，具备相应的功能和有益效果。
实施例五
图5为本申请实施例五提供的一种设备的结构示意图，如图5所示，该设备包括处理器50、存储装置51、通信装置52、输入装置53和输出装置54；设备中处理器50的数量可以是一个或多个，图5中以一个处理器50为例；设备中的处理器50、存储装置51、通信装置52、输入装置53和输出装置54可以通过总线或其他方式连接，图5中以通过总线连接为例。
存储装置51作为一种计算机可读存储介质，可设置为存储软件程序、计算机可执行程序以及模块，如本申请任意实施例所述的视频跟踪的方法对应的程序指令/模块。处理器50通过运行存储在存储装置51中的软件程序、指令以及模块，从而执行设备的至少一种功能应用以及数据处理，即实现上述视频跟踪的方法。
存储装置51可主要包括存储程序区和存储数据区，存储装置51中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端的使用所创建的数据等。此外，存储装置51可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储装置51可包括相对于处理器50远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
通信装置52可设置为实现设备间的网络连接或者移动数据连接。
输入装置53可设置为接收输入的数字或字符信息，以及产生与设备的用户设置以及功能控制有关的键信号输入。
输出装置54可包括显示屏等显示设备。
本实施例提供的一种设备可设置为执行上述任意实施例提供的视频跟踪的方法，具备相应的功能和有益效果。
实施例六
本申请实施例六还提供了一种计算机可读存储介质，存储有计算机程序，该程序被处理器执行时可实现上述任意实施例中的视频跟踪的方法。该方法可以包括：获取当前视频帧中目标特征点的关联特征点，该关联特征点为与目标特征点邻近的具有整数像素坐标的像素点；确定关联特征点在下一视频帧中的跟踪位置；根据关联特征点与目标特征点在当前视频帧中的位置差，以及关联特征点在下一视频帧中的跟踪位置，确定目标特征点在下一视频帧中的跟踪位置。
本申请实施例所提供的一种包含计算机可执行指令的存储介质，该计算机可执行指令不限于如上所述的方法操作，还可以执行本申请任意实施例所提供的视频跟踪的方法中的相关操作。
通过以上关于实施方式的描述，所属领域的技术人员可以了解到，本申请可借助软件及通用硬件来实现，也可以通过硬件实现，。本申请的技术方案可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、闪存(FLASH)、硬盘或光盘等，包括多个指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请任意实施例所述的方法。
上述视频跟踪的装置的实施例中，所包括的单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，每个功能单元的名称也只是为了便于相互区分，并不用于限制本申请的保护范围。

Claims (10)

  1. 一种视频跟踪的方法，包括：
    获取当前视频帧中目标特征点的关联特征点，所述关联特征点为与所述目标特征点邻近的具有整数像素坐标的像素点；
    确定所述关联特征点在下一视频帧中的跟踪位置；
    根据所述关联特征点与所述目标特征点在所述当前视频帧中的位置差，以及所述关联特征点在所述下一视频帧中的跟踪位置，确定所述目标特征点在所述下一视频帧中的跟踪位置。
  2. 根据权利要求1所述的方法，其中，所述确定所述关联特征点在下一视频帧中的跟踪位置，包括：
    根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定所述关联特征点在所述下一视频帧中的初始跟踪位置，其中，所述第一跟踪区域为所述当前视频帧中以所述关联特征点为中心的区域，所述第二跟踪区域内为所述下一视频帧中与所述第一跟踪区域对应的区域；
    根据所述关联特征点在所述下一视频帧中的预设位置以及所述初始跟踪位置确定所述关联特征点的当前位移；
    在所述当前位移超出预设阈值的情况下，将所述初始跟踪位置作为新的预设位置，并以所述初始跟踪位置为中心，在所述下一视频帧中确定新的第二跟踪区域，并返回执行所述根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定所述关联特征点在所述下一视频帧中的初始跟踪位置，得到新的初始跟踪位置，直至所述当前位移未超出所述预设阈值，将所述新的初始跟踪位置作为所述关联特征点在所述下一视频帧中的跟踪位置。
  3. 根据权利要求2所述的方法，其中，在所述初始跟踪位置的像素坐标为非整数像素坐标的情况下，所述下一视频帧中第二跟踪区域内多个像素点的像素坐标通过线性插值算法确定。
  4. 根据权利要求2或3所述的方法，其中，所述关联特征点在所述下一视频帧中的预设位置通过目标定位算法确定。
  5. 根据权利要求2所述的方法，在所述将所述新的初始跟踪位置作为所述关联特征点在所述下一视频帧中的跟踪位置之后，还包括：
    根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及所述下一视频帧中第二跟踪区域内多个像素点的像素灰度，确定本次视频跟踪的跟 踪置信度。
  6. 一种视频跟踪的装置，包括：
    关联点获取模块，设置为获取当前视频帧中目标特征点的关联特征点，所述关联特征点为与所述目标特征点邻近的具有整数像素坐标的像素点；
    关联位置确定模块，设置为确定所述关联特征点在下一视频帧中的跟踪位置；
    目标位置确定模块，设置为根据所述关联特征点与所述目标特征点在所述当前视频帧中的位置差，以及所述关联特征点在所述下一视频帧中的跟踪位置，确定所述目标特征点在所述下一视频帧中的跟踪位置。
  7. 根据权利要求6所述的装置，其中，所述关联位置确定模块是设置为：
    根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定所述关联特征点在所述下一视频帧中的初始跟踪位置，其中，所述第一跟踪区域为所述当前视频帧中以所述关联特征点为中心的区域，所述第二跟踪区域内为所述下一视频帧中与所述第一跟踪区域对应的区域；
    根据所述关联特征点在所述下一视频帧中的预设位置以及所述初始跟踪位置确定所述关联特征点的当前位移；
    在所述当前位移超出预设阈值的情况下，将所述初始跟踪位置作为新的预设位置，并以所述初始跟踪位置为中心，在所述下一视频帧中确定新的第二跟踪区域，并返回执行所述根据所述当前视频帧中第一跟踪区域内多个像素点的像素灰度，以及下一视频帧中第二跟踪区域内的多个像素点的像素灰度，确定所述关联特征点在所述下一视频帧中的初始跟踪位置，得到新的初始跟踪位置，直至所述当前位移未超出所述预设阈值，将所述新的初始跟踪位置作为所述关联特征点在所述下一视频帧中的跟踪位置。
  8. 根据权利要求7所述的装置，其中，在所述初始跟踪位置的像素坐标为非整数像素坐标的情况下，所述下一视频帧中第二跟踪区域内多个像素点的像素坐标通过线性插值算法确定。
  9. 一种设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-5中任一所述的视频跟踪的方法。
  10. 一种计算机可读存储介质，存储有计算机程序，所述程序被处理器执行时实现如权利要求1-5中任一所述的视频跟踪的方法。
PCT/CN2019/106852 2018-11-29 2019-09-20 视频跟踪的方法、装置、设备和存储介质 Ceased WO2020108037A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811447836.0 2018-11-29
CN201811447836.0A CN109598744B (zh) 2018-11-29 2018-11-29 一种视频跟踪的方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2020108037A1 true WO2020108037A1 (zh) 2020-06-04

Family

ID=65959325

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/106852 Ceased WO2020108037A1 (zh) 2018-11-29 2019-09-20 视频跟踪的方法、装置、设备和存储介质

Country Status (2)

Country Link
CN (1) CN109598744B (zh)
WO (1) WO2020108037A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4641498A1 (en) * 2024-04-23 2025-10-29 The Boeing Company Real-time error monitor of receiver aircraft based on machine learning keypoint solutions for automated aerial refueling

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109598744B (zh) * 2018-11-29 2020-12-08 广州市百果园信息技术有限公司 一种视频跟踪的方法、装置、设备和存储介质
CN110198409B (zh) * 2019-05-30 2021-07-09 Oppo广东移动通信有限公司 终端的控制方法及控制装置、终端及计算机可读存储介质
CN110654983A (zh) * 2019-09-16 2020-01-07 广东马上到网络科技有限公司 一种利用视频数据测算制动距离的起重机制动报警方法
CN111093077A (zh) * 2019-12-31 2020-05-01 深圳云天励飞技术有限公司 一种视频编码方法、装置、电子设备及存储介质
CN114095780B (zh) * 2020-08-03 2025-07-18 影石创新科技股份有限公司 一种全景视频剪辑方法、装置、存储介质及设备
CN112492196B (zh) * 2020-10-29 2022-01-07 贝壳技术有限公司 一种直播进行中主播跟踪方法、装置及系统
CN112488911B (zh) * 2020-11-26 2023-04-14 四三九九网络股份有限公司 在html5画布canvas上渲染gif文件的方法
CN112907652B (zh) * 2021-01-25 2024-02-02 脸萌有限公司 相机姿态获取方法、视频处理方法、显示设备和存储介质
CN115082516B (zh) * 2021-03-15 2025-04-29 北京字跳网络技术有限公司 一种目标跟踪方法、装置、设备及介质
CN115082515A (zh) * 2021-03-15 2022-09-20 北京字跳网络技术有限公司 一种目标跟踪方法、装置、设备及介质
CN113518214B (zh) * 2021-05-25 2022-03-15 上海哔哩哔哩科技有限公司 全景视频数据处理方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002075660A2 (en) * 2001-03-21 2002-09-26 Consiglio Nazionale Delle Ricerche Method and apparatus for real time contour tracking of objects on video images sequences
CN107818573A (zh) * 2016-09-12 2018-03-20 杭州海康威视数字技术股份有限公司 一种目标跟踪方法及装置
CN108898619A (zh) * 2018-06-08 2018-11-27 上海大学 一种基于pvanet神经网络的目标跟踪方法
CN109598744A (zh) * 2018-11-29 2019-04-09 广州市百果园信息技术有限公司 一种视频跟踪的方法、装置、设备和存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104599286B (zh) * 2013-10-31 2018-11-16 展讯通信(天津)有限公司 一种基于光流的特征跟踪方法及装置
CN105323420B (zh) * 2014-07-29 2019-02-22 腾讯科技（深圳）有限公司 视频图像处理方法及装置
US10268901B2 (en) * 2015-12-04 2019-04-23 Texas Instruments Incorporated Quasi-parametric optical flow estimation
WO2018058476A1 (zh) * 2016-09-29 2018-04-05 华为技术有限公司 一种图像校正方法及装置
CN107633247A (zh) * 2017-08-16 2018-01-26 歌尔股份有限公司 图像区域的确定方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002075660A2 (en) * 2001-03-21 2002-09-26 Consiglio Nazionale Delle Ricerche Method and apparatus for real time contour tracking of objects on video images sequences
CN107818573A (zh) * 2016-09-12 2018-03-20 杭州海康威视数字技术股份有限公司 一种目标跟踪方法及装置
CN108898619A (zh) * 2018-06-08 2018-11-27 上海大学 一种基于pvanet神经网络的目标跟踪方法
CN109598744A (zh) * 2018-11-29 2019-04-09 广州市百果园信息技术有限公司 一种视频跟踪的方法、装置、设备和存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4641498A1 (en) * 2024-04-23 2025-10-29 The Boeing Company Real-time error monitor of receiver aircraft based on machine learning keypoint solutions for automated aerial refueling
US12612177B2 (en) 2024-04-23 2026-04-28 The Boeing Company Real-time error monitor of receiver aircraft based on machine learning keypoint solutions for automated aerial refueling

Also Published As

Publication number Publication date
CN109598744B (zh) 2020-12-08
CN109598744A (zh) 2019-04-09

Similar Documents

Publication Publication Date Title
WO2020108037A1 (zh) 视频跟踪的方法、装置、设备和存储介质
CN108492287B (zh) 一种视频抖动检测方法、终端设备及存储介质
US9811921B2 (en) Apparatus and method for processing a depth image
US11514676B2 (en) Method and apparatus for detecting region of interest in video, device and medium
WO2021232652A1 (zh) 目标跟踪方法、装置、电子设备及计算机可读存储介质
US20150253864A1 (en) Image Processor Comprising Gesture Recognition System with Finger Detection and Tracking Functionality
CN110631554A (zh) 机器人位姿的确定方法、装置、机器人和可读存储介质
KR102432561B1 (ko) 에지를 기반으로 하는 증강현실 3차원 추적 등록 방법, 장치 및 전자 기기
CN110147750B (zh) 一种基于运动加速度的图像搜索方法、系统及电子设备
JP2016526713A (ja) トラッカ補助画像キャプチャ
US11935214B2 (en) Video content removal using flow-guided adaptive learning
CN111462174A (zh) 多目标跟踪方法、装置以及电子设备
CN113436226B (zh) 关键点检测的方法和装置
KR20210093820A (ko) 고해상도 객체 검출장치 및 방법
JP2022160646A (ja) ビデオジッタの検出方法、装置、電子機器および記憶媒体
CN112381071A (zh) 一种视频流中目标的行为分析方法、终端设备及介质
CN111583280B (zh) 图像处理方法、装置、设备及计算机可读存储介质
CN103679130B (zh) 手追踪方法、手追踪设备和手势识别系统
US20200134323A1 (en) Information processing apparatus, control method, and program
CN108961316A (zh) 图像处理方法、装置及服务器
CN113766117A (zh) 一种视频去抖动方法和装置
CN112085002A (zh) 人像分割方法、装置、存储介质及电子设备
CN113965697A (zh) 基于连续帧信息的视差成像方法、电子设备及存储介质
JP5643147B2 (ja) 動きベクトル検出装置、動きベクトル検出方法及び動きベクトル検出プログラム
JPWO2018179119A1 (ja) 映像解析装置、映像解析方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19891545

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19891545

Country of ref document: EP

Kind code of ref document: A1