WO2020014901A1 - 一种目标跟踪方法、装置、电子设备和可读存储介质 - Google Patents

一种目标跟踪方法、装置、电子设备和可读存储介质 Download PDF

Info

Publication number
WO2020014901A1
WO2020014901A1 PCT/CN2018/096161 CN2018096161W WO2020014901A1 WO 2020014901 A1 WO2020014901 A1 WO 2020014901A1 CN 2018096161 W CN2018096161 W CN 2018096161W WO 2020014901 A1 WO2020014901 A1 WO 2020014901A1
Authority
WO
WIPO (PCT)
Prior art keywords
video image
image
target
level
resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/096161
Other languages
English (en)
French (fr)
Inventor
杨文超
王恺
廉士国
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cloudminds Shenzhen Robotics Systems Co Ltd
Original Assignee
Cloudminds Shenzhen Robotics Systems Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cloudminds Shenzhen Robotics Systems Co Ltd filed Critical Cloudminds Shenzhen Robotics Systems Co Ltd
Priority to CN201880001302.7A priority Critical patent/CN109074657B/zh
Priority to PCT/CN2018/096161 priority patent/WO2020014901A1/zh
Publication of WO2020014901A1 publication Critical patent/WO2020014901A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present application relates to the field of computer vision, and in particular, to a method, an apparatus, an electronic device, and a readable storage medium for tracking an object.
  • Augmented Reality is a technology that calculates the position and angle of the camera image in real time and adds the corresponding image.
  • the goal of this technology is to put the virtual world on the screen and perform interactive.
  • Target detection and target tracking are key technologies in augmented reality.
  • Target detection can detect the precise position of the template map (flat target) in the video image, but the calculation of target detection is very time-consuming.
  • the initial position of the template image in the video image is usually obtained through target detection, and the precise position of the template image in the video image is subsequently determined by the target tracking method.
  • the general process of the target tracking method is: extract the feature points of the template map and search for the feature points in the image to be matched to obtain the homography matrix from the template map to the image to be matched, so as to determine that the template map is to be matched. Precise position in the image; according to the determined precise position and the historical position of the template image, predict the position of the template image in the next image to be matched, and continue to track the template image according to the predicted position.
  • the inventors discovered during the research of the prior art that currently, the following two methods are usually used for tracking a high-resolution image of a planar target: First, a large search radius is used to search for feature points of a planar target, but this method is time-consuming Serious; Second, the method uses a small search radius to search for feature points of a planar target, but this method is prone to search failures, which leads to the failure of tracking flat targets.
  • the plane target is tracked in real time, if the plane target moves relatively fast relative to the camera, it will often lead to the loss of tracking of the plane target or the phenomenon of jitter in the superimposed image, which reduces the User experience on AR.
  • a technical problem to be solved in some embodiments of the present application is to provide a method, an apparatus, an electronic device, and a readable storage medium for target tracking, so that when a target in a video image is tracked in real time, the target can be quickly and accurately realized.
  • the positioning position in the video image improves the user's experience of AR.
  • An embodiment of the present application provides a target tracking method, including: acquiring a frame of video image and acquiring a predicted position of the target in the video image; acquiring a reduced template image of the target; and according to the reduced template image and the predicted position, Determine the reduction ratio of the video image, and reduce the video image according to the reduction ratio to obtain a reduced video image; determine the predicted position of the target in the reduced video image according to the predicted position; use the predicted position of the target in the reduced video image, use The reduced template image is matched with the reduced video image to determine the rough positioning position information of the reduced template image in the reduced video image; the precise positioning position of the target in the video image is determined based on the rough positioning position information.
  • An embodiment of the present application further provides a target tracking device, including: a first acquisition module, a second acquisition module, an image reduction module, a predicted position reduction module, a coarse positioning module, and an accurate positioning module; the first acquisition module is used for Acquire a frame of video image, and obtain the predicted position of the target in the video image; the second acquisition module is used to acquire the reduced template image of the target; the image reduction module is used to determine the video image based on the reduced template image and the predicted position.
  • the prediction position reduction module is used to determine the predicted position of the target in the reduced video image according to the predicted position
  • the coarse positioning module is used to determine the target in the reduced video
  • the predicted position in the image is matched with the reduced template image and the reduced video image to determine the information of the rough positioning position of the reduced template image in the reduced video image
  • the precise positioning module is configured to determine the position based on the rough positioning Information to determine the target in the video map The precise positioning location.
  • An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are at least One processor executes to enable at least one processor to perform the above-mentioned target tracking method.
  • An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which is implemented by the processor to implement the foregoing target tracking method.
  • the target template image and the obtained video image are reduced, and the resolution of the template image and the video image is greatly reduced, so that the template image and the video image are being processed.
  • it can quickly determine the coarse positioning position information of the reduced template image in the reduced video image; the coarse positioning position information makes the process of determining the precise positioning position of the target in the video image reduced.
  • Searching the scope of the template image in the video image greatly improves the speed of determining the precise positioning position and shortens the time to determine the precise positioning position of the target in the video image; and in the case of an error in the predicted position, due to the reduced template
  • the low resolution and small image size of images and reduced video images make it possible to quickly determine the information of the rough positioning position without changing the scope of the search template image, so as not to lose the target template image, which improves the Accuracy of target tracking to improve user experience on AR .
  • FIG. 1 is a specific flowchart of a target tracking method in the first embodiment of the present application
  • FIG. 2 is a schematic flowchart of a specific process for determining a precise positioning position of a target in a video image in a target tracking method according to a second embodiment of the present application;
  • FIG. 3 is a detailed flowchart of sub-pixel processing in a target tracking method in a third embodiment of the present application.
  • FIG. 4 is a schematic diagram of a specific structure of a target tracking device in a fourth embodiment of the present application.
  • FIG. 5 is a detailed structural diagram of an electronic device in a fifth embodiment of the present application.
  • the first embodiment of the present application relates to a target tracking method.
  • the target tracking method is suitable for a terminal, especially a mobile terminal, such as a smart phone, a smart tablet, and the like.
  • the specific process of the target tracking method is shown in Figure 1:
  • Step 101 Obtain a frame of video image, and obtain a predicted position of a target in the video image.
  • the video image is acquired by the terminal through a camera, for example, a smartphone is acquired by a camera, and a head-mounted device is acquired by a built-in camera or an external camera.
  • the target tracking in AR is achieved by tracking the targets in continuous frame images.
  • a target in a video image refers to a two-dimensional object in the video image (such as an image of a potted plant, an image of a lamp).
  • the way to obtain the predicted position of the target in the video image can be obtained by the target in the previous frame of the video image. Obtained from the position estimation; of course, the predicted position can also be obtained by a detection method. It should be noted that, for the first frame image, the predicted position of the target in the video image is obtained by a detection method, and the detection method is not described in detail in this embodiment.
  • Step 102 Obtain a reduced template image of the target.
  • the reduced template image is a reduced template image of a pre-stored target; or, the reduced template image is obtained by reducing the template image of the target according to a preset resolution.
  • the template image of the target refers to a two-dimensional image of the target.
  • the template image of a target may be an image of a lamp, an image of a pot, and the like.
  • the template image can be obtained from a cloud server, or it can be acquired by the terminal itself.
  • the method of obtaining the template image is not limited, and can be selected according to actual needs.
  • the reduced template image may be stored in the terminal in a fixed resolution in advance. Since the reduced template image is directly obtained, the speed of acquiring the reduced template image is greatly mentioned.
  • the full-resolution template image may be reduced according to a preset resolution.
  • the preset resolution is 25 ⁇ 25 pixels
  • the full resolution of the template image is 720 ⁇ 720. Pixels, the template image at full frequency is reduced to 25 ⁇ 25 pixels to obtain a reduced template image.
  • Step 103 Determine a reduction ratio of the video image according to the reduced template image and the predicted position, and reduce the video image according to the reduction ratio to obtain a reduced video image.
  • the estimated pixel area of the template image in the video image is determined according to the predicted position; the reduction ratio of the video image is determined according to the pixel area and the estimated pixel area of the reduced template image.
  • the predicted position is the predicted position of the template image in the video image.
  • the estimated pixel area of the template image in the video image can be calculated, and the pixel area of the reduced template image is calculated.
  • the reduction ratio of the video image is equal to The pixel area of the reduced template image divided by the square root of the quotient of the estimated pixel area.
  • the pixel area of the reduced template image is sm
  • the estimated pixel area of the full-resolution template image in the video image is sf.
  • the reduction ratio t is equal to the square root of (sm / sf).
  • the method of calculating the reduction ratio is not limited in this embodiment.
  • Step 104 Determine the predicted position of the target in the reduced video image according to the predicted position.
  • the predicted position of the target in the video image is reduced according to the reduction ratio of the reduced video image to obtain the predicted position of the target in the reduced video image.
  • Step 105 According to the predicted position of the target in the reduced video image, use the reduced template image to match the reduced video image, and determine information about the rough positioning position of the reduced template image in the reduced video image.
  • the reduced video image is searched for image blocks that match the reduced template image, and the rough positioning position information is determined according to the matched image blocks.
  • feature points are extracted from the reduced template image, and image blocks to be matched on the reduced template image are determined according to the feature points, and then the predicted position of the reduced template image in the reduced video image is used as a starting point.
  • search for the image block matching the reduced template image in the reduced video image with a preset radius can be determined according to the resolution of the reduced video image, which is not limited in this embodiment.
  • the matched image block in the reduced video image is an image containing the reduced template image feature points, and the size of the image block is the same as the size of the image block to be matched on the reduced template image.
  • the image blocks found in the reduced video image that match the reduced template image are the same image as the image blocks in the template image by default, so according to the feature points in the searched image block and the corresponding features in the reduced template image Point to determine the homography matrix of the reduced template image mapped to the reduced video image.
  • the position of the image block that is searched and matched with the reduced template image can be used as information to determine the rough positioning position of the reduced template image in the reduced video image; the determined reduced template image can also be mapped
  • the homography matrix in the reduced video image is used as information for determining a rough positioning position of the reduced template image in the reduced video image.
  • the reduced template image is already very small, in this embodiment, a small number of feature points are extracted from the reduced template image (for example, 4 feature points are extracted), and then the reduced template image is directly used. As the image blocks to be matched, the position of the image block matching the reduced template image is searched in the reduced video image, that is, the position of the reduced template image in the reduced video image.
  • Step 106 Determine the precise positioning position of the target in the video image according to the information of the rough positioning position.
  • the position of the reduced template image in the reduced video image is used as an example of the rough positioning position information.
  • the initial homography matrix of the reduced template image mapped to the reduced video image can be determined, and the initial homography matrix is mapped as the full-resolution template image to the homography matrix of the full-resolution video image.
  • the initial position of the full-resolution template image in the full-resolution video image is calculated, and according to the initial position of the template image in the full-resolution video image, the full-resolution The template image is matched with the full-resolution video image to determine the position of the full-resolution template image in the full-resolution video image, and this position is used as the precise positioning position of the target in the video image.
  • the matching process is the same as The matching process in step 104 is substantially the same, and is not repeated here.
  • Step 107 Output the precise positioning position.
  • the terminal may superimpose other images on the target in the video image according to the precise positioning position of the output target on the video image of the current frame.
  • the precise positioning position of the target on the video image of the current frame and the precise positioning position of the target on the video image of the historical frame the predicted position of the target in the next frame of video image is predicted.
  • the prediction method is here I will not repeat them here.
  • the target template image and the obtained video image are reduced, and the resolution of the template image and the video image is greatly reduced, so that the template image and the video image are being processed.
  • it can quickly determine the coarse positioning position information of the reduced template image in the reduced video image; the coarse positioning position information makes the process of determining the precise positioning position of the target in the video image reduced.
  • Searching the scope of the template image in the video image greatly improves the speed of determining the precise positioning position and shortens the time to determine the precise positioning position of the target in the video image; and in the case of an error in the predicted position, due to the reduced template
  • the low resolution and small image size of images and reduced video images make it possible to quickly determine the information of the rough positioning position without changing the scope of the search template image, so as not to lose the target template image, which improves the Accuracy of target tracking to improve user experience on AR .
  • the second embodiment of the present application relates to a target tracking method.
  • the second embodiment is a further improvement on the first embodiment.
  • the main improvement lies in that in this embodiment, the target is determined in the video image according to the information of the rough positioning position. Pyramid matching is used in the process of accurate positioning. The specific flow of this process is shown in Figure 2.
  • Step 201 Determine an initial homography matrix that maps the reduced template image to the reduced video image according to the information of the coarse positioning position.
  • this step 201 is substantially the same as the process of determining the homography matrix in step 106 in the first embodiment, that is, by determining the position information of the coarse positioning and the position of the reduced template image, it can be determined that the reduced template image is mapped to The initial homography matrix of the reduced video image.
  • Step 202 Determine the N-level resolution required in the pyramid matching process according to the initial homography matrix, where N is an integer greater than 1.
  • the use context of shooting video images is determined according to the initial homography matrix.
  • the use context includes: the shooting angle and the shooting distance; according to the use context, the N-level resolution required in the pyramid matching process is determined.
  • the posture information of the reduced template image in the reduced video image can be obtained according to the initial homography matrix, and the position of the reduced template image in the reduced video image can be obtained based on the coarse positioning position information.
  • the position in the reduced video image and the posture of the reduced template image in the reduced video image are used to determine the use situation of the captured video image. According to the use situation, the N-level resolution required in the pyramid matching process is determined.
  • the use situation is large-angle shooting, then determine two levels of resolution, the first level resolution is 1/2 resolution, and the second level resolution is full resolution; if the use situation is close range shooting, then Determine three levels of resolution, the first level of resolution is 1/4 resolution, the second level of resolution is 1/2 resolution, and the third level of resolution is full resolution.
  • Step 203 Perform pyramid matching on the video image according to the initial homography matrix and the determined N-level resolution, and determine the precise positioning position of the target in the video image according to the result of the pyramid matching.
  • the video image is scaled according to the N-level resolution to obtain the N-level video image corresponding to the N-level resolution, and the video images at each level are selected and obtained from the pre-stored template images of different resolutions.
  • Corresponding N-level template image the following processing is performed in order of resolution from low to high: according to the initial position corresponding to the i + 1-level resolution, the i + 1-level template image and the i + 1-level video image are processed Match to determine the homography matrix of the i + 1 level template image mapped to the i + 1 level video image, where the first level resolution is the lowest resolution among the N level resolutions and the initial corresponding to the first level resolution
  • the position is determined according to the initial homography matrix, and the initial position corresponding to the i + 1th level resolution is determined according to the homography matrix corresponding to the ith level resolution, N> 1, 1 ⁇ i ⁇ N-1; resolution according to the Nth level
  • the homography matrix corresponding to the rate determines the precise location of the target in the video image.
  • the video image can be pyramid-matched according to the initial homography matrix.
  • a specific example will be used to illustrate the process of pyramid matching.
  • the scene is used for large-angle shooting, determine that the first-level resolution is 1/2 resolution and the second-level resolution is full resolution.
  • the video image is scaled according to the first-level resolution to obtain a 1 / 2-resolution video image.
  • template images with different levels of resolution are stored in advance.
  • the pixel area S1 of the template image in the 1 / 2-resolution video image is calculated.
  • the template image 1 closest to the pixel area S1 is selected from the template images, and the template image 1 is used as the 1 / 2-resolution template image corresponding to the 1 / 2-resolution video image.
  • the second-level resolution is full resolution Rate, the video image does not need to be scaled, and the full-resolution template image is directly selected as the template image corresponding to the full-resolution video image.
  • Process in order of resolution from low to high According to the initial homography matrix H0, map a 1 / 2-resolution template image to a 1 / 2-resolution video image, and obtain a 1 / 2-resolution template image in The initial position in the 1 / 2-resolution video image. Based on the initial position corresponding to the first-level resolution, the first-level template image is matched with the first-level video image to determine that the first-level template image is mapped to the first level.
  • the matching process of the homography matrix H1 of the video image is substantially the same as the matching process in the first embodiment, which will not be repeated here.
  • the Nth level resolution is the full resolution of the video image; if not, according to the homography matrix corresponding to the Nth level resolution
  • the full-resolution template image is matched with the full-resolution video image to obtain the homography matrix corresponding to the full-resolution, and the full-resolution template image is determined based on the full-resolution video in the full-resolution video.
  • the positioning position in the image is used as the precise positioning position of the target in the video image; if it is, the positioning position of the full-resolution template image in the full-resolution video image is determined according to the homography matrix corresponding to the Nth level resolution And as the precise location of the target in the video image.
  • the N-th level resolution is not full resolution in the pyramid matching process, it is necessary to determine the positioning position of the full resolution template image in the full resolution video image.
  • the second-level resolution determines whether the second-level resolution is full resolution. Because it is determined that the second-level resolution is full resolution, you can directly convert the full resolution The positioning position of the resolution template image in the full-resolution video image is used as the precise positioning position of the target in the video image.
  • the process of matching the i + 1th level template image with the i + 1th level video image is: searching for the image block matching the i + 1th level template image in the i + 1th level video image, A homography matrix that maps the i + 1th-level template image to the i + 1th-level video image is determined according to the image block, and during the matching process, some pixels in the image block are used for matching.
  • the process of searching for an image block matching the i + 1 level template image in the i + 1 level video image, and finding a certain number of stable feature points in the i + 1 level template image (such as using Harris Corners to find feature points)
  • the matched image blocks can use a preset shape, Such as circles, rectangles, etc.
  • some pixels in the image block are used for matching. Among them, some of the pixels in the image block are distributed in a m-shaped or X-shape, and are distributed through the m-shaped or X-shaped pixels.
  • the target tracking method provided in this embodiment adopts a pyramid matching method in the process of determining the precise positioning position of a target in a video image, because the pyramid matching method is a hierarchical matching method , First match the low-resolution video image, then match the high-resolution video image, and constantly update the homography matrix that the target maps to the video image, so that the target can be accurately determined in the video image.
  • the full-resolution video image is not directly searched for the full-resolution template image, it starts from the low resolution and is based on the homography matrix obtained from the low-resolution video matching, which can be quickly determined.
  • the initial position of the full-resolution template image in the full-resolution video image is obtained, thereby quickly determining the precise positioning position of the target in the video image.
  • the third embodiment of the present application relates to a target tracking method.
  • the third embodiment is a further improvement of the second embodiment.
  • the main improvement is that if the precise positioning position is a pixel-level coordinate position in this embodiment,
  • the homography matrix corresponding to the N-level resolution determines the precise positioning position of the target in the video image, and then obtains the precise positioning position at the sub-pixel level.
  • the specific process of obtaining the accurate positioning position at the sub-pixel level is shown in Figure 3:
  • Step 301 Obtain a matching degree value of a precise positioning position, where the matching degree value is a similarity value matching a feature point of a target and a feature point of an image block in a video image.
  • the precise location of the determined target in the video image is also the pixel-level coordinate position.
  • the feature points of the target and the characteristics of the image block in the video image may be a sub-pixel. Therefore, in order to improve the accuracy of the precise positioning position of the target in the video image, the pixel level is processed.
  • the matching degree value of the precise location can be obtained during the matching process.
  • Step 302 Perform sub-pixel processing on the precisely-positioned position according to the matching degree value to obtain a precisely-positioned position at the sub-pixel level.
  • the coordinates corresponding to the feature points with the highest matching value are selected;
  • a two-dimensional Gaussian surface is constructed according to the pixel-level coordinates corresponding to the feature points with the highest matching degree value, for example, taking points around the highest matching point value to construct a two-dimensional Gaussian surface.
  • sub-pixel level accurate positioning can be determined.
  • point A is pixel-level coordinates A (3, 5)
  • the method provided in this embodiment obtains a sub-pixel-level precise positioning position by performing sub-pixel processing on the precise positioning position, which improves the accuracy of the precise positioning position and further improves the accuracy of the determined precise positioning position. degree.
  • the fourth embodiment of the present application relates to a target tracking device 40, including: a first acquisition module 401, a second acquisition module 402, an image reduction module 403, a predicted position reduction module 404, a coarse positioning module 405, and an accurate positioning module 406,
  • a target tracking device 40 including: a first acquisition module 401, a second acquisition module 402, an image reduction module 403, a predicted position reduction module 404, a coarse positioning module 405, and an accurate positioning module 406,
  • a target tracking device 40 including: a first acquisition module 401, a second acquisition module 402, an image reduction module 403, a predicted position reduction module 404, a coarse positioning module 405, and an accurate positioning module 406,
  • the specific structure is shown in Figure 4.
  • the first acquisition module 401 is used to acquire a frame of video image and the predicted position of the target in the video image; the second acquisition module 402 is used to acquire a reduced template image of the target; the image reduction module 403 is based on the reduced template image and Prediction position, determine the reduction ratio of the video image, and reduce the video image according to the reduction ratio to obtain a reduced video image; the prediction position reduction module 404 is used to determine the predicted position of the target in the reduced video image according to the predicted position; the coarse positioning module 405 is used to match the reduced template image with the reduced video image according to the predicted position of the target in the reduced video image to determine the rough positioning information of the reduced template image in the reduced video image; precise positioning The module 406 is configured to determine the precise positioning position of the target in the video image according to the rough positioning position information.
  • This embodiment is an embodiment of a virtual device corresponding to the foregoing method.
  • the technical details in the foregoing method embodiment are still applicable in this embodiment, and details are not described herein again.
  • a fifth embodiment of the present application relates to an electronic device 50, and its structure is shown in FIG. It includes: at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501.
  • the memory 502 stores instructions executable by at least one processor 501.
  • the instructions are executed by at least one processor 501 to enable the at least one processor 501 to execute the above-mentioned target tracking method.
  • the memory 502 and the processor 501 are connected in a bus manner.
  • the bus may include any number of interconnected buses and bridges.
  • the bus links one or more processors 501 and various circuits of the memory 502 together.
  • the bus can also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, so they are not described further herein.
  • the bus interface provides an interface between the bus and the transceiver.
  • a transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium.
  • the data processed by the processor 501 is transmitted on a wireless medium through an antenna. Further, the antenna also receives the data and transmits the data to the processor 501.
  • the processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions.
  • the memory 502 may be used to store data used by the processor when performing operations.
  • processor in this embodiment can execute the implementation steps in the foregoing method embodiments, and the specific execution functions are not described in detail. For technical details in the method embodiments, details are not described herein again.
  • the sixth embodiment of the present application relates to a computer-readable storage medium.
  • the readable storage medium is a computer-readable storage medium.
  • the computer-readable storage medium stores computer instructions that enable a computer to execute the first application of the present application.
  • the display method in the above embodiments is implemented by a program instructing related hardware.
  • the program is stored in a storage medium and includes several instructions to make a device (may It is a single-chip microcomputer, a chip, or the like) or a processor that executes all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), and a random access memory (RAM, Random-Access Memory), magnetic disks, or compact discs, which can store program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及计算机视觉领域，尤其涉及一种目标跟踪方法、装置、电子设备和可读存储介质。该目标跟踪方法包括：获取一帧视频图像，以及获取目标在视频图像中的预测位置；根据缩小的模板图像和预测位置，确定视频图像的缩小比例，并将视频图像按照缩小比例缩小，得到缩小的视频图像；根据预测位置确定目标在缩小的视频图像中的预测位置；根据目标在缩小的视频图像中的预测位置，采用缩小的模板图像与缩小的视频图像进行匹配，确定缩小的模板图像在缩小的视频图像中的粗略定位位置的信息；根据粗略定位位置的信息，确定目标在视频图像中的精确定位位置。该方法可以快速、且准确的实现对目标在该视频图像中的定位位置，提高用户对AR的体验效果。

Description

一种目标跟踪方法、装置、电子设备和可读存储介质 技术领域
本申请涉及计算机视觉领域，尤其涉及一种目标跟踪方法、装置、电子设备和可读存储介质。
 背景技术
增强现实技术（Augmented Reality，简称“AR”），是一种实时地计算摄影机影像的位置及角度并加上相应图像的技术，该技术的目标是在屏幕上把虚拟世界套在现实世界并进行互动。
目标检测和目标跟踪是增强现实中的关键技术，目标检测可以检测视频图像中模板图（平面目标）的精确位置，但是，目标检测的计算非常耗时。为了克服该缺陷，在移动终端的AR应用中，通常通过目标检测获取模板图在视频图像中的初始位置，后续通过目标跟踪方法确定模板图在视频图像中的精确位置。
其中，目标跟踪方法的通常处理过程为：提取模板图的特征点，并在待匹配图像中搜寻该特征点，得到从模板图到待匹配图像的单应矩阵，从而确定出模板图在待匹配图像中的精确位置；根据确定的精确位置和模板图的历史位置，预测模板图在下一待匹配图像中的位置，根据预测的位置继续对模板图进行跟踪。
 技术问题
发明人在研究现有技术过程中发现，目前针对高分辨率的图像进行平面目标跟踪通常采用以下两种方式：第一，采用大搜索半径的方式搜寻平面目标的特征点，但是该方法耗时严重；第二，采用小搜索半径的方式搜寻平面目标的特征点，但是这种方式很容易出现搜寻失败的情况，从而导致对平面目标跟踪失败的情况。以上两种方式中，在对平面目标进行实时跟踪时，若出现平面目标相对于摄像机运动较快情况，则往往会导致对平面目标跟踪丢失的情况或者导致叠加的图像出现抖动的现象，降低了用户对AR的体验效果。
 技术解决方案
本申请部分实施例所要解决的技术问题在于提供一种目标跟踪方法、装置、电子设备和可读存储介质，使得对视频图像中的目标进行实时跟踪时，可以快速、且准确的实现对目标在该视频图像中的定位位置，提高用户对AR的体验效果。
本申请的一个实施例提供了一种目标跟踪方法，包括：获取一帧视频图像，以及获取目标在视频图像中的预测位置；获取目标的缩小的模板图像；根据缩小的模板图像和预测位置，确定视频图像的缩小比例，并将视频图像按照缩小比例缩小，得到缩小的视频图像；根据预测位置确定目标在缩小的视频图像中的预测位置；根据目标在缩小的视频图像中的预测位置，采用缩小的模板图像与缩小的视频图像进行匹配，确定缩小的模板图像在缩小的视频图像中的粗略定位位置的信息；根据粗略定位位置的信息，确定目标在视频图像中的精确定位位置。
本申请的一个实施例还提供了一种目标跟踪装置，包括：第一获取模块、第二获取模块、图像缩小模块、预测位置缩小模块、粗略定位模块和精确定位模块；第一获取模块用于获取一帧视频图像，以及获取目标在视频图像中的预测位置；第二获取模块用于获取该目标的缩小的模板图像；图像缩小模块用于根据缩小的模板图像和预测位置，确定视频图像的缩小比例，并将视频图像按照缩小比例缩小，得到缩小的视频图像；预测位置缩小模块用于根据预测位置确定目标在缩小的视频图像中的预测位置；粗略定位模块用于根据目标在缩小的视频图像中的预测位置，采用缩小的模板图像与缩小的视频图像进行匹配，确定缩小的模板图像在所述缩小的视频图像中的粗略定位位置的信息；精确定位模块用于根据所述粗略定位位置的信息，确定目标在所述视频图像中的精确定位位置。
本申请实施例还提供了一种电子设备，包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，该指令被至少一个处理器执行，以使至少一个处理器能够执行上述的目标跟踪方法。
本申请实施例还提供了一种计算机可读存储介质，存储有计算机程序，该计算机程序被处理器执行时实现上述的目标跟踪方法。
 有益效果
相对于现有技术而言，本申请部分实施例中将目标的模板图像和获取得到的视频图像进行缩小，极大程度的降低模板图像和视频图像的分辨率，使得模板图像和视频图像在进行匹配时，可以快速确定出缩小后的模板图像在缩小后的视频图像中的粗略定位位置的信息；通过粗略定位位置的信息，使得确定目标在视频图像中的精确定位位置的过程中，缩小了在视频图像中搜寻模板图像的范围，进而大大提高了确定精确定位位置的速度，缩短确定目标在视频图像中的精确定位位置的时间；且在预测位置出现了错误的情况下，由于缩小的模板图像和缩小的视频图像分辨率低、图像尺寸小，使得在不改变搜寻模板图像的范围的情况下，也可以快速确定出粗略定位位置的信息，进而不会跟丢目标的模板图像，提高了对目标跟踪的准确性，提高用户对AR的体验效果。
 附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请第一实施例中目标跟踪方法的具体流程图；
图2是本申请第二实施例中目标跟踪方法中确定目标在视频图像中精确定位位置的具体流程示意图；
图3是本申请第三实施例中目标跟踪方法中亚像素处理的具体流程示意图；
图4是本申请第四实施例中目标跟踪装置的具体结构示意图；
图5是本申请第五实施例中电子设备的具体结构示意图。
 本发明的实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请部分实施例进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。然而，本领域的普通技术人员可以理解，在本申请的各实施例中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施例的种种变化和修改，也可以实现本申请所要求保护的技术方案。
本申请的第一实施例涉及一种目标跟踪方法，该目标跟踪方法适用于终端，尤其是移动终端，如：智能手机、智能平板等。该目标跟踪方法的具体流程如图1中所示：
步骤101：获取一帧视频图像，以及获取目标在视频图像中的预测位置。
具体的说，视频图像由终端通过摄像头采集获得，如，智能手机通过摄像头获取、头戴式设备通过自带的摄像头或者外接摄像头采集获得。由于视频是由一帧一帧的连续图像组成，在AR中对目标的跟踪即是通过对连续帧图像中的目标进行跟踪实现的。在视频图像中的目标是指该视频图像中的二维物体（如一个盆栽的图像、一个灯的图像），获取目标在视频图像中的预测位置的方式可以通过该目标在上一帧视频图像中的位置估测获得；当然，该预测位置还可以是通过检测方法获取。需要说明的是，对于第一帧图像而言，目标在该视频图像中的预测位置是由检测方法获取得到，检测方法在本实施例中不再进行赘述。
步骤102：获取目标的缩小的模板图像。
一个具体实现中，缩小的模板图像为预存的目标的缩小的模板图像；或者，缩小的模板图像为目标的模板图像按照预设分辨率缩小后得到。
具体的说，目标的模板图像是指目标的二维图像，例如，一个目标的模板图像可以是一个灯的图像、一个盆栽的图像等。模板图像的获取方式有多种，例如，可以从云端服务器中获取，还可以是终端自身采集获取。本实施例中不限制模板图像的获取方式，可以根据实际需要进行选择。
其中，缩小的模板图像可以是以固定分辨率预先存储在终端中，由于直接获取缩小的模板图，大大提到了获取缩小的模板图像的速度。
当然，也可以不预存缩小的模板图像，而是根据预设分辨率对全分辨率的模板图像进行缩小，例如，预设分辨率为25×25像素，模板图像的全分辨率为720×720像素，则将全频率的模板图像缩小到25×25像素，得到缩小的模板图像。
步骤103：根据缩小的模板图像和预测位置，确定该视频图像的缩小比例，并将该视频图像按照缩小比例缩小，得到缩小的视频图像。
一个具体的实现中，根据预测位置，确定模板图像在视频图像中预估像素面积；根据缩小的模板图像的像素面积和预估像素面积，确定视频图像的缩小比例。
具体的说，预测位置为模板图像在视频图像中预测的位置，即可计算出该模板图像在视频图像中的预估像素面积，计算缩小的模板图像的像素面积，那么视频图像的缩小比例等于缩小的模板图像的像素面积除以预估像素面积的商的平方根，例如，缩小的模板图像的像素面积为sm，该全分辨率的模板图像在该视频图像中的预估像素面积为sf，那么缩小比例t等于（sm/sf）的平方根。本实施例中并不限制计算缩小比例的方法。
步骤104：根据预测位置确定目标在缩小的视频图像中的预测位置。
具体的说，将目标在视频图像中预测位置按照视频图像缩小的缩小比例进行缩小，得到目标在缩小的视频图像中的预测位置。
步骤105：根据目标在缩小的视频图像中的预测位置，采用缩小的模板图像与缩小的视频图像进行匹配，确定缩小的模板图像在缩小的视频图像中的粗略定位位置的信息。
一个具体的实现中，以目标在缩小的视频图像中的预测位置为起点，在缩小的视频图像中搜寻与缩小的模板图像匹配的图像块，根据匹配的图像块确定粗略定位位置的信息。
具体的说，在缩小的模板图像中提取特征点，并根据特征点确定缩小的模板图像上的待匹配的图像块，之后，以缩小的模板图像在缩小的视频图像中的预测位置为起点，并以预设半径在缩小的视频图像中搜寻与缩小的模板图像匹配的图像块，预设半径可以根据缩小的视频图像的分辨率确定，本实施例中不对此作限制。当然，在缩小的视频图像中匹配的图像块为包含有缩小的模板图像特征点的图像，且该图像块的尺寸与缩小的模板图像上的待匹配的图像块尺寸相同。
缩小的视频图像中搜寻到的与缩小的模板图像匹配的图像块，默认与模板图像中的图像块为同一个图像，从而根据搜寻的图像块中的特征点以及缩小的模板图像中对应的特征点，确定出缩小的模板图像映射到缩小的视频图像中单应矩阵。本实施例中可以将搜寻到的与缩小的模板图像匹配的图像块的位置，作为确定缩小的模板图像在缩小的视频图像中的粗略定位位置的信息；也可以将确定出缩小的模板图像映射到缩小的视频图像中单应矩阵，作为确定缩小的模板图像在缩小的视频图像中的粗略定位位置的信息。
需要说明的是，由于缩小的模板图像已经非常小了，因而本实施例中在缩小的模板图像中提取较少数量的特征点（如提取4个特征点）后，直接以该缩小的模板图像作为待匹配的图像块，因而在缩小的视频图像中搜寻到与缩小的模板图像匹配的图像块的位置，即为缩小的模板图像在缩小的视频图像中的位置。
步骤106：根据粗略定位位置的信息，确定目标在视频图像中的精确定位位置。
具体的说，本实施例中以缩小的模板图像在缩小的视频图像中的位置，作为粗略定位位置的信息为例进行说明。根据粗略定位位置信息，可以确定出缩小的模板图像映射至缩小的视频图像的初始单应矩阵，将该初始单应矩阵作为全分辨率的模板图像映射至全分辨率的视频图像的单应矩阵，从而根据初始单应矩阵，计算出全分辨率的模板图像在全分辨的视频图像中的初始位置，并根据分辨率的模板图像在全分辨的视频图像中的初始位置，采用全分辨率的模板图像与全分辨率的视频图像进行匹配，确定全分辨率的模板图像在全分辨率的视频图像中的位置，并将该位置作为目标在视频图像中的精确定位位置，其中匹配的过程与步骤104中的匹配过程大致相同，此处不再进行赘述。
步骤107：输出精确定位位置。
具体的说，输出精确定位位置，终端可以根据输出的目标在当前帧的视频图像上的精确定位位置，在该视频图像中的目标上叠加其他图像。当然，根据目标在当前帧的视频图像上的精确定位位置，以及该目标在历史帧的视频图像上的精确定位位置，预测出该目标在下一帧视频图像中的预测位置，预测方法在此处将不再进行赘述。
相对于现有技术而言，本申请部分实施例中将目标的模板图像和获取得到的视频图像进行缩小，极大程度的降低模板图像和视频图像的分辨率，使得模板图像和视频图像在进行匹配时，可以快速确定出缩小后的模板图像在缩小后的视频图像中的粗略定位位置的信息；通过粗略定位位置的信息，使得确定目标在视频图像中的精确定位位置的过程中，缩小了在视频图像中搜寻模板图像的范围，进而大大提高了确定精确定位位置的速度，缩短确定目标在视频图像中的精确定位位置的时间；且在预测位置出现了错误的情况下，由于缩小的模板图像和缩小的视频图像分辨率低、图像尺寸小，使得在不改变搜寻模板图像的范围的情况下，也可以快速确定出粗略定位位置的信息，进而不会跟丢目标的模板图像，提高了对目标跟踪的准确性，提高用户对AR的体验效果。
本申请的第二实施例涉及一种目标跟踪方法，第二实施例是对第一实施例的进一步改进，主要改进之处在于：本实施例中根据粗略定位位置的信息，确定目标在视频图像中的精确定位位置的过程中采用了金字塔匹配的方式。该过程的具体流程如图2所示。
步骤201：根据粗略定位位置的信息，确定将缩小的模板图像映射至缩小的视频图像的初始单应矩阵。
具体的说，本步骤201与第一实施例中的步骤106中确定单应矩阵的过程大致相同，即通过根据粗略定位位置信息以及缩小的模板图像的位置，可以确定出缩小的模板图像映射至缩小的视频图像的初始单应矩阵。
步骤202：根据初始单应矩阵，确定金字塔匹配过程中需要的N级分辨率，N为大于1的整数。
一个具体的实现中，根据初始单应矩阵，确定拍摄视频图像的使用情境，使用情境包括：拍摄的角度以及拍摄的距离；根据使用情境，确定金字塔匹配过程中需要的N级分辨率。
具体的说，根据初始单应矩阵可以获取到缩小的模板图像在缩小视频图像中的姿态信息，根据粗略定位位置信息可以获知缩小的模板图像在缩小的视频图像中的位置，根据缩小的模板图像在缩小的视频图像中的位置以及缩小的模板图像在缩小的视频图像中的姿态，判断拍摄视频图像的使用情境，根据使用情境，确定金字塔匹配过程中需要的N级分辨率。例如，若确定出使用情境为大角度拍摄，那么确定两级分辨率，第一级分辨率为1/2分辨率，第二级分辨率为全分辨率；若使用情境为近距离拍摄，那么确定三级分辨率，第一级分辨率为1/4分辨率，第二级分辨率为1/2分辨率，第三级分辨率为全分辨率。
步骤203：根据初始单应矩阵以及确定的N级分辨率，对视频图像进行金字塔匹配，根据金字塔匹配的结果确定目标在视频图像中的精确定位位置。
一个具体的实现中，按照N级分辨率对视频图像进行缩放，得到N级分辨率各自对应的N级视频图像，以及从预存的不同分辨率的模板图像中选取与得到的各级视频图像各自对应的N级模板图像；按照分辨率从低到高的顺序进行以下处理：根据第i+1级分辨率对应的初始位置，将第i+1级模板图像与第i+1级视频图像进行匹配，确定第i+1级模板图像映射至第i+1级视频图像的单应矩阵，其中，第1级分辨率为N级分辨率中的最低分辨率，第1级分辨率对应的初始位置根据初始单应矩阵确定，第i+1级分辨率对应的初始位置为根据第i级分辨率对应的单应矩阵确定，N>1，1≤i≤N-1；根据第N级分辨率对应的单应矩阵，确定目标在视频图像中的精确定位位置。
具体的说，在确定了分辨率级数以及每级的分辨率，则可以根据初始单应矩阵，对视频图像进行金字塔匹配。下面将以一个具体的例子进行说明金字塔匹配的过程。
例如，若使用情境为大角度拍摄，确定第一级分辨率为1/2分辨率，第二级分辨率为全分辨率。按照第一级分辨率对视频图像进行缩放，获得1/2分辨率的视频图像。在金子塔匹配中，预先存储有各级分辨率的模板图像，根据初始单应矩阵H0，计算模板图像在1/2分辨率的视频图像中的像素面积S1，从预存的各级分辨率的模板图像中选出与像素面积S1最接近的模板图像1，并将模板图像1作为与1/2分辨率的视频图像对应1/2分辨率的模板图像，由于第二级分辨率为全分辨率，则无需对视频图像进行缩放，直接选取全分辨率的模板图像作为全分辨率的视频图像对应的模板图像。
按照分辨率从低到高的顺序进行处理：根据初始单应矩阵H0，将1/2分辨率的模板图像映射到1/2分辨率的视频图像中，获取1/2分辨率的模板图像在1/2分辨率的视频图像中的初始位置，根据第一级分辨率对应的初始位置，将第一级模板图像与第一级视频图像进行匹配，确定第一级模板图像映射至第一级视频图像的单应矩阵H1，匹配过程与第一实施例中的匹配过程大致相同，此处将不再进行赘述。根据单应矩阵H1，将全分辨率的模板图像映射至全分辨率的视频图像中，获得第二级模板图像在第二级视频图像中的初始位置，根据第二级分辨率对应的初始位置，将第二级模板图像与第二级视频图像进行匹配，确定第二级模板图像映射至第二级视频图像的单应矩阵H2，通过单应矩阵H2，即可确定出第二级模板图像在第二级视频图像中的定位位置。
一个具体的实现中，确定了第N级分辨率对应的单应矩阵后，判断第N级分辨率是否为视频图像的全分辨率；若不是，则根据第N级分辨率对应的单应矩阵，采用全分辨率的模板图像与全分辨率的视频图像进行匹配，得到全分辨率对应的单应矩阵，并根据全分辨率对应的单应矩阵确定全分辨率的模板图像在全分辨率视频图像中的定位位置，并作为目标在视频图像中的精确定位位置；若是，则根据第N级分辨率对应的单应矩阵，确定全分辨率的模板图像在全分辨率视频图像中的定位位置，并作为目标在视频图像中的精确定位位置。
具体的说，若在金字塔匹配过程中，第N级分辨率不是全分辨率，则需要确定全分辨率的模板图像在全分辨率的视频图像中的定位位置。
如上述的例子，在确定出第二级分辨率对应的单应矩阵后，判断第二级分辨是否为全分辨率，由于判断出为第二级分辨率为全分辨率，因而可以直接将全分辨率的模板图像在全分辨率视频图像中的定位位置，作为目标在视频图像中的精确定位位置。
值得一提的是，将第i+1级模板图像与第i+1级视频图像进行匹配过程为：在第i+1级视频图像中搜寻与第i+1级模板图匹配的图像块，根据图像块确定第i+1级模板图像映射至第i+1级视频图像的单应矩阵，其中，匹配过程中使用图像块中的部分像素进行匹配。
具体的说，在第i+1级视频图像中搜寻与第i+1级模板图像匹配的图像块的过程，在第i+1级模板图像中找出一定数量的稳定特征点（如使用Harris角点的方式找寻特征点），可以理解的是，为了便于快速在第i+1级视频图像中搜寻第i+1级模板图像中的特征点，匹配的图像块可以使用预设的形状，如圆形、矩形等。当然，为了进一步提高匹配的速度，匹配过程中使用图像块中的部分像素进行匹配，其中，使用图像块中的部分像素呈米字型或者X型分布，通过米字形或X型的像素分布。
与现有技术相比，本实施例提供的目标跟踪方法，在确定目标在视频图像中的精确定位位置的过程中采用金字塔匹配的方式，由于金字塔匹配的方式是一种分层级的匹配方式，先对低分辨率的视频图像进行匹配，再对高分辨率的视频图像进行匹配，不停更新该目标映射到视频图像中的单应矩阵，从而可以精确地确定出目标在视频该图像中的精确定位，同时，由于不是直接在全分辨的视频图像中搜寻全分辨率的模板图像，而是从低分辨率开始，并根据低分辨率级视频匹配获得的单应矩阵，可以快速地确定出全分辨的模板图像在全分辨率视频图像中初始位置，从而加快确定目标在视频图像中的精确定位位置。
本申请的第三实施例涉及一种目标跟踪方法，第三实施例是第二实施例的进一步改进，主要改进之处在于：本实施例中若精确定位位置为像素级坐标位置，则根据第N级分辨率对应的单应矩阵，确定目标在视频图像中的精确定位位置之后，获得亚像素级别的精确定位位置，获得亚像素级别的精确定位位置的具体流程如图3所示：
步骤301：获取精确定位位置的匹配度值，其中，匹配度值为目标的特征点与视频图像中图像块的特征点匹配的相似度值。
具体的说，由于视频图像是以像素为单位组成，确定的目标在视频图像中的精确定位置也为像素级坐标位置，但是，由于在实际中目标的特征点与视频图像中图像块的特征点匹配度值最高的位置可能是亚像素，因而，为了提高目标在视频图像中的精确定位位置的精确度，对像素级进行处理。精确定位位置的匹配度值可以在匹配过程获取。
步骤302：根据匹配度值，对精确定位位置进行亚像素处理，获得亚像素级别的精确定位位置。
一个具体的实现中，选取匹配度值最高的特征点对应的坐标；
根据选取的坐标构建高斯曲面；对高斯曲面进行拟合，获得亚像素级别的精确定位。
具体的说，根据选取匹配度值最高的特征点对应的像素级坐标构建二维高斯曲面，比如在匹配度值最高点周围取点，构建二维高斯曲面。并对构建的二维高斯曲面进行拟合，即可确定出亚像素级别的精确定位。像素级坐标变为亚像素级别坐标的效果如：点A为像素级坐标A（3,5），经过处理后得到的亚像素级坐标A’（3.212，4.867）。
与现有技术相比，本实施例提供的方法，通过对精确定位位置进行亚像素处理，得到亚像素级精确定位位置，提高了精确定位位置的精度，进一步提高了确定的精确定位位置的精确度。
本申请的第四实施例涉及一种目标跟踪装置40，包括：第一获取模块401、第二获取模块402、图像缩小模块403、预测位置缩小模块404、粗略定位模块405和精确定位模块406，具体的结构如图4所示。
第一获取模块401用于获取一帧视频图像，以及获取目标在视频图像中的预测位置；第二获取模块402用于获取该目标的缩小的模板图像；图像缩小模块403根据缩小的模板图像和预测位置，确定视频图像的缩小比例，并将视频图像按照缩小比例缩小，得到缩小的视频图像；预测位置缩小模块404用于根据预测位置确定目标在缩小的视频图像中的预测位置；粗略定位模块405用于根据目标在缩小的视频图像中的预测位置，采用缩小的模板图像与缩小的视频图像进行匹配，确定缩小的模板图像在所述缩小的视频图像中的粗略定位位置的信息；精确定位模块406用于根据粗略定位位置的信息，确定目标在视频图像中的精确定位位置。
本实施例是与上述方法对应的虚拟装置实施例，上述方法实施例中技术细节在本实施例中依然适用，此处不再赘述。
需要说明的是，以上所述的装置实施例仅仅是示意性的，并不对本申请的保护范围构成限定，在实际应用中，本领域的技术人员可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的，此处不做限制。
本申请的第五实施例涉及一种电子设备50，其结构如图5所示。包括：至少一个处理器501；以及，与至少一个处理器501通信连接的存储器502。存储器502存储有可被至少一个处理器501执行的指令。指令被至少一个处理器501执行，以使至少一个处理器501能够执行上述的目标跟踪方法。
存储器502和处理器501采用总线方式连接，总线可以包括任意数量的互联的总线和桥，总线将一个或多个处理器501和存储器502的各种电路链接在一起。总线还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路链接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件，也可以是多个元件，比如多个接收器和发送器，提供用于在传输介质上与各种其他装置通信的单元。经处理器501处理的数据通过天线在无线介质上进行传输，进一步，天线还接收数据并将数据传送给处理器501。
处理器501负责管理总线和通常的处理，还可以提供各种功能，包括定时，外围接口，电压调节、电源管理以及其他控制功能。而存储器502可以被用于存储处理器在执行操作时所使用的数据。
需要说明的是，本实施例中的处理器能够执行上述的方法实施例中实施步骤，具体的执行功能并未详细说明，可参见方法实施例中的技术细节，此处不再赘述。
本申请的第六实施例涉及一种计算机可读存储介质，该可读存储介质为计算机可读存储介质，该计算机可读存储介质中存储有计算机指令，该计算机指令使计算机能够执行本申请第一或第二方法实施例中涉及的目标跟踪方法。
需要说明的是，本领域的技术人员能够理解，上述实施例中显示方法是通过程序来指令相关的硬件来完成的，该程序存储在一个存储介质中，包括若干指令用以使得一个设备（可以是单片机，芯片等）或处理器（processor）执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random-Access Memory）、磁碟或者光盘等各种可以存储程序代码的介质。
本领域的普通技术人员可以理解，上述各实施例是实现本申请的具体实施例，而在实际应用中，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (16)

  1. 一种目标跟踪方法，其中，包括：
    获取一帧视频图像，以及获取目标在所述视频图像中的预测位置；
    获取所述目标的缩小的模板图像；
    根据所述缩小的模板图像和所述预测位置，确定所述视频图像的缩小比例，并将所述视频图像按照所述缩小比例缩小，得到缩小的视频图像；
    根据所述预测位置确定所述目标在所述缩小的视频图像中的预测位置；
    根据所述目标在所述缩小的视频图像中的预测位置，采用所述缩小的模板图像与所述缩小的视频图像进行匹配，确定所述缩小的模板图像在所述缩小的视频图像中的粗略定位位置的信息；
    根据所述粗略定位位置的信息，确定所述目标在所述视频图像中的精确定位位置。
  2. 根据权利要求1所述的目标跟踪方法，其中，根据所述目标在所述缩小的视频图像中的预测位置，采用所述缩小的模板图像与所述缩小的视频图像进行匹配，确定所述缩小的模板图像在所述缩小的视频图像中的粗略定位位置的信息，具体包括：
    以所述目标在所述缩小的视频图像中的预测位置为起点，在所述缩小的视频图像中搜寻与所述缩小的模板图像匹配的图像块，根据所述匹配的图像块确定所述粗略定位位置的信息。
  3. 根据权利要求1或2所述的目标跟踪方法，其中，根据所述粗略定位位置的信息，确定所述目标在所述视频图像中的精确定位位置，具体包括：
    根据所述粗略定位位置的信息，确定将所述缩小的模板图像映射至所述缩小的视频图像的初始单应矩阵；
    根据所述初始单应矩阵，确定金字塔匹配过程中需要的N级分辨率，N为大于1的整数；
    根据初始单应矩阵以及确定的N级分辨率，对所述视频图像进行所述金字塔匹配，根据所述金字塔匹配的结果确定所述目标在所述视频图像中的精确定位位置。
  4. 根据权利要求3所述的目标跟踪方法，其中，根据初始单应矩阵以及确定的N级分辨率，对所述视频图像进行金字塔匹配，确定所述目标在所述视频图像中的精确定位位置，具体包括：
    按照所述N级分辨率对所述视频图像进行缩放，得到N级分辨率各自对应的N级视频图像，以及从预存的不同分辨率的模板图像中选取与得到的各级视频图像各自对应的N级模板图像；
    按照分辨率从低到高的顺序进行以下处理：根据第i+1级分辨率对应的初始位置，将第i+1级模板图像与第i+1级视频图像进行匹配，确定第i+1级模板图像映射至第i+1级视频图像的单应矩阵，其中，第1级分辨率为所述N级分辨率中的最低分辨率，第1级分辨率对应的初始位置根据所述初始单应矩阵确定，第i+1级分辨率对应的初始位置为根据第i级分辨率对应的单应矩阵确定，N>1,1≤i≤N-1；
    根据第N级分辨率对应的单应矩阵，确定所述目标在所述视频图像中的精确定位位置。
  5. 根据权利要求3或4所述的目标跟踪方法，其中，根据所述初始单应矩阵，确定金字塔匹配过程中需要的N级分辨率，具体包括：
    根据所述初始单应矩阵，确定拍摄所述视频图像的使用情境，所述使用情境包括：拍摄的角度以及拍摄的距离；
    根据所述使用情境，确定所述金字塔匹配过程中需要的N级分辨率。
  6. 根据权利要求4所述的目标跟踪方法，其中，根据第N级分辨率对应的单应矩阵，确定所述目标在所述视频图像中的精确定位位置，具体包括：
    判断第N级分辨率是否为所述视频图像的全分辨率；
    若不是，则根据所述第N级分辨率对应的单应矩阵，采用全分辨率的模板图像与全分辨率的视频图像进行匹配，得到全分辨率对应的单应矩阵，并根据所述全分辨率对应的单应矩阵确定全分辨率的模板图像在全分辨率视频图像中的定位位置，并作为所述目标在所述视频图像中的精确定位位置；
    若是，则根据所述第N级分辨率对应的单应矩阵，确定全分辨率的模板图像在全分辨率视频图像中的定位位置，并作为所述目标在所述视频图像中的精确定位位置。
  7. 根据权利要求4或6所述的目标跟踪方法，其中，若所述精确定位位置为像素级坐标位置，则根据第N级分辨率对应的单应矩阵，确定所述目标在所述视频图像中的精确定位位置之后，所述目标的跟踪方法还包括：
    获取所述精确定位位置的匹配度值，其中，匹配度值为所述目标的特征点与所述视频图像中图像块的特征点匹配的相似度值；
    根据所述匹配度值，对所述精确定位位置进行亚像素处理，获得亚像素级别的精确定位位置。
  8. 根据权利要求7所述的目标跟踪方法，其中，根据所述匹配度值，对所述精确定位位置进行亚像素处理，获得亚像素级别的精确定位位置，具体包括：
    选取匹配度值最高的特征点对应的坐标；
    根据选取的坐标构建高斯曲面；
    对所述高斯曲面进行拟合，获得亚像素级别的精确定位。
  9. 根据权利要求1至8中任一项所述的目标跟踪方法，其中，在确定所述目标在所述视频图像中的精确定位位置之后，所述目标的跟踪方法还包括：输出精确定位位置。
  10. 根据权利要求4所述的目标跟踪方法，其中，所述将第i+1级模板图像与第i+1级视频图像进行匹配，具体包括：
    在所述第i+1级视频图像中搜寻与所述第i+1级模板图匹配的图像块，根据所述图像块确定第i+1级模板图像映射至第i+1级视频图像的单应矩阵，其中，匹配过程中使用所述图像块中的部分像素进行匹配。
  11. 根据权利要求9所述的目标跟踪方法，其中，所述匹配过程中使用所述图像块中的部分像素呈米字型或者X型分布。
  12. 根据权利要求1至11中任一项所述的目标跟踪方法，其中，所述缩小的模板图像为预存的所述目标的缩小的模板图像；
    或者，
    所述缩小的模板图像为目标的模板图像按照预设分辨率缩小后得到。
  13. 根据权利要求1至12中任一项所述的目标跟踪方法，其中，根据缩小的模板图像和所述预测位置，确定所述视频图像的缩小比例，具体包括：
    根据所述预测位置，确定所述模板图像在所述视频图像中预估像素面积；
    根据所述缩小的模板图像的像素面积和所述预估像素面积，确定所述视频图像的缩小比例。
  14. 一种目标跟踪装置，其中，包括：第一获取模块、第二获取模块、图像缩小模块、预测位置缩小模块、粗略定位模块和精确定位模块；
    所述第一获取模块用于获取一帧视频图像，以及获取目标在所述视频图像中的预测位置；
    所述第二获取模块用于获取所述目标的缩小的模板图像；
    所述图像缩小模块用于根据所述缩小的模板图像和所述预测位置，确定所述视频图像的缩小比例，并将所述视频图像按照所述缩小比例缩小，得到缩小的视频图像；
    预测位置缩小模块用于根据所述预测位置确定所述目标在所述缩小的视频图像中的预测位置；
    粗略定位模块用于根据所述目标在所述缩小的视频图像中的预测位置，采用所述缩小的模板图像与所述缩小的视频图像进行匹配，确定所述缩小的模板图像在所述缩小的视频图像中的粗略定位位置的信息；
    精确定位模块用于根据所述粗略定位位置的信息，确定所述目标在所述视频图像中的精确定位位置。
  15. 一种电子设备，其中，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1~13任一项所述的目标跟踪方法。
  16. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1~13任一项所述的目标跟踪方法。
PCT/CN2018/096161 2018-07-18 2018-07-18 一种目标跟踪方法、装置、电子设备和可读存储介质 Ceased WO2020014901A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880001302.7A CN109074657B (zh) 2018-07-18 2018-07-18 一种目标跟踪方法、装置、电子设备和可读存储介质
PCT/CN2018/096161 WO2020014901A1 (zh) 2018-07-18 2018-07-18 一种目标跟踪方法、装置、电子设备和可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/096161 WO2020014901A1 (zh) 2018-07-18 2018-07-18 一种目标跟踪方法、装置、电子设备和可读存储介质

Publications (1)

Publication Number Publication Date
WO2020014901A1 true WO2020014901A1 (zh) 2020-01-23

Family

ID=64789301

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/096161 Ceased WO2020014901A1 (zh) 2018-07-18 2018-07-18 一种目标跟踪方法、装置、电子设备和可读存储介质

Country Status (2)

Country Link
CN (1) CN109074657B (zh)
WO (1) WO2020014901A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111598923A (zh) * 2020-05-08 2020-08-28 腾讯科技（深圳）有限公司 目标跟踪方法、装置、计算机设备及存储介质
CN112633198A (zh) * 2020-12-28 2021-04-09 浙江大华技术股份有限公司 一种图片处理方法、装置、存储介质及电子装置
CN113283442A (zh) * 2020-02-19 2021-08-20 北京四维图新科技股份有限公司 特征点的提取方法和装置
CN113344980A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
CN115311296A (zh) * 2022-10-12 2022-11-08 湖南视比特机器人有限公司 数据生成、图像识别方法及计算机存储介质和终端设备
CN117115250A (zh) * 2023-08-03 2023-11-24 四川中电启明星信息技术有限公司 一种基于网格的图像匹配定位方法、系统、设备及介质

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111476065A (zh) * 2019-01-23 2020-07-31 北京奇虎科技有限公司 目标跟踪方法、装置、计算机设备及存储介质
CN110163894B (zh) * 2019-05-14 2021-04-06 中国科学院半导体研究所 基于特征匹配的亚像素级目标跟踪方法
CN110363172A (zh) * 2019-07-22 2019-10-22 曲靖正则软件开发有限公司 一种视频处理方法、装置、电子设备及可读存储介质
CN110689556A (zh) * 2019-09-09 2020-01-14 苏州臻迪智能科技有限公司 跟踪方法、装置及智能设备
WO2021128243A1 (zh) * 2019-12-27 2021-07-01 威创集团股份有限公司 一种目标图案查找方法及计算机可读存储介质
CN110930428B (zh) * 2020-02-19 2020-08-14 成都纵横大鹏无人机科技有限公司 目标跟踪方法、装置、电子设备及存储介质
CN112584234B (zh) * 2020-12-09 2023-06-16 广州虎牙科技有限公司 视频图像的补帧方法及相关装置
TWI817116B (zh) * 2021-05-12 2023-10-01 和碩聯合科技股份有限公司 物件定位方法及物件定位系統

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826228A (zh) * 2010-05-14 2010-09-08 上海理工大学 基于背景估计的公交乘客运动目标检测方法
CN102915545A (zh) * 2012-09-20 2013-02-06 华东师范大学 一种基于OpenCV的视频目标跟踪算法
CN108257153A (zh) * 2017-12-29 2018-07-06 中国电子科技集团公司第二十七研究所 一种基于方向梯度统计特征的目标跟踪方法

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739692B (zh) * 2009-12-29 2012-05-30 天津市亚安科技股份有限公司 实时视频目标的快速相关跟踪方法
CN102215377B (zh) * 2010-04-07 2012-10-31 北京智安邦科技有限公司 Ptz单目标自动跟踪的装置及方法
CN102490523B (zh) * 2011-12-16 2014-06-04 武汉大学 一种基于计算机视觉的自动激光雕刻系统及其使用方法
CN103700115B (zh) * 2012-09-27 2016-08-31 中国航天科工集团第二研究院二O七所 一种复杂背景下运动目标相关匹配跟踪方法
CN104424634B (zh) * 2013-08-23 2017-05-03 株式会社理光 对象跟踪方法和装置
CN103996208A (zh) * 2014-05-21 2014-08-20 国家电网公司 一种在视频图像中进行ptz单个目标自动跟踪方法
JP6420605B2 (ja) * 2014-09-24 2018-11-07 Kddi株式会社 画像処理装置
CN104408725B (zh) * 2014-11-28 2017-07-04 中国航天时代电子公司 一种基于tld优化算法的目标重捕获系统及方法
CN104637052B (zh) * 2015-01-22 2017-07-11 西南交通大学 基于目标引导显著性检测的对象跟踪方法
CN104851111B (zh) * 2015-04-23 2017-07-07 北京环境特性研究所 一种使用连续变焦探测器的目标跟踪方法
CN106296729A (zh) * 2016-07-27 2017-01-04 南京华图信息技术有限公司 一种鲁棒的实时红外成像地面运动目标跟踪方法和系统
CN106407975B (zh) * 2016-08-24 2019-11-15 南京理工大学 基于空间-光谱结构约束的多尺度分层目标检测方法
CN106408593B (zh) * 2016-09-18 2019-05-17 东软集团股份有限公司 一种基于视频的车辆跟踪方法及装置
CN106780620B (zh) * 2016-11-28 2020-01-24 长安大学 一种乒乓球运动轨迹识别定位与跟踪系统及方法
CN107315992A (zh) * 2017-05-05 2017-11-03 深圳电航空技术有限公司 一种基于电子云台的跟踪方法及装置
CN107330917B (zh) * 2017-06-23 2019-06-25 歌尔股份有限公司 移动目标的跟踪拍摄方法和跟踪设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826228A (zh) * 2010-05-14 2010-09-08 上海理工大学 基于背景估计的公交乘客运动目标检测方法
CN102915545A (zh) * 2012-09-20 2013-02-06 华东师范大学 一种基于OpenCV的视频目标跟踪算法
CN108257153A (zh) * 2017-12-29 2018-07-06 中国电子科技集团公司第二十七研究所 一种基于方向梯度统计特征的目标跟踪方法

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283442A (zh) * 2020-02-19 2021-08-20 北京四维图新科技股份有限公司 特征点的提取方法和装置
CN113283442B (zh) * 2020-02-19 2024-05-10 北京四维图新科技股份有限公司 特征点的提取方法和装置
CN111598923A (zh) * 2020-05-08 2020-08-28 腾讯科技（深圳）有限公司 目标跟踪方法、装置、计算机设备及存储介质
CN111598923B (zh) * 2020-05-08 2022-09-20 腾讯科技（深圳）有限公司 目标跟踪方法、装置、计算机设备及存储介质
CN112633198A (zh) * 2020-12-28 2021-04-09 浙江大华技术股份有限公司 一种图片处理方法、装置、存储介质及电子装置
CN113344980A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
CN115311296A (zh) * 2022-10-12 2022-11-08 湖南视比特机器人有限公司 数据生成、图像识别方法及计算机存储介质和终端设备
CN115311296B (zh) * 2022-10-12 2023-04-28 湖南视比特机器人有限公司 数据生成、图像识别方法及计算机存储介质和终端设备
CN117115250A (zh) * 2023-08-03 2023-11-24 四川中电启明星信息技术有限公司 一种基于网格的图像匹配定位方法、系统、设备及介质

Also Published As

Publication number Publication date
CN109074657B (zh) 2022-04-08
CN109074657A (zh) 2018-12-21

Similar Documents

Publication Publication Date Title
WO2020014901A1 (zh) 一种目标跟踪方法、装置、电子设备和可读存储介质
US9646406B2 (en) Position searching method and apparatus based on electronic map
CN105283905B (zh) 使用点和线特征的稳健跟踪
JP5746937B2 (ja) オブジェクト追跡装置
US20200074646A1 (en) Method for obtaining image tracking points and device and storage medium thereof
WO2018153313A1 (zh) 立体视觉摄像机及其高度获取方法、高度获取系统
CN110689014B (zh) 感兴趣区域的检测方法、装置、电子设备及可读存储介质
US20190199992A1 (en) Information processing apparatus, method for controlling the same, and recording medium
US11682212B2 (en) Hierarchical data organization for dense optical flow processing in a computer vision system
KR101738572B1 (ko) 뎁스 이미지를 이용한 마커리스 증강현실 데이터처리 디바이스 및 뎁스 이미지를 이용한 증강현실 데이터처리방법
KR20180022539A (ko) 중첩영역의 제어점들을 이용한 시차 최소화 스티칭 장치 및 방법
JP2018537748A (ja) 可変の計算量を用いた画像のライトフィールドレンダリング
CN114096994A (zh) 图像对齐方法及装置、电子设备、存储介质
WO2020228353A1 (zh) 一种基于运动加速度的图像搜索方法、系统及电子设备
CN118817261A (zh) 一种图像采集设备的参数确定方法、装置、设备及介质
CN114610150A (zh) 图像处理方法及装置
US9852542B1 (en) Methods and apparatus related to georeferenced pose of 3D models
CN115331143B (zh) 一种视频对齐方法、装置、设备以及存储介质
US20140270360A1 (en) Edgel sampling for edge-based tracking
US11386525B2 (en) Image stitching apparatus and operation method thereof
CN120111208A (zh) 利用双目差别机制提升vr设备细节显示质量的方法
CN113313648A (zh) 图像矫正方法、装置、电子设备和介质
CN114596198A (zh) 一种图像处理方法、装置、设备及存储介质
CN108650465B (zh) 摄像机画面增强现实标签的计算方法、装置及电子设备
US8824794B1 (en) Graduated color correction of digital terrain assets across different levels of detail

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18926967

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.05.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18926967

Country of ref document: EP

Kind code of ref document: A1