WO2020215865A1 - Target tracking system and method - Google Patents

Target tracking system and method Download PDF

Info

Publication number
WO2020215865A1
WO2020215865A1 PCT/CN2020/075916 CN2020075916W WO2020215865A1 WO 2020215865 A1 WO2020215865 A1 WO 2020215865A1 CN 2020075916 W CN2020075916 W CN 2020075916W WO 2020215865 A1 WO2020215865 A1 WO 2020215865A1
Authority
WO
WIPO (PCT)
Prior art keywords
tracking
target tracking
target
module
information
Prior art date
Application number
PCT/CN2020/075916
Other languages
French (fr)
Chinese (zh)
Inventor
董鹏宇
冯晓光
刘文江
刘学彦
尹东
Original Assignee
上海富瀚微电子股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海富瀚微电子股份有限公司 filed Critical 上海富瀚微电子股份有限公司
Publication of WO2020215865A1 publication Critical patent/WO2020215865A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration by the use of local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details

Definitions

  • the present invention relates to the technical field of image processing, in particular to a target tracking system and method.
  • the target tracking unit is a common functional module in front-end equipment such as smart cameras. In practical applications, this unit generally belongs to a part of the intelligent analysis engine. After receiving the output video signal of the image processing unit (ISP), it analyzes and processes it to realize the movement tracking or further behavior analysis of the target of interest.
  • ISP image processing unit
  • the implementation method of the target tracking unit is generally to first automatically detect or manually specify the target tracking object by the target detection module, then perform feature extraction on the target area, perform feature calculation and matching in the search domain of subsequent video frames, and then update the corresponding target tracking parameters.
  • target tracking methods There are currently two types of features commonly used in target tracking methods: gradient-related features represented by "High-Speed Tracking with Kernelized Correlation Filters” and color-related features represented by "In defense of color-based model-free tracking”.
  • the parameter update method generally adopts first-order smoothing, and the search range of feature matching is generally fixed within 3 times the target size.
  • the most common problem of the above target tracking method is the drift of target tracking coordinates. There are several reasons for this: 1. There are interference objects similar to the target feature in the search field; 2. The target feature is triggered by the object being blocked or the environment is drastically changed to trigger the camera to greatly adjust the automatic exposure/auto white balance/autofocus, etc. Dramatic changes have taken place. For example, the camera will automatically turn on the fill light and switch to the black and white mode when the light is low. At this time, the three-dimensional color histogram in the method "In defense of color-based model-free tracking" automatically degenerates into a one-dimensional brightness histogram, which leads to The target feature dimension is reduced.
  • patents such as CN104537689B improve the robustness and accuracy of target tracking by simultaneously using color, texture, and gradient histogram methods to increase feature operators.
  • the performance improvement also greatly increases the implementation cost.
  • the purpose of the present invention is to provide a target tracking system and method to improve the robustness and accuracy of target tracking.
  • the present invention provides a target tracking system.
  • the target tracking system includes an image processing unit and a target tracking unit.
  • the image processing unit includes a statistical module, a statistical credibility calculation module, and a sharpness calculation module.
  • a frame buffer module and a motion adaptive determination module, the target tracking unit includes a target tracking parameter control module and a target tracking feature calculation module;
  • the statistics module is used to count the brightness, chroma and sharpness of each frame of image, or to count the chroma and brightness of each frame of image;
  • the statistical credibility calculation module is configured to obtain sensitivity information to determine whether the statistical information of the statistical module is credible, and transmit the result of the judgment to the current target tracking parameter control module;
  • the sharpness calculation module is used to obtain the sharpness information of each pixel in the image
  • the frame buffer module is used for buffering accumulated frame information
  • the motion adaptive determination module is used to obtain frame information of the current frame image and read the accumulated frame information from the frame buffer module to obtain the current motion information of the tracking target;
  • the target tracking parameter control module is configured to read the statistical information and the current movement information of the tracking target after receiving the judgment result of the statistical credibility calculation module as being credible while tracking the target Adjusting the current target tracking parameters according to the statistical information and the current movement information of the tracking target and determining whether to switch the tracking feature used in the current target tracking method and sending the result of the determination to the target tracking feature calculation module;
  • the target tracking feature calculation module is used to receive the judgment result of the target tracking parameter control module to determine whether to read the sharpness information of each pixel in the image acquired by the sharpness calculation module to track the current target
  • the tracking feature used is switched.
  • the statistics module is a 3A statistics module for statistics of brightness, chroma, and sharpness of the image.
  • the statistics module is a 2A statistics module, which is used to count the brightness and chromaticity of the image.
  • the tracking features include color, texture, and/or gradient histogram.
  • the sensitivity information includes exposure time, analog gain, and digital gain.
  • the present invention also provides a target tracking method, including:
  • adjusting the current target tracking parameter update speed according to changes in the statistical information of the statistics module includes:
  • the statistical information of the statistical module changes, it is determined whether the tracking feature used in the current tracking method is affected by the change of the statistical information of the statistical module, if yes, then the update speed of the current target tracking parameter is reduced, if not, keep Current target tracking parameter update speed.
  • filtering out the motion confidence region in the current target search area according to the current motion information of the tracking target includes:
  • the circumscribed rectangle of the current motion information of the tracking target after the filtering operation is calculated to obtain the motion confidence region.
  • the search range is adjusted and the motion confidence region is searched first.
  • the present invention also provides another target tracking method, including:
  • the target tracking system includes an image processing unit and a target tracking unit, and the image processing unit includes a statistical module, a statistical credibility calculation module, a sharpness calculation module, and a motion adaptive A determination module and a frame buffer module.
  • the target tracking unit includes a target tracking parameter control module and a target tracking feature calculation module.
  • the target tracking parameter control module reads the statistical information of the statistics module and the motion adaptive determination module
  • the obtained motion information of the tracking target is used to adjust the update speed of the current target tracking parameters to track the target, and to determine whether it is necessary to switch the tracking feature used in the target tracking method by reading the statistical information of the statistical module, and if necessary,
  • the target and feature calculation module reads the sharpness information of each pixel of the image acquired by the statistical credibility calculation module to switch the tracking feature used in the target tracking method. In this way, the image processing unit and the The combination of the target tracking unit improves the robustness and accuracy of target tracking.
  • FIG. 1 is a schematic diagram of the connection relationship of a target tracking system according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the process of a target tracking method provided by an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of another target tracking method provided by an embodiment of the present invention.
  • Figure 4 is a schematic diagram of screening motion confidence regions in an embodiment of the present invention.
  • 1-image processing unit 11-statistics module; 12-statistical credibility calculation module; 13-sharpness calculation module; 14-frame buffer module; 15-motion adaptive determination module; 2-target tracking unit; 21-target Tracking parameter control module; 22-target tracking feature calculation module; 3-image input unit.
  • FIG. 1 is a schematic diagram of the connection relationship of the target tracking system provided by the embodiment of the present invention.
  • the target tracking system provided by the present invention includes an image processing unit 1 and a target tracking unit 2.
  • the image processing unit 1 includes a statistical module 11, a statistical credibility calculation module 12, a sharpness calculation module 13, a motion adaptive determination module 15, a frame buffer module 1414, and a motion adaptive determination module 15;
  • the target tracking unit 2 includes The target tracking parameter control module 21 and the target tracking feature calculation module 22.
  • the statistics module 11 is used to count the brightness, chroma and sharpness of each frame of images, or to count the chroma and sharpness of each frame of images; specifically, the statistics module 11 may be a 3A statistics module or a 2A statistics module. Module, if the image processing unit 1 supports auto focus (AF), the statistics module 11 is a 3A statistics module, used to count the brightness, chroma and sharpness of the image, if the image processing unit 1 does not support AF , Then the statistics module 11 is a 2A statistics module, used to count the brightness and chromaticity of the image.
  • AF auto focus
  • the statistical credibility calculation module 12 is configured to obtain sensitivity information to determine whether the statistical information of the statistical module 11 is credible, and transmit the judgment result to the target tracking parameter control module 21.
  • the sensitivity information includes exposure time, analog gain, and digital gain.
  • the judging process includes: converting the exposure time, analog gain, and digital gain, and if the converted index is less than the threshold, the statistical information of the statistical module 11 is considered credible.
  • a typical threshold is 5000, that is, if the converted index is less than 5000, the statistical information of the statistical module 11 is considered to be credible.
  • the sharpness calculation module 13 is used to obtain the sharpness information of each pixel in the image.
  • the difference from the sharpness information obtained by the statistical module 11 is that the sharpness information of the entire image obtained by the statistical module 11, and the sharpness information obtained by the sharpness calculation module 13 is the sharpness of each pixel. Information, in comparison, is more accurate.
  • the frame buffer module 14 is used to buffer the accumulated frame information counted by the statistics module 11; the accumulated frame information may specifically be pre-stored in the frame buffer module 14 through time-domain filtering.
  • the motion adaptive determination module 15 is used to obtain the frame information of the current frame image and read the accumulated frame information from the frame buffer module 14 to obtain the current motion information of the tracking target.
  • the target tracking parameter control module 21 is configured to read the statistical information and the current tracking target after receiving the judgment result of the statistical credibility calculation module 12 as being credible while tracking the target.
  • the current target tracking parameter is adjusted according to the statistical information and the current motion information of the tracking target, and the tracking feature used in the current target tracking method is judged whether to switch.
  • the tracking features include color, texture, and/or gradient histograms.
  • the premise that the target tracking parameter control module 21 adjusts the current target tracking parameters according to the statistical information of the statistical module 11 and the current movement information of the tracking target is that the credibility of the statistical information of the statistical module 11 is greater than a threshold. Specifically, if the result of the determination by the statistical credibility calculation module 12 is that the credibility of the statistical information of the statistical module 11 is greater than or equal to the threshold, the target tracking parameter control module 21 uses the statistical module 11 If the statistical information of the statistical information to adjust the current target tracking parameters, if the statistical credibility calculation module 12 determines that the credibility of the statistical information of the statistical module 11 is less than the threshold, the target tracking parameter control module 21 uses the default current Target tracking parameters. In addition, the adjustment of the current target tracking parameter mentioned here specifically refers to adjusting the update speed of the current target tracking parameter.
  • the target tracking feature calculation module 22 is configured to receive the judgment result of the target tracking parameter control module 21 to decide whether to read the sharpness information of each pixel in the image acquired by the sharpness calculation module 13 to compare The tracking feature used by the current target tracking method is switched. If the target tracking parameter control module 21 determines that the tracking feature used in the target tracking method needs to be switched, it reads the sharpness information of each pixel acquired by the sharpness calculation module 13 to switch the tracking feature pair.
  • the target tracking system is connected to an image input unit 3 through the image processing unit 1 to obtain the statistical information of the statistical module 11, the sensitivity information, and the image input unit 3 from the image input unit 3. Describe the sharpness information of each pixel and so on.
  • an embodiment of the present invention provides a target tracking method to adjust the update speed of the current target tracking parameters.
  • FIG. 2 is a schematic flowchart of a target tracking method for adjusting the update speed of current target tracking parameters provided by an embodiment of the present invention, including:
  • the comparison value is less than or equal to the threshold, keep the default current target tracking parameter update speed, and preferably, while keeping the default current target tracking parameter update speed, adjust the search range, first search the motion confidence area, and then Feature matching is performed in the area to reduce the search cost, and at the same time reduce interference and reduce the probability of target center coordinate drift.
  • the comparison value is greater than the threshold, it is necessary to determine whether it is caused by occlusion or sudden changes in ambient light or lens adjustment focal length. At this time, read the statistical information of the statistical module 11 to see whether the statistical information of the statistical module 11 occurs Change, adjust the current target tracking parameter update speed according to the change of the statistical information of the statistical module 11.
  • adjusting the current target tracking parameter update speed according to the change of the statistical information of the statistical module 11 includes:
  • the statistical information of the statistical module 11 changes, it is determined whether the tracking feature used in the current tracking method is affected by the change of the statistical information of the statistical module 11. If so, the update speed of the current target tracking parameters is reduced, if not, Keep the default current target tracking parameter update speed.
  • the statistical information of the statistical module 11 read by the target tracking unit 2 includes the brightness, chroma, and sharpness of the image. If there is no change in the degree of statistical information, it is determined that the drastic changes in the motion confidence area are likely to be caused by occlusion. At this time, the update speed of the current target tracking parameters is reduced to prevent the tracking features used in the target tracking method from being updated. Incorporate features of occluded objects.
  • the front-end camera is likely to perform automatic exposure, automatic white balance, or automatic focus/zoom operations.
  • the environmental conditions corresponding to the statistical information are changing at this time.
  • the overall brightness statistics information corresponding to auto exposure changes, the scene illumination is considered to be changing; the color statistics corresponding to the auto white balance overall change, then the scene color temperature is considered to be changing; the sharpness statistics corresponding to the auto focus change as a whole, and the scene is considered The sharpness is changing.
  • the tracking feature only contains sharpness-related information, it means that the tracking feature is only affected by the sharpness statistics.
  • the parameter update speed is reduced when the sharpness statistics change overall, and when the automatic exposure or automatic white balance statistics change Use the default parameter update speed.
  • the parameter update speed will be reduced when the brightness or color statistics change as a whole, and the default parameter update speed will be used when the sharpness statistics change.
  • filtering out the motion confidence area in the current target search area according to the current motion information of the tracking target includes:
  • the circumscribed rectangle of the current motion information of the tracking target after the filtering operation is calculated to obtain the motion confidence region.
  • the filtering operation includes median filtering, corrosion and/or expansion and other processing.
  • FIG. 4 is a schematic diagram of screening motion confidence regions.
  • 4a illustrates the current motion information of the tracking target after the filtering operation
  • 4b illustrates the tracking target after the filtering operation
  • the circumscribed rectangle of the current motion information is the motion confidence area obtained by filtering.
  • the target tracking unit 2 is used in conjunction with the image processing unit 1, and the current target is tracked based on the statistical information of the image characteristic parameters obtained by the image processing unit 1.
  • the update speed of the parameters is adjusted to adapt to different application scenarios to prevent the target tracking coordinate drift and improve the target tracking accuracy.
  • the embodiment of the present invention also provides another target tracking method to supplement the tracking feature.
  • FIG. 3 is a schematic flowchart of a target tracking method for switching tracking features provided by an embodiment of the present invention, including:
  • the front-end camera switches from color mode to black and white mode due to changes in ambient lighting, the original three-channel color feature will be switched to a single-channel brightness feature, resulting in decreased tracking accuracy.
  • feature channels can be added without increasing additional calculation costs, and the tracking effect under extreme conditions can be enhanced.
  • the tracking device and method provided by the present invention improve the robustness and accuracy of target tracking.

Abstract

A target tracking system and method. The target tracking system comprises an image processing unit (1) and a target tracking unit (2). The target tracking unit (2) adjusts the update speed of a target tracking parameter by reading the brightness, chroma and sharpness of each frame of image which are subjected to the statistics of the image processing unit (1), or the chroma and brightness of each frame of image subjected to the statistics thereof, and by means of the calculated current motion information of a tracked object, and determines, according to the statistical information of the image processing unit (1), whether it is necessary to switch tracked features used by the target tracking method; and if so, reads sharpness information of images acquired by the image processing unit (1) so as to switch the tracked features. In this way, the robustness and accuracy of target tracking are improved by means of the combination of the image processing unit (1) and the target tracking unit (2).

Description

目标跟踪系统及方法Target tracking system and method 技术领域Technical field
本发明涉及图像处理技术领域，特别涉及一种目标跟踪系统及方法。The present invention relates to the technical field of image processing, in particular to a target tracking system and method.
背景技术Background technique
目标跟踪单元是智能摄像机等前端设备中的常见功能模块。在实际应用中该单元一般属于智能分析引擎的一部分，在接收到图像处理单元(ISP)的输出视频信号后对其进行分析处理，实现对关注目标的运动跟踪或进一步的行为分析。The target tracking unit is a common functional module in front-end equipment such as smart cameras. In practical applications, this unit generally belongs to a part of the intelligent analysis engine. After receiving the output video signal of the image processing unit (ISP), it analyzes and processes it to realize the movement tracking or further behavior analysis of the target of interest.
目标跟踪单元的实现方法一般是先由目标检测模块自动检测或人工指定目标跟踪对象，然后对目标区域进行特征提取，在后续视频帧的搜索域内进行特征计算与匹配，随后更新目标跟踪相应参数。目标跟踪方法目前常用的有两大类特征：以《High-Speed Tracking with Kernelized Correlation Filters》为代表的梯度相关特征以及以《In defense of color-based model-free tracking》为代表的颜色相关特征。参数更新方法一般采用一阶平滑，而且特征匹配的搜索范围一般固定为目标大小的3倍范围内进行。The implementation method of the target tracking unit is generally to first automatically detect or manually specify the target tracking object by the target detection module, then perform feature extraction on the target area, perform feature calculation and matching in the search domain of subsequent video frames, and then update the corresponding target tracking parameters. There are currently two types of features commonly used in target tracking methods: gradient-related features represented by "High-Speed Tracking with Kernelized Correlation Filters" and color-related features represented by "In defense of color-based model-free tracking". The parameter update method generally adopts first-order smoothing, and the search range of feature matching is generally fixed within 3 times the target size.
上述目标跟踪方法最常见的问题是目标跟踪坐标漂移。其产生的原因有几点：1.搜索域中存在与目标特征相似的干扰物；2.目标特征由于物体被遮挡或者环境剧烈变化触发摄像机大幅度调整自动曝光/自动白平衡/自动对焦等原因发生剧烈变化。例如摄像机会自动在光照较低时开启补光并切换到黑白模式，此时方法《In defense of color-based model-free tracking》中的三维颜色直方图自动退化为一维亮度直方图，从而导致目标特征维数降低。The most common problem of the above target tracking method is the drift of target tracking coordinates. There are several reasons for this: 1. There are interference objects similar to the target feature in the search field; 2. The target feature is triggered by the object being blocked or the environment is drastically changed to trigger the camera to greatly adjust the automatic exposure/auto white balance/autofocus, etc. Dramatic changes have taken place. For example, the camera will automatically turn on the fill light and switch to the black and white mode when the light is low. At this time, the three-dimensional color histogram in the method "In defense of color-based model-free tracking" automatically degenerates into a one-dimensional brightness histogram, which leads to The target feature dimension is reduced.
为此专利如CN104537689B通过同时使用颜色、纹理、梯度直方图这种增加特征算子的方法来提升目标跟踪的鲁棒性与准确度。但性能提升的同时也大幅增加了实现代价。For this reason, patents such as CN104537689B improve the robustness and accuracy of target tracking by simultaneously using color, texture, and gradient histogram methods to increase feature operators. However, the performance improvement also greatly increases the implementation cost.
因此，如何提供一种高性价比的目标跟踪方法成了目前需要解决的问题。Therefore, how to provide a cost-effective target tracking method has become a problem that needs to be solved at present.
发明内容Summary of the invention
本发明的目的在于提供一种目标跟踪系统及方法，以提升目标跟踪的鲁棒性与准确度。The purpose of the present invention is to provide a target tracking system and method to improve the robustness and accuracy of target tracking.
为解决上述技术问题，本发明提供一种目标跟踪系统，所述目标跟踪系统包括图像处理单元和目标跟踪单元，所述图像处理单元包括统计模块、统计可信度计算模块、锐度计算模块、帧缓存模块和运动自适应判定模块，所述目标跟踪单元包括目标跟踪参数控制模块和目标跟踪特征计算模块；To solve the above technical problems, the present invention provides a target tracking system. The target tracking system includes an image processing unit and a target tracking unit. The image processing unit includes a statistical module, a statistical credibility calculation module, and a sharpness calculation module. A frame buffer module and a motion adaptive determination module, the target tracking unit includes a target tracking parameter control module and a target tracking feature calculation module;
所述统计模块用于统计每帧图像的亮度、色度和锐度，或用于统计每帧图像的色度和亮度；The statistics module is used to count the brightness, chroma and sharpness of each frame of image, or to count the chroma and brightness of each frame of image;
所述统计可信度计算模块用于获取感光度信息以判断所述统计模块的统计信息是否可信，并将判断的结果传输给所述当前目标跟踪参数控制模块；The statistical credibility calculation module is configured to obtain sensitivity information to determine whether the statistical information of the statistical module is credible, and transmit the result of the judgment to the current target tracking parameter control module;
所述锐度计算模块用于得到图像中每个像素点的锐度信息；The sharpness calculation module is used to obtain the sharpness information of each pixel in the image;
所述帧缓存模块用于缓存累加帧信息；The frame buffer module is used for buffering accumulated frame information;
所述运动自适应判定模块用于获取当前帧图像的帧信息并从所述帧缓存模块中读取累加帧信息以得到跟踪目标当前的运动信息；The motion adaptive determination module is used to obtain frame information of the current frame image and read the accumulated frame information from the frame buffer module to obtain the current motion information of the tracking target;
所述目标跟踪参数控制模块用于在对目标进行跟踪的同时在接收到所述统计可信度计算模块的判断结果为可信之后，读取所述统计信息和所述跟踪目标当前的运动信息，根据所述统计信息和所述跟踪目标当前的运动信息的变化调整当前目标跟踪参数以及判断是否切换当前目标跟踪方法所用跟踪特征并将判断的结果发送给所述目标跟踪特征计算模块；The target tracking parameter control module is configured to read the statistical information and the current movement information of the tracking target after receiving the judgment result of the statistical credibility calculation module as being credible while tracking the target Adjusting the current target tracking parameters according to the statistical information and the current movement information of the tracking target and determining whether to switch the tracking feature used in the current target tracking method and sending the result of the determination to the target tracking feature calculation module;
所述目标跟踪特征计算模块用于接收所述目标跟踪参数控制模块的判断结果，以决定是否读取所述锐度计算模块获取的图像中每个像素点的锐度信息以对当前目标跟踪方法所用跟踪特征进行切换。The target tracking feature calculation module is used to receive the judgment result of the target tracking parameter control module to determine whether to read the sharpness information of each pixel in the image acquired by the sharpness calculation module to track the current target The tracking feature used is switched.
可选的，在所述的目标跟踪系统中，所述统计模块为3A统计模块，用于统计图像的亮度、色度和锐度。Optionally, in the target tracking system, the statistics module is a 3A statistics module for statistics of brightness, chroma, and sharpness of the image.
可选的，在所述的目标跟踪系统中，所述统计模块为2A统计模块，用于统计图像的亮度和色度。Optionally, in the target tracking system, the statistics module is a 2A statistics module, which is used to count the brightness and chromaticity of the image.
可选的，在所述的目标跟踪系统中，所述跟踪特征包括颜色、纹理和/或梯 度直方图。Optionally, in the target tracking system, the tracking features include color, texture, and/or gradient histogram.
可选的，在所述的目标跟踪系统中，所述感光度信息包括曝光时间、模拟增益和数字增益。Optionally, in the target tracking system, the sensitivity information includes exposure time, analog gain, and digital gain.
本发明还提供一种目标跟踪方法，包括：The present invention also provides a target tracking method, including:
读取所述运动自适应判定模块得到的所述跟踪目标当前的运动信息，根据所述跟踪目标当前的运动信息，在当前目标搜索区域内筛选出运动置信区域；Reading the current motion information of the tracking target obtained by the motion adaptive determination module, and filtering out the motion confidence region in the current target search area according to the current motion information of the tracking target;
计算所述运动置信区域的面积，并和上一帧运动置信区域的面积比较；Calculate the area of the motion confidence region and compare it with the area of the motion confidence region in the previous frame;
若比较值小于或等于阈值，则保持默认当前目标跟踪参数更新速度；If the comparison value is less than or equal to the threshold, keep the default current target tracking parameter update speed;
若比较值大于阈值，则读取所述统计模块的统计信息，并根据所述统计模块的统计信息的变化情况调整当前目标跟踪参数更新速度。If the comparison value is greater than the threshold, read the statistical information of the statistical module, and adjust the current target tracking parameter update speed according to the change of the statistical information of the statistical module.
可选的，在所述的目标跟踪方法中，根据所述统计模块的统计信息的变化情况调整当前目标跟踪参数更新速度包括：Optionally, in the target tracking method, adjusting the current target tracking parameter update speed according to changes in the statistical information of the statistics module includes:
若所述统计模块的统计信息没有发生变化，则降低当前目标跟踪参数的更新速度；If the statistical information of the statistical module does not change, reduce the update speed of the current target tracking parameters;
若所述统计模块的统计信息发生变化，则判断当前跟踪方法所用跟踪特征是否受所述统计模块的统计信息的变化的影响，若是，则降低当前目标跟踪参数的更新速度，若否，则保持当前目标跟踪参数更新速度。If the statistical information of the statistical module changes, it is determined whether the tracking feature used in the current tracking method is affected by the change of the statistical information of the statistical module, if yes, then the update speed of the current target tracking parameter is reduced, if not, keep Current target tracking parameter update speed.
可选的，在所述的目标跟踪方法中，根据所述跟踪目标当前的运动信息在当前目标搜索区域内筛选出运动置信区域包括：Optionally, in the target tracking method, filtering out the motion confidence region in the current target search area according to the current motion information of the tracking target includes:
将所述跟踪目标当前的运动信息在空域及时域上进行滤波操作；以及Filtering the current motion information of the tracking target in the airspace and time domain; and
计算经滤波操作后的所述跟踪目标当前的运动信息的外接矩形，以得到所述运动置信区域。可选的，在所述的目标跟踪方法中，若比较值小于或等于阈值，保持当前目标跟踪参数更新速度的同时，调整搜索范围，先搜索所述运动置信区域。The circumscribed rectangle of the current motion information of the tracking target after the filtering operation is calculated to obtain the motion confidence region. Optionally, in the target tracking method, if the comparison value is less than or equal to the threshold value, while maintaining the current target tracking parameter update speed, the search range is adjusted and the motion confidence region is searched first.
本发明还提供另外一种目标跟踪方法，包括：The present invention also provides another target tracking method, including:
读取当前跟踪方法所用跟踪特征，判断所述跟踪特征是否只包括颜色信息；Read the tracking feature used in the current tracking method, and determine whether the tracking feature only includes color information;
若否，则不切换所述跟踪特征；If not, do not switch the tracking feature;
若是，则读取所述图像处理单元获取的图像特征参数的统计信息以判断图像是否为黑白模式，若是，则读取所述锐度计算模块获取的图像中每个像素点的 锐度信息，利用所述图像每人像素点的锐度信息将所述跟踪特征从三通道颜色切换为亮度、水平方向锐度和垂直方向锐度，若否，则将所述跟踪特征从亮度、水平方向锐度和垂直方向锐度切换成三通道颜色。If yes, read the statistical information of the image feature parameters obtained by the image processing unit to determine whether the image is in black and white mode, if yes, read the sharpness information of each pixel in the image obtained by the sharpness calculation module, Use the sharpness information of each pixel of the image to switch the tracking feature from three-channel color to brightness, horizontal sharpness, and vertical sharpness. If not, change the tracking feature from brightness to horizontal sharpness. The sharpness and vertical sharpness switch to three-channel color.
在本发明提供的目标跟踪系统及方法中，所述目标跟踪系统包括图像处理单元和目标跟踪单元，所述图像处理单元包括统计模块、统计可信度计算模块、锐度计算模块、运动自适应判定模块和帧缓存模块，所述目标跟踪单元包括目标跟踪参数控制模块和目标跟踪特征计算模块，所述目标跟踪参数控制模块通过读取所述统计模块的统计信息和所述运动自适应判定模块得到的跟踪目标的运动信息来调整当前目标跟踪参数的更新速度以对目标进行跟踪，以及通过读取所述统计模块的统计信息来判断是否需要切换目标跟踪方法所用跟踪特征，若需要，则所述目标跟特征计算模块读取所述统计可信度计算模块获取的图像每个像素点的锐度信息以对目标跟踪方法所用跟踪特征进行切换，如此，便通过所述图像处理单元和所述目标跟踪单元的联用提升了目标跟踪的鲁棒性与准确度。In the target tracking system and method provided by the present invention, the target tracking system includes an image processing unit and a target tracking unit, and the image processing unit includes a statistical module, a statistical credibility calculation module, a sharpness calculation module, and a motion adaptive A determination module and a frame buffer module. The target tracking unit includes a target tracking parameter control module and a target tracking feature calculation module. The target tracking parameter control module reads the statistical information of the statistics module and the motion adaptive determination module The obtained motion information of the tracking target is used to adjust the update speed of the current target tracking parameters to track the target, and to determine whether it is necessary to switch the tracking feature used in the target tracking method by reading the statistical information of the statistical module, and if necessary, The target and feature calculation module reads the sharpness information of each pixel of the image acquired by the statistical credibility calculation module to switch the tracking feature used in the target tracking method. In this way, the image processing unit and the The combination of the target tracking unit improves the robustness and accuracy of target tracking.
附图说明Description of the drawings
图1是本发明实施例的目标跟踪系统的连接关系示意图；FIG. 1 is a schematic diagram of the connection relationship of a target tracking system according to an embodiment of the present invention;
图2是本发明实施例提供的一种目标跟踪方法的过程示意图；2 is a schematic diagram of the process of a target tracking method provided by an embodiment of the present invention;
图3是本发明实施例提供的另一种目标跟踪方法的过程示意图；3 is a schematic diagram of another target tracking method provided by an embodiment of the present invention;
图4是本发明实施例中的筛选运动置信区域的示意图；Figure 4 is a schematic diagram of screening motion confidence regions in an embodiment of the present invention;
其中，各附图标记说明如下：Among them, the reference signs are described as follows:
1-图像处理单元；11-统计模块；12-统计可信度计算模块；13-锐度计算模块；14-帧缓存模块；15-运动自适应判定模块；2-目标跟踪单元；21-目标跟踪参数控制模块；22-目标跟踪特征计算模块；3-图像输入单元。1-image processing unit; 11-statistics module; 12-statistical credibility calculation module; 13-sharpness calculation module; 14-frame buffer module; 15-motion adaptive determination module; 2-target tracking unit; 21-target Tracking parameter control module; 22-target tracking feature calculation module; 3-image input unit.
具体实施方式Detailed ways
以下结合附图和具体实施例对本发明提出的目标跟踪系统及方法作进一步详细说明。需说明的是，附图均采用非常简化的形式且均使用非精准的比例，仅用以方便、明晰地辅助说明本发明实施例的目的。此外，附图所展示的结构往往是实际结构的一部分。特别的，各附图需要展示的侧重点不同，有时会采用不同 的比例。The target tracking system and method proposed by the present invention will be described in further detail below in conjunction with the drawings and specific embodiments. It should be noted that the drawings are in a very simplified form and all use imprecise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often part of the actual structure. In particular, the focus of each drawing is different, and different scales are sometimes used.
首先，请参阅图1，其为本发明实施例提供的目标跟踪系统的连接关系示意图，本发明提供的目标跟踪系统包括图像处理单元1和目标跟踪单元2。First, please refer to FIG. 1, which is a schematic diagram of the connection relationship of the target tracking system provided by the embodiment of the present invention. The target tracking system provided by the present invention includes an image processing unit 1 and a target tracking unit 2.
所述图像处理单元1包括统计模块11、统计可信度计算模块12、锐度计算模块13、运动自适应判定模块15帧缓存模块1414和运动自适应判定模块15；所述目标跟踪单元2包括目标跟踪参数控制模块21和目标跟踪特征计算模块22。The image processing unit 1 includes a statistical module 11, a statistical credibility calculation module 12, a sharpness calculation module 13, a motion adaptive determination module 15, a frame buffer module 1414, and a motion adaptive determination module 15; the target tracking unit 2 includes The target tracking parameter control module 21 and the target tracking feature calculation module 22.
所述统计模块11，用于统计每帧图像的亮度、色度和锐度，或用于统计每帧图像的色度和锐度；具体的，所述统计模块11可为3A统计模块或2A模块，若所述图像处理单元1支持自动对焦(AF)，则所述统计模块11为3A统计模块，用于统计图像的亮度、色度和锐度，若所述图像处理单元1不支持AF，则所述统计模块11为2A统计模块，用于统计图像的亮度和色度。The statistics module 11 is used to count the brightness, chroma and sharpness of each frame of images, or to count the chroma and sharpness of each frame of images; specifically, the statistics module 11 may be a 3A statistics module or a 2A statistics module. Module, if the image processing unit 1 supports auto focus (AF), the statistics module 11 is a 3A statistics module, used to count the brightness, chroma and sharpness of the image, if the image processing unit 1 does not support AF , Then the statistics module 11 is a 2A statistics module, used to count the brightness and chromaticity of the image.
所述统计可信度计算模块12，用于获取感光度信息以判断所述统计模块11的统计信息是否可信，并将判断的结果传输给所述目标跟踪参数控制模块21。所述感光度信息包括曝光时间、模拟增益和数字增益。判断的过程包括：对曝光时间、模拟增益和数字增益进行换算，若换算后的指标小于阈值，则认为所述统计模块11的统计信息可信。比如一个典型的阈值是5000，即若换算后的指标小于5000，则认为所述统计模块11的统计信息可信。The statistical credibility calculation module 12 is configured to obtain sensitivity information to determine whether the statistical information of the statistical module 11 is credible, and transmit the judgment result to the target tracking parameter control module 21. The sensitivity information includes exposure time, analog gain, and digital gain. The judging process includes: converting the exposure time, analog gain, and digital gain, and if the converted index is less than the threshold, the statistical information of the statistical module 11 is considered credible. For example, a typical threshold is 5000, that is, if the converted index is less than 5000, the statistical information of the statistical module 11 is considered to be credible.
所述锐度计算模块13，用于得到图像中每个像素点的锐度信息。与所述统计模块11得到的锐度信息不同的是，所述统计模块11获取的整个图像的锐度信息，而所述锐度计算模块13得到的锐度信息是每个像素点的锐度信息，比较来说，精度更高。The sharpness calculation module 13 is used to obtain the sharpness information of each pixel in the image. The difference from the sharpness information obtained by the statistical module 11 is that the sharpness information of the entire image obtained by the statistical module 11, and the sharpness information obtained by the sharpness calculation module 13 is the sharpness of each pixel. Information, in comparison, is more accurate.
所述帧缓存模块14用于缓存所述统计模块11统计的累加帧信息；所述累加帧信息具体可通过时域滤波预存在所述帧缓存模块14中。The frame buffer module 14 is used to buffer the accumulated frame information counted by the statistics module 11; the accumulated frame information may specifically be pre-stored in the frame buffer module 14 through time-domain filtering.
所述运动自适应判定模块15，用于获取当前帧图像的帧信息并从所述帧缓存模块14中读取累加帧信息以得到跟踪目标当前的运动信息。The motion adaptive determination module 15 is used to obtain the frame information of the current frame image and read the accumulated frame information from the frame buffer module 14 to obtain the current motion information of the tracking target.
所述目标跟踪参数控制模块21，用于在对目标进行跟踪的同时在接收到所述统计可信度计算模块12的判断结果为可信之后，读取所述统计信息和所述跟踪目标当前的运动信息，根据所述统计信息和所述跟踪目标当前的运动信息的 变化调整当前目标跟踪参数，以及判断是否切换当前目标跟踪方法所用跟踪特征。所述跟踪特征包括颜色、纹理和/或梯度直方图。The target tracking parameter control module 21 is configured to read the statistical information and the current tracking target after receiving the judgment result of the statistical credibility calculation module 12 as being credible while tracking the target. The current target tracking parameter is adjusted according to the statistical information and the current motion information of the tracking target, and the tracking feature used in the current target tracking method is judged whether to switch. The tracking features include color, texture, and/or gradient histograms.
所述目标跟踪参数控制模块21根据所述统计模块11的统计信息和所述跟踪目标当前的运动信息的变化调整当前目标跟踪参数的前提是所述统计模块11的统计信息的可信度大于阈值，具体的，若所述统计可信度计算模块12判断的结果是所述统计模块11的统计信息的可信度大于或等于阈值，则所述目标跟踪参数控制模块21利用所述统计模块11的统计信息调整当前目标跟踪参数，若所述统计可信度计算模块12判断的结果是所述统计模块11的统计信息的可信度小于阈值，则所述目标跟踪参数控制模块21使用默认当前目标跟踪参数。另外，这里所述调整当前目标跟踪参数具体是指调整当前目标跟踪参数更新速度。The premise that the target tracking parameter control module 21 adjusts the current target tracking parameters according to the statistical information of the statistical module 11 and the current movement information of the tracking target is that the credibility of the statistical information of the statistical module 11 is greater than a threshold. Specifically, if the result of the determination by the statistical credibility calculation module 12 is that the credibility of the statistical information of the statistical module 11 is greater than or equal to the threshold, the target tracking parameter control module 21 uses the statistical module 11 If the statistical information of the statistical information to adjust the current target tracking parameters, if the statistical credibility calculation module 12 determines that the credibility of the statistical information of the statistical module 11 is less than the threshold, the target tracking parameter control module 21 uses the default current Target tracking parameters. In addition, the adjustment of the current target tracking parameter mentioned here specifically refers to adjusting the update speed of the current target tracking parameter.
所述目标跟踪特征计算模块22，用于接收所述目标跟踪参数控制模块21的判断结果，以决定是否读取所述锐度计算模块13获取的图像中每个像素点的锐度信息以对当前目标跟踪方法所用跟踪特征进行切换。若所述目标跟踪参数控制模块21判断需要切换目标跟踪方法所用跟踪特征，则读取所述锐度计算模块13获取的每个像素点的锐度信息对所述跟踪特征对进行切换。The target tracking feature calculation module 22 is configured to receive the judgment result of the target tracking parameter control module 21 to decide whether to read the sharpness information of each pixel in the image acquired by the sharpness calculation module 13 to compare The tracking feature used by the current target tracking method is switched. If the target tracking parameter control module 21 determines that the tracking feature used in the target tracking method needs to be switched, it reads the sharpness information of each pixel acquired by the sharpness calculation module 13 to switch the tracking feature pair.
实际应用时，所述目标跟踪系统通过所述图像处理单元1与一图像输入单元3相连，以从所述图像输入单元3中获取所述统计模块11的统计信息、所述感光度信息、所述每个像素点的锐度信息等等。In actual application, the target tracking system is connected to an image input unit 3 through the image processing unit 1 to obtain the statistical information of the statistical module 11, the sensitivity information, and the image input unit 3 from the image input unit 3. Describe the sharpness information of each pixel and so on.
基于以上所述目标跟踪系统，本发明实施例提供一种目标跟踪方法，以调整当前目标跟踪参数的更新速度。Based on the target tracking system described above, an embodiment of the present invention provides a target tracking method to adjust the update speed of the current target tracking parameters.
请参阅图2，其为本发明实施例提供的用以调整当前目标跟踪参数的更新速度的目标跟踪方法的流程示意图，包括：Please refer to FIG. 2, which is a schematic flowchart of a target tracking method for adjusting the update speed of current target tracking parameters provided by an embodiment of the present invention, including:
读取所述运动自适应判定模块15得到的所述跟踪目标当前的运动信息，根据所述跟踪目标当前的运动信息，在当前目标搜索区域内筛选出运动置信区域；Read the current motion information of the tracking target obtained by the motion adaptive determination module 15, and filter out the motion confidence region in the current target search area according to the current motion information of the tracking target;
计算所述运动置信区域的面积，并和上一帧运动置信区域的面积比较；Calculate the area of the motion confidence region and compare it with the area of the motion confidence region in the previous frame;
若比较值小于或等于阈值，则保持默认当前目标跟踪参数更新速度，且优选地，保持默认当前目标跟踪参数更新速度的同时，调整搜索范围，先搜索所述运动置信区域，在所述运动置信区域内进行特征匹配，达到降低搜索代价，同时除低干扰，减少目标中心坐标漂移概率的目的。If the comparison value is less than or equal to the threshold, keep the default current target tracking parameter update speed, and preferably, while keeping the default current target tracking parameter update speed, adjust the search range, first search the motion confidence area, and then Feature matching is performed in the area to reduce the search cost, and at the same time reduce interference and reduce the probability of target center coordinate drift.
若比较值大于阈值，则需要判断是由于遮挡导致还是环境光照突变或者镜头调节焦距等原因导致的，此时读取所述统计模块11的统计信息，看所述统计模块11的统计信息是否发生变化，根据所述统计模块11的统计信息的变化情况调整当前目标跟踪参数更新速度。If the comparison value is greater than the threshold, it is necessary to determine whether it is caused by occlusion or sudden changes in ambient light or lens adjustment focal length. At this time, read the statistical information of the statistical module 11 to see whether the statistical information of the statistical module 11 occurs Change, adjust the current target tracking parameter update speed according to the change of the statistical information of the statistical module 11.
其中，根据所述统计模块11的统计信息的变化情况调整当前目标跟踪参数更新速度包括：Wherein, adjusting the current target tracking parameter update speed according to the change of the statistical information of the statistical module 11 includes:
若所述统计模块11的统计信息没有发生变化，则降低当前目标跟踪参数的更新速度；If the statistical information of the statistical module 11 does not change, reduce the update speed of the current target tracking parameters;
若所述统计模块11的统计信息发生变化，则判断当前跟踪方法所用跟踪特征是否受所述统计模块11的统计信息的变化的影响，若是，则降低当前目标跟踪参数的更新速度，若否，则保持默认当前目标跟踪参数更新速度。If the statistical information of the statistical module 11 changes, it is determined whether the tracking feature used in the current tracking method is affected by the change of the statistical information of the statistical module 11. If so, the update speed of the current target tracking parameters is reduced, if not, Keep the default current target tracking parameter update speed.
例如，对于支持AF的所述图像处理单元1，所述目标跟踪单元2读取的所述统计模块11的统计信息包括图像的亮度、色度和锐度，若图像的亮度、色度和锐度统计信息均没有发生变化，则认定所述运动置信区域的剧烈变化很大概率上是由于遮挡导致的，此时降低当前目标跟踪参数的更新速度以防止目标跟踪方法所用跟踪特征在更新过程中融入遮挡物体的特征。For example, for the image processing unit 1 supporting AF, the statistical information of the statistical module 11 read by the target tracking unit 2 includes the brightness, chroma, and sharpness of the image. If there is no change in the degree of statistical information, it is determined that the drastic changes in the motion confidence area are likely to be caused by occlusion. At this time, the update speed of the current target tracking parameters is reduced to prevent the tracking features used in the target tracking method from being updated. Incorporate features of occluded objects.
若图像的亮度、色度或锐度统计信息发生变化，则认定前端摄像机大概率或即将进行自动曝光、自动白平衡或自动对焦/变焦操作。此时需要进一步判断跟踪方法所用跟踪特征是否受亮度、色度和/或锐度统计信息的变化的影响，若是，则降低当前目标跟踪参数的更新速度，若否，则保持默认当前目标跟踪参数更新速度。If the brightness, chroma, or sharpness statistics of the image change, it is determined that the front-end camera is likely to perform automatic exposure, automatic white balance, or automatic focus/zoom operations. At this time, it is necessary to further determine whether the tracking features used in the tracking method are affected by the changes in brightness, chroma and/or sharpness statistics. If so, reduce the update speed of the current target tracking parameters, if not, keep the default current target tracking parameters Update Speed.
具体地，若所述统计模块11的统计信息中亮度、色度和/或锐度统计信息至少有一个在发生变化，则认为此时该统计信息对应的环境条件在发生变化。如自动曝光对应的亮度统计信息整体变化，则认为场景照度在变化；自动白平衡对应的颜色统计信息整体变化，则认为场景色温在变化；自动对焦对应的锐度统计信息整体变化，则认为场景锐度在变化。此时需要根据当前目标跟踪方法所述跟踪特征进一步判断。若跟踪特征中只包含锐度相关信息，则说明跟踪特征只受锐度统计信息的影响，故在锐度统计信息整体变化时降低参数更新速度，而在自动曝光或自动白平衡统计信息变化时使用默认参数更新速度。类似地，若跟踪特征中 只包含颜色相关信息，则在亮度或颜色统计信息整体发生变化时降低参数更新速度，而在锐度统计信息变化时使用默认参数更新速度。Specifically, if at least one of the brightness, chroma, and/or sharpness statistical information in the statistical information of the statistical module 11 is changing, it is considered that the environmental conditions corresponding to the statistical information are changing at this time. If the overall brightness statistics information corresponding to auto exposure changes, the scene illumination is considered to be changing; the color statistics corresponding to the auto white balance overall change, then the scene color temperature is considered to be changing; the sharpness statistics corresponding to the auto focus change as a whole, and the scene is considered The sharpness is changing. At this time, it is necessary to further judge according to the tracking characteristics of the current target tracking method. If the tracking feature only contains sharpness-related information, it means that the tracking feature is only affected by the sharpness statistics. Therefore, the parameter update speed is reduced when the sharpness statistics change overall, and when the automatic exposure or automatic white balance statistics change Use the default parameter update speed. Similarly, if the tracking feature only contains color-related information, the parameter update speed will be reduced when the brightness or color statistics change as a whole, and the default parameter update speed will be used when the sharpness statistics change.
另外，根据所述跟踪目标当前的运动信息在当前目标搜索区域内筛选出运动置信区域包括：In addition, filtering out the motion confidence area in the current target search area according to the current motion information of the tracking target includes:
将所述跟踪目标当前的运动信息在空域及时域上进行滤波操作；以及Filtering the current motion information of the tracking target in the airspace and time domain; and
计算经滤波操作后的所述跟踪目标当前的运动信息的外接矩形，以得到所述运动置信区域。The circumscribed rectangle of the current motion information of the tracking target after the filtering operation is calculated to obtain the motion confidence region.
其中，所述滤波操作包括中值滤波、腐蚀和/或膨胀等处理。Wherein, the filtering operation includes median filtering, corrosion and/or expansion and other processing.
请参考图4，图4为筛选运动置信区域的示意图，图4中4a示意了经滤波操作后的所述跟踪目标当前的运动信息，相应的，4b示意了经滤波操作后的所述跟踪目标当前的运动信息的外接矩形，即筛选得到的运动置信区域。Please refer to FIG. 4, which is a schematic diagram of screening motion confidence regions. In FIG. 4, 4a illustrates the current motion information of the tracking target after the filtering operation, and correspondingly, 4b illustrates the tracking target after the filtering operation The circumscribed rectangle of the current motion information is the motion confidence area obtained by filtering.
本发明实施例提供的所述目标跟踪方法，通过将所述目标跟踪单元2与所述图像处理单元1联用，基于所述图像处理单元1获取的图像特征参数的统计信息，对当前目标跟踪参数的更新速度进行调整以适应不同的应用场景，防止目标跟踪坐标漂移，提高了目标跟踪精度。According to the target tracking method provided by the embodiment of the present invention, the target tracking unit 2 is used in conjunction with the image processing unit 1, and the current target is tracked based on the statistical information of the image characteristic parameters obtained by the image processing unit 1. The update speed of the parameters is adjusted to adapt to different application scenarios to prevent the target tracking coordinate drift and improve the target tracking accuracy.
此外，基于以上所述目标跟踪系统，本发明实施例还提供另一种目标跟踪方法，以补充跟踪特征。In addition, based on the above-mentioned target tracking system, the embodiment of the present invention also provides another target tracking method to supplement the tracking feature.
请参阅图3，其为本发明实施例提供的用以切换跟踪特征的目标跟踪方法的流程示意图，包括：Please refer to FIG. 3, which is a schematic flowchart of a target tracking method for switching tracking features provided by an embodiment of the present invention, including:
读取当前跟踪方法所用跟踪特征，判断所述跟踪特征是否只包括颜色信息；Read the tracking feature used in the current tracking method, and determine whether the tracking feature only includes color information;
若否，则不切换所述跟踪特征；If not, do not switch the tracking feature;
若是，则读取所述图像处理单元1获取的图像特征参数的统计信息以判断图像是否为黑白模式，若是，则读取锐度计算模块13所述锐度计算模块13获取的图像每个像素点的锐度信息，利用所述图像每人像素点的锐度信息将所述跟踪特征从三通道颜色(红色、黄色、蓝色)切换为亮度、水平方向锐度和垂直方向锐度，若否，则将所述跟踪特征从亮度、水平方向锐度和垂直方向锐度切换成三通道颜色。If yes, read the statistical information of the image characteristic parameters obtained by the image processing unit 1 to determine whether the image is in black and white mode, if yes, read each pixel of the image obtained by the sharpness calculation module 13 Point sharpness information, using the sharpness information of each pixel of the image to switch the tracking feature from three-channel color (red, yellow, blue) to brightness, horizontal sharpness and vertical sharpness, if If not, switch the tracking feature from brightness, horizontal sharpness and vertical sharpness to three-channel color.
当前端摄像机由于环境光照变化由彩色模式切换为黑白模式时，原来的三通道颜色特征会切换为单通道亮度特征，导致跟踪精度下降。此时复用所述锐度 计算单元获取的每个像素点的锐度统计信息，可以在不增加额外运算代价的前提下增加特征通道，增强极限条件下的跟踪效果。When the front-end camera switches from color mode to black and white mode due to changes in ambient lighting, the original three-channel color feature will be switched to a single-channel brightness feature, resulting in decreased tracking accuracy. At this time, by reusing the sharpness statistical information of each pixel obtained by the sharpness calculation unit, feature channels can be added without increasing additional calculation costs, and the tracking effect under extreme conditions can be enhanced.
综上所述，本发明提供的跟踪装置及方式提升了目标跟踪的鲁棒性与准确度。In summary, the tracking device and method provided by the present invention improve the robustness and accuracy of target tracking.
上述描述仅是对本发明较佳实施例的描述，并非对本发明范围的任何限定，本发明领域的普通技术人员根据上述揭示内容做的任何变更、修饰，均属于权利要求书的保护范围。The foregoing description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention in any way. Any changes or modifications made by persons of ordinary skill in the field of the present invention based on the foregoing disclosure shall fall within the protection scope of the claims.

Claims (10)

  1. 一种目标跟踪系统，其特征在于，所述目标跟踪系统包括图像处理单元和目标跟踪单元，所述图像处理单元包括统计模块、统计可信度计算模块、锐度计算模块、帧缓存模块和运动自适应判定模块，所述目标跟踪单元包括目标跟踪参数控制模块和目标跟踪特征计算模块；A target tracking system, wherein the target tracking system includes an image processing unit and a target tracking unit, and the image processing unit includes a statistical module, a statistical credibility calculation module, a sharpness calculation module, a frame buffer module, and a motion An adaptive determination module, the target tracking unit includes a target tracking parameter control module and a target tracking feature calculation module;
    所述统计模块用于统计每帧图像的亮度、色度和锐度，或用于统计每帧图像的色度和亮度；The statistics module is used to count the brightness, chroma and sharpness of each frame of image, or to count the chroma and brightness of each frame of image;
    所述统计可信度计算模块用于获取感光度信息以判断所述统计模块的统计信息是否可信，并将判断的结果传输给所述目标跟踪参数控制模块；The statistical credibility calculation module is configured to obtain sensitivity information to determine whether the statistical information of the statistical module is credible, and transmit the result of the judgment to the target tracking parameter control module;
    所述锐度计算模块用于得到图像中每个像素点的锐度信息；The sharpness calculation module is used to obtain the sharpness information of each pixel in the image;
    所述帧缓存模块用于缓存累加帧信息；The frame buffer module is used for buffering accumulated frame information;
    所述运动自适应判定模块用于获取当前帧图像的帧信息并从所述帧缓存模块中读取累加帧信息以得到跟踪目标当前的运动信息；The motion adaptive determination module is used to obtain frame information of the current frame image and read the accumulated frame information from the frame buffer module to obtain the current motion information of the tracking target;
    所述目标跟踪参数控制模块用于在对目标进行跟踪的同时在接收到所述统计可信度计算模块的判断结果为可信之后，读取所述统计信息和所述跟踪目标当前的运动信息，根据所述统计信息和所述跟踪目标当前的运动信息的变化调整当前目标跟踪参数以及判断是否切换当前目标跟踪方法所用跟踪特征并将判断的结果发送给所述目标跟踪特征计算模块；The target tracking parameter control module is configured to read the statistical information and the current movement information of the tracking target after receiving the judgment result of the statistical credibility calculation module to be credible while tracking the target , Adjust the current target tracking parameters according to the statistical information and the current movement information of the tracking target, determine whether to switch the tracking feature used in the current target tracking method, and send the determined result to the target tracking feature calculation module;
    所述目标跟踪特征计算模块用于接收所述目标跟踪参数控制模块的判断结果，以决定是否读取所述锐度计算模块获取的图像中每个像素点的锐度信息以对当前目标跟踪方法所用跟踪特征进行切换。The target tracking feature calculation module is used to receive the judgment result of the target tracking parameter control module to determine whether to read the sharpness information of each pixel in the image acquired by the sharpness calculation module to track the current target The tracking feature used is switched.
  2. 如权利要求1所述的目标跟踪系统，其特征在于，所述统计模块为3A统计模块，用于统计图像的亮度、色度和锐度。3. The target tracking system according to claim 1, wherein the statistics module is a 3A statistics module for statistics of brightness, chromaticity and sharpness of the image.
  3. 如权利要求1所述的目标跟踪系统，其特征在于，所述统计模块为2A统计模块，用于统计图像的亮度和色度。8. The target tracking system according to claim 1, wherein the statistics module is a 2A statistics module for statistics of brightness and chromaticity of the image.
  4. 如权利要求1所述的目标跟踪系统，其特征在于，所述跟踪特征包括颜色、纹理和/或梯度直方图。The target tracking system according to claim 1, wherein the tracking feature includes color, texture and/or gradient histogram.
  5. 如权利要求1所述的目标跟踪系统，其特征在于，所述感光度信息包括曝光时间、模拟增益和数字增益。8. The target tracking system of claim 1, wherein the sensitivity information includes exposure time, analog gain, and digital gain.
  6. 一种采用如权利要求1～5中任一项所述的目标跟踪系统的目标跟踪方法，其特征在于，包括：A target tracking method using the target tracking system according to any one of claims 1 to 5, characterized in that it comprises:
    读取所述运动自适应判定模块得到的所述跟踪目标当前的运动信息，根据所述跟踪目标当前的运动信息，在当前目标搜索区域内筛选出运动置信区域；Reading the current motion information of the tracking target obtained by the motion adaptive determination module, and filtering out the motion confidence region in the current target search area according to the current motion information of the tracking target;
    计算所述运动置信区域的面积，并和上一帧运动置信区域的面积比较；Calculate the area of the motion confidence region and compare it with the area of the motion confidence region in the previous frame;
    若比较值小于或等于阈值，则保持当前目标跟踪参数更新速度；If the comparison value is less than or equal to the threshold, keep the current target tracking parameter update speed;
    若比较值大于阈值，则读取所述统计模块的统计信息，并根据所述统计模块的统计信息的变化情况调整当前目标跟踪参数更新速度。If the comparison value is greater than the threshold, read the statistical information of the statistical module, and adjust the current target tracking parameter update speed according to the change of the statistical information of the statistical module.
  7. 如权利要求6所述的目标跟踪方法，其特征在于，根据所述统计模块的统计信息的变化情况调整当前目标跟踪参数更新速度包括：7. The target tracking method according to claim 6, wherein adjusting the current target tracking parameter update speed according to the change of the statistical information of the statistical module comprises:
    若所述统计模块的统计信息没有发生变化，则保持当前目标跟踪参数的更新速度；If the statistical information of the statistical module does not change, the update speed of the current target tracking parameters is maintained;
    若所述统计模块的统计信息发生变化，则判断当前跟踪方法所用跟踪特征是否受所述统计模块的统计信息的变化的影响，若是，则降低当前目标跟踪参数的更新速度，若否，则保持当前目标跟踪参数更新速度。If the statistical information of the statistical module changes, it is determined whether the tracking feature used in the current tracking method is affected by the change of the statistical information of the statistical module, if yes, then the update speed of the current target tracking parameter is reduced, if not, keep Current target tracking parameter update speed.
  8. 如权利要求6所述的目标跟踪方法，其特征在于，根据所述跟踪目标当前的运动信息在当前目标搜索区域内筛选出运动置信区域包括：The target tracking method according to claim 6, wherein the filtering out the motion confidence region in the current target search area according to the current motion information of the tracking target comprises:
    将所述跟踪目标当前的运动信息在空域及时域上进行滤波操作；以及Filtering the current motion information of the tracking target in the airspace and time domain; and
    计算经滤波操作后的所述跟踪目标当前的运动信息的外接矩形，以得到所述运动置信区域。(比如中值滤波或腐蚀、膨胀等处理；然后计算滤波平滑后的运动信息外接矩形，以得到所述置运动置信区域。)The circumscribed rectangle of the current motion information of the tracking target after the filtering operation is calculated to obtain the motion confidence region. (For example, median filtering or processing such as corrosion and expansion; then calculate the circumscribed rectangle of the filtered and smoothed motion information to obtain the motion confidence region.)
  9. 如权利要求6所述的目标跟踪方法，其特征在于，若比较值小于或等于阈值，保持当前目标跟踪参数更新速度的同时，调整搜索范围，先搜索所述运动置信区域。7. The target tracking method according to claim 6, wherein if the comparison value is less than or equal to the threshold value, while maintaining the current target tracking parameter update speed, the search range is adjusted and the motion confidence region is searched first.
  10. 一种采用如权利要求1～5中任一项所述的目标跟踪系统的目标跟踪方法，其特征在于，包括：A target tracking method using the target tracking system according to any one of claims 1 to 5, characterized in that it comprises:
    读取当前跟踪方法所用跟踪特征，判断所述跟踪特征是否只包括颜色信息；Read the tracking feature used in the current tracking method, and determine whether the tracking feature only includes color information;
    若否，则不切换所述跟踪特征；If not, do not switch the tracking feature;
    若是，则读取所述图像处理单元获取的图像特征参数的统计信息以判断图 像是否为黑白模式，若是，则读取所述锐度计算模块获取的图像中每个像素点的锐度信息，利用所述图像每人像素点的锐度信息将所述跟踪特征从三通道颜色切换为亮度、水平方向锐度和垂直方向锐度，若否，则将所述跟踪特征从亮度、水平方向锐度和垂直方向锐度切换成三通道颜色。If yes, read the statistical information of the image feature parameters obtained by the image processing unit to determine whether the image is in black and white mode, if yes, read the sharpness information of each pixel in the image obtained by the sharpness calculation module, Use the sharpness information of each pixel of the image to switch the tracking feature from three-channel color to brightness, horizontal sharpness, and vertical sharpness. If not, change the tracking feature from brightness to horizontal sharpness. The sharpness and vertical sharpness switch to three-channel color.
PCT/CN2020/075916 2019-04-23 2020-02-20 Target tracking system and method WO2020215865A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910330691.4A CN111833376A (en) 2019-04-23 2019-04-23 Target tracking system and method
CN201910330691.4 2019-04-23

Publications (1)

Publication Number Publication Date
WO2020215865A1 true WO2020215865A1 (en) 2020-10-29

Family

ID=72911923

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/075916 WO2020215865A1 (en) 2019-04-23 2020-02-20 Target tracking system and method

Country Status (2)

Country Link
CN (1) CN111833376A (en)
WO (1) WO2020215865A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115657000A (en) * 2022-12-28 2023-01-31 成都空御科技有限公司 Low-slow small-adaptive radar detection system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101729872A (en) * 2009-12-11 2010-06-09 南京城际在线信息技术有限公司 Video monitoring image based method for automatically distinguishing traffic states of roads
CN104270570A (en) * 2014-10-17 2015-01-07 北京英泰智软件技术发展有限公司 Binocular video camera and image processing method thereof
CN104537689A (en) * 2014-12-25 2015-04-22 中国科学院自动化研究所 Target tracking method based on local contrast prominent union features
US20160358341A1 (en) * 2015-06-07 2016-12-08 Apple Inc. Fast Template-Based Tracking
CN108496138A (en) * 2017-05-25 2018-09-04 深圳市大疆创新科技有限公司 A kind of tracking and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101729872A (en) * 2009-12-11 2010-06-09 南京城际在线信息技术有限公司 Video monitoring image based method for automatically distinguishing traffic states of roads
CN104270570A (en) * 2014-10-17 2015-01-07 北京英泰智软件技术发展有限公司 Binocular video camera and image processing method thereof
CN104537689A (en) * 2014-12-25 2015-04-22 中国科学院自动化研究所 Target tracking method based on local contrast prominent union features
US20160358341A1 (en) * 2015-06-07 2016-12-08 Apple Inc. Fast Template-Based Tracking
CN108496138A (en) * 2017-05-25 2018-09-04 深圳市大疆创新科技有限公司 A kind of tracking and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DAI, SHIJIE ET AL.: "Mean Shift (Mean Shift Tracking Algorithm Using Joint Histogram of Colors and Oriented Gradients", OPTICS AND PRECISION ENGINEERING, vol. 23, no. 10, 31 October 2015 (2015-10-31), pages 460 - 462 *

Also Published As

Publication number Publication date
CN111833376A (en) 2020-10-27

Similar Documents

Publication Publication Date Title
US10929718B2 (en) Image recognition apparatus, learning apparatus, image recognition method, learning method, and storage medium
WO2019148912A1 (en) Image processing method, apparatus, electronic device and storage medium
US9117139B2 (en) Video processing apparatus and video processing method
CN113766125B (en) Focusing method and device, electronic equipment and computer readable storage medium
US9823331B2 (en) Object detecting apparatus, image capturing apparatus, method for controlling object detecting apparatus, and storage medium
US9113059B2 (en) Image pickup apparatus and image region discrimination method
US8411159B2 (en) Method of detecting specific object region and digital camera
KR20140022446A (en) Motion detecting device, motion detecting method, imaging device, and monitoring system
KR20100134085A (en) Systems and methods to increase speed of object detection in a digital image
US11836903B2 (en) Subject recognition method, electronic device, and computer readable storage medium
CN109922325B (en) Image processing apparatus, control method thereof, and computer-readable storage medium
KR101038815B1 (en) Image capture system capable of fast auto focus
JP2592985B2 (en) Imaging device
JP2016126750A (en) Image processing system, image processing device, imaging device, image processing method, program, and recording medium
CN110365897B (en) Image correction method and device, electronic equipment and computer readable storage medium
WO2020215865A1 (en) Target tracking system and method
CN108289170B (en) Photographing apparatus, method and computer readable medium capable of detecting measurement area
JP2011134117A (en) Object region extraction apparatus, method for controlling the same, object tracking apparatus, and program
US10609275B2 (en) Image processing device, image processing method, and recording medium
US11716536B2 (en) Control device, image capturing apparatus, and control method for detecting obstacle
US8471950B2 (en) Signal processor for adjusting image quality of an input picture signal
CN113792708B (en) ARM-based remote target clear imaging system and method
KR101761947B1 (en) Device and method for reducing blooming of camera image
WO2023245391A1 (en) Preview method and apparatus for camera
JP2007333909A (en) Electronic camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20795868

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20795868

Country of ref document: EP

Kind code of ref document: A1