CN102682454A - Method and device for tracking region of interest in video - Google Patents

Abstract

The invention discloses a method and device for tracking a region of interest in a video, comprising: firstly, obtaining a motion vector of a pixel or a macroblock in a current frame, and determining a moving speed distribution parameter of a region of interest ROI according to the motion vector, Also determine the ROI scaling parameter according to the state information of the ROI in the reference frame; then, use the moving speed distribution parameter and the scaling parameter of the ROI to perform state transition processing on the particles obtained by sampling in the current frame, and determine the current state according to the particles after the state transition The ROI position and size of the frame. The embodiment of the present invention can use the motion vector information existing in the compressed code stream or generated during encoding to guide the particle state transfer process, thereby reducing the number of particles required in the tracking process while ensuring the tracking effect, thereby reducing the tracking process. complexity, and better tracking results can be obtained.

Description

Method and device for tracking region of interest in video

technical field

The present invention relates to the technical field of video processing, in particular to a method and device for tracking a region of interest during video processing.

Background technique

With the popularization and development of communication technology, corresponding video services such as mobile TV, video conferencing, and video surveillance have also developed rapidly. In the process of users accessing and developing corresponding video services through various terminals and different access methods, the diversity of user terminals and the complexity of the network environment make how to effectively transmit video content a huge challenge in the design of video service systems. challenge.

At present, SVC (Scalable Video Coding) technology can effectively transmit corresponding video content to a certain extent. SVC technology is to simultaneously encode sub-streams of various code rates, resolutions, and frame rates in a code stream, and perform a simple extraction operation at the transmission node according to the network conditions and the needs of users or user equipment to generate correspondingly suitable sub-streams. Match code flow. Compared with single code stream technology, SVC technology can provide a code stream with scalable space, time and quality, that is, some sub-code streams can be extracted from this code stream. The corresponding sub-code stream can meet the network transmission rate and the requirements of end users for video space, time and signal-to-noise ratio. Therefore, SVC technology enables video streams to better adapt to various network environments and user terminals.

In SVC technology, the lowest quality coding layer that can be provided is called BL (base layer), and the coding layer that can enhance spatial resolution, time resolution or signal-to-noise ratio strength is called EL (enhancement layer). Among them, the spatial scalability uses the method of layered coding (Layered Coding), which uses the motion, texture and residual information between layers; the temporal scalability uses hierarchical bidirectional predictive frame (Hierarchical B) coding technology; for the signal-to-noise ratio For scalability, the methods of CGS (coarse-grained quality scalable) and MGS (medium-grained quality scalable) can be used.

The corresponding SVC technology also provides support for ROI (Region of Interest) encoding. ROI usually refers to a region in a video frame that contains objects with clear high-level semantics for viewers, such as a person, an object, and so on. When a user browses a video, if the display size of the device is small, or the available bandwidth is reduced, the definition of the region of interest can be kept as clear as possible so as not to affect the user's viewing experience of the video. For example, when the access bandwidth is insufficient, some non-interest regions can be deleted to adapt to the impact of bandwidth requirements on the subjective video quality, that is, when the bandwidth is not enough to transmit the coded streams of the base layer and enhancement layer, the base layer and ROI can be transmitted The encoded bit stream can make full use of the bandwidth, which can maintain the video quality to a certain extent and ensure the subjective experience of users.

In order to use the ROI coding technology to adapt to various applications, it is necessary to determine the position and size of the ROI in each frame of the video. Usually, video tracking technology can be used to determine the size and position of the ROI in each frame of the video.

The implementation scheme of determining the ROI in each frame of the video by using the video tracking technology currently adopted will be described below.

A currently used ROI tracking method is based on a particle filter algorithm. Specifically, the tracked area is represented as a particle (rectangle or ellipse, etc.), and the particle can include various states, such as motion speed, direction, and area size. During tracking, a certain number of particles are generated by importance sampling in the current frame, and according to the correlation between the certain number of particles and the particles in the area to be tracked in the reference frame, the stable state of the particles in the current frame is obtained by weighting, thus Get the area to be tracked in the current frame.

Correspondingly, the specific processing methods for obtaining the region to be tracked in the current frame based on the particle filter algorithm include:

i＝1，…，N，其中p(x₀)是指初始化的目标概率分布，具体地，可以设定为第一帧中以目标的位置大小为均值的高斯分布。(1) Initialization: take k=0 (that is, the initial moment), and draw N sample points according to p(x ₀ )

i=1, . . . , N, where p(x ₀ ) refers to the initialized target probability distribution, specifically, it can be set as a Gaussian distribution with the position and size of the target as the mean value in the first frame.

令

其中i＝1，…，N，其中，

表示k时刻第i个粒子的状态，

表示从0时刻(初始时刻)到k时刻为止粒子的状态，z_1:k表示从1时刻(初始跟踪时刻)到k时刻为止目标的观测值(一般指跟踪目标的颜色直方图)，

是指，

是指有了1至k帧的观测值和第i个粒子至k-1帧的状态的条件下，第k帧中粒子的状态分布的估计，即重要性函数。(2) Importance sampling:

make

where i=1,...,N, where,

Indicates the state of the i-th particle at time k,

Represents the state of the particle from time 0 (initial time) to time k, z _1:k represents the observed value of the target from time 1 (initial tracking time) to time k (generally refers to the color histogram of the tracking target),

Refers to,

It refers to the estimation of the state distribution of the particle in the kth frame under the condition that there are observation values from 1 to k frames and the state of the i-th particle to k-1 frame, that is, the importance function.

其中，

是指观测模型，即表示粒子是所跟踪目标的概率，

是指状态转移模型，即目标由k-1帧向k帧运动的概率分布模型。(3) Calculate the weight If one-step transfer posterior state distribution is adopted, the formula can be simplified as:

in,

refers to the observation model, that is, the particle is the probability of the tracked target,

It refers to the state transition model, that is, the probability distribution model of the target moving from frame k-1 to frame k.

(4) Normalized weights:

的大小复制或舍弃样本

得到N个近似服从

分布的样本令 ${\mathrm{\ω}}_k^{\left(i\right)}={\widetilde{\mathrm{\ω}}}_k^{\left(i\right)}=1/N,$ i＝1，…，N。(5) Resampling: according to the respective normalized weights

Replicate or discard samples of size

get N approximate obedience

Sample of the distribution make ${\mathrm{\ω}}_k^{\left(i\right)}={\widetilde{\mathrm{\ω}}}_k^{\left(i\right)}=1/N,$ i=1,...,N.

(6) Output result: the output of the algorithm is the particle set It can be used to approximate the posterior probability and the expectation of the function x _0:k , where:

Posterior probability: $\hat{p}\left(x_{0:k}\right|z_{1:k})=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\δ}}_{x_{0:k}^{\left(i\right)}}\left({\mathrm{dx}}_{0:k}\right);$

The expectation of the function x _0:k : $\mathrm{E.}\left(x_{0:k}\right)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{x}_{0:k}^i.$

(7) Let k=k+1, repeat the above process (2) to process (6).

In the above implementation scheme, to obtain a stable tracking effect, a larger number of particles is required, and the larger the number of particles, the greater the amount of calculation required for tracking, resulting in a greatly increased processing complexity.

Contents of the invention

The object of the present invention is to provide a method and device for tracking a region of interest in a video, so as to reduce the processing complexity in the tracking process under the premise of ensuring the tracking effect.

The purpose of the present invention is achieved through the following technical solutions:

A region of interest tracking method in a video, comprising:

Obtain the motion vector of the pixel or macroblock in the current frame, and determine the moving speed distribution parameter of the region of interest ROI according to the motion vector, and determine the ROI scaling parameter according to the state information of the ROI in the reference frame;

Using the moving speed distribution parameters and scaling parameters of the ROI to perform state transition processing on the particles sampled in the current frame, and determine the ROI position and size in the current frame according to the state-transferred particles.

Optionally, the step of determining the moving speed distribution parameters of the ROI includes:

Determine the position in the reference frame of the reference pixel or macroblock corresponding to the pixel or macroblock in the current frame according to the motion vector of the pixel or macroblock in the current frame, and select the position from the motion vector of the pixel or macroblock in the current frame The motion vector of the pixel or macroblock in the current frame corresponding to the reference pixel or macroblock in the ROI in the reference frame; determine the moving speed distribution parameter of the ROI according to the motion vector of the pixel or macroblock in the current frame selected and obtained;

Optionally, the step of determining ROI scaling parameters includes:

Determine the corresponding pixel or macroblock in the current frame according to the motion vector of the selected pixel or macroblock in the current frame, and determine the ROI according to the pixel or macroblock in the current frame and the pixel or macroblock in the corresponding reference frame scaling parameter.

Optionally, the moving speed distribution parameters of the ROI include:

其中，a为，b为为仿射参数，该仿射参数采用最小二乘法结合四参数变换模型求解，相应的四参数变换模块为根据该当前帧中像素或宏块和对应的参考帧中的像素或宏块建立。 $P\left({\mathrm{MV}}^{\mathrm{ROI}}\right)=\underset{i=0}{\overset{\left|G\right|-1}{\mathrm{\Σ}}}\mathrm{\δ}({\mathrm{MV}}^i-{\mathrm{MV}}^{\mathrm{ROI}})/\left|G\right|,$ Among them, δ is a Dirac function, G is the motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixels or macroblocks in the ROI in the reference frame, and MV _i refers to the i-th motion in G Vector, MV ^ROI is the moving speed of the ROI from the previous frame to the current frame; the ROI scaling parameters include:

Among them, a is, b is an affine parameter, and the affine parameter is solved by the least square method combined with the four-parameter transformation model, and the corresponding four-parameter transformation module is based on the pixel or macroblock in the current frame and the corresponding reference frame. Pixel or macroblock build.

Optionally, the step of performing state transition processing on the particles obtained by sampling in the current frame includes:

标志变量

否则，以分布P(MV^ROI)选取在上一模块中统计的运动矢量集合G中的一个元素对应的两个分量(MV′_x，MV′_y)分别作为v₁、v₂的值，即令v₁＝MV′_x、v₂＝MV′_y，并记

α为状态转移参数，其初始值为预先设定，在后续的粒子更新过程中更新该值的方式包括：

为k-1帧中粒子n中的v_x分量，

为k-1帧中粒子n中的v_y分量；Determine the speed v ₁ and v ₂ of the particle state transition according to the moving speed distribution parameters of the ROI, including; generate a random number μ with a uniform distribution of 0 to 1, if μ<α, then set

flag variable

Otherwise, select the two components (MV′ _x , MV′ _y ) corresponding to an element in the motion vector set G counted in the previous module as the values of v ₁ and v ₂ respectively by distribution P(MV ^ROI ), that is, v ₁ =MV′ _x , v ₂ =MV′ _y , and record

α is a state transition parameter, its initial value is preset, and the way to update this value in the subsequent particle update process includes:

is the v _x component in particle n in frame k-1,

is the v _y component of particle n in frame k-1;

Determine the particle scaling parameter η according to the ROI scaling parameter, including; generate a random number γ with a uniform distribution of 0 to 1, if γ<β, then make η=ρ, otherwise let η=1, where β is the target size change A degree parameter, whose value is preset, and ρ is the scaling parameter;

Particle state transfer processing is performed according to the speed of the particle state transfer and the particle scaling parameter, and the result after obtaining the particle state transfer of the nth particle in the kth frame includes: the position of the particle ( ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{x\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{x\right\}+{\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="HDA0000158966620000043.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+{\mathrm{\&epsiv;}}_x,$ ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}+v_2+{\mathrm{\&epsiv;}}_{\mathrm{the\; y}}$ ), the moving speed of the particle in the x and y directions are respectively: ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_x\right\}={\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="HDA0000158966620000043.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+{\mathrm{\&epsiv;}}_{v_x}$ and ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_{\mathrm{the\; y}}\right\}=v_2+{\mathrm{\&epsiv;}}_{v_{\mathrm{the\; y}}},$ The semi-major axis of the ellipse corresponding to the particle at the position is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{a\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{a\right\}+{\mathrm{\&epsiv;}}_a,$ semi minor axis is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{b\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{b\right\}+{\mathrm{\&epsiv;}}_b,$ Where ε _x , ε _y are random variables that obey the N(0, 0.2) distribution, is a random variable subject to N(0, 0.25), and ε _a and ε _b are random variables subject to N(0, 0.1).

Optionally, the step of determining the ROI of the current frame according to the particles after the state transition includes:

Perform the statistical processing of the color histogram of each particle area on the particles after the state transition processing, and perform particle update processing according to the statistical processing results of the color histogram of each particle area;

Calculate the ROI position size based on the results obtained from the particle update process.

A device for tracking a region of interest in a video, comprising:

The moving speed distribution parameter and scaling parameter determination module is used to obtain the motion vector of the pixel or macroblock in the current frame, and determine the moving speed distribution parameter of the region of interest ROI according to the motion vector, and is also used to determine the moving speed distribution parameter of the ROI in the reference frame according to the motion vector The state information of determines the scaling parameters;

The particle state transfer module is used to use the moving speed distribution parameter and the scaling parameter of the ROI determined by the moving speed distribution parameter and the scaling parameter determination module to perform state transfer processing on the particles obtained by sampling in the current frame;

The ROI determination module is used to determine the position and size of the ROI of the current frame according to the particles after the state transfer by the particle state transfer module.

Optionally, the moving speed distribution parameter and scaling parameter determination module specifically includes:

The motion vector acquisition module is used to determine the position of the reference pixel or macroblock corresponding to the pixel or macroblock in the current frame in the reference frame according to the motion vector of the pixel or macroblock in the current frame, and determine the position of the pixel or macroblock in the current frame In the motion vector, the motion vector of the pixel or macroblock in the current frame corresponding to the reference pixel or macroblock corresponding to the position in the ROI in the reference frame is selected;

The parameter determination module is used to determine the moving speed distribution parameter and scaling parameter of the ROI according to the motion vector of the pixels or macroblocks in the current frame selected and obtained by the motion vector acquisition module; The motion vector of the block determines the corresponding pixel or macroblock in the current frame, and the ROI scaling parameter is determined according to the pixel or macroblock in the current frame and the corresponding pixel or macroblock in the reference frame.

Optionally, the parameter determination module includes:

The moving speed distribution parameter estimation module of ROI is used for estimating the moving speed distribution parameter of ROI, and the moving speed distribution parameter of the ROI obtained by estimation includes: $P\left({\mathrm{MV}}^{\mathrm{ROI}}\right)=\underset{i=0}{\overset{\left|G\right|-1}{\mathrm{\&Sigma;}}}\mathrm{\&delta;}({\mathrm{MV}}^i-{\mathrm{MV}}^{\mathrm{ROI}})/\left|G\right|,$ Among them, δ is a Dirac function, G is the motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixels or macroblocks in the ROI in the reference frame, and MV ⁱ refers to the i-th motion in G Vector, MV ^ROI is the moving speed of ROI from the previous frame to the current frame;

其中，a为，b为仿射参数，该仿射参数为采用最小二乘法结合四参数变换模型求解，相应的四参数变换模块为根据该当前帧中像素或宏块和对应的参考帧中的像素或宏块建立。ROI scaling parameter estimation module, for estimating ROI scaling parameters, and the estimated ROI scaling parameters obtained include:

Among them, a is, b is an affine parameter, and the affine parameter is solved by using the least squares method combined with a four-parameter transformation model, and the corresponding four-parameter transformation module is based on the pixel or macroblock in the current frame and the corresponding reference frame. Pixel or macroblock build.

Optionally, the particle state transfer module includes:

标志变量

否则，以分布P(MV^ROI)选取在上一模块中统计的运动矢量集合G中的一个元素对应的两个分量(MV′_x，MV′_y)分别作为v₁、v₂的值，即令v₁＝MV′_x、v₂＝MV′_y，并记

α为状态转移参数，其初始值为预先设定，在后续的粒子更新过程中更新该值的方式包括：

为k-1帧中粒子n中的v_x分量，

为k-1帧中粒子n中的v_y分量；The particle position and speed transfer module is used to determine the speed v ₁ and v ₂ of the particle state transfer according to the ROI moving speed distribution parameter obtained by the ROI moving speed distribution parameter estimation module, including: generating random number μ, if μ<α, then let

flag variable

Otherwise, select the two components (MV′ _x , MV′ _y ) corresponding to an element in the motion vector set G counted in the previous module as the values of v ₁ and v ₂ respectively by distribution P(MV ^ROI ), that is, v ₁ =MV′ _x , v ₂ =MV′ _y , and record

α is a state transition parameter, its initial value is preset, and the way to update this value in the subsequent particle update process includes:

is the v _x component in particle n in frame k-1,

is the v _y component of particle n in frame k-1;

The particle size transfer module is used to determine the particle scaling parameter η according to the ROI scaling parameter obtained by the ROI scaling parameter estimation module, including; generating a random number γ with a uniform distribution of 0 to 1, if γ<β, then let η=ρ , otherwise let η=1, wherein, β is the target size change degree parameter, and its value is preset, and ρ is the ROI scaling parameter;

The particle state transfer result determination module is used to perform particle state transfer processing according to the particle state transfer speed determined by the particle position speed transfer module and the particle scaling parameter determined by the particle size transfer module, to obtain the nth frame in the kth frame The result after the particle state transfer of a particle includes: the position of the particle ( ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{x\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{x\right\}+v_1+{\mathrm{\&epsiv;}}_x,$ ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}+v_2+{\mathrm{\&epsiv;}}_{\mathrm{the\; y}}$ ), the moving speed of the particle in the x and y directions are respectively: ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_x\right\}=v_1+{\mathrm{\&epsiv;}}_{v_x}$ and ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_{\mathrm{the\; y}}\right\}=v_2+{\mathrm{\&epsiv;}}_{v_{\mathrm{the\; y}}},$ The semi-major axis of the ellipse corresponding to the particle at the position is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{a\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{a\right\}+{\mathrm{\&epsiv;}}_a,$ semi minor axis is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{b\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{b\right\}+{\mathrm{\&epsiv;}}_b,$ Where ε _x , ε _y are random variables that obey the N(0, 0.2) distribution, is a random variable subject to N(0, 0.25), and ε _a and ε _b are random variables subject to N(0, 0.1).

Optionally, the ROI determination module includes:

The statistical processing module of the color histogram of each particle area is used for performing the statistical processing of the color histogram of each particle area on the particles after the state transition processing;

The particle update module is used to perform particle update processing according to the statistical processing results of the color histogram of each particle area;

The ROI position and size calculation module is used to calculate the ROI position and size according to the result obtained by particle update processing.

It can be seen from the above-mentioned technical solution provided by the present invention that the ROI tracking technology provided by the embodiment of the present invention can use the motion vector information existing in the compressed code stream or generated during encoding to guide the particle state transfer process, thereby ensuring the tracking effect In the case of , the number of particles required in the tracking process is reduced, thereby reducing the complexity of the tracking process and obtaining a better tracking effect.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings on the premise that s does not pay creative efforts.

Fig. 1 is a schematic diagram of the processing process of the method provided by the embodiment of the present invention;

Fig. 2 is a structural schematic diagram 1 of the device provided by the embodiment of the present invention;

Fig. 3 is a schematic structural diagram II of the device provided by the embodiment of the present invention;

Fig. 4 is a schematic structural diagram III of the device provided by the embodiment of the present invention;

Fig. 5 is a structural schematic diagram 4 of the device provided by the embodiment of the present invention;

Fig. 6 is the MV distribution histogram provided by the embodiment of the present invention;

Fig. 7 is a schematic diagram of particle state transition provided by the embodiment of the present invention;

Fig. 8 is a schematic diagram five of the structure of the device provided by the embodiment of the present invention;

Fig. 9 is a histogram of particle region colors provided by an embodiment of the present invention;

Fig. 10 is a schematic diagram of the particle update process provided by the embodiment of the present invention;

Fig. 11 is a schematic diagram of the application effect of the embodiment of the present invention;

Fig. 12 is a second schematic diagram of the application effect of the embodiment of the present invention;

FIG. 13 is a first schematic diagram of the application environment of the embodiment of the present invention;

FIG. 14 is a second schematic diagram of the application environment of the embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

An embodiment of the present invention provides a method for tracking a region of interest in a video, and its specific implementation is shown in Figure 1, which may include the following steps:

Step 11, obtaining the motion vector of the pixel (or macroblock) in the current frame, and determining the moving speed distribution parameter of ROI according to the corresponding motion vector, and also determining the ROI scaling parameter according to the status information of the ROI in the reference frame;

Wherein, the current frame whose corresponding ROI position is to be determined is referred to as the current frame for short, and the reference frame of the current frame on the encoding structure is referred to as the reference frame for short, and the ROI position in the reference frame is known;

Specifically, in this step, the process of determining the moving speed distribution parameters of the ROI may specifically include, but is not limited to:

First, determine the position of the reference pixel or macroblock corresponding to the pixel or macroblock in the current frame in the reference frame according to the motion vector of the pixel or macroblock in the current frame; The corresponding position is located in the reference pixel or macroblock in the ROI in the reference frame, and the motion vector of the pixel or macroblock in the current frame corresponding to the reference pixel or macroblock is obtained; after that, the pixel or macroblock in the current frame can be obtained according to the selection. The motion vector of the macroblock determines the moving speed distribution parameter of the corresponding ROI;

The step of determining ROI scaling parameters according to the status information of the ROI in the reference frame may include, but is not limited to:

First, determine the corresponding pixel or macroblock in the current frame according to the motion vector of the selected pixel or macroblock in the current frame; then, according to the pixel or macroblock in the current frame and the pixel or macroblock in the corresponding reference frame Determine the ROI scaling parameters.

Wherein, the calculation methods of the above-mentioned ROI moving speed distribution parameters and ROI scaling parameters may include, but are not limited to:

ROI moving speed distribution parameters $P\left({\mathrm{MV}}^{\mathrm{ROI}}\right)=\underset{i=0}{\overset{\left|G\right|-1}{\mathrm{\&Sigma;}}}\mathrm{\&delta;}({\mathrm{MV}}^i-{\mathrm{MV}}^{\mathrm{ROI}})/\left|G\right|,$ Among them, δ is a Dirac function, G is the motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixels or macroblocks in the ROI in the reference frame, and MV ⁱ refers to the i-th motion in G Vector, MV ^ROI is the moving speed of ROI from the previous frame (ie reference frame) to the current frame;

其中，a为，b为仿射参数，该仿射参数采用最小二乘法结合四参数变换模型求解，相应的四参数变换模块为根据该当前帧中像素或宏块和对应的参考帧中的像素或宏块建立。ROI scaling parameters

Among them, a is, b is an affine parameter, and the affine parameter is solved by using the least squares method combined with the four-parameter transformation model, and the corresponding four-parameter transformation module is based on the pixel or macroblock in the current frame and the pixel in the corresponding reference frame or macroblock build.

Step 12, using the determined ROI moving speed distribution parameters and ROI scaling parameters to perform state transition processing on the particles obtained by sampling in the current frame;

The specific implementation of this step may include:

标志变量

否则，以分布P(MV^ROI)选取在上一模块中统计的运动矢量集合G中的一个元素对应的两个分量(MV′_x，MV′_y)分别作为v₁、v₂的值，即令v₁＝MV′_x、v₂＝MV′_y，并记

α为状态转移参数，其初始值为预先设定，在后续的粒子更新过程中更新该值的方式包括：

为k-1帧中粒子n中的v_x分量，

为k-1帧中粒子n中的v_y分量，且当前帧为第k帧，参考帧为第k-1帧；Firstly, according to the moving velocity distribution parameters of the above ROI, the velocity v ₁ and v ₂ of particle state transition are determined, including: generating a random number μ with a uniform distribution of 0 to 1, if μ<α, then set

flag variable

Otherwise, select the two components (MV′ _x , MV′ _y ) corresponding to an element in the motion vector set G counted in the previous module as the values of v ₁ and v ₂ respectively by distribution P(MV ^ROI ), that is, v ₁ =MV′ _x , v ₂ =MV′ _y , and record

α is a state transition parameter, its initial value is preset, and the way to update this value in the subsequent particle update process includes:

is the v _x component in particle n in frame k-1,

is the v _y component of particle n in frame k-1, and the current frame is frame k, and the reference frame is frame k-1;

Afterwards, determine the particle scaling parameter η according to the above-mentioned ROI scaling parameters, including; generate a random number γ with a uniform distribution of 0 to 1, if γ<β, then make η=ρ, otherwise let η=1, where β is the target size The degree of change parameter, its value is preset, and ρ is the scaling parameter;

是服从N(0，0.25)的随机变量，ε_a、ε_b是服从N(0，0.1)的随机变量。Finally, the particle state transfer process is performed according to the speed of the above-mentioned particle state transfer and the particle scaling parameter, and the result after obtaining the particle state transfer of the nth particle in the kth frame includes: the position of the particle ( ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{x\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{x\right\}+v_1+{\mathrm{\&epsiv;}}_x,$ ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}+v_2+{\mathrm{\&epsiv;}}_{\mathrm{the\; y}}$ ), the moving speed of the particle in the x and y directions are respectively: ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_x\right\}=v_1+{\mathrm{\&epsiv;}}_{v_x}$ and ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_{\mathrm{the\; y}}\right\}=v_2+{\mathrm{\&epsiv;}}_{v_{\mathrm{the\; y}}},$ The semi-major axis of the ellipse corresponding to the particle at the position is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{a\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{a\right\}+{\mathrm{\&epsiv;}}_a,$ semi minor axis is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{b\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{b\right\}+{\mathrm{\&epsiv;}}_b,$ Where ε _x , ε _y are random variables that obey the N(0, 0.2) distribution,

is a random variable subject to N(0, 0.25), and ε _a and ε _b are random variables subject to N(0, 0.1).

Step 13, determine the ROI position and size of the current frame according to the particles after the state transition, and realize the tracking processing for the ROI in the video;

Specifically, the specific implementation of this step may include:

Perform the statistical processing of the color histogram of each particle area on the particles after the state transition processing, and perform particle update processing according to the statistical processing results of the color histogram of each particle area;

Calculate the ROI position size based on the results obtained from the particle update process.

The embodiments of the present invention mainly use the additional information in the code stream or generated during encoding to guide particle state transition, thereby ensuring the tracking effect on the ROI and reducing the processing complexity of the tracking process. Specifically, the present invention utilizes the MV information existing in the compressed code stream or generated during encoding to guide the particle state transition process in the particle filter algorithm, thereby reducing the number of particles required while ensuring the tracking effect, thereby reducing tracking The complexity of the processing process; or, the present invention can obtain a more stable tracking effect under the condition that the same number of particles is used for tracking processing. Moreover, the present invention also utilizes the stability of the particle filter theory to noise to obtain the robustness of the tracking algorithm, which further ensures the tracking effect of the tracking technical solution provided by the embodiment of the present invention.

The embodiment of the present invention also provides a device for tracking a region of interest in a video, and its specific implementation structure is shown in Figure 2, which may include:

(1) Moving speed distribution parameter and scaling parameter determining module 21, used to obtain the motion vector of the pixel or macroblock in the current frame, and determine the moving speed distribution parameter of the region of interest ROI according to the motion vector, and also according to the moving speed distribution parameter in the reference frame The status information of the ROI determines the ROI scaling parameter;

Further, as shown in FIG. 3 , the moving speed distribution parameter and scaling parameter determination module 21 may specifically include:

The motion vector acquisition module 211 is configured to determine the position of the reference pixel or macroblock corresponding to the pixel or macroblock in the current frame in the reference frame according to the motion vector of the pixel or macroblock in the current frame, and determine the position of the pixel or macroblock in the current frame Select the reference pixel or macroblock whose position is located in the ROI in the reference frame from the motion vector, and then obtain the motion vector of the pixel or macroblock in the current frame corresponding to the reference pixel or macroblock;

The parameter determining module 212 is used to determine the moving speed distribution parameter of the corresponding ROI according to the motion vector of the pixel or macroblock in the current frame selected and obtained by the above-mentioned motion vector obtaining module 211; The motion vector determines the corresponding pixel or macroblock in the current frame, and determines the corresponding ROI scaling parameter according to the pixel or macroblock in the current frame and the pixel or macroblock in the corresponding reference frame;

The parameter determination module 212 may specifically include an ROI moving speed distribution parameter estimation module 2121 and an ROI scaling parameter estimation module 2122, wherein:

The ROI moving speed distribution parameter estimation module 2121 is used to estimate the ROI moving speed distribution parameter, and the estimated moving speed distribution parameter of the ROI includes: $P\left({\mathrm{MV}}^{\mathrm{ROI}}\right)=\underset{i=0}{\overset{\left|G\right|-1}{\mathrm{\&Sigma;}}}\mathrm{\&delta;}({\mathrm{MV}}^i-{\mathrm{MV}}^{\mathrm{ROI}})/\left|G\right|,$ Among them, δ is a Dirac function, G is the motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixels or macroblocks in the ROI in the reference frame, and MV ⁱ refers to the i-th motion in G Vector, MV ^ROI is the moving speed of ROI from the previous frame (ie reference frame) to the current frame;

其中，a为，b为为仿射参数，该仿射参数采用最小二乘法结合四参数变换模型求解，相应的四参数变换模块为根据该当前帧中像素或宏块和对应的参考帧中的像素或宏块建立。The ROI scaling parameter estimation module 2122 is configured to estimate the ROI scaling parameter, and the estimated ROI scaling parameter includes:

Among them, a is, b is an affine parameter, and the affine parameter is solved by the least square method combined with the four-parameter transformation model, and the corresponding four-parameter transformation module is based on the pixel or macroblock in the current frame and the corresponding reference frame. Pixel or macroblock build.

(2) particle state transfer module 22, for utilizing the moving speed distribution parameter and the scaling parameter of the ROI determined by the above-mentioned moving speed distribution parameter and scaling parameter determining module 21 to carry out state transition processing to the particles sampled and obtained in the current frame;

Further, as shown in FIG. 8, the corresponding particle state transfer module may include:

标志变量

否则，以分布P(MV^ROI)选取在上一模块中统计的运动矢量集合G中的一个元素对应的两个分量(MV′_x，MV′_y)分别作为v₁、v₂的值，即令v₁＝MV′_x、v₂＝MV′_y，并记α为状态转移参数，其初始值为预先设定，在后续的粒子更新过程中更新该值的方式包括：

为k-1帧中粒子n中的v_x分量，

为k-1帧中粒子n中的v_y分量；The particle position and velocity transfer module 221 is used to determine the velocity v ₁ and v ₂ of the particle state transition according to the ROI moving velocity distribution parameter obtained by the above ROI moving velocity distribution parameter estimation module 2121, including; Random number μ, if μ<α, then let

flag variable

Otherwise, select the two components (MV′ _x , MV′ _y ) corresponding to an element in the motion vector set G counted in the previous module as the values of v ₁ and v ₂ respectively by distribution P(MV ^ROI ), that is, v ₁ =MV′ _x , v ₂ =MV′ _y , and record α is a state transition parameter, its initial value is preset, and the way to update this value in the subsequent particle update process includes:

is the v _x component in particle n in frame k-1,

is the v _y component of particle n in frame k-1;

The particle size transfer module 222 is used to determine the particle scaling parameter η according to the ROI scaling parameter obtained by the above-mentioned ROI scaling parameter estimation module 2122, which may specifically include: generating a random number γ with a uniform distribution of 0 to 1, if γ<β, then set η=ρ, otherwise make η=1, wherein, β is the target size change degree parameter, and its value is preset, and ρ is the above-mentioned ROI scaling parameter;

是服从N(0，0.25)的随机变量，ε_a、ε_b是服从N(0，0.1)的随机变量。Particle state transfer result determination module 223, for performing particle state transfer processing according to the particle state transfer speed determined by the corresponding particle position speed transfer module 221 and the particle scaling parameter determined by the particle size transfer module 222, to obtain the kth frame The results of the particle state transfer of the nth particle include: the position of the particle ( ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{x\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{x\right\}+v_1+{\mathrm{\&epsiv;}}_x,$ ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}+v_2+{\mathrm{\&epsiv;}}_{\mathrm{the\; y}}$ ), the moving speed of the particle in the x and y directions are respectively: ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_x\right\}=v_1+{\mathrm{\&epsiv;}}_{v_x}$ and ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_{\mathrm{the\; y}}\right\}=v_2+{\mathrm{\&epsiv;}}_{v_{\mathrm{the\; y}}},$ The semi-major axis of the ellipse corresponding to the particle at the position is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{a\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{a\right\}+{\mathrm{\&epsiv;}}_a,$ semi minor axis is ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{b\right\}=\mathrm{\&eta;}\&times;{\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{b\right\}+{\mathrm{\&epsiv;}}_b,$ Where ε _x , ε _y are random variables that obey the N(0, 0.2) distribution,

is a random variable subject to N(0, 0.25), and ε _a and ε _b are random variables subject to N(0, 0.1).

(3) ROI determination module 23, used for determining the ROI position and size of the current frame according to the particle after the state transfer of the above-mentioned particle state transfer module 22;

Specifically, as shown in Figure 4, the ROI determination module may specifically include:

The statistical processing module 231 of the color histogram of each particle area is used for performing the statistical processing of the color histogram of each particle area on the particles after the state transition process;

The particle update module 232 is used to perform particle update processing according to the results obtained by performing the statistical processing of the color histogram of each particle area by the above-mentioned color histogram statistical processing module 231 of each particle area;

The ROI position and size calculation module 233 is configured to calculate the ROI position and size according to the results obtained by the above particle update module 232 executing particle update processing.

Similarly, in the above-mentioned device, the MV information existing in the compressed code stream or generated during encoding is used to guide the particle state transfer process, so that the number of required particles can be reduced while ensuring the tracking effect, or, when using the same number of particles In the case of tracking processing, a more stable tracking effect can be obtained.

That is to say, in order to improve the accuracy of ROI tracking and reduce the complexity of tracking, the embodiment of the present invention uses MV information generated in the corresponding encoding process or existing in the code stream to guide the state of particles in the particle filter algorithm. The implementation of the transfer to obtain a more superior tracking effect.

In order to facilitate a better understanding of the embodiments of the present invention, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings and specific application processes.

In the embodiment of the present invention, specifically, according to the particle filter theory, the tracking of the ROI is realized by utilizing the MV information generated during the encoding process or already existing in the code stream. Wherein, the ROI is obtained frame by frame, and data such as the position and size of the ROI in the current frame is obtained according to the information of the ROI in the reference frame. The corresponding tracking process mainly includes:

First, obtain the reference frame of the current frame from the cache of the reference frame, and estimate the direction of the ROI to the current frame by the ROI moving speed distribution parameter and the scaling parameter estimation module according to the state information of the ROI in the reference frame and the MV of the pixel (or macroblock). Speed distribution and scaling parameters for movement;

Secondly, the ROI position and size information distribution in the current frame is estimated by the particle state transfer module;

Third, use the color histogram of each particle area obtained by statistics of the color histogram statistical module of each particle coverage area, and further correct the information distribution of the ROI obtained by the particle state transfer module through the particle update module, and calculate and output the ROI in the current frame The location and size information of the current frame is stored in the reference frame buffer together with the current frame, so as to estimate the ROI information in the subsequent frame.

Specifically, as shown in Figure 4, the device provided by the embodiment of the present invention may specifically include:

(1) Reference frame acquisition module

This module is used to obtain the reference frame of the current frame, which is the same as the method of obtaining the reference frame in the traditional coding method, which is not shown in the figure.

(2) R0I moving speed distribution parameter and scaling parameter estimation module

This module is the moving speed distribution parameter and scaling parameter determination module, which is used to obtain the moving speed distribution parameter distribution estimation and scaling parameter estimation of the ROI area, as shown in Figure 5, this module can further include MV acquisition module, ROI moving speed The distribution parameter estimation module and the ROI scaling parameter estimation module, the following will describe the processing functions completed by each module respectively:

(21) The MV acquisition module, that is, the motion vector acquisition module, is used to count the reference pixels (or macroblocks, the pixels appearing in the following descriptions can be replaced by macroblocks) MV of pixels located in the ROI area of the reference frame, corresponding MV is a two-dimensional vector (MV _x , MV _y ), that is, the component of the MV of the pixel in the x and y directions;

i＝0，1，…，|G|-1；进一步，将G中各元素MV的集合记为M；Specifically, according to the H.264/SVC coding standard, it is easy to know that the position of its reference pixel in the reference frame can be obtained from the MV of the pixel in the current frame. Therefore, if the reference pixel of a certain pixel in the current frame is in the reference frame In the ROI area of the current frame, the MV of the pixel in the current frame is obtained, and the multiple pixels in the current frame whose reference pixels are in the ROI area in the reference frame are recorded as a set G, and the number of elements in the set G is |G| , the i-th element is g _i , and the MV of g _i is recorded as

i=0, 1, ..., |G|-1; further, the set of each element MV in G is recorded as M;

For example, as shown in Figure 6, the reference pixels associated with the corresponding MV1, MV2, and MV3 are located in the ROI region in the reference frame, so the pixels in the current frame corresponding to the MV1, MV2, and MV3 are included in the statistics, that is The corresponding pixels need to be recorded in the set G; and the reference pixels associated with the MVO are located outside the ROI in the reference frame, therefore, the pixels in the current frame corresponding to the MVO are not included in the statistics.

(22) The moving speed distribution estimation module is used to count the (-MV) distribution histogram of the pixels in the current frame where the reference pixels are located in the ROI region in the reference frame, wherein -MV is obtained by taking the negative sign of the two components of the MV respectively , that is, the -MV distribution histogram of the pixels in the set G obtained by the MV acquisition module, and normalized by the L1 norm, used as an estimate of the probability distribution of the moving speed of the ROI area from the reference frame to the current frame, specifically It can be recorded as the moving velocity distribution parameter P(MV ^ROI ), that is:

$P\left({\mathrm{MV}}^{\mathrm{ROI}}\right)=\underset{i=0}{\overset{\left|G\right|-1}{\mathrm{\&Sigma;}}}\mathrm{\&delta;}({\mathrm{MV}}^i-{\mathrm{MV}}^{\mathrm{ROI}})/\left|G\right|,$ Among them, δ is a Dirac function, G is the motion vector set of the pixel or macroblock in the current frame corresponding to the reference pixel in the ROI where the position is located in the reference frame, MV ⁱ refers to the i-th motion vector in G, MV ^{The ROI} is the moving speed of the ROI from the previous frame (ie, the reference frame) to the current frame.

Figure 6 only gives an example of statistics P(MV ^ROI ), where '+' represents a pixel, such as the pixel reference pixel corresponding to the tail of the dotted arrow is not located in the ROI area in the reference frame, and its motion vector MV ⁰ is not included in the statistics list; other motion vectors such as MV ¹ , MV ² etc. are included in the statistics.

其中，a、b为仿射变换参数；(23) The scaling parameter estimation module of the ROI area is used to estimate and obtain the scaling parameter of the ROI area. The scaling parameter is used to indicate the scaling relationship between the ROI area in the reference frame and the ROI area in the current frame. Specifically, it can be based on The scaling parameter is obtained by means of affine transformation, and the scaling parameter can be:

Among them, a and b are affine transformation parameters;

Specifically, the manner of obtaining the ROI scaling parameter may include:

采用仿射变换的一种特例，即四参数变换模型作为目标的移动变换模型，即：Let the coordinates of the i-th element g _i in G be (x _i , y _i ), then the coordinates of its corresponding reference pixel are

A special case of using affine transformation, that is, a four-parameter transformation model as the moving transformation model of the target, namely:

$\left[\begin{array}{c}{x}_i\\ {\mathrm{the\; y}}_i\end{array}\right]=\left[\begin{array}{cc}a& -b\\ b& a\end{array}\right]\&times;\left[\begin{array}{c}{u}_i\\ v_i\end{array}\right]+\left[\begin{array}{c}c\\ d\end{array}\right],$ Where a, b, c, and d are affine transformation parameters;

Using the least squares criterion to estimate a four-parameter transformation model with a unified ROI for all pixels in the above set G, the following matrices A, B, and C can be constructed, namely:

$\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}& <span\; class="notranslate">\; .</span>& & \\ & <span\; class="notranslate">\; .</span>& & \\ & <span\; class="notranslate">\; .</span>& & \\ {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& <span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ & <span\; class="notranslate">\; .</span>& & \\ & <span\; class="notranslate">\; .</span>& & \\ & <span\; class="notranslate">\; .</span>& & \end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}<span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ <span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>\end{array}\right]<span\; class="notranslate">\; ;</span>$

In this way, the mobile transformation model of the corresponding target can be converted into C=B×A, and A=(B ^T B) ^-1 B ^T C can be solved by the least square criterion, where B ^T is the transposition matrix of B, as The projection transformation parameters a and b are the a and b components in the matrix A. Since the matrices B and C are known quantities, the values of the affine transformation parameters a and b can be obtained, and then the value of the corresponding scaling parameter ρ can be calculated. .

(3) Particle transfer module

This module is the particle state transfer module, which can be used to estimate the probability distribution of the moving speed of the ROI area from the reference frame to the current frame according to the ROI moving speed distribution parameter and the scaling parameter estimation module P(MV ^ROI ) and the estimation of the scaling parameter ρ, to guide the state transition of particles to achieve better tracking results.

Specifically, the purpose of this module is to make a preliminary estimate of the ROI distribution of the current frame. As shown in Figure 7, each circle represents a particle (including information such as the position and size of the ROI), and the particle set in the reference frame represents the particle set in the reference frame. The state information distribution of the ROI, the particle set in the reference frame is transferred through the particle state to obtain the particle set in the current frame, and the particle set in the current frame represents the state information distribution of the ROI in the current frame;

During the process of particle state transfer, the information that this module needs to use includes: the particle state representing the ROI distribution in the reference frame, the state transfer parameter α, and P(MV ^ROI ) and ρ, where the initial value of α can be The setting is not limited to 0.5, and the value of the state transition parameter α will be automatically updated in the subsequent processing.

Specifically, it is necessary to perform the same processing for each particle. Here, only the state transition processing for the nth particle is taken as an example for illustration. As shown in FIG. 8, the state transition processing process for the nth particle may include :

(即k-1帧中粒子n中的v_x分量，以下此类表达式含义类似)、

否则，以根据ROI区域由参考帧向当前帧的移动速度概率分布的估计结果P(MV^ROI)在之前获得的运动矢量集合M中选取一个元素对应的两个分量(MV′_x，MV′_y)分别作为v₁、v₂的值，即令v₁＝MV′_x、v₂＝Mv′_y，并记

其中，v₁、v₂临时变量；First, determine the speed of particle state transfer, which may specifically include: generating a random number μ with a uniform distribution of 0 to 1, if μ<α, then let

(that is, the v _x component in particle n in frame k-1, the following expressions have similar meanings),

Otherwise, according to the estimation result P(MV ^ROI ) of the probability distribution of the moving speed of the ROI area from the reference frame to the current frame, select two components (MV′ _x , MV′ _y ) corresponding to an element in the previously obtained motion vector set M ) as the values of v ₁ and v ₂ respectively, that is, set v ₁ =MV′ _x , v ₂ =Mv′ _y , and record

Among them, v ₁ and v ₂ are temporary variables;

Secondly, determine the particle scaling parameters, which may specifically include: generating a random number γ with a uniform distribution of 0 to 1, if γ<β, then set η=ρ, otherwise set η=1, where η is a temporary variable, and β is the target Size change degree parameter, which is a constant parameter, used to control the change degree of the target size, if the size of the target changes faster, the value of β is larger, otherwise, the value of β is smaller, generally the target in two adjacent frames The size change is usually not very drastic, so you can but not limited to set the parameter β to 0.2;

该粒子在相应位置上对应的椭圆的长半轴该粒子在相应位置上对应的椭圆的短半轴

其中ε_x、ε_y是服从N(0，0.2)分布的随机变量，

是服从N(0，0.25)的随机变量，ε_a、ε_b是服从N(0，0.1)的随机变量。其中，

表示第k帧中的第n个粒子，其含义为：第k帧中ROI以概率

出现在(x，y)处，长半轴和短半轴分别为a、b，并以(v_x，v_y)的速度向下一帧移动；Third, particle state transfer, according to the previously determined temporary variables v ₁ , v ₂ and η, perform particle state transfer processing, specifically, calculate the state transfer result of the particle corresponding to the nth particle in the kth frame is: the position of the particle in the x-axis direction The position of the particle in the y-axis direction ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}={\mathrm{the\; s}}_{k-1}^{\mathrm{no}}\left\{\mathrm{the\; y}\right\}+v_2+{\mathrm{\&epsiv;}}_{\mathrm{the\; y}},$ The moving speed of the particle in the x-axis direction ${\mathrm{the\; s}}_k^{\mathrm{no}}\left\{v_x\right\}={\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="HDA0000158966620000043.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+{\mathrm{\&epsiv;}}_{v_x},$ The moving speed of the particle in the y-axis direction

The semi-major axis of the ellipse corresponding to the particle at the corresponding position The semi-minor axis of the ellipse corresponding to the particle at the corresponding position

Where ε _x , ε _y are random variables that obey the N(0, 0.2) distribution,

is a random variable subject to N(0, 0.25), and ε _a and ε _b are random variables subject to N(0, 0.1). in,

Represents the nth particle in the kth frame, which means: the ROI in the kth frame is calculated with probability

Appears at (x, y), the semi-major axis and semi-minor axis are a and b respectively, and moves to the next frame at the speed of (v _x , v _y );

(4) The color histogram statistical module of each particle area

This module is used to calculate the observation model of the particle filter theory, and the same operation needs to be performed for each particle. The following uses the calculation of the observation model of the particle filter theory for particle n as an example to illustrate.

其在当前帧的图像上可对应于以(x，y)为中心，以a、b为半长轴的椭圆形，记为

如图9所示，该模块即用于统计图中椭圆形区域的颜色直方图，并用L1范数归一化，可记为

其计算方式如下：Particle n in the current frame, the state is

It can correspond to an ellipse with (x, y) as the center and a, b as the semi-major axis on the image of the current frame, denoted as

As shown in Figure 9, this module is used for the color histogram of the elliptical area in the statistical graph, and normalized by the L1 norm, which can be recorded as

It is calculated as follows:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; sb</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; V</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; ;</span>$

是落在

中的第t个像素在各颜色通道上落入的bin的序号(相应的每个颜色通道的bin的序号从0开始递增，直到9)；

表示椭圆

中的像素个数。Among them, assuming that the video input is YUV color space, N _Y , N _U , and N _V are the number of bins divided by each color channel (obtained by quantizing the YUV space, such as dividing each dimension of the YUV space into 10 parts, then N _Y , N _U , N _V are all equal to 10);

is falling on

The serial number of the bin that the tth pixel in each color channel falls into (the corresponding serial number of the bin of each color channel increases from 0 to 9);

Represents an ellipse

The number of pixels in .

Specifically, the realization method used to calculate the observation model of particle filter theory in this module is the same as that of the prior art, except that the input of this module is the state transition result of particles obtained according to the method of the present invention ${\mathrm{the\; s}}_k^{\mathrm{no}}=\{x,v_x,\mathrm{the\; y},v_{\mathrm{the\; y}},a,b\}.$

(5) Particle update module

This module is used to perform corresponding particle update processing, specifically for calculating the weight of the particles (that is, the output of the particle transfer module) that represent the ROI state distribution in the current frame obtained through state transfer, and obtain the weight of the particles in the current frame through resampling. The distribution of ROIs is estimated, and the state transition parameter α is updated.

Specifically, as shown in Figure 10, the corresponding particle update process may include:

表示第k帧中第n个粒子的权重(粒子所代表的区域是ROI区域的概率)，这里的N则表示所用颜色直方图的bin数目，N＝N_YN_UN_V；

表示在上一模块(统计各粒子区域颜色直方图模块)中统计得到的粒子n的颜色直方图第j个bin的值；P(sb₀)是指初始化的ROI区域的颜色直方图；参数σ为常数，可以但不限于设置为0.25；归一化各粒子权重 ${\mathrm{\&pi;}}_k^n={\mathrm{\&pi;}}_k^n/\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^n.$ First, update the weight of each particle ${\mathrm{\&pi;}}_k^{\mathrm{no}}={\mathrm{\&pi;}}_{k-1}^{\mathrm{no}}\&times;\exp {\{-\{1-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\sqrt{P\left({\mathrm{sb}}_0\right)\left[j\right]\&times;P\left({\mathrm{sb}}_k^{\mathrm{no}}\right)\left[j\right]}\}}^2/\mathrm{\&sigma;}\},$ in,

Represents the weight of the nth particle in the kth frame (the region represented by the particle is the probability of the ROI region), where N represents the number of bins in the color histogram used, N=N _Y N _U N _V ;

Indicates the value of the jth bin of the color histogram of particle n obtained in the previous module (statistical color histogram module of each particle area); P(sb ₀ ) refers to the color histogram of the initialized ROI area; parameter σ is a constant, it can be set to 0.25 but not limited to; normalize the weight of each particle ${\mathrm{\&pi;}}_k^{\mathrm{no}}={\mathrm{\&pi;}}_k^{\mathrm{no}}/\underset{\mathrm{no}=1}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^{\mathrm{no}}.$

之后判断

如果有效粒子数

则进行相应的重采样操作，否则不执行重采样操作，直接执行后面第三步更新状态转移参数α，其中，N为粒子数目，λ是一常数参数，可以预先设定；具体地，相应的重采样操作包括：对当前帧的粒子集合

按照权重

进行采样，放入新的粒子集合

中，即粒子

有

的可能性加入新的粒子集合S′中，重采样后更新S′中各粒子权重

令S＝S′，以完成相应的重采样操作过程。Secondly, perform a resampling operation, which may specifically include: First, calculate the number of effective particles

judge later

If the effective number of particles

Then perform the corresponding resampling operation, otherwise do not perform the resampling operation, and directly execute the third step to update the state transition parameter α, where N is the number of particles, and λ is a constant parameter, which can be set in advance; specifically, the corresponding The resampling operation includes: collection of particles for the current frame

according to weight

Sampling, into a new collection of particles

in the particle

have

The possibility of adding the new particle set S′, update the weight of each particle in S′ after resampling

Let S=S' to complete the corresponding resampling operation process.

Third, update the state transition parameter α:

(6) Calculate ROI position size module

现在可以计算当前帧中的ROI的位置大小，ROI中心的位置 $(x,y)=(\underset{n=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^n\&times;s_k^n\{x\},\underset{n=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^n\&times;s_k^n\{y\left\}\right),$ ROI的边长 $a=\underset{n=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^n\&times;s_k^n\left\{a\right\},$ $b=\underset{n=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^n\&times;s_k^n\left\{b\right\}.$ After the calculation of the above modules, the particle set in the current frame has been obtained

Now it is possible to calculate the position size of the ROI in the current frame, and the position of the center of the ROI $(x,\mathrm{the\; y})=(\underset{\mathrm{no}=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^{\mathrm{no}}\&times;{\mathrm{the\; s}}_k^{\mathrm{no}}\{x\},\underset{\mathrm{no}=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^{\mathrm{no}}\&times;{\mathrm{the\; s}}_k^{\mathrm{no}}\{\mathrm{the\; y}\left\}\right),$ side length of ROI $a=\underset{\mathrm{no}=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^{\mathrm{no}}\&times;{\mathrm{the\; s}}_k^{\mathrm{no}}\left\{a\right\},$ $b=\underset{\mathrm{no}=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{\mathrm{\&pi;}}_k^{\mathrm{no}}\&times;{\mathrm{the\; s}}_k^{\mathrm{no}}\left\{b\right\}.$

存入参考帧的buffer(缓存)中，以便于后续的跟踪处理过程中应用。Then, it is also necessary to gather the particles representing the state of the ROI in the current frame

Stored in the buffer (cache) of the reference frame, so as to be applied in the subsequent tracking process.

The implementation of the above embodiments of the present invention makes it possible to obtain a better tracking effect with a lower complexity in the process of tracking the ROI.

Specifically, the ROI tracking scheme provided by the embodiment of the present invention is compared with the tracking scheme in the prior art, and the coastguard and stephen sequences are used as input to conduct a comparative experiment. It is found that: in the coastguard sequence, the tracking scheme in the prior art starts from the 49th frame Start to lose the tracking result, and completely lose the tracking result at frame 56, while the tracking solution proposed in the embodiment of the present invention shows that the tracking result is good; in the stephen sequence, the tracking solution in the prior art starts to lose the tracking result from the 33rd frame, to The tracking result is completely lost in the 78th frame, but the tracking solution proposed by the embodiment of the present invention also shows that the tracking result is good.

In particle filter theory, a standard used to measure the quality of the tracking scheme is the change of the number of effective particles with the tracking time. As shown in Fig. 11 and Fig. 12, the situation that the number of effective particles changes with the tracking time in the tracking scheme of the prior art and the tracking scheme proposed in the embodiment of the present invention are respectively shown in the sequence coastguard, stephen, in the two figures The upper line corresponds to the tracking scheme proposed by the embodiment of the present invention, and the lower line corresponds to the tracking scheme of the prior art. It can be seen that the number of effective particles in the tracking scheme proposed by the embodiment of the present invention decreases more slowly with the tracking time, that is, the present invention The tracking scheme proposed by the embodiment is obviously better than the tracking scheme in the prior art.

The environment in which the method and apparatus for tracking a region of interest in a video provided by an embodiment of the present invention can be applied will be illustrated below.

Application Example 1

Embodiments of the present invention provide a method and device for tracking a region of interest in a video, which can be applied to SVC (Scalable Video Coding, scalable video coding) encoding including ROI.

Specifically, the structure of an SVC encoder with ROI scalability is shown in Figure 13, and the corresponding SVC provides support for ROI encoding. ROI is often a region in a video frame that contains objects with clear high-level semantics for viewers, such as a person, an object, and so on. When a user browses a video, if the display size of the device is small, or the available bandwidth is reduced, in order not to affect the viewing experience of the video, it is necessary to maintain the clarity of the region of interest as much as possible. As shown in Figure 13, the present invention provides a method and device for tracking a region of interest in a video, which can be used in the acquisition process of the ROI region in the SVC encoder with scalable ROI, that is, it can be used to achieve the enhancement shown in Figure 13 Layer ROI area acquisition module. The input of the module is the encoded basic layer MV and video data information of the current frame. After tracking processing, the size and position information of the ROI in the current frame is output to facilitate the encoding of the ROI region of the enhancement layer.

Application Example 2

Embodiments of the present invention provide a method and device for tracking a region of interest in a video, which can be applied not only to scalable video coding technology, but also to transcoding of ROI regions.

Due to the limitations of video terminal screen size and network bandwidth, it is often necessary to transcode the existing compressed video stream into the target stream required by the client. In order to ensure the visual quality, the original compressed stream can be transcoded into the ROI stream, that is, Discard visual information that is irrelevant to the customer, and only keep the high quality of the ROI area. The structure of the corresponding cascaded transcoder is shown in Figure 14. The embodiment of the present invention provides a method and device for tracking a region of interest in a video, which can be placed in the ROI of the cascaded transcoder shown in Figure 14 The tracking module is used to obtain the location and size information of the ROI, so as to encode the ROI into a high-quality code stream.

Of course, the method and device for tracking a region of interest in a video provided by the embodiment of the present invention can also be applied to other similar application environments that need to track an ROI. No more examples here.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technical field can easily conceive of changes or changes within the technical scope disclosed in the present invention. Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. A method for tracking regions of interest in a video, comprising:

acquiring a motion vector of a pixel or a macro block in a current frame, determining a moving speed distribution parameter of an ROI (region of interest) according to the motion vector, and determining an ROI scaling parameter according to state information of the ROI in a reference frame;

and performing state transition processing on the particles obtained by sampling in the current frame by using the moving speed distribution parameter and the scaling parameter of the ROI, and determining the position and the size of the ROI of the current frame according to the particles after the state transition.

2. The method of claim 1,

the step of determining the moving speed distribution parameter of the ROI includes:

determining the position of a reference pixel or a macro block corresponding to the pixel or the macro block in the current frame in the reference frame according to the motion vector of the pixel or the macro block in the current frame, and selecting the motion vector of the pixel or the macro block in the current frame, wherein the position of the reference pixel or the macro block is positioned in the ROI in the reference frame, in the motion vector of the pixel or the macro block in the current frame; determining the moving speed distribution parameter of the ROI according to the selected motion vector of the pixel or the macro block in the current frame;

the step of determining ROI scaling parameters comprises:

and determining the corresponding pixel or macro block in the current frame according to the selected motion vector of the pixel or macro block in the current frame, and determining the ROI scaling parameter according to the pixel or macro block in the current frame and the pixel or macro block in the corresponding reference frame.

3. The method of claim 2, wherein the moving speed distribution parameters of the ROI comprise:wherein δ is a dirac function, G is a motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixel or macroblock whose position is located in the ROI in the reference frame, and MV isⁱRefers to the ith motion vector, MV, in G^ROIMoving speed of ROI from previous frame to current frame; the ROI scaling parameters include:

and a is, b is an affine parameter, the affine parameter is solved by combining a least square method and a four-parameter transformation model, and a corresponding four-parameter transformation module is established according to the pixel or the macro block in the current frame and the pixel or the macro block in the corresponding reference frame.

4. The method of claim 3, wherein the step of performing state transition processing on the sampled particles in the current frame comprises:

determining the velocity v of the particle state transition according to the moving velocity distribution parameter of the ROI₁、v₂Comprises the following steps of; generating a random number mu in a uniform distribution of 0-1, and if mu is less than alpha, making

Sign variable

Otherwise, with distribution P (MV)^ROI) Selecting two components (MV ') corresponding to one element in the motion vector set G counted in the previous module'_x，MV′_y) Respectively as v₁、v₂Value of, i.e. order v₁＝MV′_x、v₂＝MV′_yTo and from

α is a state transition parameter, an initial value of which is preset, and a mode of updating the value in a subsequent particle updating process includes:

is v in particle n in the k-1 frame_xThe components of the first and second images are,is v in particle n in the k-1 frame_yA component;

determining a particle scaling parameter η from the ROI scaling parameter, including; generating a random number gamma in a uniform distribution of 0-1, if gamma is less than beta, making eta equal to rho, otherwise making eta equal to 1, wherein beta is a target size change degree parameter, the value of beta is preset, and rho is the scaling parameter;

performing particle state transition processing according to the particle state transition speed and the particle scaling parameter, and obtaining a result after particle state transition of an nth particle in a kth frame includes: the position of the particle ( <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>x</mi> <mo>}</mo> <mo>=</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>x</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>v</mi> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>x</mi> </msub> <mo>,</mo> </mrow> </math> <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>y</mi> <mo>}</mo> <mo>=</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>y</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>v</mi> <mn>2</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>y</mi> </msub> </mrow> </math> ) The moving speeds of the particles in the x and y directions are respectively as follows: <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <msub> <mi>v</mi> <mi>x</mi> </msub> <mo>}</mo> <mo>=</mo> <msub> <mi>v</mi> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <msub> <mi>v</mi> <mi>x</mi> </msub> </msub> </mrow> </math> and <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <msub> <mi>v</mi> <mi>y</mi> </msub> <mo>}</mo> <mo>=</mo> <msub> <mi>v</mi> <mn>2</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <msub> <mi>v</mi> <mi>y</mi> </msub> </msub> <mo>,</mo> </mrow> </math> the major semi-axis of the ellipse corresponding to the particle at the position is <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>a</mi> <mo>}</mo> <mo>=</mo> <mi>&eta;</mi> <mo>&times;</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>a</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>a</mi> </msub> <mo>,</mo> </mrow> </math> Short semi-axis is <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>b</mi> <mo>}</mo> <mo>=</mo> <mi>&eta;</mi> <mo>&times;</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>b</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>b</mi> </msub> <mo>,</mo> </mrow> </math> Wherein epsilon_x、ε_yIs a random variable obeying a distribution of N (0, 0.2),

is a random variable, ε, obeying N (0, 0.25)_a、ε_bIs a random variable obeying N (0, 0.1)。

5. The method according to any of claims 1-4, wherein the step of determining the ROI of the current frame from the state-transferred particles comprises:

performing the statistical processing of the color histogram of each particle area on the particles subjected to the state transition processing, and performing particle updating processing according to the statistical processing result of the color histogram of each particle area;

the ROI position size is calculated based on the result obtained by the particle update process.

6. An apparatus for tracking regions of interest in a video, comprising:

a moving speed distribution parameter and scaling parameter determining module, for obtaining the motion vector of the pixel or macro block in the current frame, and determining the moving speed distribution parameter of the ROI according to the motion vector, and also for determining the scaling parameter according to the state information of the ROI in the reference frame;

the particle state transition module is used for carrying out state transition processing on the particles sampled and obtained in the current frame by utilizing the moving speed distribution parameters and the scaling parameters of the ROI determined by the moving speed distribution parameter and scaling parameter determining module;

and the ROI determining module is used for determining the position and the size of the ROI of the current frame according to the particles subjected to state transfer by the particle state transfer module.

7. The apparatus of claim 6, wherein the moving speed distribution parameter and scaling parameter determining module specifically comprises:

the motion vector acquisition module is used for determining the position of a reference pixel or a macro block corresponding to the pixel or the macro block in the current frame in the reference frame according to the motion vector of the pixel or the macro block in the current frame, and selecting the motion vector of the pixel or the macro block in the current frame corresponding to the reference pixel or the macro block of which the position is positioned in the ROI in the reference frame from the motion vector of the pixel or the macro block in the current frame;

the parameter determining module is used for determining the moving speed distribution parameter and the zooming parameter of the ROI according to the motion vector of the pixel or the macro block in the current frame selected and obtained by the motion vector obtaining module; and determining the corresponding pixel or macro block in the current frame according to the selected motion vector of the pixel or macro block in the current frame, and determining the ROI scaling parameter according to the pixel or macro block in the current frame and the pixel or macro block in the corresponding reference frame.

8. The apparatus of claim 7, wherein the parameter determination module comprises:

a moving speed distribution parameter estimation module of the ROI, configured to estimate a moving speed distribution parameter of the ROI, and the estimating of the obtained moving speed distribution parameter of the ROI includes: <math> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msup> <mi>MV</mi> <mi>ROI</mi> </msup> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mo>|</mo> <mi>G</mi> <mo>|</mo> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <mi>&delta;</mi> <mrow> <mo>(</mo> <msup> <mi>MV</mi> <mi>i</mi> </msup> <mo>-</mo> <msup> <mi>MV</mi> <mi>ROI</mi> </msup> <mo>)</mo> </mrow> <mo>/</mo> <mo>|</mo> <mi>G</mi> <mo>|</mo> <mo>,</mo> </mrow> </math> wherein δ is a dirac function, G is a motion vector set of pixels or macroblocks in the current frame corresponding to the reference pixel or macroblock whose position is located in the ROI in the reference frame, and MV isⁱRefers to the ith motion vector, MV, in G^ROIMoving speed of ROI from previous frame to current frame;

an ROI scaling parameter estimation module, configured to estimate an ROI scaling parameter, and estimating the obtained ROI scaling parameter includes:

and a is, b is an affine parameter, the affine parameter is solved by combining a least square method with a four-parameter transformation model, and a corresponding four-parameter transformation module is established according to the pixel or the macro block in the current frame and the pixel or the macro block in the corresponding reference frame.

9. The apparatus of claim 8, wherein the particle state transition module comprises:

a particle position and velocity transfer module for determining the velocity v of particle state transfer according to the ROI moving velocity distribution parameter obtained by the ROI moving velocity distribution parameter estimation module₁、v₂Comprises the following steps of; generating a random number mu in a uniform distribution of 0-1, and if mu is less than alpha, making

Sign variable

Otherwise, with distribution P (MV)^ROI) Selecting two components (MV ') corresponding to one element in the motion vector set G counted in the previous module'_x，MV′_y) Respectively as v₁、v₂Value of, i.e. order v₁＝MV′_x、v₂＝MV′_yTo and fromα is a state transition parameter, an initial value of which is preset, and a mode of updating the value in a subsequent particle updating process includes:

is v in particle n in the k-1 frame_xThe components of the first and second images are,

is v in particle n in the k-1 frame_yA component;

the particle size transfer module is used for determining a particle scaling parameter eta according to the ROI scaling parameter obtained by the ROI scaling parameter estimation module, and comprises the following steps of; generating a random number gamma in a uniform distribution of 0-1, if gamma is less than beta, making eta equal to rho, otherwise making eta equal to 1, wherein beta is a target size change degree parameter, the value of beta is preset, and rho is the ROI scaling parameter;

a particle state transition result determining module, configured to perform particle state transition processing according to the particle state transition speed determined by the particle position and speed transferring module and the particle scaling parameter determined by the particle size transferring module, where obtaining a result after particle state transition of an nth particle in a kth frame includes: the position of the particle ( <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>x</mi> <mo>}</mo> <mo>=</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>x</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>v</mi> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>x</mi> </msub> <mo>,</mo> </mrow> </math> <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>y</mi> <mo>}</mo> <mo>=</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>y</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>v</mi> <mn>2</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>y</mi> </msub> </mrow> </math> ) The moving speeds of the particles in the x and y directions are respectively as follows: <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <msub> <mi>v</mi> <mi>x</mi> </msub> <mo>}</mo> <mo>=</mo> <msub> <mi>v</mi> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <msub> <mi>v</mi> <mi>x</mi> </msub> </msub> </mrow> </math> and <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <msub> <mi>v</mi> <mi>y</mi> </msub> <mo>}</mo> <mo>=</mo> <msub> <mi>v</mi> <mn>2</mn> </msub> <mo>+</mo> <msub> <mi>&epsiv;</mi> <msub> <mi>v</mi> <mi>y</mi> </msub> </msub> <mo>,</mo> </mrow> </math> the major semi-axis of the ellipse corresponding to the particle at the position is <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>a</mi> <mo>}</mo> <mo>=</mo> <mi>&eta;</mi> <mo>&times;</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>a</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>a</mi> </msub> <mo>,</mo> </mrow> </math> Short semi-axis is <math> <mrow> <msubsup> <mi>s</mi> <mi>k</mi> <mi>n</mi> </msubsup> <mo>{</mo> <mi>b</mi> <mo>}</mo> <mo>=</mo> <mi>&eta;</mi> <mo>&times;</mo> <msubsup> <mi>s</mi> <mrow> <mi>k</mi> <mo>-</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mo>{</mo> <mi>b</mi> <mo>}</mo> <mo>+</mo> <msub> <mi>&epsiv;</mi> <mi>b</mi> </msub> <mo>,</mo> </mrow> </math> Wherein epsilon_x、ε_yIs a random variable obeying a distribution of N (0, 0.2),

is a random variable, ε, obeying N (0, 0.25)_a、ε_bIs a random variable obeying N (0, 0.1).

10. The apparatus of any one of claims 6-9, wherein the ROI determination module comprises:

each particle area color histogram statistical processing module is used for executing each particle area color histogram statistical processing on the particles after the state transition processing;

the particle updating module is used for performing particle updating processing according to the statistical processing result of the color histogram of each particle area;

and the ROI position size calculation module is used for calculating the ROI position size according to the result obtained by the particle updating processing.

Images