CN114494338A - Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking - Google Patents
Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking Download PDFInfo
- Publication number
- CN114494338A CN114494338A CN202111574194.2A CN202111574194A CN114494338A CN 114494338 A CN114494338 A CN 114494338A CN 202111574194 A CN202111574194 A CN 202111574194A CN 114494338 A CN114494338 A CN 114494338A
- Authority
- CN
- China
- Prior art keywords
- hand
- forearm
- key point
- kalman filtering
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000001914 filtration Methods 0.000 title claims abstract description 55
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 25
- 210000000245 forearm Anatomy 0.000 claims abstract description 138
- 238000001514 detection method Methods 0.000 claims abstract description 85
- 210000000707 wrist Anatomy 0.000 claims abstract description 45
- 230000008447 perception Effects 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 10
- 230000004048 modification Effects 0.000 claims description 3
- 238000012986 modification Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 210000004247 hand Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Abstract
The application relates to a hand real-time perception method based on adaptive positioning and Kalman filtering tracking. The method comprises the following steps: acquiring an RGB image and preprocessing the RGB image; inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points; extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair; obtaining a first prediction result for a forearm key point pair based on an optimized Kalman filtering forearm tracker; generating a hand local area in a self-adaptive manner by utilizing a first prediction result, and inputting an RGB image corresponding to the hand local area into a trained BlazePalm hand detection model to obtain a hand detection frame; and acquiring a second prediction result for the hand detection box based on an optimized Kalman filtering hand detection box tracker to serve as a hand real-time sensing result. This application can realize stable, real-time and accurate hand perception result.
Description
Technical Field
The application relates to the technical field of computer vision, in particular to a hand real-time perception method based on adaptive positioning and Kalman filtering tracking.
Background
The ability of perceiving hand shape and hand motion can be applied to a plurality of scenes, realizes the intellectuality of equipment. For example, it may form the basis for sign language understanding and gesture control, and may enable digital content and information overlays of the physical world in augmented reality scenarios.
However, the actual application scene is complex, and a detection model in the hand perception algorithm is easily affected by target distance, illumination, shielding, background contrast and the like, so that real-time detection of hands is unstable, and further problems occur in downstream application. For example, under the condition of 720p resolution and using a lightweight hand detection model, the distance between a detection target and a camera exceeds 2.5 meters, at this time, the hand target is too small, and the lightweight hand detection model cannot be stably sufficient for detecting small targets, so that a large number of missed detection situations occur in hand detection, and further relevant applications based on hands cannot be completed. Stable real-time hand detection is therefore a very challenging computer vision task.
Disclosure of Invention
Based on the technical problems, the invention aims to solve the unstable situation of hand perception, and provides a hand real-time perception method adopting adaptive hand region coarse positioning and improved Kalman filtering, wherein a human body posture estimation model, a hand detector and a Kalman filtering tracker are used in the method.
The invention provides a hand real-time sensing method based on adaptive positioning and Kalman filtering tracking, which comprises the following steps:
acquiring an RGB image and preprocessing the RGB image;
inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points;
extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair;
if the forearm key point pair has a point in a default state, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker;
a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame;
and if the hand detection frame has a point in a default state, acquiring a second prediction result for the hand detection frame as a hand real-time perception result based on an optimized Kalman filtering hand detection frame tracker.
Further, the method further comprises: and if the forearm key point pair is in a complete state and is in the first needle image, setting a forearm key point tracker for the forearm key point pair, wherein the forearm key point tracker needs to initialize parameters.
Specifically, the optimization-based kalman filtering forearm tracker obtains a first prediction result for the forearm key-point pair, including:
the forearm tracker predicts the t-moment state of the t-1 moment state of the forearm key point pair and records the t-moment state as a predicted value;
acquiring the observation state of the forearm key point pair at the time t and recording the observation state as an observation value;
and acquiring the optimal matching pair of the forearm key point pair as a first prediction result based on the predicted value and the observed value.
Further preferably, before the obtaining an optimal matching pair of a forearm key point pair based on the predicted value and the observed value as a first prediction result, the method further includes:
respectively calculating Euclidean distances corresponding to a wrist point and an elbow point;
calculating an average distance based on Euclidean distances corresponding to the wrist point and the elbow point to obtain a loss matrix;
and obtaining the minimum loss by adopting a Hungarian algorithm based on the loss matrix.
Still more specifically, said adaptively generating a local area of a hand using said first prediction comprises:
acquiring a wrist point and an elbow point in the first prediction result;
calculating a forearm two-dimensional vector mode according to the wrist point and the elbow point;
calculating a two-dimensional vector module of the upper arm according to the elbow point and the shoulder point;
calculating the coordinates and the length and width values of the hand local area based on the judgment formulas of the forearm two-dimensional vector model and the upper arm two-dimensional vector model;
and adaptively adjusting the coordinates and the length and width values to the hand local area.
Still further, the judgment formula based on the forearm two-dimensional vector model and the upper arm two-dimensional vector model calculates the coordinates and the length and width values of the hand local area, and the formula is as follows:
wherein, cx，cyX-axis coordinate and y-axis coordinate, Dist, respectively, representing local areas of the handafRepresenting the two-dimensional vector norm, Dist, of the forearmauRepresenting the two-dimensional vector norm, w, of the upper armiIndicates the wrist point, eiRepresenting the elbow points, siIndicates shoulder point and l indicates width.
Further preferably, the optimized kalman filtering includes a prediction phase and an update phase and a correction to the update phase; the correction to the update phase is based on a kalman gain, wherein the kalman gain is multiplied by an enhancement factor or by an attenuation factor.
The invention provides a hand real-time sensing device based on adaptive positioning and Kalman filtering tracking, which comprises:
the system comprises an acquisition module, a preprocessing module and a display module, wherein the acquisition module is used for acquiring an RGB image and preprocessing the RGB image;
the preprocessing module is used for inputting the preprocessed RGB images into the trained OpenPose human posture estimation model to obtain a first preset number of human key points;
the extraction module is used for extracting a second preset number of wrist points, elbow points and shoulder points from the first preset number of human body key points to form a forearm key point pair and an upper arm key point pair;
the first prediction module is used for obtaining a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker if the forearm key point pair has a point in a default state;
the hand detection module is used for generating a hand local area in a self-adaptive mode by utilizing the first prediction result, inputting the RGB image corresponding to the hand local area into a trained BlazePalm hand detection model, and obtaining a hand detection frame;
and the second prediction module is used for acquiring a second prediction result for the hand detection frame based on the optimized Kalman filtering hand detection frame tracker as a hand real-time perception result if a point of a default state exists in the hand detection frame.
A third aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring an RGB image and preprocessing the RGB image;
inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points;
extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair;
if the forearm key point pair has a point in a default state, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker;
a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame;
and if the hand detection frame has a default state point, acquiring a second prediction result for the hand detection frame as a hand real-time sensing result based on the optimized Kalman filtering hand detection frame tracker.
A fourth aspect of the invention provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
acquiring an RGB image and preprocessing the RGB image;
inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points;
extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair;
if the forearm key point pair has a point in a default state, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker;
a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame;
and if the hand detection frame has a default state point, acquiring a second prediction result for the hand detection frame as a hand real-time sensing result based on the optimized Kalman filtering hand detection frame tracker.
The beneficial effect of this application does: the method and the device make up for the situation that an OpenPose human posture estimation model and a BlazePalm hand detection model are insufficient in detection by using an optimized Kalman filtering forearm tracker and an optimized hand detection frame tracker, so that the hand detection result is more real-time and accurate. Because the human posture estimation model has prediction errors, factors such as illumination change, shielding and background can cause the predicted position of a wrist point to drift, and the local hand area generated by depending on the wrist point can also deviate.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.
The present application may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
FIG. 1 is a schematic diagram illustrating steps of a real-time hand perception method based on adaptive positioning and Kalman filtering tracking in an exemplary embodiment of the present application;
FIG. 2 is a schematic structural diagram of a hand real-time sensing device based on adaptive positioning and Kalman filtering tracking according to an exemplary embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application;
fig. 4 is a schematic diagram of a storage medium provided in an exemplary embodiment of the present application.
Detailed Description
Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present application. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present application. It will be apparent to one skilled in the art that the present application may be practiced without one or more of these details. In other instances, well-known features of the art have not been described in order to avoid obscuring the present application.
It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the application. As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Exemplary embodiments according to the present application will now be described in more detail with reference to the accompanying drawings. These exemplary embodiments may, however, be embodied in many different forms and should not be construed as limited to only the embodiments set forth herein. The figures are not drawn to scale, wherein certain details may be exaggerated and omitted for clarity. The shapes of various regions, layers, and relative sizes and positional relationships therebetween shown in the drawings are merely exemplary, and deviations may occur in practice due to manufacturing tolerances or technical limitations, and a person skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions, as actually required.
Several examples are given below in conjunction with the description of figures 1-4 to describe exemplary embodiments according to the present application. It should be noted that the following application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. Rather, embodiments of the present application may be applied to any scenario where applicable.
Example 1:
in this embodiment, a hand real-time sensing method based on adaptive positioning and kalman filter tracking is implemented, as shown in fig. 1, including:
s1, acquiring an RGB image and preprocessing the RGB image;
s2, inputting the preprocessed RGB images into the trained OpenPose human posture estimation model to obtain a first preset number of human key points;
s3, extracting wrist points, elbow points and shoulder points with a second preset number from the human body key points with the first preset number to form a forearm key point pair and an upper arm key point pair;
s4, if the forearm key point pair has a point in a default state, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker;
s5, generating a hand local area in a self-adaptive mode by utilizing the first prediction result, inputting the RGB image corresponding to the hand local area into a trained BlazePalm hand detection model, and obtaining a hand detection frame;
and S6, if the hand detection frame has a default state point, acquiring a second prediction result for the hand detection frame as a hand real-time perception result based on the optimized Kalman filtering hand detection frame tracker.
It should be noted here that kalman filtering is a state equation based on a linear system, and is an algorithm for performing optimal estimation on a system state by inputting and outputting observation data through the system; SSD is a shorthand for Single Shot multitox Detector, representing a target detection algorithm; BlazePalm is a hand detection model developed by google; blazeface is a face detection model developed by Google; BlazeBlock is a sub-network module developed by Google, and Blazeface and BlazePalm are composed of a plurality of sub-network modules; openpos is an open-source human posture estimation model developed by the university of kangsu meilong Perceptual Computing Lab (CMU Perceptual Computing Lab); the Hungarian algorithm is a combined optimization algorithm for solving task allocation problems in polynomial time; IoU is an abbreviation for interaction over Union, a standard that measures the accuracy with which a corresponding object is detected in a particular dataset.
Further, the method further comprises: and if the forearm key point pair is in a complete state and is in the first needle image, setting a forearm key point tracker for the forearm key point pair, wherein the forearm key point tracker needs to initialize parameters.
Specifically, the method for obtaining a first prediction result for a forearm key point pair based on an optimized kalman filtering forearm tracker includes:
the forearm tracker predicts the t-moment state of the t-1 moment state of the forearm key point pair and records the t-moment state as a predicted value;
acquiring the observation state of the forearm key point pair at the time t and recording the observation state as an observation value;
and acquiring the optimal matching pair of the forearm key point pair based on the predicted value and the observed value and recording the optimal matching pair as a first prediction result.
Further preferably, before obtaining the optimal matching pair of the forearm key point pair as the first prediction result based on the predicted value and the observed value, the method further includes:
respectively calculating Euclidean distances corresponding to a wrist point and an elbow point;
calculating an average distance based on Euclidean distances corresponding to the wrist point and the elbow point to obtain a loss matrix;
and obtaining the minimum loss by adopting a Hungarian algorithm based on the loss matrix.
More specifically, the method for adaptively generating a local hand region by using a first prediction result comprises the following steps:
acquiring a wrist point and an elbow point in the first prediction result;
calculating a forearm two-dimensional vector mode according to the wrist point and the elbow point;
calculating a two-dimensional vector module of the upper arm according to the elbow point and the shoulder point;
calculating the coordinates and the length and width values of the hand local area based on the judgment formulas of the forearm two-dimensional vector model and the upper arm two-dimensional vector model;
the adaptive adjustment of coordinates and length and width values is worth to the hand local area.
And further, calculating the coordinates and the length and width values of the hand local area based on a judgment formula of a forearm two-dimensional vector model and an upper arm two-dimensional vector model, wherein the formula is as follows:
wherein, cx，cyX-axis coordinate and y-axis coordinate, Dist, respectively, representing local areas of the handafRepresenting the two-dimensional vector norm, Dist, of the forearmauRepresenting the two-dimensional vector norm, w, of the upper armiIndicates the wrist point, eiRepresenting the elbow points, siIndicates shoulder point and l indicates width.
Further preferably, the optimized kalman filtering includes a prediction phase and an update phase and a correction to the update phase; the correction to the update phase is corrected based on a kalman gain, wherein the kalman gain is multiplied by an enhancement factor or by an attenuation factor.
Example 2:
the embodiment provides a hand real-time perception method based on adaptive positioning and Kalman filtering tracking, which comprises the following steps: acquiring an RGB image and preprocessing the RGB image; inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points; extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair; if a point in a default state exists in the forearm key point pair, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker; a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame; and if the hand detection frame has a point in a default state, acquiring a second prediction result for the hand detection frame as a hand real-time perception result based on an optimized Kalman filtering hand detection frame tracker.
In one possible embodiment, the RGB image is acquired and preprocessed, the RGB image may be read from the camera device, preprocessed, resized to the input size of the human pose estimation module, and preprocessedImage pixel values are normalized, etc. And then inputting the preprocessed image into an OpenPose human posture estimation model to obtain 18 pieces of human body key point position information. The key points of the human body are extracted into 6 points of the left wrist, the right elbow, the left shoulder and the right shoulder. The left and right wrist points and the left and right elbow points constitute the left and right forearms, and the left and right elbow points and the left and right shoulder points constitute the left and right upper arms. Considering that the OpenPose human body posture estimation module has instability, which has a probability of causing the human body key point to have default condition, namely failure in position value regression, and the xy axis position information of the human body key point in the default state is assigned as-1. Thus, if there is a default state point in a pair of forearm keypoints, a first prediction result is obtained for the pair of forearm keypoints based on an optimized kalman filter forearm tracker. For example: ith forearm key point pair afiContaining the corresponding wrist point wiAnd elbow point eiI.e. afi＝{wi，eiSimilarly, the ith upper arm key point pair auiIncluding the corresponding elbow point eiAnd shoulder point siAu is ai＝{ei，siWherein a forearm key-point pair may be divided into default forearm key-points afnullFor and complete forearm key point pair afvaildTwo kinds. If the wrist point w of a forearm key point pairiOr elbow point eiBoth are in the default state (i.e., xy axis position information is-1), then the forearm keypoint is classified as the default forearm keypoint, otherwise, the forearm keypoint is classified as the complete forearm keypoint pair.
As an alternative embodiment, in some embodiments, the method further comprises: and if the forearm key point pair is in a complete state and is in the first needle image, setting a forearm key point tracker for the forearm key point pair, wherein the forearm key point tracker needs to initialize parameters. For example, if the input is the first frame image, then the key point pairs are based on each complete forearm
Wrist point of
And elbow point
Position information as initialization input for initializing a forearm key point pair tracker
Assuming that there are m complete forearm key point pairs currently, m forearm key point pair trackers can be obtained, and the m forearm key point pair trackers form a forearm key point pair tracker set
Where superscript 0 refers to the initial frame time point.
In an implementation, the method for obtaining a first prediction result for a forearm key-point pair based on an optimized kalman filtering forearm tracker includes: the forearm tracker predicts the t-moment state of the t-1 moment state of the forearm key point pair and records the t-moment state as a predicted value; acquiring the observation state of the forearm key point pair at the time t and recording the observation state as an observation value; and acquiring the optimal matching pair of the forearm key point pair based on the predicted value and the observed value and recording the optimal matching pair as a first prediction result.
Further preferably, before obtaining the optimal matching pair of the forearm key point pair as the first prediction result based on the predicted value and the observed value, the method further includes: respectively calculating Euclidean distances corresponding to a wrist point and an elbow point; calculating an average distance based on Euclidean distances corresponding to the wrist point and the elbow point to obtain a loss matrix; and obtaining the minimum loss by adopting a Hungarian algorithm based on the loss matrix.
More specifically, the method for adaptively generating a local hand region by using a first prediction result comprises the following steps: acquiring a wrist point and an elbow point in the first prediction result; calculating a forearm two-dimensional vector mode according to the wrist point and the elbow point; calculating a two-dimensional vector module of the upper arm according to the elbow point and the shoulder point; calculating the coordinates and the length and width values of the hand local area based on the judgment formulas of the forearm two-dimensional vector model and the upper arm two-dimensional vector model; the adaptive adjustment of coordinates and length and width values is worth to the hand local area. The calculation formulas of the forearm two-dimensional vector model and the upper arm two-dimensional vector model are as follows:
Distaf＝||wi-ei||2
Distau＝||ei-si||2
and further, calculating the coordinates and the length and width values of the hand local area based on a judgment formula of a forearm two-dimensional vector model and an upper arm two-dimensional vector model, wherein the formula is as follows:
wherein, cx，cyX-axis coordinate and y-axis coordinate, Dist, respectively, representing local areas of the handafRepresenting the two-dimensional vector norm, Dist, of the forearmauRepresenting the two-dimensional vector norm, w, of the upper armiIndicates the wrist point, eiRepresenting the elbow points, siIndicates shoulder points, l indicates width, since the aspect ratio of the hand detection box is 1: 1, the length and width of the hand region frame are also the same, and l is set. 0.33 in the above formula is obtained by counting the ratio of the length of the palm of the human body to the length of the forearm and the length of the upper arm, 0.9 is obtained by counting the ratio of the length of the forearm and the length of the upper arm of the human body, and 1.6 is the optimal regional scale factor found in practice. The resulting hand region box can be expressed in "(upper left x-axis position, upper left y-axis position, width, height)" form, i.e. as follows:
Areahand＝(cx-l*0.5，cy-l*0，5，l，l)
through the formula, the algorithm can adaptively adjust the hand region frame according to the current arm condition, for example, when the hand is straightened forward, the wrist point and the elbow point are almost overlapped on the two-dimensional plane, the distance between the wrist point and the elbow point is very small, and the distance between the elbow point and the shoulder point is still kept to be a certain distance, so that the size and the position of the hand region frame can be determined by the length of the upper arm of the algorithm, and the problem of excessively depending on the wrist point is solved.
It should be noted that the forearm keypoint tracker and the hand detection frame tracker in the present application are implemented based on modified enhanced kalman filtering, which optimizes a modification stage relative to conventional kalman filtering, i.e., the optimized kalman filtering includes a prediction stage, an update stage, and a modification to the update stage; the correction to the update phase is corrected based on a kalman gain, wherein the kalman gain is multiplied by an enhancement factor or by an attenuation factor. The tracking algorithm of the traditional kalman filter comprises a prediction phase, in which the coordinates (x) of the moving object at time t are first assumed, and an update phaset，yt) The transport speed is (v)x，vy) Then the state value statt＝(xt，yt，vx，vy)TObserved value of
The state transition matrix a in the system state prediction equation is:
the general tracking algorithm defaults to having no input for the control variables, so the state equation can be defined as:
in addition to the need for a state prediction equation, an observation equation is also needed, where the observation matrix is H, with the purpose of transforming the observation matrix shape to correspond to the shape of the state matrix, and the observation equation can be defined as:
based on the above definition, let
The a priori state estimate at time t is an unreliable estimate made by the algorithm based on the a posteriori estimates at the previous time point. The corresponding prediction equation obtained by combining the state equation is as follows:
wherein, wtThen it is expressed as:
in addition, let the prior estimated covariance at time t be
Where Q is the covariance of the excitation noise.
In the update stage, a correction is mainly made to the unreliable estimation, and the update formula is as follows:
wherein the content of the first and second substances,
a representation of an unreliable estimate is made of,
representing the estimated value of the posterior state, KtThe method is expressed in terms of the kalman gain,
representing the a posteriori estimated covariance. Modified enhanced Kalman Filter is mainly used for the update stage
And (3) optimizing, namely calculating errors of the posterior estimation state value and the observed value at a previous time point, wherein the error calculation modes are different under different applications, the error calculation of the forearm key point tracker is based on Euclidean distance, and the hand detection box tracker is based on IoU values. When the error value is smaller than the set weak qualified threshold value, the Kalman gain is multiplied by an enhancement factor; if the error value is larger than the set strong qualified threshold value, the Kalman gain is multiplied by an attenuation factor; if the error value is between the weak qualified threshold and the strong qualified threshold, the Kalman gain is not changed. The corresponding error formula is as follows:
the error calculation of the forearm keypoint tracker is based on the Euclidean distance (below the corresponding formula), the hand detection frame tracker is based on IoU values (above the corresponding formula), and the corresponding prior state estimation value correction formula is as follows:
where Φ is the enhancement factor, set to 2 optimal in practice; Γ is the attenuation factor, which in practice is preferably set to 0.9.
The Kalman gain actually plays a balance role between the prior state estimation and the observation value, if the Kalman gain is enhanced, the system is shown to believe the observation value more currently, and therefore the correction strength of the prior state estimation is increased; if the kalman gain is attenuated, the representation system considers that the current posterior state estimation may over-fit the observed value, which results in reduced robustness, and if the observed value has an error, the system estimation is also easily affected, thereby weakening the correction strength of the prior state estimation.
The optimized Kalman filtering provides support for the forearm tracker and the hand detection frame tracker, so that the forearm tracker and the hand detection frame tracker can obtain an optimal matching pair, and the first prediction result and the second prediction result obtained by the forearm tracker and the hand detection frame tracker are clear and stable and achieve a real-time technical effect by correcting the updating stage of the optimized Kalman filtering. For example, assume that it is currently the t-th frame and there is mt-1A tracker, m firstt-1The tracker needs to predict the state of the current time point t according to the state value at the time t-1 to obtain the prior predicted value of the wrist point
And elbow point prior predicted value
For example, get n forearm key point pairs, including all afsnullAnd afvaildTwo kinds, each afiWrist point w iniAnd elbow point eiIs required to react with mt-1Wrist point prior prediction value of individual tracker
And elbow point prior predicted value
Respectively calculating corresponding Euclidean distances to obtain dwijAnd deijThen calculate the average distance
To obtainLoss matrix
And then, using a Hungarian algorithm based on the loss matrix D to search for a minimum loss combination, and calculating minimum loss pairing sets of all current forearm key point pairs and all current trackers
Where k denotes that there are k pairs of optimal matching pairs kmt＝{af，ktt-1K is less than or equal to mt-1And K is greater than or equal to n. If k' forearm key point pairs af incapable of being paired existmissAnd k ' of k ' forearm keypoints are k 'vaildInitializing a forearm key point pair tracker and putting the forearm key point pair tracker set KT into based on the wrist point and elbow point position information of each complete forearm key point pair as initialization inputt(ii) a If k' trackers which cannot be paired appear
The lifetime counters of k "trackers self-increment by 1, and when the lifetime counter reaches the clear threshold, the tracker goes from KTtMiddle elimination, assuming that m is eliminated altogetherdelA tracker, the last t frame's forearm key point pair tracker set KTtThe number of middle trackers is updated as: m ist＝mt-1+k′vaild-mdel。
The difference between the work process of the hand detection box tracker and the work process of the forearm tracker is that before the optimized kalman filtering-based hand detection box tracker obtains a second prediction result for the hand detection box as a real-time hand sensing result, the method further comprises the following steps: iou values corresponding to the wrist point and the elbow point are respectively calculated; deriving a loss matrix based on the value of Iou; and obtaining the minimum loss by adopting a Hungarian algorithm based on the loss matrix. Other similar operations to those of the forearm tracker will not be described in detail herein. In addition, when the trackers cannot be paired, the reason is that the human body key points in the default state cannot be obtained through the estimation and prediction of the human body posture, and therefore the priori predicted values are supplemented to the corresponding missing key points.
The method and the device make up for the situation that an OpenPose human posture estimation model and a BlazePalm hand detection model are insufficient in detection by using an optimized Kalman filtering forearm tracker and an optimized hand detection frame tracker, so that the hand detection result is more real-time and accurate. Because the human posture estimation model has prediction errors, factors such as illumination change, shielding and background can cause the predicted position of a wrist point to drift, and the local hand area generated by depending on the wrist point can also deviate.
Example 3:
this embodiment provides a hand real-time perception device based on self-adaptation location and kalman filter tracking, as shown in fig. 2, the device includes:
an obtaining module 301, configured to obtain an RGB image and perform preprocessing on the RGB image;
the preprocessing module 302 is configured to input the preprocessed RGB images into a trained openpos human posture estimation model to obtain a first preset number of human key points;
the extracting module 303 is configured to extract a second preset number of wrist points, elbow points, and shoulder points from the first preset number of human body key points to form a forearm key point pair and an upper arm key point pair;
a first prediction module 304, configured to obtain a first prediction result for a forearm keypoint pair based on an optimized kalman filter forearm tracker if there is a point in a default state in the forearm keypoint pair;
the hand detection module 305 is configured to generate a hand local region in a self-adaptive manner by using the first prediction result, and input an RGB image corresponding to the hand local region into a trained BlazePalm hand detection model to obtain a hand detection frame;
the second prediction module 306 is configured to, if there is a point in the hand detection box in the default state, obtain a second prediction result for the hand detection box based on the optimized kalman filter hand detection box tracker as a real-time hand sensing result.
Reference is now made to fig. 3, which is a schematic diagram illustrating an electronic device provided in some embodiments of the present application. As shown in fig. 3, the electronic device 2 includes: the system comprises a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program that can be executed on the processor 200, and the processor 200 executes the computer program to perform the method for real-time hand perception based on adaptive positioning and kalman filter tracking provided by any of the foregoing embodiments of the present application, where the electronic device may be an electronic device with a touch-sensitive display.
The Memory 201 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The processor 200 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 200. The Processor 200 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the method in combination with the hardware thereof.
The electronic equipment provided by the embodiment of the application and the hand real-time sensing method based on the adaptive positioning and the Kalman filtering tracking provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the electronic equipment.
Referring to fig. 4, the computer readable storage medium shown in fig. 4 is an optical disc 30, on which a computer program (i.e., a program product) is stored, and when the computer program is executed by a processor, the hand real-time sensing method based on adaptive positioning and kalman filter tracking provided in any of the foregoing embodiments is executed.
In addition, examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, or other optical and magnetic storage media, which are not described in detail herein.
The computer-readable storage medium provided by the above-mentioned embodiment of the present application and the quantum key distribution channel allocation method in the spatial division multiplexing optical network provided by the embodiment of the present application have the same inventive concept, and have the same beneficial effects as the method adopted, run, or implemented by the application program stored in the computer-readable storage medium.
The present application further provides a computer program product, including a computer program, which when executed by a processor, implements the steps of the hand real-time sensing method based on adaptive positioning and kalman filter tracking provided in any of the foregoing embodiments, where the steps of the method include: acquiring an RGB image and preprocessing the RGB image; inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points; extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair; if the forearm key point pair has a point in a default state, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker; a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame; and if the hand detection frame has a default state point, acquiring a second prediction result for the hand detection frame as a hand real-time sensing result based on the optimized Kalman filtering hand detection frame tracker.
It should be noted that: the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application. In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification, and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except that at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the creation apparatus of a virtual machine according to embodiments of the present application. The present application may also be embodied as an apparatus or device program for carrying out a portion or all of the methods described herein. A program implementing the application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A hand real-time perception method based on adaptive positioning and Kalman filtering tracking is characterized by comprising the following steps:
acquiring an RGB image and preprocessing the RGB image;
inputting the preprocessed RGB images into a trained OpenPose human posture estimation model to obtain a first preset number of human key points;
extracting wrist points, elbow points and shoulder points of a second preset number from the human body key points of the first preset number to form a forearm key point pair and an upper arm key point pair;
if a point in a default state exists in the forearm key point pair, acquiring a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker;
a hand local area is generated by utilizing the first prediction result in a self-adaptive mode, and RGB images corresponding to the hand local area are input into a trained BlazePalm hand detection model to obtain a hand detection frame;
and if the hand detection frame has a default state point, acquiring a second prediction result for the hand detection frame as a hand real-time sensing result based on the optimized Kalman filtering hand detection frame tracker.
2. The real-time hand perception method based on adaptive positioning and Kalman filtering tracking according to claim 1, characterized in that the method further comprises: and if the forearm key point pair is in a complete state and is in the first needle image, setting a forearm key point tracker for the forearm key point pair, wherein the forearm key point tracker needs to initialize parameters.
3. The method of claim 1 or 2, wherein the Kalman filtering based hand real-time perception method is characterized in that the optimization-based Kalman filtering forearm tracker obtains a first prediction result for the forearm key point pair, and comprises:
the forearm tracker predicts the t-moment state of the t-1 moment state of the forearm key point pair and records the t-moment state as a predicted value;
acquiring the observation state of the forearm key point pair at the time t and recording the observation state as an observation value;
and acquiring the optimal matching pair of the forearm key point pair as a first prediction result based on the predicted value and the observed value.
4. The method for real-time hand perception based on adaptive positioning and kalman filter tracking according to claim 3, wherein before obtaining the optimal matching pair of the forearm key point pair based on the predicted value and the observed value as the first prediction result, the method further comprises:
respectively calculating Euclidean distances corresponding to a wrist point and an elbow point;
calculating an average distance based on Euclidean distances corresponding to the wrist point and the elbow point to obtain a loss matrix;
and obtaining the minimum loss by adopting a Hungarian algorithm based on the loss matrix.
5. The method for real-time hand perception based on adaptive positioning and Kalman filtering tracking according to claim 3, wherein the adaptively generating a local hand region by using the first prediction result comprises:
acquiring a wrist point and an elbow point in the first prediction result;
calculating a forearm two-dimensional vector mode according to the wrist point and the elbow point;
calculating a two-dimensional vector module of the upper arm according to the elbow point and the shoulder point;
calculating the coordinates and the length and width values of the hand local area based on the judgment formulas of the forearm two-dimensional vector model and the upper arm two-dimensional vector model;
and adaptively adjusting the coordinates and the length and width values to the hand local area.
6. The method for real-time hand perception based on adaptive positioning and Kalman filtering tracking according to claim 5, wherein the judgment formula based on the forearm two-dimensional vector model and the upper arm two-dimensional vector model is used for calculating the coordinates and the length and width values of the local hand area, and the formula is as follows:
wherein, cx，cyX-axis coordinate and y-axis coordinate, Dist, respectively, representing local areas of the handafRepresenting two-dimensional orientation of forearmMeasuring mode, DistauRepresenting the two-dimensional vector norm, w, of the upper armiIndicates the wrist point, eiRepresenting the elbow points, siIndicates shoulder point and l indicates width.
7. The real-time hand perception method based on adaptive positioning and Kalman filtering tracking according to claim 4, characterized in that the optimized Kalman filtering includes a prediction phase and an update phase and a modification to the update phase; the correction to the update phase is based on a kalman gain, wherein the kalman gain is multiplied by an enhancement factor or by an attenuation factor.
8. A hand real-time perception device based on adaptive positioning and Kalman filtering tracking is characterized in that the device comprises:
the system comprises an acquisition module, a preprocessing module and a display module, wherein the acquisition module is used for acquiring an RGB image and preprocessing the RGB image;
the preprocessing module is used for inputting the preprocessed RGB images into the trained OpenPose human posture estimation model to obtain a first preset number of human key points;
the extraction module is used for extracting a second preset number of wrist points, elbow points and shoulder points from the first preset number of human body key points to form a forearm key point pair and an upper arm key point pair;
the first prediction module is used for obtaining a first prediction result for the forearm key point pair based on an optimized Kalman filtering forearm tracker if the forearm key point pair has a point in a default state;
the hand detection module is used for generating a hand local area in a self-adaptive mode by utilizing the first prediction result, inputting the RGB image corresponding to the hand local area into a trained BlazePalm hand detection model, and obtaining a hand detection frame;
and the second prediction module is used for acquiring a second prediction result for the hand detection frame based on the optimized Kalman filtering hand detection frame tracker as a real-time hand sensing result if the hand detection frame has a point in a default state.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111574194.2A CN114494338A (en) | 2021-12-21 | 2021-12-21 | Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111574194.2A CN114494338A (en) | 2021-12-21 | 2021-12-21 | Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494338A true CN114494338A (en) | 2022-05-13 |
Family
ID=81493786
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111574194.2A Pending CN114494338A (en) | 2021-12-21 | 2021-12-21 | Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494338A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170263005A1 (en) * | 2016-03-10 | 2017-09-14 | Sony Corporation | Method for moving object detection by a kalman filter-based approach |
| US20180024641A1 (en) * | 2016-07-20 | 2018-01-25 | Usens, Inc. | Method and system for 3d hand skeleton tracking |
| CN108919943A (en) * | 2018-05-22 | 2018-11-30 | 南京邮电大学 | A kind of real-time hand method for tracing based on depth transducer |
| CN110909685A (en) * | 2019-11-25 | 2020-03-24 | 广州织点智能科技有限公司 | Posture estimation method, device, equipment and storage medium |
| CN111860274A (en) * | 2020-07-14 | 2020-10-30 | 清华大学 | Traffic police command gesture recognition method based on head orientation and upper half body skeleton characteristics |
| US20210124942A1 (en) * | 2019-10-25 | 2021-04-29 | 7-Eleven, Inc. | Object detection based on wrist-area region-of-interest |
-
2021
- 2021-12-21 CN CN202111574194.2A patent/CN114494338A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170263005A1 (en) * | 2016-03-10 | 2017-09-14 | Sony Corporation | Method for moving object detection by a kalman filter-based approach |
| US20180024641A1 (en) * | 2016-07-20 | 2018-01-25 | Usens, Inc. | Method and system for 3d hand skeleton tracking |
| CN108919943A (en) * | 2018-05-22 | 2018-11-30 | 南京邮电大学 | A kind of real-time hand method for tracing based on depth transducer |
| US20210124942A1 (en) * | 2019-10-25 | 2021-04-29 | 7-Eleven, Inc. | Object detection based on wrist-area region-of-interest |
| CN110909685A (en) * | 2019-11-25 | 2020-03-24 | 广州织点智能科技有限公司 | Posture estimation method, device, equipment and storage medium |
| CN111860274A (en) * | 2020-07-14 | 2020-10-30 | 清华大学 | Traffic police command gesture recognition method based on head orientation and upper half body skeleton characteristics |
Non-Patent Citations (2)
| Title |
|---|
| PASCHALIS PANTELERIS等: "Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild", 2018 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION, 7 May 2018 (2018-05-07), pages 436 - 445 * |
| 马敬奇;钟震宇;雷欢;吴亮生;: "人体结构化特征与核相关滤波器算法融合的目标跟踪方法", 计算机应用, no. 1, 10 July 2020 (2020-07-10), pages 61 - 65 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Engel et al. | Direct sparse odometry | |
| Hermann et al. | Iterative semi-global matching for robust driver assistance systems | |
| CN107967693B (en) | Video key point processing method and device, computing equipment and computer storage medium | |
| CN109934065B (en) | Method and device for gesture recognition | |
| US20150062010A1 (en) | Pointing-direction detecting device and its method, program and computer readable-medium | |
| JP2010500677A (en) | Image processing method | |
| CN111709973B (en) | Target tracking method, device, equipment and storage medium | |
| CN110986969B (en) | Map fusion method and device, equipment and storage medium | |
| CN108229305B (en) | Method and device for determining outer frame of target object and electronic equipment | |
| CN111696130A (en) | Target tracking method, target tracking apparatus, and computer-readable storage medium | |
| CN108122280A (en) | The method for reconstructing and device of a kind of three-dimensional point cloud | |
| JP2023502192A (en) | Visual positioning method and related apparatus, equipment and computer readable storage medium | |
| US20150262362A1 (en) | Image Processor Comprising Gesture Recognition System with Hand Pose Matching Based on Contour Features | |
| CN109948624A (en) | Method, apparatus, electronic equipment and the computer storage medium of feature extraction | |
| US9323995B2 (en) | Image processor with evaluation layer implementing software and hardware algorithms of different precision | |
| CN115705651A (en) | Video motion estimation method, device, equipment and computer readable storage medium | |
| US10713847B2 (en) | Method and device for interacting with virtual objects | |
| CN114494338A (en) | Hand real-time sensing method based on adaptive positioning and Kalman filtering tracking | |
| CN112989872A (en) | Target detection method and related device | |
| CN113255700B (en) | Image feature map processing method and device, storage medium and terminal | |
| CN114037977B (en) | Road vanishing point detection method, device, equipment and storage medium | |
| CN115546681A (en) | Asynchronous feature tracking method and system based on events and frames | |
| WO2023003646A1 (en) | Focused computer detection of objects in images | |
| CN112241967B (en) | Target tracking method, device, medium and equipment | |
| JP6388744B1 (en) | Ranging device and ranging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |