CN109785367B - Method and device for filtering foreign points in three-dimensional model tracking - Google Patents
Method and device for filtering foreign points in three-dimensional model tracking Download PDFInfo
- Publication number
- CN109785367B CN109785367B CN201910052458.4A CN201910052458A CN109785367B CN 109785367 B CN109785367 B CN 109785367B CN 201910052458 A CN201910052458 A CN 201910052458A CN 109785367 B CN109785367 B CN 109785367B
- Authority
- CN
- China
- Prior art keywords
- color
- model
- contour
- points
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides a method and a device for filtering foreign points in three-dimensional model tracking, wherein the method comprises the following steps: extracting a contour or extracting a characteristic point from the current frame image; acquiring a color model based on contour sampling points or characteristic points; reading in the next frame image, and predicting the initial position of the contour or the characteristic point in the next frame according to the motion model; optimizing the contour or searching the corresponding characteristic point in the next frame according to the initial position; calculating an energy value of the color probability distribution of the contour sampling points or the characteristic points on the basis of the color model, and filtering out external points according to the calculated energy value; and solving the posture of the next frame and replacing the next frame with the current frame. The method can solve the problems of highlight, complex environment, shielding and the like of the three-dimensional model tracking in the actual scene, thereby improving the success rate and robustness of the three-dimensional tracking.
Description
Technical Field
The embodiment of the invention relates to the field of computer vision, in particular to a method and a device for filtering out foreign points in three-dimensional model tracking.
Background
Tracking technology was primarily applied earlier to planar tracking, and specifically to tracking a specific target (such as a vehicle, a person, a billboard, etc.) in a certain amount of video sequence, and marking the position of the target in the video sequence. Target tracking has wide application in the field of computer vision: such as video monitoring, traffic monitoring, unmanned driving, face recognition, Augmented Reality (AR), and the like. For example, in face recognition, the position of a face needs to be captured in a video sequence in real time, and then features related to the face can be extracted for recognition; in the AR field, positions of markers need to be grabbed in a video sequence in real time for rendering and the like.
The above-mentioned tracking is generally directed to a specific pixel region or a planar object in the image. With the increase of computer computing power and the demand of markets, tracking technology gradually becomes excessive to three-dimensional objects. In tracking a three-dimensional object, it is necessary to specify the position of the three-dimensional object on a video screen in real time and to calculate detailed attitude information such as the size and orientation of the three-dimensional object. Especially in the AR field, if the pose information is not accurate enough, the rendered virtual object will greatly affect the user experience.
US patent application US20140369557 of prior art 1 discloses a system and method for feature-based tracking, wherein the tracking approach takes NCC prediction + planar IC iteration + feature tracking (feature tracking). Under the condition that a template exists, firstly, rough search is carried out by using NCC to determine an approximate region where a target is located, then homography (homograph) is extracted from a plane region in an image, iteration is carried out by using an Inverse combination (Inverse composite) mode, a more accurate position is further obtained, and finally, on the basis, a feature tracking and feature point matching mode is used to determine a final accurate position. The system defined in this patent includes a series of outputs for computing positions of anomalies, such as positions of Inverse composition iterations when feature tracking fails, etc.
Chinese patent application CN 106845435 of prior art 2 discloses an augmented reality technology implementation method based on a real object detection tracking algorithm, which mainly adopts the steps of presetting a three-dimensional real object template and a posture for tracking. In a specific tracking process, a region frame is divided for a tracking region, and feature points are extracted from the region. And if the number of the characteristic points meets the requirement of the threshold value, tracking the characteristic points. When the ORB feature points are extracted from the current frame, the corresponding feature points are searched and searched in the next frame, so that the pose of the three-dimensional model of the next frame is estimated. If the number of the feature points is not enough, extracting edges in the area, and estimating the attitude (pos) of the three-dimensional model by utilizing edge contour optimization.
However, in the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
for the prior art 1, a plane iteration mode is adopted, and a plane area with rich characteristic points can exist in a tracking sequence, otherwise, failure is caused. In addition, tracking of three-dimensional objects often does not conform to the mathematical assumption of a plane, and problems are also easily caused in the solving process.
For the prior art 2, in the scheme of adopting the feature points, when the perspective of the three-dimensional object is changed, the originally visible corner points or faces may become invisible, which easily causes tracking failure. When the texture of an object is switched to the edge-based tracking, due to the problems of various illumination or complex backgrounds and the like, the edge is likely to be converged to the local optimum, an incorrect pose is solved, and subsequent tracking failure is caused.
In addition, in the tracking of the three-dimensional model, because a part of the three-dimensional model lacks textures, and has highlight reflection and the like, a fixed template is difficult to generate for tracking the feature points, the posture of the target is often calculated in real time through region tracking or edge tracking, and subsequent processing is facilitated. But external points are easily introduced due to intrinsic defects of the target model and the ambient illumination and the like; in practical application, occlusion exists widely, and the traditional algorithm is easy to introduce outliers. The above phenomena are easy to cause the posture of the algorithm solution to be deviated or cause the loss.
It should be noted that the above background description is only for the sake of clarity and complete description of the technical solutions of the present invention and for the understanding of those skilled in the art. Such solutions are not considered to be known to the person skilled in the art merely because they have been set forth in the background section of the invention.
Disclosure of Invention
In view of the above problems, an object of the embodiments of the present invention is to provide a method and an apparatus for filtering out outliers in three-dimensional model tracking, which can solve the problems of highlight, complex environment, occlusion, etc. in an actual scene of three-dimensional model tracking, thereby improving the success rate and robustness of three-dimensional tracking.
In order to achieve the above object, an embodiment of the present invention provides a method for filtering out outliers in three-dimensional model tracking, including: extracting a contour of the current frame; acquiring a color model based on contour sampling points, wherein the color model is a color probability distribution model of each point on the edge of a projection contour or a global color probability distribution model; reading in the next frame image, predicting the initial position of the contour in the next frame according to the motion model, and optimizing the contour in the next frame according to the initial position; calculating an energy value of color probability distribution of the contour sampling points on the basis of the color model, and filtering out external points according to the calculated energy value; and solving the posture of the next frame and replacing the next frame with the current frame.
Specifically, in the contour-based approach, the contour of a three-dimensional object is projected onto a two-dimensional image according to the pose of the current frame, wherein the contour is solved in the algorithm assuming that the image appears at the position corresponding to the contour.
Counting the color distribution in a preset range with the contour sampling point as the center, counting the color value of each point in the contour sampling area, quantizing the color value, calculating the probability of each color in the quantized contour sampling area, and forming a color probability distribution model of each point; and recalculating the color probability distribution model of each point of the contour edge after the tracking of each frame is finished, or updating the color probability distribution model of each point of the contour edge at a preset rate. Or counting the color distribution of all sampling points of the contour within a preset range, quantizing the color values, calculating the probability of each color appearing in the contour sampling region after quantization, and forming a contour global color probability distribution model; after each frame of tracking is finished, the probability distribution of all sampling points of the contour is recalculated to form a global color probability distribution model; alternatively, the global color probability distribution model is updated at a preset rate.
Reading in the next frame image, predicting the initial position of the contour sampling point extracted from the previous frame in the next frame according to the motion model, and searching the edge in the next frame according to the initial position to optimize the edge.
In order to achieve the above object, an embodiment of the present invention further provides a method for filtering out outliers in three-dimensional model tracking, including: extracting feature points of the current frame; acquiring a color model based on the feature points; the color model is a color probability distribution model of each point of the edge of the contour or a global color probability distribution model; reading in the next frame of image, predicting the position of the feature point according to the motion model, and searching the corresponding feature point in the next frame according to the initial position; calculating the energy value of the probability distribution of the characteristic points on the basis of the color model, and filtering out the outer points according to the calculated energy value; and solving the posture of the next frame and replacing the next frame with the current frame.
Specifically, in the scheme based on the feature points, a region needing to be calculated in the image is determined according to the posture obtained by tracking the three-dimensional image, the corner points are extracted in the region, and corresponding descriptors are calculated to generate the feature points.
The color model is a color probability distribution model for each feature point or a global color probability distribution model for all feature points.
Counting the color distribution in a preset range with each characteristic point as the center, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a color probability distribution model of each characteristic point; and recalculating the color probability distribution model of each feature point after the tracking of each frame is finished, or updating the color probability distribution model of each feature point at a preset rate. Or counting the color distribution in a preset range of all the feature points, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a global color probability distribution model of the feature points; after each frame of tracking is finished, the probability distribution of all the feature points is recalculated to form a global color probability distribution model of the feature points; or, updating the global color probability distribution model of the feature points at a preset rate.
Reading in the next frame of image, and predicting the possible initial position of the feature point extracted from the previous frame in the next frame according to the motion model; searching corresponding characteristic points in the next frame according to the initial position; or extracting characteristic points near the initial region and searching corresponding characteristic points by using descriptor comparison.
The embodiment of the invention also provides a device for filtering the foreign points in the three-dimensional model tracking, and the device for filtering the foreign points in the three-dimensional model tracking realizes the method for filtering the foreign points in the three-dimensional model tracking when being executed.
The embodiment of the invention also provides a device for filtering out external points in three-dimensional model tracking, which comprises a memory and a processor, wherein: the memory is used for storing codes and documents; the processor is used for executing the codes and the documents stored in the memory to realize the external point filtering method in the three-dimensional model tracking as described above.
As can be seen from the above, through multiple experimental studies of the inventor, the method and the device for filtering out outliers in three-dimensional model tracking provided by the embodiment of the present invention calculate the energy of the current tracking point in real time by counting the tracking point or the historical color model of the tracking area, and provide a new formula for calculating the energy value E, where the smaller E is, the higher the correct probability of the sampling point is, so that according to the energy value E, and by setting an energy threshold or other adaptive threshold processing modes, the wrong contour or feature point can be simply and accurately filtered out, thereby avoiding the influence of the incorrectly tracked contour or feature point on the overall tracking effect, and further improving the stability and robustness of three-dimensional model tracking in each scene. In particular, in a contour-based approach, samples closer to the contour can be given more weight in the energy function, which will make the result more reliable.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a feature point-based three-dimensional image tracking method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a contour-based three-dimensional image tracking method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart illustrating a method for filtering outliers in three-dimensional model tracking according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of contour-based three-dimensional image tracking according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart illustrating a method for filtering outliers in three-dimensional model tracking according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an outlier filtering device in three-dimensional model tracking according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings of the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The embodiment of the invention provides a method for filtering out external points in three-dimensional model tracking, wherein the three-dimensional image tracking can be carried out based on feature points and contours.
As shown in fig. 1, one method of three-dimensional image tracking may be contour-based, generally comprising:
101. calculating the projection of the current frame three-dimensional object outline on the image;
102. reading in the next frame of image;
103. predicting the position of the contour transformation according to the motion model;
104. extracting the edge of the next frame of image, and fitting and optimizing the edge with the contour; and
105. and solving the posture of the next frame and replacing the next frame with the current frame.
As shown in fig. 2, another method of three-dimensional image tracking may be feature point-based, generally comprising:
201. extracting and storing characteristic points of the current frame;
202. reading in the next frame of image;
203. predicting the position of the feature point according to the motion model;
204. searching for feature points; and
205. and solving the posture of the next frame and replacing the next frame with the current frame.
According to the method for filtering the outliers in the three-dimensional model tracking, provided by the embodiment of the invention, on the basis of a three-dimensional image tracking scheme based on the feature points and the contour, a new module is introduced based on the color model, and the outliers can be effectively filtered, so that the problems of highlight, complex environment, shielding and the like of the three-dimensional model tracking in an actual scene are solved, and the robustness of the three-dimensional object tracking is further improved.
As shown in fig. 3, the method for filtering out outliers in the three-dimensional model tracking based on the color model specifically includes the following steps:
step 301, extracting a contour or extracting a feature point from the current frame.
In the scheme of carrying out three-dimensional image tracking based on contour extraction:
according to the pose of the current frame, the contour of the three-dimensional object is projected onto the two-dimensional image, wherein the algorithm assumes that an edge should appear at the position corresponding to the contour of the image to solve the contour, for example, the contour can be solved by integrally optimizing the position of the edge or searching for the position with high gradient as an edge sampling point through normal search, and the specific method for solving the contour is not limited herein.
In the scheme of three-dimensional image tracking based on the feature points:
the region to be calculated in the image is determined from the pose (position) tracked by the three-dimensional image. And extracting corner points in the region and calculating to generate corresponding feature descriptors. The extracted corner may be a fast (features from computed segment test) corner, a Harris corner, or a similar corner extraction algorithm, and the specific method for extracting the corner is not limited herein.
And calculating corresponding descriptors to generate feature points of the extracted corner points. The way of generating the feature descriptors can be to generate matching templates of NCC (normalized Cross correlation)/SAD (sum of Absolute Differences)/SSD (sum of Squared Differences) and the like for simple patch extraction; the descriptor may be an ORB (ordered FAST and indexed BRIEF) descriptor, or a SIFT (Scale Invariant feature transform)/SURF (speed Up Robust feature), and the descriptor is not limited herein.
And 2, acquiring a color model based on the contour sampling points or the characteristic points.
In this step, statistics are calculated or the color model based on contour sampling points or based on feature points is updated.
The color model is a color probability distribution model of each point of the edge of the contour or a global color probability distribution model, wherein the color can refer to a color model of each channel of RGB stored in an original computer conventional image format, and can also refer to a color model of converting the RGB color model into HSV/HSI and the like. RGB color model: the RGB color scheme is a color standard in the industry, and various colors are obtained by changing three color channels of red (R), green (G) and blue (B) and superimposing the three color channels on each other, where RGB represents colors of the three channels of red, green and blue, and the color standard almost includes all colors that can be perceived by human vision, and is one of the most widely used color systems at present. HSV color model: HSV (Hue, Saturation) is a color space created by a.r. smith in 1978, also known as the hexagonal cone Model (Hexcone Model), based on the intuitive nature of color. The parameters of the colors in this model are: hue (H), saturation (S), lightness (V). HSV can be interconverted with RGB.
Because there may be slight color difference between the pixels in the same position, in order to consider efficiency and robustness, the color channel value is generally quantized, for example, each channel value of original RGB is 0-255, that is, there are 65535 colors in total, and it can be quantized to 512 or 256 colors, and HSV/HIS is also the same. The quantization mode may adopt different quantization values for different color channels, for example, in HSV, the H channel may be quantized more finely in consideration of color discrimination.
The color probability distribution model may refer to the probability distribution of colors in the peripheral regions of a single feature point or an edge sampling point, or may refer to a global color model formed by gathering the peripheral regions of all points. For the manner of the feature points, due to uncertainty of distribution positions of the feature points, an independent color model is often adopted. For the edge mode, since the sampling points are gathered near the edge, the overall color model is usually counted, and the color model is divided into a foreground (inside the object) model and a background model to be stored respectively.
The probability distribution refers to the probability of such a color occurrence in the color model of the current region, where the sum of the probabilities of all color occurrences is 1. The probability distribution can be generated in real time for each frame, that is, the color probability distribution model of the feature point or the edge sampling point is recalculated after tracking of each frame is finished, or the probability distribution model can form a binding relationship with the feature point or the edge sampling point, and after the initial color probability distribution model is generated, the probability distribution is updated at a certain rate, so that the error caused by sudden change of the picture can be prevented.
Taking the RGB color model as an example, suppose that the target region has two colors, one 60% of the region is (255, 0, 0), and the remaining 40% of the region is (0, 255, 0); the target area has n pixel points in total. In the original RGB model, there are 256 total values per channel, and each channel is quantized to m values. The color channels total m3And (4) distributing the seeds. Let j be 256/m. The distribution is ((0-j-1), 0- (j-1)), ((1-2 j-1), 0- (j-1)), ("' (256-j) -255, (256-j) -255); the color distributions ((256-j) to 255, 0 to (j-1)) have values of 0.6n, 0 to (j-1), and 256-j to 255) have values of 0.4 n. Normalized to 0.6 and 0.4, respectively, with the remainder being 0. These values reflect the probability that a color within the corresponding color range appears within the target region.
In the scheme of three-dimensional image tracking based on the contour, obtaining a color model based on contour sampling points specifically comprises:
counting the color distribution in a preset range with the contour sampling point as the center, counting the color value of each point in the contour sampling area, quantizing the color value, calculating the probability of each color in the quantized contour sampling area, and forming a color probability distribution model of each point; recalculating the color probability distribution model of each point of the contour edge after the tracking of each frame is finished, or updating the color probability distribution model of each point of the contour edge at a preset rate;
alternatively, the first and second electrodes may be,
counting the color distribution of all sampling points of the contour within a preset range, quantizing the color values, calculating the probability of each color appearing in the contour sampling region after quantization, and forming a contour global color probability distribution model; after each frame of tracking is finished, the probability distribution of all sampling points of the contour is recalculated to form a global color probability distribution model; alternatively, the global color probability distribution model is updated at a preset rate.
In the scheme of performing three-dimensional image tracking based on the feature points, obtaining a color model based on the feature points specifically includes:
counting the color distribution in a preset range with each characteristic point as the center, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a color probability distribution model of each characteristic point; recalculating the color probability distribution model of each characteristic point after the tracking of each frame is finished, or updating the color probability distribution model of each characteristic point at a preset rate;
alternatively, the first and second electrodes may be,
counting the color distribution in a preset range of all the feature points, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a global color probability distribution model of the feature points; after each frame of tracking is finished, the probability distribution of all the feature points is recalculated to form a global color probability distribution model of the feature points; or, updating the global color probability distribution model of the feature points at a preset rate.
And 3, reading in the next frame of image, and predicting the contour or the position of the feature point according to the motion model.
In this step, the next frame image is read in, and the initial position of the edge sampling point or the feature point extracted from the previous frame in the next frame is predicted according to the motion model, so that the algorithm can search for the optimized contour or the feature point conveniently.
The motion model may be an acceleration of a historical gesture, or an initial position provided by a simple rough search, or an initial displacement and rotation provided by a device gyroscope or the like, and is not particularly limited herein.
And 4, optimizing the contour or searching the corresponding characteristic point in the next frame according to the initial position.
In the scheme of carrying out three-dimensional image tracking based on the contour:
the contour can be determined simply by searching the gradient direction along the normal, or an edge distance field can be generated, and the correct contour position can be obtained by optimizing the compensation function of the distance field, and the specific method for solving the contour is not limited herein.
Taking fig. 4 as an example, when tracking the three-dimensional model, the inner side frame (sampling point) is the contour of the previous frame of object projected to the current frame according to the previous frame posture, and the outer side frame (sampling point) is the contour map of the current frame obtained by the search method. The image 402 is a middle map of foreground and background segmentation in the process, the image 403 is an edge image of the current frame, and the image 404 is a distance field image in distance field optimization.
In the scheme of three-dimensional image tracking based on the feature points:
the template matching search may be a simple SAD/SSD/NNC template matching search, or feature points such as ORB/SIFT/SURF may be extracted near the initial region, and the corresponding feature points may be determined by descriptor comparison, and the specific method for determining the feature points is not limited herein.
And 5, calculating the energy value of the color probability distribution of the contour sampling points or the characteristic points on the basis of the color model, and filtering out the outer points according to the calculated energy value.
In the scheme of carrying out three-dimensional image tracking based on the contour:
due to the influence of a complex background, or highlights, etc., false contours may be generated near the real contours. False contours also have gradients that can be misjudged by algorithms for direct search classes. These false contours can create disturbances on the distance field, allowing the algorithm to converge on the false contours, and therefore also require filtering of samples near the false contours.
Taking fig. 5 as an example, the pose deviation caused by "spoofing" by a false edge is shown, wherein false contours are generated near the real contours in the 501 image due to a complex background, for example, optimization errors are caused by false contours which are lines inside a wire frame in the 503 image, and therefore, the false contours need to be filtered out.
In the definition of the profile-based energy function, the energy function E is defined as follows:
E＝-log(P(Φ|Ω))
wherein, Ic(x) Representing the pixel value at the x coordinate of the current frame;
omega is a sampling area of the contour sampling point;
Pf(Ic(x) A posterior probability of the current color in a color model of the global foreground;
Pb(Ic(x) Represents the posterior probability of the current color in the color model of the global background;
Φ (x) is a directed distance field that returns a positive distance when x is in the foreground and a negative distance when x is in the background.
He(x) The smoothing heaviside function gives a larger weight to points in the sampling region that are closer to the sampling point, and other linear smoothing functions may be adopted without specific limitation.
The energy function E in equation (1) is the probability that the currently found edge point is located at the true edge (and object contour) under the constraint of the statistical global color model. The low-probability points are screened out by setting the threshold value, so that the rest searched sampling points or optimization points are positioned on the real edge instead of the false edge caused by disordered background or highlight and the like, and the robustness of the subsequent solving posture is improved.
In addition, as the distribution of the sampling points of the contour is similar to that of the sampling points of the contour, global statistics can be adopted to screen the interior points and the exterior points. In addition, samples closer to the profile can be given more weight in the energy function, which will make the result more reliable.
In the scheme of three-dimensional image tracking based on the feature points:
the error matching may cause deviation of the final solved pose, for example, due to disappearance of the feature points in view transformation, or due to occlusion, highlight, etc., which may cause incorrect matching of the feature points.
In the feature point-based energy function definition, an energy function E is defined as follows:
wherein the content of the first and second substances,a neighborhood representing the feature point;
x is a pixel coordinate;
Ic(x) Representing the pixel value at the x coordinate under the current frame.
The energy function E in the formula (2) intuitively measures that the color sampling near the characteristic point is consistent with the historical color distribution requirement, and can effectively filter out external points under the condition that a threshold value is set in advance.
In the above-described scheme of three-dimensional image tracking based on a contour or based on a feature point, the larger P (Φ | Ω) is, the higher the probability that the point is a true contour point or a feature point is. Generally, for the convenience of subsequent optimization and calculation, E is selected as the final energy expression, i.e., the smaller E, the higher the probability of correct sampling point. Generally, if there is a case of occlusion or highlight, the local color model will not match the global color model, and a larger E will be obtained, and if the color model converges to the wrong edge, the color model will not match the history information. The global color model or the local historical color model and the like can select different measures according to actual application scenes. Then, a simple threshold filtering may be set according to the calculated E, and a point where E is greater than a certain threshold may be considered as an outlier for filtering, or may be processed in a self-adaptive threshold processing manner, such as adding a fixed value to the mean value, or fixedly deleting a point with the maximum energy of 10%, and the like, which is not limited herein.
And 6, solving the next frame attitude, and replacing the next frame with the current frame.
In this step, after the outliers are filtered, it is ensured that the remaining points are all correctly searched feature points or edge sampling points, the posture of the three-dimensional object under the current frame is solved by adopting PNP or gauss-newton iteration and other modes, and the tracking of the next frame is performed. And solving the posture of the next frame by adopting the same calculation method, replacing the next frame with the current frame, and circularly processing.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
The embodiment of the invention also provides a device for filtering the foreign points in the three-dimensional model tracking, and the device for filtering the foreign points in the three-dimensional model tracking realizes the method for filtering the foreign points in the three-dimensional model tracking when being executed.
As shown in fig. 6, an embodiment of the present invention further provides a device for filtering out outliers in three-dimensional model tracking, including a memory and a processor, wherein:
a memory 601 for storing codes and documents;
a processor 602 for executing the code and documents stored in the memory to implement the method for filtering out outliers in three-dimensional model tracking as described above.
The specific technical details of the device for filtering out the external points in the three-dimensional model tracking are similar to those of the method for filtering out the external points in the three-dimensional model tracking, and thus detailed descriptions are omitted.
Those skilled in the art will understand that all or part of the steps in the method according to the above embodiments may be implemented by a program instructing related hardware to complete, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, etc.) or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Therefore, the method and the device for filtering the external points in the three-dimensional model tracking provided by the invention can be used for calculating the energy of the current tracking point in real time by counting the tracking point or the historical color model of the tracking area, filtering the wrong edge or tracking point by using the energy threshold value, avoiding the influence of the wrong tracking edge or tracking point on the overall tracking effect, and improving the stability and the robustness of the three-dimensional model tracking in each scene.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments.
Finally, it should be noted that: the foregoing description of various embodiments of the invention is provided to those skilled in the art for the purpose of illustration. It is not intended to be exhaustive or to limit the invention to a single disclosed embodiment. Various alternatives and modifications of the invention, as described above, will be apparent to those skilled in the art. Thus, while some alternative embodiments have been discussed in detail, other embodiments will be apparent or relatively easy to derive by those of ordinary skill in the art. The present invention is intended to embrace all such alternatives, modifications, and variances which have been discussed herein, and other embodiments which fall within the spirit and scope of the above application.
Claims (12)
1. A method for filtering out foreign points in three-dimensional model tracking is characterized by comprising the following steps:
extracting a contour from the current frame image;
acquiring a color model based on a contour sampling point, wherein the color model is a color probability distribution model of each point on the edge of a contour or a global color probability distribution model;
reading in the next frame image, predicting the initial position of the contour in the next frame according to the motion model, and optimizing the contour in the next frame according to the initial position;
calculating an energy value of color probability distribution of the contour sampling points on the basis of the color model, and filtering out external points according to the calculated energy value;
and solving the posture of the next frame and replacing the next frame with the current frame.
2. The method for filtering out outliers in three-dimensional model tracking according to claim 1, wherein the extracting contours from the current frame image specifically comprises:
the contour of the three-dimensional object is projected onto the two-dimensional image according to the pose of the current frame, wherein the contour is solved by assuming that the image presents the contour at the position corresponding to the contour in the algorithm.
3. The method for filtering out outliers in three-dimensional model tracking according to claim 1, wherein the obtaining of the color model based on the contour sampling points specifically comprises:
counting the color distribution in a preset range with the contour sampling point as the center, counting the color value of each point in the contour sampling area, quantizing the color value, calculating the probability of each color in the quantized contour sampling area, and forming a color probability distribution model of each point; recalculating the color probability distribution model of each point of the contour edge after the tracking of each frame is finished, or updating the color probability distribution model of each point of the contour edge at a preset rate;
alternatively, the first and second electrodes may be,
counting the color distribution of all sampling points of the contour within a preset range, quantizing the color values, calculating the probability of each color appearing in the contour sampling region after quantization, and forming a contour global color probability distribution model; after each frame of tracking is finished, the probability distribution of all sampling points of the contour is recalculated to form a global color probability distribution model; alternatively, the global color probability distribution model is updated at a preset rate.
4. The method for filtering out outliers in three-dimensional model tracking according to claim 1, wherein reading in an image of a next frame, predicting an initial position of a contour in the next frame according to the motion model, and optimizing the contour in the next frame according to the initial position specifically comprises:
reading in the next frame image, predicting the initial position of the contour sampling point extracted from the previous frame in the next frame according to the motion model, and searching the edge in the next frame according to the initial position to optimize the contour.
5. The method for filtering out outliers in three-dimensional model tracking according to claim 1, wherein the energy value based on the probability distribution of the contour sampling points is calculated by an energy function E, and the calculation formula of the energy function E is as follows:
E＝-log(P(Φ|Ω))
wherein, Ic(x) Representing the pixel value at the x coordinate of the current frame;
omega is a sampling area of the contour sampling point;
Pf(Ic(x) A posterior probability of the current color in a color model of the global foreground;
Pb(Ic(x) Represents the posterior probability of the current color in the color model of the global background;
Φ (x) is a directed distance field that returns a positive distance when x is in the foreground and a negative distance when x is in the background;
he (x) gives greater weight to points within the sampling region that are closer to the sampling point.
6. A method for filtering out foreign points in three-dimensional model tracking is characterized by comprising the following steps:
extracting feature points of the current frame;
acquiring a color model based on the feature points, wherein the color model is a color probability distribution model of each feature point or a global color probability distribution model of all the feature points;
reading in the next frame of image, predicting the position of the feature point according to the motion model, and searching the corresponding feature point in the next frame according to the initial position;
calculating the energy value of the probability distribution of the characteristic points on the basis of the color model, and filtering out the outer points according to the calculated energy value;
and solving the posture of the next frame and replacing the next frame with the current frame.
7. The method for filtering outliers in three-dimensional model tracking according to claim 6, wherein the extracting feature points of the current frame specifically comprises:
determining a region needing to be calculated in the image according to the posture obtained by tracking the three-dimensional image, extracting angular points in the region and calculating corresponding descriptors to generate feature points.
8. The method for filtering out outliers in three-dimensional model tracking according to claim 6, wherein the obtaining of the color model based on the feature points specifically includes:
counting the color distribution in a preset range with each characteristic point as the center, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a color probability distribution model of each characteristic point; recalculating the color probability distribution model of each characteristic point after the tracking of each frame is finished, or updating the color probability distribution model of each characteristic point at a preset rate;
alternatively, the first and second electrodes may be,
counting the color distribution in a preset range of all the feature points, quantizing the color values, calculating the occurrence probability of each color after quantization, and forming a global color probability distribution model of the feature points; after each frame of tracking is finished, the probability distribution of all the feature points is recalculated to form a global color probability distribution model of the feature points; or, updating the global color probability distribution model of the feature points at a preset rate.
9. The method for filtering out outliers in three-dimensional model tracking according to claim 6, wherein reading in the next frame image, predicting the positions of feature points according to the motion model, and searching for corresponding feature points in the next frame according to the initial position specifically comprises:
reading in the next frame of image, and predicting the possible initial position of the feature point extracted from the previous frame in the next frame according to the motion model;
searching corresponding characteristic points in the next frame according to the initial position; or extracting characteristic points near the initial region and searching corresponding characteristic points by using descriptor comparison.
10. The method for filtering out outliers in three-dimensional model tracking according to claim 6, wherein the energy value based on the probability distribution of the feature points is calculated by an energy function E, and the calculation formula of the energy function E is as follows:
wherein the content of the first and second substances,representing the vicinity of the feature point;
x is a pixel coordinate;
Ic(x) Representing the pixel value at the x coordinate under the current frame.
11. A device for filtering out outliers in three-dimensional model tracking, wherein the device for filtering out outliers in three-dimensional model tracking is implemented to realize the method steps of any one of claims 1 to 10.
12. An apparatus for filtering outliers in three-dimensional model tracking, the apparatus comprising a memory and a processor, wherein:
the memory is used for storing codes and documents;
the processor for executing the code and documents stored in the memory for implementing the method steps of any of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910052458.4A CN109785367B (en) | 2019-01-21 | 2019-01-21 | Method and device for filtering foreign points in three-dimensional model tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910052458.4A CN109785367B (en) | 2019-01-21 | 2019-01-21 | Method and device for filtering foreign points in three-dimensional model tracking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109785367A CN109785367A (en) | 2019-05-21 |
| CN109785367B true CN109785367B (en) | 2019-12-20 |
Family
ID=66500949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910052458.4A Active CN109785367B (en) | 2019-01-21 | 2019-01-21 | Method and device for filtering foreign points in three-dimensional model tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109785367B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112614161A (en) * | 2020-12-28 | 2021-04-06 | 之江实验室 | Three-dimensional object tracking method based on edge confidence |
| CN116091588A (en) * | 2021-11-02 | 2023-05-09 | 中兴通讯股份有限公司 | Three-dimensional object detection method, apparatus, and computer-readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345881A (en) * | 2018-02-01 | 2018-07-31 | 福州大学 | A kind of document quality detection method based on computer vision |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6192156B1 (en) * | 1998-04-03 | 2001-02-20 | Synapix, Inc. | Feature tracking using a dense feature array |
| CN1873656A (en) * | 2005-06-03 | 2006-12-06 | 中国科学院自动化研究所 | Detection method of natural target in robot vision navigation |
| US8761439B1 (en) * | 2011-08-24 | 2014-06-24 | Sri International | Method and apparatus for generating three-dimensional pose using monocular visual sensor and inertial measurement unit |
| KR101926563B1 (en) * | 2012-01-18 | 2018-12-07 | 삼성전자주식회사 | Method and apparatus for camera tracking |
| CN103606186B (en) * | 2013-02-02 | 2016-03-30 | 浙江大学 | The virtual hair style modeling method of a kind of image and video |
| CN106096577B (en) * | 2016-06-24 | 2019-05-31 | 安徽工业大学 | A kind of target tracking method in camera distribution map |
| CN106845435A (en) * | 2017-02-10 | 2017-06-13 | 深圳前海大造科技有限公司 | A kind of augmented reality Implementation Technology based on material object detection tracing algorithm |
-
2019
- 2019-01-21 CN CN201910052458.4A patent/CN109785367B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345881A (en) * | 2018-02-01 | 2018-07-31 | 福州大学 | A kind of document quality detection method based on computer vision |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109785367A (en) | 2019-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9830701B2 (en) | Static object reconstruction method and system | |
| Feng et al. | Local background enclosure for RGB-D salient object detection | |
| US10373380B2 (en) | 3-dimensional scene analysis for augmented reality operations | |
| US9349194B2 (en) | Method for superpixel life cycle management | |
| CN107633526B (en) | Image tracking point acquisition method and device and storage medium | |
| US8411932B2 (en) | Example-based two-dimensional to three-dimensional image conversion method, computer readable medium therefor, and system | |
| US8494297B2 (en) | Automatic detection and mapping of symmetries in an image | |
| US9367762B2 (en) | Image processing device and method, and computer readable medium | |
| US20180174301A1 (en) | Iterative method for salient foreground detection and multi-object segmentation | |
| JP2002342756A (en) | Method for detecting position of eye and mouth in digital image | |
| CN109785367B (en) | Method and device for filtering foreign points in three-dimensional model tracking | |
| CN113723399A (en) | License plate image correction method, license plate image correction device and storage medium | |
| Kumar et al. | Automatic image segmentation using wavelets | |
| CN113436251A (en) | Pose estimation system and method based on improved YOLO6D algorithm | |
| CN111161219B (en) | Robust monocular vision SLAM method suitable for shadow environment | |
| CN111179281A (en) | Human body image extraction method and human body action video extraction method | |
| CN114820987A (en) | Three-dimensional reconstruction method and system based on multi-view image sequence | |
| CN113840135A (en) | Color cast detection method, device, equipment and storage medium | |
| CN113343987A (en) | Text detection processing method and device, electronic equipment and storage medium | |
| CN113033256A (en) | Training method and device for fingertip detection model | |
| CN110599517A (en) | Target feature description method based on local feature and global HSV feature combination | |
| Fu et al. | Adaptive Multi-Level Region Merging for Salient Object Detection. | |
| CN113570667B (en) | Visual inertial navigation compensation method and device and storage medium | |
| Lv et al. | Saliency detection via fusing color contrast and hash fingerprint | |
| EP2765556A1 (en) | Method for superpixel life cycle management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |