WO2022252487A1 - 位姿获取方法及装置、电子设备、存储介质和程序 - Google Patents

位姿获取方法及装置、电子设备、存储介质和程序 Download PDF

Info

Publication number
WO2022252487A1
WO2022252487A1 PCT/CN2021/127307 CN2021127307W WO2022252487A1 WO 2022252487 A1 WO2022252487 A1 WO 2022252487A1 CN 2021127307 W CN2021127307 W CN 2021127307W WO 2022252487 A1 WO2022252487 A1 WO 2022252487A1
Authority
WO
WIPO (PCT)
Prior art keywords
point
weight
sampling
pose
value
Prior art date
Application number
PCT/CN2021/127307
Other languages
English (en)
French (fr)
Inventor
黄鸿
钟凡
秦学英
宋修强
Original Assignee
浙江商汤科技开发有限公司
山东大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江商汤科技开发有限公司, 山东大学 filed Critical 浙江商汤科技开发有限公司
Priority to JP2023525620A priority Critical patent/JP2023549069A/ja
Priority to KR1020237014463A priority patent/KR20230073331A/ko
Publication of WO2022252487A1 publication Critical patent/WO2022252487A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Definitions

  • the present application relates to the technical field of computer vision, and relates to but not limited to a pose acquisition method and device, electronic equipment, storage media and programs.
  • pose parameters have been more and more widely used in many scenarios such as augmented reality systems, robot hand-eye calibration, interactive games, and human-computer interaction.
  • augmented reality system virtual objects can be rendered and superimposed on real objects in a video image according to pose parameters, so as to achieve a virtual-real fusion effect with spatial and geometric consistency.
  • Embodiments of the present application provide a pose acquisition method and device, electronic equipment, a storage medium, and a program.
  • the first aspect of the embodiment of the present application provides a pose acquisition method, which is applied to an electronic device, including:
  • the attribute information can reflect the possibility that the sampling point belongs to the target object, and the reference weight can reflect the reference of the sampling point in the subsequent process of solving the pose parameters. Therefore, when constructing the objective function based on the attribute information of sampling points and reference weights, the influence of interference factors on the pose solution can be reduced, which is beneficial to improve the accuracy of pose parameters.
  • obtaining the reference weight of the sampling point includes: searching for the target point among several sampling points on the search line segment to obtain the search result; wherein, the target point is used to represent the object contour point of the target object; based on the search result , obtain the weight information of several sampling points on the search line segment respectively; wherein, the weight information includes at least one of the first weight and the second weight, the first weight is related to the predicted probability value of the target point, and the predicted probability value represents the sampling point as The possibility of the contour point of the object, and the second weight is related to the first distance from the target point to the sampling point; based on the weight information, the reference weight of the sampling point is obtained.
  • the target point is used to represent the object contour point of the target object, and based on the search result, respectively obtain the weight information of several sampling points on the search line segment
  • the weight information includes at least one of the first weight and the second weight, the first weight is related to the predicted probability value of the target point, the predicted probability value represents the possibility of the sampling point as the object contour point, and the second weight is related to the target point to
  • the first distance of the sampling point is related, so the first weight and the second weight can represent the reference value of the sampling point from different angles, and then based on this and then based on the weight information, the reference weight of the sampling point can be obtained, which can improve the reference weight in the subsequent solution position The reference value in the attitude parameter process.
  • the attribute information includes: the first probability value that the sampling point belongs to the target object; searching for the target point among several sampling points on the search line segment to obtain the search result, including: for each search line segment, several sampling points When the reference probability difference of the current point satisfies the first condition, the current point is used as the candidate point, and the candidate point whose predicted cost value satisfies the second condition is selected as the target point.
  • the reference probability difference of the current point is the difference between the first probability values of two sampling points having a preset position relationship with the current point
  • the predicted cost value includes at least one of the first generation value and the second generation value
  • the second The first-generation value is related to the predicted probability value of the candidate point
  • the second-generation value is related to the second distance from the candidate point to the projected contour point on the search line segment.
  • the attribute information includes the first probability value that the sampling point belongs to the target object, and for each search line segment, several sampling points are respectively used as the current point, and when the reference probability difference of the current point satisfies the first condition, the current point as a candidate point, and select a candidate point whose predicted cost value satisfies the second condition as the target point, and the reference probability difference of the current point is the difference between the first probability values of two sampling points that have a preset position relationship with the current point , the prediction cost value includes at least one of the first generation value and the second generation value, the first generation value is related to the predicted probability value of the candidate point, the second generation value is related to the second generation of the candidate point to the projected contour point on the search line segment Distance correlation, that is, the first-generation value and the second-generation value respectively represent the cost of the candidate point as an object contour point from different angles, so the candidate point is first roughly selected by the reference probability difference, and then the target point is obtained by fine selection based on the predicted cost value , which can help improve
  • the method before selecting candidate points whose predicted cost values satisfy the second condition as target points, the method further includes: filtering candidate points whose predicted probability values satisfy the third condition.
  • the preset position relationship is adjacent to the current point; and/or, the second condition includes the minimum predicted cost value; and/or, the first generation value is negatively correlated with the predicted probability value of the candidate point, and The second generation value is positively related to the second distance.
  • the preset position relationship to be adjacent to the current point, it can help to accurately evaluate the sudden change of the first probability value of each sampling point, which is conducive to improving the accuracy of the candidate point; and by setting the second condition as Including the minimum prediction cost value, it can help to further alleviate the influence of interference factors on the selection of target points as much as possible, and improve the accuracy of pose parameters; and by setting the first generation value to be negatively correlated with the predicted probability value of the candidate point, and The second-generation value is positively correlated with the second distance, which can help improve the accuracy of the first-generation value and the second-generation value.
  • the weight information includes the first weight; based on the search results, respectively obtaining the weight information of several sampling points on the search line segment, including: when the search results include the searched target point, based on the prediction of the target point The probability value determines the first weight of the sampling point, wherein the first weight is positively correlated with the predicted probability value of the target point; and/or, when the search result includes no target point, the first weight is determined as the first Numerical values; wherein, the first numerical value is the lower limit value of the first weight when the search result includes the searched target point.
  • the weight information includes the first weight
  • the first weight of the sampling point is determined based on the predicted probability value of the target point, and the first weight is positively correlated with the predicted probability value of the target point
  • the first weight is determined as the first numerical value
  • the first numerical value is the lower limit value of the first weight when the search result includes the searched target point, which can be searched
  • the line segment as a whole is a dimension, and determining the first weight of each sampling point on the search line segment is conducive to improving the efficiency of obtaining the first weight.
  • the weight information includes a second weight; based on the search results, obtaining the weight information of several sampling points on the search line segment respectively includes: when the search results include the searched target point, based on the corresponding The first distance determines the second weight of the sampling point, wherein the second weight is negatively correlated with the first distance; and/or, when the search result includes no target point, the second weight is determined as the second value; Wherein, the second value is the upper limit value of the second weight when the search result includes the searched target point.
  • the weight information includes a second weight
  • the second weight of the sampling point is determined based on the first distance corresponding to the sampling point, and the second weight is negatively correlated with the first distance.
  • the search result includes that the target point is not found
  • the second weight is determined as the second value
  • the second value is the upper limit value of the second weight when the target point is found
  • the whole search line segment can be used as the dimension , to determine the second weight of each sampling point on the search line segment, which is beneficial to improve the efficiency of obtaining the second weight.
  • the weight information includes a first weight and a second weight, and both the first weight and the second weight are positively correlated with the reference weight.
  • the weight information is set to include both the first weight and the second weight, and both the first weight and the second weight are positively correlated with the reference weight, so the sampling point can be characterized from two different dimensions of the first weight and the second weight
  • the reference value in the subsequent process of solving the pose parameters is conducive to improving the reference value of the reference weight itself.
  • the attribute information includes: a first probability value and a first credibility that the sampling point belongs to the target object, and a second probability value and a second credibility that the sampling point does not belong to the target object; based on the sampling point
  • the attribute information and reference weights of , constructing the objective function including: obtaining the first product of the first credibility and the first probability value and the second product of the second credibility and the second probability value, and based on the first product sum
  • the sum of the second products is used to obtain the joint probability value of the sampling point; the weighted result of the joint probability value based on the reference weight of each sampling point is obtained to obtain the objective function.
  • the attribute information includes the first probability value and the first reliability that the sampling point belongs to the target object, and the second probability value and the second reliability that the sampling point does not belong to the target object.
  • the first possible The first product of the reliability and the first probability value and the second product of the second reliability and the second probability value, and based on the sum of the first product and the second product, the joint probability value of the sampling point is obtained, so that it can be obtained from
  • the joint probability value of the sampling point is represented by the two angles of the sampling point belonging to the target object and the sampling point not belonging to the target object, and the objective function is constructed by weighting the joint probability value with the reference weight of each sampling point, which can improve the target
  • the accuracy of the function is beneficial to improve the accuracy of the reference pose.
  • the first credibility and the second credibility are negatively correlated, and the first credibility of the sampling point is negatively correlated with the directed Euclidean distance from the corresponding projected contour point to the sampling point,
  • the corresponding projected contour points are located on the same search line segment as the sampling points.
  • the first credibility and the second credibility are negatively correlated, and the first credibility of the sampling point is negatively correlated with the directed Euclidean distance from the corresponding projected contour point to the sampling point, and the corresponding projected contour point Located on the same search line segment as the sampling point, that is, the smaller the directed Euclidean distance, the higher the first confidence that the sampling point belongs to the target object, and the lower the second confidence that the sampling point does not belong to the target object. It is beneficial to alleviate the influence of interference factors such as partial occlusion as much as possible.
  • the captured image includes the foreground area and the background area divided based on the projection contour; after obtaining the first product of the first credibility and the first probability value and the second product of the second credibility and the second probability value Before the second product, the method further includes: when the directed Euclidean distance of the sampling point is greater than the first distance value, and the sampling point belongs to the foreground area, filtering the sampling point; and/or, the directed Euclidean distance of the sampling point If it is smaller than the second distance value and the sampling point belongs to the background area, filter the sampling point.
  • the captured image includes the foreground area and the background area based on the projection contour.
  • the projection profile is obtained by using the reference pose projection of the target object; before obtaining several sampling points on the search line segment in the captured image, the method includes: down-sampling the captured image to obtain several resolutions Pyramid image with high rate; according to the resolution from small to large, select the pyramid image as the current captured image in turn, and perform the steps of obtaining several sampling points on the search line segment in the captured image and subsequent steps for the current captured image; wherein, this time
  • the reference pose used in the execution is the pose parameter obtained in the last execution, and the pose parameter obtained in the last execution is used as the final pose parameter of the target object in the captured image.
  • the projection profile is obtained by using the reference pose projection of the target object, so that before the projection sampling, the captured image is first down-sampled to obtain several resolutions of the pyramid image, and the resolutions are from small to large, in order Select the pyramid image as the current captured image, and perform the above-mentioned steps of obtaining several sampling points on the search line segment in the captured image and subsequent steps on the current captured image, and the reference pose used in this execution is the position obtained in the previous execution Pose parameters, the pose parameters obtained by the last execution are used as the final pose parameters of the target object in the captured image, so that the pose parameters can be estimated from coarse to fine in the process of obtaining the pose parameters, which in turn can help improve the pose parameters. Acquisition efficiency and accuracy of attitude parameters.
  • the projection profile is obtained by using the reference pose projection of the target object; based on the objective function, the pose parameters of the target object in the captured image are obtained, including: solving the objective function to obtain an update of the reference pose Parameters; use the update parameters to optimize the reference pose to obtain the pose parameters.
  • the projection profile is obtained by using the reference pose projection of the target object, which is the pose parameter of the target object in the reference image, and the reference image is obtained before the image is captured, and the objective function is solved to obtain
  • the update parameters of the reference pose, and the optimization of the reference pose by using the update parameters to obtain the pose parameters are beneficial to accurately and continuously tracking the pose parameters during the shooting process of the target object.
  • the second aspect of the embodiment of the present application provides a pose acquisition device, which is applied to electronic equipment, including:
  • the projection sampling module, the information acquisition module, the function building module and the pose solving module is configured to obtain a number of sampling points located on the search line segment in the captured image; wherein, the search line segment passes through the projected contour point of the target object in the captured image, The projected contour point is located on the projected contour of the target object; the information extraction module is configured to obtain the attribute information of the sampling point, and obtain the reference weight of the sampling point; wherein, the attribute information represents the possibility that the sampling point belongs to the target object; the function building module is configured to be based on The attribute information and reference weights of the sampling points are used to construct the objective function; the pose solving module is configured to obtain the pose parameters of the target object in the captured image based on the objective function.
  • the third aspect of the embodiment of the present application provides an electronic device, including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory, so as to implement the pose acquisition method in the first aspect above.
  • a fourth aspect of the embodiments of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the pose acquisition method in the first aspect above is implemented.
  • the fifth aspect of the embodiment of the present application provides a computer program, including computer readable code, when the computer readable code is run in the electronic device, the processor in the electronic device executes any one of the above Pose acquisition method.
  • the above solution obtains several sampling points located on the search line segment in the captured image, and the search line segment passes through the projected contour point of the target object in the captured image, and the projected contour point is located on the projected contour point of the target object, and then obtains the attribute information of the sample point based on this, And obtain the reference weight of the sampling point, and the attribute information indicates the possibility that the sampling point belongs to the target object, thus, based on the attribute information of the sampling point and the reference weight, construct an objective function, and based on the objective function, obtain the position of the target object in the captured image Attitude parameters.
  • the attribute information can reflect the possibility that the sampling point belongs to the target object, and the reference weight can reflect the reference of the sampling point in the subsequent process of solving the pose parameters. Therefore, when constructing the objective function based on the attribute information of sampling points and reference weights, the influence of interference factors on the pose solution can be reduced, which is beneficial to improve the accuracy of pose parameters.
  • Fig. 1 is a schematic flow chart of an embodiment of the pose acquisition method of the present application
  • Figure 2 is a schematic diagram of an embodiment of a contour mask
  • Fig. 3 is a schematic diagram of an embodiment of a projection profile and a search line segment
  • Fig. 4a is a schematic diagram of an embodiment of capturing an image
  • Fig. 4b is a schematic diagram of another embodiment of a mask image
  • Fig. 4c is a schematic diagram of an embodiment of a search line segment
  • Fig. 5a is a schematic diagram of another embodiment of capturing an image
  • Fig. 5b is a schematic diagram of an embodiment of a layout area
  • Fig. 5c is a schematic diagram of another embodiment of a partial area
  • Fig. 6 is a schematic flow chart of an embodiment of step S12 in Fig. 1;
  • Figure 7a is a bundled image of the search line segment in Figure 3;
  • Fig. 7 b is the cluster image of the first probability value of each sampling point on the search line segment in Fig. 3;
  • Fig. 8 is a schematic flow chart of another embodiment of the pose acquisition method of the present application.
  • Fig. 9 is a schematic frame diagram of an embodiment of a pose acquisition device of the present application.
  • FIG. 10 is a schematic frame diagram of an embodiment of the electronic device of the present application.
  • Fig. 11 is a schematic diagram of an embodiment of a computer-readable storage medium of the present application.
  • system and “network” are often used interchangeably herein.
  • the term “and/or” in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations.
  • the character "/” in this article generally indicates that the contextual objects are an “or” relationship.
  • “many” herein means two or more than two.
  • the pose acquisition method provided in the embodiment of the present application can be applied to an augmented reality system.
  • the augmented reality system it is necessary to accurately estimate the pose of the real object relative to the camera, so that the virtual object can be accurately rendered and superimposed on the real object in the video according to the pose parameters, so as to achieve a virtual-real fusion effect with spatial and geometric consistency. .
  • the pose acquisition method improves the pose estimation accuracy of the object by detecting and processing pixels that are partially occluded and interfered by similar colors, and realizes accurate virtual-real registration and geometric consistency. Virtual reality fusion effect.
  • FIG. 1 is a schematic flowchart of an embodiment of a pose acquisition method in the present application. Specifically, the following steps may be included:
  • Step S11 Obtain several sampling points located on the search line segment in the captured image.
  • the search line segment passes through the projected contour point of the target object in the captured image, and the projected contour point is located on the projected contour of the target object.
  • the projected contour is obtained by projecting a reference pose of the target object, and the reference pose is a pose parameter of the target object in a reference image, and the reference image is captured before the image is captured.
  • video data can be taken for the target object, and the video data can contain multiple frames of images, then for the t-1th frame image, the steps in the embodiments of the present disclosure can be sampled to obtain the target object at the t-th
  • T t-1 can be used as the reference pose
  • the target can be obtained by using the steps in the embodiment of the present disclosure
  • the pose parameter T t of the object in the t-th frame of image, and so on, will not be exemplified here one by one.
  • a three-dimensional modeling of the target object may be performed in advance to obtain a three-dimensional model of the target object.
  • a three-dimensional model may include several vertices and edges connecting the vertices.
  • the reference pose can be recorded as T, which can be expressed as a 4*4 homogeneous matrix:
  • a contour mask in order to facilitate the subsequent determination of the relevant attributes of each pixel, can be obtained based on the projection result of the target object, and each pixel in the contour mask corresponds to a pixel at the same position in the captured image point.
  • FIG. 2 is a schematic diagram of an embodiment of the contour mask.
  • the projection profile can be obtained, and the captured image is divided into the foreground area (ie, the foreground area ⁇ f in Figure 2 ) and the background area (ie, the background area ⁇ b in Figure 2 ) based on the projection profile.
  • a search line segment l i passing through the projected contour point m i on the projected contour may be constructed.
  • the search line segment may be constructed along the normal vector n i of the projected contour at the projected contour point mi .
  • several sampling points can be extracted on the search line segment.
  • the projected contour point m i on the search line segment l i and N (such as, 7, 8, 9) pixel points located on both sides of the projected contour point m i can be extracted as the points on the search line segment l i
  • sampling points that is, solid circles in Figure 2.
  • FIG. 2 is only a projection profile that may exist in the actual application process, and does not limit the specific stroke of the projection profile. Other situations can be deduced by analogy, and no examples will be given here.
  • FIG. 3 is a schematic diagram of an embodiment of projecting a contour and searching for a line segment.
  • multiple search line segments may be constructed based on each projected contour point on the projected contour.
  • Other captured images can be deduced by analogy, and no more examples will be given here.
  • Step S12 Obtain the attribute information of the sampling point, and obtain the reference weight of the sampling point.
  • the attribute information indicates the possibility that the sampling point belongs to the target object.
  • the attribute information may include a first probability value and a first confidence level that the sampling point belongs to the target object, and a second probability value and second confidence level that the sampling point does not belong to the target object.
  • the above-mentioned first degree of credibility may indicate the degree of credibility of the first probability value
  • the above-mentioned second degree of credibility may indicate the degree of credibility of the second probability value.
  • the sampling point belongs to the target object, it can be considered that the sampling point belongs to the actual foreground area in the captured image; otherwise, if the sampling point does not belong to the target object, it can be considered that the sampling point belongs to the actual background area in the captured image.
  • the jth sampling point on the search line segment l i can be recorded as x ij
  • the first probability value of the sampling point x ij can be recorded as P f (x ij )
  • the sampling point x ij The second probability value can be denoted as P b (x ij ).
  • first probability value and the second probability value can be determined through the time-continuous local color histogram, and the specific acquisition process of the first probability value and the second probability value can refer to the specific technique of the time-continuous local color histogram The details will not be repeated here.
  • the first credibility and the second credibility are negatively correlated, that is, the higher the first credibility, the lower the second credibility, and vice versa, the lower the first credibility, the lower the second credibility. Two, the higher the reliability.
  • the first reliability of the sampling point is negatively correlated with the directed Euclidean distance from the corresponding projected contour point to the sampling point, and the corresponding projected contour point and the sampling point are located on the same search line segment. Please refer to Fig.
  • the directed Euclidean distance from the sampling point to the corresponding projected contour point can be obtained.
  • its directional Euclidean distance d(x ij ) can be expressed as:
  • m i represents the projected contour point on the search line segment l i , Indicates the transpose of the normal vector of the projected contour at the projected contour point mi .
  • a smooth and differentiable step function (eg, Heaviside function) may be used to process the directed Euclidean distance to obtain the first reliability.
  • Heaviside function e.g, Heaviside function
  • s represents the smoothing factor. The larger s is, the more severe the change of the first reliability He(d(x ij )) will be with the directed Euclidean distance d(x ij ); otherwise, the more s Smaller, the first reliability He(d(x ij )) changes more smoothly with the directional Euclidean distance d(x ij ).
  • the sum of the first reliability and the second reliability may be 1. Still taking the jth sampling point x ij on the search line segment l i as an example, after obtaining the first reliability He(d(x ij )) of the sampling point x ij , 1-He(d(x ij ) ) as the second reliability of the sampling point x ij .
  • the target point may be searched for in several sampling points on the search line segment to obtain a search result, and the target point is used to represent the object contour point of the target object.
  • the weight information of several sampling points on the search line segment can be respectively obtained based on the search results, and the weight information includes at least one of the first weight and the second weight, and the first weight is related to the predicted probability value of the target point , the predicted probability value represents the possibility of the sampling point as an object contour point, and the second weight is related to the first distance from the target point to the sampling point, so that the reference weight of the sampling point can be obtained based on the weight information.
  • the object contour point is the actual contour point of the target object in the captured image, as shown in Figure 2, the sampling point si on the search line segment l i is also located in the object contour (that is, the object contour in Figure 2), then the sampling point s i is also the actual contour point of the target object in the captured image.
  • the weight information may include a first weight
  • the first weight of the sampling point may be determined based on the predicted probability value of the target point, and the first weight and The predicted probability value of the target point is positively correlated, and in the case that the search result includes no target point, the first weight can be determined as the first value, and the first value is when the search result includes the searched target point The lower limit value of the first weight.
  • C) represents the predicted probability value of the target point s i , that is, the possibility that the target point s i is the object contour point, and the prediction of the target point s i
  • k 1 represents a negative constant, which is used to control the decay speed of the first weight with the predicted probability value, which can be set according to application requirements, for example, it can be set to -1.25, etc., which is not limited here.
  • the predicted probability value of the target point s i when the predicted probability value of the target point s i is 1, it indicates that the target point s i has the highest possibility of being the object contour point, and at this time the target point s i has the largest first weight (that is, 1 ), and when the predicted probability value of the target point s i is 0, it indicates that the target point s i is the least likely to be an object contour point, and at this time the target point s i has the smallest first weight (ie exp(k 1 )) , the first weight is the lower limit value of the first weight when the search result includes the searched target point.
  • the search line segment l i where the target point s i is located is likely to be in an interfered state (for example, in a partially occluded state, is in the local area interfered by similar colors))
  • the weight of these sampling points in the subsequent acquisition process of pose parameters can be reduced Reference value, in order to alleviate the influence of interference factors on pose parameters as much as possible, and improve the accuracy of pose parameters.
  • the weight information may include a second weight
  • the second weight of the sampling point may be determined based on the first distance corresponding to the sampling point, and the second The weight is negatively correlated with the first distance
  • the second weight can be determined as a second value
  • the second value is the second value when the search result includes a searched target point.
  • k 2 represents a negative constant, which is used to control the decay speed of the second weight along with the first distance, which can be set according to application requirements, for example, it can be set to -3.5, etc., which is not limited here.
  • the first distance D(x ij , s i ) between them reaches the minimum (ie 0).
  • the target point s i has the largest second weight (namely 1), and when the sampling point x ij and the target point s i are at the two ends of the search line segment l i respectively, the second weight between them - The distance D(x ij , s i ) reaches the maximum (ie 1), and at this time the target point s i has the smallest second weight (ie exp(k 2 )).
  • the second weight can reduce the reference value of the sampling point x ij in the subsequent acquisition process of pose parameters, so as to alleviate the influence of interference factors on pose parameters as much as possible, and improve the accuracy of pose parameters.
  • the weight information of the sampling point may include the first weight and the second weight at the same time. The method of obtaining the second weight will not be repeated here.
  • the weight information includes both the first weight and the second weight, both the first weight and the second weight are positively correlated with the reference weight.
  • the product of the first weight and the second weight may be used as the reference weight.
  • the product of the first weight w c (x ij ) and the second weight w d (x ij ) can be used as the reference weight w ( x ij ).
  • Other sampling points can be deduced in the same way, and no more examples will be given here.
  • Step S13 Construct an objective function based on the attribute information and reference weights of the sampling points.
  • the attribute information of the sampling point may include: the first probability value and the first credibility that the sampling point belongs to the target object, and the second probability value and the first reliability that the sampling point does not belong to the target object Two credibility, then the first product of the first credibility and the first probability value and the second product of the second credibility and the second probability value can be obtained, and based on the first product and the second product, the sampling On this basis, the weighted combination of the joint probability value based on the reference weight of each sampling point is used to obtain the objective function.
  • the logarithm of the sum of the first product and the second product can be taken to obtain the joint probability value, and the weighted results of the joint probability value are summed by the reference weights of each sampling point to obtain the objective function E (p):
  • He(d(x ij )) represents the first reliability
  • P f (x ij ) represents the first probability value
  • 1-He(d(x ij )) represents the second reliability
  • P b (x ij ) represents the second probability value
  • w(x ij ) represents the reference weight.
  • L represents the collection of sampling points on all search line segments.
  • the captured image includes a foreground area (ie, the foreground area ⁇ f in FIG. 2 ) and a background area (ie, the foreground area in FIG. In the background area ⁇ b ), before constructing the objective function, each sampling point on the search line segment can be further verified. Taking the direction of the normal vector at the projected contour point as an example from the foreground area to the background area, when the directed Euclidean distance of the sampling point is greater than the first distance value (eg, 0), it can be considered that the sampling point belongs to the background area .
  • the first distance value eg, 0
  • the sampling point actually belongs to the foreground area, it can be filtered, that is, filtered from the above sampling point set L, as shown in Figure 2, the bottommost search line segment, the two sampling points located on the leftmost side of the search line segment are directed
  • the K-distances are both greater than the first distance value (eg, 0), so it can be considered that these two sampling points belong to the background area, but actually these two sampling points belong to the foreground area, then these two sampling points can be filtered.
  • the directed Euclidean distance of the sampling point is less than the second distance value (such as, 0)
  • the second distance value such as, 0
  • Step S14 Obtain the pose parameters of the target object in the captured image based on the objective function.
  • the projection profile is obtained by using the reference pose projection of the target object, the reference pose is the pose parameter of the target object in the reference image, and the reference image is obtained before the image is captured , then the objective function can be solved to obtain the updated parameters of the reference pose, and the updated parameters can be used to optimize the reference pose to obtain the pose parameters of the target object in the captured image.
  • the reference image may be an image of one frame before the captured image.
  • the reference image may be the t-1th frame image in the video data
  • the captured image may be the t-th frame image in the video data.
  • the objective function can be rewritten as the standard form of the nonlinear weighted least squares problem:
  • the above update parameter ⁇ p is expressed by Lie algebra.
  • ⁇ p can be converted into a Euclidean transformation matrix ⁇ T.
  • ⁇ T Euclidean transformation matrix
  • the pose parameter T′ of the target object in the captured image can be expressed as:
  • the pose parameters of each target object in the captured image can be obtained using the steps in the embodiments of the present disclosure. get.
  • the aforementioned mask image I s and depth image I d can be used to filter the sampling points in the disturbed state (eg, occluded).
  • the depth image I d may be obtained by rendering the captured image, and the specific rendering process will not be repeated here.
  • the depth image Id may specifically include depth values of each pixel in the captured image.
  • the background area corresponding to the kth target object can be searched first The sampling point x ij within and adjacent to the projected contour point m i of the search line segment, and check whether I s (x ij ) is equal to the index of another target object, and the depth value I d of the sampling point x ij (x ij ) is less than the depth value I d (m i ) of the projected contour point m i of the search line segment, if so, it can be considered that the search line segment l i where the sampling point x ij is located is in a disturbed state (for example, blocked), then you can filter This search all sampling points on the line segment l i .
  • FIG. 4a is a schematic diagram of an embodiment of a captured image
  • FIG. 4b is a schematic diagram of another embodiment of a mask image
  • FIG. 4c is a schematic diagram of an embodiment of a search line segment.
  • the foregoing local color histogram is constructed based on local regions surrounding the outline of the object, and at the same time, in order to enhance temporal continuity, each local region corresponds to a model vertex.
  • each local region corresponds to a model vertex.
  • the 3D model of the target object contains fewer vertices (for example, less than 50)
  • these local areas cannot completely cover the object outline, thus affecting the above-mentioned first probability value P f and second probability value P the accuracy of b .
  • the 3D model of the target object contains fewer vertices than the preset threshold (eg, 50), several (eg, 4) vertices can be added to each side of the 3D model to improve the local area , so that the local area can cover the outline of the object as much as possible.
  • the preset threshold eg, 50
  • several (eg, 4) vertices can be added to each side of the 3D model to improve the local area , so that the local area can cover the outline of the object as much as possible.
  • FIG. 5a is a schematic diagram of another embodiment of a captured image
  • FIG. 5b is a schematic diagram of an embodiment of a layout area
  • FIG. 5c is a schematic diagram of another embodiment of a partial area.
  • the local area (as shown by the hollow circle in the figure) does not Completely cover the object outline of the target object, in this case, by adding vertices (eg, up to 8) on each edge, the local area can completely cover the object outline.
  • the pose parameters of the target object in the captured image may be obtained through multiple iterations (for example, 7 iterations).
  • the pose parameters of the target object in the reference image can be used as the reference position, and the steps in the embodiments of the present disclosure are executed to obtain the pose of the target object in the captured image during the first iteration parameter, and use it as the reference pose of the second iteration, and re-execute the steps in the embodiment of the present disclosure to obtain the pose parameters of the target object in the captured image during the second iteration.
  • the pose parameters obtained in the i-1th iteration are used as the reference pose, and the steps in the embodiments of the present disclosure are performed to obtain the target object in the captured image in the i-th iteration Pose parameters, until the last iteration, the pose parameters of the target object in the captured image at the last iteration can be directly used as the final pose parameters of the target object in the captured image.
  • the above solution obtains several sampling points located on the search line segment in the captured image, and the search line segment passes through the projected contour point of the target object in the captured image, and the projected contour point is located on the projected contour point of the target object, and then obtains the attribute information of the sample point based on this, And obtain the reference weight of the sampling point, and the attribute information indicates the possibility that the sampling point belongs to the target object, so based on the attribute information of the sampling point and the reference weight, construct the objective function, and based on the objective function, obtain the pose of the target object in the captured image parameter.
  • the attribute information can refer to the possibility that the sampling point belongs to the target object
  • the reference weight can The reference value of the reference sampling points in the subsequent process of solving the pose parameters can help to alleviate the influence of interference factors on the pose solution as much as possible, and is conducive to improving the accuracy of the pose parameters.
  • FIG. 6 is a schematic flowchart of an embodiment of step S12 in FIG. 1 . Specifically, the following steps may be included:
  • Step S61 Search for the target point among several sampling points on the search line segment, and obtain the search result.
  • the target point is used to represent the object outline point of the target object.
  • the sampling point s i on the search line segment l i can be used to represent the object contour point, other cases can be deduced by analogy, and no more examples are given here.
  • each search line segment several sampling points on the search line segment can be used as current points respectively, and when the reference probability difference of the current point satisfies the first condition, the current point can be used as a candidate point, And select the candidate point whose predicted cost value satisfies the second condition as the target point.
  • rough selection can be performed based on the reference probability difference first, and then fine selection can be performed based on the predicted cost value, which is conducive to improving the efficiency and accuracy of screening target points.
  • the reference probability difference value of the current point may be the difference between the first probability values of two sampling points having a preset position relationship (eg, adjacent to the current point) with the current point.
  • all the search line segments in Figure 3 can be stacked in rows to construct a bundled image Ib of the search line segments, and as shown in Figure 7b, The first probability values of each sampling point on all the search line segments in Fig. 3 are also stacked row by row to construct a bundle image I p about the first probability values.
  • the search line segment contains 2*N+1 sampling points, wherein the sampling point at the middle position is the projected contour point, one side of the projected contour point corresponds to the foreground area, and the other side of the projected contour point corresponds to Therefore, the middle column of the cluster image I b corresponds to the projection contour, one side of the middle column corresponds to the foreground area, and the other side of the middle column corresponds to the background area.
  • the above candidate points may have three situations, that is, the candidate points may be object contour points, the candidate points may also be foreground interference points, and the candidate points may also be background interference points.
  • the candidate points may also be background interference points.
  • several consecutive sampling points can be selected from the section where the search line segment l i points to the background area to form the first sampling point set (For example, can include xi , j-1 , xi , j-2 , xi , j-3 ), and select a number of continuous sampling points in the section where the search line segment l i points to the foreground area to form the second sampling point set (eg, can include xi ,j+1 , xi ,j+2 , xi ,j+3 ).
  • the first sampling point set Theoretically, it should belong to the background area, while the second set of sampling points In theory, it should belong to the foreground area, so the probability value P(h ij
  • C) is the predicted probability value of the candidate point as the object contour point; and as described in the aforementioned disclosed embodiments, the candidate point h ij can be used as the target
  • C) can be written as P(s i
  • the sampling points are foreground interference points
  • the first set of sampling points and the second set of sampling points In theory, they should all belong to the foreground area, so the probability value P(h ij
  • F) is the predicted probability value of the candidate point as a foreground interference point.
  • sampling points are background interference points
  • a set of sampling points and the second set of sampling points In theory, they should all belong to the background area, so the probability value P(h ij
  • B) is the predicted probability value of the candidate point as the background interference point.
  • sampling point can be further defined as the normalized probability value P c (h ij ) of the object contour point:
  • P c (h ij ) is the normalized probability value of the candidate point as the object contour point.
  • C) satisfies the third condition can be further filtered.
  • the above-mentioned predicted cost value may include at least one of the first-generation value and the second-generation value, for example, it may include both the first-generation value and the second-generation value, or it may only include the first-generation value value, or only second-generation values.
  • the first-generation value can be related to the predicted probability value of the candidate point. For example, the first-generation value can be negatively correlated with the predicted probability value of the candidate point.
  • the first-generation value can be recorded as E d (h ij ), then the first-generation value Generation value E d (h ij ) can be expressed as:
  • the second-generation value is related to the second distance from the candidate point to the projected contour point on the search line segment.
  • the second-generation value can be positively correlated with the above-mentioned second distance.
  • the second-generation value can be recorded as E S (h ij ), then the second generation value E S (h ij ) can be expressed as:
  • the second generation value E of the target point is S (h ij ) is larger.
  • the first generation value and the second generation value can be weighted as the predicted cost value E(h ij ):
  • ⁇ represents a weighting factor, which can be set according to actual application needs, for example, it can be set to 0.015, which is not limited here.
  • the above-mentioned second condition may specifically include the smallest prediction cost value, that is, when the inter-frame motion of the target object or the camera is relatively peaceful, the second generation value will impose an additional penalty on the candidate points that are far away from the projection contour, so as to preferentially select Candidate points that are closer to the projected contour are used as target points.
  • the target point may not be found on the search line segment l i , then at this time, for the search line segment l i , you can mark To indicate that for the search line segment l i , its search results include not finding the target point.
  • Step S62 Based on the search results, respectively obtain weight information of several sampling points on the search line segment.
  • the weight information includes at least one of the first weight and the second weight, the first weight is related to the predicted probability value of the target point, the predicted probability value indicates the possibility of the sampling point as the object contour point, and the second The weight is related to the first distance from the target point to the sampling point.
  • Step S63 Obtain the reference weight of the sampling point based on the weight information.
  • the target point is searched in several sampling points on the search line segment to obtain the search result, and the target point is used to represent the object contour point of the target object, and based on the search result, the weight information of several sampling points on the search line segment is respectively obtained , and the weight information includes at least one of the first weight and the second weight, the first weight is related to the predicted probability value of the target point, the predicted probability value represents the possibility of the sampling point as the object contour point, and the second weight is related to the target point
  • the first distance to the sampling point is related, so the first weight and the second weight can represent the reference value of the sampling point from different angles, and based on this and then based on the weight information, the reference weight of the sampling point can be obtained, which can improve the reference weight in the subsequent solution Reference value during pose parameterization.
  • FIG. 8 is a schematic flow chart of another embodiment of the method for obtaining a pose in the present application. Specifically, the following steps may be included:
  • Step S801 down-sampling the captured image to obtain pyramid images of several resolutions.
  • 2 can be used as the downsampling ratio to downsample the captured image to obtain a pyramid image with 1/4 resolution, a pyramid image with 1/2 resolution, and a pyramid image with original resolution (that is, the captured image itself) .
  • Other situations can be deduced by analogy, and no more examples will be given here.
  • Step S802 According to the resolution from small to large, select the pyramid image as the current captured image in sequence.
  • the pyramid image with the highest rate is used as the currently captured image, and this cycle is repeated, and no more examples are given here.
  • Step S803 Obtain several sampling points located on the search line segment in the currently captured image.
  • the search line segment passes through the projected contour point of the target object in the captured image, the projected contour point is located on the projected contour of the target object, and the projected contour is obtained by using the reference pose projection of the target object.
  • Step S804 Obtain the attribute information of the sampling point, and obtain the reference weight of the sampling point.
  • the attribute information indicates the possibility that the sampling point belongs to the target object.
  • the attribute information includes: a first probability value and a first credibility that the sampling point belongs to the target object, a second probability value and a second credibility that the sampling point does not belong to the target object, and the first credibility
  • the smoothing factor s in formula (4) is negatively correlated.
  • the smoothing factor s can be set to 1.2
  • the smoothing factor s can be set to 0.8
  • the smoothing factor s can be set to 0.6, which is not limited here.
  • Step S805 Construct an objective function based on the attribute information and reference weights of the sampling points.
  • Step S806 Obtain the pose parameters of the target object in the currently captured image based on the objective function.
  • the pose parameters of the target object in the captured image may be obtained through multiple iterations for the captured image.
  • the pose parameters of the target object in the pyramid image can be obtained through several iterations, and the lower the resolution of the pyramid image, the more iterations. For example, for a pyramid image of 1/4 resolution, it can be iterated 4 times, for a pyramid image of 1/2 resolution, it can be iterated 2 times, for a pyramid image of original resolution (that is, the captured image itself) and In other words, it can be iterated once.
  • a specific iterative process reference may be made to relevant descriptions in the foregoing disclosed embodiments, and details are not repeated here.
  • Step S807 Determine whether the currently captured image is the last frame of the pyramid image, if not, execute step S808, otherwise execute step S810.
  • the pose parameters of the target object in the last frame of the pyramid image can be used as the final pose parameters of the target object in the captured image, otherwise the iterative process can be continued.
  • Step S808 Use the pose parameters obtained in this execution as the reference pose.
  • the pose parameters obtained in this execution can be used as the reference pose, and an iterative operation is performed on the next frame of the pyramid image.
  • Step S809 Re-execute step S802 and subsequent steps.
  • Step S810 Use the pose parameters obtained in this execution as the final pose parameters of the target object in the captured image.
  • the iterative operation can be ended to obtain the final pose parameters of the target object in the captured image.
  • the projection profile is obtained by using the reference pose projection of the target object, so that before the projection sampling, the captured image is down-sampled to obtain pyramid images of several resolutions, and the resolutions are from small to large, Sequentially select the pyramid image as the current captured image, and perform the above-mentioned steps of obtaining several sampling points on the search line segment in the captured image and subsequent steps on the current captured image, and the reference pose used in this execution is the one obtained in the previous execution Pose parameters, the pose parameters obtained by the last execution are used as the final pose parameters of the target object in the captured image, so that the pose estimation can be performed from coarse to fine in the process of obtaining the pose parameters, which can help improve Acquisition efficiency and accuracy of pose parameters.
  • FIG. 9 is a schematic frame diagram of an embodiment of a pose acquisition device 90 of the present application.
  • the pose acquisition device 90 is applied to electronic equipment, and includes: a projection sampling module 91, an information extraction module 92, a function construction module 93 and a pose solving module 94, and the projection sampling module 91 is configured to acquire several Sampling point; wherein, the search line segment passes through the projected contour point of the target object in the captured image, and the projected contour point is located at the projected contour of the target object; the information extraction module 92 is configured to obtain the attribute information of the sampling point, and obtain the reference weight of the sampling point; wherein , the attribute information indicates the possibility that the sampling point belongs to the target object; the function construction module 93 is configured to construct an objective function based on the attribute information and reference weight of the sampling point; the pose solving module 94 is configured to obtain the target object in the captured image based on the objective function pose parameters.
  • the information extraction module 92 includes a target point search submodule configured to search for the target point among several sampling points on the search line segment to obtain search results; wherein the target point is configured to represent the object contour point of the target object
  • the information extraction module 92 includes a weight information acquisition sub-module configured to obtain weight information of several sampling points on the search line segment respectively based on the search results; wherein the weight information includes at least one of the first weight and the second weight, the first The weight is related to the predicted probability value of the target point, the predicted probability value represents the possibility of the sampling point as an object contour point, and the second weight is related to the first distance from the target point to the sampling point;
  • the information extraction module 92 includes a reference weight acquisition submodule , configured to obtain the reference weight of the sampling point based on the weight information.
  • the attribute information includes: the first probability value that the sampling point belongs to the target object; the target point search submodule includes a current point acquisition unit configured to use several sampling points as the current point for each search line segment, respectively, The target point search submodule includes a candidate point acquisition unit configured to use the current point as a candidate point when the reference probability difference of the current point satisfies the first condition, and the target point search submodule includes a target point acquisition unit configured to select The candidate point whose predicted cost value satisfies the second condition is taken as the target point; wherein, the reference probability difference of the current point is the difference between the first probability values of two sampling points having a preset position relationship with the current point, and the predicted cost value includes the first probability value of At least one of the first-generation value and the second-generation value, the first-generation value is related to the predicted probability value of the candidate point, and the second-generation value is related to the second distance from the candidate point to the projected contour point on the search line segment.
  • the target point searching submodule includes a candidate point filtering unit configured to filter candidate points whose predicted probability values satisfy the third condition.
  • the preset position relationship is adjacent to the current point; and/or, the second condition includes the minimum predicted cost value; and/or, the first generation value is negatively correlated with the predicted probability value of the candidate point, and The second generation value is positively related to the second distance.
  • the weight information includes a first weight; the weight information acquisition submodule includes a first determination unit configured to determine the sampling point based on the predicted probability value of the target point when the search result includes the target point. The first weight, wherein the first weight is positively correlated with the predicted probability value of the target point; the weight information acquisition submodule includes a second determination unit configured to determine the first weight when the search result includes no target point. is the first value; wherein, the first value is the lower limit value of the first weight when the search result includes the searched target point.
  • the weight information includes a second weight
  • the weight information acquisition submodule includes a third determination unit, configured to determine the sampling point based on the first distance corresponding to the sampling point when the search result includes a searched target point The second weight, wherein, the second weight is negatively correlated with the first distance
  • the weight information acquisition submodule includes a fourth determination unit configured to determine the second weight as the second weight when the search result includes no target point. Two values; wherein, the second value is the upper limit value of the second weight when the search result includes the searched target point.
  • the weight information includes a first weight and a second weight, and both the first weight and the second weight are positively correlated with the reference weight.
  • the attribute information includes: a first probability value and a first degree of confidence that the sampling point belongs to the target object, and a second probability value and second degree of confidence that the sampling point does not belong to the target object;
  • the function building block 93 includes a joint probability calculation submodule configured to obtain a first product of the first credibility and the first probability value and a second product of the second credibility and the second probability value, and based on the first product and the second product sum to obtain the joint probability value of the sampling point;
  • the function construction module 93 includes a joint probability weighting sub-module configured to weight the joint probability value based on the reference weight of each sampling point to obtain the objective function.
  • the first credibility and the second credibility are negatively correlated, and the first credibility of the sampling point is negatively correlated with the directed Euclidean distance from the corresponding projected contour point to the sampling point,
  • the corresponding projected contour points are located on the same search line segment as the sampling points.
  • the captured image includes a foreground area and a background area divided based on the projection contour;
  • the function construction module 93 includes a first filtering submodule configured such that the directional Euclidean distance at the sampling point is greater than the first distance value, and When the sampling point belongs to the foreground area, filter the sampling point;
  • the function construction module 93 includes a second filtering submodule configured to be less than the second distance value when the directed Euclidean distance of the sampling point, and the sampling point belongs to the background area , to filter sampling points.
  • the projection profile is obtained by using the reference pose projection of the target object;
  • the pose acquisition device 90 includes a down-sampling module configured to down-sample the captured image to obtain pyramid images of several resolutions;
  • the posture acquisition device 90 includes an image selection module configured to select the pyramid image as the current captured image in turn according to the resolution from small to large, and the projection sampling module 91, the information extraction module 92, the function construction module 93 and the pose solving module 94 are configured as Execute the steps of acquiring several sampling points on the search line segment in the captured image and subsequent steps for the current captured image; wherein, the reference pose used in this execution is the pose parameter obtained in the previous execution, and the position and posture obtained in the last execution The pose parameters are used as the final pose parameters of the target object in the captured image.
  • the projection profile is obtained by using the reference pose projection of the target object, the reference pose is the pose parameter of the target object in the reference image, and the reference image is obtained before the image is captured;
  • the pose solution Module 94 includes a function solving submodule configured to solve the objective function to obtain the update parameters of the reference pose;
  • the pose solving module 94 includes a pose optimization submodule configured to optimize the reference pose using the update parameters to obtain the pose Attitude parameters.
  • FIG. 10 is a schematic frame diagram of an embodiment of an electronic device 100 of the present application.
  • the electronic device 100 includes a memory 101 and a processor 102 coupled to each other, and the processor 102 is configured to execute program instructions stored in the memory 101, so as to implement the steps in any of the above embodiments of the attitude acquisition method.
  • the electronic device 100 may include, but is not limited to: a microcomputer and a server.
  • the electronic device 100 may also include mobile devices such as notebook computers and tablet computers, which are not limited here.
  • the processor 102 is configured to control itself and the memory 101 to implement the steps in any of the above embodiments of the pose acquisition method.
  • the processor 102 may also be called a CPU (Central Processing Unit, central processing unit).
  • the processor 102 may be an integrated circuit chip with signal processing capability.
  • the processor 102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the processor 102 may be jointly implemented by integrated circuit chips.
  • FIG. 11 is a schematic diagram of an embodiment of a computer-readable storage medium 110 of the present application.
  • the computer-readable storage medium 110 stores program instructions 111 that can be executed by the processor, and the program instructions 111 are used to implement the steps in any of the above embodiments of the pose acquisition method.
  • the above solution can help to alleviate the influence of interference factors on the pose solution as much as possible, and is beneficial to improve the accuracy of the pose parameters.
  • An embodiment of the present disclosure provides a computer program, including computer readable codes.
  • a processor in the electronic device executes any one of the above pose acquisitions. method.
  • the disclosed methods and devices may be implemented in other ways.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .
  • the embodiment of the present application discloses a pose acquisition method and device, electronic equipment, a storage medium and a program, wherein the pose acquisition method includes: acquiring a number of sampling points located on the search line segment in the captured image; wherein the search line segment is photographed The projected contour point of the target object in the image, the projected contour point is located on the projected contour of the target object; obtain the attribute information of the sampling point, and obtain the reference weight of the sampling point; where the attribute information indicates the possibility that the sampling point belongs to the target object; based on sampling Point attribute information and reference weights are used to construct an objective function; based on the objective function, the pose parameters of the target object in the captured image are obtained.
  • the above solution can improve the accuracy of pose parameters.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种位姿获取方法及装置、电子设备、存储介质和程序，其中，位姿获取方法包括：获取拍摄图像中位于搜索线段上的若干采样点；其中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓；获取采样点的属性信息，以及获取采样点的参考权重；其中，属性信息表示采样点属于目标物体的可能性；基于采样点的属性信息和参考权重，构建目标函数；基于目标函数，得到拍摄图像中目标物体的位姿参数。

Description

位姿获取方法及装置、电子设备、存储介质和程序
相关申请的交叉引用
本专利申请要求2021年06月07日提交的中国专利申请号为202110626207.X、申请人为浙江商汤科技开发有限公司和山东大学，申请名称为“位姿获取方法及装置和电子设备、存储介质”的专利申请的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本申请涉及计算机视觉技术领域，涉及但不限于一种位姿获取方法及装置、电子设备、存储介质和程序。
背景技术
随着信息技术的发展，位姿参数已经在诸如增强现实系统、机器人手眼标定、互动游戏、人机交互等诸多场景受到越来越广泛的应用。例如，在增强现实系统中，根据位姿参数可以将虚拟物体渲染并叠加至视频图像中现实物体上，以实现具有空间和几何一致性的虚实融合效果。
目前，在实际场景中，由于诸如局部遮挡、相似颜色等干扰因素，位姿参数的精度往往受到严重影响。有鉴于此，如何提高位姿参数的精度成为亟待解决的问题。
发明内容
本申请实施例提供一种位姿获取方法及装置、电子设备、存储介质和程序。
本申请实施例第一方面提供了一种位姿获取方法，应用于电子设备中，包括：
获取拍摄图像中位于搜索线段上的若干采样点；其中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓；获取采样点的属性信息，以及获取采样点的参考权重；其中，属性信息表示采样点属于目标物体的可能性；基于采样点的属性信息和参考权重，构建目标函数；基于目标函数，得到拍摄图像中目标物体的位姿参数。
因此，获取拍摄图像中位于搜索线段上的若干采样点，且搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓，基于此再获取采样点的属性信息，以及获取采样点的参考权重，且属性信息表示采样点属于目标物体的可能性，从而，基于采样点的属性信息和参考权重，构建目标函数，并基于目标函数，得到拍摄图像中目标物体的位姿参数。由于目标函数是基于采样点的属性信息和参考权重两者而构建的，其中，属性信息能够反映采样点属于目标物体的可能性，参考权重能够反映采样点在后续求解位姿参数过程中的参考价值，因此，在基于采样点的属性信息和参考权重构建目标函数时，能够降低干扰因素对于位姿求解的影响，有利于提高位姿参数的精度。
在一种实现方式中，获取采样点的参考权重，包括：在搜索线段上的若干采样点中搜索目标点，得到搜索结果；其中，目标点用于表示目标物体的物体轮廓点；基于搜索结果，分别获取搜索线段上的若干采样点的权重信息；其中，权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，且第二权重与目标点至采样点的第一距离相关；基于权重信息，得到采样点的参考权重。
因此，在搜索线段上的若干采样点中搜索目标点，得到搜索结果，且目标点用于表示目标物体的物体轮廓点，并基于搜索结果，分别获取搜索线段上的若干采样点的权重信息，且权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，而第二权重与目标点至采样点的第一距离相关，故第一权重和第二权重能够从不同角度表征采样点的参考价值，进而基于此再基于权重信息，得到采样点的参考权重，能够提高参考权重在后续求解位姿参数过程中的参考价值。
在一种实现方式中，属性信息包括：采样点属于目标物体的第一概率值；在搜索线段上的若干采样点中搜索目标点，得到搜索结果，包括：对于每条搜索线段，将若干采样点分别作为当前点，在当前点的参考概率差值满足第一条件的情况下，将当前点作为候选点，并选取预测代价值满足第二条件的候选点作为目标点。其中，当前点的参考概率差值为与当前点具有预设位置关系的两个采样点的第一概率值之差，预测代价值包括第一代价值和第二代价值中至少一者，第一代价值与候选点的预测概率值相关，第二代价值与候选点至搜索线段上的投影轮廓点的第二距离相关。
因此，属性信息包括采样点属于目标物体的第一概率值，并对于每条搜索线段，将若干采样点分别作为当前点，在当前点的参考概率差值满足第一条件的情况下，将当前点作为候选点，以及选取预 测代价值满足第二条件的候选点作为目标点，且当前点的参考概率差值为与当前点具有预设位置关系的两个采样点的第一概率值之差，预测代价值包括第一代价值和第二代价值中至少一者，第一代价值与候选点的预测概率值相关，第二代价值与候选点至搜索线段上的投影轮廓点的第二距离相关，即第一代价值和第二代价值分别不同角度表征候选点视为物体轮廓点的代价，故通过参考概率差值先粗选得到候选点，再基于预测代价值细选得到目标点，能够有利于提高筛选目标点的效率和精度。
在一种实现方式中，在选取预测代价值满足第二条件的候选点作为目标点之前，方法还包括：过滤预测概率值满足第三条件的候选点。
因此，在从候选点中细选得到目标点之前，先过滤预测概率值满足第三条件的候选点，而由于预测概率值表示采样点作为物体轮廓点的可能性，故能够有利于进一步提高目标点的筛选效率。
在一种实现方式中，预设位置关系为与当前点相邻；和/或，第二条件包括预测代价值最小；和/或，第一代价值与候选点的预测概率值负相关，且第二代价值与第二距离正相关。
因此，通过将预设位置关系设置为与当前点相邻，能够有利于准确评估每一采样点的第一概率值突变情况，有利于提高候选点的准确性；而通过将第二条件设置为包括预测代价值最小，能够有利于进一步尽可能地缓解干扰因素对于选取目标点的影响，提高位姿参数的精度；而通过将第一代价值设置为与候选点的预测概率值负相关，且第二代价值与第二距离正相关，能够有利于提高第一代价值和第二代价值的准确性。
在一种实现方式中，权重信息包括第一权重；基于搜索结果，分别获取搜索线段上的若干采样点的权重信息，包括：在搜索结果包括搜索到目标点的情况下，基于目标点的预测概率值确定采样点的第一权重，其中，第一权重与目标点的预测概率值正相关；和/或，在搜索结果包括未搜索到目标点的情况下，将第一权重确定为第一数值；其中，第一数值为在搜索结果包括搜索到目标点的情况下第一权重的下限值。
因此，权重信息包括第一权重，且在搜索结果包括搜索到目标点的情况下，基于目标点的预测概率值确定采样点的第一权重，且第一权重与目标点的预测概率值正相关，在搜索结果包括未搜索到目标点的情况下，将第一权重确定为第一数值，且第一数值为搜索结果包括搜索到目标点的情况下第一权重的下限值，能够以搜索线段整体为维度，确定搜索线段上各个采样点的第一权重，有利于提高获取第一权重的效率。
在一种实现方式中，权重信息包括第二权重；基于搜索结果，分别获取搜索线段上的若干采样点的权重信息，包括：在搜索结果包括搜索到目标点的情况下，基于采样点对应的第一距离确定采样点的第二权重，其中，第二权重与第一距离负相关；和/或，在搜索结果包括未搜索到目标点的情况下，将第二权重确定为第二数值；其中，第二数值为在搜索结果包括搜索到目标点的情况下第二权重的上限值。
因此，权重信息包括第二权重，且在搜索结果包括搜索到目标点的情况下，基于采样点对应的第一距离确定采样点的第二权重，且第二权重与第一距离负相关，在搜索结果包括未搜索到目标点的情况下，将第二权重确定为第二数值，且第二数值为在搜索到目标点的情况下第二权重的上限值，能够以搜索线段整体为维度，确定搜索线段上各个采样点的第二权重，有利于提高获取第二权重的效率。
在一种实现方式中，权重信息包括第一权重和第二权重，且第一权重、第二权重均与参考权重正相关。
因此，将权重信息设置为同时包含第一权重和第二权重，且第一权重、第二权重均与参考权重正相关，故能够同时从第一权重、第二权重两种不同维度表征采样点在后续求解位姿参数过程中的参考价值，有利于提高参考权重本身的参考价值。
在一种实现方式中，属性信息包括：采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度；基于采样点的属性信息和参考权重，构建目标函数，包括：获取第一可信度与第一概率值的第一积以及第二可信度与第二概率值的第二积，并基于第一积和第二积之和，得到采样点的联合概率值；基于各个采样点的参考权重对联合概率值的加权结果，得到目标函数。
因此，属性信息包括采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度，在此基础上，获取第一可信度与第一概率值的第一积以及第二可信度与第二概率值的第二积，并基于第一积和第二积之和，得到采样点的联合概率值，从而能够从采样点属于目标物体和采样点不属于目标物体两种角度来表征采样点的联合概率值，并以此通过各个采样点的参考权重对联合概率值的加权结果来构建得到目标函数，能够提高目标函数的准确性，有利于提高参考位姿的精度。
在一种实现方式中，第一可信度与第二可信度为负相关关系，采样点的第一可信度与相应投影轮廓点至采样点的有向欧氏距离为负相关关系，相应投影轮廓点与采样点位于相同搜索线段上。
因此，第一可信度与第二可信度为负相关关系，且采样点的第一可信度与相应投影轮廓点至采样点的有向欧氏距离为负相关关系，相应投影轮廓点与采样点位于相同搜索线段上，即有向欧氏距离越小，采样点属于目标物体的第一可信度越高，而采样点不属于目标物体的第二可信度越低，能够有利于尽可能地缓解诸如局部遮挡等干扰因素的影响。
在一种实现方式中，拍摄图像包括基于投影轮廓划分的前景区域和背景区域；在获取第一可信度与第一概率值的第一积以及第二可信度与第二概率值的第二积之前，方法还包括：在采样点的有向欧氏距离大于第一距离值，且采样点属于前景区域的情况下，过滤采样点；和/或，在采样点的有向欧氏距离小于第二距离值，且采样点属于背景区域的情况下，过滤采样点。
因此，拍摄图像包括基于投影轮廓划分的前景区域和背景区域，在计算联合概率值之前，先检测有向欧氏距离大于第一距离值且属于前景区域的采样点，即可将其视为干扰点，并将其过滤，有利于尽可能地降低干扰点对于后续求解位姿参数的影响，而先检测有向欧氏距离小于第二距离值且属于背景区域的采样点，即可将其视为干扰点，并将其过滤，有利于尽可能地降低干扰点对于后续求解位姿参数的影响。
在一种实现方式中，投影轮廓是利用目标物体的参考位姿投影得到的；在获取拍摄图像中位于搜索线段上的若干采样点之前，方法包括：对拍摄图像进行降采样，得到若干种分辨率的金字塔图像；按照分辨率由小到大，依次选择金字塔图像作为当前拍摄图像，并对当前拍摄图像执行获取拍摄图像中位于搜索线段上的若干采样点的步骤以及后续步骤；其中，本次执行所采用的参考位姿为上一次执行得到的位姿参数，最后一次执行得到的位姿参数作为拍摄图像中目标物体最终的位姿参数。
因此，投影轮廓是利用目标物体的参考位姿投影得到的，从而在投影采样之前，先对拍摄图像进行降采样，以得到若干种分辨率的金字塔图像，并按照分辨率由小到大，依次选择金字塔图像作为当前拍摄图像，以及对当前拍摄图像执行上述获取拍摄图像中位于搜索线段上的若干采样点的步骤以及后续步骤，且本次执行所采用的参考位姿为上一次执行得到的位姿参数，最后一次执行得到的位姿参数作为拍摄图像中目标物体最终的位姿参数，从而能够在位姿参数的获取过程中，由粗到细地进行位姿估计，进而能够有利于提高位姿参数的获取效率和精度。
在一种实现方式中，投影轮廓是利用目标物体的参考位姿投影得到的；基于目标函数，得到拍摄图像中目标物体的位姿参数，包括：对目标函数进行求解，得到参考位姿的更新参数；利用更新参数对参考位姿进行优化，得到位姿参数。
因此，投影轮廓是利用目标物体的参考位姿投影得到的，参考位姿是参考图像中目标物体的位姿参数，且参考图像是在拍摄图像之前拍摄得到的，并对目标函数进行求解，得到参考位姿的更新参数，以及利用更新参数对参考位姿进行优化，得到位姿参数，有利于在对目标物体拍摄过程中，准确地对位姿参数进行持续跟踪。
本申请实施例第二方面提供了一种位姿获取装置，应用于电子设备中，包括：
投影采样模块、信息获取模块、函数构建模块和位姿求解模块，投影采样模块配置为获取拍摄图像中位于搜索线段上的若干采样点；其中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓；信息提取模块配置为获取采样点的属性信息，以及获取采样点的参考权重；其中，属性信息表示采样点属于目标物体的可能性；函数构建模块配置为基于采样点的属性信息和参考权重，构建目标函数；位姿求解模块配置为基于目标函数，得到拍摄图像中目标物体的位姿参数。
本申请实施例第三方面提供了一种电子设备，包括相互耦接的存储器和处理器，处理器用于执行存储器中存储的程序指令，以实现上述第一方面中的位姿获取方法。
本申请实施例第四方面提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述第一方面中的位姿获取方法。
本申请实施例第五方面提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一种位姿获取方法。
上述方案，获取拍摄图像中位于搜索线段上的若干采样点，且搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓，基于此再获取采样点的属性信息，以及获取采样点的参考权重，且属性信息表示采样点属于目标物体的可能性，从而，基于采样点的属性信息和参考权重，构建目标函数，并基于目标函数，得到拍摄图像中目标物体的位姿参数。由于目标函数是基于采样点的属性信息和参考权重两者而构建的，其中，属性信息能够反映采样点属于目标物体的可能 性，参考权重能够反映采样点在后续求解位姿参数过程中的参考价值，因此，在基于采样点的属性信息和参考权重构建目标函数时，能够降低干扰因素对于位姿求解的影响，有利于提高位姿参数的精度。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开实施例。根据下面参考附图对示例性实施例的详细说明，本公开实施例的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1是本申请位姿获取方法一实施例的流程示意图；
图2是轮廓掩码一实施例的示意图；
图3是投影轮廓和搜索线段一实施例的示意图；
图4a是拍摄图像一实施例的示意图；
图4b是掩码图像另一实施例的示意图；
图4c是搜索线段一实施例的示意图；
图5a是拍摄图像另一实施例的示意图；
图5b是布局区域一实施例的示意图；
图5c是局部区域另一实施例的示意图；
图6是图1中步骤S12一实施例的流程示意图；
图7a是图3中搜索线段的集束图像；
图7b是图3中搜索线段上各个采样点的第一概率值的集束图像；
图8是本申请位姿获取方法另一实施例的流程示意图；
图9是本申请位姿获取装置一实施例的框架示意图；
图10是本申请电子设备一实施例的框架示意图；
图11是本申请计算机可读存储介质一实施例的框架示意图。
具体实施方式
下面结合说明书附图，对本申请实施例的方案进行详细说明。
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、接口、技术之类的具体细节，以便透彻理解本申请。
本文中术语“系统”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。
本申请实施例提供的位姿获取方法，可以应用于增强现实系统。在增强现实系统中，需要准确估计现实物体相对于相机的位姿，才能根据位姿参数将虚拟物体准确地渲染并叠加到视频中的现实物体上，实现具有空间和几何一致性的虚实融合效果。
然而，在实际应用中，复杂环境下存在局部遮挡和相似颜色，容易产生对现实物体的跟踪偏移和跟踪失败，导致虚拟物体与真实物体的配准产生偏移。
为了解决上述技术问题，本申请实施例提供的位姿获取方法，通过检测并处理被局部遮挡和相似颜色干扰的像素点，提高物体的位姿估计精度，实现精确的虚实配准和几何一致的虚实融合效果。
以下，对本申请实施例提供的位姿获取方法进行详细介绍。
请参阅图1，图1是本申请位姿获取方法一实施例的流程示意图。具体而言，可以包括如下步骤：
步骤S11：获取拍摄图像中位于搜索线段上的若干采样点。
本公开实施例中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓。
在一个实施场景中，投影轮廓是利用目标物体的参考位姿投影得到的，而参考位姿是参考图像中目标物体的位姿参数，且参考图像是在拍摄图像之前拍摄得到的。例如，在现实场景中，可以对目标物体拍摄视频数据，视频数据可以包含多帧图像，则对于其中第t-1帧图像而言，可以采样本公开实 施例中步骤得到目标物体在第t-1帧图像中的位姿参数T t-1，在获取其中第t帧图像中目标物体的位姿参数时，可以将T t-1作为参考位姿，并利用本公开实施例中步骤得到目标物体在第t帧图像中的位姿参数T t，以此类推，在此不再一一举例。
在一个实施场景中，为了提高投影便利性，可以预先对目标物体进行三维建模，得到目标物体的三维模型。
应理解，三维模型可以包括若干顶点以及连接顶点的边。三维建模的具体过程，可以参阅三维建模的相关技术细节，在此不再赘述。
在一个实施场景中，为了便于描述，参考位姿可以记为T，其可以表示为一个4*4的齐次矩阵：
Figure PCTCN2021127307-appb-000001
上述公式(1)中，
Figure PCTCN2021127307-appb-000002
表示特殊欧氏群，R表示旋转参数，t表示平移参数，且R为
Figure PCTCN2021127307-appb-000003
(即特殊正交群)，t为实数矩阵。在此基础上，可以利用相机内参K以及上述参考位姿T将目标物体上的三维点X投影至拍摄图像中，得到三维点X对应于拍摄图像中的像素点x：
Figure PCTCN2021127307-appb-000004
上述公式(2)中，π(X)＝[X/Z,Y/Z] T，
Figure PCTCN2021127307-appb-000005
表示三维点X的齐次坐标，即
Figure PCTCN2021127307-appb-000006
三维点X的普通坐标表示为
Figure PCTCN2021127307-appb-000007
此外，在如前述持续跟踪场景下，帧间相对位姿△T可以利用李代数由六维捻度矢量(即twist vector)表示，即p＝[w 1 w 2 w 3 v 1 v 2 v 3]。
在一个实施场景中，为了便于后续确定各个像素点的相关属性，可以基于目标物体的投影结果，得到轮廓掩码，且该轮廓掩码中每个像素点分别对应于拍摄图像中相同位置的像素点。请结合参阅图2，图2是轮廓掩码一实施例的示意图。如图2所示，目标物体经投影之后，可以得到投影轮廓，拍摄图像基于投影轮廓划分为前景区域(即图2中前景区域Ω f)和背景区域(即图2中背景区域Ω b)。
在一个实施场景中，可以构造经过投影轮廓上投影轮廓点m i的搜索线段l i。搜索线段具体可以沿投影轮廓在投影轮廓点m i处的法向量n i进行构造。在此基础上，可以在搜索线段上提取得到若干采样点。例如，可以提取搜索线段l i上的投影轮廓点m i以及分别位于投影轮廓点m i两侧的N个(如，7个、8个、9个)像素点，作为搜索线段l i上的若干采样点(即图2中实心圆点)。
应理解，图2所示仅仅为实际应用过程中，可能存在的一种投影轮廓，并不因此而限定投影轮廓的具体行程，其他情况可以以此类推，在此不再一一举例。
在一个实施场景中，请结合参阅图3，图3是投影轮廓和搜索线段一实施例的示意图。如图3所示，在现实场景中，可以基于投影轮廓上各个投影轮廓点，构造多条搜索线段。其他拍摄图像可以以此类推，在此不再一一举例。
步骤S12：获取采样点的属性信息，以及获取采样点的参考权重。
本公开实施例中，属性信息表示采样点属于目标物体的可能性。例如，属性信息可以包括采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度。
应理解，上述第一可信度可以表示第一概率值的可信程度，上述第二可信度可以表示第二概率值可信程度。此外，若采样点属于目标物体，则可以认为该采样点属于拍摄图像中实际的前景区域，反之，若采样点不属于目标物体，则可以认为该采样点属于拍摄图像中实际的背景区域。
在一个实施场景中，为了便于描述，搜索线段l i上第j个采样点可以记为x ij，采样点x ij的第一概率值可以记为P f(x ij)，采样点x ij的第二概率值可以记为P b(x ij)。
应理解，第一概率值和第二概率值可以通过时间连续的局部颜色直方图来确定，第一概率值和第二概率值的具体获取过程，可以参阅时间连续的局部颜色直方图的具体技术细节，在此不再赘述。
在一个实施场景中，第一可信度与第二可信度为负相关关系，即第一可信度越高，第二可信度越低，反之，第一可信度越低，第二可信度越高。此外，采样点的第一可信度与相应投影轮廓点至采样点的有向欧氏距离为负相关关系，且相应投影轮廓点与采样点位于相同搜索线段上。请结合参阅图2， 以搜索线段l i为例，该搜索线段l i上某一采样点的第一可信度与该采样点至搜索线段l i上投影轮廓点m i的有向欧氏距离负相关，其他搜索线段上各个采样点可以以此类推获取其第一可信度，在此不再一一举例。
在一个具体的实施场景中，可以基于采样点的第一坐标、相应投影轮廓点的第二坐标以及前述法向量，获取采样点至相应投影轮廓点的有向欧氏距离。仍以搜索线段l i上第j个采样点x ij为例，其有向欧氏距离d(x ij)可以表示为：
Figure PCTCN2021127307-appb-000008
上述公式(3)中，m i表示搜索线段l i上的投影轮廓点，
Figure PCTCN2021127307-appb-000009
表示投影轮廓在投影轮廓点m i处的法向量的转置。
在另一个具体的实施场景中，为了平滑第一可信度，可以采用光滑可导的阶跃函数(如，Heaviside函数)处理有向欧氏距离，得到第一可信度。仍以搜索线段l i上第j个采样点x ij为例，其第一可信度He(d(x ij))可以表示为：
Figure PCTCN2021127307-appb-000010
上述公式(4)中，s表示平滑因子，s越大，第一可信度He(d(x ij))随有向欧氏距离d(x ij)变化越趋于剧烈；反之，s越小，第一可信度He(d(x ij))随有向欧氏距离d(x ij)变化越趋于平缓。
在又一个实施场景中，第一可信度与第二可信度之和可以为1。仍以搜索线段l i上第j个采样点x ij为例，在得到采样点x ij的第一可信度He(d(x ij))之后，可以将1-He(d(x ij))作为采样点x ij的第二可信度。
在一个实施场景中，对于每条搜索线段而言，可以在搜索线段上的若干采样点中搜索目标点，得到搜索结果，且目标点用于表示目标物体的物体轮廓点。在此基础上，可以基于搜索结果，分别获取搜索线段上的若干采样点的权重信息，且权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，而第二权重与目标点至采样点的第一距离相关，从而可以基于权重信息，得到采样点的参考权重。
应理解，目标点的搜索过程以及预测概率值的计算过程，可以参阅下述公开实施例中相关描述，在此暂不赘述。此外，物体轮廓点为目标物体在拍摄图像中实际轮廓点，如图2所示，搜索线段l i上采样点s i同时也位于物体轮廓(即图2中物体轮廓)，则该采样点s i同时也为目标物体在拍摄图像中实际轮廓点。
在一个具体的实施场景中，权重信息可以包括第一权重，则在搜索结果包括搜索到目标点的情况下，可以基于目标点的预测概率值确定采样点的第一权重，且第一权重与目标点的预测概率值正相关，而在搜索结果包括未搜索到目标点的情况下，可以将第一权重确定为第一数值，且第一数值为在搜索结果包括搜索到目标点的情况下第一权重的下限值。仍以搜索线段l i上第j个采样点x ij为例，第一权重w c(x ij)可以表示为：
Figure PCTCN2021127307-appb-000011
上述公式(5)中，
Figure PCTCN2021127307-appb-000012
表示搜索线段l i能够搜索到目标点s i，P(s i|C)表示目标点s i的预测概率值，即表示目标点s i为物体轮廓点的可能性，目标点s i的预测概率值越大，目标点s i为物体轮廓点的可能性越高，反之，目标点s i的预测概率值越小，目标点s i为物体轮廓点的可能性越低。此外，k 1表示一个负常数，用于控制第一权重随预测概率值的衰减速度，具体可以根据应用需要进行设置，如可以设置为-1.25等，在此不做限定。
如公式(5)所示，当目标点s i的预测概率值为1时，表明目标点s i为物体轮廓点的可能性最高，此时目标点s i具有最大的第一权重(即1)，而当目标点s i的预测概率值为0时，表明目标点s i为物体轮廓点的可能性最低，此时目标点s i具有最小的第一权重(即exp(k 1))，该第一权重即为在搜索结果包括搜索到目标点的情况下第一权重的下限值。
应理解，若目标点s i预测概率值P(s i|C)过小，则该目标点s i所在的搜索线段l i很可能出于被干扰状态(如，处于被局部遮挡的状态、处于被相似颜色干扰的局部区域内)，则通过调小该目标点s i所在的搜索线段l i上各个采样点的第一权重，可以调低这些采样点在后续位姿参数获取过程中的参考价值，以尽可能地缓解干扰因素对于位姿参数的影响，提高位姿参数的精度。
在另一个具体的实施场景中，权重信息可以包括第二权重，则在搜索结果包括搜索到目标点的情况下，可以基于采样点对应的第一距离确定采样点的第二权重，且第二权重与第一距离负相关，而在搜索结果包括未搜索到目标点的情况下，可以将第二权重确定为第二数值，且第二数值为在搜索结果包括搜索到目标点的情况下第二权重的上限值。仍以搜索线段l i上第j个采样点x ij为例，第二权重w d(x ij)可以表示为：
Figure PCTCN2021127307-appb-000013
上述公式(6)中，
Figure PCTCN2021127307-appb-000014
表示搜索线段l i能够搜索到目标点s i，D(x ij,s i)表示采样点x ij对应的第一距离，即目标点s i至采样点x ij之间的距离。具体地，第一距离D(x ij,s i)可以基于采样点x ij的第一坐标、目标点s i的第三坐标以及采样点x ij所在的搜索线段l i的长度N i(即搜索线段l i所包含的采样点个数)计算得到，如D(x ij,s i)＝||x ij-s i||/N i。
此外，k 2表示一个负常数，用于控制第二权重随第一距离的衰减速度，具体可以根据应用需要进行设置，如可以设置为-3.5等，在此不做限定。如公式(6)所示，当采样点x ij与目标点s i为同一采样点的情况下，两者之间的第一距离D(x ij,s i)达到最小(即0)。
在此情况下，目标点s i具有最大的第二权重(即1)，而当采样点x ij与目标点s i分别处于搜索线段l i两个末端的情况下，两者之间的第一距离D(x ij,s i)达到最大(即1)，此时目标点s i具有最小的第二权重(即exp(k 2))。
应理解，采样点x ij距离目标点s i越远，采样点x ij越容易处于被干扰状态(如，被复杂背景干扰、被相似颜色干扰)，则通过调小该采样点x ij的第二权重，可以调低该采样点x ij在后续位姿参数获取过程中的参考价值，以尽可能地缓解干扰因素对于位姿参数的影响，提高位姿参数的精度。
在又一个具体的实施场景中，采样点的权重信息可以同时包含第一权重和第二权重，在此情况下，第一权重和第二权重的获取过程，可以分别参阅前述第一权重、第二权重的获取方式，在此不再赘述。此外，在权重信息同时包含第一权重和第二权重的情况下，第一权重、第二权重均与参考权重正相关。
例如，可以将第一权重与第二权重之积作为参考权重。仍以搜索线段l i上第j个采样点x ij为例，可以将第一权重w c(x ij)与第二权重w d(x ij)之积作为采样点x ij的参考权重w(x ij)。其他采样点可以以此类推，在此不再一一举例。
步骤S13：基于采样点的属性信息和参考权重，构建目标函数。
在一个实施场景中，如前所述，采样点的属性信息可以包括：采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度，则可以获取第一可信度与第一概率值的第一积以及第二可信度与第二概率值的第二积，并基于第一积和第二积，得到采样点的联合概率值，在此基础上，再基于各个采样点的参考权重对联合概率值的加权结合，得到目标函数。
在一个具体的实施场景中，可以对第一积和第二积之和取对数，得到联合概率值，并对各个采样 点的参考权重对联合概率值的加权结果求和，得到目标函数E(p)：
Figure PCTCN2021127307-appb-000015
上述公式(7)中，He(d(x ij))表示第一可信度，P f(x ij)表示第一概率值，1-He(d(x ij))表示第二可信度，P b(x ij)表示第二概率值，w(x ij)表示参考权重，具体获取过程，可以参阅前述相关描述，在此不再赘述。此外，L表示所有搜索线段上采样点的集合。
在另一个具体的实施场景中，请结合参阅图2，拍摄图像包括基于投影轮廓(即图2中投影轮廓)划分的前景区域(即图2中前景区域Ω f)和背景区域(即图2中背景区域Ω b)，则在构造目标函数之前，还可以进一步对搜索线段上各个采样点进行校验。以投影轮廓点处的法向量的方向为从前景区域指向背景区域为例，则在采样点的有向欧氏距离大于第一距离值(如，0)时，可以认为该采样点属于背景区域。
若该采样点实际属于前景区域，则可以将其过滤，即从上述采样点集合L中过滤，如图2中最下侧搜索线段，位于该搜索线段最左侧两个采样点其有向欧氏距离均大于第一距离值(如，0)，故可以认为这两个采样点属于背景区域，而实际上这两个采样点属于前景区域，则可以将这两个采样点过滤。
类似地，在采样点的有向欧氏距离小于第二距离值(如，0)时，可以认为该采样点属于前景区域，若干采样点实际属于背景区域，则可以将其过滤，即从上述采样点集合L中过滤，如图2中左上角搜索线段，位于该搜索线段最右侧两个采样点其有向欧氏距离均小于第二距离值(如，0)，故可以认为这两个采样点属于前景区域，而实际上这两个采样点属于背景区域，则可以将这两个采样点过滤。
步骤S14：基于目标函数，得到拍摄图像中目标物体的位姿参数。
在一个实施场景中，如前所述，投影轮廓是利用目标物体的参考位姿投影得到的，参考位姿是参考图像中目标物体的位姿参数，且参考图像是在拍摄图像之前拍摄得到的，则可以对目标函数进行求解，得到参考位姿的更新参数，并利用更新参数对参考位姿进行优化，得到拍摄图像中目标物体的位姿参数。
例如，参考图像可以是拍摄图像之前一帧图像，如参考图像可以为视频数据中第t-1帧图像，而拍摄图像可以为视频数据中第t帧图像，具体可以参阅前述相关描述，在此不再赘述。
在一个具体的实施场景中，为了便于使用非线性算法最小化目标函数，可以将目标函数改写为非线性加权最小二乘问题的标准形式：
Figure PCTCN2021127307-appb-000016
F(x ij,p)＝-log[He(d(x ij))P f(x ij)+(1-He(d(x ij)))P f(x ij)]……(9)
上述公式(8)中，ψ(x ij)＝1/F(x ij,p)。在此基础上，可以通过高斯牛顿算法迭代解决优化问题，将雅可比向量定义为：
Figure PCTCN2021127307-appb-000017
Figure PCTCN2021127307-appb-000018
Figure PCTCN2021127307-appb-000019
上述公式(12)中，
Figure PCTCN2021127307-appb-000020
表示平滑的狄拉克δ函数，可以由第一可信度He(d(x ij))引出，具体可以参阅前述公式(4)。此外，
Figure PCTCN2021127307-appb-000021
可以由前述公式(2)推导，具体推导过程可以参阅高斯牛顿算法的相关细节，在此不再赘述。基于上述雅可比向量以及高斯牛顿算法，可以推导得到更新参数△p：
Figure PCTCN2021127307-appb-000022
在另一个具体的实施场景中，上述更新参数△p采用李代数表达，为了便于优化参考位姿，可以将△p转换为欧式变换矩阵△T，具体转换过程可以参阅李群、李代数相关技术细节，在此不再赘述。在此基础上，拍摄图像中目标物体的位姿参数T′可以表示为：
T′＝△T·T……(14)
在一个实施场景中，在存在多个目标物体的情况下，若多个目标物体彼此之间互不遮挡，则各个目标物体在拍摄图像中的位姿参数可以利用本公开实施例中步骤分别获取得到。反之，若多个目标物体彼此之间存在遮挡，则可以使用前述掩码图像I s和深度图像I d来过滤处于被干扰状态(如，被遮挡)的采样点。
在一个具体的实施场景中，深度图像I d可以是对拍摄图像进行渲染得到的，具体渲染过程在此不再赘述。深度图像I d具体可以包括拍摄图像中各个像素点的深度值。
在另一个具体的实施场景中，在获取第k个目标物体在拍摄图像中的位姿参数过程中，可以先搜索位于第k个目标物体对应的背景区域
Figure PCTCN2021127307-appb-000023
内且与所在搜索线段的投影轮廓点m i毗邻的采样点x ij，并校验I s(x ij)是否等于另一目标物体的索引，且采样点x ij的深度值I d(x ij)小于所在搜索线段的投影轮廓点m i的深度值I d(m i)，若是，则可以认为该采样点x ij所在搜索线段l i处于被干扰状态(如，被遮挡)，则可以过滤这条搜索线段l i上所有采样点。
请结合参阅图4a至图4c，其中，图4a是拍摄图像一实施例的示意图，图4b是掩码图像另一实施例的示意图，图4c是搜索线段一实施例的示意图。
如图4a至4c所示，拍摄图像中存在鸭子和松鼠两个目标物体，且鸭子被松鼠遮挡，故在获取鸭子在拍摄图像中的位姿参数过程中，可以过滤遮挡的投影轮廓附近的搜索线段，以尽可能地环节诸如布局遮挡等干扰因素对获取位姿参数的负面影响，有利于提高位姿参数的精度。
在一个实施场景中，前述局部颜色直方图是基于围绕物体轮廓的局部区域而构造得到的，与此同时，为了增强时间连续性，每个局部区域对应于一个模型顶点。然而，目标物体的三维模型所包含的顶点较少(如，少于50个)的情况下，这些局部区域并不能完全覆盖物体轮廓，从而影响上述第一概率值P f和第二概率值P b的准确性。
有鉴于此，在目标物体的三维模型所包含的顶点少于预设阈值(如，50)的情况下，可以在三维模型的每条边增加若干(如，4个)顶点，以提升局部区域的数量，从而尽可能地使得局部区域能够覆盖物体轮廓。
请结合参阅图5a至图5c，其中，图5a是拍摄图像另一实施例的示意图，图5b是布局区域一实施例的示意图，图5c是局部区域另一实施例的示意图。
如图5a至图5c所示，在顶点(如图中实心圆所示)数量较少(如图5b中仅有4个)的情况下，局部区域(如图中空心圆所示)并不完全覆盖目标物体的物体轮廓，在此情况下，通过在每条边上增加顶点(如，增加至8个)，局部区域可以完全覆盖物体轮廓。
在一个实施场景中，拍摄图像中目标物体的位姿参数可以是通过多次迭代(如，7次迭代)得到的。具体地，在第一次迭代过程中，可以将参考图像中目标物体的位姿参数作为参考位置，并执行本公开实施例中步骤，得到第一次次迭代时拍摄图像中目标物体的位姿参数，并将其作为第二次迭代的参考位姿，并重新执行本公开实施例中步骤，得到第二次迭代时拍摄图像中目标物体的位姿参数。
以此类推，在第i次迭代时，将第i-1次迭代所得到的位姿参数作为参考位姿，并执行本公开实施例中步骤，得到第i次迭代时拍摄图像中目标物体的位姿参数，直至最后一次迭代，可以直接将最后一次迭代时拍摄图像中目标物体的位姿参数，作为拍摄图像中目标物体最终的位姿参数。
上述方案，获取拍摄图像中位于搜索线段上的若干采样点，且搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓，基于此再获取采样点的属性信息，以及获取采样点的参考权重，且属性信息表示采样点属于目标物体的可能性，从而基于采样点的属性信息和参考权重，构建目标函数，并基于目标函数，得到拍摄图像中目标物体的位姿参数。
应理解，由于目标函数是基于采样点的属性信息和参考权重两者而构建的，故一方面得益于属性 信息能够参考采样点属于目标物体的可能性，另一方面得益于参考权重能够参考采样点在后续求解位姿参数过程中的参考价值，进而能够有利于尽可能地缓解干扰因素对于位姿求解的影响，有利于提高位姿参数的精度。
请参阅图6，图6是图1中步骤S12一实施例的流程示意图。具体而言，可以包括如下步骤：
步骤S61：在搜索线段上的若干采样点中搜索目标点，得到搜索结果。
本公开实施例中，目标点用于表示目标物体的物体轮廓点。请结合参阅图2，仍以搜索线段l i为例，搜索线段l i上采样点s i可以用于表示物体轮廓点，其他情况可以以此类推，在此不再一一举例。
在一个实施场景中，对于每条搜索线段，可以将该搜索线段上若干采样点分别作为当前点，在当前点的参考概率差值满足第一条件的情况下，可以将当前点作为候选点，并选取预测代价值满足第二条件的候选点作为目标点。上述方式，可以先基于参考概率差值进行粗选，再基于预测代价值进行细选，有利于提高筛选目标点的效率和准确性。
在一个具体的实施场景中，当前点的参考概率差值可以为与当前点具有预设位置关系(如，与当前点相邻)的两个采样点的第一概率值之差。为了提高访问不同采样点的第一概率值的便利性，如图7a所示，可以将图3中所有搜索线段按行堆叠，构造得到搜索线段的集束图像I b，以及如图7b所示，将图3中所有搜索线段上各个采样点的第一概率值也按行堆叠，构建得到关于第一概率值的集束图像I p。
如前述公开实施例所述，搜索线段包含2*N+1个采样点，其中中间位置的采样点为投影轮廓点，投影轮廓点一侧对应于前景区域，投影轮廓点的另一侧对应于背景区域，故集束图像I b中间一列对应投影轮廓，中间一列的一侧对应前景区域，中间一列的另一侧对应背景区域。
在另一个具体的实施场景中，第一条件可以包括参考概率差值大于预设阈值，则在构造得到上述集束图像I p之后，可以利用预设卷积核(如，f＝[-1 0 1])在集束图像I p每行分别进行滑动卷积，并将卷积响应高于预设阈值ε(如，0.3)的采样点作为候选点。
在又一个具体的实施场景中，上述候选点有可能存在三种情况，即候选点可能为物体轮廓点，候选点也可能为前景干扰点，候选点还可能为背景干扰点。为了提高分类准确性，对于搜索线段l i上第j个采样点x ij而言，可以在搜索线段l i指向背景区域的一段选取若干连续采样点，构成第一采样点集
Figure PCTCN2021127307-appb-000024
(如，可以包括x i,j-1,x i,j-2,x i,j-3)，并在搜索线段l i指向前景区域的一段选取若干连续采样点，构成第二采样点集
Figure PCTCN2021127307-appb-000025
(如，可以包括x i,j+1,x i,j+2,x i,j+3)。
故此，在采样点为物体轮廓点的情况下，第一采样点集
Figure PCTCN2021127307-appb-000026
理论上应属于背景区域，而第二采样点集
Figure PCTCN2021127307-appb-000027
理论上应属于前景区域，故采样点作为物体轮廓点的概率值P(h ij|C)可以表示为：
Figure PCTCN2021127307-appb-000028
应理解，在h ij为候选点的情况下，P(h ij|C)即为候选点作为物体轮廓点的预测概率值；而如前述公开实施例所述，在候选点h ij可作为目标点s i的情况下，P(h ij|C)可以写作P(s i|C)，即可以作为目标点可作为物体轮廓点的预测概率值。
类似地，在采样点为前景干扰点的情况下，第一采样点集
Figure PCTCN2021127307-appb-000029
和第二采样点集
Figure PCTCN2021127307-appb-000030
理论上应均属于前景区域，故采样点作为前景干扰点的概率值P(h ij|F)可以表示为：
Figure PCTCN2021127307-appb-000031
应理解，在h ij为候选点的情况下，P(h ij|F)即为候选点作为前景干扰点的预测概率值。
类似地，在采样点为背景干扰点的情况下，一采样点集
Figure PCTCN2021127307-appb-000032
和第二采样点集
Figure PCTCN2021127307-appb-000033
理论上应均属于背景区域，故采样点作为背景干扰点的概率值P(h ij|B)可以表示为：
Figure PCTCN2021127307-appb-000034
应理解，在h ij为候选点的情况下，P(h ij|B)即为候选点作为背景干扰点的预测概率值。
在此基础上，可以进一步定义采样点作为物体轮廓点的归一化概率值P c(h ij)：
Figure PCTCN2021127307-appb-000035
应理解，在h ij为候选点的情况下，P c(h ij)即为候选点作为物体轮廓点的归一化概率值。
在又一个具体的实施场景中，在获取候选点作为物体轮廓点的预测概率值P(h ij|C)或者在获取候选点作为物体轮廓点的归一化概率值P c(h ij)之后，可以进一步过滤预测概率值P(h ij|C)满足第三条件的候选点。例如，可以过滤预测概率值P(h ij|C)小于前述概率值P(h ij|F)和概率值P(h ij|B)两者中最大值的候选点，即对于候选点h ij，若满足P(h ij|C)<max(P(h ij|B),P(h ij|F))，则可以将该候选点h ij过滤；或者，可以如前所述，基于预测概率值P(h ij|C)得到归一化概率值P c(h ij)，并过滤归一化概率值P c(h ij)小于预设阈值(如，0.5)的候选点，在此不做限定。
在又一个具体的实施场景中，上述预测代价值可以包括第一代价值和第二代价值中至少一者，如可以同时包括第一代价值和第二代价值，也可以仅包括第一代价值，或者仅包括第二代价值。第一代价值可以与候选点的预测概率值相关，如第一代价值可以与候选点的预测概率值负相关，为了便于描述，第一代价值可以记为E d(h ij)，则第一代价值E d(h ij)可以表示为：
Figure PCTCN2021127307-appb-000036
如公式(19)所示，候选点h ij的预测概率值P(h ij|C)越大，其作为目标点的第一代价值E d(h ij)越小。
此外，第二代价值与候选点至搜索线段上的投影轮廓点的第二距离相关，如第二代价值可以与上述第二距离正相关，为了便于描述，第二代价值可以记为E S(h ij)，则第二代价值E S(h ij)可以表示为：
E S(h ij)＝||h ij-m i|| 2……(20)
如公式(20)所示，候选点h ij至搜索线段l i上的投影轮廓点m i的第二距离||h ij-m i||越大，其作为目标点的第二代价值E S(h ij)越大。
应理解，在预测代价值同时包含第一代价值和第二代价值的情况下，可以将第一代价值和第二代价值进行加权处理，作为预测代价值E(h ij)：
E(h ij)＝E d(h ij)+λE s(h ij,m i)……(21)
上述公式(21)中，λ表示加权因子，具体可以根据实际应用需要进行设置，如可以设置为0.015，在此不做限定。上述第二条件具体可以包括预测代价值最小，即在目标物体或相机的帧间运动较为平和的情况下，第二代价值会对离投影轮廓较远的候选点施加额外惩罚，以优先选择离投影轮廓较近的候选点作为目标点。
应理解，在上述筛选过程中，搜索线段l i上可能搜索不到目标点，则此时可以对于该搜索线段l i，可以标记
Figure PCTCN2021127307-appb-000037
以表示对于搜索线段l i而言，其搜索结果包括未搜索到目标点。
步骤S62：基于搜索结果，分别获取搜索线段上的若干采样点的权重信息。
本公开实施例中，权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，且第二权重与目标点至采样点的第一距离相关。具体可以参阅前述公开实施例中相关描述，在此不再赘述。
步骤S63：基于权重信息，得到采样点的参考权重。
具体可以参阅前述公开实施例中相关描述，在此不再赘述。
上述方案，在搜索线段上的若干采样点中搜索目标点，得到搜索结果，且目标点用于表示目标物体的物体轮廓点，并基于搜索结果，分别获取搜索线段上的若干采样点的权重信息，且权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，而第二权重与目标点至采样点的第一距离相关，故第一权重和第二权重能够从不同角度表征采样点的参考价值，进而基于此再基于权重信息，得到采样点的参考权重，能够提高参考权重在后续求解位姿参数过程中的参考价值。
请参阅图8，图8是本申请位姿获取方法另一实施例的流程示意图。具体可以包括如下步骤：
步骤S801：对拍摄图像进行降采样，得到若干种分辨率的金字塔图像。
例如，可以将2作为降采样倍率，以对拍摄图像进行降采样处理，得到1/4分辨率的金字塔图像、1/2分辨率的金字塔图像以及原始分辨率的金字塔图像(即拍摄图像本身)。其他情况可以以此类推，在此不再一一举例。
步骤S802：按照分辨率由小到大，依次选择金字塔图像作为当前拍摄图像。
例如，可以先选择1/4分辨率的金字塔图像作为当前拍摄图像，并执行下述步骤以得到1/4分辨率的金字塔图像中目标物体的位姿参数，自此之后再选择1/2分辨率的金字塔图像作为当前拍摄图像，如此循环，在此不再一一举例。
步骤S803：获取当前拍摄图像中位于搜索线段上的若干采样点。
本公开实施例中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓，且投影轮廓是利用目标物体的参考位姿投影得到的。具体可以参阅前述公开实施例中相关描述，在此不再赘述。
步骤S804：获取采样点的属性信息，以及获取采样点的参考权重。
本公开实施例中，属性信息表示采样点属于目标物体的可能性。具体可以参阅前述公开实施例中相关描述，在此不再赘述。
应理解，属性信息包括：采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度，且第一可信度的计算过程可以参阅前述公开实施例中公式(4)及相关描述，且公式(4)中平滑因子s与负相关。例如，对于1/4分辨率的金字塔图像而言，平滑因子s可以设置为1.2，而对于1/2分辨率的金字塔图像而言，平滑因子s可以设置为0.8，对于原始分辨率的金字塔图像(即拍摄图像本身)而言，平滑因子s可以设置为0.6，在此不做限定。
步骤S805：基于采样点的属性信息和参考权重，构建目标函数。
具体可以参阅前述公开实施例中相关描述，在此不再赘述。
步骤S806：基于目标函数，得到当前拍摄图像中目标物体的位姿参数。
具体可以参阅前述公开实施例中相关描述，在此不再赘述。
应理解，如前述公开实施例所述，对于拍摄图像而言可以通过多次迭代，得到拍摄图像中目标物体的位姿参数。类似地，对于金字塔图像而言，可以经过若干次迭代，得到金字塔图像中目标物体的位姿参数，且金字塔图像分辨率越低，迭代次数越多。例如，对于1/4分辨率的金字塔图像而言，可以迭代4次，对于1/2分辨率的金字塔图像而言，可以迭代2次，对于原始分辨率的金字塔图像(即拍摄图像本身)而言，可以迭代1次。具体的迭代过程，可以参阅前述公开实施例中相关描述，在此不再赘述。
步骤S807：判断当前拍摄图像是否为最后一帧金字塔图像，若否，则执行步骤S808，否则执行步骤S810。
在当前拍摄图像为最后一帧金字塔图像的情况下，即可将最后一帧金字塔图像中目标物体的位姿参数，作为拍摄图像中目标物体最终的位姿参数，否则可以继续进行迭代过程。
步骤S808：将本次执行得到的位姿参数作为参考位姿。
在当前拍摄图像不为最后一帧金字塔图像的情况下，可以将本次执行得到的位姿参数作为参考位姿，并对下一帧金字塔图像进行执行迭代操作。
步骤S809：重新执行步骤S802以及后续步骤。
即在当前拍摄图像不为最后一帧金字塔图像的情况下，对下一帧金字塔图像进行执行迭代操作。
步骤S810：将本次执行得到的位姿参数作为拍摄图像中目标物体最终的位姿参数。
在当前拍摄图像为最后一帧金字塔图像的情况下，可以结束迭代操作，得到拍摄图像中目标物体最终的位姿参数。
上述方案，投影轮廓是利用目标物体的参考位姿投影得到的，从而在投影采样之前，先对拍摄图像进行降采样，以得到若干种分辨率的金字塔图像，并按照分辨率由小到大，依次选择金字塔图像作为当前拍摄图像，以及对当前拍摄图像执行上述获取拍摄图像中位于搜索线段上的若干采样点的步骤以及后续步骤，且本次执行所采用的参考位姿为上一次执行得到的位姿参数，最后一次执行得到的位姿参数作为拍摄图像中目标物体最终的位姿参数，从而能够在位姿参数的获取过程中，由粗到细地进行位姿估计，进而能够有利于提高位姿参数的获取效率和精度。
请参阅图9，图9是本申请位姿获取装置90一实施例的框架示意图。位姿获取装置90应用于电子设备中，包括：投影采样模块91、信息提取模块92、函数构建模块93和位姿求解模块94，投影采样模块91配置为获取拍摄图像中位于搜索线段上的若干采样点；其中，搜索线段经过拍摄图像中目标物体的投影轮廓点，投影轮廓点位于目标物体的投影轮廓；信息提取模块92配置为获取采样点的属性信息，以及获取采样点的参考权重；其中，属性信息表示采样点属于目标物体的可能性；函数构建模块93配置为基于采样点的属性信息和参考权重，构建目标函数；位姿求解模块94配置为基于目标函数，得到拍摄图像中目标物体的位姿参数。
在一些公开实施例中，信息提取模块92包括目标点搜索子模块，配置为在搜索线段上的若干采样点中搜索目标点，得到搜索结果；其中，目标点配置为表示目标物体的物体轮廓点；信息提取模块92包括权重信息获取子模块，配置为基于搜索结果，分别获取搜索线段上的若干采样点的权重信息；其中，权重信息包括第一权重和第二权重中至少一者，第一权重与目标点的预测概率值相关，预测概率值表示采样点作为物体轮廓点的可能性，且第二权重与目标点至采样点的第一距离相关；信息提取模块92包括参考权重获取子模块，配置为基于权重信息，得到采样点的参考权重。
在一些公开实施例中，属性信息包括：采样点属于目标物体的第一概率值；目标点搜索子模块包括当前点获取单元，配置为对于每条搜索线段，将若干采样点分别作为当前点，目标点搜索子模块包括候选点获取单元，配置为在当前点的参考概率差值满足第一条件的情况下，将当前点作为候选点，目标点搜索子模块包括目标点获取单元，配置为选取预测代价值满足第二条件的候选点作为目标点；其中，当前点的参考概率差值为与当前点具有预设位置关系的两个采样点的第一概率值之差，预测代价值包括第一代价值和第二代价值中至少一者，第一代价值与候选点的预测概率值相关，第二代价值与候选点至搜索线段上的投影轮廓点的第二距离相关。
在一些公开实施例中，目标点搜索子模块包括候选点过滤单元，配置为过滤预测概率值满足第三条件的候选点。
在一些公开实施例中，预设位置关系为与当前点相邻；和/或，第二条件包括预测代价值最小；和/或，第一代价值与候选点的预测概率值负相关，且第二代价值与第二距离正相关。
在一些公开实施例中，权重信息包括第一权重；权重信息获取子模块包括第一确定单元，配置为在搜索结果包括搜索到目标点的情况下，基于目标点的预测概率值确定采样点的第一权重，其中，第一权重与目标点的预测概率值正相关；权重信息获取子模块包括第二确定单元，配置为在搜索结果包括未搜索到目标点的情况下，将第一权重确定为第一数值；其中，第一数值为在搜索结果包括搜索到目标点的情况下第一权重的下限值。
在一些公开实施例中，权重信息包括第二权重；权重信息获取子模块包括第三确定单元，配置为在搜索结果包括搜索到目标点的情况下，基于采样点对应的第一距离确定采样点的第二权重，其中，第二权重与第一距离负相关；权重信息获取子模块包括第四确定单元，配置为在搜索结果包括未搜索到目标点的情况下，将第二权重确定为第二数值；其中，第二数值为在搜索结果包括搜索到目标点的情况下第二权重的上限值。
在一些公开实施例中，权重信息包括第一权重和第二权重，且第一权重、第二权重均与参考权重正相关。
在一些公开实施例中，属性信息包括：采样点属于目标物体的第一概率值和第一可信度，以及采样点不属于目标物体的第二概率值和第二可信度；函数构建模块93包括联合概率计算子模块，配置为获取第一可信度与第一概率值的第一积以及第二可信度与第二概率值的第二积，并基于第一积和第二积之和，得到采样点的联合概率值；函数构建模块93包括联合概率加权子模块，配置为基于各个采样点的参考权重对联合概率值的加权结果，得到目标函数。
在一些公开实施例中，第一可信度与第二可信度为负相关关系，采样点的第一可信度与相应投影轮廓点至采样点的有向欧氏距离为负相关关系，相应投影轮廓点与采样点位于相同搜索线段上。
在一些公开实施例中，拍摄图像包括基于投影轮廓划分的前景区域和背景区域；函数构建模块93包括第一过滤子模块，配置为在采样点的有向欧氏距离大于第一距离值，且采样点属于前景区域 的情况下，过滤采样点；函数构建模块93包括第二过滤子模块，配置为在采样点的有向欧氏距离小于第二距离值，且采样点属于背景区域的情况下，过滤采样点。
在一些公开实施例中，投影轮廓是利用目标物体的参考位姿投影得到的；位姿获取装置90包括降采样模块，配置为对拍摄图像进行降采样，得到若干种分辨率的金字塔图像；位姿获取装置90包括图像选择模块，配置为按照分辨率由小到大，依次选择金字塔图像作为当前拍摄图像，投影采样模块91、信息提取模块92、函数构建模块93和位姿求解模块94配置为对当前拍摄图像执行获取拍摄图像中位于搜索线段上的若干采样点的步骤以及后续步骤；其中，本次执行所采用的参考位姿为上一次执行得到的位姿参数，最后一次执行得到的位姿参数作为拍摄图像中目标物体最终的位姿参数。
在一些公开实施例中，投影轮廓是利用目标物体的参考位姿投影得到的，参考位姿是参考图像中目标物体的位姿参数，且参考图像是在拍摄图像之前拍摄得到的；位姿求解模块94包括函数求解子模块，配置为对目标函数进行求解，得到参考位姿的更新参数；位姿求解模块94包括位姿优化子模块，配置为利用更新参数对参考位姿进行优化，得到位姿参数。
请参阅图10，图10是本申请电子设备100一实施例的框架示意图。电子设备100包括相互耦接的存储器101和处理器102，处理器102用于执行存储器101中存储的程序指令，以实现上述任一位姿获取方法实施例的步骤。在一个具体的实施场景中，电子设备100可以包括但不限于：微型计算机、服务器，此外，电子设备100还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。
具体而言，处理器102用于控制其自身以及存储器101以实现上述任一位姿获取方法实施例的步骤。处理器102还可以称为CPU(Central Processing Unit，中央处理单元)。处理器102可能是一种集成电路芯片，具有信号的处理能力。处理器102还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器102可以由集成电路芯片共同实现。
请参阅图11，图11为本申请计算机可读存储介质110一实施例的框架示意图。计算机可读存储介质110存储有能够被处理器运行的程序指令111，程序指令111用于实现上述任一位姿获取方法实施例的步骤。
上述方案，能够有利于尽可能地缓解干扰因素对于位姿求解的影响，有利于提高位姿参数的精度。
本公开实施例提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一种位姿获取方法。
在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
工业实用性
本申请实施例公开了一种位姿获取方法及装置、电子设备、存储介质和程序，其中，位姿获取方法包括：获取拍摄图像中位于搜索线段上的若干采样点；其中，搜索线段经过拍摄图像中目标物体的 投影轮廓点，投影轮廓点位于目标物体的投影轮廓；获取采样点的属性信息，以及获取采样点的参考权重；其中，属性信息表示采样点属于目标物体的可能性；基于采样点的属性信息和参考权重，构建目标函数；基于目标函数，得到拍摄图像中目标物体的位姿参数。上述方案，能够提高位姿参数的精度。

Claims (17)

  1. 一种位姿获取方法，应用于电子设备中，包括：
    获取拍摄图像中位于搜索线段上的若干采样点；其中，所述搜索线段经过所述拍摄图像中目标物体的投影轮廓点，所述投影轮廓点位于所述目标物体的投影轮廓；
    获取所述采样点的属性信息，以及获取所述采样点的参考权重；其中，所述属性信息表示所述采样点属于所述目标物体的可能性；
    基于所述采样点的所述属性信息和所述参考权重，构建目标函数；
    基于所述目标函数，得到所述拍摄图像中所述目标物体的位姿参数。
  2. 根据权利要求1所述的方法，其中，所述获取所述采样点的参考权重，包括：
    在所述搜索线段上的所述若干采样点中搜索目标点，得到搜索结果；其中，所述目标点用于表示所述目标物体的物体轮廓点；
    基于所述搜索结果，分别获取所述搜索线段上的所述若干采样点的权重信息；其中，所述权重信息包括第一权重和第二权重中至少一者，所述第一权重与所述目标点的预测概率值相关，所述预测概率值表示所述采样点作为所述物体轮廓点的可能性，且所述第二权重与所述目标点至所述采样点的第一距离相关；
    基于所述权重信息，得到所述采样点的参考权重。
  3. 根据权利要求2所述的方法，其中，所述属性信息包括：所述采样点属于所述目标物体的第一概率值；所述在所述搜索线段上的所述若干采样点中搜索目标点，得到搜索结果，包括：
    对于每条所述搜索线段，将所述若干采样点分别作为当前点，在所述当前点的参考概率差值满足第一条件的情况下，将所述当前点作为候选点，并选取预测代价值满足第二条件的候选点作为所述目标点；
    其中，所述当前点的参考概率差值为与所述当前点具有预设位置关系的两个所述采样点的所述第一概率值之差，所述预测代价值包括第一代价值和第二代价值中至少一者，所述第一代价值与所述候选点的预测概率值相关，所述第二代价值与所述候选点至所述搜索线段上的投影轮廓点的第二距离相关。
  4. 根据权利要求3所述的方法，其中，在所述选取预测代价值满足第二条件的候选点作为所述目标点之前，所述方法还包括：
    过滤所述预测概率值满足第三条件的候选点。
  5. 根据权利要求3所述的方法，其中，所述预设位置关系为与所述当前点相邻；
    和/或，所述第二条件包括所述预测代价值最小；
    和/或，所述第一代价值与所述候选点的预测概率值负相关，且所述第二代价值与所述第二距离正相关。
  6. 根据权利要求2所述的方法，其中，所述权重信息包括所述第一权重；所述基于所述搜索结果，分别获取所述搜索线段上的所述若干采样点的权重信息，包括：
    在所述搜索结果包括搜索到所述目标点的情况下，基于所述目标点的预测概率值确定所述采样点的第一权重，其中，所述第一权重与所述目标点的预测概率值正相关；
    和/或，在所述搜索结果包括未搜索到所述目标点的情况下，将所述第一权重确定为第一数值；其中，所述第一数值为在所述搜索结果包括搜索到所述目标点的情况下所述第一权重的下限值。
  7. 根据权利要求2所述的方法，其中，所述权重信息包括所述第二权重；所述基于所述搜索结果，分别获取所述搜索线段上的所述若干采样点的权重信息，包括：
    在所述搜索结果包括搜索到所述目标点的情况下，基于所述采样点对应的所述第一距离确定所述采样点的第二权重，其中，所述第二权重与所述第一距离负相关；
    和/或，在所述搜索结果包括未搜索到所述目标点的情况下，将所述第二权重确定为第二数值；其中，所述第二数值为在所述搜索结果包括搜索到所述目标点的情况下所述第二权重的上限值。
  8. 根据权利要求2至7任一项所述的方法，其中，所述权重信息包括第一权重和第二权重，且所述第一权重、所述第二权重均与所述参考权重正相关。
  9. 根据权利要求1所述的方法，其中，所述属性信息包括：所述采样点属于所述目标物体的第一概率值和第一可信度，以及所述采样点不属于所述目标物体的第二概率值和第二可信度；所述基于所述采样点的所述属性信息和所述参考权重，构建目标函数，包括：
    获取所述第一可信度与所述第一概率值的第一积以及所述第二可信度与所述第二概率值的第二积，并基于所述第一积和第二积之和，得到所述采样点的联合概率值；
    基于各个所述采样点的参考权重对所述联合概率值的加权结果，得到目标函数。
  10. 根据权利要求9所述的方法，其中，所述第一可信度与所述第二可信度为负相关关系，所述采样点的第一可信度与相应所述投影轮廓点至所述采样点的有向欧氏距离为负相关关系，相应所述投影轮廓点与所述采样点位于相同所述搜索线段上。
  11. 根据权利要求10所述的方法，其中，所述拍摄图像包括基于所述投影轮廓划分的前景区域和背景区域；在所述获取所述第一可信度与所述第一概率值的第一积以及所述第二可信度与所述第二概率值的第二积之前，所述方法还包括：
    在所述采样点的所述有向欧氏距离大于第一距离值，且所述采样点属于所述前景区域的情况下，过滤所述采样点；
    和/或，在所述采样点的所述有向欧氏距离小于第二距离值，且所述采样点属于所述背景区域的情况下，过滤所述采样点。
  12. 根据权利要求1至11任一项所述的方法，其中，所述投影轮廓是利用所述目标物体的参考位姿投影得到的；在所述获取拍摄图像中位于搜索线段上的若干采样点之前，所述方法包括：
    对所述拍摄图像进行降采样，得到若干种分辨率的金字塔图像；
    按照所述分辨率由小到大，依次选择所述金字塔图像作为当前所述拍摄图像，并对当前所述拍摄图像执行所述获取拍摄图像中位于搜索线段上的若干采样点的步骤以及后续步骤；其中，本次执行所采用的所述参考位姿为上一次执行得到的所述位姿参数，最后一次执行得到的所述位姿参数作为所述拍摄图像中所述目标物体最终的所述位姿参数。
  13. 根据权利要求1至12任一项所述的方法，其中，所述投影轮廓是利用所述目标物体的参考位姿投影得到的，所述参考位姿是参考图像中所述目标物体的位姿参数，且所述参考图像是在所述拍摄图像之前拍摄得到的；所述基于所述目标函数，得到所述拍摄图像中所述目标物体的位姿参数，包括：
    对所述目标函数进行求解，得到所述参考位姿的更新参数；
    利用所述更新参数对所述参考位姿进行优化，得到所述位姿参数。
  14. 一种位姿获取装置，应用于电子设备中，包括：
    投影采样模块，配置为获取拍摄图像中位于搜索线段上的若干采样点；其中，所述搜索线段经过所述拍摄图像中目标物体的投影轮廓点，所述投影轮廓点位于所述目标物体的投影轮廓；
    信息提取模块，配置为获取所述采样点的属性信息，以及获取所述采样点的参考权重；其中，所述属性信息表示所述采样点属于所述目标物体的可能性；
    函数构建模块，配置为基于所述采样点的所述属性信息和所述参考权重，构建目标函数；
    位姿求解模块，配置为基于所述目标函数，得到所述拍摄图像中所述目标物体的位姿参数。
  15. 一种电子设备，包括相互耦接的存储器和处理器，所述处理器用于执行所述存储器中存储的程序指令，以实现权利要求1至13任一项所述的位姿获取方法。
  16. 一种计算机可读存储介质，其上存储有程序指令，所述程序指令被处理器执行时实现权利要求1至13任一项所述的位姿获取方法。
  17. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1至13中任一所述的图像处理方法。
PCT/CN2021/127307 2021-06-04 2021-10-29 位姿获取方法及装置、电子设备、存储介质和程序 WO2022252487A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023525620A JP2023549069A (ja) 2021-06-04 2021-10-29 位置姿勢取得方法及びその装置、電子機器、記憶媒体並びにプログラム
KR1020237014463A KR20230073331A (ko) 2021-06-04 2021-10-29 포즈 획득 방법 및 장치, 전자 기기, 저장 매체 및 프로그램

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110626207.XA CN113407027B (zh) 2021-06-04 2021-06-04 位姿获取方法及装置和电子设备、存储介质
CN202110626207.X 2021-06-04

Publications (1)

Publication Number Publication Date
WO2022252487A1 true WO2022252487A1 (zh) 2022-12-08

Family

ID=77676442

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/127307 WO2022252487A1 (zh) 2021-06-04 2021-10-29 位姿获取方法及装置、电子设备、存储介质和程序

Country Status (4)

Country Link
JP (1) JP2023549069A (zh)
KR (1) KR20230073331A (zh)
CN (1) CN113407027B (zh)
WO (1) WO2022252487A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116542945A (zh) * 2023-05-11 2023-08-04 哈尔滨工业大学重庆研究院 晶圆图像分割处理方法、电子设备及存储介质
CN116684748B (zh) * 2023-08-01 2023-09-26 世优（北京）科技有限公司 摄影构图框的生成方法、装置及摄影设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113407027B (zh) * 2021-06-04 2023-03-03 浙江商汤科技开发有限公司 位姿获取方法及装置和电子设备、存储介质
CN114937125B (zh) * 2022-07-25 2022-10-25 深圳大学 可重建度量信息预测方法、装置、计算机设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704821A (zh) * 2017-09-29 2018-02-16 河北工业大学 一种弯道的车辆位姿计算方法
US20180060677A1 (en) * 2016-08-29 2018-03-01 Neusoft Corporation Method, apparatus and device for detecting lane lines
CN110706285A (zh) * 2019-10-08 2020-01-17 中国人民解放军陆军工程大学 基于cad模型的物体位姿预测方法
CN113407027A (zh) * 2021-06-04 2021-09-17 浙江商汤科技开发有限公司 位姿获取方法及装置和电子设备、存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104748746B (zh) * 2013-12-29 2017-11-03 刘进 智能机姿态测定及虚拟现实漫游方法
US10380763B2 (en) * 2016-11-16 2019-08-13 Seiko Epson Corporation Hybrid corner and edge-based tracking
CN109712172A (zh) * 2018-12-28 2019-05-03 哈尔滨工业大学 一种初始位姿测量结合目标追踪的位姿测量方法
CN110689573B (zh) * 2019-09-06 2022-07-01 重庆邮电大学 一种基于边缘模型的增强现实无标识跟踪注册方法及装置
CN111369571B (zh) * 2020-02-27 2023-06-23 北京百度网讯科技有限公司 三维物体位姿准确性判断方法、装置及电子设备
CN112614161A (zh) * 2020-12-28 2021-04-06 之江实验室 一种基于边缘置信度的三维物体跟踪方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180060677A1 (en) * 2016-08-29 2018-03-01 Neusoft Corporation Method, apparatus and device for detecting lane lines
CN107704821A (zh) * 2017-09-29 2018-02-16 河北工业大学 一种弯道的车辆位姿计算方法
CN110706285A (zh) * 2019-10-08 2020-01-17 中国人民解放军陆军工程大学 基于cad模型的物体位姿预测方法
CN113407027A (zh) * 2021-06-04 2021-09-17 浙江商汤科技开发有限公司 位姿获取方法及装置和电子设备、存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116542945A (zh) * 2023-05-11 2023-08-04 哈尔滨工业大学重庆研究院 晶圆图像分割处理方法、电子设备及存储介质
CN116542945B (zh) * 2023-05-11 2024-01-05 哈尔滨工业大学重庆研究院 晶圆图像分割处理方法、电子设备及存储介质
CN116684748B (zh) * 2023-08-01 2023-09-26 世优（北京）科技有限公司 摄影构图框的生成方法、装置及摄影设备

Also Published As

Publication number Publication date
KR20230073331A (ko) 2023-05-25
JP2023549069A (ja) 2023-11-22
CN113407027B (zh) 2023-03-03
CN113407027A (zh) 2021-09-17

Similar Documents

Publication Publication Date Title
WO2022252487A1 (zh) 位姿获取方法及装置、电子设备、存储介质和程序
US20210166426A1 (en) Mapping object instances using video data
Baak et al. A data-driven approach for real-time full body pose reconstruction from a depth camera
KR20190038808A (ko) 비디오 데이터의 객체 검출
US8363902B2 (en) Moving object detection method and moving object detection apparatus
JP2008518331A (ja) リアルタイムビデオ動き解析を通じたビデオコンテンツ理解
Wang et al. Robust edge-based 3D object tracking with direction-based pose validation
WO2019157922A1 (zh) 一种图像处理方法、装置及ar设备
WO2021098545A1 (zh) 一种姿势确定方法、装置、设备、存储介质、芯片及产品
CN116740126A (zh) 目标跟踪方法、高速相机及存储介质
JP2024508024A (ja) 画像データの処理方法及び装置
WO2022247126A1 (zh) 视觉定位方法、装置、设备、介质及程序
CN115035456A (zh) 视频去噪方法、装置、电子设备和可读存储介质
Seifi et al. How to improve CNN-based 6-DoF camera pose estimation
Chen et al. Robust visual tracking for planar objects using gradient orientation pyramid
Favorskaya et al. Digital video stabilization in static and dynamic scenes
CN112417991A (zh) 基于沙漏胶囊网络的双注意力人脸对齐方法
Martínez et al. HMPMR strategy for real-time tracking in aerial images, using direct methods
CN112184766B (zh) 一种对象的跟踪方法、装置、计算机设备和存储介质
KR20230083212A (ko) 객체 자세 추정 장치 및 방법
Favorskaya et al. Warping techniques in video stabilization
Zhang et al. An improved SLAM algorithm based on feature contour extraction for camera pose estimation
Kaveti et al. Towards robust VSLAM in dynamic environments: a light field approach
CN113570535A (zh) 视觉定位方法及相关装置、设备
Huang et al. Weighted large margin nearest center distance-based human depth recovery with limited bandwidth consumption

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943839

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023525620

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 20237014463

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE