WO2024060981A1 - 三维网格优化方法、设备及存储介质 - Google Patents

三维网格优化方法、设备及存储介质 Download PDF

Info

Publication number
WO2024060981A1
WO2024060981A1 PCT/CN2023/116808 CN2023116808W WO2024060981A1 WO 2024060981 A1 WO2024060981 A1 WO 2024060981A1 CN 2023116808 W CN2023116808 W CN 2023116808W WO 2024060981 A1 WO2024060981 A1 WO 2024060981A1
Authority
WO
WIPO (PCT)
Prior art keywords
error
candidate
initial
pixel
image
Prior art date
Application number
PCT/CN2023/116808
Other languages
English (en)
French (fr)
Inventor
王鹏
Original Assignee
深圳市其域创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市其域创新科技有限公司 filed Critical 深圳市其域创新科技有限公司
Publication of WO2024060981A1 publication Critical patent/WO2024060981A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing

Definitions

  • Embodiments of the present invention relate to the field of computer vision technology, and specifically relate to a three-dimensional grid optimization method, equipment and storage medium.
  • the traditional 3D reconstruction algorithm process roughly includes: structure recovery from motion, multi-view depth estimation, mesh construction. Since the initial point cloud contains a lot of noise, traditional mesh construction algorithms such as Poisson reconstruction, recovery from Visual Hull The mesh quality is generally poor, and some details are lost, resulting in poor accuracy of the 3D reconstruction model.
  • embodiments of the present invention provide a three-dimensional mesh optimization method, equipment and storage medium to solve the problems existing in the existing technology.
  • a three-dimensional mesh optimization method includes:
  • each of the candidate image pairs includes a first candidate image and a second candidate image from different viewing angles, and the initial surface has the first candidate image. and the second candidate image are jointly mapped on the common viewing area of the initial surface, and the initial surface has at least one mesh patch;
  • the mesh optimization error is calculated based on the target image pair.
  • selecting a target image pair from the plurality of sets of candidate image pairs according to the first condition further comprises:
  • the candidate image pair whose parallax quality error meets the quality error range is used as the target image pair.
  • selecting target image pairs among the plurality of candidate image pairs according to the first condition further includes:
  • the candidate image pair whose overlap degree error conforms to the overlap degree error range is used as the target image pair.
  • selecting target image pairs among the plurality of candidate image pairs according to the first condition further includes:
  • the candidate image pair whose symmetry error conforms to the symmetry error range is used as the target image pair.
  • selecting target image pairs among the plurality of candidate image pairs according to the first condition further includes:
  • selecting target image pairs among the plurality of candidate image pairs according to the first condition further includes:
  • the corresponding candidate image pair is used as the target image pair.
  • the first condition is further based on disparity quality error, overlap error, symmetry error and resolution error settings of the candidate image pairs, and the plurality of sets of candidate image pairs are selected according to the first condition
  • the target image pair in further includes:
  • the candidate image pair whose first error conforms to the first range value is used as the target image pair.
  • the first condition is further based on disparity quality error, overlap error, symmetry error and resolution error settings of the candidate image pairs, and the plurality of sets of candidate image pairs are selected according to the first condition
  • the target image pair in further includes:
  • the undetermined image pair whose undetermined error is less than or equal to the initial error multiplied by the preset coefficient is used as the target image pair.
  • the method further includes:
  • the mesh optimization error is calculated based on the target pixels.
  • selecting the target pixel of the image according to the second condition further includes:
  • each candidate area having a central pixel and a plurality of candidate pixels
  • the depth value difference is the difference between the pixel depth value of the candidate pixel and the pixel depth value of the center pixel
  • the second condition is set according to the depth value variance, and whether the candidate pixel meets the second condition is determined;
  • the candidate pixel that meets the second condition is used as the target pixel.
  • using the candidate pixel that meets the second condition as the target pixel further includes:
  • dd(h,k) represents the depth value difference corresponding to each candidate pixel in the first pixel set
  • max ⁇ dd(h,k) ⁇ represents the maximum depth value difference in the first pixel set.
  • a computing device including: a processor, a memory, a communication interface, and a communication bus.
  • the processor, the memory, and the communication interface complete each other through the communication bus. communication between;;
  • the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operation of the three-dimensional mesh optimization method described in any one of the above.
  • a computer-readable storage medium in which at least one executable instruction is stored, and the executable instruction executes any one of the above-mentioned instructions when running. Operation of three-dimensional mesh optimization methods.
  • the embodiment of the present invention selects a target image pair based on the first condition based on the area setting of the common viewing area, so that the selection of the image pair is based on the area setting of the common viewing area to measure the common characteristics of different images in the common viewing area, thereby selecting the image pair that meets the common viewing area.
  • the image pairs required in the common viewing area make the selected image pairs have higher quality, which is beneficial to reducing the redundancy of grid optimization error calculation, improving the solution speed of grid optimization error calculation, and can reduce regional discontinuity and two-dimensional
  • the error effects of each image due to their own reasons are conducive to more accurate calculation of grid optimization errors.
  • FIG1 is a schematic diagram showing a flow chart of a three-dimensional mesh optimization method provided by an embodiment of the present invention
  • Figure 2 shows a schematic structural diagram of a three-dimensional mesh optimization device provided by an embodiment of the present invention
  • FIG3 is a schematic diagram showing the structure of a computing device provided by some embodiments of the present invention.
  • Figure 4 shows a schematic diagram of the image mapping relationship when the fi area and the hi area are discontinuous provided by some embodiments of the present invention
  • Figure 5 shows a three-dimensional schematic diagram of the image mapping relationship when the fi area and the hi area are discontinuous provided by some embodiments of the present invention
  • Figure 6 shows a schematic diagram of two images mapped to an initial surface provided by some embodiments of the present invention.
  • Figure 7 shows a schematic diagram of the symmetrical relationship between two images provided by some embodiments of the present invention.
  • the processor first performs preliminary modeling based on multiple images to form an initial three-dimensional model.
  • the mesh quality of this initial three-dimensional model is poor.
  • the inventor found that the selection of image pairs has a significant impact on the final quality of mesh optimization. If the two cameras are very close, the reconstruction will be noisy. On the other hand, if the distance between the two cameras is too far, the overlap range will be small, which means that the common viewing range of the two cameras is too small. Correct image pair selection facilitates consistent photometric error calculations across image pairs, which in turn makes the gradient descent method faster.
  • image i and image j have a common view area mapped to the initial surface S, where the h area in the initial surface S , g area and f area are areas to which image i and image j are jointly mapped.
  • image j h j represents the area between mapping lines l 1 and l 2 when mapped to area h
  • gj represents the area located between mapping lines l 1 and l 2 when mapped to area g.
  • the area between rays l2 and l3, fj represents the area between mapping lines l3 and l4 when mapped to area f.
  • hi represents the area between the mapping lines l5 and l6 when mapped to the h area
  • fj represents the area between the mapping lines ll6 and l7 when mapped to the f area.
  • the area shown in the circle is not shown.
  • the gi area should be mapped to the g area. In this case, the gi area is occluded.
  • the fi area and hi area in image i are discontinuous.
  • the existing algorithm will consider the information of the f i region and the h i region at the same time. In this case, due to the discontinuity of the f i region and the h i region, the error calculation will be inaccurate, affecting Mesh quality.
  • the inventor provides a three-dimensional grid optimization method that selects a target image pair based on the first condition based on the area setting of the common viewing area, so that the selection of the image pair is based on the area setting of the common viewing area to measure the performance of different images in the common viewing area.
  • Common characteristics thereby selecting an appropriate image pair to reduce regional discontinuity and the error impact of the two images due to their own reasons, which is conducive to more accurate grid optimization error calculation.
  • FIG. 1 shows a flow chart of a three-dimensional mesh optimization method provided by an embodiment of the present invention.
  • the method is executed by a computing device.
  • the computing device may be a computing device including one or more processors.
  • the processor may be a central processing unit.
  • the processor CPU may be an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention, which are not limited here.
  • the one or more processors included in the computing device may be the same type of processor, such as one or more CPUs; or they may be different types of processors, such as one or more CPUs and one or more ASICs, where No restrictions. .
  • the method includes the following steps:
  • Step 110 Obtain multiple sets of candidate image pairs from multiple images and an initial surface of the three-dimensional model.
  • Each candidate image pair includes a first candidate image and a second candidate image from different perspectives, and the initial surface has a first candidate image and a second candidate image.
  • the candidate images are jointly mapped in the common viewing area of the initial surface, and the initial surface has at least one grid patch;
  • Step 120 Select a target image pair among the plurality of candidate image pairs according to a first condition, which is set based on the area of the common viewing area;
  • Step 130 Calculate the grid optimization error according to the target image pair.
  • the initial surface of the initial three-dimensional model has at least one mesh patch.
  • the initial surface can be formed by one mesh patch; in some cases, the initial surface can also be formed by multiple mesh patches. composition.
  • the mesh quality of the initial three-dimensional model is poor and requires mesh optimization, in which any two images among multiple images form an image pair, thus having multiple sets of image pairs. Among them, if two images in an image pair have a common view area mapped on the initial surface, then the image pair is used as a candidate image pair. Usually there are multiple groups of candidate image pairs, and each candidate image pair includes the first candidate from different perspectives. image and the second candidate image.
  • the first condition is set based on the area of the common view area, and target image pairs among multiple groups of candidate image pairs are selected according to the first condition to measure the common characteristics of different images in the common view area, thereby selecting an appropriate image pair, It is conducive to more accurate grid optimization error calculation.
  • the first condition can be set according to at least one of the image characteristics such as parallax quality error, overlap error, symmetry error, resolution error, coverage, etc. Or it can also be set according to other image characteristics related to the common viewing area, which is not limited here and can be set as needed.
  • the image characteristics such as parallax quality error, overlap error, symmetry error, resolution error, coverage, etc.
  • it can also be set according to other image characteristics related to the common viewing area, which is not limited here and can be set as needed.
  • step 130 after selecting a suitable target image pair, a grid optimization error calculation needs to be performed based on the relevant parameters of the target image pair to obtain an optimized three-dimensional grid.
  • the grid optimization error E E photo +E smooth , where E photo represents the photometric consistency error of the three-dimensional model relative to the image.
  • E smooth represents the smooth regular term of the initial surface.
  • E photo The discrete gradient calculated by E(s) for vertex X i can be expressed as Among them ,
  • the integral can be transformed from the initial surface S region to the image i by transforming the integration variable from x to xi .
  • the integration variable represents the gradient at x toward the outside of the surface
  • xi is the projection of x in image i
  • xj is the projection of The depth distance of the camera center c i .
  • the smooth regular term E smooth can be obtained through the Laplacian operator.
  • the target image pair is selected according to the first condition based on the area setting of the common viewing area, so that the selection of the image pair is based on the area setting of the common viewing area to measure the common characteristics of different images in the common viewing area, thereby Selecting image pairs that meet the requirements of the common view area makes the quality of the selected image pairs higher, which is beneficial to reducing the redundancy of grid optimization error calculation, improving the solution speed of grid optimization error calculation, and reducing regional inconsistencies. Continuous and error effects caused by two images due to their own reasons are conducive to more accurate calculation of grid optimization errors.
  • step 120 further includes:
  • Step a01 Determine the first camera center of the first candidate image and the second camera center of the second candidate image
  • Step a02 Calculate the average disparity from the first camera center and the second camera center to the common viewing area
  • Step a03 Calculate the parallax quality error based on the average parallax
  • Step a04 Determine whether the parallax quality error meets the quality error range
  • Step a05 If the parallax quality error meets the quality error range, use the candidate image pair whose parallax quality error meets the quality error range as the target image pair.
  • step a01 and step a02 the average disparity in step a02 in, Represents the area of the common view area, represents the common view area where the first candidate image i and the second candidate image j are jointly mapped on the initial surface S, x i represents any point located in the common view area, c i x i represents the first connection line between the first camera center c i and xi in the common view area, c i x i ,c j x i ) represents the angle between the first connection line and the second connection line.
  • parallax quality error In step a03, parallax quality error Among them, the angle with small parallax can make the image pairs have better overlap, while the large angle will make the grid optimization more stable. Therefore, the parallax quality error needs to balance the parallax angle to obtain better parallax quality error.
  • ⁇ p is the average reference parallax, usually set to 40°-60°;
  • ⁇ p is the preset angle variance, usually set to 40°-50°.
  • ⁇ p is preferably set to 50°, and ⁇ p is preferably set to 45° to better measure the parallax quality error.
  • the first condition is whether the parallax quality error meets the quality error range. If the processor determines that the parallax quality error conforms to the quality error range, it selects a candidate image pair that conforms to the quality error range as the target image pair, so that the grid optimization error can be calculated based on the relevant parameters of the target image pair. Among them, the quality error range is between 40° and 60°. The parallax quality error in this range is better. The target image obtained based on the parallax quality error has a smaller error on the calculated grid optimization, thereby optimizing the three-dimensional grid. advanced optimization.
  • steps a01 to a04 by calculating the parallax quality error, select the target image pair corresponding to the parallax quality error that meets the requirements, so as to achieve a better balance between the target image pair within the reasonable parallax and common view area, and optimize the subsequent calculation grid.
  • the error provides good parameters for the target image pair.
  • step 120 further includes:
  • Step b01 Calculate the overlap error based on the mapping of the first candidate image and the second candidate image in the common view area and the mapping of the first candidate image in the first area of the initial surface.
  • Step b02 Determine whether the overlap error meets the overlap error range
  • Step b03 If the overlap degree error conforms to the overlap degree error range, use the candidate image pair whose overlap degree error conforms to the overlap degree error range as the target image pair.
  • the overlap error Represents the area of the common view area, Indicates that the first candidate image i and the second candidate image j are jointly mapped in the common view area of the initial surface S, and A(Ii) indicates that the first candidate image i is mapped in the first area of the initial surface S.
  • the negative sign in means that the common viewing area between the two views is expected to be larger.
  • the first condition is whether the overlap degree error meets the overlap degree error range. If the processor determines that the overlap error conforms to the overlap error range, it selects a candidate image pair that conforms to the overlap error range as the target image pair, so that the grid optimization error can be calculated based on the relevant parameters of the target image pair.
  • the overlap error range can be set to 0.5 to 0.8, so that the overlap degree of the target image pair is more in line with reasonable expectations and provides good target image pair parameters for subsequent calculation of grid optimization errors.
  • the resolution error range can also be set to 0.5 to 0.9, or 0.6 to 0.8, or other range values, which are not limited here and can be set as needed.
  • the candidate image pairs that meet the overlap error range are determined as the target image pairs, ensuring that the overlap degree of the target image pairs meets reasonable expectations, so that the target image pairs are based on the overlap degree.
  • the error is selected to provide good target image pair parameters for subsequent calculation of grid optimization error.
  • step 120 further includes:
  • Step c01 determining a first camera center of a first candidate image and a second camera center of a second candidate image
  • Step c02 Calculate the average direction angle difference between the first camera center and the second camera center with respect to the normal vector of the common view area;
  • Step c03 Calculate the symmetry error based on the average direction angle difference
  • Step c04 Determine whether the symmetry error meets the symmetry error range
  • Step c05 If the symmetry error is within the symmetry error range, the candidate image pair whose symmetry error is within the symmetry error range is used as Target image pair.
  • the average direction angle difference in Represents the area of the common view area, represents the common view area where the first candidate image i and the second candidate image j are jointly mapped on the initial surface S, x represents any point located in the common view area, c i x represents the distance between the first camera center c i and x in the common view area
  • the first connection line, c j x represents the second connection line between the second camera center c j and x in the common view area
  • the initial surface S is observed by the reference camera c ref and the other two cameras c i and c j . If the initial surface S is completely observed by the three cameras, that is to say, the image The overlap is 100%, the baselines c ref -c i and c ref -c j have similar values, and the image pairs may be good or bad at the same time. Intuitively, the initial surface S along its normal direction changes while assuming that the image rendering is affected by Gaussian noise. Therefore, by calculating the symmetry error of candidate image pairs and judging the symmetry error range, similar noise will be transferred to significantly different along Different gradient noises.
  • the symmetry error ⁇ s is a preset symmetry angle variance, which is usually set to 40°-50°, and preferably set to 45°.
  • the first condition is whether the symmetry error meets the symmetry error range. If the processor determines that the symmetry error conforms to the symmetry error range, it selects a candidate image pair that conforms to the symmetry error range as the target image pair, so that the grid optimization error can be calculated based on the relevant parameters of the target image pair.
  • the average direction angle difference and the symmetry error are calculated to determine whether the symmetry error conforms to the symmetry error range, and then the candidate image pair whose symmetry error conforms to the symmetry error range is used as the target image pair, to Eliminate image centering camera
  • the noise effect caused by the symmetry degree is selected, so that the target image pair with a smaller symmetry error is selected, so that the grid optimization error calculation is further optimized.
  • the symmetry error range can be set to 0.6 ⁇ 0.9 to control the symmetry of the two cameras corresponding to the target image pair within the expected range and reduce the impact of noise.
  • the symmetry error range can also be set to 0.5 to 0.8, or 0.7 to 0.9, or other range values, which are not limited here and can be set as needed.
  • step 120 further includes:
  • Step d01 Determine the first camera center of the first candidate image and the second camera center of the second candidate image
  • Step d02 Calculate the average distance from the center of the first camera and the center of the second camera to the common view area;
  • Step d03 Calculate the resolution error based on the average distance
  • Step d04 Select a target image pair based on the resolution error.
  • the average distance in Represents the area of the common view area, Indicates the common view area where the first candidate image i and the second candidate image j are jointly mapped on the initial surface S, x i represents any point located in the common view area, f i represents the camera focal length corresponding to the first candidate image i, f j represents the camera focal length corresponding to the second candidate image j, c i x i represents the first camera center c i and the common The first connecting line distance of x i in the viewing area, c j x i represents the first connecting line between the first camera center c j and x i in the common viewing area.
  • step d03 in order to tend to select image pairs with similar resolutions and thereby produce more consistent optimization results, the resolution error needs to be calculated to evaluate the resolution consistency of the candidate image pairs based on the resolution error.
  • resolution error ⁇ r is the preset resolution variance, which is usually set to 0.2-0.4.
  • ⁇ r is set to 0.25, which means that the difference in resolution is 25%.
  • the first condition is whether the resolution error meets the resolution error range. If the processor determines that the resolution error conforms to the resolution error range, it selects a candidate image pair that conforms to the resolution error range as the target image pair, so that the grid optimization error can be calculated based on the relevant parameters of the target image pair.
  • the setting of the resolution error range can control the difference in resolution of the target image within the expected range, and the impact of resolution on subsequent grid optimization error calculations.
  • the resolution error range can be set to 0.6 to 1.0 to control the resolution difference of the target image within a reasonable expected range.
  • the resolution error range can also be set to 0.5 to 0.9, or 0.5 to 1.0, or other range values, which are not limited here and can be set as needed.
  • the processor determines whether the resolution error conforms to the resolution error range, and accordingly selects target image pairs with similar resolutions, thereby producing more consistent optimization results, so that the grid optimization error can be based on the target image pair.
  • the relevant parameters are calculated. Setting the resolution error range can control the resolution difference of the target image within the expected range and reduce the impact of the resolution error on subsequent grid optimization error calculations.
  • step 120 further includes:
  • Step e01 Use a candidate image pair as an initial image pair, and calculate the initial average coverage and initial coverage variance of the initial image pair in the common view area;
  • Step e02 Calculate the candidate average coverage and candidate coverage variance of other candidate image pairs in the common view area, determine whether the candidate average coverage is greater than the initial average coverage, and determine whether the candidate coverage variance is less than the initial coverage variance;
  • Step e03 If the candidate average coverage is greater than the initial average coverage and the candidate coverage variance is less than the initial coverage variance, the corresponding Candidate image pairs serve as target image pairs.
  • the coverage rate is defined as the number of all visible mesh patches in the initial surface that are observed by the image pair. Specifically, let F represent all visible mesh patches in the initial surface, and define one of its mesh patches f ⁇ F, the visible function relative to the image pair i and j is If f can be observed in both images, then otherwise Then, the global visibility function of f is defined as Then the coverage can be expressed as The obtained CP represents the coverage rate of the candidate image pair in the common view area.
  • the initial average coverage ⁇ p0 E(C P0 ), where C P0 represents the coverage rate of the initial image pair in the common view area, and E(C P0 ) represents the average coverage rate, which is C P0 /2.
  • Initial coverage variance ⁇ p0 stddev(C P0 ).
  • the corresponding candidate image pair is considered to be better, and the corresponding candidate image pair is regarded as the target image pair, so that the target image pair
  • the corresponding camera pair is configured to provide good coverage, which further facilitates grid optimization error calculation and optimizes the three-dimensional grid.
  • the first condition is further set based on the disparity quality error, overlap error, symmetry error and resolution error of the candidate image pair, and step 120 further includes:
  • Step f02 Determine whether the first error meets the first range value
  • Step f03 If the first error conforms to the first range value, use the candidate image pair whose first error conforms to the first range value as the target image pair.
  • the first error is obtained based on the parallax quality error, overlap error, symmetry error and resolution error.
  • the grid quality can be simultaneously evaluated through different dimensions such as parallax quality error, overlap error, symmetry error and resolution error.
  • ⁇ 1 , ⁇ 2 , ⁇ 3 , and ⁇ 4 may be set to the same value, such as 0.25. Or, in some embodiments, ⁇ 1 , ⁇ 2 , ⁇ 3 , and ⁇ 4 can be set as needed, for example, according to the degree of importance.
  • ⁇ 1 is set to be larger relative to , overlap error, symmetry error, and resolution error have little impact on grid optimization
  • ⁇ 2 , ⁇ 3 , and ⁇ 4 can be set to the same value, for example, ⁇ 1 is set to 0.4, and ⁇ 2 , ⁇ 3 , ⁇ 4 Set both to 0.2; or set ⁇ 1 to 0.4, ⁇ 2 to 0.3, ⁇ 3 to 0.2, and ⁇ 4 to 0.1.
  • the values of ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 can be set as needed and are not limited here.
  • the first condition is set to whether the first error meets the first range value. If the first error meets the first range value, it means that the first error meets expectations.
  • the candidate image pair whose first error meets the first range value is used as the target image pair to optimize the mesh optimization error and obtain a three-dimensional mesh with good mesh quality.
  • the first range value can be set to 0.6-0.8 to control the first error difference of the target image pair within a more reasonable expected range.
  • the first range value can also be set to 0.5-0.9, or 0.5-0.8, or other range values, which are not limited here and are set as needed.
  • the first error is obtained according to the parallax quality error, overlap error, symmetry error and resolution error.
  • the first error can be measured simultaneously through different dimensions such as parallax quality error, overlap error, symmetry error and resolution error.
  • the mesh quality is optimized to make the 3D mesh optimization better.
  • the first error may also be calculated from two or three of the disparity quality error, the overlap error, the symmetry error, and the resolution error, and the corresponding weights change accordingly.
  • the first condition is further based on parallax quality error, overlap error, symmetry error and resolution error settings of the candidate image pairs, and the target image pair among the plurality of candidate image pairs is selected according to the first condition, further comprising: :
  • Step g01 Calculate the first error based on parallax quality error, overlap error, symmetry error and resolution error;
  • Step g02 Use the first candidate image pair with the smallest error as the initial image pair, and calculate the initial average coverage and initial coverage variance of the initial image pair on the initial surface;
  • Step g03 Calculate the candidate average coverage and candidate coverage variance of other candidate image pairs on the initial surface, determine whether the candidate average coverage is greater than the initial average coverage, and determine whether the candidate coverage variance is less than the initial coverage variance;
  • Step g04 If the candidate average coverage is greater than the initial average coverage and the candidate coverage variance is less than the initial coverage variance, use the corresponding candidate image as a pending image pair;
  • Step g05 Calculate the first error of each undetermined image pair, and use the sum of the first errors of n undetermined image pairs as the undetermined error, where n is a positive integer greater than 0;
  • Step g06 Use the sum of the first errors of the n initial image pairs as the initial error, and determine whether the undetermined error is less than or equal to the initial error multiplied by the preset coefficient.
  • the preset coefficient is less than 1 and greater than or equal to 0.8;
  • Step g07 If the undetermined error is less than or equal to the initial error multiplied by the preset coefficient, use the undetermined image pair whose undetermined error is less than or equal to the initial error multiplied by the preset coefficient as the target image pair.
  • the candidate image pair with the smallest first error is used as the initial image pair, and the first error is calculated based on the parallax quality error, overlap error, symmetry error and resolution error. If the first error is the smallest, Correspondingly, the initial image pair is the optimal result obtained based on the first error, and its corresponding initial average coverage and initial coverage variance are also good, which is conducive to controlling the subsequent selection of average coverage and coverage variance to be more consistent. Expected target image pair.
  • steps g03 and g04 by using the candidate images whose candidate average coverage is greater than the initial average coverage and whose candidate coverage variance is less than the initial coverage variance as a pending image pair, the coverage is improved at the expense of the optimization result of the first error.
  • the joint optimization effects of the first error and coverage are weighed, and the corresponding to-be-determined image pair after trade-off optimization is obtained.
  • step g05 to step g07 there is at least one undetermined image pair obtained after step g03 and step g04, wherein each undetermined image pair can calculate the corresponding first error, by summing the first errors of n undetermined image pairs As the undetermined error, the sum of the first errors of the n initial image pairs is used as the initial error to determine whether the undetermined error is less than or equal to the initial error multiplied by the preset coefficient, so as to determine the corresponding target image pair by weighing the undetermined error.
  • the processor determines that the undetermined error is less than or equal to the initial error multiplied by the preset coefficient, it means that the n undetermined image pairs meet the trade-off requirements of coverage and first error, and the corresponding n undetermined image pairs are used as target image pairs. According to the n The corresponding calculated mesh optimization error of each undetermined image can be optimized, thereby obtaining a good three-dimensional mesh optimization effect.
  • the undetermined error is the sum of the first errors of n undetermined image pairs, so that under the condition that the coverage requirements are met, a large difference between the first errors of each undetermined image pair is allowed, as long as the n undetermined image pairs It is sufficient that the first error sum meets the corresponding error requirements, thereby balancing the first error and coverage to a large extent, which is beneficial to grid optimization error calculation.
  • the initial error is the sum of the first errors of n initial image pairs
  • the initial image pair is selected by the smallest first error among multiple candidate image pairs, which has a good optimization effect on the grid optimization error calculation, and will be preset Setting the coefficient to less than 1 and greater than or equal to 0.8 will help ensure that the deviation between the undetermined error and the initial error is small, making the undetermined error in line with expectations, and better balancing the undetermined error and coverage.
  • the network can be further calculated.
  • Grid optimization error further optimize the three-dimensional grid.
  • the undetermined error of the n target image pairs is equal to the initial error.
  • the deviation is small, and the initial error is the sum of the first errors of n initial image pairs.
  • the initial image pair is selected by the smallest first error among multiple candidate image pairs.
  • the undetermined error is in line with expectations and better The undetermined error and coverage rate are uniformly balanced.
  • the grid optimization error can be further calculated based on the n target image pairs to further optimize the three-dimensional grid.
  • the method further includes:
  • Step h01 Select the target pixel of the image according to the second condition, and the second condition is set according to the pixel depth value of the image;
  • Step h02 Calculate the grid optimization error based on the target pixels.
  • the target pixel of the image is selected according to the second condition, so as to calculate the grid optimization error correspondingly according to the target pixel.
  • grid optimization requires aggregating the gradient of the similarity measurement pixel p(x,y) as the center to calculate the gradient of the image area P.
  • the depth of each pixel in the image area P is discontinuous, which in turn causes the calculation of the derivative of the surrounding pixel error to the scene geometry to be incorrect. Therefore, the pixel depth value can measure the occlusion situation, and the second condition is set according to the pixel depth value of the image to find the target pixel with less impact of occlusion, thereby obtaining a good grid optimization error.
  • the second condition is set based on the variance of the pixel depth values of all pixels in the image area, and the pixels whose variance meets the requirements are used as target pixels; or in some embodiments, the second condition is set based on the variance of all pixels in the image area.
  • the standard deviation of the pixel depth value is set, and the pixel whose standard deviation meets the requirements is used as the target pixel; or in some embodiments, the second condition is set according to the difference between the pixel depth value of the central pixel of the image area and other pixels, Use the pixel whose difference meets the requirements as the target pixel; or the second condition sets the pixel depth value according to other methods to obtain the corresponding target pixel.
  • step h01 further includes:
  • Step h011 Divide the image into multiple candidate areas, each candidate area has a central pixel and multiple candidate pixels;
  • Step h012 Calculate the depth value difference, which is the difference between the pixel depth value of the candidate pixel and the pixel depth value of the center pixel;
  • Step h013 Classify all candidate pixels in the candidate area into a first pixel set and a second pixel set according to the depth value difference.
  • the minimum depth value difference in the second pixel set is greater than the maximum depth value difference in the first pixel set;
  • Step h014 Calculate the depth value variance of all first pixel sets based on the depth value difference
  • Step h015 the second condition is set according to the depth value variance, and whether the candidate pixel meets the second condition is determined;
  • Step h016 If the depth value difference meets the second condition, use the candidate pixel that meets the second condition as the target pixel.
  • each image can be divided into multiple candidate areas.
  • the candidate area can be considered as the above-mentioned image area P.
  • the candidate area can be an area of 5*5 pixels, or in some implementations, the candidate area can also be Areas with other pixel sizes are not limited here and can be set as needed.
  • Each candidate area has multiple pixels, including a central pixel and multiple candidate pixels, where the multiple candidate pixels are located around the central pixel.
  • the pixel depth z i value of the candidate pixel x i is the depth distance from the point x of the candidate pixel x i mapped on the nearest initial surface S to the center of the corresponding camera of the image, and the pixel depth value of the center pixel corresponds to The depth distance from the point where the center pixel is mapped to the nearest initial surface S to the center of the corresponding camera in the image is used to obtain the corresponding depth value difference.
  • each candidate pixel can calculate the corresponding depth value difference, and there are multiple corresponding depth value differences, when dividing, it can be divided in a variety of ways. For example, in some cases, divide according to the average value, that is, calculate the average value of multiple depth value differences, use candidate pixels corresponding to depth value differences that are less than the average value as the first pixel set, and use depth values that are greater than or equal to the average value as the first pixel set.
  • the candidate pixels corresponding to the difference are used as the second pixel set; or, in some cases, divided according to the median, that is, the median of multiple depth value differences is calculated, and the candidates corresponding to the depth value difference smaller than the median are
  • the pixels are used as the first pixel set, and the candidate pixels corresponding to the depth value difference greater than or equal to the median are used as the second pixel set; or the corresponding division threshold is set as needed to classify all candidate pixels in the candidate area as the first pixel set.
  • the pixel set and the second pixel set are such that the minimum depth value difference in the second pixel set is greater than the maximum depth value difference in the first pixel set.
  • step h014 to step h016 since the depth value difference of the first pixel set is small, it means that the difference between the pixel depth value of all candidate pixels in the first pixel set and the pixel depth value of the central pixel of the candidate area is small. In this case, it means that the occlusion effect on these candidate pixels is relatively small. Then, by calculating the depth value variance of the first pixel set and setting the second condition according to the depth value variance, it is determined whether the candidate pixel meets the second condition, and the candidate pixel that meets the second condition is used as the target pixel.
  • the second condition links the depth value difference with the depth value variance, so that the difference between the depth value difference corresponding to the target pixel that meets the second condition and the pixel depth value of the central pixel is further reduced, thereby further reducing the occlusion effect and improving the optimization result of the grid optimization error.
  • the second condition can be set as: the difference between the depth value difference and the depth value variance corresponding to the candidate pixel meets a preset threshold; or, in some embodiments, the second condition can be set as: the difference between the depth value difference and the depth value variance corresponding to the candidate pixel The difference between the depth value difference and the depth value variance meets the preset conditions; or, the second condition is set according to other methods, which is not limited here and can be set as needed.
  • step h011 to h016 candidate pixels with smaller pixel depth values are selected as the first pixel set, so that the candidate pixels in the first pixel set are relatively less affected by occlusion. Then, by calculating the depth value variance of the first pixel set, and setting the second condition according to the depth value variance, it is determined whether the candidate pixel meets the second condition, and the candidate pixel that meets the second condition is used as the target pixel, so that the The second condition relates the depth value difference to the depth value variance. Correspondingly, the difference between the depth value difference corresponding to the target pixel that meets the second condition and the pixel depth value of the central pixel is further reduced, thereby further reducing the impact of occlusion and improving the network Optimization results of lattice optimization error.
  • step h016 further includes:
  • Step h017 Will match candidate pixels as target pixels, where dd(h,k) represents the depth value difference corresponding to each candidate pixel in the first pixel set, Represents the depth value variance, max ⁇ dd(h,k) ⁇ represents the maximum depth value difference in the first pixel set.
  • step h017 by controlling the difference between the depth value difference and the depth value variance to be 10 times the difference between the depth value difference and the maximum depth value difference, the depth value difference is made smaller to meet the expected depth value difference, and the result is consistent with The required target pixels are used to calculate the grid optimization error.
  • Figure 2 shows a schematic structural diagram of a three-dimensional mesh optimization device provided by an embodiment of the present invention.
  • the device 200 includes:
  • the first acquisition module 210 is used to acquire multiple sets of candidate image pairs from multiple images and an initial surface of the three-dimensional model.
  • Each candidate image pair includes a first candidate image and a second candidate image from different viewing angles.
  • the initial surface has a first
  • the candidate image and the second candidate image are jointly mapped in a common viewing area of the initial surface, and the initial surface has at least one grid patch;
  • the first selection module 220 is configured to select a target image pair among multiple sets of candidate image pairs according to a first condition, where the first condition is set based on the area of the common view area;
  • the first calculation module 230 is used to calculate the grid optimization error according to the target image pair.
  • the first selection module 220 further includes:
  • a first determination unit configured to determine the first camera center of the first candidate image and the second camera center of the second candidate image
  • the first computing unit is used to calculate the average disparity from the center of the first camera and the center of the second camera to the common viewing area;
  • a second computing unit configured to calculate a disparity quality error according to the average disparity
  • the first judgment unit is used to judge whether the parallax quality error meets the quality error range
  • the second determination unit is configured to, if the parallax quality error conforms to the quality error range, use the candidate image pair whose parallax quality error conforms to the quality error range as the target image pair.
  • the first selection module 220 further includes:
  • a third computing unit configured to calculate the overlap error based on the mapping of the first candidate image and the second candidate image in the common view area and the mapping of the first candidate image in the first area of the initial surface;
  • the second judgment unit is used to judge whether the overlap error conforms to the overlap error range
  • the third determination unit is configured to, if the overlap degree error conforms to the overlap degree error range, use the candidate image pair whose overlap degree error conforms to the overlap degree error range as the target image pair.
  • the first selection module 220 further includes:
  • a fourth determination unit configured to determine the first camera center of the first candidate image and the second camera center of the second candidate image
  • the fourth computing unit is used to calculate the average direction angle difference between the first camera center and the second camera center with respect to the normal vector of the common view area;
  • a fifth calculation unit used for calculating the symmetry error according to the average direction angle difference
  • the third judgment unit is used to judge whether the symmetry error conforms to the symmetry error range
  • the fifth determination unit is used to, if the symmetry error conforms to the symmetry error range, use the candidate image pair whose symmetry error conforms to the symmetry error range as the target image pair.
  • the first selection module 220 further includes:
  • a sixth determination unit configured to determine the first camera center of the first candidate image and the second camera center of the second candidate image
  • the sixth computing unit is used to calculate the average distance from the center of the first camera and the center of the second camera to the common viewing area;
  • the seventh computing unit is used to calculate the resolution error based on the average distance
  • the fourth judgment unit is used to judge whether the resolution error meets the resolution error range
  • the seventh determination unit is configured to, if the resolution error conforms to the resolution error range, use the candidate image pair whose resolution error conforms to the resolution error range as the target image pair.
  • the first selection module 220 further includes:
  • the eighth computing unit is used to use a candidate image pair as an initial image pair and calculate the initial average coverage and initial coverage variance of the initial image pair in the common view area;
  • the fifth judgment unit is used to calculate the candidate average coverage and candidate coverage variance of other candidate image pairs in the common view area, determine whether the candidate average coverage is greater than the initial average coverage, and determine whether the candidate coverage variance is less than the initial coverage variance. ;
  • the eighth determination unit is configured to use the corresponding candidate image pair as the target image pair if the candidate average coverage is greater than the initial average coverage and the candidate coverage variance is less than the initial coverage variance.
  • the first condition is also set based on the disparity quality error, overlap error, symmetry error and resolution error of the candidate image pair, and the first selection module 220 further includes:
  • the sixth judgment unit is used to judge whether the first error meets the first range value
  • the ninth determination unit is configured to use the candidate image pair whose first error conforms to the first range value as the target image pair if the first error conforms to the first range value.
  • the first condition is also set based on the disparity quality error, overlap error, symmetry error and resolution error of the candidate image pair, and the first selection module 220 further includes:
  • the tenth computing unit is used to calculate the first error based on the parallax quality error, overlap error, symmetry error and resolution error;
  • an eleventh operation unit configured to use the candidate image pair with the smallest first error as an initial image pair, and calculate an initial average coverage rate and an initial coverage rate variance of the initial image pair on an initial surface
  • the seventh judgment unit is used to calculate the candidate average coverage and candidate coverage variance of other candidate image pairs on the initial surface, determine whether the candidate average coverage is greater than the initial average coverage, and determine whether the candidate coverage variance is less than the initial coverage variance;
  • the tenth determination unit is used to use the corresponding candidate image as a pending image pair if the candidate average coverage is greater than the initial average coverage and the candidate coverage variance is less than the initial coverage variance;
  • the twelfth operating unit is used to calculate the first error of each undetermined image pair, and use the first error of n undetermined image pairs as the undetermined error, where n is a positive integer greater than 0;
  • the eighth judgment unit is used to use the sum of the first errors of the n initial image pairs as the initial error, and judge whether the undetermined error is less than or equal to the initial error multiplied by the preset coefficient, and the preset coefficient is less than 1 and greater than or equal to 0.8;
  • An eleventh determination unit configured to use an undetermined image pair whose undetermined error is less than or equal to the initial error multiplied by the preset coefficient as a target image pair if the undetermined error is less than or equal to the initial error multiplied by the preset coefficient.
  • the device further includes:
  • the second selection module is used to select the target pixel of the image according to the second condition, and the second condition is set according to the pixel depth value of the image;
  • the second calculation module is used to calculate the grid optimization error based on the target pixels.
  • the second selection module further includes:
  • a first dividing unit used to divide the image into multiple candidate areas, each candidate area having a central pixel and multiple candidate pixels;
  • the thirteenth operation unit is used to calculate the depth value difference, which is the difference between the pixel depth value of the candidate pixel and the pixel depth value of the central pixel;
  • a twelfth determination unit configured to classify all candidate pixels in the candidate area into a first pixel set and a second pixel set according to the depth value difference, where the minimum depth value difference in the second pixel set is greater than the maximum depth value difference in the first pixel set ;
  • the fourteenth computing unit is used to calculate the depth value variance of the first pixel set in each candidate area based on the depth value difference;
  • the ninth judgment unit is used to judge whether the candidate pixel meets the second condition according to the depth value variance setting
  • the thirteenth determination unit is configured to use the candidate pixel that meets the second condition as the target pixel if the depth value difference meets the second condition.
  • the thirteenth determining unit further includes:
  • the fourteenth determination unit is used to match the candidate pixels as target pixels, where dd(h,k) represents the depth value difference corresponding to each candidate pixel in the first pixel set, Represents the depth value variance, max ⁇ dd(h,k) ⁇ represents the maximum depth value difference in the first pixel set.
  • Figure 3 shows a schematic structural diagram of a computing device provided by an embodiment of the present invention. Specific embodiments of the present invention do not limit the specific implementation of the computing device.
  • the computing device may include: a processor (processor) 302, a communications interface (Communications Interface) 304, a memory (memory) 306, and a communication bus 308.
  • processor processor
  • communications interface Communication Interface
  • memory memory
  • the processor 302 the communication interface 304, and the memory 306 complete communication with each other through the communication bus 308.
  • the communication interface 304 is used to communicate with network elements of other devices such as clients or other servers.
  • the processor 302 is configured to execute the program 310. Specifically, it can execute the above-mentioned related steps in the three-dimensional mesh optimization method embodiment.
  • program 310 may include program code including computer-executable instructions. .
  • the processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
  • the one or more processors included in the computing device may be the same type of processor, such as one or more CPUs; or they may be different types of processors, such as one or more CPUs and one or more ASICs.
  • Memory 306 is used to store program 310.
  • the memory 306 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
  • Embodiments of the present invention also provide a computer-readable storage medium, in which at least one executable instruction is stored, and the executable instruction executes the operation of any one of the above three-dimensional mesh optimization methods during runtime.
  • modules in the devices in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment.
  • the modules or units or components in the embodiments may be combined into one module or unit or component, and they may be divided into multiple sub-modules or sub-units or sub-components.
  • All features disclosed in this specification including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of the equipment are combined.
  • Each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

本发明实施例涉及计算机视觉技术领域，具体涉及一种三维网格优化方法、设备及存储介质，该方法包括：获取多个图像中的多组候选图像对和三维模型的初始表面；根据第一条件选择多组候选图像对中的目标图像对，第一条件基于共视区域的面积设置；根据目标图像对计算网格优化误差。本发明实施例通过根据基于共视区域面积设置的第一条件选择目标图像对，使得图像对的选择基于共视区域的面积设置，以衡量不同图像在共视区域的共同特性，从而选择符合共视区域要求的图像对，以降低区域不连续以及两个图像由于各自原因造成的误差影响，有利于网格优化误差计算更准确。

Description

三维网格优化方法、设备及存储介质 技术领域
本发明实施例涉及计算机视觉技术领域，具体涉及一种三维网格优化方法、设备及存储介质。
背景技术
传统的三维重建算法流程大致包含：从运动中恢复结构，多视图深度估计，网格构建，由于初始点云中包含大量的噪声，而传统的网格构建算法如泊松重建，从Visual Hull恢复的网格质量一般都比较差，此外也损失了些细节，导致三维重建模型精度较差。
因此，有必要提供一种三维网格优化方法、设备及存储介质，以克服上述问题。
发明内容
鉴于上述问题，本发明实施例提供了一种三维网格优化方法、设备及存储介质，用于解决现有技术中存在的问题。
根据本发明实施例的第一方面，提供了一种三维网格优化方法，所述方法包括：
获取多个图像中的多组候选图像对和三维模型的初始表面，每一所述候选图像对包括不同视角的第一候选图像和第二候选图像，所述初始表面具有所述第一候选图像和所述第二候选图像共同映射在所述初始表面的共视区域，所述初始表面具有至少一网格面片；
根据第一条件选择多组所述候选图像对中的目标图像对，所述第一条件基于所述共视区域的面积设置；
根据所述目标图像对计算网格优化误差。
在一些实施例中，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
计算所述第一相机中心和所述第二相机中心到所述共视区域的平均视差；
根据所述平均视差计算视差质量误差；
判断所述视差质量误差是否符合质量误差范围；
若所述视差质量误差符合所述质量误差范围，将所述视差质量误差符合所述质量误差范围的所述候选图像对作为所述目标图像对。
在一些实施例中，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
根据第一候选图像与第二候选图像映射在所述共视区域与第一候选图像映射在所述初始表面的第一区域，计算重叠度误差；
判断所述重叠度误差是否符合重叠度误差范围；
若所述重叠度误差符合重叠度误差范围，将所述重叠度误差符合所述重叠度误差范围的所述候选图像对作为所述目标图像对。
在一些实施例中，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
计算所述第一相机中心和所述第二相机中心关于所述共视区域的法向量的平均方向角度差；
根据所述平均方向角度差计算对称度误差；
判断所述对称度误差是否符合对称度误差范围；
若所述对称度误差符合所述对称度误差范围，将所述对称度误差符合所述对称度误差范围的所述候选图像对作为所述目标图像对。
在一些实施例中，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
计算所述第一相机中心和所述第二相机中心到所述共视区域的平均距离；
根据所述平均距离计算分辨率误差；
判断所述分辨率误差是否符合分辨率误差范围；
若所述分辨率误差符合所述分辨率误差范围，将所述分辨率误差符合所述分辨率误差范围的所述候选 图像对作为所述目标图像对。
在一些实施例中，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
将一所述候选图像对作为初始图像对，计算所述初始图像对在所述共视区域的初始平均覆盖率和初始覆盖率方差；
计算其他所述候选图像对在所述共视区域的候选平均覆盖率和候选覆盖率方差，判断所述候选平均覆盖率是否大于所述初始平均覆盖率，判断所述候选覆盖率方差是否小于所述初始覆盖率方差；
若所述候选平均覆盖率大于所述初始平均覆盖率且所述候选覆盖率方差小于所述初始覆盖率方差，将对应的所述候选图像对作为所述目标图像对。
在一些实施例中，所述第一条件还基于所述候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，所述根据第一条件选择多组所述候选图像对中的目标图像对，，进一步包括：
计算第一误差其中，μ1、μ2、μ3、μ4均为权重，μ1+μ2+μ3+μ4＝1，表示所述视差质量误差，表示所述重叠度误差，表示所述对称度误差，表示所述分辨率误差；
判断所述第一误差是否符合第一范围值；
若所述第一误差符合所述第一范围值，将所述第一误差符合所述第一范围值的所述候选图像对作为所述目标图像对。
在一些实施例中，所述第一条件还基于所述候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，所述根据第一条件选择多组所述候选图像对中的目标图像对，，进一步包括：
根据所述视差质量误差、所述重叠度误差、所述对称度误差和所述分辨率误差计算第一误差；
将所述第一误差最小的所述候选图像对作为初始图像对，计算所述初始图像对在所述初始表面的初始平均覆盖率和初始覆盖率方差；
计算其他所述候选图像对在所述初始表面的候选平均覆盖率和候选覆盖率方差，判断所述候选平均覆盖率是否大于所述初始平均覆盖率，判断所述候选覆盖率方差是否小于所述初始覆盖率方差；
若所述候选平均覆盖率大于所述初始平均覆盖率且所述候选覆盖率方差小于所述初始覆盖率方差，将对应的所述候选图像作为待定图像对；
计算每一所述待定图像对的所述第一误差，将n个所述待定图像对的所述第一误差作为待定误差，n为大于0的正整数；
将n个所述初始图像对的所述第一误差总和作为初始误差，判断所述待定误差是否小于或等于所述初始误差乘以预设系数，所述预设系数小于1大于等于0.8；
若所述待定误差小于或等于所述初始误差乘以预设系数，将所述待定误差小于或等于所述初始误差乘以预设系数的所述待定图像对作为所述目标图像对。
在一些实施例中，所述获取多个图像中的多组候选图像对和三维模型的初始表面之后，所述方法还包括：
根据第二条件选择所述图像的目标像素，所述第二条件根据所述图像的像素深度值设置；
根据所述目标像素计算网格优化误差。
在一些实施例中，所述根据第二条件选择所述图像的目标像素，进一步包括：
将所述图像分成多个候选区域，每一所述候选区域中具有中心像素和多个候选像素；
计算深度值差，所述深度值差为所述候选像素的所述像素深度值与所述中心像素的所述像素深度值之差；
根据所述深度值差将所述候选区域的所有所述候选像素分类为第一像素集合和第二像素集合，所述第二像素集合中最小所述深度值差大于所述第一像素集合中最大所述深度值差；
根据所述深度值差计算每一所述候选区域中所述第一像素集合的深度值方差；
所述第二条件根据所述深度值方差设置，判断所述候选像素是否符合所述第二条件；
若所述深度值差符合所述第二条件，将符合所述第二条件的所述候选像素作为所述目标像素。
在一些实施例中，所述若所述候选像素符合所述第二条件，将符合所述第二条件的所述候选像素作为所述目标像素，进一步包括：
将符合的所述候选像素作为所述目标像素，其中，dd(h,k)表示所述第一像素集合中每一所述候选像素对应的所述深度值差，表示所述深度值方差，max{dd(h,k)}表示所述第一像素集合中最大所述深度值差。
根据本发明实施例的第二方面，提供了一种计算设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；；
所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如上述任一项所述的三维网格优化方法的操作。
根据本发明实施例的第三方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令在运行时执行如上述任一项所述的三维网格优化方法的操作。
本发明实施例通过根据基于共视区域面积设置的第一条件选择目标图像对，，使得图像对的选择基于共视区域的面积设置，以衡量不同图像在共视区域的共同特性，从而选择符合共视区域要求的图像对，使得该选择的图像对的质量较高，有利于减小网格优化误差计算冗余性，提高网格优化误差计算的求解速度，并且可以降低区域不连续以及两个图像由于各自原因造成的误差影响，有利于网格优化误差计算更准确。
上述说明仅是本发明实施例技术方案的概述，为了能够更清楚了解本发明实施例的技术手段，而可依照说明书的内容予以实施，并且为了让本发明实施例的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
附图仅用于示出实施方式，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1示出了本发明实施例提供的三维网格优化方法的流程示意图；
图2示出了本发明实施例提供的三维网格优化装置的结构示意图；
图3示出了本发明一些实施例提供的计算设备的结构示意图；
图4示出了本发明一些实施例提供的fi区域和hi区域不连续时图像映射关系的示意图；
图5示出了本发明一些实施例提供的fi区域和hi区域不连续时图像映射关系的立体示意图；
图6示出了本发明一些实施例提供的两个图像映射到初始表面的示意图；
图7示出了本发明一些实施例提供的两个图像对称关系的示意图。
具体实施方式
下面将参照附图更详细地描述本发明的示例性实施例。虽然附图中显示了本发明的示例性实施例，然而应当理解，可以以各种形式实现本发明而不应被这里阐述的实施例所限制。
在进行三维重建过程中，处理器先根据多个图像进行初步建模，形成初始三维模型，该初始三维模型的网格质量较差。针对现有三维重建模型网格质量较差的问题，发明人发现，图像对的选择对网格优化的最终质量有很重要的影响。如果两个相机距离很近，会导致重建的噪声很大。另一方面，如果两个相机距离太远，会导致重叠范围较少，也就是说两个相机共视范围太小。正确的图像对选择有利于图像对间一致的光度误差计算，进而使梯度下降法速度更快。
此外，发明人还发现，模型遮挡也会影响网格优化结果，如图4和图5中，图像i和图像j具有映射到初始表面S的共视区域，其中，初始表面S中的h区域、g区域和f区域均为图像i和图像j共同映射到的区域。图像j中，hj表示映射到h区域时位于映射线l1与l2之间的区域，gj表示映射到g区域时位于映 射线l2与l3之间的区域，fj表示映射到f区域时位于映射线l3与l4之间的区域。图像i中，hi表示映射到h区域时位于映射线l5与l6之间的区域，fj表示映射到f区域时位于映射线ll6与l7之间的区域，其中，圆圈内所示的区域未显示本应映射到g区域的gi区域，在此情况下，gi区域被遮挡，实际上图像i中fi区域和hi区域不连续。在计算gi区域投影误差时，现有的算法会同时考虑来fi区域和hi区域的信息，在此情况下，由于fi区域和hi区域不连续，导致误差计算不准确，影响网格质量。
发明人提供了一种三维网格优化方法，根据基于共视区域面积设置的第一条件选择目标图像对，使得图像对的选择基于共视区域的面积设置，以衡量不同图像在共视区域的共同特性，从而选择合适的图像对，以降低区域不连续以及两个图像由于各自原因造成的误差影响，有利于网格优化误差计算更准确。
图1示出了本发明实施例提供的三维网格优化方法的流程图，该方法由计算设备执行，该计算设备可以是包括一个或多个处理器的计算设备，该处理器可能是中央处理器CPU，或者是特定集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本发明实施例的一个或多个集成电路，在此不做限定。计算设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC，在此不做限定。。
如图1所示，该方法包括以下步骤：
步骤110：获取多个图像中的多组候选图像对和三维模型的初始表面，每一候选图像对包括不同视角的第一候选图像和第二候选图像，初始表面具有第一候选图像和第二候选图像共同映射在初始表面的共视区域，初始表面具有至少一网格面片；
步骤120：根据第一条件选择多组候选图像对中的目标图像对，第一条件基于共视区域的面积设置；
步骤130：根据目标图像对计算网格优化误差。
步骤110中，初始三维模型的初始表面具有至少一网格面片，在一些情况下，初始表面可以由一个网格面片形成；在一些情况下，初始表面也可以由多个网格面片组成。初始的三维模型的网格质量较差，需要进行网格优化，其中，多个图像中任意两张图像形成图像对，由此具有多组图像对。其中，若图像对中两个图像具有共同映射在初始表面的共视区域，则将该图像对作为候选图像对，通常候选图像对具有多组，每一候选图像对包括不同视角的第一候选图像和第二候选图像。
步骤120中，第一条件基于共视区域的面积设置，根据第一条件选择多组候选图像对中的目标图像对，以衡量不同图像在共视区域的共同特性，从而选择合适的图像对，有利于网格优化误差计算更准确。
第一条件的设置有多种情况，可以根据视差质量误差、重叠度误差、对称度误差、分辨率误差、覆盖率等图像特性中的至少一个设置。或者也可以根据其他图像特性与共视区域相关设置，在此不做限定，根据需要设置。
步骤130中，在选择了合适的目标图像对后，需要根据该目标图像对的相关参数进行网格优化误差计算，以得到优化后的三维网格。其中，网格优化误差E＝Ephoto+Esmooth，其中Ephoto表示的是三维模型相对于图像光度一致性误差。Esmooth表示初始表面的光滑正则项。
为了最小化Ephoto，网格优化程序需要使用梯度下降法。得到Ephoto的计算公式为其中，表示梯度误差函数，如果x在目标图像对i，j的投影点的周围的patch的相似度增加，下降，表示的是图像j的点通过初始表面S映射到图像i坐标系下，表示共视区域。Ephoto＝E(s)对顶点Xi计算的离散梯度可以表示为 其中，Xi∈R3初始表面S的一个顶点，φi(x)表示初始表面上一个x在对应三角形相对于顶点Xi三角形的重心坐标。
通过公式我们可以得到通过将积分变量从x变换为xi，可以将积分从初始表面S区域变换到图像i。如图6所示，表示在x处朝向表面外的梯度，xi是x在图像i中的投影，xj是x在图像j中的投影，di表示从相机中心ci到x的向量，zi是x到相机中心ci的深度距离。
光滑正则项Esmooth可以通过拉普拉斯算子得到。
步骤110至步骤130中，通过根据基于共视区域面积设置的第一条件选择目标图像对，使得图像对的选择基于共视区域的面积设置，以衡量不同图像在共视区域的共同特性，从而选择符合共视区域要求的图像对，使得该选择的图像对的质量较高，有利于减小网格优化误差计算冗余性，，提高网格优化误差计算的求解速度，并且可以降低区域不连续以及两个图像由于各自原因造成的误差影响，有利于网格优化误差计算更准确。
在一些实施例中，步骤120进一步包括：
步骤a01：确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
步骤a02：计算第一相机中心和第二相机中心到共视区域的平均视差；
步骤a03：根据平均视差计算视差质量误差；
步骤a04：判断视差质量误差是否符合质量误差范围；
步骤a05：若视差质量误差符合质量误差范围，将视差质量误差符合质量误差范围的候选图像对作为目标图像对。
步骤a01和步骤a02中，步骤a02中平均视差其中，表示共视区域的面积，表示第一候选图像i和第二候选图像j共同映射在初始表面S的共视区域，xi表示位于共视区域内的任意一点，cixi表示第一相机中心ci与共视区域内xi的第一连线，cixi表示第二相机中心cj与共视区域内xi的第二连线，∠(cixi,cjxi)表示第一连线与第二连线的夹角。
步骤a03中，视差质量误差其中，视差小的角度可以使图像对有更好的重叠度，而大角度会使网格优化更加稳定，因此，在视差质量误差需要平衡视差角度，得到更良好的视差质量误差。αp为平均参考视差，通常设为40°-60°；σp为预设角度方差，通常设为40°-50°。αp优选设为50°，σp优选设为45°，以更好地衡量视差质量误差。
步骤a04和步骤a05中，对应的，第一条件即为视差质量误差是否符合质量误差范围。若处理器判断视差质量误差符合质量误差范围，则选择符合质量误差范围的候选图像对作为目标图像对，使得网格优化误差可以根据目标图像对的相关参数进行计算。其中，质量误差范围在40°到60°范围之间，在此范围的视差质量误差较好，根据该视差质量误差所得到目标图像对计算的网格优化误差较小，从而对三维网格优化进一步优化。
步骤a01至步骤a04中，通过计算视差质量误差，选择符合要求的视差质量误差对应的目标图像对，以实现目标图像对在合理视差和共视区域内更好地权衡，以为后续计算网格优化误差提供良好的目标图像对的参数。
在一些实施例中，步骤120进一步包括：
步骤b01：根据第一候选图像与第二候选图像映射在共视区域与第一候选图像映射在初始表面的第一区域，计算重叠度误差。
步骤b02：判断重叠度误差是否符合重叠度误差范围；
步骤b03：若重叠度误差符合重叠度误差范围，将重叠度误差符合重叠度误差范围的候选图像对作为目标图像对。
步骤b01中，重叠度误差 表示共视区域的面积，表示第一候选图像i和第二候选图像j共同映射在初始表面S的共视区域，A(Ii)表示第一候选图像i映射在初始表面S的第一区域。其中，因为希望最小化所有的重叠度误差函数来选取合适的图像对，所以中的负号，即希望两个视图间的共视区域更大些。
步骤b02和步骤b03中，对应的，第一条件即为重叠度误差是否符合重叠度误差范围。若处理器判断重叠度误差符合重叠度误差范围，则选择符合重叠度误差范围的候选图像对作为目标图像对，使得网格优化误差可以根据目标图像对的相关参数进行计算。其中，重叠度误差范围可以设置为0.5～0.8，以使得目标图像对的重叠度较为符合合理预期，为后续计算网格优化误差提供良好的目标图像对参数。当然，在一些实施例中，分辨率误差范围也可以设置为0.5～0.9，或者0.6～0.8，或者其他范围值，在此不做限定，根据需要设置。
步骤b01至步骤b03中，通过计算重叠度误差，以将符合重叠度误差范围的候选图像对确定为目标图像对，保证目标图像对的重叠度处符合合理预期，以使得目标图像对根据重叠度误差进行选择，为后续计算网格优化误差提供良好的目标图像对的参数。
在一些实施例中，步骤120进一步包括：
步骤c01：确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
步骤c02：计算第一相机中心和第二相机中心关于共视区域的法向量的平均方向角度差；
步骤c03：根据平均方向角度差计算对称度误差；
步骤c04：判断对称度误差是否符合对称度误差范围；
步骤c05：若对称度误差符合对称度误差范围，将对称度误差符合对称度误差范围的候选图像对作为 目标图像对。
步骤c01和步骤c02中，平均方向角度差 其中，表示共视区域的面积，表示第一候选图像i和第二候选图像j共同映射在初始表面S的共视区域，x表示位于共视区域内的任意一点，cix表示第一相机中心ci与共视区域内x的第一连线，cjx表示第二相机中心cj与共视区域内x的第二连线，表示在共视区域的x处的法向，表示第一连线与的夹角，表示第二连线与的夹角，如果cix和cjx位于的同侧，sign＝1，如果cix和cjx位于的不同侧，sign＝-1。其中，在一些场景下，如图7所示，初始表面S被参考相机cref和其他两个相机ci和cj观测到，若初始表面S完全被三个相机观测到，也就是说图像的重叠度为100％，基线cref-ci和cref-cj具有相似的数值，图像对可能同时好或者同时坏。直观上，初始表面S沿着其法向进行变化，同时假设图像呈像受到高斯噪声的影响。因此，通过计算候选图像对的对称度误差以及判断对称度误差范围，使得相似的噪声将会转移到显著不同的沿着不同的梯度噪声。统计上，若候选图像对位于的同一侧，则沿着法向的噪声梯度会进行累加；若候选图像对位于的不同侧，则沿着法向的噪声梯度会抵消。由此，可以判断，若在初始表面S的不同xi的位置位于对应不同侧的部分较多，所得的Si,j数值相应较小，噪声影响较小。
步骤c03中，对称度误差σs为预设对称角度方差，通常设为40°-50°，优选设为45°。
步骤c04和步骤c05中，对应的，第一条件即为对称度误差是否符合对称度误差范围。若处理器判断对称度误差符合对称度误差范围，则选择符合对称度误差范围的候选图像对作为目标图像对，使得网格优化误差可以根据目标图像对的相关参数进行计算。
步骤c01至步骤c05中，通过计算平均方向角度差以及对称度误差，以判断对称度误差是否符合对称度误差范围，从而将对称度误差符合对称度误差范围的候选图像对作为目标图像对，以消除图像对中相机 对称度造成的噪声影响，从而选择对称度误差较小的目标图像对，使得网格优化误差计算进一步优化。对称度误差范围可以设置为0.6～0.9，以将目标图像对对应的两个相机对称度控制在预期范围，减少噪声影响。当然，在一些实施例中，对称度误差范围也可以设置为0.5～0.8，或者0.7～0.9，或者其他范围值，在此不做限定，根据需要设置。
在一些实施例中，步骤120进一步包括：
步骤d01：确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
步骤d02：计算第一相机中心和第二相机中心到共视区域的平均距离；
步骤d03：根据平均距离计算分辨率误差；
步骤d04：根据分辨率误差选择目标图像对。
步骤d01和步骤d02中，平均距离其中，表示共视区域的面积，表示第一候选图像i和第二候选图像j共同映射在初始表面S的共视区域， xi表示位于共视区域内的任意一点，fi表示第一候选图像i对应的相机焦距，fj表示第二候选图像j对应的相机焦距，cixi表示第一相机中心ci与共视区域内xi的第一连线距离，cjxi表示第一相机中心cj与共视区域内xi的第一连线。
步骤d03中，为了倾向于选择具有相似分辨率的图像对，从而产生更加一致的优化结果，需要计算分辨率误差，以根据分辨率误差评价候选图像对的分辨率一致性。分辨率误差σr为预设分辨率方差，通常设置为0.2-0.4，其中，优选的，σr设为0.25，表示分辨率的差异为25％。
步骤d04和步骤d05中，对应的，第一条件即为分辨率误差是否符合分辨率误差范围。若处理器判断分辨率误差符合分辨率误差范围，则选择符合分辨率误差范围的候选图像对作为目标图像对，使得网格优化误差可以根据目标图像对的相关参数进行计算。分辨率误差范围的设置，可以将目标图像对分辨率差异控制在预期范围，分辨率对后续网格优化误差计算的影响。其中，分辨率误差范围可以设置为0.6～1.0，以将目标图像对分辨率差异控制在较为合理的预期范围。当然，在一些实施例中，，分辨率误差范围也可以设置为0.5～0.9，或者0.5～1.0，或者其他范围值，在此不做限定，根据需要设置。
通过步骤d01至步骤d05，处理器判断分辨率误差是否符合分辨率误差范围，以相应选择具有相似分辨率的目标图像对，从而产生更加一致的优化结果，使得网格优化误差可以根据目标图像对的相关参数进行计算。分辨率误差范围的设置，可以将目标图像对分辨率差异控制在预期范围，减小分辨率误差对后续网格优化误差计算的影响。
在一些实施例中，步骤120进一步包括：
步骤e01：将一候选图像对作为初始图像对，计算初始图像对在共视区域的初始平均覆盖率和初始覆盖率方差；
步骤e02：计算其他候选图像对在共视区域的候选平均覆盖率和候选覆盖率方差，判断候选平均覆盖率是否大于初始平均覆盖率，判断候选覆盖率方差是否小于初始覆盖率方差；
步骤e03：若候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差，将对应的 候选图像对作为目标图像对。
步骤e01中，将覆盖率定义为初始表面中所有可见网格面片被图像对观测的数量，具体的，令F表示初始表面中所有可见网格面片，定义其某个网格面片f∈F，相对于图像对i和j的可见函数为如果f在两个图像都可以观测到，则否则然后，f的全局可见性函数定义为然后覆盖率可以表达为所得的CP即表示候选图像对在共视区域的覆盖率。初始平均覆盖率μp0＝E(CP0)，其中CP0表示初始图像对在共视区域的覆盖率，E(CP0)表示覆盖率的平均值即CP0/2。初始覆盖率方差σp0＝stddev(CP0)。
步骤e02和步骤e03中，其他候选图像对中每一候选图像对的候选平均覆盖率μp＝E(CP)，初始覆盖率方差σp＝stddev(CP)，通过判断候选平均覆盖率是否大于初始平均覆盖率，判断候选覆盖率方差是否小于初始覆盖率方差，相应选择目标图像对，得到对应的目标图像对的集合，该集合中的所有目标图像对用于网格优化误差计算，以优化三维网格。其中，若候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差，则认为该对应的候选图像对更优，将对应的候选图像对作为目标图像对，以使目标图像对所对应的相机对配置为可以提供良好的覆盖范围，从而进一步有利于网格优化误差计算，优化三维网格。
在一些实施例中，第一条件还基于候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，步骤120进一步包括：
步骤f01：计算第一误差其中，μ1、μ2、μ3、μ4均为权重，μ1+μ2+μ3+μ4＝1，表示视差质量误差，表示重叠度误差，表示对称度误差，表示分辨率误差；
步骤f02：判断第一误差是否符合第一范围值；
步骤f03：若第一误差符合第一范围值，，将第一误差符合第一范围值的候选图像对作为目标图像对。
步骤f01中，根据视差质量误差、重叠度误差、对称度误差和分辨率误差得到第一误差，可以通过视差质量误差、重叠度误差、对称度误差和分辨率误差等不同维度同时对网格质量进行优化，使得三维网格优化更好。其中，在一些实施例中，μ1、μ2、μ3、μ4可以设置为同一数值，例如0.25。或者，在一些实施例中，μ1、μ2、μ3、μ4可以根据需要设置，例如根据重要程度设置，例如视差质量误差对网格优化的影响更大，则μ1相对设置更大，重叠度误差、对称度误差、分辨率误差对网格优化的影响较小，则μ2、μ3、μ4可以设置为相同，例如μ1设置为0.4，μ2、μ3、μ4均设置为0.2；或者将μ1设置为0.4、μ2设置为0.3、μ3设置为0.2、μ4设置为0.1。μ1、μ2、μ3、μ4的数值可以根据需要设置，在此不做限定。
步骤f02和步骤f03中，第一条件设为第一误差是否符合第一范围值，若第一误差符合第一范围值，说明第一误差符合预期，将第一误差符合第一范围值的候选图像对作为目标图像对，以优化网格优化误差，得到良好网格质量的三维网格。其中，第一范围值可以设置为0.6～0.8，以将目标图像对第一误差异控制在较为合理的预期范围。当然，在一些实施例中，第一范围值也可以设置为0.5～0.9，或者0.5～0.8，或者其他范围值，在此不做限定，根据需要设置。
通过步骤f01至步骤f03，根据视差质量误差、重叠度误差、对称度误差和分辨率误差得到第一误差，可以通过视差质量误差、重叠度误差、对称度误差和分辨率误差等不同维度同时对网格质量进行优化，使得三维网格优化更好。
在一些实施例中，第一误差也可以由视差质量误差、重叠度误差、对称度误差和分辨率误差中的两个或者三个计算得到，对应的权重相应发生变化。例如，在一些情况下，第一误差由视差质量误差、重叠度误差计算得到，相应的，μ1+μ2＝1，则μ1和μ2可以设置为同一数值，例如0.5，或者μ1设置为0.6，μ2设置为0.4；或者μ1和μ2设置为其他数值。在一些情况下，第一误差由视差质量误差、重叠度误差、分辨率误差计算得到，相应的，μ1+μ2+μ4＝1，则μ1、μ2和μ3可以设置为同一数值，例如0.333；或者μ1设置为0.4，μ2和μ4设置为0.3；或者μ1、μ2和μ3设置为其他数值。以此类推，在此不做赘述。
在一些实施例中，第一条件还基于候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，根据第一条件选择多组候选图像对中的目标图像对，进一步包括：
步骤g01：根据视差质量误差、重叠度误差、对称度误差和分辨率误差计算第一误差；
步骤g02：将第一误差最小的候选图像对作为初始图像对，计算初始图像对在初始表面的初始平均覆盖率和初始覆盖率方差；
步骤g03：计算其他候选图像对在初始表面的候选平均覆盖率和候选覆盖率方差，判断候选平均覆盖率是否大于初始平均覆盖率，判断候选覆盖率方差是否小于初始覆盖率方差；
步骤g04：若候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差，将对应的候选图像作为待定图像对；
步骤g05：计算每一待定图像对的第一误差，将n个待定图像对的第一误差总和作为待定误差，n为大于0的正整数；
步骤g06：将n个初始图像对的第一误差总和作为初始误差，判断待定误差是否小于或等于初始误差乘以预设系数，预设系数小于1大于等于0.8；
步骤g07：若待定误差小于或等于初始误差乘以预设系数，将待定误差小于或等于初始误差乘以预设系数的待定图像对作为目标图像对。
步骤g01和步骤g02中，通过将第一误差最小的候选图像对作为初始图像对，而第一误差根据视差质量误差、重叠度误差、对称度误差和分辨率误差计算，若第一误差最小，相应的，初始图像对为根据第一误差得到的最优结果，其对应的初始平均覆盖率和初始覆盖率方差也较好，有利于控制后续通过平均覆盖率和覆盖率方差选择更够选择符合预期的目标图像对。
步骤g03和步骤g04中，通过将候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差的候选图像作为待定图像对，以牺牲第一误差的优化结果获得覆盖率的提高，从而权衡第一误差和覆盖率共同的优化作用，得到对应经过权衡优化后的待定图像对。
步骤g05至步骤g07中，步骤g03和步骤g04后得到的待定图像对至少有一个，其中，每一待定图像对均能计算对应的第一误差，通过将n个待定图像对的第一误差总和作为待定误差，将n个初始图像对的第一误差总和作为初始误差，以判断待定误差是否小于或等于初始误差乘以预设系数，从而通过待定误差进行权衡，确定对应的目标图像对。若处理器判断待定误差小于或等于初始误差乘以预设系数，说明n个待定图像对符合覆盖率和第一误差的权衡要求，将对应的n个待定图像对作为目标图像对，根据该n个待定图像对相应计算得到的网格优化误差能够得到优化，从而获得良好的三维网格优化效果。
其中，待定误差为n个待定图像对的第一误差总和，使得在符合覆盖率要求的情况下，允许每一个待定图像对的第一误差之间有较大差别，只要n个待定图像对的第一误差总和符合相应的误差要求即可，从而在较大程度上将第一误差与覆盖率均衡，有利于网格优化误差计算。
此外，由于初始误差为n个初始图像对的第一误差总和，该初始图像对由多个候选图像对中第一误差最小选择的，对网格优化误差计算具有良好的优化效果，将预设系数设为小于1大于等于0.8，有利于保证待定误差与初始误差的偏差较小，使得待定误差符合预期，以更好地均衡待定误差与覆盖率，根据该n个目标图像对可以进一步计算网格优化误差，进一步优化三维网格。
步骤g01至步骤g07中，通过使得n个目标图像对的待定误差小于或等于初始误差乘以预设系数，且预设系数小于1大于等于0.8，使得n个目标图像对的待定误差与初始误差的偏差较小，而该初始误差为n个初始图像对的第一误差总和，该初始图像对由多个候选图像对中第一误差最小选择的，相应的，待定误差符合预期，以更好地均衡待定误差与覆盖率，根据该n个目标图像对可以进一步计算网格优化误差，进一步优化三维网格。
在一些实施例中，步骤110之后，该方法还包括：
步骤h01：根据第二条件选择图像的目标像素，第二条件根据图像的像素深度值设置；
步骤h02：根据目标像素计算网格优化误差。
步骤h01和步骤h02中，针对图像遮挡影响网格优化误差计算的情况，根据第二条件选择图像的目标像素，以根据该目标像素对应计算网格优化误差。其中，对于每个像素，网格优化需要聚合相似度测量的梯度像素p(x,y)为中心，计算图像区域P的梯度。在图像被遮挡的情况下，导致图像区域P各像素深度的不连续，进而使得其周围的像素误差对场景几何的导数计算不正确。因此，像素深度值可以衡量遮挡情况，通过第二条件根据图像的像素深度值设置，以找到遮挡影响较小的目标像素，从而获得良好的网格优化误差。
在一些实施例中，第二条件根据图像区域中所有像素的像素深度值的方差进行设置，将方差符合要求的像素作为目标像素；或者在一些实施例中，第二条件根据图像区域中所有像素的像素深度值的标准差进行设置，将标准差符合要求的像素作为目标像素；或者在一些实施例中，第二条件根据图像区域的中心像素与其他像素的像素深度值的差值进行设置，将差值符合要求的像素作为目标像素；或者第二条件根据其他方式对像素深度值进行设置，得到对应的目标像素。
在一些实施例中，步骤h01进一步包括：
步骤h011：将图像分成多个候选区域，每一候选区域中具有中心像素和多个候选像素；
步骤h012：计算深度值差，深度值差为候选像素的像素深度值与中心像素的像素深度值之差；
步骤h013：根据深度值差将候选区域的所有候选像素分类为第一像素集合和第二像素集合，第二像素集合中最小深度值差大于第一像素集合中最大深度值差；
步骤h014：根据深度值差计算所有第一像素集合的深度值方差；
步骤h015：第二条件根据深度值方差设置，判断候选像素是否符合第二条件；
步骤h016：若深度值差符合第二条件，将符合第二条件的候选像素作为目标像素。
步骤h011中，每一图像可以划分多个候选区域，该候选区域即可以认为是上述的图像区域P，该候选区域可以为5*5像素的区域，或者在一些实施中，候选区域也可以为其他像素尺寸的区域，在此不做限定，根据需要设置。其中每一候选区域均具有多个像素，包括中心像素和多个候选像素，其中，多个候选像素位于中心像素的周围。
步骤h012中，如图6所示，候选像素xi的像素深度zi值为候选像素xi映射在最近初始表面S的点x到图像对应相机中心的深度距离，中心像素的像素深度值对应为中心像素映射到最近初始表面S的点到图像对应相机中心的深度距离，从而得出对应的深度值差。
步骤h013中，由于每一候选像素可以计算对应的深度值差，相应的深度值差有多个，在进行划分时，可以通过多种方式划分。例如，在一些情况下，根据平均值划分，即计算多个深度值差的平均值，将小于平均值的深度值差对应的候选像素作为第一像素集合，将大于或等于平均值的深度值差对应的候选像素作为第二像素集合；或者，在一些情况下，根据中位数划分，即计算得出多个深度值差的中位数，将小于中位数的深度值差对应的候选像素作为第一像素集合，将大于或等于中位数的深度值差对应的候选像素作为第二像素集合；或者根据需要设定对应的划分阈值，以将候选区域的所有候选像素分类为第一像素集合和第二像素集合，使得第二像素集合中最小深度值差大于第一像素集合中最大深度值差。
步骤h014至步骤h016中，由于第一像素集合的深度值差较小，说明第一像素集合中所有的候选像素的像素深度值与候选区域的中心像素的像素深度值的差异较小，在此情况下，说明该些候选像素所受的遮挡影响相对较小。而后，通过计算第一像素集合的深度值方差，以及第二条件根据深度值方差设置，以判断候选像素是否符合第二条件，将符合第二条件的候选像素作为目标像素。该第二条件将深度值差与深度值方差联系，使得符合第二条件的目标像素对应的深度值差与中心像素的像素深度值的差异进一步减小，从而进一步减小遮挡影响，提高网格优化误差的优化结果。
在一些实施例中，第二条件可以设置为：候选像素对应的深度值差与深度值方差的差值符合预设阈值；或者，在一些实施例中，第二条件可以设置为：候选像素对应的深度值差与深度值方差的差值符合预设条件；或者，第二条件根据其他方式设置，在此不做限定，根据需要设置。
步骤h011至步骤h016中，通过选择像素深度值较小的候选像素作为第一像素集合，使得第一像素集合中候选像素所受的遮挡影响相对较小。而后，通过计算第一像素集合的深度值方差，以及第二条件根据深度值方差设置，以判断候选像素是否符合第二条件，将符合第二条件的候选像素作为目标像素，使得该 第二条件将深度值差与深度值方差联系，相应的，符合第二条件的目标像素对应的深度值差与中心像素的像素深度值的差异进一步减小，从而进一步减小遮挡影响，提高网格优化误差的优化结果。
在一些实施例中，步骤h016进一步包括：
步骤h017：将符合的候选像素作为目标像素，其中，dd(h,k)表示第一像素集合中每一候选像素对应的深度值差，表示深度值方差，max{dd(h,k)}表示第一像素集合中最大深度值差。
步骤h017中，通过将深度值差与深度值方差的差值控制在深度值差与最大深度值差的差值的10倍关系，使得深度值差较小以符合预期的深度值差异，得到符合要求的目标像素，从而计算网格优化误差。
图2示出了本发明实施例提供的三维网格优化装置的结构示意图，该装置200包括：
第一获取模块210，用于获取多个图像中的多组候选图像对和三维模型的初始表面，每一候选图像对包括不同视角的第一候选图像和第二候选图像，初始表面具有第一候选图像和第二候选图像共同映射在初始表面的共视区域，初始表面具有至少一网格面片；
第一选择模块220，用于根据第一条件选择多组候选图像对中的目标图像对，第一条件基于共视区域的面积设置；
第一计算模块230，用于根据目标图像对计算网格优化误差。
在一些实施例中，第一选择模块220进一步包括：
第一确定单元，用于确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
第一运算单元，用于计算第一相机中心和第二相机中心到共视区域的平均视差；
第二运算单元，用于根据平均视差计算视差质量误差；
第一判断单元，用于判断视差质量误差是否符合质量误差范围；
第二确定单元，用于若视差质量误差符合质量误差范围，将视差质量误差符合质量误差范围的候选图像对作为目标图像对。
在一些实施例中，第一选择模块220进一步包括：
第三运算单元，用于根据第一候选图像与第二候选图像映射在共视区域与第一候选图像映射在初始表面的第一区域，计算重叠度误差；
第二判断单元，用于判断重叠度误差是否符合重叠度误差范围；
第三确定单元，用于若重叠度误差符合重叠度误差范围，将重叠度误差符合重叠度误差范围的候选图像对作为目标图像对。
在一些实施例中，第一选择模块220进一步包括：
第四确定单元，用于确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
第四运算单元，用于计算第一相机中心和第二相机中心关于共视区域的法向量的平均方向角度差；
第五运算单元，用于根据平均方向角度差计算对称度误差；
第三判断单元，用于判断对称度误差是否符合对称度误差范围；
第五确定单元，用于若对称度误差符合对称度误差范围，将对称度误差符合对称度误差范围的候选图像对作为目标图像对。
在一些实施例中，第一选择模块220进一步包括：
第六确定单元，用于确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
第六运算单元，用于计算第一相机中心和第二相机中心到共视区域的平均距离；
第七运算单元，用于根据平均距离计算分辨率误差；
第四判断单元，用于判断分辨率误差是否符合分辨率误差范围；
第七确定单元，用于若分辨率误差符合分辨率误差范围，将分辨率误差符合分辨率误差范围的候选图像对作为目标图像对。
在一些实施例中，第一选择模块220进一步包括：
第八运算单元，用于将一候选图像对作为初始图像对，计算初始图像对在共视区域的初始平均覆盖率和初始覆盖率方差；
第五判断单元，用于计算其他候选图像对在共视区域的候选平均覆盖率和候选覆盖率方差，判断候选平均覆盖率是否大于初始平均覆盖率，判断候选覆盖率方差是否小于初始覆盖率方差；
第八确定单元，用于若候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差，将对应的候选图像对作为目标图像对。
在一些实施例中，第一条件还基于候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，第一选择模块220进一步包括：
第九运算单元，计算第一误差其中，μ1、μ2、μ3、μ4均为权重，μ1+μ2+μ3+μ4＝1，表示视差质量误差，表示重叠度误差，，表示对称度误差，表示分辨率误差；
第六判断单元，用于判断第一误差是否符合第一范围值；
第九确定单元，用于若第一误差符合第一范围值，将第一误差符合第一范围值的候选图像对作为目标图像对。
在一些实施例中，第一条件还基于候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，第一选择模块220进一步包括：
第十运算单元，用于根据视差质量误差、重叠度误差、对称度误差和分辨率误差计算第一误差；
第十一运算单元，用于将第一误差最小的候选图像对作为初始图像对，计算初始图像对在初始表面的初始平均覆盖率和初始覆盖率方差；
第七判断单元，用于计算其他候选图像对在初始表面的候选平均覆盖率和候选覆盖率方差，判断候选平均覆盖率是否大于初始平均覆盖率，判断候选覆盖率方差是否小于初始覆盖率方差；
第十确定单元，用于若候选平均覆盖率大于初始平均覆盖率且候选覆盖率方差小于初始覆盖率方差，将对应的候选图像作为待定图像对；
第十二运算单元，用于计算每一待定图像对的第一误差，将n个待定图像对的第一误差作为待定误差，n为大于0的正整数；
第八判断单元，用于将n个初始图像对的第一误差总和作为初始误差，判断待定误差是否小于或等于初始误差乘以预设系数，预设系数小于1大于等于0.8；
第十一确定单元，用于若待定误差小于或等于初始误差乘以预设系数，将待定误差小于或等于初始误差乘以预设系数的待定图像对作为目标图像对。
在一些实施例中，该装置还包括：
第二选择模块，用于根据第二条件选择图像的目标像素，第二条件根据图像的像素深度值设置；
第二计算模块，用于根据目标像素计算网格优化误差。
在一些实施例中，第二选择模块进一步包括：
第一划分单元，用于将图像分成多个候选区域，每一候选区域中具有中心像素和多个候选像素；
第十三运算单元，用于计算深度值差，深度值差为候选像素的像素深度值与中心像素的像素深度值之差；
第十二确定单元，用于根据深度值差将候选区域的所有候选像素分类为第一像素集合和第二像素集合，第二像素集合中最小深度值差大于第一像素集合中最大深度值差；
第十四运算单元，用于根据深度值差计算每一候选区域中第一像素集合的深度值方差；
第九判断单元，用于第二条件根据深度值方差设置，判断候选像素是否符合第二条件；
第十三确定单元，用于若深度值差符合第二条件，将符合第二条件的候选像素作为目标像素。
在一些实施例中，第十三确定单元进一步包括：
第十四确定单元，用于将符合的候选像素作为目标像素，其中，dd(h,k)表示第一像素集合中每一候选像素对应的深度值差，表示深度值方差，max{dd(h,k)}表示第一像素集合中最大深度值差。
图3示出了本发明实施例提供的计算设备的结构示意图，本发明具体实施例并不对计算设备的具体实现做限定。
如图3所示，该计算设备可以包括：处理器(processor)302、通信接口(Communications Interface)304、存储器(memory)306、以及通信总线308。
其中：处理器302、通信接口304、以及存储器306通过通信总线308完成相互间的通信。通信接口304，用于与其它设备比如客户端或其它服务器等的网元通信。处理器302，用于执行程序310，具体可以执行上述用于三维网格优化方法实施例中的相关步骤。
具体地，程序310可以包括程序代码，该程序代码包括计算机可执行指令。。
处理器302可能是中央处理器CPU，或者是特定集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本发明实施例的一个或多个集成电路。计算设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。
存储器306，用于存放程序310。存储器306可能包含高速RAM存储器，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。
本发明实施例还提供一种计算机可读存储介质，存储介质中存储有至少一可执行指令，可执行指令在运行时执行上述任一项的三维网格优化方法的操作。
在此提供的算法或显示不与任何特定计算机、虚拟系统或者其它设备固有相关。各种通用系统也可以与基于在此的示教一起使用。根据上面的描述，构造这类系统所要求的结构是显而易见的。此外，本发明实施例也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本发明的内容，并且上面对特定语言所做的描述是为了披露本发明的最佳实施方式。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本发明并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明实施例的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。
本领域技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。上述实施例中的步骤，除有特殊说明外，不应理解为对执行顺序的限定。

Claims (13)

  1. 一种三维网格优化方法，其特征在于，所述方法包括：
    获取多个图像中的多组候选图像对和三维模型的初始表面，每一所述候选图像对包括不同视角的第一候选图像和第二候选图像，所述初始表面具有所述第一候选图像和所述第二候选图像共同映射在所述初始表面的共视区域，所述初始表面具有至少一网格面片；
    根据第一条件选择多组所述候选图像对中的目标图像对，所述第一条件基于所述共视区域的面积设置；
    根据所述目标图像对计算网格优化误差。
  2. 根据权利要求1所述的三维网格优化方法，其特征在于，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
    计算所述第一相机中心和所述第二相机中心到所述共视区域的平均视差；
    根据所述平均视差计算视差质量误差；
    判断所述视差质量误差是否符合质量误差范围；
    若所述视差质量误差符合所述质量误差范围，将所述视差质量误差符合所述质量误差范围的所述候选图像对作为所述目标图像对。
  3. 根据权利要求1所述的三维网格优化方法，其特征在于，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    根据第一候选图像与第二候选图像映射在所述共视区域与第一候选图像映射在所述初始表面的第一区域，计算重叠度误差；
    判断所述重叠度误差是否符合重叠度误差范围；
    若所述重叠度误差符合重叠度误差范围，将所述重叠度误差符合所述重叠度误差范围的所述候选图像对作为所述目标图像对。
  4. 根据权利要求1所述的三维网格优化方法，其特征在于，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
    计算所述第一相机中心和所述第二相机中心关于所述共视区域的法向量的平均方向角度差；
    根据所述平均方向角度差计算对称度误差；
    判断所述对称度误差是否符合对称度误差范围；
    若所述对称度误差符合所述对称度误差范围，将所述对称度误差符合所述对称度误差范围的所述候选图像对作为所述目标图像对。
  5. 根据权利要求1所述的三维网格优化方法，其特征在于，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    确定第一候选图像的第一相机中心和第二候选图像的第二相机中心；
    计算所述第一相机中心和所述第二相机中心到所述共视区域的平均距离；
    根据所述平均距离计算分辨率误差；
    判断所述分辨率误差是否符合分辨率误差范围；
    若所述分辨率误差符合所述分辨率误差范围，将所述分辨率误差符合所述分辨率误差范围的所述候选图像对作为所述目标图像对。
  6. 根据权利要求1所述的三维网格优化方法，其特征在于，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    将一所述候选图像对作为初始图像对，计算所述初始图像对在所述共视区域的初始平均覆盖率和初始覆盖率方差；
    计算其他所述候选图像对在所述共视区域的候选平均覆盖率和候选覆盖率方差，判断所述候选平均覆盖率是否大于所述初始平均覆盖率，判断所述候选覆盖率方差是否小于所述初始覆盖率方差；
    若所述候选平均覆盖率大于所述初始平均覆盖率且所述候选覆盖率方差小于所述初始覆盖率方差，将对应的所述候选图像对作为所述目标图像对。
  7. 根据权利要求1-6任一项所述的三维网格优化方法，其特征在于，所述第一条件还基于所述候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    计算第一误差
    其中，μ1、μ2、μ3、μ4均为权重，μ1+μ2+μ3+μ4＝1，表示所述视差质量误差，表示所述重叠度误差，表示所述对称度误差，表示所述分辨率误差；
    判断所述第一误差是否符合第一范围值；
    若所述第一误差符合所述第一范围值，将所述第一误差符合所述第一范围值的所述候选图像对作为所述目标图像对。
  8. 根据权利要求1所述的三维网格优化方法，其特征在于，所述第一条件还基于所述候选图像对的视差质量误差、重叠度误差、对称度误差和分辨率误差设置，所述根据第一条件选择多组所述候选图像对中的目标图像对，进一步包括：
    根据所述视差质量误差、所述重叠度误差、所述对称度误差和所述分辨率误差计算第一误差；
    将所述第一误差最小的所述候选图像对作为初始图像对，计算所述初始图像对在所述初始表面的初始平均覆盖率和初始覆盖率方差；
    计算其他所述候选图像对在所述初始表面的候选平均覆盖率和候选覆盖率方差，判断所述候选平均覆盖率是否大于所述初始平均覆盖率，判断所述候选覆盖率方差是否小于所述初始覆盖率方差；
    若所述候选平均覆盖率大于所述初始平均覆盖率且所述候选覆盖率方差小于所述初始覆盖率方差，将对应的所述候选图像作为待定图像对；
    计算每一所述待定图像对的所述第一误差，将n个所述待定图像对的所述第一误差作为待定误差，n为大于0的正整数；
    将n个所述初始图像对的所述第一误差总和作为初始误差，判断所述待定误差是否小于或等于所述初始误差乘以预设系数，所述预设系数小于1大于等于0.8；
    若所述待定误差小于或等于所述初始误差乘以预设系数，将所述待定误差小于或等于所述初始误差乘以预设系数的所述待定图像对作为所述目标图像对。
  9. 根据权利要求1-6、8中任一项所述的三维网格优化方法，其特征在于，所述获取多个图像中的多组候选图像对和三维模型的初始表面之后，所述方法还包括：
    根据第二条件选择所述图像的目标像素，所述第二条件根据所述图像的像素深度值设置；
    根据所述目标像素计算网格优化误差。
  10. 根据权利要求9所述的三维网格优化方法，其特征在于，所述根据第二条件选择所述图像的目标像素，进一步包括：
    将所述图像分成多个候选区域，每一所述候选区域中具有中心像素和多个候选像素；
    计算深度值差，所述深度值差为所述候选像素的所述像素深度值与所述中心像素的所述像素深度值之差；
    根据所述深度值差将所述候选区域的所有所述候选像素分类为第一像素集合和第二像素集合，所述第二像素集合中最小所述深度值差大于所述第一像素集合中最大所述深度值差；
    根据所述深度值差计算每一所述候选区域中所述第一像素集合的深度值方差；
    所述第二条件根据所述深度值方差设置，判断所述候选像素是否符合所述第二条件；
    若所述深度值差符合所述第二条件，将符合所述第二条件的所述候选像素作为所述目标像素。
  11. 根据权利要求10所述的三维网格优化方法，其特征在于，若所述候选像素符合所述第二条件，将符合所述第二条件的所述候选像素作为所述目标像素，进一步包括：
    将符合
    的所述候选像素作为所述目标像素，其中，dd(h,k)表示所述第一像素集合中每一所述候选像素对应的所述深度值差，表示所述深度值方差，max{dd(h,k)}表示所述第一像素集合中最大所述深度值差。
  12. 一种计算设备，其特征在于，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
    所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如权利要求1-11中任一项所述的三维网格优化方法的操作。
  13. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一可执行指令，所述可执行指令在运行时执行如权利要求1-11中任一项所述的三维网格优化方法的操作。
PCT/CN2023/116808 2022-09-20 2023-09-04 三维网格优化方法、设备及存储介质 WO2024060981A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211143432.9 2022-09-20
CN202211143432.9A CN115272618B (zh) 2022-09-20 2022-09-20 三维网格优化方法、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024060981A1 true WO2024060981A1 (zh) 2024-03-28

Family

ID=83756151

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/116808 WO2024060981A1 (zh) 2022-09-20 2023-09-04 三维网格优化方法、设备及存储介质

Country Status (2)

Country Link
CN (1) CN115272618B (zh)
WO (1) WO2024060981A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272618B (zh) * 2022-09-20 2022-12-20 深圳市其域创新科技有限公司 三维网格优化方法、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110132242A (zh) * 2018-02-09 2019-08-16 驭势科技（北京）有限公司 多摄像机即时定位与地图构建的三角化方法及其运动体
CN110349251A (zh) * 2019-06-28 2019-10-18 深圳数位传媒科技有限公司 一种基于双目相机的三维重建方法及装置
CN111784841A (zh) * 2020-06-05 2020-10-16 中国人民解放军军事科学院国防科技创新研究院 重建三维图像的方法、装置、电子设备及介质
CN111914715A (zh) * 2020-07-24 2020-11-10 廊坊和易生活网络科技股份有限公司 一种基于仿生视觉的智能车目标实时检测与定位方法
WO2022143237A1 (zh) * 2020-12-31 2022-07-07 华为技术有限公司 一种目标定位的方法、系统及相关设备
CN115272618A (zh) * 2022-09-20 2022-11-01 深圳市其域创新科技有限公司 三维网格优化方法、设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2991029A1 (en) * 2014-08-29 2016-03-02 Thomson Licensing Method for inserting features into a three-dimensional object and method for obtaining features from a three-dimensional object
CN109146941A (zh) * 2018-06-04 2019-01-04 成都通甲优博科技有限责任公司 一种基于网格区域划分的深度图像优化方法及系统
CN113610952A (zh) * 2020-04-20 2021-11-05 广州极飞科技股份有限公司 一种三维场景重建方法、装置、电子设备和存储介质
WO2022021309A1 (zh) * 2020-07-31 2022-02-03 深圳市大疆创新科技有限公司 建立模型的方法、装置、电子设备及计算机可读存储介质
CN114419278A (zh) * 2022-01-19 2022-04-29 厦门大学 室内三维彩色网格模型生成方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110132242A (zh) * 2018-02-09 2019-08-16 驭势科技（北京）有限公司 多摄像机即时定位与地图构建的三角化方法及其运动体
CN110349251A (zh) * 2019-06-28 2019-10-18 深圳数位传媒科技有限公司 一种基于双目相机的三维重建方法及装置
CN111784841A (zh) * 2020-06-05 2020-10-16 中国人民解放军军事科学院国防科技创新研究院 重建三维图像的方法、装置、电子设备及介质
CN111914715A (zh) * 2020-07-24 2020-11-10 廊坊和易生活网络科技股份有限公司 一种基于仿生视觉的智能车目标实时检测与定位方法
WO2022143237A1 (zh) * 2020-12-31 2022-07-07 华为技术有限公司 一种目标定位的方法、系统及相关设备
CN115272618A (zh) * 2022-09-20 2022-11-01 深圳市其域创新科技有限公司 三维网格优化方法、设备及存储介质

Also Published As

Publication number Publication date
CN115272618B (zh) 2022-12-20
CN115272618A (zh) 2022-11-01

Similar Documents

Publication Publication Date Title
WO2018214505A1 (zh) 立体匹配方法与系统
CN106780590B (zh) 一种深度图的获取方法及系统
WO2021174939A1 (zh) 人脸图像的获取方法与系统
CN107230225B (zh) 三维重建的方法和装置
CN109493375B (zh) 三维点云的数据匹配及合并方法、装置、可读介质
KR102318023B1 (ko) 에지를 이용한 3차원 모델 생성
WO2024060981A1 (zh) 三维网格优化方法、设备及存储介质
WO2017091927A1 (zh) 图像处理方法和双摄像头系统
US10957062B2 (en) Structure depth-aware weighting in bundle adjustment
CN112561978B (zh) 深度估计网络的训练方法、图像的深度估计方法、设备
CN110996082B (zh) 投影调节方法、装置、投影仪及可读存储介质
CN110223222B (zh) 图像拼接方法、图像拼接装置和计算机可读存储介质
TW201917562A (zh) 資料處理方法、裝置、系統和儲存介質
CN114387319B (zh) 点云配准方法、装置、设备以及存储介质
CN104574508A (zh) 一种面向虚拟现实技术的多分辨率模型简化方法
WO2022061850A1 (zh) 点云运动畸变修正方法和装置
WO2023160426A1 (zh) 视频插帧方法、训练方法、装置和电子设备
CN114926316A (zh) 距离测量方法、装置、电子设备及存储介质
US20210118172A1 (en) Target detection method, target detection apparatus, and unmanned aerial vehicle
WO2022120988A1 (zh) 基于混合2d卷积和伪3d卷积的立体匹配方法
CN111179327B (zh) 一种深度图的计算方法
CN113435367A (zh) 社交距离评估方法、装置及存储介质
US20230384085A1 (en) Phase unwrapping method based on multi-view constraints of light field and related components
CN117152330A (zh) 一种基于深度学习的点云3d模型贴图方法和装置
CN107247833B (zh) 一种云计算下的cae海量数据轻量化方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23867272

Country of ref document: EP

Kind code of ref document: A1