WO2016179825A1 - 基于三维场景的导航方法 - Google Patents

基于三维场景的导航方法 Download PDF

Info

Publication number
WO2016179825A1
WO2016179825A1 PCT/CN2015/078949 CN2015078949W WO2016179825A1 WO 2016179825 A1 WO2016179825 A1 WO 2016179825A1 CN 2015078949 W CN2015078949 W CN 2015078949W WO 2016179825 A1 WO2016179825 A1 WO 2016179825A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
building
sampling point
value
point
Prior art date
Application number
PCT/CN2015/078949
Other languages
English (en)
French (fr)
Inventor
黄惠
郝竹明
龚明伦
利辛斯基•丹尼尔
科恩•丹尼尔
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2015/078949 priority Critical patent/WO2016179825A1/zh
Publication of WO2016179825A1 publication Critical patent/WO2016179825A1/zh
Priority to US15/390,000 priority patent/US10066956B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3602—Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/003—Navigation within 3D models or images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/3635—Guidance using 3D or perspective road maps
    • G01C21/3638—Guidance using 3D or perspective road maps including 3D objects and buildings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30184—Infrastructure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present invention relates to the field of computer graphics, and in particular, to a navigation method based on a three-dimensional scene.
  • the current automatic navigation methods for 3D scenes mainly have the following problems: First, many navigation technologies are based on fixed camera speed, viewing angle, and altitude navigation methods, without considering the characteristics of the scene and the user's attention; second, Many navigation technologies are not fully fully automated. Most of them require users to do a lot of calibration and input. Third, existing navigation technologies have strong limitations in use and can only be used in a specific scene. Fourth, navigation The visual experience in the process is not continuous enough and not smooth enough.
  • the present invention proposes a navigation method based on a three-dimensional scene to overcome one or more problems in the prior art.
  • the present invention provides a navigation method based on a three-dimensional scene, the method comprising: calculating an interest value of a camera's perspective based on height, volume, irregularity, and uniqueness of a building in the scene; generating an interest value according to the perspective A trajectory parameter of the camera to navigate according to the trajectory parameter.
  • calculating the interest value of the camera's perspective based on the height, volume, irregularity, and uniqueness of the building in the scene includes: based on the height, volume, irregularity, and uniqueness of the building Calculating an importance value of the building; generating an interest value map of the viewing angle according to the importance value of the building; correcting the interest value map by a center weight and a depth weight; and inversely calculating the corrected interest value Figure, the interest value of the view angle is obtained.
  • the generating the trajectory parameter of the camera according to the interest value of the view includes: step 101: selecting a planned route in the scene, uniformly sampling the planned route, and obtaining a plurality of sampling points; Step 102: Perform weighted smoothing on the planned route according to the interest value of the viewing angle at the sampling point, and use the position corresponding to the sampling point on the planned route after weighting smoothing as the corrected sampling.
  • step 103 correcting the camera motion speed at the sampling point according to the set total navigation time and the interest value of the angle of view at the sampling point; step 104: according to the corrected sampling point
  • the camera motion speed at the correction corrects the camera pose at the sampling point; wherein the corrected camera pose at the sampling point and the corrected camera motion speed at the sampling point are the trajectory parameters of the camera.
  • generating the trajectory parameter of the camera according to the interest value of the viewing angle further comprises: setting an initial camera focus point, an initial camera motion speed, and an initial camera posture at the sampling point.
  • the generating a trajectory parameter of the camera according to the interest value of the viewing angle further comprising: according to the corrected camera focus point at the sampling point, the corrected camera motion speed at the sampling point, and Correcting the camera posture at the sampling point, re-acquiring the interest value of the viewing angle at the sampling point, if the difference between the next interest value of the viewing angle at the sampling point and the previous interest value is greater than a setting a threshold value, wherein a previous interest value of the angle of view at the sampling point is replaced with a previous interest value of the angle of view at the sampling point, and a previous camera focus point at the sampling point is used to replace the previous point at the sampling point
  • One camera focus point, replacing the previous camera motion speed at the sampling point with the next camera motion speed at the sampling point, replacing the previous time at the sampling point with the last camera posture at the sampling point The camera pose repeats the step 102, the step 103, and the step 104.
  • the importance values of the building are:
  • ⁇ , ⁇ , ⁇ , ⁇ are weight coefficients
  • S h (b) is the high interest value of building b
  • S v (b) is the volume importance value of building b
  • S r (b) is the building
  • S u (b) is the uniqueness value of the building b
  • the building b is the building;
  • height(b) is the height of the building b
  • ⁇ height(c) is the set of heights of the building c near the planned route
  • volume(b) is the volume of building b
  • ⁇ volume(c) is the set of volumes of building c near the planned route
  • volume (MVBB(b)) is the volume of the body bounding box MVBB(b) of building b
  • ⁇ b is the set of buildings d within a predetermined range near the building b, and the uniqueness difference value of the building b and the building d within the predetermined range
  • volume ( ⁇ (MVBB(b), MVBB(d))) is the result of the intersection of the body bounding box MVBB(b) of the building b and the body bounding box MVBB(d) of the building d (MVBB(b) , MVBB(d)) volume
  • volume (U(MVBB(b), MVBB(d)))) is the phase bounding box MVBB(b) of building b and the body bounding box MVBB(d) of building d
  • the result is the volume of U (MVBB(b), MVBB(d)).
  • the three-dimensional scene-based navigation method further includes:
  • the weight coefficients ⁇ , ⁇ , ⁇ , ⁇ are obtained by solving an optimization weight equation, and the optimization weight equation is:
  • R i is a given user score based on the height, volume, irregularity, and uniqueness of a building in a given building set
  • R( ⁇ , ⁇ , ⁇ , ⁇ ) is given according to a group
  • K(R( ⁇ , ⁇ , ⁇ , ⁇ ), R i ) is the score value R i of the given user and the importance value R ( ⁇ , ⁇ , ⁇ of the building in the given building set) a first distance between ⁇ ), if the first distance is less than a set distance, the value of the set of given weight coefficients is taken as the value of the weight coefficients ⁇ , ⁇ , ⁇ , ⁇ .
  • the interest value map is corrected in the pass center weight and depth weight:
  • the center weight is Where i is the position of the pixel in the interest value map, o is the center of the interest value map, and r is half the diagonal length of the interest value map;
  • the depth weight is Where d * is a set depth of observation and d(i) is the depth of observation at position i of the pixel;
  • N is the number of pixels of the interest value map, N ⁇ 1, N is an integer, j is the sequence number of the sampling point, j ⁇ [0,n], n>1, n is a positive integer, S( i) is the value of interest of the view at position i of the pixel in the interest value map.
  • the position of the camera focus point at the corrected sampling point is
  • M is the number of sampling points that the tracking target of the camera at the sampling point has passed within a predetermined time
  • M is an integer
  • M ⁇ 1 is the camera at the hth sampling point.
  • p h is the camera at the hth sampling point. The position of the initial focus point.
  • the step 103 includes:
  • the optimization time t j is the motion time of the camera from the position of the camera at the jth sampling point to the position of the camera at the j+1th sampling point, and the most constrained equation is:
  • I j is the interest value of the view angle at the jth sampling point
  • T is the total navigation time
  • the corrected camera pose at the sampling point is obtained by a minimum energy equation
  • E d (c j , f j , v j ) is the distance term
  • E p (c j , d j , f j ) is the projection term
  • E s (c j , d j )) is the smooth term
  • a. b, c is a predetermined coefficient
  • the desired distance between the initial position c j of the camera at the jth sampling point and the position f j of the initial camera focus point at the sampling point is ⁇ is a given angle value, ⁇ is the given motion time of the tracking target, Is the vertical component of c j , f j z is the vertical component of f j ;
  • v min is the minimum of all camera motion speeds at the sample points after correction
  • v max is the maximum of all camera motion speeds at the sample points after correction
  • ⁇ max is all said a preset maximum pitch angle value in a pitch angle of the camera at the sampling point, ⁇ min being a preset minimum pitch angle value among the camera's pitch angles at all of the sampling points;
  • R(d j ) is a projection unit vector of the tracking target on the navigation image at the jth sampling point
  • d j is a unit vector of the initial orientation of the camera at the jth sampling point
  • ⁇ 1 and ⁇ 2 are predetermined constants
  • d j-1 is a unit vector of the initial orientation of the camera at the j-1th sampling point
  • c j-1 is the j-1th sampling point
  • c j-2 is the initial position of the camera at the j-2th sampling point.
  • the interest value map in the interest value map that generates the perspective of the camera according to the importance value of the building: the interest value map is a color energy map.
  • the center weight is reduced by a trigonometric function from the center to the outside.
  • the first distance is a distance calculated according to the Kendall tau method.
  • the optimization weight equation is solved by a random search method or a quasi-Newton method.
  • the three-dimensional scene-based navigation method calculates the interest value of the viewing angle based on the height, volume, irregularity and uniqueness of the building, and has high navigation quality.
  • the embodiment of the present invention can automatically perform importance analysis on a given scene, and generate an adaptive speed, a view angle, and a height according to the interest value of the view, and ensure a smooth camera track, which is a very effective fully automatic navigation path. The way it is generated.
  • the embodiment of the invention automatically analyzes the importance value of the viewing angle, and calculates a smooth camera trajectory after the adaptive camera speed and the camera posture based on the analysis result, and the user interaction is particularly simple.
  • FIG. 1 is a schematic flow chart of a navigation method based on a three-dimensional scene according to an embodiment of the present invention
  • FIG. 2 is a schematic flow chart of calculating an interest value of a viewing angle in an embodiment of the present invention
  • FIG. 3 is a schematic flow chart of generating a trajectory parameter in an embodiment of the present invention.
  • FIG. 4 is a schematic flow chart of generating a trajectory parameter in an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of generating a trajectory parameter according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a building set used for calculating a weight coefficient according to an embodiment of the present invention.
  • Figure 7 is a result of the user's weighted scoring of the building in Figure 6;
  • 8A-8C are schematic diagrams of textures of scenes at three different viewing angles
  • FIGS. 8D-8F are schematic diagrams showing effects before weighting corresponding to the viewing angles of Figs. 8A-8C, respectively, according to an embodiment of the present invention.
  • FIGS. 8A-8I are schematic diagrams showing effects before weighting corresponding to the viewing angles of FIGS. 8A-8C, respectively, according to an embodiment of the present invention.
  • FIG. 9 is a schematic flow chart of an initialization process in an embodiment of the present invention.
  • FIG. 10A is a schematic diagram showing smooth results after two iterations according to an embodiment of the present invention.
  • FIG. 10B is a schematic diagram of a trajectory of a camera corresponding to a smooth result after the second iteration of FIG. 10A;
  • Figure 11 is a diagram showing changes in the desired distance and camera posture as speed increases in accordance with an embodiment of the present invention.
  • FIG. 1 is a schematic flow chart of a navigation method based on a three-dimensional scene according to an embodiment of the present invention. As shown in FIG. 1, the navigation method of the embodiment of the present invention includes:
  • Step S101 calculating an interest value of a camera's perspective based on the height, volume, irregularity, and uniqueness of the building in the scene;
  • Step S102 Generate a trajectory parameter of the camera according to the interest value of the perspective to navigate according to the trajectory parameter.
  • the navigation method based on the three-dimensional scene in the embodiment of the present invention fully considers the user's interest in the building in the scene, including the height, volume, irregularity and uniqueness of the building, so as to satisfactorily satisfy the user in the navigation image.
  • the embodiments of the present invention illustrate the advantages of the present invention in a building, but those skilled in the art know that the navigation method based on the three-dimensional scene of the embodiment of the present invention can also be considered in the scene.
  • Other objects such as transportation equipment, trees, etc. are important to accommodate the needs of different navigation scenarios.
  • each camera position corresponds to one view, and each view corresponds to one scene, and each scene corresponds to a plurality of buildings.
  • the importance value of each building in the scene corresponding to the viewing angle is obtained, and then the importance value of the building is further processed, and the interest value of the viewing angle can be obtained.
  • Step S101 can include:
  • Step S201 calculating an importance value of the building based on height, volume, irregularity, and uniqueness of the building;
  • Step S202 Generate an interest value map of the viewing angle according to the importance value of the building
  • Step S203 correcting the interest value map by using a center weight and a depth weight
  • Step S204 Backcalculating the corrected interest value map to obtain the interest value of the perspective.
  • the method for calculating the interest value of the perspective in the embodiment of the present invention not only considers the importance value of the building in the scene, but also incorporates the user's more attention to the central area through the center weight, and incorporates the user's favorite interval in the depth weight. Observe the factors of the building at a certain distance, and further make the navigation results closer to the user's attention.
  • the angle of view of each camera corresponds to a trajectory parameter of a group of cameras
  • the trajectory parameters of the camera may include a camera focus point, a camera posture, a camera motion speed, and a speed of the tracking target, and the camera posture may use the camera three-dimensional position, the camera
  • the parameters such as orientation, camera pitch angle, and camera yaw angle are indicated.
  • the step S102 of generating a trajectory parameter of the camera according to the interest value of the view to navigate according to the trajectory parameter may include:
  • Step S301 Select a planned route in the scenario, and uniformly sample the planned route to obtain multiple sampling points;
  • Step S303 performing weighted smoothing on the planned route according to the interest value of the viewing angle at the sampling point, and using the position corresponding to the sampling point on the planned route after weighting smoothing as the corrected sampling.
  • Step S304 correcting the camera moving speed at the sampling point according to the set total navigation time and the interest value of the viewing angle at the sampling point;
  • Step S305 Correct the camera posture at the sampling point according to the camera movement speed at the corrected sampling point.
  • the corrected camera posture at the sampling point and the corrected camera motion speed at the sampling point are used as the trajectory parameters of the camera.
  • the three-dimensional scene-based navigation method of the embodiment of the present invention divides the huge constrained optimization problem into a series of small optimization steps, and each optimization step is more and more controllable, and then iteratively solves these optimization steps.
  • the embodiment of the invention resolves the contradiction existing in the three-dimensional scene navigation, and can bring the navigation experience more suitable to the user.
  • the parameters of the camera at each of the above sampling points are initialized as an initial value of the subsequent correction (optimization) step before the start of the above step S303.
  • generating a trajectory parameter of the camera may further include:
  • Step S302 setting an initial camera focus point, an initial camera motion speed, and an initial camera posture at the sampling point.
  • the appropriate initial value is optimized or corrected by setting an appropriate initial value for a plurality of parameters to achieve a better navigation effect.
  • the first corrected camera pose and the camera motion speed may not be directly used as the trajectory parameters of the camera, but multiple iteration calculations may be performed to obtain better trajectory parameters.
  • FIG. 5 is a schematic flow chart of generating trajectory parameters in an embodiment of the present invention. As shown in FIG. 5, generating a trajectory parameter of the camera may further include:
  • Step S306 If the difference between the next interest value of the view angle at the sampling point and the previous interest value is greater than a set threshold, repeating the iterative execution of the step S303, the step S304, and the step S305.
  • the next interest value of the view angle at the sampling point is based on the camera focus point at the sampling point after the previous correction (for example, the first correction), the previous time.
  • the camera movement speed at the sampling point after the correction (for example, the first correction) and the camera posture at the sampling point after the previous correction (for example, the first correction) are obtained.
  • the previous interest value of the view angle at the sampling point (for example, the second interest value) is used to replace the previous time of the view angle at the sampling point.
  • a value of interest eg, a first interest value/initial interest value
  • replacing the previous camera focus point at the sample point with a subsequent camera focus point eg, a second camera focus point
  • the sample point eg, First camera focus point / initial camera focus point
  • replacing the previous camera motion speed at the sampling point with the next camera motion speed (eg, the second camera motion speed) at the sampling point eg, first Camera movement speed/initial camera movement speed
  • replacing the previous camera posture first camera posture/initial camera posture
  • second camera posture at the sampling point.
  • the difference between the next interest value of the view angle at the sampling point (for example, the second interest value) and the previous interest value (for example, the first interest value/initial interest value) is less than or equal to the above set threshold value.
  • the iterative algorithm described above may end at the third iteration.
  • the initial viewing angle interest value (the first interest value) is obtained, and the initial camera focusing point, the initial camera moving speed, and the initial camera posture are obtained.
  • Two viewing angle interest values if the difference between the second viewing angle interest value and the initial viewing angle interest value (the first interest value) is greater than the above set threshold
  • a third camera focus point, a third camera motion speed, and a third camera pose are obtained.
  • the iterative calculation is performed in turn until the difference between the next interest value of the viewing angle and the previous interest value is less than or equal to the set threshold, and n times of optimization or correction is performed, and the camera pose and camera motion obtained by the nth optimization or correction are obtained.
  • Speed is taken as the trajectory parameter of the camera, where n ⁇ 1, and n is an integer.
  • the trajectory parameters of the camera are gradually optimized by an iterative correction method, so that the navigation effect is closer to the user's expectation and demand.
  • the height, volume, irregularity and uniqueness of the building can be incorporated into the importance value of the building through a plurality of different models.
  • the importance value of the building can be calculated by the following model, the importance values of the building are:
  • ⁇ , ⁇ , ⁇ , ⁇ are weight coefficients
  • S h (b) is the high interest value of building b
  • S v (b) is the volume importance value of building b
  • S r ( b) is the irregularity value of the building b
  • S u (b) is the uniqueness value of the building b
  • the building b is the above building.
  • the high interest values of the above building b are:
  • height (b) is the height of the building b
  • ⁇ height (c) is a set of the heights of the building c in the vicinity of the planned route.
  • the high interest value of the building in the embodiment of the invention takes into account the effect of altitude on the user's point of interest.
  • the high interest value in the embodiment of the present invention is a normalized height score item, taking into account the relative height of the buildings in the scene.
  • a skyscraper is usually a landmark or landscape of a city, and the high interest value of the above buildings takes this factor into account.
  • the volume importance values of the above building b are:
  • volume (b) is the volume of the building b
  • ⁇ volume (c) is the set of the volume of the building c in the vicinity of the planned route.
  • Volume (b) in equation (3) measures the outer shell volume of building b. Since the model of the building b is not necessarily watertight, in one embodiment, the volume volume(b) of the building b can be calculated by two parallel projected depth images rendered from the front and back of the building b, respectively. The depth interval of the building b at the corresponding pixel point in the depth image is accumulated to obtain a volume value similar to the volume of the building b.
  • the method for calculating the volume of a building according to an embodiment of the present invention is accurate when the building b has no hole structure along the projection direction, and most buildings can satisfy such structural conditions, so the above method for calculating the volume of the building can be The effect of the volume of the building on the value of interest of the viewing angle is more precisely considered.
  • the volume importance value in the embodiment of the invention takes into account the effect of volume on the user's point of interest.
  • Large-volume building structures in the scene, such as stadiums or shopping centers, are often used as landmarks in navigation, and the above-mentioned volume importance values can include this factor.
  • the irregularity values of the above building b are:
  • volume (MVBB(b)) is the volume of the body bounding box MVBB(b) of the building b.
  • the irregularity value of a building can also be called the value of the opposite sex.
  • the embodiment of the present invention divides the building into a plurality of small body bounding boxes MVBB, and the irregularity of the building b is defined as the difference between the volume and the MVBB of the box, thereby quantifying the building The irregularity of b.
  • ⁇ b is a set of buildings d within a predetermined range near the building b
  • M(b, d) is a unique difference value of the building b and the building d within the predetermined range .
  • volume ( ⁇ (MVBB(b), MVBB(d)))) is the result of the intersection of the body bounding box MVBB(b) of the building b and the body bounding box MVBB(d) of the building d.
  • (MVBB(b), MVBB(d)) volume, volume (U(MVBB(b), MVBB(d)))) is the body bounding box MVBB(b) of building b and the body bounding box MVBB of building d
  • the phase of the result of (d) is the volume of U (MVBB(b), MVBB(d)).
  • the model of the uniqueness value of the building b in the embodiment of the present invention mainly considers the difference between the building and the building adjacent thereto, quantifies the uniqueness value of the building, and simplifies the solution of the uniqueness value of the building. process.
  • the uniqueness values of embodiments of the present invention are obtained by evaluating their body bounding boxes. In a scene, such as a very unique building The building is generally more attractive to the user, and the value of the uniqueness of the building in the embodiment of the present invention can take into account this influencing factor.
  • the weighting coefficient in the above formula (1) can be obtained by various methods. In one embodiment, it can be obtained by solving an optimization weighting equation, wherein the optimization weighting equation is:
  • R i is a given user score based on the height, volume, irregularity, and uniqueness of the building in a given building set
  • R( ⁇ , ⁇ , ⁇ , ⁇ ) is Calculated from the values of a given set of weight coefficients and the S h (b), S v (b), S r (b), and S u (b) values of the building in the given building set
  • K(R( ⁇ , ⁇ , ⁇ , ⁇ ), R i ) is the given user score value R i and the importance value R of the building in the given building set ( a first distance between ⁇ , ⁇ , ⁇ , ⁇ ), if the first distance is less than a set distance, the value of the set of given weight coefficients is used as the weight coefficient ⁇ , ⁇ , ⁇ , ⁇ Value.
  • the first distance K (R( ⁇ , ⁇ , ⁇ , ⁇ ), R i ) can be calculated by the Kendall tau method, wherein the Kendall tau method is published by Kendall et al. in Biometrica (1983) , Vol. 30, pp. 81-39) The method used in the article "A new measure of rank correlation. Biometrica".
  • FIG. 6 is a block diagram showing the structure of a building set used to calculate weight coefficients in an embodiment of the present invention.
  • the buildings contained in the scene of a three-dimensional virtual city constitute a collection of buildings.
  • a plurality of users are required to sort each building in the building collection according to the size of the interest in the building in Fig. 6, and obtain a plurality of sets of weighted scoring results, as shown in Fig. 7.
  • the weighting coefficients of the items are obtained by fitting using the formula (7).
  • Figure 7 shows the importance values of the buildings in Figure 6, and the histogram corresponding to the importance values is drawn.
  • the higher buildings #2 and #5 have the highest height interest value and volume importance value score; the irregularities of the irregularly shaped buildings #9 and buildings #3
  • the scores of the building #1, building #8 and building #11 are relatively similar, and their uniqueness values are very low; compared with the other buildings in Fig. 6, the volume of the building #12 appears Very small, its uniqueness score is higher.
  • the scores of the height, volume, irregularity, and uniqueness of the building are weighted. From the histogram in Fig. 7, the building #2 and the building #5 are of the highest importance, and the building #12 The least importance. This result is also consistent with the statistical results of user surveys. This further demonstrates that the model of the height, volume, irregularity, and uniqueness of the building of the embodiment of the present invention is extremely effective.
  • step S202 is performed to generate an interest value map of the perspective corresponding to the scene, for example, the interest value map is a color energy map.
  • the importance values of each building are mapped to a color model scene whose color values are represented by red to blue colors, and warm colors represent high scores.
  • a two-dimensional energy map containing the faces of all building models at that particular perspective can be obtained, and the interest value score of the viewing angle can be simply accumulated by all the pixel values in the energy map of the viewing angle.
  • step S203 by introducing the center weight and the depth weight to correct the interest value map (interest value) of the perspective, a more reasonable perspective interest value score can be obtained.
  • the center weight may be a trigonometric function from the center to reduce the weight so that the center position of the image has a high interest value score, and the position far from the center has a lower interest value score, thereby satisfying the user more attention.
  • the needs of the central area may be a trigonometric function from the center to reduce the weight so that the center position of the image has a high interest value score, and the position far from the center has a lower interest value score, thereby satisfying the user more attention.
  • the center weight can be expressed as:
  • i is the position of the pixel in the above interest value map
  • o is the center of the above interest value map
  • r is half the diagonal length of the interest value map.
  • the depth weight is a distance based weight ⁇ d .
  • the projected pixel area of the surface of the scene model is A
  • the distance-based weight term depth weight
  • d * is a set observation depth
  • d(i) is the observation depth at the position i of the pixel in the image.
  • the depth weights in the embodiments of the present invention take into account the above factors, so that it is possible to prevent the buildings in the short distance from causing excessive adverse effects on the interest value of the viewing angle.
  • the interest value of the perspective corresponding to the interest value map that is, the sum of the scores of the entire interest value of the specific perspective can be expressed as:
  • N is the number of pixels of the interest value map, N ⁇ 1, N is an integer, j is the serial number of the sampling point, j ⁇ [0,n], n>1, n is A positive integer, S(i) is the interest value of the view angle at the position i of the pixel in the interest value map.
  • Both the original interest value map and the weighted range can be normalized between [0, 1], so the weighted interest value map can also be within this range.
  • Figure 8G corresponds to the first perspective of interest
  • I 0.199
  • the three-dimensional scene-based navigation method of the embodiment of the present invention grasps the essence of efficient navigation, that is, the control of the camera.
  • the camera's trajectory parameters look for a sequence of camera positions, including the camera's three-dimensional position and camera orientation, and study the camera motion speed at the camera's position. Regardless of the change in yaw angle, the position of the camera is represented by five degrees of freedom, including the camera's three-dimensional position, camera orientation, camera pitch angle, camera motion speed, and tracking target motion speed.
  • the navigation method of the embodiment of the present invention is based on some key ideas.
  • calculate the optimization speed of the tracking target for each iteration (same as the camera motion speed).
  • the interest value of the determined viewing angle is obtained, and the speed of the tracking target is obtained by a simple solvable solution.
  • the motion path is adaptively smoothed, thereby generating a series of camera positions and camera focus points depending on the speed of the tracking target (the area where the high speed motion is smoothed more).
  • the optimized camera pose at each focus point is calculated by solving a minimum target equation.
  • a planned route is given, and is uniformly sampled along the planned route, for example, one sampling point (dense sampling) is taken every 10 m, and the positions of the sampling points are marked as p j , j ⁇ [0, n] Each p j is a three-dimensional position on the path.
  • Figure 9 is a flow chart showing the initialization process in one embodiment of the present invention.
  • the step S302 of setting an initial camera focus point, an initial camera motion speed, and an initial camera posture at the sampling point may include:
  • Step S901 setting the position f j of the initial camera focus point to the position p j of the sampling point, where j represents the serial number of the sampling point;
  • Step S902 setting the initial camera motion speed to a uniform speed
  • Step S903 setting the initial camera posture to ⁇ c j , d j >.
  • c j f j-2 +[0,0,e]
  • c j is the initial position of the camera at the jth sampling point
  • d j is the unit vector of the initial orientation of the camera at the jth sampling point
  • e is the initial of the camera and the ground at the sampling point
  • the height, f j-2 is the position of the initial camera focus point at the j-2th sampling point, j ⁇ [0, n], n > 1, and n is a positive integer.
  • the aspect ratio of the screen is 16:9
  • the corresponding horizontal and vertical fields of view are 60° and 36°, respectively
  • the camera's pitch angle is raised by 6° in the direction vector, thus ensuring that the camera focus point is always At the bottom 1/3 of the frame.
  • the orientation of the camera is determined by the horizontal portion of dj .
  • the initial moving speed of the tracking target and the camera are both constant speeds, so the camera will always be a constant distance and height from the tracking target.
  • the optimized time tj is found , ie the tracking target is at pj (the position of the jth sample point) to pj+1 (j+1th) The movement time between the positions of the sampling points).
  • Each camera focus point corresponds to a camera pose ⁇ c j , d j >, and a view angle corresponding to a camera pose can render a map of interest values, thereby correspondingly calculating an interest value score I j .
  • t j can be solved first by solving the following most constrained equation:
  • set I j is the interest value of the view angle at the jth sampling point, and T is the total navigation time.
  • the function f(t j ) determines how the speed changes with the value of interest. It guarantees a strong nonlinear correlation between speed and interest value, and at the same time, a simple and solvable solution is obtained, which is to multiply the high-dimensional vector by the maximum point.
  • can be a constant, and the maximization of this point multiplication only needs to guarantee ⁇ 0, in other words, the two vectors need to be collinear.
  • Set a constant ⁇ so that For each j, Set each segment of the planned route from p j to p j+1 to the distance ⁇ of equal length, then the velocity v j can be simply expressed as ⁇ /t j , then optimize the speed (corrected camera motion speed / tracking target speed) )Satisfy C is a constant (C ⁇ / ⁇ 2 ), and
  • the interest value of the angle of view in the embodiment of the present invention is limited to a range, so the camera motion speed (the speed of the tracking target) is also limited to a range.
  • the navigation method of the embodiment of the present invention can overcome the above disadvantages by performing adaptive smoothing according to the camera motion speed in each iteration.
  • the position of the focus point of the camera is recalculated according to the trajectory of the adaptive weighted smoothing camera of the camera motion speed, and the corrected position of the camera focus point at the sampling point is:
  • M is the number of sampling points that the tracking target of the camera at the sampling point has traveled within a predetermined time, for example, the predetermined time is 6 seconds, then M is tracking in 6 seconds.
  • the number of the target passing through the sampling point, M is an integer, M ⁇ 1, and p h is the position of the initial focus point of the camera at the hth sampling point.
  • Figure 10A is a schematic illustration of smoothing results after two iterations in accordance with one embodiment of the present invention. As shown in FIG. 10A, after correction, the focus point of the camera is more in line with the user's expectation.
  • Embodiment corresponds to the focal point, j machine according to the velocity v of each sample point correction at p j / optimize one embodiment, the camera should be tracked and f j (correction / optimized), can adjust the position of the camera. For each camera focus point f j , the camera pose ⁇ c j , d j > is recalculated.
  • the corrected camera pose at the sampling point is obtained by a minimum energy equation, which is:
  • E d (c j , f j , v j ) is the distance term
  • E p (c j , d j , f j ) is the projection term
  • E s (c j , d j )) is For smooth terms, a, b, and c are predetermined coefficients.
  • the distance term E d ensures that the camera and the focus point are at an appropriate distance and height.
  • the smooth term E s attenuates a large change in camera pose between adjacent focus points.
  • the distance term is determined by the two constraints of the desired pitch angle ⁇ and the desired distance D:
  • the expected distance between the initial position c j of the camera at the jth sampling point to the position f j of the initial camera focus point at the sampling point is:
  • ⁇ is a given angle value
  • ⁇ is the given motion time of the tracking target. Is the vertical component of c j , f j z is the vertical component of f j .
  • ⁇ v j is the distance that the tracking target moves within ⁇ seconds. As shown in FIG. 11, the desired distance D(v j ) is guaranteed to be within an angular range in which the moving distance includes ⁇ .
  • the constant parameter ⁇ can be set to 20, and the constant parameter ⁇ can be set to 20°.
  • the desired height between the camera and the ground at the jth sampling point is:
  • v min is the minimum speed of the camera motion at all after the correction of the sampling points
  • v max is the maximum speed of the camera motion at all after the correction of the sampling points
  • ⁇ Max is the preset maximum pitch angle value of the camera's pitch angle at all of the sample points
  • ⁇ min is the preset minimum pitch angle value among the camera's pitch angles at all of the sample points.
  • ⁇ max is set to 40° and ⁇ min is set to 10°.
  • the projection item is:
  • R(d j ) is the projection unit vector of the tracking target (camera focus point) on the navigation image at the jth sampling point (center in the horizontal direction, vertical direction distance 1/3 Position), d j is the unit vector of the initial orientation of the camera at the jth sampling point.
  • ⁇ 1 and ⁇ 2 are predetermined constants
  • d j-1 is a unit vector of the initial orientation of the camera at the j-1th sampling point
  • c j-1 is the j-1th
  • the initial position of the camera at the sampling point, c j-2 is the initial position of the camera at the j-2th sampling point.
  • the parameters such as the position of the camera, the camera focus point and the like use the initial value because this is the case of the first iteration, and in the subsequent iteration, the embodiment according to the present invention is required. Iteratively updates the above parameters.
  • the navigation method of the embodiment of the present invention spends more time where the interest value is high.
  • the camera is tighter and close to the ground level, so this high-speed navigation is basically equivalent to the driver's perspective.
  • speed increases both height and distance increase simultaneously to avoid visual discomfort.
  • FIG. 10B is a schematic diagram of the trajectory of the camera corresponding to the smooth result after the second iteration of FIG. 10A. As shown in FIG. 10B, the position of the camera at each sampling point corresponds one-to-one with the camera focus point after the second iteration in FIG. 10A. According to the calculation result of the above embodiment, continuous camera focus points can be obtained by linear interpolation.
  • the navigation method based on the three-dimensional scene in the embodiment of the present invention comprehensively considers various influencing factors in the navigation process by considering the height, volume, irregularity and uniqueness of the building in the scene, so that the navigation screen can better satisfy the user's Expectation.
  • the navigation method of the embodiment of the present invention has the following advantages:
  • Fully automatic navigation The process of generating the interest value of the angle of view is a fully automatic analysis process, which does not require manual labeling and other operations. When generating the camera track, there is no need to manually adjust, correct or set the initial value.
  • the navigation quality is very high: the generated camera motion trajectory has better smoothness, and the smoothing effect far exceeds the smoothing effect in the prior art, especially when the viewing angle and the camera moving speed change, the camera trajectory can be absolutely Flat
  • the camera in the embodiment of the present invention can track a certain road surface target all the time, and ensure that the user does not have a sense of loss when browsing the navigation image.
  • the navigation method of the embodiment of the present invention can be applied not only to driving navigation, but also to instant noodles including automatic path finding, military parade, and unmanned aircraft.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Automation & Control Theory (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

一种基于三维场景的导航方法，所述方法包括：基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值（S101），根据所述视角的兴趣值生成相机的轨迹参数，以根据所述轨迹参数进行导航（S102）。该导航方法基于建筑物的高度、体积、不规则性及独特性，得到合理的视角的兴趣值，可以实现高质量导航。

Description

基于三维场景的导航方法 技术领域
本发明涉及计算机图形学领域，尤其涉及一种基于三维场景的导航方法。
背景技术
在过去十几年内，随着三维建模技术的快速发展，大量诸如谷歌地球这样的三维场景迅速产生，然而基于三维场景的导航技术却一直没有得到很好的发展。
目前，针对相机控制和三维虚拟场景导航的技术研究已有很多，主要是一些关于视角选择、相机轨迹选择及相机运动控制的自动和半自动的技术。
与视角选择有关的研究，例如，利用视角熵度量当前视角下所能看到的建筑物面的分布状况；利用包含建筑物表面可见性、目标重要性、曲率、轮廓及拓扑复杂度的描述子来分析视角的重要性；基于智能学习，分析一些定义(包括建筑物风格、位置、结构等)的语义特征。
相机轨迹规划方面的研究，例如，利用路径规划和图的搜索技术在三维博物馆里进行自动导航；从几何分析角度出发的碰撞检测技术、目标可见性分析以及路径平滑等技术。上述方法主要考虑了相机轨迹的生成，忽略了相机运动速度的问题。之后，又有研究提出基于预定义相机路径自动计算相机速度的优化方法，主要考虑到保持用户注意力的问题。
与相机运动有关的研究，例如，从用户预定义的兴趣值点之间运动的方式。上述方法增加了导航过程中接受用户反馈和简单交互的方式。
目前的三维场景的自动导航方法，主要存在着以下一些问题：第一，很多导航技术还是基于固定相机速度、视角、高度的导航方式，没有考虑场景的特征和用户注意力的问题；第二，很多导航技术还不能完全全自动，大多数还是需要用户做很多标定和输入；第三，现有导航技术在使用方面具有很强的局限性，只能用于某个特定场景；第四，导航过程中的视觉感受不够连续、不够光滑。
发明内容
本发明提出一种基于三维场景的导航方法，以克服现有技术中的一个或多个问题。
本发明提出一种基于三维场景的导航方法，所述方法包括：基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值；根据所述视角的兴趣值生成相机的轨迹参数，以根据所述轨迹参数进行导航。
一个实施例中，所述基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值，包括：基于所述建筑物的高度、体积、不规则性及独特性计算所述建筑物的重要性值；根据所述建筑物的重要性值生成所述视角的兴趣值图；通过中心权重和深度权重修正所述兴趣值图；反算修正后的所述兴趣值图，得到所述视角的兴趣值。
一个实施例中，所述根据所述视角的兴趣值生成相机的轨迹参数，包括：步骤101：在所述场景中选取一规划路线，对所述规划路线进行均匀采样，得到多个采样点；步骤102：依据所述采样点处的视角的兴趣值对所述规划路线作加权光滑，并将作加权光滑后的所述规划路线上与所述采样点对应的位置作为修正后的所述采样点处的相机聚焦点；步骤103：根据设定的导航总时间和所述采样点处的视角的兴趣值修正所述采样点处的相机运动速度；步骤104：根据所述修正后的采样点处的相机运动速度修正所述采样点处的相机姿势；其中，修正后的所述采样点处的相机姿势和修正后的所述采样点处的相机运动速度即为所述相机的轨迹参数。
一个实施例中，在步骤102之前，所述根据所述视角的兴趣值生成相机的轨迹参数，还包括：设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势。
一个实施例中，所述根据所述视角的兴趣值生成相机的轨迹参数，还包括：依据修正后的所述采样点处的相机聚焦点、修正后的所述采样点处的相机运动速度及修正后的所述采样点处的相机姿势，重新获取所述采样点处的视角的兴趣值，如果所述采样点处的视角的后一次兴趣值与前一次兴趣值的差值大于一设定阈值，则用所述采样点处的视角的后一次兴趣值代替所述采样点处的视角的前一次兴趣值，用所述采样点处的后一次相机聚焦点代替所述采样点处的前一次相机聚焦点，用所述采样点处的后一次相机运动速度代替所述采样点处的前一次相机运动速度，用所述采样点处的后一次相机姿势代替所述采样点处的前一次相机姿势，重复迭代执行所述步骤102、所述步骤103及所述步骤104。
一个实施例中，所述建筑物的重要性值为：
S(b)＝αSh(b)+βSv(b)+γSr(b)+δSu(b)，
其中，α、β、γ、δ为权重系数，Sh(b)为建筑物b的高度兴趣值，Sv(b)为建筑物b的体积重要性值，Sr(b)为建筑物b的不规则性值，Su(b)为建筑物b的独特性值，建筑物b即为所述建筑物；
Figure PCTCN2015078949-appb-000001
其中，height(b)为建筑物b的高度，Ωheight(c)为所述规划路线的附近的建筑物c的高度的集合，
Figure PCTCN2015078949-appb-000002
其中，volume(b)为建筑物b的体积，Ωvolume(c)为所述规划路线的附近的建筑物c的体积的集合，
Figure PCTCN2015078949-appb-000003
其中，volume(MVBB(b))为建筑物b的体包围盒MVBB(b)的体积，
Figure PCTCN2015078949-appb-000004
其中，Ωb为建筑物b附近的预定范围内的建筑物d的集合，建筑物b和所述预定范围内的建筑物d的独特性差异值
Figure PCTCN2015078949-appb-000005
其中，volume(∩(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相交结果∩(MVBB(b)，MVBB(d))的体积，volume(U(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相并结果U(MVBB(b)，MVBB(d))的体积。
一个实施例中，所述基于三维场景的导航方法还包括：
通过求解一优化权重方程得到所述权重系数α、β、γ、δ，所述优化权重方程为：
Figure PCTCN2015078949-appb-000006
其中，Ri为基于一给定建筑物集合中的建筑物的高度、体积、不规则性及独特性的给定用户打分值，R(α，β，γ，δ)为根据一组给定权重系数的值和所述给定建筑物集合中的建筑物的Sh(b)值、Sv(b)值、Sr(b)值、Su(b)值计算得到的重要性值，K(R(α，β，γ，δ)，Ri)为所述给定用户打分值Ri和所述给定建筑物集合中的建筑物的重要性值R(α，β，γ，δ)之间的第一距离，如果所述第一距离小于一设定距离，则将所述组给定权重系数的值作为所述权重系数α、β、γ，δ的值。
一个实施例中，在所述通过中心权重和深度权重修正所述兴趣值图中：
所述中心权重为
Figure PCTCN2015078949-appb-000007
其中i为所述兴趣值图中的像素的位置，o为所述兴趣值图的中心，r是所述兴趣值图的对角线长度的一半；
所述深度权重为
Figure PCTCN2015078949-appb-000008
其中d*为一设定观察深度，d(i)为所述像素的位置i处的观察深度；
所述修正后的兴趣值图对应的视角的兴趣值为
Figure PCTCN2015078949-appb-000009
其中，N是所述兴趣值图的像素个数，N≥1，N为整数，j为所述采样点的序号，j∈[0，n]，n＞1，n为正整数，S(i)为所述兴趣值图中的像素的位置i处的视角的兴趣值。
一个实施例中，所述设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势，包括：将所述初始相机聚焦点的位置fj设定为所述采样点的位置pj，其中，j表示所述采样点的序号；将所述初始相机运动速度设定为一匀速度；将所述初始相机姿势设定为<cj，dj>，其中，cj＝fj-2+[0，0，e]，
Figure PCTCN2015078949-appb-000010
其中，cj为第j个所述采样点处的相机的初始位置，dj为第j个所述采样点处的相机的初始朝向的单位向量，e为所述采样点处的相机与地面的初始高度，fj-2为第j-2个所述采样点处的初始相机聚焦点的位置，j∈[0，n]，n＞1，n为正整数。
一个实施例中，在所述步骤102中：修正后的所述采样点处的相机聚焦点的位置为
Figure PCTCN2015078949-appb-000011
其中，M为所述采样点处的相机的跟踪目标在一预定时间内所走过的采样点的个数，M为整数，M≥1，ph为第h个所述采样点处的相机初始聚焦点的位置。
一个实施例中，所述步骤103包括：
通过最大点乘高维向量
Figure PCTCN2015078949-appb-000012
和高维向量I＝{Ij}求解一最有约束方程，得到一优化时间tj；
其中，所述优化时间tj为相机从第j个所述采样点处的相机的位置到第j+1个所述采样点处的相机的位置的运动时间，所述最有约束方程为：
Figure PCTCN2015078949-appb-000013
其中，∑tj＝T，
其中，
Figure PCTCN2015078949-appb-000014
Ij为第j个所述采样点处的视角的兴趣值，T即为所述导航总时间；
将从第j个所述采样点处的相机的位置到第j+1个所述采样点处的相机的位置的间距均设定为一给定第二距离Δ；
根据所述优化时间tj及所述第二距离Δ得到修正后的所述采样点处的相机运动速度为
Figure PCTCN2015078949-appb-000015
其中，常数C＝Δ/β2，
Figure PCTCN2015078949-appb-000016
||I||为高维向量I的模。
一个实施例中，在所述步骤104中：修正后的所述采样点处的相机姿势通过一最小化能量方程得到；
其中，所述最小化能量方程为：
Figure PCTCN2015078949-appb-000017
其中，Ed(cj，fj，vj)为距离项，Ep(cj，dj，fj)为投影项，Es(cj，dj))为光滑项，a、b、c为预定系数；
所述距离项
Figure PCTCN2015078949-appb-000018
其中，第j个所述采样点处的相机的初始位置cj到所述采样点处的初始相机聚焦点的位置fj的之间的期望距离为
Figure PCTCN2015078949-appb-000019
μ为一给定角度值，α为跟踪目标的给定运动时间，
Figure PCTCN2015078949-appb-000020
为cj的垂直分量，fj z为fj的垂直分量；
第j个所述采样点处的相机与地面之间的期望高度为H(vj)＝D(vj)sin(Φ(vj))，其中，所述采样点处的相机的期望俯仰角为
Figure PCTCN2015078949-appb-000021
其中，vmin为所有修正后的所述采样点处的相机运动速度中的最小值，vmax为所有修正后的所述采样点处的相机运动速度中的最大值，Φmax为所有所述采样点处的相机的俯仰角中的预设最大俯仰角值，Φmin为所有所述采样点处的相机的俯仰角中的预设最小俯仰角值；
所述投影项
Figure PCTCN2015078949-appb-000022
其中，R(dj)为跟踪目标在第j个所述采样点处的导航图像上的投影单位向量，dj为第j个所述采样点处的相机的初始朝向的单位向量；
所述光滑项
Figure PCTCN2015078949-appb-000023
其中，λ1和λ2为预定常量，dj-1为第j-1个所述采样点处的相机的初始朝向的单位向量，cj-1为第j-1个所述采样点处的相机的初始位置，cj-2为第j-2个所述采样点处的相机的初始位置。
一个实施例中，在所述根据所述建筑物的重要性值生成所述相机的视角的兴趣值图中：所述兴趣值图为颜色能量图。
一个实施例中，所述中心权重为从中心往外呈三角函数降低。
一个实施例中，所述第一距离是根据Kendall tau方法计算得到的距离。
一个实施例中，所述优化权重方程通过随机搜索法或拟牛顿法求解。
一个实施例中，所述权重系数α、β、γ、δ的值分别为：α＝0.35，β＝0.3，γ＝0.15，δ＝0.2。
本发明实施例的基于三维场景的导航方法基于建筑物的高度、体积、不规则性及独特性计算视角的兴趣值，具有较高的导航质量。
进一步，本发明实施例能够对给定场景自动进行重要性分析，并根据视角的兴趣值生成自适应的速度、视角、高度，同时保证平滑的相机轨迹，是一种非常有效的全自动导航路径生成的方式。本发明实施例自动化分析视角的重要性值，并基于该分析结果计算出自适应相机速度及相机姿势后的光滑的相机轨迹，用户交互特别简洁。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1是本发明一实施例的基于三维场景的导航方法的流程示意图；
图2是本发明一个实施例中的计算视角的兴趣值的流程示意图；
图3是本发明一实施例中的生成轨迹参数的流程示意图；
图4是本发明一实施例中的生成轨迹参数的流程示意图；
图5是本发明一实施例中的生成轨迹参数的流程示意图；
图6是本发明一实施例中计算权重系数所用的建筑物集合的结构示意图；
图7是用户对图6中建筑物的加权打分结果；
图8A-8C是三个不同视角下的场景的纹理示意图；
图8D-8F是本发明一实施例的分别与图8A-8C的视角对应的加权前的效果示意图；
图8G-8I是本发明一实施例的分别与图8A-8C的视角对应的加权前的效果示意图；
图9是本发明一个实施例中的初始化过程的流程示意图；
图10A是本发明一实施例的经过两次迭代后的光滑结果示意图；
图10B是与图10A的第二次迭代后的光滑结果相对应的相机的轨迹示意图；
图11是本发明一实施例中随着速度增加期望距离及相机姿势的变化示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚明白，下面结合附图对本发明实施例做进一步详细说明。在此，本发明的示意性实施例及其说明用于解释本发明，但并不作为对本发明的限定。
图1是本发明一实施例的基于三维场景的导航方法的流程示意图。如图1所示，本发明实施例的导航方法包括：
步骤S101：基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值；
步骤S102：根据所述视角的兴趣值生成相机的轨迹参数，以根据所述轨迹参数进行导航。
本发明实施例的基于三维场景的导航方法，充分考虑了用户对场景中建筑物的兴趣倾向，包括建筑物的高度、体积、不规则性及独特性，从而很好地满足用户对导航图像中所视建筑物的期望。
因为建筑物在场景中占据重要地位，所以本发明的各实施例以建筑物来说明本发明的优点，但本领域技术人员知道，本发明实施例的基于三维场景的导航方法也可以考虑场景中其他物体(例如交通设备、树木等)重要性，从而适应不同导航场景的需要。
在上述步骤S101中，每一个相机位置对应一种视角，每个视角对应一个场景，每个场景中对应多个建筑物。对于一个视角，求出该视角所对应场景中的每个建筑的重要性值，然后对建筑物的重要性值作进一步处理，可以得到该视角的兴趣值。
图2是本发明一个实施例中的计算视角的兴趣值的流程示意图，如图2所示，基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值的步骤S101可包括：
步骤S201：基于所述建筑物的高度、体积、不规则性及独特性计算所述建筑物的重要性值；
步骤S202：根据所述建筑物的重要性值生成所述视角的兴趣值图；
步骤S203：通过中心权重和深度权重修正所述兴趣值图；
步骤S204：反算修正后的所述兴趣值图，得到所述视角的兴趣值。
本发明实施例中的计算视角的兴趣值的方法，不仅考虑了场景中建筑物的重要性值，还通过中心权重纳入了用户对中心区域更加关注的因素，通过深度权重纳入了用户喜欢在间隔一定距离观察建筑物的因素，进一步使导航结果更贴近用户的关注点。
在上述步骤S102中，每个相机的视角对应一组相机的轨迹参数，相机的轨迹参数可以包括相机聚焦点、相机姿势、相机运动速度及跟踪目标的速度，相机姿势可以用相机三维位置、相机朝向、相机俯仰角及相机偏航角等参数来表示。
图3是本发明一实施例中的生成轨迹参数的流程示意图。如图3所示，根据所述视角的兴趣值生成相机的轨迹参数，以根据所述轨迹参数进行导航的骤S102可包括：
步骤S301：在所述场景中选取一规划路线，对所述规划路线进行均匀采样，得到多个采样点；
步骤S303：依据所述采样点处的视角的兴趣值对所述规划路线作加权光滑，并将作加权光滑后的所述规划路线上与所述采样点对应的位置作为修正后的所述采样点处的相机聚焦点；
步骤S304：根据设定的导航总时间和所述采样点处的视角的兴趣值修正所述采样点处的相机运动速度；
步骤S305：根据所述修正后的采样点处的相机运动速度修正所述采样点处的相机姿势。
本发明实施例中，将修正后的所述采样点处的相机姿势和修正后的所述采样点处的相机运动速度作为所述相机的轨迹参数。
在三维场景导航中，给定庞大的搜索空间和复杂的导航需求总是相互矛盾的。直接求解一个整体的优化方案基本是不可行的。
本发明实施例的基于三维场景的导航方法将庞大的约束优化问题分成一系列小的优化步骤，而且每一优化步骤越来越可控，然后迭代求解这些优化步骤。本发明实施例化解三维场景导航中存在的矛盾，可以给用户带来更符合其需求导航体验。
一个实施例中，在上述步骤S303开始之前，对上述各采样点处的相机的参数进行初始化，以作为后续修正(优化)步骤的初始值。
图4是本发明一实施例中的生成轨迹参数的流程示意图。如图4所示，生成相机的轨迹参数还可包括：
步骤S302：设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势。
本发明实施例中的生成相机的轨迹参数的方法，通过给多个参数设定合适的初始值，对所述合适的初始值进行优化或修正，以达到较好的导航效果。
在上述步骤S305后，可以不直接将第一次修正的相机姿势及相机运动速度作为相机的轨迹参数，而是进行多次迭代计算，求得更佳的轨迹参数。
图5是本发明一实施例中的生成轨迹参数的流程示意图。如图5所示，生成相机的轨迹参数还可包括：
步骤S306：如果所述采样点处的视角的后一次兴趣值与前一次兴趣值的差值大于一设定阈值，重复迭代执行所述步骤S303、所述步骤S304及所述步骤S305。
在上述步骤S306中，采样点处的视角的后一次兴趣值(例如第二个兴趣值)，依据前一次修正(例如第一次修正)后的所述采样点处的相机聚焦点、前一次修正(例如第一次修正)后的所述采样点处的相机运动速度及前一次修正(例如第一次修正)后的所述采样点处的相机姿势获得。
在迭代执行述步骤S303、所述步骤S304及所述步骤S305中，用所述采样点处的视角的后一次兴趣值(例如第二个兴趣值)代替所述采样点处的视角的前一次兴趣值(例如第一个兴趣值/初始兴趣值)，用所述采样点处的后一次相机聚焦点(例如第二个相机聚焦点)代替所述采样点处的前一次相机聚焦点(例如第一个相机聚焦点/初始相机聚焦点)，用所述采样点处的后一次相机运动速度(例如第二个相机运动速度)代替所述采样点处的前一次相机运动速度(例如第一个相机运动速度/初始相机运动速度)，用所述采样点处的后一次相机姿势(第二个相机姿势)代替所述采样点处的前一次相机姿势(第一个相机姿势/初始相机姿势)。
当所述采样点处的视角的后一次兴趣值(例如第二个兴趣值)与前一次兴趣值(例如第一个兴趣值/初始兴趣值)的差值小于或等于上述设定阈值时，上述迭代步骤结束。一个实施例中，上述迭代算法在第三次迭代时可以结束。
换言之，本发明实施例中的生成轨迹参数的方法，在第一次修正过程中，求取初始视角兴趣值(第一个兴趣值)，对初始相机聚焦点、初始相机运动速度及初始相机姿势进行优化或修正，得到第二个相机聚焦点、第二个相机运动速度及第二个相机姿势；依据上述第二个相机聚焦点、第二个相机运动速度及第二个相机姿势求得第二个视角兴趣值，如果第二个视角兴趣值与初始视角兴趣值(第一个兴趣值)的差值大于上述设定阈 值，则进行第二次优化或修正，得到第三个相机聚焦点、第三个相机运动速度及第三个相机姿势。依次迭代计算，直到视角的后一次兴趣值与前一次兴趣值的差值小于或等于该设定阈值，进行了n次优化或修正，则将第n次优化或修正所得的相机姿势和相机运动速度作为相机的轨迹参数，其中，n≥1，n为整数。
本发明实施例中的生成轨迹参数的方法，通过迭代修正的方法，逐步优化了相机的轨迹参数，从而使导航效果更加贴近用户的期望和需求。
在上述步骤S201中，可以通过多种不同模型，将建筑物的高度、体积、不规则性及独特性纳入建筑物的重要性值。
在一个实施例中，可以通过以下模型计算建筑物的重要性值，建筑物的重要性值为：
S(b)＝αSh(b)+βSv(b)+γSr(b)+δSu(b)    (1)，
在公式(1)中，α、β、γ、δ为权重系数，Sh(b)为建筑物b的高度兴趣值，Sv(b)为建筑物b的体积重要性值，Sr(b)为建筑物b的不规则性值，Su(b)为建筑物b的独特性值，建筑物b即为上述建筑物。
上述建筑物b的高度兴趣值为：
Figure PCTCN2015078949-appb-000024
在公式(2)中，height(b)为建筑物b的高度，Ωheight(c)为所述规划路线的附近的建筑物c的高度的集合。
本发明实施例中的建筑物的高度兴趣值考虑了高度对用户关注点的影响。本发明实施例中的高度兴趣值是一个归一化的高度得分项，考虑了场景中建筑物的相对高度。例如，摩天大厦通常是一个城市的地标性建筑或风景线，上述建筑物的高度兴趣值即考虑到这一因素。
上述建筑物b的体积重要性值为：
Figure PCTCN2015078949-appb-000025
在公式(3)中，volume(b)为建筑物b的体积，Ωvolume(c)为所述规划路线的附近的建筑物c的体积的集合。
公式(3)中volume(b)度量了建筑物b的外壳体积。由于建筑物b的模型不一定具有水密性，所以在一个实施例中，可以通过分别从建筑物b的正面和背面渲染的两张平行投影的深度图像计算建筑物b的体积volume(b)。建筑物b在深度图像中对应像素点处的深度间隔会被累加，从而得到一个与建筑物b的体积近似的体积值。
利用本发明实施例的计算建筑物的体积方法，当建筑物b沿着投影方向没有洞结构的时候是精确的，而且大多数建筑能够满足这样的结构条件，所以上述计算建筑物的体积方法可以较精确考虑建筑物的体积对视角的兴趣值的影响。
本发明实施例中的体积重要性值考虑了体积对用户关注点的影响。场景中的大体积建筑结构，例如体育场馆或购物中心通常作为导航中使用的地标，上述体积重要性值便可囊括这一影响因素。
上述建筑物b的不规则性值为：
Figure PCTCN2015078949-appb-000026
在公式(4)中，volume(MVBB(b))为建筑物b的体包围盒MVBB(b)的体积。建筑物的不规则性值也可称作异性值。
对于用户，规则的盒子形状的建筑物看起来总是比较枯燥，而不规则结构的建筑会比较有意思。
本发明实施例计算建筑物的不规则性值时，将建筑物分割成许多很小的体包围盒MVBB，建筑物b的不规则性被定义成其体积与其盒MVBB的差异，从而量化建筑物b的不规则性。
上述建筑物b的独特性值为：
Figure PCTCN2015078949-appb-000027
在公式(5)中，Ωb为建筑物b附近的预定范围内的建筑物d的集合，M(b，d)为建筑物b和所述预定范围内的建筑物d的独特性差异值。
其中，
Figure PCTCN2015078949-appb-000028
在公式(6)中，volume(∩(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相交结果∩(MVBB(b)，MVBB(d))的体积，volume(U(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相并结果U(MVBB(b)，MVBB(d))的体积。
本发明实施例中的建筑物b的独特性值的模型，主要考虑了建筑物和与其相邻的建筑物的差异性，量化建筑物的独特性值，并简化了建筑物的独特性值求解过程。本发明实施例的独特性值通过评估它们的体包围盒得到。在一个场景中，例如一栋很独特的建 筑物通常比较吸引用户注意力，本发明实施例中的建筑物的独特性值即可考虑这一影响因素。
上述公式(1)中的权重系数可以通过多种方法求得，在一个实施例中，可通过求解一优化权重方程得到，其中，该优化权重方程为：
Figure PCTCN2015078949-appb-000029
在公式(7)中，Ri为基于一给定建筑物集合中的建筑物的高度、体积、不规则性及独特性的给定用户打分值；R(α，β，γ，δ)为根据一组给定权重系数的值和所述给定建筑物集合中的建筑物的Sh(b)值、Sv(b)值、Sr(b)值、Su(b)值计算得到的重要性值；K(R(α，β，γ，δ)，Ri)为所述给定用户打分值Ri和所述给定建筑物集合中的建筑物的重要性值R(α，β，γ，δ)之间的第一距离，如果所述第一距离小于一设定距离，则将所述组给定权重系数的值作为所述权重系数α、β、γ、δ的值。
一个实施例中，上述第一距离K(R(α，β，γ，δ)，Ri)可以通过Kendall tau方法计算得到，其中，Kendall tau方法为Kendall等人在发表于Biometrica期刊(1983年，第30卷，81-39页)的文章“A new measure of rank correlation.Biometrica”中所用到的方法。
图6是本发明一实施例中计算权重系数所用的建筑物集合的结构示意图。如图6所示，一个三维虚拟城市的场景中包含的建筑物构成一个建筑物的集合。多个用户被要求根据对图6中建筑物的兴趣大小，对建筑物集合中的每一个建筑物进行排序，得到多组加权的打分结果，如图7所示。然后，根据上述多组加权的打分结果，利用公式(7)进行拟合得到各项的权重系数。
公式(7)可以通过多种不同方法进行求解，例如逐步精细的随机搜索方法或拟牛顿法。对图7中的加权打分结果进行随机搜索，可求得公式(7)的权重解为α＝0.35，β＝0.3，γ＝0.15，δ＝0.2。
图7列出了图6中各建筑的重要性值，同时画出了重要性值所对应的柱状图。如图7所示，较高的建筑物#2和建筑物#5具有最高的高度兴趣值和体积重要性值得分；形状较不规则的建筑物#9和建筑物#3的不规则性值的得分很高；建筑物#1、建筑物#8及建筑物#11的形状较相似，它们的独特性值的得分很低；与图6中其它建筑相比，建筑物#12的体积显得特别小，其独特性值得分较高。
如此一来，图6中各建筑物的高度兴趣值、体积重要性值、不规则性值及独特性值与他们的实际外形一致，这有力地说明了本发明实施例的基于三维导航方法的有效性。
此外，将建筑物的高度、体积、不规则性、独特性四项的得分加权，由图7中的柱状图得知，建筑物#2和建筑物#5的重要性最高，建筑物#12的重要性最低。这一结果与用户调查的统计结果也是吻合的。这进一步说明了本发明实施例的建筑物的高度、体积、不规则性、独特性的模型是极为有效的。
通过上述步骤S201得出每个导航点处的三维模型场景中的建筑物的重要性值后，进行步骤S202，生成所述场景所对应视角的兴趣值图，例如该兴趣值图为颜色能量图。各建筑物的重要性值被映射成由从红到蓝的颜色代表其重要性值得分的颜色模型场景，暖色代表了高的得分。在特定视角下，便可得到一张包含了该特定视角下所有建筑物模型的面的二维能量图，该视角的兴趣值得分便可以通过简单的累加该视角的能量图中的所有像素值得到。
然而，在上述步骤S203中，通过引入中心权重和深度权重修正视角的兴趣值图(兴趣值)，可得到更合理的视角的兴趣值得分。
在一个实施例中，中心权重可以是从中心往外呈三角函数降低权重，以使图像的中心位置具有高的兴趣值得分，远离中心的位置具有较低的兴趣值得分，从而可以满足用户更加关注中心区域的需求。
一个实施例中，中心权重可表示为：
Figure PCTCN2015078949-appb-000030
在公式(8)中，i为上述兴趣值图中的像素的位置，o为上述兴趣值图的中心，r是所述兴趣值图的对角线长度的一半。
一个实施例中，深度权重为基于距离的权重ωd。假设理想的观察距离是d*，例如是d*＝150m，在该理想的观察距离d*上，场景模型表面的投影的像素面积是A，若该观察深度在图像的像素i处是d(i)，则面积A在像素i处的投影就近似等于
Figure PCTCN2015078949-appb-000031
则基于距离的权重项(深度权重)可表示为：
Figure PCTCN2015078949-appb-000032
在公式(9)中，d*为一设定观察深度，d(i)为在图像中的像素的位置i处的观察深度。
当相机距离一个给定建筑越来越近，该建筑物的投影将占据越来越多像素位置。在这种情况下，每个像素得分的简单的求和将会导致一种完全由建筑物b带来的很高的得分。
然而，用户通常更喜欢在有一定距离的情况下观察某个建筑物。将相机移动比这个距离更近的时候并不会得到更多的信息，因此其对这个视角的贡献值并不会增加。实际上，当相机特别近导致只有建筑物的一部分能够被看到的时候，它的贡献值应该更低。
本发明实施例中的深度权重便考虑了上述因素，从而可以避免近距离的建筑物给视角的兴趣值带来过多的不利影响。
经过中心权重公式(8)和深度权重公式(9)修正后，兴趣值图对应的视角的兴趣值，即特定视角的整个的兴趣值的得分总和可以表示为：
Figure PCTCN2015078949-appb-000033
在公式(10)中，N是所述兴趣值图的像素个数，N≥1，N为整数，j为所述采样点的序号，j∈[0，n]，n>1，n为正整数，S(i)为所述兴趣值图中的像素的位置i处的视角的兴趣值。原始兴趣值图和加权的范围都可归一化在[0,1]之间，因此加权后的兴趣值图也可在这个范围内。
图8A至图8C是三个不同视角下的场景的纹理示意图；图8D至图8F是本发明一实施例的分别与图8A至图8C的视角对应的加权前的效果示意图；图8G至图8I是本发明一实施例的分别与图8A至图8C的视角对应的加权前的效果示意图。图8D对应第一视角的兴趣值为I＝0.36，图8E对应第二视角的兴趣值为I＝0.29，图8F对应第三视角的兴趣值为I＝0.28，图8G对应第一视角的兴趣值为I＝0.199，图8H对应第二视角的兴趣值为I＝0.247，图8I对应第三视角的兴趣值为I＝0.264。
如图8A至图8I所示，添加不同权重，兴趣值I不同，所产生的图像效果不同，从而说明本发明实施例引入权重可以使图像更好地展示用户期望的场景。
本发明实施例的基于三维场景的导航方法抓住高效导航的本质，即相机的控制。在生成相机的轨迹参数时，寻找一系列相机的位置的序列，包括相机的三维位置和相机朝向，并研究这些相机的位置处的相机运动速度。不考虑偏航角的变化，相机的位置通过五个自由度来表示，包括相机的三维位置、相机朝向、相机俯仰角、相机的运动速度和跟踪目标的运动速度。
本发明实施例的导航方法是基于一些关键的想法。首先，计算出每一次迭代的跟踪目标的优化速度(与相机运动速度相同)。给定相机位置，得到确定的视角的兴趣值，通过一个简单的可求解的方案得到跟踪目标的速度。然后，为了避免相机位置的剧烈变化，自适应地光滑了运动路径，从而根据跟踪目标的速度(高速运动的区域会被光滑的更厉害)生成了一系列相机的位置和相机聚焦点。再者，为了保证视觉的舒适性，同时 保持连续地跟踪目标，随着跟踪目标的加速提升相机的高度和与目标的距离。最终，通过求解一个最小目标方程计算出每个聚焦点处的优化后的相机姿势。
一个实施例中，给定一个规划路线，沿着该规划路线均匀采样，例如，每10m取一个采样点(密集采样)，并标志这些采样点的位置为pj，j∈[0，n]，每一个pj都是路径上的一个三维位置。
图9是本发明一个实施例中的初始化过程的流程示意图。如图9所示，所述设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势的步骤S302可包括：
步骤S901：将所述初始相机聚焦点的位置fj设定为所述采样点的位置pj，其中，j表示所述采样点的序号；
步骤S902：将所述初始相机运动速度设定为一匀速度；
步骤S903：将所述初始相机姿势设定为<cj，dj>。
其中，cj＝fj-2+[0，0，e]，
Figure PCTCN2015078949-appb-000034
cj为第j个所述采样点处的相机的初始位置，dj为第j个所述采样点处的相机的初始朝向的单位向量，e为所述采样点处的相机与地面的初始高度，fj-2为第j-2个所述采样点处的初始相机聚焦点的位置，j∈[0，n]，n>1，n为正整数。
在一个实施例中，屏幕的宽高比为16：9，对应的水平和垂直视场分别是60°和36°，相机的俯仰角在方向向量向上抬6°，这样可保证相机聚焦点始终在帧底部1/3的位置处。相机的朝向由dj的水平部分决定。
在上述步骤S901中，各采样点处的初始化的相机聚焦点与导航点(采样点)的意义对应(即fj＝pj)。在上述步骤S902中，跟踪目标和相机的初始运动速度均为匀速，所以相机将离跟踪目标始终为一个恒定的距离和高度。
修正了相机姿势后，再调整规划路线上各采样点处的跟踪目标的速度。这将会反过来影响相机运动速度和相机姿势。用户希望在兴趣值高的地方花费更多时间，希望当跟踪目标运动速度慢时，相机跟得更紧，希望当速度增加时，视觉舒适。
一个实施例中，给定一组规划路线上的均匀采样点，找到优化的时间tj，即跟踪目标在pj(第j个采样点的位置)到pj+1(第j+1个采样点的位置)之间的运动时间。每一个相机聚焦点，对应一个相机姿势<cj，dj>，一个相机姿势对应的视角可以渲染出一张兴趣值图，从而对应计算出一个兴趣值得分Ij。
本发明实施例中，可首先通过求解下面的最有约束方程，求解出tj：
Figure PCTCN2015078949-appb-000035
其中，∑tj＝T    (11)，
在公式(11)中，设定
Figure PCTCN2015078949-appb-000036
Ij为第j个所述采样点处的视角的兴趣值，T即为所述导航总时间。函数f(tj)决定了速度怎样随着兴趣值改变的。
Figure PCTCN2015078949-appb-000037
保证了速度和兴趣值之间的强烈的非线性关联，同时得出了一个简单可求解的方案，即通过最大点乘高维向量
Figure PCTCN2015078949-appb-000038
和高维向量I＝{Ij}求解最有约束方程公式(11)，得到优化时间tj。
最大点乘高维向量
Figure PCTCN2015078949-appb-000039
和
Figure PCTCN2015078949-appb-000040
为：
Figure PCTCN2015078949-appb-000041
在公式(12)中，θ是向量
Figure PCTCN2015078949-appb-000042
和向量I＝{Ij}之间的夹角。由于T是给定的，||I||可为常量，这个点乘的最大化只需要保证θ＝0，换言之，这两个向量需要共线。设定一个常量β，使得
Figure PCTCN2015078949-appb-000043
对每一个j，
Figure PCTCN2015078949-appb-000044
将规划路线从pj到pj+1的每段都设为等长的距离Δ，则速度vj可以简单表示成Δ/tj，则优化速度(修正的相机运动速度/跟踪目标的速度)满足
Figure PCTCN2015078949-appb-000045
C为常量(C＝Δ/β2)，||I||为高维向量I的模。
从而，本发明实施例中的视角的兴趣值被限定在一个范围内，所以相机运动速度(跟踪目标的速度)也就被限定在一个范围内。
若如初始的相机轨迹，简单的跟随规划路线和相机聚焦点，易导致较差的用户体验，因为由于速度会在枯燥的部分提高了，导致相机的摆动和转弯会非常剧烈。本发明实施例的导航方法，通过在每次迭代中作一次根据相机运动速度的自适应平滑可以克服上述缺点。
一个实施例中，根据相机运动速度的自适应加权平滑相机的轨迹，重新计算相机聚焦点的位置，修正后的所述采样点处的相机聚焦点的位置为：
Figure PCTCN2015078949-appb-000046
在公式(13)中，M为所述采样点处的相机的跟踪目标在一预定时间内所走过的采样点的个数，例如预定时间为6秒，则M为在6秒时间内跟踪目标走过采样点的个数，M为整数，M≥1，ph为第h个所述采样点处的相机初始聚焦点的位置。
图10A是本发明一实施例的经过两次迭代后的光滑结果示意图。如图10A所示，经过修正后，相机的聚焦点更符合用户的期望。
一个实施例中，根据每一个采样点pj处修正/优化后的机运动速度vj，及相机应该跟踪的对应聚焦点fj(修正/优化后)，可以调整相机的姿势。对于每一个相机聚焦点fj，重新计算相机姿势<cj，dj>。
修正后的所述采样点处的相机姿势通过一最小化能量方程得到，所述最小化能量方程为：
Figure PCTCN2015078949-appb-000047
在公式(14)中，Ed(cj，fj，vj)为距离项，Ep(cj，dj，fj)为投影项，Es(cj，dj))为光滑项，a、b、c为预定系数。
公式(14)中，距离项Ed确保了相机和聚焦点相距一个合适的距离和高度。投影项Ep确保了投影在帧上期望的位置。最后，光滑项Es削弱了相邻聚焦点之间相机姿态较大的改变。
通过期望俯仰角Φ和期望距离D两个约束，求取所述距离项：
Figure PCTCN2015078949-appb-000048
在公式(15)中，第j个所述采样点处的相机的初始位置cj到所述采样点处的初始相机聚焦点的位置fj的之间的期望距离为：
Figure PCTCN2015078949-appb-000049
在公式(16)中，μ为一给定角度值，α为跟踪目标的给定运动时间，
Figure PCTCN2015078949-appb-000050
为cj的垂直分量，fj z为fj的垂直分量。
在上述期望距离D(vj)中，αvj为跟踪目标在α秒内运动的距离。如图11所示，期望距离D(vj)保证了运动距离包含μ的角度范围内。常量参数α可以设置成20，常量参数μ可以设置成20°。
第j个所述采样点处的相机与地面之间的期望高度为：
H(vj)＝D(vj)sin(Φ(vj))    (17)，
在公式(17)中，所述采样点处的相机的期望俯仰角为：
Figure PCTCN2015078949-appb-000051
在公式(18)中，vmin为所有修正后的所述采样点处的相机运动速度中的最小值，vmax为所有修正后的所述采样点处的相机运动速度中的最大值，Φmax为所有所述采样点处的相机的俯仰角中的预设最大俯仰角值，Φmin为所有所述采样点处的相机的俯仰角中的预设最小俯仰角值。一个实施例中，Φmax设定为40°，Φmin设定为10°。
所述投影项为：
Figure PCTCN2015078949-appb-000052
在公式(19)中，R(dj)为跟踪目标(相机聚焦点)在第j个所述采样点处的导航图像上的投影单位向量(水平方向的中心，垂直方向距离地步1/3的位置)，dj为第j个所述采样点处的相机的初始朝向的单位向量。
所述光滑项为：
Figure PCTCN2015078949-appb-000053
在公式(18)中，λ1和λ2为预定常量，dj-1为第j-1个所述采样点处的相机的初始朝向的单位向量，cj-1为第j-1个所述采样点处的相机的初始位置，cj-2为第j-2个所述采样点处的相机的初始位置。一个实施例中，λ1＝500，λ2＝1200。
需要说明的是，上述多处计算中，相机的位置、相机聚焦点等参数使用初始值，是因为这是第一次迭代的情况，当在后续迭代过程中，则需要根据本发明实施例的迭代规律更新上述各参数。
本发明实施例的导航方法在兴趣值高的地方花费更多时间。当跟踪目标慢速运动时，相机跟得更紧一些，同时接近于地面高度，所以这个高速导航就基本等同于驾驶员视角。当速度增加，高度和距离都会同时增加来避免视觉上的不舒适感。
图10B是与图10A的第二次迭代后的光滑结果相对应的相机的轨迹示意图。如图10B所示，各采样点处的相机的位置与图10A中第二次迭代后的相机聚焦点一一对应。根据上述实施例的计算结果，可以通过线性插值的方式得到连续的相机聚焦点。
本发明实施例中，使用了两个约束来达到上述效果。首先，向量fj-cj和水平面之间的俯仰角
Figure PCTCN2015078949-appb-000054
应该随着这个速度成比例的增加。其次，不管速度如何，这个地面距离需要保持一个时间常量覆盖的范围，包含在一个常量角度μ之内。
本发明实施例的基于三维场景的导航方法，通过考虑场景中建筑物的高度、体积、不规则性、独特性，全面考虑了导航过程中的多种影响因素，使得导航画面更能满足用户的期望。此外，本发明实施例的导航方法还具有以下优点：
1)可实现全自动导航：生成视角的兴趣值的过程是全自动分析的过程，不需要人工标注等操作，生成相机轨迹时，不需要人工调整、纠正或设定初始值等操作。
2)导航质量很高：生成的相机运动轨迹具有较好的平滑性，平滑效果远远超过现有技术中的平滑效果，尤其在视角和相机运动速度发生变化时，相机的轨迹可做到绝对平 滑；同时，本发明实施例中的相机能一直跟踪某个确定的路面目标，保证用户浏览导航图像时不会有丢失感。
3)导航效率高：对于很长的一段场景，可以在保证获取到绝大多数关键信息的前提下，以最快的速度完成场景的导航，保证用户不在枯燥的场景上花费太多时间，同时能注意到兴趣值很高的位置。
4)交互简单：用户不需要做太多例如标定的复杂、繁琐操作，也不需要设定相机的一些参数，只需指定规划路线的起止点，设定导航总时间，不需额外学习。
5)可移植性高：只需更改算法的兴趣值方程，便可针对各种应用场景自动生成相机路径。因此本发明实施例的导航方法不仅可以用于驾车导航，还可在包括游戏的自动寻路、军事阅兵、无人飞机等方便面得到应用。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述的具体实施例，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施例而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (17)

  1. 一种基于三维场景的导航方法，其特征在于，所述方法包括：
    基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值；
    根据所述视角的兴趣值生成相机的轨迹参数，以根据所述轨迹参数进行导航。
  2. 如权利要求1所述的基于三维场景的导航方法，其特征在于，所述基于场景中的建筑物的高度、体积、不规则性及独特性计算相机的视角的兴趣值，包括：
    基于所述建筑物的高度、体积、不规则性及独特性计算所述建筑物的重要性值；
    根据所述建筑物的重要性值生成所述视角的兴趣值图；
    通过中心权重和深度权重修正所述兴趣值图；
    反算修正后的所述兴趣值图，得到所述视角的兴趣值。
  3. 如权利要求2所述的基于三维场景的导航方法，其特征在于，所述根据所述视角的兴趣值生成相机的轨迹参数，包括：
    步骤101：在所述场景中选取一规划路线，对所述规划路线进行均匀采样，得到多个采样点；
    步骤102：依据所述采样点处的视角的兴趣值对所述规划路线作加权光滑，并将作加权光滑后的所述规划路线上与所述采样点对应的位置作为修正后的所述采样点处的相机聚焦点；
    步骤103：根据设定的导航总时间和所述采样点处的视角的兴趣值修正所述采样点处的相机运动速度；
    步骤104：根据所述修正后的采样点处的相机运动速度修正所述采样点处的相机姿势；
    其中，修正后的所述采样点处的相机姿势和修正后的所述采样点处的相机运动速度即为所述相机的轨迹参数。
  4. 如权利要求3所述的基于三维场景的导航方法，其特征在于，在步骤102之前，所述根据所述视角的兴趣值生成相机的轨迹参数，还包括：
    设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势。
  5. 如权利要求4所述的基于三维场景的导航方法，其特征在于，所述根据所述视角的兴趣值生成相机的轨迹参数，还包括：
    依据修正后的所述采样点处的相机聚焦点、修正后的所述采样点处的相机运动速度及修正后的所述采样点处的相机姿势，重新获取所述采样点处的视角的兴趣值，如果所 述采样点处的视角的后一次兴趣值与前一次兴趣值的差值大于一设定阈值，则用所述采样点处的视角的后一次兴趣值代替所述采样点处的视角的前一次兴趣值，用所述采样点处的后一次相机聚焦点代替所述采样点处的前一次相机聚焦点，用所述采样点处的后一次相机运动速度代替所述采样点处的前一次相机运动速度，用所述采样点处的后一次相机姿势代替所述采样点处的前一次相机姿势，重复迭代执行所述步骤102、所述步骤103及所述步骤104。
  6. 如权利要求5所述的基于三维场景的导航方法，其特征在于，所述建筑物的重要性值为：
    S(b)＝αSh(b)+βSv(b)+γSr(b)+δSu(b)，
    其中，α、β、γ、δ为权重系数，Sh(b)为建筑物b的高度兴趣值，Sv(b)为建筑物b的体积重要性值，Sr(b)为建筑物b的不规则性值，Su(b)为建筑物b的独特性值，建筑物b即为所述建筑物；
    Figure PCTCN2015078949-appb-100001
    其中，height(b)为建筑物b的高度，Ωheight(c)为所述规划路线的附近的建筑物c的高度的集合，
    Figure PCTCN2015078949-appb-100002
    其中，volume(b)为建筑物b的体积，Ωvolume(c)为所述规划路线的附近的建筑物c的体积的集合，
    Figure PCTCN2015078949-appb-100003
    其中，volume(MVBB(b))为建筑物b的体包围盒MVBB(b)的体积，
    Figure PCTCN2015078949-appb-100004
    其中，Ωb为建筑物b附近的预定范围内的建筑物d的集合，建筑物b和所述预定范围内的建筑物d的独特性差异值
    Figure PCTCN2015078949-appb-100005
    其中，volume(∩(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相交结果∩(MVBB(b)，MVBB(d))的体积，volume(U(MVBB(b)，MVBB(d)))为建筑物b的体包围盒MVBB(b)与建筑物d的体包围盒MVBB(d)的相并结果U(MVBB(b)，MVBB(d))的体积。
  7. 如权利要求6所述的基于三维场景的导航方法，其特征在于，所述基于三维场景的导航方法还包括：
    通过求解一优化权重方程得到所述权重系数α、β、γ、δ，所述优化权重方程为：
    Figure PCTCN2015078949-appb-100006
    其中，Ri为基于一给定建筑物集合中的建筑物的高度、体积、不规则性及独特性的给定用户打分值，R(α，β，γ，δ)为根据一组给定权重系数的值和所述给定建筑物集合中的建筑物的Sh(b)值、Sv(b)值、Sr(b)值、Su(b)值计算得到的重要性值，K(R(α，β，γ，δ)，Ri)为所述给定用户打分值Ri和所述给定建筑物集合中的建筑物的重要性值R(α，β，γ，δ)之间的第一距离，如果所述第一距离小于一设定距离，则将所述组给定权重系数的值作为所述权重系数α、β、γ、δ的值。
  8. 如权利要求6所述的基于三维场景的导航方法，其特征在于，在所述通过中心权重和深度权重修正所述兴趣值图中：
    所述中心权重为
    Figure PCTCN2015078949-appb-100007
    其中i为所述兴趣值图中的像素的位置，o为所述兴趣值图的中心，r是所述兴趣值图的对角线长度的一半；
    所述深度权重为
    Figure PCTCN2015078949-appb-100008
    其中d*为一设定观察深度，d(i)为所述像素的位置i处的观察深度；
    所述修正后的兴趣值图对应的视角的兴趣值为
    Figure PCTCN2015078949-appb-100009
    其中，N是所述兴趣值图的像素个数，N≥1，N为整数，j为所述采样点的序号，j∈[0，n]，n＞1，n为正整数，S(i)为所述兴趣值图中的像素的位置i处的视角的兴趣值。
  9. 如权利要求8所述的基于三维场景的导航方法，其特征在于，所述设定所述采样点处的初始相机聚焦点、初始相机运动速度及初始相机姿势，包括：
    将所述初始相机聚焦点的位置fj设定为所述采样点的位置pj，其中，j表示所述采样点的序号；
    将所述初始相机运动速度设定为一匀速度；
    将所述初始相机姿势设定为<cj，dj>，其中，cj＝fj-2+[0，0，e]，
    Figure PCTCN2015078949-appb-100010
    其中，cj为第j个所述采样点处的相机的初始位置，dj为第j个所述采样点处的相机的初始朝向的单位向量，e为所述采样点处的相机与地面的初始高度，fj-2为第j-2个所述采样点处的初始相机聚焦点的位置，j∈[0，n]，n＞1，n为正整数。
  10. 如权利要求9所述的基于三维场景的导航方法，其特征在于，在所述步骤102中：修正后的所述采样点处的相机聚焦点的位置为
    Figure PCTCN2015078949-appb-100011
    其中，M为所述采样点处的相机的跟踪目标在一预定时间内所走过的采样点的个数，M为整数，M≥1，ph为第h个所述采样点处的相机初始聚焦点的位置。
  11. 如权利要求10所述的基于三维场景的导航方法，其特征在于，所述步骤103包括：
    通过最大点乘高维向量
    Figure PCTCN2015078949-appb-100012
    和高维向量I＝{Ij}求解一最有约束方程，得到一优化时间tj；
    其中，所述优化时间tj为相机从第j个所述采样点处的相机的位置到第j+1个所述采样点处的相机的位置的运动时间，所述最有约束方程为：
    Figure PCTCN2015078949-appb-100013
    其中，∑tj＝T，
    其中，
    Figure PCTCN2015078949-appb-100014
    Ij为第j个所述采样点处的视角的兴趣值，T即为所述导航总时间；
    将从第j个所述采样点处的相机的位置到第j+1个所述采样点处的相机的位置的间距均设定为一给定第二距离Δ；
    根据所述优化时间tj及所述第二距离Δ得到修正后的所述采样点处的相机运动速度为
    Figure PCTCN2015078949-appb-100015
    其中，常数C＝Δ/β2，
    Figure PCTCN2015078949-appb-100016
    ||I||为高维向量I的模。
  12. 如权利要求11所述的基于三维场景的导航方法，其特征在于，在所述步骤104中：
    修正后的所述采样点处的相机姿势通过一最小化能量方程得到；
    其中，所述最小化能量方程为：
    Figure PCTCN2015078949-appb-100017
    其中，Ed(cj，fj，vj)为距离项，Ep(cj，dj，fj)为投影项，Es(cj，dj))为光滑项，a、b、c为预定系数；
    所述距离项
    Figure PCTCN2015078949-appb-100018
    其中，第j个所述采样点处的相机的初始位置cj到所述采样点处的初始相机聚焦点的位置fj的之间的期望距离为
    Figure PCTCN2015078949-appb-100019
    μ为一给定角度值，α为跟踪目标的给定运动时间，
    Figure PCTCN2015078949-appb-100020
    为cj的垂直分量，
    Figure PCTCN2015078949-appb-100021
    为fj的垂直分量；
    第j个所述采样点处的相机与地面之间的期望高度为H(vj)＝D(vj)sin(Φ(vj))，其中，所述采样点处的相机的期望俯仰角为
    Figure PCTCN2015078949-appb-100022
    其中，vmin为所有修正后的所述采样点处的相机运动速度中的最小值，vmax为所有修正后的所述采样点处的相机运动速度中的最大值，Φmax为所有所述采样点处的相机的俯仰角中的预设最大俯仰角值，Φmin为所有所述采样点处的相机的俯仰角中的预设最小俯仰角值；
    所述投影项
    Figure PCTCN2015078949-appb-100023
    其中，R(dj)为跟踪目标在第j个所述采样点处的导航图像上的投影单位向量，dj为第j个所述采样点处的相机的初始朝向的单位向量；
    所述光滑项
    Figure PCTCN2015078949-appb-100024
    其中，λ1和λ2为预定常量，dj-1为第j-1个所述采样点处的相机的初始朝向的单位向量，cj-1为第j-1个所述采样点处的相机的初始位置，cj-2为第j-2个所述采样点处的相机的初始位置。
  13. 如权利要求2所述的基于三维场景的导航方法，其特征在于，在所述根据所述建筑物的重要性值生成所述相机的视角的兴趣值图中：
    所述兴趣值图为颜色能量图。
  14. 如权利要求2所述的基于三维场景的导航方法，其特征在于，所述中心权重为从中心往外呈三角函数降低。
  15. 如权利要求7所述的基于三维场景的导航方法，其特征在于，所述第一距离是根据Kendall tau方法计算得到的距离。
  16. 如权利要求7所述的基于三维场景的导航方法，其特征在于，所述优化权重方程通过随机搜索法或拟牛顿法求解。
  17. 如权利要求7所述的基于三维场景的导航方法，其特征在于，所述权重系数α、β、γ、δ的值分别为：α＝0.35，β＝0.3，γ＝0.15，δ＝0.2。
PCT/CN2015/078949 2015-05-14 2015-05-14 基于三维场景的导航方法 WO2016179825A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2015/078949 WO2016179825A1 (zh) 2015-05-14 2015-05-14 基于三维场景的导航方法
US15/390,000 US10066956B2 (en) 2015-05-14 2016-12-23 Navigation technology in three-dimensional scenes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/078949 WO2016179825A1 (zh) 2015-05-14 2015-05-14 基于三维场景的导航方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/390,000 Continuation US10066956B2 (en) 2015-05-14 2016-12-23 Navigation technology in three-dimensional scenes

Publications (1)

Publication Number Publication Date
WO2016179825A1 true WO2016179825A1 (zh) 2016-11-17

Family

ID=57247667

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/078949 WO2016179825A1 (zh) 2015-05-14 2015-05-14 基于三维场景的导航方法

Country Status (2)

Country Link
US (1) US10066956B2 (zh)
WO (1) WO2016179825A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150035823A1 (en) * 2013-07-31 2015-02-05 Splunk Inc. Systems and Methods for Using a Three-Dimensional, First Person Display to Convey Data to a User
US10726745B2 (en) 2017-06-13 2020-07-28 At&T Intellectual Property I, L.P. Autonomous multi-pass data acquisition using unmanned aerial vehicles
US10460492B2 (en) * 2017-09-13 2019-10-29 Canon Kabushiki Kaisha Method, system and apparatus for navigating a virtual camera using a navigation device
US10902676B2 (en) * 2018-03-13 2021-01-26 Canon Kabushiki Kaisha System and method of controlling a virtual camera
CN111652892A (zh) * 2020-05-02 2020-09-11 王磊 一种基于深度学习的遥感影像建筑物矢量提取及优化方法
CN117809000B (zh) * 2024-02-28 2024-05-10 四川省公路规划勘察设计研究院有限公司 一种基于高斯滤波算法的公路路径漫游方法及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005316702A (ja) * 2004-04-28 2005-11-10 Mitsubishi Electric Corp ３次元景観表示装置
US8044953B2 (en) * 2002-06-28 2011-10-25 Autodesk, Inc. System for interactive 3D navigation for proximal object inspection
US8326442B2 (en) * 2007-05-25 2012-12-04 International Business Machines Corporation Constrained navigation in a three-dimensional (3D) virtual arena
WO2014041188A1 (en) * 2012-09-17 2014-03-20 SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH Method and an apparatus for determining a gaze point on a three-dimensional object
CN103971392A (zh) * 2013-01-31 2014-08-06 北京四维图新科技股份有限公司 面向导航的三维视频数据处理方法、装置、系统及终端
CN104867142A (zh) * 2015-05-14 2015-08-26 中国科学院深圳先进技术研究院 基于三维场景的导航方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9420234B2 (en) * 2006-04-13 2016-08-16 Virtual Observer Pty Ltd Virtual observer
US8532367B2 (en) 2010-08-17 2013-09-10 Raytheon Company System and method for 3D wireframe reconstruction from video
US9326442B1 (en) * 2011-02-07 2016-05-03 Shield Industries, Inc. Side mounted fertilizer knife assembly with interchangeable strip till and low draft points
CN102332175A (zh) 2011-09-28 2012-01-25 中山大学 一种基于形状约束的Flock动画方法
US20150206343A1 (en) * 2014-01-17 2015-07-23 Nokia Corporation Method and apparatus for evaluating environmental structures for in-situ content augmentation

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8044953B2 (en) * 2002-06-28 2011-10-25 Autodesk, Inc. System for interactive 3D navigation for proximal object inspection
JP2005316702A (ja) * 2004-04-28 2005-11-10 Mitsubishi Electric Corp ３次元景観表示装置
US8326442B2 (en) * 2007-05-25 2012-12-04 International Business Machines Corporation Constrained navigation in a three-dimensional (3D) virtual arena
WO2014041188A1 (en) * 2012-09-17 2014-03-20 SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH Method and an apparatus for determining a gaze point on a three-dimensional object
CN103971392A (zh) * 2013-01-31 2014-08-06 北京四维图新科技股份有限公司 面向导航的三维视频数据处理方法、装置、系统及终端
CN104867142A (zh) * 2015-05-14 2015-08-26 中国科学院深圳先进技术研究院 基于三维场景的导航方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
GRABLER, F. ET AL.: "Automatic Generation of Tourist Maps", ACM TRANSACTIONS ON GRAPHICS, vol. 27, no. 3, August 2008 (2008-08-01), pages 100, XP055330782 *
LI, YUPING: "Path Design Based on 3 D City Models", GEOMATICS & SPATIAL INFORMATION TECHNOLOGY, vol. 27, no. 3, 30 March 2014 (2014-03-30), pages 145 - 148, ISSN: 1672-5867 *
MA, LINSHAN E ET AL.: "Design and Implementation of Virtual Library Roaming Navigation System", RESEARCHES IN LIBRARY SCIENCE, 15 July 2010 (2010-07-15), pages 54 - 57, ISSN: 1001-0424 *

Also Published As

Publication number Publication date
US10066956B2 (en) 2018-09-04
US20170153116A1 (en) 2017-06-01

Similar Documents

Publication Publication Date Title
WO2016179825A1 (zh) 基于三维场景的导航方法
US11488380B2 (en) Method and apparatus for 3-D auto tagging
US10349033B2 (en) Three-dimensional map generating and displaying apparatus and method
US10582120B2 (en) Systems, methods, and media for providing interactive refocusing in images
Huang et al. Visual odometry and mapping for autonomous flight using an RGB-D camera
Weiss et al. Intuitive 3D maps for MAV terrain exploration and obstacle avoidance
Krull et al. 6-dof model based tracking via object coordinate regression
CN104867142B (zh) 基于三维场景的导航方法
WO2018009473A1 (en) Motion capture and character synthesis
US10140765B2 (en) Staged camera traversal for three dimensional environment
US20220301252A1 (en) View synthesis of a dynamic scene
CN109887003A (zh) 一种用于进行三维跟踪初始化的方法与设备
Greene et al. Flame: Fast lightweight mesh estimation using variational smoothing on delaunay graphs
US20180165860A1 (en) Motion edit method and apparatus for articulated object
JP7138361B2 (ja) ３次元仮想空間モデルを利用したユーザポーズ推定方法および装置
Tatzgern et al. Dynamic compact visualizations for augmented reality
Li et al. 3D reconstruction of indoor scenes via image registration
Huang et al. Network algorithm real-time depth image 3D human recognition for augmented reality
Fanani et al. Keypoint trajectory estimation using propagation based tracking
CN109582134B (zh) 信息显示的方法、装置及显示设备
Gebhardt et al. Optimization-based user support for cinematographic quadrotor camera target framing
US11158122B2 (en) Surface geometry object model training and inference
US20090213121A1 (en) Image processing method and apparatus
Deng et al. Interactive panoramic map-like views for 3D mountain navigation
US20040155886A1 (en) Image simulation processing method, processing apparatus, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15891528

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 19.01.2018)

122 Ep: pct application non-entry in european phase

Ref document number: 15891528

Country of ref document: EP

Kind code of ref document: A1