WO2019242418A1 - 相机定位方法、装置、终端及存储介质 - Google Patents

相机定位方法、装置、终端及存储介质 Download PDF

Info

Publication number
WO2019242418A1
WO2019242418A1 PCT/CN2019/085746 CN2019085746W WO2019242418A1 WO 2019242418 A1 WO2019242418 A1 WO 2019242418A1 CN 2019085746 W CN2019085746 W CN 2019085746W WO 2019242418 A1 WO2019242418 A1 WO 2019242418A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
feature point
position information
image
feature points
Prior art date
Application number
PCT/CN2019/085746
Other languages
English (en)
French (fr)
Inventor
陈超
吴伟
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2020552267A priority Critical patent/JP7026819B2/ja
Priority to EP19822963.5A priority patent/EP3813014B1/en
Publication of WO2019242418A1 publication Critical patent/WO2019242418A1/zh
Priority to US16/928,427 priority patent/US11210810B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P15/00—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
    • G01P15/18—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration in two or more dimensions
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/01—Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
    • G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/42—Determining position
    • G01S19/48—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
    • G01S19/485—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system whereby the further system is an optical system or imaging system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • Embodiments of the present invention relate to the field of computer technology, and in particular, to a camera positioning method, device, terminal, and storage medium.
  • Vision SLAM Simultaneous Localization and Mapping
  • SLAM Simultaneous Localization and Mapping
  • Visual SLAM technology includes a mapping phase and a positioning phase.
  • the camera first captures multiple images of the target environment, extracts features from each image to obtain the feature points of each image, and matches the feature points in each image.
  • the location change determines the location information of each image and the location information of each feature point in each image, thereby constructing an environment map, which includes the location information of each feature point.
  • the target environment can be photographed to obtain the target image, and the target image can be feature extracted to obtain the feature points in the target image, and the feature points in the target image Match the feature points in the environment map, and obtain the camera's position information based on the position information of the matched feature points to achieve camera positioning.
  • each position in the target environment can be captured from multiple different angles to obtain multiple images from multiple perspectives.
  • an environment map containing feature points with multiple perspectives is constructed.
  • the feature points in the target image and the multi-viewpoint feature points in the environment map are matched to obtain the position information of the camera.
  • the camera actually only captures the target environment from one perspective, and feature points that can be observed in other perspectives but cannot be observed in the current perspective are invalid for the camera. Therefore, matching the feature points in the target image with the feature points in the multi-viewpoint in the environment map does not match the actual shooting situation of the camera, resulting in a large amount of invalid calculations and increasing the possibility of incorrect matching.
  • the embodiments of the present application provide a camera positioning method, device, terminal, and storage medium.
  • a camera positioning method executed by a terminal includes:
  • An environment map of a target environment is obtained, where the environment map includes position information of multiple feature points in the target environment and perspective area information of the multiple feature points, and the perspective area information of the feature points is used to represent the target A viewing area where the feature point can be observed in the environment;
  • a camera positioning device the device includes:
  • a map acquisition module configured to acquire an environment map of a target environment, where the environment map includes position information of multiple feature points in the target environment and perspective area information of the multiple feature points, and the perspective area information of the feature points A view area used to indicate that the feature point can be observed in the target environment;
  • a position prediction module configured to predict the position when the camera shoots the target image according to the position information of the camera when the historical image is captured before the target image is captured, to obtain the predicted position information of the camera;
  • a filtering module configured to, based on the predicted position information of the camera, the position information of each feature point in the environment map, and the perspective area information of each feature point, convert the currently unobservable camera in the environment map Feature points are filtered out;
  • a positioning module configured to match the feature points in the target image with the remaining feature points filtered by the environment map to obtain the feature point correspondence, and determine the position information of the camera according to the feature point correspondence, so that The feature point correspondence includes a correspondence between feature points in the target image and matched feature points.
  • a terminal wherein the terminal includes a processor and a memory, and the memory stores computer-readable instructions.
  • the processor causes the processor to perform the following steps. :
  • An environment map of a target environment is obtained, where the environment map includes position information of multiple feature points in the target environment and perspective area information of the multiple feature points, and the perspective area information of the feature points is used to represent the target A viewing area where the feature point can be observed in the environment;
  • a non-transitory computer-readable storage medium stores computer-readable instructions.
  • the one or more processors execute the following steps:
  • An environment map of a target environment is obtained, where the environment map includes position information of multiple feature points in the target environment and perspective area information of the multiple feature points, and the perspective area information of the feature points is used to represent the target A viewing area where the feature point can be observed in the environment;
  • FIG. 1 is a schematic diagram of a feature point provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of another feature point provided by an embodiment of the present application.
  • 3A is a flowchart of a method for constructing an environment map according to an embodiment of the present application
  • FIG. 3B is a flowchart of process steps for obtaining position information and viewing angle area information of the specified feature point for any specified feature point in the original image according to an embodiment of the present application;
  • FIG. 5 is a top view of a target environment provided by an embodiment of the present application.
  • FIG. 6 is a side view of a target environment provided by an embodiment of the present application.
  • FIG. 7 is a top view of a target environment provided by an embodiment of the present application.
  • FIG. 8 is a side view of a target environment provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of a camera positioning method according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a camera positioning device according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • a feature-based map construction and camera positioning method is proposed.
  • an environment map is constructed by matching feature points between images.
  • the current image is captured by the camera, and the feature points in the current image are matched with the feature points in the environment map to achieve camera positioning.
  • a multi-view environment map can be constructed to ensure the full availability of the environment map, but it also brings a problem: when the camera is positioned, the camera is actually only Shooting the target environment from one angle of view, many feature point cameras at the shooting position cannot be captured in the current angle of view, and these feature points are invalid for the camera. At this time, if the feature points in the target image captured by the camera are matched with all the feature points in the environment map, a large amount of invalid calculation will be generated, and the probability of incorrect matching will be increased.
  • an embodiment of the present application provides a camera positioning method, which can determine the position information of each feature point and the perspective area information of each feature point in the environment map during the mapping stage. Information to represent the area where feature points can be observed.
  • the approximate position of the camera is predicted first, and the feature points currently unobservable by the camera are filtered out based on the perspective area information of each feature point, and then the feature points in the target image captured by the camera and the rest of the environment map
  • the feature points are matched to determine the position of the camera, which not only reduces the calculation amount, but also the feature points used for matching are consistent with the actual shooting conditions of the camera, reducing the possibility of matching errors, which is equivalent to improving the positioning accuracy.
  • the “camera” mentioned in this application refers to a device or device with an image acquisition function, which can be used to capture an image.
  • the "camera” may be an image acquisition device specifically used to acquire an image; it may also be an image acquisition device configured in a terminal, and the image acquisition device may be a camera.
  • the embodiments of the present application can be applied to various scenarios for positioning a terminal configured with a camera.
  • a user issues an item delivery task to a delivery robot through a client and a server, and the item delivery task includes a loading location and a delivery location of the item.
  • the delivery robot may first use the method provided in the embodiment of the present application to perform positioning, and then plan the route according to the current position, the place where the item is loaded, and the place of delivery: first, plan the route from the current position to the place where it is loaded , Then plan the route from the loading point to the delivery point, and then move according to the planned path.
  • the distribution robot will also perform real-time positioning, control the speed and direction of the movement to ensure that it follows the planned route and avoids deviating from the route .
  • an environment map can be constructed.
  • the environment map contains not only a map of the real scene, but also virtual elements added at certain locations in the real scene, such as virtual pets, virtual coupons, etc. .
  • the method provided in the embodiment of the present application may be used for positioning, determining the current location of the terminal and the virtual elements set near the terminal, and displaying the currently captured real scene image and the virtual elements set nearby on the display interface of the terminal. To form a display interface combined with virtual reality.
  • FIG. 3A is a flowchart of a method for constructing an environment map provided by an embodiment of the present application
  • FIG. 4 is a flowchart of an operation for constructing an environment map provided by an embodiment of the present application. Taking the execution subject of the method for constructing the environment map as a terminal, the process of constructing the environment map is exemplified. 3A and 4, the method includes:
  • the terminal acquires multiple original images obtained by the camera shooting the target environment.
  • the terminal may be a mobile phone, a tablet computer, a robot, a smart camera, etc.
  • the terminal is equipped with a camera, and the camera can shoot the current environment.
  • the target environment can be a variety of environments such as parks, buildings, and warehouses for storing items. When an environment map of the target environment needs to be constructed, the terminal can move in the target environment and obtain multiple original images by shooting the target environment with a camera.
  • the number of the plurality of original images may be determined according to the accuracy requirement of the environment map. And for the same position in the target environment, the position can be photographed from multiple different perspectives to obtain a multi-angle original image.
  • the terminal determines a standard image according to the multiple original images, and determines position information when the camera shoots the standard image.
  • the terminal can determine the standard image and the position information when the camera shoots the standard image. In the subsequent process, the terminal will use the standard image as a reference and determine the position information when the camera shoots other original images according to the position difference between the other original image and the standard image.
  • the terminal may determine the first original image captured by the camera as a standard image, and initialize position information when the camera captures the standard image. For example, the terminal may use the position when the terminal captured the standard image as the origin position. Create the environment coordinate system, that is, the position information of the standard image is (0,0,0). For each original image taken after that, position information can be determined based on the position difference from the standard image. In addition, considering that as the camera moves continuously, the original image captured is becoming less and less the same as the standard image, a new original image can be selected as the updated standard image, and it is determined that the camera takes the updated image. Position information at the time of the standard image, and each original image captured thereafter can be determined based on the position difference from the updated standard image.
  • the terminal matches the feature points in the original image with the feature points in the standard image to obtain the feature point correspondence, and according to the feature point correspondence and the camera when shooting the standard image Position information, to obtain the position information when the camera takes the original image.
  • the terminal After the terminal obtains multiple original images, it can perform feature extraction on each original image, determine the position information of the feature points in each original image in the original image, and determine the descriptors of each feature point.
  • Feature points are described.
  • the terminal can use FAST (Features from Accelerated Segment Test, accelerated segmentation test feature), Harris (a corner detection algorithm), SURF (Speed Up Robust Feature, accelerated robust features), BRISK (Binary Robust Invariant Variable Variable Key Points, Binary Lu) Key points such as stick-invariant and scalable), ORB (a feature point detection algorithm) and other algorithms extract feature points.
  • FAST Features from Accelerated Segment Test, accelerated segmentation test feature
  • Harris a corner detection algorithm
  • SURF Speed Up Robust Feature, accelerated robust features
  • BRISK Binary Robust Invariant Variable Variable Key Points, Binary Lu
  • Key points such as stick-invariant and scalable
  • ORB a feature point detection algorithm
  • the feature points in the original image can be matched with the feature points in the standard image.
  • the feature point correspondence includes the correspondence between the feature points in the original image and the feature points matched in the standard image, which is equivalent to generating the original The pair of feature points that match each other between the image and the standard image.
  • each feature point in the original image can be combined with each feature point in the standard image in pairs according to the distance metric, and the descriptors of the two feature points in each combination can be calculated.
  • Distance for each feature point in the original image, the feature point closest to the feature point in the standard image is selected as the matching feature point of the feature point.
  • a feature point in the standard image that is closest to the feature point and less than a preset distance is selected as the matching feature point of the feature point, thereby establishing a relationship between the feature point in the original image and the matching feature point in the standard image. Corresponding relationship.
  • the position change information of the original image relative to the standard image can be determined, and then iteration is performed based on the position information of the standard image and the position change information of the original image relative to the standard image. Calculate to get the position information when the camera takes the original image.
  • the image 1 is determined as a standard image, and the position information is (x1, y1, z1), and then the position change information of the image 2 relative to the image 1 can be determined as ( ⁇ x1, ⁇ y1, ⁇ z1) according to the correspondence relationship with the feature points of the image ),
  • the position change information of image 3 relative to image 1 is ( ⁇ x2, ⁇ y2, ⁇ z2), then it is determined that the position information (x2, y2, z2) of image 2 is (x1 + ⁇ x1, y1 + ⁇ y1, z1 + ⁇ z1),
  • the position information (x3, y3, z3) of the image 3 is (x1 + ⁇ x2, y1 + ⁇ y2, z1 + ⁇ z2).
  • the image 3 can be updated to a second standard image, and then the position change information of the image 4 relative to the image 3 is determined according to the correspondence relationship with the feature points of the image 3 as ( ⁇ x3, ⁇ y3, ⁇ z3), then the position information of the image 4 Is (x3 + ⁇ x3, y3 + ⁇ y3, z3 + ⁇ z3).
  • the position information of the camera when capturing any image may be expressed in the form of a rotation and translation matrix, and the terminal may obtain a rotation and translation matrix of a standard image, and according to the correspondence between the feature points and the position of each feature point Information to determine the rotation and translation matrix of the original image relative to the standard image.
  • the rotation and translation matrix includes rotation parameters and displacement parameters of the original image relative to the standard image. Then, the rotation and translation matrix of the standard image and the rotation and translation matrix of the original image relative to the standard image are iteratively calculated to obtain the rotation and translation of the original image.
  • Matrix, the displacement parameter in the rotation and translation matrix is the position information when the camera captures the original image.
  • the terminal after the terminal obtains the position information when the camera takes the original image, it can also remove the noisy feature points in the original image, such as the feature points of dynamic objects or unmatchable feature points in the original image. .
  • the terminal can analyze the captured multiple original images to obtain position change information between any two original images.
  • a feature point is in any two original images
  • the actual position change information of the two does not match the position change information of the two original images, it means that during the process of the camera taking the two original images, the object where the feature point is moved within the target environment, so it can be determined
  • the feature point is a feature point of a dynamic object, not a feature point of the target environment itself, and the feature point is filtered out at this time.
  • the terminal obtains the position information of each feature point and the angle of view area information of each feature point according to the position information when the camera captures the original image.
  • the process of obtaining the position information and the perspective area information of the specified feature point may include the following steps S3041-S3042:
  • the depth of the specified feature point in the original image is used to indicate the distance between the camera and the specified feature point when the original image is captured, and it can indicate the relative position relationship between the specified feature point and the camera, and determine the After the position information, according to the position information of the camera and the depth of the specified feature point in the original image, the position information of the specified feature point can be obtained.
  • plane estimation is performed within a preset range of the designated feature point to determine at least one plane passing the designated feature point.
  • a complete object is composed of one or more surfaces, and the position of each surface will affect the observable area of each feature point on the surface of the object. Therefore, when the terminal performs an observable area analysis on the specified feature point, it comprehensively considers the position information of the designated feature point and the position information of other points in the original image, performs plane estimation within a preset range of the designated feature point, and determines that the designated feature point has passed. At least one plane of the point. At this time, the at least one plane may be considered to be at least one surface of the object on which the feature point is located. Then, a viewing angle area where the designated feature point can be observed is determined according to the position of the at least one plane.
  • the preset range may be determined in the original image according to the position of the specified feature point.
  • the preset range may be a circular range with the specified feature point as the center and a certain distance as the radius, or it may be a specified range
  • the feature point is a range formed by a plurality of fan-shaped regions selected by the vertices.
  • an edge line segmentation algorithm of a grayscale image or an edge line segmentation algorithm of a depth map may be used to predict the plane parameters of at least one plane.
  • the terminal may select at least three feature points within a preset range of the designated feature points, and specify the feature points on the original image within a polygon formed by the at least three feature points, and the at least three features
  • the points are on the same plane but not on the same straight line in the environment map, and the distance between the designated feature point and the plane where the at least three feature points are located is calculated to determine whether the distance is less than a preset distance. If the distance between the designated feature point and the plane is less than the preset distance, it is determined that the designated feature point is located on the plane where the at least three feature points are located, that is, only one plane passes the designated feature point.
  • At least three fan-shaped regions with the specified feature points as vertices can be determined, and from each fan-shaped region, a feature point closest to the specified feature point is selected, thereby selecting at least three Feature points.
  • FIG. 5 is a top view of the target environment provided by the embodiment of the present application
  • FIG. 6 is a side view of the target environment provided by the embodiment of the present application.
  • feature point 1 is a feature point to be subjected to a perspective analysis.
  • the three feature points of feature point 2, feature point 3, and feature point 4 are selected near point 1. These three feature points are not located on the same straight line. Based on these three feature points, a plane can be predicted. Feature point 1 and The distance between the planes is small, so it can be determined that the feature point 1 is also located on the plane.
  • edge detection is performed within the preset range to obtain the designated feature point as At least one edge line of the vertex, and obtain at least four feature points within a preset range, using the specified feature point as the intersection point of at least two planes, using at least one edge line as the intersection line of any two planes, and at least The four feature points are used as points on at least two planes to obtain the at least two planes.
  • the at least four feature points may include at least three feature points previously selected, and may also include at least one feature point that is additionally selected. It is only necessary to ensure that the at least four feature points are not located on the same straight line, and the at least four The feature points are not located on the at least one edge line, so that at least two planes can be determined according to the designated feature point, the at least four feature points, and the at least one edge line.
  • FIG. 7 is a top view of the target environment provided by the embodiment of the present application
  • FIG. 8 is a side view of the target environment provided by the embodiment of the present application.
  • the feature point 2, the feature point 3, and the feature point 4 are located in the same On the plane, the distance between feature point 1 and the plane is large, so it can be determined that feature point 1 is not located on the plane.
  • feature point 5 is selected near feature point 1 and edge detection is performed to obtain the passed feature.
  • An edge line of point 1 according to the position of the five feature points and the detected edge line, two planes can be determined, where feature point 1, feature point 2 and feature point 3 are located on one plane, feature point 1, feature Point 4 and feature point 5 are located on the other plane.
  • the camera configured by the terminal is a depth camera
  • the depth camera can obtain the depth of each point in the image when capturing the image, for the specified feature point
  • the The depths of multiple points in the preset range in the original image and based on the depth of each point in the original image
  • the precise positions of multiple feature points in the target environment can be determined, and based on the multiple feature points'
  • the accurate position can construct at least one plane such that the specified feature points are located on each of the at least one plane, and the points within a preset range of the specified feature points are located on any one or any two planes of the at least one plane.
  • the multiple points in the preset range may include feature points extracted from the original image, and may also include points other than the feature points.
  • S3043. Determine the first connected area and the second connected area obtained by dividing the target environment by at least one plane, and determine the area where the camera is located in the first connected area and the second connected area as a perspective at which the designated feature point can be observed. Area to get the perspective area information.
  • any two or more planes in the at least one plane may divide the target environment into two connected regions, called a first connected region and a second connected region.
  • a plane may divide the target environment into an upper side and a lower side.
  • Two connected areas on the side, two planes can divide the target environment into two connected areas inside the dihedral angle and two outside the dihedral angle, etc.
  • Three planes can also divide the target environment into two connected areas.
  • the angle of view area is the area information of the specified feature point.
  • the feature point 1 is located on the plane and the observable angle is 180 degrees.
  • the feature point 1 can be observed, and when it is located on the lower side of the plane, the feature point 1 cannot be observed.
  • the feature point 1 is located on the intersection of two planes, the dotted line represents the center vector of the normal vectors of the two planes, and the observable area of the feature point 1 is the center vector with the center axis and the angle w Area, feature point 1 can be observed when the camera is located in the area, and feature point 1 cannot be observed when it is not located in the area.
  • the terminal constructs an environment map according to the position information of each feature point in the multiple original images and the perspective area information of each feature point.
  • the environment map includes the position information of multiple feature points and the perspectives of the multiple feature points. Area information.
  • the terminal can subsequently match the feature points in the target image currently captured by the camera with the feature points in the environment map, determine the position of the camera according to the matching result, and implement camera positioning.
  • the position information and the perspective area information of the matching feature points in different images may also be optimized to obtain Globally consistent environment map.
  • the position information of the feature points in an image is different from the position information of the matching feature points in other multiple images, the position information of the feature points in the image can be corrected to ensure the position change information between different images. It is consistent with the position change information of the matching feature points between different images.
  • the method provided in the embodiment of the present application obtains position information of each original image by using multiple original images obtained by shooting a target environment according to a camera, and obtains position information of each feature point and each of the original images according to the position information of each original image.
  • the perspective area information of the feature points not only constructs a multi-view environment map, but also can use the perspective area information to represent the area where the feature points can be observed.
  • the camera can then be disabled based on the predicted position information of the camera and the perspective area information of the feature points.
  • the observed feature points are filtered out, reducing the amount of calculation and improving the accuracy of positioning.
  • FIG. 9 is a flowchart of a camera positioning method according to an embodiment of the present application. Taking the execution subject of the camera positioning method as a terminal, the process of positioning a camera when it moves in a target environment is exemplified. Referring to FIG. 9, the method includes:
  • the terminal obtains an environment map of a target environment.
  • the environment map includes position information of multiple feature points in the target environment and perspective area information of the multiple feature points.
  • the perspective area information of the feature points is used to represent the perspective area where the feature points can be observed in the target environment.
  • the terminal predicts the position when the camera captures the target image according to the position information when the camera captures the historical image before capturing the target image, and obtains the predicted position information of the camera.
  • the terminal is configured with a camera, and during the process of the terminal moving in the target environment, the camera can shoot the current target environment to obtain one or more images.
  • the historical image is an image whose shooting time is before the shooting time of the target image.
  • the location information of the historical image captured by the camera may specifically be the location information of a historical image captured by the camera (for example, the previous image captured by the camera before the target image is captured), or may be based on the camera.
  • step S902 may include any one of step A, step B, and step C:
  • Step A Use the constant speed model to obtain the predicted position information of the camera according to the position information of the previous image before the camera captures the target image and the time interval of the camera capturing any two adjacent images in time.
  • the camera can be set at a fixed time interval, and one image is captured at each time interval, so as to obtain multiple images.
  • any two temporally adjacent images refer to any two temporally adjacent images among a plurality of images captured by the camera, and the time interval between the two images is equal to the shooting time interval set by the camera.
  • a constant speed model can be obtained.
  • the constant speed model sets the camera to move at a fixed speed, which can be set by the terminal by default or according to the walking speed of ordinary people.
  • a constant speed model can be used to obtain the displacement amount of the camera within the time interval, and according to the position when the last image was taken Information and the amount of displacement to obtain the predicted position information of the camera.
  • Step B According to the position information of the previous image before the target image is taken by the camera, the time interval between the two adjacent images in the time taken by the camera, and the acceleration detected by the sensor, a variable speed model is used to obtain the predicted position information of the camera .
  • the camera can be set at a fixed time interval, and one image is taken at each time interval to obtain multiple images.
  • any two temporally adjacent images refer to any two temporally adjacent images among a plurality of images captured by the camera, and the time interval between the two images is equal to the shooting time interval set by the camera. It is assumed that the camera can obtain a gear shift model when the camera moves at a variable speed.
  • the gear shift model sets the camera to move at a certain acceleration, and can calculate the displacement of the camera within the time interval according to the input initial speed, acceleration, and time interval.
  • the terminal detects the acceleration through the configured sensors, and uses a variable speed model based on the initial speed and acceleration when the camera takes the last image. Obtain the amount of displacement of the camera during the time interval, and obtain the predicted position information of the camera according to the position information when the camera captured the previous image and the amount of displacement.
  • Step C During the process from the last image captured by the camera to the target image, the terminal acquires the position change information of the target image relative to the previous image by tracking the feature points in the previous image, and captures the last image according to the camera.
  • the position information at the time of the image and the position change information of the target image relative to the previous image are used to obtain the predicted position information of the camera.
  • the position information of the camera when the previous image is captured can be obtained, and during the process from when the camera captures the last image to the shooting target image, the terminal can track the feature points in the previous image. According to the movement information of the feature point, position change information of the target image relative to the previous image is obtained, and then iterative calculation is performed according to the position information of the camera when the previous image was taken and the position change information of the target image relative to the previous image. Get the camera's predicted position information.
  • the terminal may use the optical flow matching algorithm to obtain the motion information of the feature points, and obtain the position information of the target image relative to the previous image based on the motion information.
  • the terminal may use the optical flow matching algorithm to obtain the motion information of multiple feature points, and determine the position information of the multiple feature points in the previous image and the position information in the target image based on the motion information, thereby determining the relativeness of the target image.
  • the displacement parameters in the rotation and translation matrix are the position change information of the target image relative to the previous image.
  • the terminal may also use other tracking algorithms to track the feature points.
  • the terminal has not yet obtained the position information when the previous image was taken before the camera, or the difference between the target image and the previous image was too large because the position of the terminal changed too much , There is no matching feature point between the two.
  • the predicted position information of the target image cannot be obtained based on the position information of the previous image taken by the camera before the target image.
  • relocation can be performed to compare the feature points of the target image with those in the environment map. The multiple feature points are matched separately, and the feature point that most closely matches the feature point of the target image is selected, and the predicted position information of the target image is initially determined according to the position of the selected matched feature point.
  • each feature point in the target image can be combined with each feature point in the environment map in pairs according to the distance metric, and the descriptors between the two feature points in each combination can be calculated.
  • Distance the smaller the distance, the smaller the difference between the descriptors representing the two feature points, the more likely they are the same feature points.
  • the feature point closest to the feature point in the environment map is selected as the matching feature point of the feature point.
  • a feature point in the environment map that is closest to the feature point and is less than a preset distance is selected as the matching feature point of the feature point.
  • the terminal may determine the geographic area where the camera is located by using geographic information detected by an external sensor, limit feature matching to a relatively small geographic area, and perform feature point matching in the geographic area where the camera is located Get the current predicted position information of the camera.
  • the external sensor may be a WIFI (Wireless Fidelity) device, a UWB (Ultra Wideband) device, etc. connected to the terminal.
  • the external sensor configures geographic location information according to its location. As the terminal moves, When the terminal is located near an external sensor, it can connect to the external sensor and receive geographic location information sent by the external sensor, and determine the current geographic area based on the geographic location information.
  • the terminal when the terminal takes the position information of other historical images (for example, the penultimate image taken by the camera before the target image is taken), the terminal predicts the position when the camera takes the target image to obtain the camera ’s
  • the predicted position information For a specific implementation of the predicted position information, reference may be made to the foregoing terminal to predict the position when the camera shoots the target image according to the position information of the previous image, and a detailed description of the steps of obtaining the predicted position information of the camera is omitted here.
  • the terminal filters out feature points that are not currently observable by the camera according to the predicted position information of the camera, the position information of each feature point in the environment map, and the perspective area information of each feature point.
  • the viewpoint area information of a feature point is used to indicate the viewpoint area where the feature point can be observed.
  • the feature point can be observed, and the captured target image may include the feature point.
  • the camera is not in the viewing area of a feature point, even if the feature point is not far from the camera position, the camera cannot observe the feature point, and the captured target image will not include the feature point.
  • the terminal can determine whether the camera is within the perspective area of the feature point according to the predicted position information of the camera and the perspective area information of the feature point.
  • the feature point is retained, and when it is determined that the camera is not within the feature point's perspective area, the feature point is filtered out.
  • the feature points currently unobservable by the camera in the environment map can be filtered out, and only the feature points currently observable by the camera are retained.
  • the terminal matches the feature points in the target image with the remaining feature points after filtering the environment map to obtain the feature point correspondence, and obtains the position information of the camera according to the feature point correspondence.
  • the feature point correspondence includes the feature points in the target image. Correspondence with matching feature points.
  • the terminal can obtain the descriptors of each feature point in the target image and the descriptors of each feature point remaining after filtering the environment map, and pair each feature point in the target image with the remaining feature points after filtering the environment map. Combination, calculate the distance between the descriptors of the two feature points in each combination. The smaller the distance, the smaller the difference between the descriptors representing the two feature points, and the more likely they are the same feature point.
  • the feature point closest to the feature point in the environment map is selected as the matching feature point.
  • a feature point in the environment map that is closest to the feature point and less than a preset distance is selected as a matching feature point to establish a feature point correspondence.
  • the position information of the camera can be obtained, and the position information is used to represent the position information of the camera currently in the target environment.
  • PnP perspective of n feature points
  • other algorithms can be used to obtain the position information of the camera.
  • the position of the camera in the environment map can be determined, and corresponding operations can be performed based on the position. For example, in the scenario of delivering goods, it can be judged whether it deviates from the planned route according to the current position of the delivery robot. If the delivery robot deviates from the planned route, the route planning is re-performed according to the current position of the delivery robot and the delivery location. Alternatively, in the scenario of a virtual reality game, after the location of the terminal is determined, the virtual element set at the location may be displayed on the display interface of the terminal for the user to interact with the virtual element.
  • the method provided in the embodiment of the present application obtains an environment map of a target environment, where the environment map includes position information of multiple feature points and perspective area information of the multiple feature points, and the perspective area information is used to indicate observable feature points
  • the angle of view area is based on the position information of the camera when the previous image was captured before the target image was captured, and the position of the camera when the target image was captured is predicted to obtain the predicted position information of the camera.
  • the position information of each feature point and the perspective area information of each feature point are used to filter out the feature points that are not currently observable by the camera, and match the feature points in the target image with the remaining feature points after filtering from the environment map to obtain the features.
  • Point correspondence to determine the camera's position information and achieve camera positioning which reduces the amount of calculation and increases the calculation speed, and the feature points used for matching are consistent with the actual shooting situation of the camera, reducing the possibility of matching errors, which is equivalent to Improved positioning accuracy and robustness of map tracking.
  • the embodiments of the present application provide a SLAM method based on the visual feature view area, which greatly improves the robustness of positioning and mapping, provides reliable information for camera positioning, and can be widely used in robot navigation, game character positioning, and drone positioning.
  • SLAM method based on the visual feature view area
  • the stability of the entire system is enhanced, the amount of calculation is reduced, the requirements for hardware configuration are reduced, and costs are saved.
  • FIG. 10 is a schematic structural diagram of a camera positioning device according to an embodiment of the present application.
  • the device includes:
  • a map acquisition module 1001 is configured to acquire an environment map of a target environment.
  • the environment map includes position information of multiple feature points and perspective area information of multiple feature points in the target environment. Viewing area where feature points are observed.
  • the position prediction module 1002 is configured to predict the position when the camera captures the target image according to the position information when the camera captures the historical image before the target image is captured, to obtain the predicted position information of the camera.
  • the filtering module 1003 is configured to filter out feature points that are not currently observable by the camera in the environment map according to the predicted position information of the camera, the position information of each feature point in the environment map, and the perspective area information of each feature point.
  • a positioning module 1004 is configured to match the feature points in the target image with the remaining feature points after filtering the environment map to obtain the feature point correspondence, and determine the position information of the camera according to the feature point correspondence.
  • the feature point correspondence includes the target image. Correspondence between the feature points and the matched feature points.
  • the location prediction module 1002 includes:
  • a first prediction unit configured to obtain the predicted position information of the camera by using a constant speed model according to the position information when the camera takes the previous image and the time interval between the two images when the camera is adjacent to each other in time; or,
  • the second prediction unit is configured to obtain the predicted position information of the camera by using a variable speed model according to the position information of the camera when the previous image was taken, the time interval between the two images when the camera was adjacent to each other, and the acceleration detected by the sensor. ;or,
  • the third prediction unit is configured to obtain position change information of the target image with respect to the previous image by tracking feature points in the previous image during the process of shooting the previous image to the target image by the camera, and shoot according to the camera.
  • the position information of the previous image and the position change information of the target image relative to the previous image are used to obtain the predicted position information of the camera.
  • the camera positioning device further includes:
  • a relocation module configured to: when the camera is started, or when the predicted position information of the camera cannot be obtained based on the position information when the previous image is taken by the camera, perform the feature point of the target image with multiple feature points in the environment map Match to get the predicted position information of the camera.
  • the filtering module 1003 is further configured to, for each feature point in the environment map, filter the feature points when it is determined that the camera is not in the perspective area of the feature points according to the predicted position information of the camera.
  • the map acquisition module includes:
  • a standard determining unit configured to obtain multiple original images obtained by the camera shooting the target environment, determine a standard image based on the multiple original images, and determine position information when the camera captures the standard image;
  • a position acquisition unit configured to match feature points in the original image with feature points in the standard image for any original image other than the standard image to obtain feature point correspondence, and the feature point correspondence includes the original image Correspondence between feature points and feature points matched in the standard image; according to the correspondence between feature points and the position information when the camera shoots the standard image, obtain the position information when the camera shoots the original image;
  • An information acquiring unit configured to acquire position information of each feature point and perspective area information of each feature point in the original image according to the position information when the camera captures the original image;
  • a map construction unit is configured to construct an environment map according to the position information of each feature point and the perspective area information of each feature point in multiple original images.
  • the information acquisition unit is further configured to obtain, for any specified feature point in the original image, the position of the specified feature point according to the position information when the camera shoots the original image and the depth of the specified feature point in the original image.
  • Information according to the position information of the specified feature point and the position information of other points in the original image, plane estimation is performed within a preset range of the specified feature point to determine at least one plane passing the specified feature point;
  • the first connected area and the second connected area obtained by dividing the environment, determine the area where the camera is located in the first connected area and the second connected area as the viewing angle area where the specified feature point can be observed, and obtain the viewing angle area information.
  • the information obtaining unit is further configured to:
  • the designated feature points are located inside a polygon formed by at least three feature points. At least three feature points are on the same plane but not on the same line in the environment map. ; And if the distance between the specified feature point and the plane on which the at least three feature points are located is less than a preset distance, determine that the designated feature point is on the plane on which the at least three feature points are located.
  • the information acquisition unit is further configured to: if the distance between the specified feature point and the plane where the at least three feature points are located is not less than a preset distance, perform edge detection within a preset range to obtain the specified feature
  • a point is at least one edge line of a vertex, and at least four feature points in a preset range are obtained; and a designated feature point is taken as an intersection point of at least two planes, and at least one edge line is taken as an intersection line of any two planes, Take at least four feature points as points on at least two planes to obtain at least two planes.
  • the camera is a depth camera
  • the information obtaining unit is further configured to obtain the depth in the original image of a plurality of points within a preset range of the specified feature points, and determine at least one of the points according to the depth of each point in the original image.
  • Plane the designated feature points are located on each of the at least one plane, and the points within the preset range of the designated feature points are located on any one or both of the planes.
  • the camera positioning device provided in the foregoing embodiment performs camera positioning
  • only the above-mentioned division of functional modules is used as an example.
  • the foregoing functions may be allocated by different functional modules according to needs. That is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above.
  • the camera positioning device and the camera positioning method embodiments provided by the foregoing embodiments belong to the same concept. For specific implementation processes, refer to the method embodiments, and details are not described herein again.
  • FIG. 11 shows a structural block diagram of a terminal 1100 provided by an exemplary embodiment of the present application.
  • the terminal 1100 can be a portable mobile terminal, such as: smartphone, tablet, MP3 player (Moving Picture Experts Group Audio Layer III, moving picture expert compression standard audio level 3), MP4 (Moving Picture Experts Group Audio Audio Layer IV, dynamic Image expert compression standard audio level 4) Player, laptop, desktop computer, head-mounted device, or any other smart terminal.
  • the terminal 1100 may also be called other names such as user equipment, portable terminal, laptop terminal, and desktop terminal.
  • the terminal 1100 includes a processor 1101 and a memory 1102.
  • the processor 1101 may include one or more processing cores, such as a 4-core processor, a 5-core processor, and the like.
  • the processor 1101 may use at least one hardware form among DSP (Digital Signal Processing, Digital Signal Processing), FPGA (Field-Programmable Gate Array, Field Programmable Gate Array), and PLA (Programmable Logic Array). achieve.
  • the processor 1101 may also include a main processor and a co-processor.
  • the main processor is a processor for processing data in the awake state, also referred to as a CPU (Central Processing Unit).
  • the co-processor is Low-power processor for processing data in standby.
  • the processor 1101 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is responsible for rendering and drawing content required to be displayed on the display screen.
  • the processor 1101 may further include an AI (Artificial Intelligence) processor, and the AI processor is configured to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the memory 1102 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 1102 may further include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash storage devices.
  • the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction, and the at least one instruction is used by the processor 1101 to implement camera positioning provided by the method embodiment in this application. method.
  • the terminal 1100 may further include a peripheral device interface 1103 and at least one peripheral device.
  • the processor 1101, the memory 1102, and the peripheral device interface 1103 may be connected through a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 1103 through a bus, a signal line, or a circuit board.
  • the peripheral device includes at least one of a radio frequency circuit 1104, a touch display screen 1105, a camera 1106, an audio circuit 1107, a positioning component 1108, and a power supply 1109.
  • the peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102.
  • the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1101, the memory 1102, and the peripheral device interface 1103 or Both can be implemented on separate chips or circuit boards, which is not limited in this embodiment.
  • the radio frequency circuit 1104 is used to receive and transmit an RF (Radio Frequency) signal, also called an electromagnetic signal.
  • the radio frequency circuit 1104 communicates with a communication network and other communication devices through electromagnetic signals.
  • the radio frequency circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like.
  • the radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G, and 11G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network.
  • the radio frequency circuit 1104 may further include circuits related to Near Field Communication (NFC), which is not limited in this application.
  • NFC Near Field Communication
  • the display screen 1105 is used to display a UI (User Interface).
  • the UI may include graphics, text, icons, videos, and any combination thereof.
  • the display screen 1105 also has the ability to capture a touch signal on or above the surface of the display screen 1105.
  • the touch signal can be input to the processor 1101 as a control signal for processing.
  • the display screen 1105 may also be used to provide a virtual button and / or a virtual keyboard, which is also called a soft button and / or a soft keyboard.
  • one display screen 1105 may be provided, and the front panel of the terminal 1100 is provided.
  • At least two display screens 1105 may be provided on different surfaces of the terminal 1100, or in a folded design.
  • the display screen 1105 may be a flexible display screen disposed on a curved surface or a folded surface of the terminal 1100.
  • the display screen 1105 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen.
  • the display 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).
  • the camera assembly 1106 is used to capture images or videos.
  • the camera assembly 1106 includes a front camera and a rear camera.
  • the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal.
  • the camera assembly 1106 may further include a flash.
  • the flash can be a monochrome temperature flash or a dual color temperature flash.
  • a dual color temperature flash is a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.
  • the audio circuit 1107 may include a microphone and a speaker.
  • the microphone is used to capture the sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1101 for processing, or input to the radio frequency circuit 1104 to implement voice communication.
  • the microphone can also be an array microphone or an omnidirectional microphone.
  • the speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves.
  • the speaker can be a traditional film speaker or a piezoelectric ceramic speaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for ranging purposes.
  • the audio circuit 1107 may further include a headphone jack.
  • the positioning component 1108 is used for positioning the current geographic position of the terminal 1100 to implement navigation or LBS (Location Based Service).
  • the positioning component 1108 may be a positioning component based on a US-based GPS (Global Positioning System), a Beidou system in China, a Granas system in Russia, or a Galileo system in the European Union.
  • the power supply 1109 is used to power various components in the terminal 1100.
  • the power source 1109 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery.
  • the rechargeable battery may support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 1100 further includes one or more sensors 1110.
  • the one or more sensors 1110 include, but are not limited to, an acceleration sensor 1111, a gyro sensor 1112, a pressure sensor 1113, a fingerprint sensor 1114, an optical sensor 1115, and a proximity sensor 1116.
  • the acceleration sensor 1111 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1100.
  • the acceleration sensor 1111 may be used to detect components of the acceleration of gravity on three coordinate axes.
  • the processor 1101 may control the touch display screen 1105 to display a user interface in a landscape view or a portrait view according to a gravity acceleration signal captured by the acceleration sensor 1111.
  • the acceleration sensor 1111 may also be used for shooting a game or a user's motion data.
  • the gyro sensor 1112 can detect the body direction and rotation angle of the terminal 1100, and the gyro sensor 1112 can cooperate with the acceleration sensor 1111 to capture a 3D action of the user on the terminal 1100. Based on the data captured by the gyro sensor 1112, the processor 1101 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 1113 may be disposed on a side frame of the terminal 1100 and / or a lower layer of the touch display screen 1105.
  • a user's holding signal to the terminal 1100 can be detected, and the processor 1101 can perform left-right hand recognition or shortcut operation according to the holding signal captured by the pressure sensor 1113.
  • the processor 1101 controls the operable controls on the UI interface according to the user's pressure operation on the touch display screen 1105.
  • the operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 1114 is used to capture a user's fingerprint, and the processor 1101 recognizes the identity of the user based on the fingerprint captured by the fingerprint sensor 1114, or the fingerprint sensor 1114 recognizes the identity of the user based on the captured fingerprint. When identifying the user's identity as a trusted identity, the processor 1101 authorizes the user to have related sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 1114 may be disposed on the front, back, or side of the terminal 1100. When a physical button or a manufacturer's logo is set on the terminal 1100, the fingerprint sensor 1114 can be integrated with the physical button or the manufacturer's logo.
  • the optical sensor 1115 is used to capture the ambient light intensity.
  • the processor 1101 may control the display brightness of the touch display screen 1105 according to the ambient light intensity captured by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1105 is reduced.
  • the processor 1101 may also dynamically adjust the shooting parameters of the camera component 1106 according to the ambient light intensity captured by the optical sensor 1115.
  • the proximity sensor 1116 also called a distance sensor, is usually disposed on the front panel of the terminal 1100.
  • the proximity sensor 1116 is used to capture the distance between the user and the front of the terminal 1100.
  • the processor 1101 controls the touch display screen 1105 to switch from the bright screen state to the closed screen state; when the proximity sensor 1116 detects When the distance between the user and the front of the terminal 1100 gradually becomes larger, the processor 1101 controls the touch display screen 1105 to switch from the rest screen state to the bright screen state.
  • FIG. 11 does not constitute a limitation on the terminal 1100, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.
  • An embodiment of the present application further provides a terminal for camera positioning.
  • the terminal includes a processor and a memory.
  • the memory stores computer-readable instructions.
  • the processor executes the foregoing. Steps of the camera positioning method.
  • the steps of the camera positioning method herein may be the steps in the camera positioning method of each of the above embodiments.
  • An embodiment of the present application further provides a computer-readable storage medium that stores computer-readable instructions.
  • the processor When the computer-readable instructions are executed by a processor, the processor causes the processor to execute the steps of the camera positioning method described above.
  • the steps of the camera positioning method herein may be the steps in the camera positioning method of each of the above embodiments.
  • the computer-readable instructions may include a program, a code set, and the like.
  • the program may be stored in a computer-readable storage medium.
  • the storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

一种相机定位方法，包括：获取目标环境的环境地图；根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息；根据相机的预测位置信息、环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将环境地图中相机当前不可观测到的特征点过滤掉；将目标图像中的特征点与环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据特征点对应关系确定相机的位置信息。

Description

相机定位方法、装置、终端及存储介质
本申请要求于2018年06月19日提交中国专利局，申请号为201810630115.7、发明名称为“相机定位方法、装置、终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明实施例涉及计算机技术领域，特别涉及一种相机定位方法、装置、终端及存储介质。
背景技术
视觉SLAM(Simultaneous Localization and Mapping，同时定位与地图创建)技术是通过相机对目标环境进行观测，实现环境地图构建和相机定位的技术，目前已逐渐成为计算机视觉领域的热门研究方向。
视觉SLAM技术包括建图阶段和定位阶段。在建图阶段，相机先拍摄目标环境的多张图像，对每张图像进行特征提取得到每张图像的特征点，并将各个图像中的特征点进行匹配，根据相互匹配的特征点之间的位置变化，确定每张图像的位置信息以及每张图像中每个特征点的位置信息，从而构建环境地图，该环境地图包括每个特征点的位置信息。在之后的定位阶段，相机位于目标环境中的某一位置时，可以对目标环境进行拍摄，得到目标图像，对目标图像进行特征提取得到目标图像中的特征点，并将目标图像中的特征点与环境地图中的特征点进行匹配，根据匹配特征点的位置信息获取相机的位置信息，实现相机定位。
其中，在建图阶段为了构建出完整全面的环境地图，拍摄目标环境的多张图像时，针对目标环境中的每个位置可以从多个不同的角度进行拍摄，得到多视角的多张图像，从而构建出包含多视角特征点的环境地图。那么，在 定位阶段会将目标图像中的特征点与环境地图中多视角的特征点均进行匹配，以获取相机的位置信息。
然而，相机实际上仅是从一个视角来拍摄目标环境，在其他视角下可以观测到而在当前视角下无法观测到的特征点对相机来说是无效的。因此，将目标图像中的特征点与环境地图中多视角的特征点均进行匹配的方式不符合相机的真实拍摄情况，产生了大量无效的计算量，而且加大了错误匹配的可能性。
发明内容
本申请实施例提供了一种相机定位方法、装置、终端及存储介质。
一种相机定位方法，由终端执行，所述方法包括：
获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；及
将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间的对应关系。
一种相机定位装置，所述装置包括：
地图获取模块，用于获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述 特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
位置预测模块，用于根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
过滤模块，用于根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；及
定位模块，用于将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间的对应关系。
一种终端，其特征在于，所述终端包括处理器和存储器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行以下步骤：
获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；及
将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间 的对应关系。
一种非易失性的计算机可读存储介质，存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；及
将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间的对应关系。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种特征点的示意图；
图2是本申请实施例提供的另一种特征点的示意图；
图3A是本申请实施例提供的一种构建环境地图的方法流程图；
图3B是本申请实施例提供的对于原始图像中的任一指定特征点，获取该指定特征点的位置信息和视角区域信息的过程步骤的流程图
图4是本申请实施例提供的一种构建环境地图的操作流程图；
图5是本申请实施例提供的目标环境的俯视图；
图6是本申请实施例提供的目标环境的侧视图；
图7是本申请实施例提供的目标环境的俯视图；
图8是本申请实施例提供的目标环境的侧视图；
图9是本申请实施例提供的一种相机定位方法的流程图；
图10是本申请实施例提供的一种相机定位装置的结构示意图；
图11是本申请实施例提供的一种终端的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
传统方案中提出了一种基于特征的地图构建和相机定位方法，在建图阶段，通过对图像之间的特征点进行匹配构建环境地图。在定位阶段，通过相机拍摄当前图像，将当前图像中的特征点与环境地图中的特征点进行匹配，实现相机定位。
由于在实际应用中，同一个物体或者同一个场景，从不同的视角观测会看到不同的画面，提取出不同的特征点。因此为了保证环境地图的可用性，针对同一个目标环境，往往需要构建包含多个视角的环境地图。例如构建一个走廊的环境地图时，需要相机从相反的两个方向各走一遍，即从两个不同的视角观测该走廊，从而保证构建的环境地图中可以包括从两个不同的视角 观测到的走廊特征点。若只从一个方向观测了走廊，在后续进行相机定位时，当相机从另一个方向经过时，所观测到的特征点不同，依靠先前构建的环境地图进行相机定位将会十分困难，最终导致定位失败。
如图1所示，相机从左侧观测三角形物体时可以看到A区域，如特征点1、特征点2和特征点3，而相机从右侧观测该三角形物体时，可以看到B区域和C区域，如特征点3、特征点4、特征点5和特征点6。如果只根据左侧观测到的三角形物体来构建环境地图，会导致环境地图中仅包括特征点1、特征点2和特征点3，而不会包括特征点4、特征点5和特征点6，当相机位于三角形物体右侧时将无法实现定位。
虽然通过相机在不同的位置对目标环境进行拍摄，可以构建出多视角的环境地图，以保证环境地图的全面可用性，但同时也带来了一个问题：在进行相机定位时，相机实际上仅是从一个视角来拍摄目标环境，所拍摄位置处的很多特征点相机在当前视角下是拍摄不到的，这些特征点对相机来说是无效的。此时如果将相机拍摄的目标图像中的特征点与环境地图中的所有特征点进行匹配，会产生大量无效的计算量，而且还会增加错误匹配的概率。
如图2所示，当相机移动到三角形物体的右侧时，只能观测到特征点3、特征点4和特征点5，而由于特征点1、特征点2和特征点6也在相机附近，故这6个特征点都会参与匹配计算，增加了计算量。并且由于相机实际拍摄的目标图像中不包括特征点1、特征点2、特征点6，将目标图像与三角形物体附近的环境地图进行匹配时，匹配结果很可能为目标图像与三角形物体不匹配，进而导致相机定位出现错误。
为了解决传统方案中存在的问题，本申请实施例提供了一种相机定位方法，能够在建图阶段确定环境地图中每个特征点的位置信息和每个特征点的视角区域信息，以视角区域信息来表示可观测到特征点的区域。在定位阶段，先预测相机的大概位置，根据每个特征点的视角区域信息，将相机当前不可观测到的特征点过滤掉，再将相机拍摄的目标图像中的特征点与环境地图中剩余的特征点进行匹配，从而确定相机的位置，既减少了计算量，而且用于 匹配的特征点符合相机的真实拍摄情况，降低了匹配错误的可能性，相当于提高了定位准确率。
应当理解的是，在本申请中提及的“相机”指具有图像采集功能的装置或设备，可以用来拍摄图像。该“相机”具体可以是专门用于采集图像的图像采集设备；也可以是终端中配置的图像采集装置，该图像采集装置具体可以是摄像头。
本申请实施例可以应用于对配置有相机的终端进行定位的多种场景下。例如，在配送物品的场景下，用户通过客户端及服务器向配送机器人下发物品配送任务，该物品配送任务中包括物品的装载地点及送达地点。配送机器人接收到物品配送任务后，可以先采用本申请实施例提供的方法进行定位，再根据当前位置、物品的装载地点及送达地点进行路径规划：首先规划出从当前位置到装载地点的路线，再规划出装载地点到送达地点的路线，之后按照规划的路径进行移动，在移动过程中配送机器人还会进行实时定位，控制移动速度及移动方向，以保证按照规划路线行进，避免偏离路线。
又如，在虚拟现实游戏的场景下，可以构建环境地图，该环境地图中不仅包含真实场景的地图，还包含在真实场景的某些位置处添加的虚拟元素，如虚拟宠物、虚拟优惠券等。终端移动过程中可以采用本申请实施例提供的方法进行定位，确定终端当前所在的位置以及在终端附近设置的虚拟元素，在终端的显示界面上显示当前拍摄的真实场景图像和附近设置的虚拟元素，构成虚拟现实相结合的显示界面。
当然，本申请实施例还可以应用于其他场景下，在此不再赘述。
图3A是本申请实施例提供的一种构建环境地图的方法流程图，图4是本申请实施例提供的一种构建环境地图的操作流程图。以该构建环境地图方法的执行主体为终端，对构建环境地图的过程进行举例说明。参见图3A和图4，该方法包括：
S301、终端获取相机拍摄目标环境得到的多张原始图像。
其中，该终端可以为手机、平板电脑、机器人、智能相机等，该终端配置有相机，可以通过相机对当前所处的环境进行拍摄。而该目标环境可以为公园、建筑物、存放物品的仓库等多种环境。当需要构建目标环境的环境地图时，终端可以在目标环境中进行移动，并通过相机拍摄目标环境得到多张原始图像。
该多张原始图像的数量可以根据对环境地图的精确度需求确定。且针对目标环境中的同一个位置，可以从多个不同的视角对该位置进行拍摄，得到多视角的原始图像。
S302、终端根据多张原始图像确定标准图像，并确定相机拍摄标准图像时的位置信息。
终端可以确定标准图像以及相机拍摄标准图像时的位置信息，后续过程中终端将以标准图像为基准，根据其他原始图像与标准图像之间的位置差异来确定相机拍摄其他原始图像时的位置信息。
在一种可能实现方式中，该终端可以将相机拍摄的第一张原始图像确定为标准图像，并初始化相机拍摄标准图像时的位置信息，例如可以将终端拍摄标准图像时的位置作为原点位置，创建环境坐标系，即标准图像的位置信息为(0，0，0)。之后拍摄的每张原始图像均可根据与标准图像之间的位置差异来确定位置信息。并且，考虑到随着相机不断进行移动，拍摄到的原始图像与标准图像的相同之处越来越少，则可以重新选取一张原始图像作为更新后的标准图像，确定相机拍摄该更新后的标准图像时的位置信息，之后拍摄的每张原始图像均可根据与更新后的标准图像的位置差异来确定位置信息。
S303、对于除标准图像以外的任一张原始图像，终端将原始图像中的特征点与标准图像中的特征点进行匹配，得到特征点对应关系，根据特征点对应关系及相机拍摄标准图像时的位置信息，获取相机拍摄原始图像时的位置信息。
终端获取到多张原始图像后，可以对每张原始图像进行特征提取，确定每张原始图像中的特征点在原始图像的位置信息，并确定每个特征点的描述子，利用描述子来对特征点进行描述。例如终端可以采用FAST(Features from Accelerated Segment Test，加速分割测试特征)、Harris(一种角点检测算法)、SURF(Speed Up Robust Feature，加速稳健特征)、BRISK(Binary Robust Invariant Scalable Keypoints，二进制鲁棒不变可扩展的关键点)、ORB(一种特征点检测算法)等算法提取特征点。
那么，将标准图像作为基准，对于除标准图像以外的任一张原始图像，均可将原始图像中的特征点与标准图像中的特征点进行匹配，根据原始图像中的每个特征点以及每个特征点在标准图像中匹配的特征点，建立特征点对应关系，即特征点对应关系包括原始图像中的特征点与在标准图像中匹配的特征点之间的对应关系，相当于生成了原始图像与标准图像之间两两匹配的特征点对。
其中，在进行特征点匹配时，可以根据距离度量准则，将原始图像中的每个特征点分别与标准图像中的每个特征点两两组合，计算每个组合中两个特征点的描述子之间的距离，则对于原始图像中的每个特征点，选取标准图像中与该特征点距离最近的特征点，作为该特征点的匹配特征点。或者，选取标准图像中与该特征点距离最近且距离小于预设距离的特征点，作为该特征点的匹配特征点，从而建立了原始图像中的特征点与标准图像中的匹配特征点之间的对应关系。
之后，根据该特征点对应关系以及每个特征点的位置信息，可以确定原始图像相对于标准图像的位置变化信息，则根据标准图像的位置信息及原始图像相对于标准图像的位置变化信息进行迭代计算，即可获取相机拍摄原始图像时的位置信息。
例如，将图像1确定为标准图像，位置信息为(x1，y1，z1)，之后可以根据与图像1的特征点对应关系确定图像2相对于图像1的位置变化信息为(Δx1，Δy1，Δz1)，图像3相对于图像1的位置变化信息为(Δx2，Δy2，Δz2)， 则确定图像2的位置信息(x2，y2，z2)为(x1+Δx1，y1+Δy1，z1+Δz1)、图像3的位置信息(x3，y3，z3)为(x1+Δx2，y1+Δy2，z1+Δz2)。此时可以将图像3更新为第二个标准图像，之后根据与图像3的特征点对应关系确定图像4相对于图像3的位置变化信息为(Δx3，Δy3，Δz3)，则图像4的位置信息为(x3+Δx3，y3+Δy3，z3+Δz3)。
在一种可能实现方式中，可以以旋转平移矩阵的形式来表示相机拍摄任一图像时的位置信息，终端可以获取标准图像的旋转平移矩阵，根据该特征点对应关系以及每个特征点的位置信息，确定原始图像相对于标准图像的旋转平移矩阵。其中，该旋转平移矩阵包括原始图像相对于标准图像的旋转参数和位移参数，则将标准图像的旋转平移矩阵以及原始图像相对于标准图像的旋转平移矩阵进行迭代计算，可以得到原始图像的旋转平移矩阵，该旋转平移矩阵中的位移参数即为相机拍摄原始图像时的位置信息。
在另一种可能实现方式中，终端获取到相机拍摄原始图像时的位置信息后，还可以去除原始图像中的噪声特征点，如原始图像中存在的动态物体的特征点或者无法匹配的特征点。
例如，相机在拍摄原始图像时，不仅会拍摄到目标环境，还可能会拍摄到目标环境中的动态物体。因此，为了准确构建目标环境的环境地图，终端可以对拍摄到的多张原始图像进行分析，获取任两个原始图像之间的位置变化信息，当某一特征点在该任两个原始图像中的实际位置变化信息与该任两个原始图像的位置变化信息不匹配时，表示在相机拍摄该任两个原始图像的过程中，特征点所在的物体在目标环境内发生了移动，因此可以确定该特征点为动态物体的特征点，而不是目标环境本身的特征点，此时将该特征点过滤掉。
或者，当某一特征点在一张原始图像中出现，而在其他原始图像中没有出现时，表示该特征点将无法进行匹配，仅根据一张原始图像上的特征点无法预测相机的位置变化，也无法实现相机定位，因此将该特征点过滤掉。
S304、终端根据相机拍摄原始图像时的位置信息，获取原始图像中每个 特征点的位置信息和每个特征点的视角区域信息。
参见图3B，对于原始图像中的任一指定特征点，获取该指定特征点的位置信息和视角区域信息的过程可以包括以下步骤S3041-S3042：
S3041、根据相机拍摄原始图像时的位置信息及指定特征点在原始图像中的深度，获取指定特征点的位置信息。
其中，指定特征点在原始图像中的深度用于表示拍摄原始图像时相机与指定特征点之间的距离，可以表示该指定特征点与相机之间的相对位置关系，确定相机拍摄原始图像时的位置信息后，根据相机的位置信息和指定特征点在原始图像中的深度，可以获取指定特征点的位置信息。
S3042、根据指定特征点的位置信息以及原始图像中其他点的位置信息，在指定特征点的预设范围内进行平面估计，确定经过指定特征点的至少一个平面。
考虑到在真实环境中，由一个或多个表面构成完整的物体，且各个表面所在的位置会影响到物体表面上每个特征点的可观测区域。因此，终端对该指定特征点进行可观测区域分析时，综合考虑指定特征点的位置信息以及原始图像中其他点的位置信息，在指定特征点的预设范围内进行平面估计，确定经过指定特征点的至少一个平面，此时可以认为该至少一个平面是特征点所在物体的至少一个表面，则根据至少一个平面的位置确定可观测到指定特征点的视角区域。
其中，该预设范围可以在原始图像中根据指定特征点所在的位置确定，例如该预设范围可以为以指定特征点为中心、以一定距离为半径的圆形范围，或者也可以为以指定特征点为顶点选取的多个扇形区域构成的范围等。
其中，在进行平面估计时，可以采用灰度图的边缘线分割算法或者深度图的边缘线分割算法等，预测出至少一个平面的平面参数。
在一种可能实现方式中，终端可以在指定特征点的预设范围内选取至少三个特征点，在原始图像上指定特征点位于至少三个特征点构成的多边形的内部，该至少三个特征点在环境地图中处于同一平面上但未处于同一直线上， 并计算指定特征点与该至少三个特征点所在的平面之间的距离，判断该距离是否小于预设距离。如果指定特征点与该平面之间的距离小于预设距离，则确定指定特征点位于该至少三个特征点所在的平面上，也即是仅有一个平面经过指定特征点。
其中，在预设范围内选取特征点时，可以确定以指定特征点为顶点的至少三个扇形区域，从每个扇形区域中选取一个距离指定特征点最近的特征点，从而选取出至少三个特征点。
图5是本申请实施例提供的目标环境的俯视图，图6是本申请实施例提供的目标环境的侧视图，参见图5和图6，特征点1为要进行视角分析的特征点，在特征点1的附近选取特征点2、特征点3和特征点4三个特征点，这三个特征点没有位于同一个直线上，根据这三个特征点可以预测出一个平面上，特征点1与该平面之间的距离较小，因此可以确定特征点1也位于该平面上。
如果指定特征点与该平面之间的距离不小于预设距离，确定指定特征点没有位于该至少三个特征点所在的平面上，则在预设范围内进行边缘检测，得到以指定特征点为顶点的至少一条边缘线，并获取预设范围内的至少四个特征点，将指定特征点作为至少两个平面的相交点，将至少一条边缘线作为任两个平面的相交线，并将至少四个特征点作为至少两个平面上的点，得到该至少两个平面。
其中，该至少四个特征点可以包括之前选取的至少三个特征点，还可以包括另外选取的至少一个特征点，只需保证该至少四个特征点没有位于同一直线上，且该至少四个特征点均没有位于该至少一条边缘线上，这样根据该指定特征点、该至少四个特征点和该至少一条边缘线可以确定至少两个平面。
图7是本申请实施例提供的目标环境的俯视图，图8是本申请实施例提供的目标环境的侧视图，参见图7和图8，特征点2、特征点3和特征点4位于同一个平面上，特征点1与该平面之间的距离较大，因此可以确定特征点1没有位于该平面上，此时在特征点1附近又选取了特征点5，并进行边缘检 测，得到经过特征点1的一条边缘线，则根据5个特征点的位置以及检测得到的边缘线，可以确定两个平面，其中特征点1、特征点2和特征点3位于一个平面上，特征点1、特征点4和特征点5位于另一个平面上。
在另一种可能实现方式中，如果终端配置的相机为深度相机，该深度相机在拍摄图像时即可获取到图像中每个点的深度，则针对指定特征点来说，获取指定特征点的预设范围内的多个点在原始图像中的深度，根据获取到的每个点在原始图像中的深度，可以确定多个特征点在目标环境中的准确位置，根据该多个特征点的准确位置可以构建至少一个平面，使指定特征点位于至少一个平面中的每个平面上，指定特征点的预设范围内的点位于至少一个平面中的任一个或任两个平面上。其中，预设范围内的多个点可以包括从原始图像中提取的特征点，也可以包括除特征点以外的其他点。
S3043、确定通过至少一个平面将目标环境进行划分所得到的第一连通区域和第二连通区域，将第一连通区域和第二连通区域中相机所在的区域确定为可观测到指定特征点的视角区域，获取视角区域信息。
该至少一个平面中任两个或多个平面相交，可以将目标环境划分成两个连通区域，称为第一连通区域和第二连通区域，例如一个平面可以将目标环境划分成上侧和下侧的两个连通区域，两个平面可以将目标环境划分成二面角内侧和二面角外侧的两个连通区域等，三个平面也可以将目标环境划分成两个连通区域。
确定第一连通区域和第二连通区域后，可以根据相机拍摄原始图像时的位置信息，判断相机位于第一连通区域还是位于第二连通区域，从而将相机所在的区域确定为可观测指定特征点的视角区域，将相机所在的区域信息作为指定特征点的视角区域信息。
例如，参见图6，特征点1位于平面上，可观测角度为180度，相机位于平面上侧区域时，可以观测到特征点1，而位于平面下侧区域时，不能观测到特征点1。又如，参见图8，特征点1位于两个平面的相交线上，虚线表示两个平面的法向量的中心向量，特征点1的可观测区域为以中心向量为中 心轴且角度为w的区域，相机位于该区域时可以观测到特征点1，而没有位于该区域时不能观测到特征点1。
S305、终端根据多张原始图像中每个特征点的位置信息和每个特征点的视角区域信息，构建环境地图，则环境地图中包括多个特征点的位置信息和该多个特征点的视角区域信息。
终端后续可以将相机当前拍摄的目标图像中的特征点与环境地图中的特征点进行匹配，根据匹配结果确定相机的位置，实现相机定位。
在一种可能实现方式中，当终端对每张原始图像分析完成，得到特征点的位置信息和视角区域信息时，还可以对不同图像中匹配特征点的位置信息和视角区域信息进行优化，得到全局一致的环境地图。例如当某一图像中特征点的位置信息与其他多张图像中匹配特征点的位置信息存在偏差时，可以对该图像中特征点的位置信息进行修正，以保证不同图像之间的位置变化信息与匹配特征点在不同图像之间的位置变化信息一致。
本申请实施例提供的方法，通过根据相机拍摄目标环境得到的多张原始图像，获取每张原始图像的位置信息，并根据每张原始图像的位置信息获取每个特征点的位置信息和每个特征点的视角区域信息，不仅构建出多视角的环境地图，而且能够以视角区域信息来表示可观测到特征点的区域，后续根据相机的预测位置信息和特征点的视角区域信息可以将相机不可观测到的特征点过滤掉，减少了计算量，提高了定位准确率。
图9是本申请实施例提供的一种相机定位方法的流程图，以该相机定位方法的执行主体为终端，对相机在目标环境中移动时进行定位的过程进行举例说明。参见图9，该方法包括：
S901、终端获取目标环境的环境地图。
其中，该环境地图包括目标环境中多个特征点的位置信息和该多个特征点的视角区域信息，特征点的视角区域信息用于表示目标环境中可观测到该特征点的视角区域。
构建环境地图的具体过程参见上述图3A所示的实施例，本申请实施例在此不再赘述。
S902、终端根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息。
本申请实施例中，该终端配置有相机，终端在目标环境中进行移动的过程中，可以通过相机对当前所处的目标环境进行拍摄，得到一张或多张图像。
其中，历史图像是在拍摄时间在该目标图像的拍摄时间之前的图像。在本申请实施例中，相机拍摄历史图像的位置信息，具体可以是相机拍摄某一张历史图像的位置信息(比如，相机在拍摄目标图像之前拍摄的上一张图像)，也可以是根据相机拍摄的多张历史图像的位置信息所确定的综合的位置信息。可以理解，在本申请实施例提供的方案中，由于是需要根据相机拍摄历史图像的位置信息来对相机拍摄目标图像的位置进行预测，因而，根据相机拍摄上一张图像的位置信息或前几张历史图像的位置信息来进行预测，预测的准确率是较高的。
在一个实施例中，终端当前要进行定位时，将相机当前拍摄的图像称为目标图像，终端先根据相机在目标图像之前拍摄的上一张图像的位置信息，对相机(终端)当前的位置进行预测，获取相机的预测位置信息，此时终端暂不将目标图像的特征点与环境地图中的特征点进行匹配。在一个实施例中，该步骤S902可以包括步骤A、步骤B和步骤C中的任一项：
步骤A、根据相机拍摄目标图像之前的上一张图像时的位置信息和相机拍摄任两张时间上相邻图像的时间间隔，采用恒速模型，获取相机的预测位置信息。
具体地，相机可以设置好固定的时间间隔，每隔该时间间隔拍摄一张图像，从而拍摄得到多张图像。其中，任两张时间上相邻图像是指相机拍摄到的多张图像中在时间上相邻的任两张图像，这两张图像的时间间隔等于相机设置的拍摄时间间隔。假设相机匀速移动的情况下，可以获取恒速模型，该恒速模型设定相机按照固定速度移动，该固定速度可以由终端默认设置，或 者根据一般人们的走路速度设置。那么，终端获取到相机拍摄上一张图像时的位置信息后，根据该时间间隔，采用恒速模型，可以获取到相机在该时间间隔内的位移量，并根据拍摄上一张图像时的位置信息和该位移量，获取相机的预测位置信息。
步骤B、根据相机拍摄目标图像之前的上一张图像时的位置信息、相机拍摄任两张时间上相邻图像的时间间隔及通过传感器检测到的加速度，采用变速模型，获取相机的预测位置信息。
相机可以设置好固定的时间间隔，每隔该时间间隔拍摄一张图像，从而拍摄得到多张图像。其中，任两张时间上相邻图像是指相机拍摄到的多张图像中在时间上相邻的任两张图像，这两张图像的时间间隔等于相机设置的拍摄时间间隔。则假设相机变速移动的情况下，可以获取变速模型，该变速模型设定相机按照一定的加速度移动，能够按照输入的初始速度、加速度和时间间隔计算出相机在该时间间隔内的位移量。那么，随着相机的移动，终端获取到相机拍摄上一张图像时的位置信息和初始速度后，通过配置的传感器检测加速度，根据相机拍摄上一张图像时的初始速度和加速度，采用变速模型获取相机在该时间间隔内的位移量，并根据相机拍摄上一张图像时的位置信息和该位移量，获取相机的预测位置信息。
步骤C、在相机拍摄上一张图像至拍摄目标图像的过程中，终端通过追踪上一张图像中的特征点，获取目标图像相对于上一张图像的位置变化信息，根据相机拍摄上一张图像时的位置信息和目标图像相对于上一张图像的位置变化信息，获取相机的预测位置信息。
在相机拍摄了上一张图像时可以获取相机拍摄上一张图像时的位置信息，且在相机拍摄上一张图像至拍摄目标图像的过程中，终端可以追踪上一张图像中的特征点，根据特征点的运动信息，获取目标图像相对于上一张图像的位置变化信息，则根据相机拍摄上一张图像时的位置信息和目标图像相对于上一张图像的位置变化信息进行迭代计算，获取相机的预测位置信息。
其中，在追踪过程中，终端可以采用光流匹配算法获取特征点的运动信 息，根据该运动信息获取目标图像相对于上一张图像的位置信息。例如，终端可以采用光流匹配算法获取多个特征点的运动信息，根据运动信息确定该多个特征点在上一张图像中的位置信息以及在目标图像中的位置信息，从而确定目标图像相对于上一张图像的旋转平移矩阵，该旋转平移矩阵中的位移参数即为目标图像相对于上一张图像的位置变化信息。或者终端也可以采用其他追踪算法来追踪特征点。
另外，当启动相机时，终端还未获取到相机之前拍摄上一张图像时的位置信息，或者由于终端的位置变化过大而导致目标图像与之前拍摄的上一张图像之间的差异过大，两者不存在匹配的特征点。这两种情况下，根据相机在目标图像之前拍摄的上一张图像的位置信息将无法获取到目标图像的预测位置信息，此时可以进行重定位，将目标图像的特征点与环境地图中的多个特征点分别进行匹配，选取与目标图像的特征点最为匹配的特征点，根据选取的匹配特征点所在的位置，初步确定目标图像的预测位置信息。
进行特征点匹配时，可以根据距离度量准则，将目标图像中的每个特征点分别与环境地图中的每个特征点两两组合，计算每个组合中两个特征点的描述子之间的距离，距离越小，表示两个特征点的描述子相差越小，越有可能是相同的特征点。则对于目标图像中的每个特征点，选取环境地图中与该特征点距离最近的特征点，作为该特征点的匹配特征点。或者，选取环境地图中与该特征点距离最近且距离小于预设距离的特征点，作为该特征点的匹配特征点，则根据目标图像中的多个特征点对应的匹配特征点的位置信息可以获取相机当前的预测位置信息。
在一种可能实现方式中，终端可以通过外部传感器检测的地理信息，确定相机所在的地理区域，将特征匹配限制于一个相对小的地理区域内，再相机所在的地理区域内通过进行特征点匹配获取相机当前的预测位置信息。其中，该外部传感器可以为终端连接的WIFI(Wireless Fidelity，无线保真)设备、UWB(Ultra Wideband，超宽带)设备等，外部传感器根据自身所在的位置配置地理位置信息，随着终端的移动，终端位于外部传感器附近时，可以 与外部传感器连接，并接收外部传感器发送的地理位置信息，根据地理位置信息确定当前所在的地理区域。
在一个实施例中，当终端根据拍摄其他的历史图像的位置信息(比如，相机在拍摄目标图像之前拍摄的倒数第二张图像)，来对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息的具体实现方式，可以参考前述终端根据拍摄上一张图像的位置信息，来对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息的步骤的具体描述，在此不再赘述。S903、终端根据相机的预测位置信息和环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将相机当前不可观测到的特征点过滤掉。
特征点的视角区域信息用于表示可观测该特征点的视角区域，当相机处于某一特征点的视角区域内时，可以观测到该特征点，所拍摄的目标图像中可能会包含该特征点，而当相机未处于某一特征点的视角区域时，即使该特征点与相机的位置相距不远，相机也无法观测到该特征点，所拍摄的目标图像中也不会包含该特征点。
因此，为了减小计算量，对于环境地图中的每个特征点，终端可以根据相机的预测位置信息和该特征点的视角区域信息，判断相机是否处于特征点的视角区域内，当确定相机处于特征点的视角区域内时，保留该特征点，当确定相机未处于特征点的视角区域内时，将该特征点过滤掉。
采用上述方式，可以将环境地图中相机当前不可观测到的特征点过滤掉，而仅保留相机当前可观测到的特征点。
S904、终端将目标图像中的特征点与环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据特征点对应关系获取相机的位置信息，特征点对应关系包括目标图像中的特征点与匹配的特征点之间的对应关系。
终端可以获取目标图像中每个特征点的描述子，以及环境地图过滤后剩余的每个特征点的描述子，将目标图像中的每个特征点分别与环境地图过滤后剩余的特征点两两组合，计算每个组合中两个特征点的描述子之间的距离，距离越小，表示两个特征点的描述子相差越小，越有可能是相同的特征点。 则对于目标图像中的每个特征点，选取环境地图中与该特征点距离最近的特征点，作为匹配特征点。或者，选取环境地图中与该特征点距离最近且距离小于预设距离的特征点，作为匹配特征点，从而建立特征点对应关系。
之后，根据该特征点对应关系以及环境地图剩余的每个特征点的位置信息，可以获取相机的位置信息，该位置信息用于表示相机当前在目标环境中的位置信息。其中，可以采用PnP(n个特征点的透视)算法或者其他算法获取相机的位置信息。
获取到相机的位置信息之后，可以确定相机在环境地图中的位置，从而基于该位置进行相应的操作。例如，在配送物品的场景下，可以根据配送机器人当前的位置判断是否偏离规划路线，如果配送机器人偏离规划路线，则根据配送机器人当前的位置和送达地点重新进行路线规划。或者，在虚拟现实游戏的场景下，确定终端的位置后，可以在终端的显示界面中显示在该位置处设置的虚拟元素，供用户与虚拟元素进行交互。
本申请实施例提供的方法，通过获取目标环境的环境地图，该环境地图中包括多个特征点的位置信息和该多个特征点的视角区域信息，视角区域信息用于表示可观测到特征点的视角区域，先根据相机在拍摄目标图像之前拍摄上一张图像时的位置信息，对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息，根据相机的预测位置信息、环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将相机当前不可观测到的特征点过滤掉，将目标图像中的特征点与环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，从而确定相机的位置信息，实现相机定位，既减少了计算量，提高了计算速度，而且用于匹配的特征点符合相机的真实拍摄情况，降低了匹配错误的可能性，相当于提高了定位准确率和地图跟踪的鲁棒性。
本申请实施例提供了基于视觉特征视角区域的SLAM方法，大幅提高了定位与建图的鲁棒性，为相机定位提供了可靠信息，可以广泛应用于机器人导航、游戏角色定位、无人机定位、水下机器人定位等多种场景下，增强了整个系统的稳定性，减小了计算量，降低了对硬件配置的要求，节省了成本。
图10是本申请实施例提供的一种相机定位装置的结构示意图。参见图10，该装置包括：
地图获取模块1001，用于获取目标环境的环境地图,环境地图包括目标环境中多个特征点的位置信息和多个特征点的视角区域信息，特征点的视角区域信息用于表示目标环境中可观测到特征点的视角区域。
位置预测模块1002，用于根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对相机拍摄目标图像时的位置进行预测，得到相机的预测位置信息。过滤模块1003，用于根据相机的预测位置信息、环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将环境地图中相机当前不可观测到的特征点过滤掉。定位模块1004，用于将目标图像中的特征点与环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据特征点对应关系确定相机的位置信息，特征点对应关系包括目标图像中的特征点与匹配的特征点之间的对应关系。
在一个实施例中，位置预测模块1002，包括：
第一预测单元，用于根据相机拍摄上一张图像时的位置信息和相机拍摄任两张时间上相邻图像的时间间隔，采用恒速模型，获取相机的预测位置信息；或者，
第二预测单元，用于根据相机拍摄上一张图像时的位置信息、相机拍摄任两张时间上相邻图像的时间间隔及通过传感器检测到的加速度，采用变速模型，获取相机的预测位置信息；或者，
第三预测单元，用于在相机拍摄上一张图像至拍摄目标图像的过程中，通过追踪上一张图像中的特征点，获取目标图像相对于上一张图像的位置变化信息，根据相机拍摄上一张图像时的位置信息和目标图像相对于上一张图像的位置变化信息，获取相机的预测位置信息。
在一个实施例中，该相机定位装置还包括：
重定位模块，用于当启动相机时，或者当根据相机拍摄上一张图像时的 位置信息无法获取到相机的预测位置信息时，将目标图像的特征点与环境地图中的多个特征点进行匹配，获取相机的预测位置信息。
在一个实施例中，过滤模块1003还用于对于环境地图中的每个特征点，当根据相机的预测位置信息确定相机未处于特征点的视角区域内时，将特征点过滤掉。
在一个实施例中，地图获取模块，包括：
标准确定单元，用于获取相机拍摄目标环境得到的多张原始图像，根据多张原始图像确定标准图像，并确定相机拍摄标准图像时的位置信息；
位置获取单元，用于对于除标准图像以外的任一张原始图像，将原始图像中的特征点与标准图像中的特征点进行匹配，得到特征点对应关系，特征点对应关系包括原始图像中的特征点与在标准图像中匹配的特征点之间的对应关系；根据特征点对应关系及相机拍摄标准图像时的位置信息，获取相机拍摄原始图像时的位置信息；
信息获取单元，用于根据相机拍摄原始图像时的位置信息，获取原始图像中每个特征点的位置信息和每个特征点的视角区域信息；
地图构建单元，用于根据多张原始图像中每个特征点的位置信息和每个特征点的视角区域信息，构建环境地图。
在一个实施例中，信息获取单元，还用于对于原始图像中的任一指定特征点，根据相机拍摄原始图像时的位置信息及指定特征点在原始图像中的深度，获取指定特征点的位置信息；根据指定特征点的位置信息以及原始图像中其他点的位置信息，在指定特征点的预设范围内进行平面估计，确定经过指定特征点的至少一个平面；及确定通过至少一个平面将目标环境进行划分所得到的第一连通区域和第二连通区域，将第一连通区域和第二连通区域中相机所在的区域确定为可观测到指定特征点的视角区域，获取视角区域信息。
在一个实施例中，信息获取单元，还用于
在指定特征点的预设范围内选取至少三个特征点，指定特征点位于至少三个特征点构成的多边形的内部，至少三个特征点在环境地图中处于同一平 面上但未处于同一直线上；及如果指定特征点与至少三个特征点所在的平面之间的距离小于预设距离，则确定指定特征点位于至少三个特征点所在的平面上。
。
在一个实施例中，信息获取单元，还用于如果指定特征点与至少三个特征点所在的平面之间的距离不小于预设距离，则在预设范围内进行边缘检测，得到以指定特征点为顶点的至少一条边缘线，并获取预设范围内的至少四个特征点；及将指定特征点作为至少两个平面的相交点，将至少一条边缘线作为任两个平面的相交线，并将至少四个特征点作为至少两个平面上的点，得到至少两个平面。
可选地，相机为深度相机，信息获取单元，还用于获取指定特征点的预设范围内的多个点在原始图像中的深度，根据每个点在原始图像中的深度，确定至少一个平面，指定特征点位于至少一个平面中的每个平面上，指定特征点的预设范围内的点位于至少一个平面中的任一个或任两个平面上。
需要说明的是：上述实施例提供的相机定位装置在进行相机定位时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将终端的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的相机定位装置与相机定位方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图11示出了本申请一个示例性实施例提供的终端1100的结构框图。该终端1100可以是便携式移动终端，比如：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑、台式电脑、头戴式设备，或其 他任意智能终端。终端1100还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1100包括有：处理器1101和存储器1102。
处理器1101可以包括一个或多个处理核心，比如4核心处理器、5核心处理器等。处理器1101可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1101也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器1101可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器1101还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1102可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1102还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1102中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1101所具有以实现本申请中方法实施例提供的相机定位方法。
在一些实施例中，终端1100还可选包括有：外围设备接口1103和至少一个外围设备。处理器1101、存储器1102和外围设备接口1103之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口1103相连。具体地，外围设备包括：射频电路1104、触摸显示屏1105、摄像头1106、音频电路1107、定位组件1108和电源1109中的至少一种。
外围设备接口1103可被用于将I/O(Input/Output，输入/输出)相关的 至少一个外围设备连接到处理器1101和存储器1102。在一些实施例中，处理器1101、存储器1102和外围设备接口1103被集成在同一芯片或电路板上；在一些其他实施例中，处理器1101、存储器1102和外围设备接口1103中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路1104用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路1104通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1104将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路1104包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路1104可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及11G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1104还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1105用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏1105是触摸显示屏时，显示屏1105还具有拍摄在显示屏1105的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1101进行处理。此时，显示屏1105还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏1105可以为一个，设置终端1100的前面板；在另一些实施例中，显示屏1105可以为至少两个，分别设置在终端1100的不同表面或呈折叠设计；在再一些实施例中，显示屏1105可以是柔性显示屏，设置在终端1100的弯曲表面上或折叠面上。甚至，显示屏1105还可以设置成非矩形的不规则图形，也即异形屏。显示屏1105可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件1106用于拍摄图像或视频。可选地，摄像头组件1106包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件1106还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路1107可以包括麦克风和扬声器。麦克风用于拍摄用户及环境的声波，并将声波转换为电信号输入至处理器1101进行处理，或者输入至射频电路1104以实现语音通信。出于立体声拍摄或降噪的目的，麦克风可以为多个，分别设置在终端1100的不同部位。麦克风还可以是阵列麦克风或全向拍摄型麦克风。扬声器则用于将来自处理器1101或射频电路1104的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路1107还可以包括耳机插孔。
定位组件1108用于定位终端1100的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件1108可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源1109用于为终端1100中的各个组件进行供电。电源1109可以是交流电、直流电、一次性电池或可充电电池。当电源1109包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1100还包括有一个或多个传感器1110。该一个或 多个传感器1110包括但不限于：加速度传感器1111、陀螺仪传感器1112、压力传感器1113、指纹传感器1114、光学传感器1115以及接近传感器1116。
加速度传感器1111可以检测以终端1100建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器1111可以用于检测重力加速度在三个坐标轴上的分量。处理器1101可以根据加速度传感器1111拍摄的重力加速度信号，控制触摸显示屏1105以横向视图或纵向视图进行用户界面的显示。加速度传感器1111还可以用于游戏或者用户的运动数据的拍摄。
陀螺仪传感器1112可以检测终端1100的机体方向及转动角度，陀螺仪传感器1112可以与加速度传感器1111协同拍摄用户对终端1100的3D动作。处理器1101根据陀螺仪传感器1112拍摄的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器1113可以设置在终端1100的侧边框和/或触摸显示屏1105的下层。当压力传感器1113设置在终端1100的侧边框时，可以检测用户对终端1100的握持信号，由处理器1101根据压力传感器1113拍摄的握持信号进行左右手识别或快捷操作。当压力传感器1113设置在触摸显示屏1105的下层时，由处理器1101根据用户对触摸显示屏1105的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器1114用于拍摄用户的指纹，由处理器1101根据指纹传感器1114拍摄到的指纹识别用户的身份，或者，由指纹传感器1114根据拍摄到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器1101授权该用户具有相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器1114可以被设置终端1100的正面、背面或侧面。当终端1100上设置有物理按键或厂商Logo时，指纹传感器1114可以与物理按键或厂商标志集成在一起。
光学传感器1115用于拍摄环境光强度。在一个实施例中，处理器1101 可以根据光学传感器1115拍摄的环境光强度，控制触摸显示屏1105的显示亮度。具体地，当环境光强度较高时，调高触摸显示屏1105的显示亮度；当环境光强度较低时，调低触摸显示屏1105的显示亮度。在另一个实施例中，处理器1101还可以根据光学传感器1115拍摄的环境光强度，动态调整摄像头组件1106的拍摄参数。
接近传感器1116，也称距离传感器，通常设置在终端1100的前面板。接近传感器1116用于拍摄用户与终端1100的正面之间的距离。在一个实施例中，当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变小时，由处理器1101控制触摸显示屏1105从亮屏状态切换为息屏状态；当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变大时，由处理器1101控制触摸显示屏1105从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图11中示出的结构并不构成对终端1100的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
本申请实施例还提供了一种用于相机定位的终端，该终端包括处理器和存储器，存储器中存储有计算机可读指令，计算机可读指令被所述处理器执行时，使得处理器执行上述相机定位方法的步骤。此处相机定位方法的步骤可以是上述各个实施例的相机定位方法中的步骤。
本申请实施例还提供了一种计算机可读存储介质，存储有计算机可读指令，计算机可读指令被处理器执行时，使得处理器执行上述相机定位方法的步骤。此处相机定位方法的步骤可以是上述各个实施例的相机定位方法中的步骤。
其中，该计算机可读指令可以包括程序、代码集等。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存 储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种相机定位方法，由终端执行，其特征在于，所述方法包括：
    获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
    根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
    根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；及
    将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间的对应关系。
  2. 根据权利要求1所述的方法，其特征在于，所述根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息，包括：
    根据所述相机拍摄所述上一张图像时的位置信息和所述相机拍摄任两张时间上相邻图像的时间间隔，采用恒速模型，获取所述相机的预测位置信息；或者，
    根据所述相机拍摄所述上一张图像时的位置信息、所述相机拍摄任两张时间上相邻图像的时间间隔及通过传感器检测到的加速度，采用变速模型，获取所述相机的预测位置信息；或者，
    在所述相机拍摄所述上一张图像至拍摄所述目标图像的过程中，通过追踪所述上一张图像中的特征点，获取所述目标图像相对于所述上一张图像的位置变化信息，根据所述相机拍摄所述上一张图像时的位置信息和所述目标图像相对于所述上一张图像的位置变化信息，获取所述相机的预测位置信息。
  3. 根据权利要求1或2所述的方法，其特征在于，所述方法还包括：
    当启动所述相机时，或者当根据所述相机拍摄所述历史图像时的位置信息无法获取到所述相机的预测位置信息时，将所述目标图像的特征点与所述环境地图中的多个特征点进行匹配，获取所述相机的预测位置信息。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉，包括：
    对于所述环境地图中的每个特征点，当根据所述相机的预测位置信息确定所述相机未处于所述特征点的视角区域内时，将所述特征点过滤掉。
  5. 根据权利要求1所述的方法，其特征在于，所述获取目标环境的环境地图，包括：
    获取相机拍摄所述目标环境得到的多张原始图像，根据所述多张原始图像确定标准图像，并确定所述相机拍摄所述标准图像时的位置信息；
    对于除所述标准图像以外的任一张原始图像，将所述原始图像中的特征点与所述标准图像中的特征点进行匹配，得到特征点对应关系，所述特征点对应关系包括所述原始图像中的特征点与在所述标准图像中匹配的特征点之间的对应关系；根据所述特征点对应关系及所述相机拍摄所述标准图像时的位置信息，获取所述相机拍摄所述原始图像时的位置信息；
    根据所述相机拍摄所述原始图像时的位置信息，获取所述原始图像中每个特征点的位置信息和每个特征点的视角区域信息；及
    根据所述多张原始图像中每个特征点的位置信息和每个特征点的视角区域信息，构建所述环境地图。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述相机拍摄所述原始图像时的位置信息，获取所述原始图像中每个特征点的位置信息和每个特征点的视角区域信息，包括：
    对于所述原始图像中的任一指定特征点，根据所述相机拍摄所述原始图像时的位置信息及所述指定特征点在所述原始图像中的深度，获取所述指定特征 点的位置信息；
    根据所述指定特征点的位置信息以及所述原始图像中其他点的位置信息，在所述指定特征点的预设范围内进行平面估计，确定经过所述指定特征点的至少一个平面；及
    确定通过所述至少一个平面将所述目标环境进行划分所得到的第一连通区域和第二连通区域，将所述第一连通区域和所述第二连通区域中所述相机所在的区域确定为可观测到所述指定特征点的视角区域，获取所述视角区域信息。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述指定特征点的位置信息以及所述原始图像中其他点的位置信息，在所述指定特征点的预设范围内进行平面估计，确定经过所述指定特征点的至少一个平面，包括：
    在所述指定特征点的预设范围内选取至少三个特征点，所述指定特征点位于所述至少三个特征点构成的多边形的内部，所述至少三个特征点在所述环境地图中处于同一平面上但未处于同一直线上；及
    如果所述指定特征点与所述至少三个特征点所在的平面之间的距离小于预设距离，则确定所述指定特征点位于所述至少三个特征点所在的平面上。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述指定特征点的位置信息以及所述原始图像中其他点的位置信息，在所述指定特征点的预设范围内进行平面估计，确定经过所述指定特征点的至少一个平面，还包括：
    如果所述指定特征点与所述至少三个特征点所在的平面之间的距离不小于所述预设距离，则在所述预设范围内进行边缘检测，得到以所述指定特征点为顶点的至少一条边缘线，并获取所述预设范围内的至少四个特征点；及
    将所述指定特征点作为至少两个平面的相交点，将所述至少一条边缘线作为任两个平面的相交线，并将所述至少四个特征点作为所述至少两个平面上的点，得到所述至少两个平面。
  9. 根据权利要求6所述的方法，其特征在于，所述相机为深度相机，根据所述指定特征点的位置信息以及所述原始图像中其他点的位置信息，在所述指定特征点的预设范围内进行平面估计，确定经过所述指定特征点的至少一个平 面，包括：
    获取所述指定特征点的预设范围内的多个点在所述原始图像中的深度，根据每个点在所述原始图像中的深度，确定至少一个平面，所述指定特征点位于所述至少一个平面中的每个平面上，所述指定特征点的预设范围内的点位于所述至少一个平面中的任一个或任两个平面上。
  10. 一种相机定位装置，其特征在于，所述装置包括：
    地图获取模块，用于获取目标环境的环境地图，所述环境地图包括所述目标环境中多个特征点的位置信息和所述多个特征点的视角区域信息，所述特征点的视角区域信息用于表示所述目标环境中可观测到所述特征点的视角区域；
    位置预测模块，用于根据相机在拍摄目标图像之前拍摄历史图像时的位置信息，对所述相机拍摄所述目标图像时的位置进行预测，得到所述相机的预测位置信息；
    过滤模块，用于根据所述相机的预测位置信息、所述环境地图中每个特征点的位置信息和每个特征点的视角区域信息，将所述环境地图中所述相机当前不可观测到的特征点过滤掉；
    定位模块，用于将所述目标图像中的特征点与所述环境地图过滤后剩余的特征点进行匹配，得到特征点对应关系，根据所述特征点对应关系确定所述相机的位置信息，所述特征点对应关系包括所述目标图像中的特征点与匹配的特征点之间的对应关系。
  11. 根据权利要求10所述的方法，其特征在于，所述位置预测模块包括
    第一预测单元，用于根据所述相机拍摄所述上一张图像时的位置信息和所述相机拍摄任两张时间上相邻图像的时间间隔，采用恒速模型，获取所述相机的预测位置信息；或者，
    第二预测单元，用于根据所述相机拍摄所述上一张图像时的位置信息、所述相机拍摄任两张时间上相邻图像的时间间隔及通过传感器检测到的加速度，采用变速模型，获取所述相机的预测位置信息；或者，
    第三预测单元，用于在所述相机拍摄所述上一张图像至拍摄所述目标图像的过程中，通过追踪所述上一张图像中的特征点，获取所述目标图像相对于所述上一张图像的位置变化信息，根据所述相机拍摄所述上一张图像时的位置信息和所述目标图像相对于所述上一张图像的位置变化信息，获取所述相机的预测位置信息。
  12. 根据权利要求10或11所述的装置，其特征在于，所述装置还包括：
    重定位模块，用于当启动所述相机时，或者当根据所述相机拍摄所述历史图像时的位置信息无法获取到所述相机的预测位置信息时，将所述目标图像的特征点与所述环境地图中的多个特征点进行匹配，获取所述相机的预测位置信息。
  13. 根据权利要求10所述的装置，其特征在于，所述过滤模块还用于对于所述环境地图中的每个特征点，当根据所述相机的预测位置信息确定所述相机未处于所述特征点的视角区域内时，将所述特征点过滤掉。
  14. 根据权利要求10所述的装置，其特征在于，所述地图获取模块，包括：
    标准确定单元，用于获取相机拍摄所述目标环境得到的多张原始图像，根据所述多张原始图像确定标准图像，并确定所述相机拍摄所述标准图像时的位置信息；
    位置获取单元，用于对于除所述标准图像以外的任一张原始图像，将所述原始图像中的特征点与所述标准图像中的特征点进行匹配，得到特征点对应关系，所述特征点对应关系包括所述原始图像中的特征点与在所述标准图像中匹配的特征点之间的对应关系；根据所述特征点对应关系及所述相机拍摄所述标准图像时的位置信息，获取所述相机拍摄所述原始图像时的位置信息；
    信息获取单元，用于根据所述相机拍摄所述原始图像时的位置信息，获取所述原始图像中每个特征点的位置信息和每个特征点的视角区域信息；及
    地图构建单元，用于根据所述多张原始图像中每个特征点的位置信息和每个特征点的视角区域信息，构建所述环境地图。
  15. 根据权利要求14所述的装置，其特征在于，所述信息获取单元，还用 于：
    对于所述原始图像中的任一指定特征点，根据所述相机拍摄所述原始图像时的位置信息及所述指定特征点在所述原始图像中的深度，获取所述指定特征点的位置信息；
    根据所述指定特征点的位置信息以及所述原始图像中其他点的位置信息，在所述指定特征点的预设范围内进行平面估计，确定经过所述指定特征点的至少一个平面；及
    确定通过所述至少一个平面将所述目标环境进行划分所得到的第一连通区域和第二连通区域，将所述第一连通区域和所述第二连通区域中所述相机所在的区域确定为可观测到所述指定特征点的视角区域，获取所述视角区域信息。
  16. 根据权利要求15所述的方法，其特征在于，所述信息获取单元，还用于：
    在所述指定特征点的预设范围内选取至少三个特征点，所述指定特征点位于所述至少三个特征点构成的多边形的内部，所述至少三个特征点在所述环境地图中处于同一平面上但未处于同一直线上；及
    如果所述指定特征点与所述至少三个特征点所在的平面之间的距离小于预设距离，则确定所述指定特征点位于所述至少三个特征点所在的平面上。
  17. 根据权利要求16所述的装置，其特征在于，所述信息获取单元还用于如果所述指定特征点与所述至少三个特征点所在的平面之间的距离不小于所述预设距离，则在所述预设范围内进行边缘检测，得到以所述指定特征点为顶点的至少一条边缘线，并获取所述预设范围内的至少四个特征点；及将所述指定特征点作为至少两个平面的相交点，将所述至少一条边缘线作为任两个平面的相交线，并将所述至少四个特征点作为所述至少两个平面上的点，得到所述至少两个平面。
  18. 根据权利要求15所述的装置，其特征在于，所述相机为深度相机，所述信息获取单元，还用于：获取所述指定特征点的预设范围内的多个点在所述原始图像中的深度，根据每个点在所述原始图像中的深度，确定至少一个平面， 所述指定特征点位于所述至少一个平面中的每个平面上，所述指定特征点的预设范围内的点位于所述至少一个平面中的任一个或任两个平面上。
  19. 一种终端，其特征在于，所述终端包括处理器和存储器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如权利要求1至10中任一项所述的方法的步骤。
  20. 一种非易失性的计算机可读存储介质，存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1至10中任一项所述的方法的步骤。
PCT/CN2019/085746 2018-06-19 2019-05-07 相机定位方法、装置、终端及存储介质 WO2019242418A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2020552267A JP7026819B2 (ja) 2018-06-19 2019-05-07 カメラの位置決め方法および装置、端末並びにコンピュータプログラム
EP19822963.5A EP3813014B1 (en) 2018-06-19 2019-05-07 Camera localization method, terminal and storage medium
US16/928,427 US11210810B2 (en) 2018-06-19 2020-07-14 Camera localization method and apparatus, terminal, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810630115.7A CN110148178B (zh) 2018-06-19 2018-06-19 相机定位方法、装置、终端及存储介质
CN201810630115.7 2018-06-19

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/928,427 Continuation US11210810B2 (en) 2018-06-19 2020-07-14 Camera localization method and apparatus, terminal, and storage medium

Publications (1)

Publication Number Publication Date
WO2019242418A1 true WO2019242418A1 (zh) 2019-12-26

Family

ID=67589086

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/085746 WO2019242418A1 (zh) 2018-06-19 2019-05-07 相机定位方法、装置、终端及存储介质

Country Status (5)

Country Link
US (1) US11210810B2 (zh)
EP (1) EP3813014B1 (zh)
JP (1) JP7026819B2 (zh)
CN (1) CN110148178B (zh)
WO (1) WO2019242418A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021170299A (ja) * 2020-04-17 2021-10-28 Kddi株式会社 点群処理システム、方法及びプログラム
CN114187349A (zh) * 2021-11-03 2022-03-15 深圳市正运动技术有限公司 产品加工方法、装置、终端设备以及存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112013844B (zh) * 2019-05-31 2022-02-11 北京小米智能科技有限公司 建立室内环境地图的方法及装置
CN111310585B (zh) * 2020-01-20 2023-08-01 北京百度网讯科技有限公司 用于生成信息的方法和装置
CN113220928B (zh) * 2020-01-21 2024-06-11 北京达佳互联信息技术有限公司 一种图像搜索方法、装置、电子设备及存储介质
US11232315B2 (en) * 2020-04-28 2022-01-25 NextVPU (Shanghai) Co., Ltd. Image depth determining method and living body identification method, circuit, device, and medium
CN112966725B (zh) * 2021-02-08 2024-01-16 深圳市优必选科技股份有限公司 匹配模板图像的方法、装置及终端设备
US11423576B1 (en) * 2021-07-23 2022-08-23 Ford Global Technologies, Llc Infrastructure node localization with respect to alternative frame of reference
CN117940739A (zh) * 2021-10-28 2024-04-26 深圳市大疆创新科技有限公司 可移动平台的定位方法和装置
WO2024015031A1 (en) * 2022-07-11 2024-01-18 Delivers Ai Robotik Otonom Surus Bilgi Teknolojileri A.S. A delivery system and hybrid localization method for a delivery robot
WO2024144926A1 (en) * 2022-12-28 2024-07-04 Qualcomm Incorporated Object aided localization without complete object information

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646391A (zh) * 2013-09-30 2014-03-19 浙江大学 一种针对动态变化场景的实时摄像机跟踪方法
US20160012588A1 (en) * 2014-07-14 2016-01-14 Mitsubishi Electric Research Laboratories, Inc. Method for Calibrating Cameras with Non-Overlapping Views
CN105469405A (zh) * 2015-11-26 2016-04-06 清华大学 基于视觉测程的同时定位与地图构建方法

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5776255B2 (ja) * 2011-03-25 2015-09-09 ソニー株式会社 端末装置、物体識別方法、プログラム及び物体識別システム
GB201208088D0 (en) * 2012-05-09 2012-06-20 Ncam Sollutions Ltd Ncam
CN103279952B (zh) * 2013-05-17 2017-10-17 华为技术有限公司 一种目标跟踪方法和装置
CN103731658B (zh) * 2013-12-25 2015-09-30 深圳市墨克瑞光电子研究院 双目摄像机复位方法和双目摄像机复位装置
JP6540108B2 (ja) * 2015-03-09 2019-07-10 富士通株式会社 画像生成方法、システム、装置、及び端末
US10062010B2 (en) * 2015-06-26 2018-08-28 Intel Corporation System for building a map and subsequent localization
CN106383123B (zh) 2016-10-26 2023-04-18 苏州誉阵自动化科技有限公司 相机定位装置及视觉检测装置
CN107564012B (zh) * 2017-08-01 2020-02-28 中国科学院自动化研究所 面向未知环境的增强现实方法及装置
CN107990899B (zh) * 2017-11-22 2020-06-30 驭势科技（北京）有限公司 一种基于slam的定位方法和系统
CN108648235B (zh) * 2018-04-27 2022-05-17 腾讯科技（深圳）有限公司 相机姿态追踪过程的重定位方法、装置及存储介质
US11636618B2 (en) * 2019-11-14 2023-04-25 Samsung Electronics Co., Ltd. Device and method with simultaneous implementation of localization and mapping
KR102216956B1 (ko) * 2019-11-26 2021-02-19 서울과학기술대학교 산학협력단 다중 시점 영상의 프리로드 시스템 및 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646391A (zh) * 2013-09-30 2014-03-19 浙江大学 一种针对动态变化场景的实时摄像机跟踪方法
US20160012588A1 (en) * 2014-07-14 2016-01-14 Mitsubishi Electric Research Laboratories, Inc. Method for Calibrating Cameras with Non-Overlapping Views
CN105469405A (zh) * 2015-11-26 2016-04-06 清华大学 基于视觉测程的同时定位与地图构建方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021170299A (ja) * 2020-04-17 2021-10-28 Kddi株式会社 点群処理システム、方法及びプログラム
JP7290599B2 (ja) 2020-04-17 2023-06-13 Kddi株式会社 点群処理システム、方法及びプログラム
CN114187349A (zh) * 2021-11-03 2022-03-15 深圳市正运动技术有限公司 产品加工方法、装置、终端设备以及存储介质
CN114187349B (zh) * 2021-11-03 2022-11-08 深圳市正运动技术有限公司 产品加工方法、装置、终端设备以及存储介质

Also Published As

Publication number Publication date
EP3813014A4 (en) 2021-08-11
JP7026819B2 (ja) 2022-02-28
US11210810B2 (en) 2021-12-28
US20200342626A1 (en) 2020-10-29
EP3813014B1 (en) 2023-08-23
EP3813014A1 (en) 2021-04-28
CN110148178B (zh) 2022-02-22
JP2021520540A (ja) 2021-08-19
CN110148178A (zh) 2019-08-20

Similar Documents

Publication Publication Date Title
US11205282B2 (en) Relocalization method and apparatus in camera pose tracking process and storage medium
WO2019242418A1 (zh) 相机定位方法、装置、终端及存储介质
US11189037B2 (en) Repositioning method and apparatus in camera pose tracking process, device, and storage medium
US11703993B2 (en) Method, apparatus and device for view switching of virtual environment, and storage medium
US11321870B2 (en) Camera attitude tracking method and apparatus, device, and system
US11481923B2 (en) Relocalization method and apparatus in camera pose tracking process, device, and storage medium
US11158083B2 (en) Position and attitude determining method and apparatus, smart device, and storage medium
CN109947886B (zh) 图像处理方法、装置、电子设备及存储介质
US20240214513A1 (en) Method and apparatus for controlling a plurality of virtual characters, device, and storage medium
US11276183B2 (en) Relocalization method and apparatus in camera pose tracking process, device, and storage medium
WO2019205851A1 (zh) 位姿确定方法、装置、智能设备及存储介质
WO2020215858A1 (zh) 基于虚拟环境的物体构建方法、装置、计算机设备及可读存储介质
CN110599593B (zh) 数据合成的方法、装置、设备及存储介质
CN109886208B (zh) 物体检测的方法、装置、计算机设备及存储介质
CN110570465A (zh) 实时定位与地图构建方法、装置及计算机可读存储介质
CN113384880A (zh) 虚拟场景显示方法、装置、计算机设备及存储介质
CN112308103B (zh) 生成训练样本的方法和装置
CN111369684B (zh) 目标跟踪方法、装置、设备及存储介质
CN113033590B (zh) 图像特征匹配方法、装置、图像处理设备及存储介质
WO2019233299A1 (zh) 地图构建方法、装置及计算机可读存储介质
CN112967261A (zh) 图像融合方法、装置、设备及存储介质
CN114093020A (zh) 动作捕捉方法、装置、电子设备及存储介质
CN112990424A (zh) 神经网络模型训练的方法和装置
CN116502382A (zh) 传感器数据的处理方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19822963

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020552267

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019822963

Country of ref document: EP

Effective date: 20210119