WO2023280038A1 - 一种三维实景模型的构建方法及相关装置 - Google Patents

一种三维实景模型的构建方法及相关装置 Download PDF

Info

Publication number
WO2023280038A1
WO2023280038A1 PCT/CN2022/102581 CN2022102581W WO2023280038A1 WO 2023280038 A1 WO2023280038 A1 WO 2023280038A1 CN 2022102581 W CN2022102581 W CN 2022102581W WO 2023280038 A1 WO2023280038 A1 WO 2023280038A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sub
satellite
intersection
area
Prior art date
Application number
PCT/CN2022/102581
Other languages
English (en)
French (fr)
Inventor
康一飞
万一
盛鑫
姚永祥
张彦峰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023280038A1 publication Critical patent/WO2023280038A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes

Definitions

  • the present application relates to the field of computer technology, in particular to a method for constructing a three-dimensional real scene model and related devices.
  • Digital city represents the development direction of urban informatization and is an important means to promote the informatization of the whole society.
  • the digital city has developed rapidly and has become one of the most promising high-tech fields.
  • the 3D real-scene model of the city is gradually replacing the 2D city map, and has become an important tool in many fields such as urban planning, urban management, public safety, heritage protection, traffic navigation, tourism and vacation, military defense, and film and television entertainment.
  • the 3D reality model constructed based on satellite images is usually relatively rough, and can only recognize the overall color of the building, but cannot extract the detailed texture of the building wall, resulting in poor accuracy of the 3D reality model constructed.
  • the present application provides a method for constructing a three-dimensional real-scene model, which can obtain a three-dimensional real-scene model with high texture resolution, and improves the precision of the three-dimensional real-scene model.
  • the first aspect of the present application provides a method for constructing a three-dimensional real scene model, and the method is applied to an electronic device, such as a server.
  • the method includes: acquiring a first corresponding relationship between a sub-image in a satellite image and a building plane in a three-dimensional model, where the satellite image includes a plurality of sub-images, and the three-dimensional model is constructed based on the satellite image.
  • the server determines the position of the building plane of a certain building in the three-dimensional model in the satellite image, so as to cut out a sub-image in the satellite image, and the sub-image includes the building plane of the building.
  • a first image is acquired, the first image includes an image of the building plane, the first image is obtained by shooting from the ground, and the resolution of the first image is higher than that of the sub-image.
  • the fact that the first image is taken from the ground means that the first image is taken at a relatively short distance from the building, which is different from satellite images taken from high altitude.
  • the second corresponding relationship represents the corresponding relationship between the sub-image and the coordinate points in the image area representing the same building plane in the first image.
  • the second correspondence indicates that the corner points of the building plane in the sub-image have a corresponding relationship with the corner points of the building plane in the first image.
  • texture mapping is performed on the building plane in the three-dimensional model according to the first image to obtain a three-dimensional real scene model. That is to say, the server can establish an accurate correspondence between the first image and the building plane in the three-dimensional model by means of sub-images in the satellite image, so that the The building plane performs texture mapping to obtain a three-dimensional real scene model.
  • texture mapping refers to a process of mapping texture pixels in a two-dimensional space to pixels in a three-dimensional space. To put it simply, performing texture mapping on the building plane in the 3D model according to the first image is to paste the first image onto the surface of the building plane in the 3D space, so as to enhance the realism of the building plane in the 3D space.
  • the 3D model is constructed based on satellite images, it is possible to obtain an accurate correspondence between a certain sub-image in the satellite image and a certain building plane in the 3D model.
  • the ground image is usually obtained based on the terminal shooting one or more buildings at different positions and angles, so it is difficult to directly establish an accurate correspondence between the picture content in the ground image and the building plane in the 3D model.
  • the acquiring the first image includes: acquiring a second image and shooting location information corresponding to the second image, the second image is taken from the ground, and the first The second image is uploaded to the server by the terminal.
  • the first image in the second image is determined based on the shooting location information and the location information of the building plane in the three-dimensional model.
  • the server can determine which buildings are specifically photographed in the second image; then, based on the location information of these buildings in the 3D model, it can determine which of these buildings The position of the building plane in the second image, so that the first image is obtained by cropping the second image, and the first image includes the building plane.
  • the corresponding relationship between a certain building plane in the 3D model and the first image in the second image is confirmed based on the shooting position information of the second image, so as to roughly establish the relationship between the building plane and the first image. so that the subsequent image matching process can directly select the sub-image of the satellite image corresponding to the building plane.
  • the process of matching the second image with multiple sub-images of the satellite image one by one is avoided, and the efficiency of image matching is improved.
  • the acquiring the first image includes: acquiring a third image, the third image is taken from the ground, and the third image is uploaded to the server by the terminal; Semantic segmentation is performed on the third image to obtain a semantic segmentation result of the third image; based on the semantic segmentation result of the third image and the two-dimensional image of the three-dimensional model, determine all Describe the first image.
  • the corresponding relationship between a certain building plane in the 3D model and the first image in the third image is confirmed based on the semantic segmentation results of the third image, so as to roughly establish the relationship between the building plane and the first image. so that the subsequent image matching process can directly select the sub-image of the satellite image corresponding to the building plane.
  • the process of matching the second image with multiple sub-images of the satellite image one by one is avoided, and the efficiency of image matching is improved.
  • the acquiring the first corresponding relationship between the sub-images in the satellite image and the building plane in the three-dimensional model includes: respectively acquiring a plurality of sub-images in the first satellite image and the second satellite image An intersection point, each intersection point in the plurality of intersection points is represented by a vertex and two branch vectors, the starting point of the two branch vectors is the vertex, and the satellite image includes the first satellite image and the The second satellite image, the first satellite image and the second satellite image are obtained from shooting the same area from different locations. Since the corner point of a house is usually a vertex of a square-shaped building plane, the feature description of the corner point of a house can be based on the intersection point. Wherein, the intersection point is an image structure composed of two intersecting line segments, and the intersection point may be specifically represented by a vertex and two branch vectors.
  • intersection points in the first satellite image and the intersection points in the second satellite image are matched to obtain multiple sets of matched intersection points.
  • each set of matched intersections includes an intersection in the first satellite image and an intersection in the second satellite image, and the intersection in the first satellite image and the intersection in the second satellite image are matched.
  • a first corresponding relationship between the sub-image obtained from the multiple sets of matched intersection points and the building plane is obtained, and the building plane is obtained from the multiple sets of matched intersection points.
  • the intersection of multiple sets of matching intersections is obtained.
  • the corner point of the house is taken as the observation object, the feature description of the corner point of the house is realized through the intersection point, and the matching intersection point is searched in the first satellite image and the second satellite image.
  • the first corresponding relationship between the sub-image in the satellite image and the building plane in the three-dimensional model is determined based on the matched intersection points, which improves the matching accuracy between the satellite image and the three-dimensional model.
  • the epipolar line where the vertex of the first intersection point is located and the epipolar line where the vertex of the second intersection point is located are epipolar lines with the same name, and the two branch vectors of the first intersection point pass through
  • the epipolar line passed by the epipolar line and the epipolar line passed by the two branch vectors of the second intersection point are epipolar lines with the same name; the first intersection point and the second intersection point belong to the multiple sets of matching intersection points A group that.
  • the vanishing point passed by the two branch vectors of the first intersection and the vanishing point passed by the two branch vectors of the second intersection are vanishing points with the same name.
  • the matching the first image and the sub-image to obtain the second corresponding relationship between the first area and the second area includes: matching the first image performing image feature point matching with the sub-image to obtain a plurality of first feature points in the first image and a plurality of second feature points in the sub-image, and the plurality of first feature points and the The multiple second feature points correspond one-to-one, the multiple first feature points are used to represent the features of the first image, and the multiple second feature points are used to represent the features of the sub-image; based on the The plurality of first feature points and the plurality of second feature points, determine the second corresponding relationship between the first area and the second area, the first area is based on the plurality of first The second region is obtained based on the plurality of second feature points.
  • the feature point refers to a representative point in the image, that is, a point with characteristic properties.
  • Feature points can represent some unique and salient features in the image. Specifically, feature points can represent objects in a similar or identical form in other similar images containing the same object. To put it simply, for the same object or scene, multiple pictures are collected from different angles, if the same place can be identified as the same; then, these scale-invariant points or blocks are called feature points.
  • Feature points are points that are analyzed by algorithms and contain rich local information, usually appearing at the corner positions in the image or the positions where the texture changes drastically.
  • the method further includes: performing color correction on the first area in the first image based on the second area in the sub-image.
  • the performing color correction on the first region in the first image includes: constructing a triangulation network based on the plurality of second feature points, and the triangulation network is based on a plurality of consecutive A mesh pattern composed of triangles; based on the gray value of the center of gravity of the triangle in the triangulation network, construct a color transformation equation; based on the color transformation equation, perform color correction on the first region in the first image.
  • the second aspect of the present application provides a model construction device, including: an acquisition unit and a processing unit; the acquisition unit is used to acquire the first corresponding relationship between the sub-image in the satellite image and the building plane in the three-dimensional model, so The satellite image includes a plurality of sub-images, the three-dimensional model is constructed based on the satellite image; the processing unit is configured to acquire a first image, the first image includes an image of the building plane, the The first image is taken from the ground, and the resolution of the first image is higher than that of the sub-image; the processing unit is further configured to match the first image and the sub-image to obtain The second corresponding relationship between the first area and the second area, the first area is the area representing the building plane in the first image, and the second area is the area representing the building in the sub-image A region of a plane; the processing unit is further configured to, based on the first correspondence and the second correspondence, perform texture mapping on the building plane in the three-dimensional model according to the first image, to obtain a three-dimensional Reality model.
  • the acquisition unit is further configured to acquire a second image and shooting location information corresponding to the second image, the second image is taken from the ground; the processing unit, It is further configured to determine the first image in the second image based on the shooting location information and the location information of the building plane in the three-dimensional model.
  • the acquiring unit is further configured to acquire a third image, and the third image is obtained from the ground; the processing unit is further configured to perform semantic processing on the third image segmentation to obtain the semantic segmentation result of the third image; the processing unit is further configured to determine the the first image.
  • the processing unit is further configured to respectively acquire multiple intersection points in the first satellite image and the second satellite image, and each intersection point in the multiple intersection points passes through the vertex and Two branch vectors represent that the starting point of the two branch vectors is the vertex, the satellite image includes the first satellite image and the second satellite image, the first satellite image and the second satellite image The images are obtained from shooting the same area from different locations; the processing unit is further configured to match the intersection points in the first satellite image and the intersection points in the second satellite image to obtain multiple sets of matched Intersection: the processing unit is further configured to obtain a first correspondence between the sub-image and the building plane based on the multiple sets of matched intersections, the sub-image is formed by the multiple sets of matched The intersection point is obtained, and the building plane is obtained by the intersection of the multiple sets of matching intersection points.
  • the epipolar line where the vertex of the first intersection point is located and the epipolar line where the vertex of the second intersection point is located are epipolar lines with the same name, and the two branch vectors of the first intersection point pass through
  • the epipolar line passed by the epipolar line and the epipolar line passed by the two branch vectors of the second intersection point are epipolar lines with the same name; the first intersection point and the second intersection point belong to the multiple sets of matching intersection points A group that.
  • the vanishing point passed by the two branch vectors of the first intersection and the vanishing point passed by the two branch vectors of the second intersection are vanishing points with the same name.
  • the processing unit is further configured to perform image feature point matching on the first image and the sub-image to obtain a plurality of first feature points in the first image and A plurality of second feature points in the sub-image, the plurality of first feature points correspond one-to-one to the plurality of second feature points, and the plurality of first feature points are used to represent the first image
  • the features of the plurality of second feature points are used to represent the features of the sub-image
  • the processing unit is further configured to determine the plurality of first feature points and the plurality of second feature points A second corresponding relationship between the first area and the second area, the first area is obtained based on the plurality of first feature points, and the second area is obtained based on the plurality of second features Get it.
  • the processing unit is further configured to perform color correction on the first area in the first image based on the second area in the sub-image.
  • the processing unit is further configured to construct a triangular network based on the plurality of second feature points, and the triangular network is a network graph formed based on a plurality of continuous triangles; the processing The unit is further configured to construct a color transformation equation based on the gray value of the center of gravity of the triangle in the triangulation; the processing unit is further configured to perform an operation on the first region in the first image based on the color transformation equation Perform color correction.
  • the third aspect of the present application provides an electronic device, which includes: a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the The electronic device executes the method in any one implementation manner in the first aspect.
  • a fourth aspect of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is run on a computer, the computer executes the method according to any one of the implementation manners in the first aspect.
  • the fifth aspect of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the method in any one of the implementation manners in the first aspect.
  • a sixth aspect of the present application provides a chip, including one or more processors. Part or all of the processor is used to read and execute the computer program stored in the memory, so as to execute the method in any possible implementation manner of any aspect above.
  • the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or wires.
  • the chip further includes a communication interface, and the processor is connected to the communication interface.
  • the communication interface is used to receive data and/or information to be processed, and the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface.
  • the communication interface may be an input-output interface.
  • FIG. 1 is a schematic diagram of a vanishing point provided in an embodiment of the present application
  • Figure 2a is a three-dimensional model of a building obtained by artificial mapping provided by the embodiment of the present application;
  • Figure 2b is a three-dimensional real-scene model constructed based on satellite images provided by the embodiment of the present application;
  • FIG. 3 is a schematic structural diagram of an electronic device 101 provided in an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a satellite tilt imaging provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a terminal collecting ground images provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of a method for constructing a three-dimensional real scene model provided in an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a three-dimensional model provided in the embodiment of the present application.
  • FIG. 9 is a schematic diagram of a first corresponding relationship between a sub-image in a satellite image and a building plane in a three-dimensional model provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a corrected sub-image provided in the embodiment of the present application.
  • FIG. 11 is a schematic diagram of the correspondence between a first feature point and a second feature point provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of selecting matching intersection points based on epipolar constraints provided by the embodiment of the present application.
  • FIG. 13 is a schematic diagram of selecting matching intersection points based on vanishing point constraints provided by the embodiment of the present application.
  • Fig. 14 is a schematic diagram of obtaining a spatial plane through forward intersection at an intersection provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of obtaining a first image based on a second image provided by an embodiment of the present application.
  • FIG. 16 is a schematic diagram of obtaining a first image based on a third image provided by an embodiment of the present application.
  • FIG. 17 is a schematic diagram of constructing a triangulation network based on feature points provided by an embodiment of the present application.
  • Fig. 18a is a schematic diagram of the construction process of a three-dimensional real scene model provided by the embodiment of the present application.
  • Fig. 18b is a comparative schematic diagram of various model construction methods provided in the embodiment of the present application.
  • FIG. 19 is a schematic structural diagram of an electronic device 1900 provided in an embodiment of the present application.
  • FIG. 20 is a schematic structural diagram of a computer-readable storage medium 2000 provided by an embodiment of the present application.
  • 3D real scene model The simulation model constructed with the help of computer 3D modeling technology can display the actual scene in the real world through virtualization.
  • Digital photogrammetry It uses computer to process digital images or digitized images, and uses computer vision (the core of which is image matching and image recognition) to replace the three-dimensional measurement and recognition of human eyes to complete the automatic extraction of image geometry and physical information.
  • the spatial position of the ground point cannot be determined by using a single photo, only the photographic direction of the ground point can be determined. To obtain the spatial position of the ground point, two overlapping photos must be used to form a stereo pair. It is the basic unit of stereo photogrammetry, and the stereo model formed by it is the basis of stereo photogrammetry.
  • Stereo pair A pair of photographs taken of the same area from two different locations.
  • the stereoscopic model of the photographed object can be seen in the overlapping image part of the image pair by using the stereoscopic observation method and special tools.
  • the inner orientation element is a parameter describing the relative position between the photography center and the photo, including three parameters, namely the vertical distance from the photography center to the photo, and the coordinates of the principal point of the image in the frame coordinate system.
  • Outer orientation element on the basis of restoring the inner orientation element (that is, the recovery of the photographic light beam), the parameters for determining the spatial position and attitude of the photographic beam at the moment of photography are called the outer orientation element.
  • the outer orientation elements of a photo include six parameters, three of which are linear elements, used to describe the spatial coordinates of the photography center; the other three are angular elements, used to describe the spatial attitude of the photo.
  • the outer orientation element is the basic data to determine the geometric relationship of the photographic light beam on the object side.
  • Same-name image point Within the overlapping range of the stereo pair, the image formed by the same object point on the left and right photos is called the same-name image point.
  • the image point with the same name is obtained from the second photography of the same object point at (two) different photographic points during aerial photography.
  • the position and coordinates of the image point with the same name must be accurately determined and measured. Value, in order to ensure the quality and accuracy of stereoscopic observation and measurement.
  • Rays with the same name Two direction lines formed by connecting the same object point and its two image points with the same name on the stereo pair. It can be divided into two cases.
  • the same object point projects light rays to form image points with the same name to different photography stations (or projection centers). of projection light.
  • Intersection in front refers to the method of determining the spatial position of the model point by the intersection of rays of the same name after recovering the light beams in the stereo pair photography and establishing the geometric model.
  • the object space coordinates of the point are determined by the inner and outer orientation elements of the left and right images of the stereo pair and the image coordinate measurement values of the same name point, which is called the spatial front intersection of the stereo pair.
  • Intersection angle The included angle of rays with the same name is the intersection angle.
  • Epipolar line The line connecting the two camera stations is called the projection baseline, and any plane containing the baseline is the epipolar plane.
  • the projection baseline For example, each object point and baseline determine an epipolar plane, and each ray pair with the same name also determines an epipolar plane.
  • the intersection line of the epipolar plane and the image plane is the epipolar line.
  • Epipolar line with the same name Two epipolar lines formed by the intersection of the same nuclear plane and the left and right images.
  • FIG. 1 is a schematic diagram of a vanishing point provided in an embodiment of the present application.
  • the top edge of pillar A, the bottom edge of pillar A, the top edge of pillar B, and the bottom edge of pillar B are parallel to each other, and stretch toward the distant horizon until they converge, and the vanishing point 1 is obtained.
  • the other side of the top of column A, the other side of the bottom of column A, the side of the top of column C and the side of the bottom of column C are parallel to each other, and stretch toward the distant horizon until they converge, and the vanishing point 2 is obtained.
  • Homogeneous Vanishing Points Two matching vanishing points in a stereo pair.
  • Vanishing line The locus of the vanishing points of all straight lines on the plane is the vanishing line.
  • 3D real scene A 3D virtual display technology that uses a digital camera to take multi-angle shots of the existing scene, and then stitches it in post-production and loads a playback program to complete it.
  • Sub-satellite point The sub-satellite point is the intersection of the line connecting the center of the earth and the satellite on the surface of the earth, expressed by geographic longitude and latitude.
  • the ground point directly below the satellite is called the sub-satellite point.
  • a collection of sub-satellite points is called a sub-satellite point trajectory.
  • Texture mapping the process of assigning texture and color information to the surface of a 3D model.
  • the texture is generally derived from a 2D image.
  • Polar coordinate system A two-dimensional coordinate system. Any position in this coordinate system can be represented by an included angle and a distance from the origin-pole. Polar coordinates are used in a wide range of fields, including mathematics, physics, engineering, navigation, aviation, computers, and robotics. The polar coordinate system is particularly useful when the relationship between two points is easily expressed by the included angle and distance; while in the planar rectangular coordinate system, such a relationship can only be expressed using trigonometric functions. For many types of curves, the polar coordinate equation is the simplest form of expression, and even for some curves, only the polar coordinate equation can be expressed.
  • AR Augmented Reality
  • AR is a technology that ingeniously integrates virtual information with the real world.
  • Computer-generated text, images, 3D models, music, video and other virtual information are simulated and applied to the real world.
  • the two kinds of information complement each other, thereby realizing the "enhancement" of the real world.
  • Augmented reality technology can not only effectively reflect the content of the real world, but also promote the display of virtual information content, and these delicate contents complement and superimpose each other.
  • the user needs to make the real world overlap with the computer graphics on the basis of the head-mounted display. After overlapping, the real world can be fully seen around it.
  • Augmented reality technology mainly includes new technologies and means such as multimedia, 3D modeling and scene fusion. There are obvious differences between the information content provided by augmented reality and the information content that humans can perceive.
  • a common modeling scheme for 3D reality models is artificial mapping.
  • FIG. 2a is a three-dimensional building model obtained by artificial mapping provided in the embodiment of the present application.
  • the construction of the city's 3D real scene model mainly relies on the satellite surveying and mapping system.
  • the satellite surveying and mapping system high-resolution optical satellites are used to image the urban area multiple times, and a three-dimensional real-scene model is constructed based on the captured satellite images.
  • the 3D reality model constructed based on satellite images is usually relatively rough, and can only recognize the overall color of the building, but cannot extract the detailed texture of the building wall, resulting in poor accuracy of the 3D reality model constructed.
  • FIG. 2b is a three-dimensional real-scene model constructed based on satellite images provided by the embodiment of the present application. It can be seen from Figure 2b that the 3D real scene model constructed based on satellite images can roughly show the overall color and shape of the building, but the detailed texture of the building cannot be identified.
  • the embodiment of the present application provides a method for constructing a 3D real-scene model. Firstly, the precise first corresponding relationship between the sub-image of the small block in the satellite image and the building plane in the 3D model is obtained, and then the sub-image is photographed with the ground The obtained high-resolution images are matched to determine the second correspondence between the content in the sub-image and the content in the ground image; finally, based on the first correspondence and the second correspondence, the ground image reflects the building plane The detailed texture of the 3D model is mapped to the architectural plane of the 3D model to realize the construction of a high-precision 3D real scene model.
  • the texture mapping of the 3D model based on the ground image can obtain a 3D real-world model with high texture resolution, which improves the accuracy of the 3D real-world model .
  • the method for constructing a three-dimensional real scene model provided in the embodiment of the present application may be applied to electronic devices.
  • the electronic device may be, for example, a server, a smart phone (mobile phone), a personal computer (personal computer, PC), a notebook computer, a tablet computer, a smart TV, a mobile internet device (mobile internet device, MID), a wearable equipment, virtual reality (virtual reality, VR) equipment, augmented reality (augmented reality, AR) equipment, wireless electronic equipment in industrial control (industrial control), wireless electronic equipment in self driving (self driving), remote surgery ( Wireless electronic devices in remote medical surgery, wireless electronic devices in smart grid, wireless electronic devices in transportation safety, wireless electronic devices in smart city, smart home home) in wireless electronic devices, etc.
  • the method provided in the embodiment of the present application will be introduced below by taking the method provided in the embodiment of the present application applied to a server as an example.
  • FIG. 3 is a schematic structural diagram of an electronic device 101 provided in an embodiment of the present application.
  • the electronic device 101 includes a processor 103 , and the processor 103 is coupled to a system bus 105 .
  • the processor 103 may be one or more processors, each of which may include one or more processor cores.
  • a display adapter (video adapter) 107 which can drive a display 109, and the display 109 is coupled to the system bus 105.
  • the system bus 105 is coupled to an input-output (I/O) bus through a bus bridge 111 .
  • the I/O interface 115 is coupled to the I/O bus.
  • the I/O interface 115 communicates with various I/O devices, such as an input device 117 (such as a touch screen, etc.), an external memory 121, (such as a hard disk, a floppy disk, a CD or a flash drive), a multimedia interface, etc.).
  • Transceiver 123 (which can send and/or receive radio communication signals), camera 155 (which can capture still and moving digital video images) and external USB port 125 .
  • the interface connected to the I/O interface 115 may be a USB interface.
  • the processor 103 may be any conventional processor, including a reduced instruction set computing (reduced instruction set computing, RISC) processor, a complex instruction set computing (complex instruction set computing, CISC) processor or a combination of the above.
  • the processor may be a special purpose device such as an ASIC.
  • the electronic device 101 can communicate with the software deployment server 149 through the network interface 129 .
  • the network interface 129 is a hardware network interface, such as a network card.
  • the network 127 may be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (virtual private network, VPN).
  • the network 127 may also be a wireless network, such as a WiFi network, a cellular network, and the like.
  • Hard disk drive interface 131 is coupled to system bus 105 .
  • the hardware driver interface is connected to the hard disk drive 133 .
  • Internal memory 135 is coupled to system bus 105 .
  • Data running on the internal memory 135 may include an operating system (OS) 137 of the electronic device 101 , application programs 143 and a scheduler.
  • OS operating system
  • application programs 143 application programs 143 and a scheduler.
  • the operating system includes a Shell 139 and a kernel (kernel) 141.
  • Shell 139 is an interface between the user and the kernel of the operating system.
  • the shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for user input, interpreting user input to the operating system, and processing various operating system output.
  • Kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources.
  • the kernel 141 directly interacts with the hardware.
  • the operating system kernel usually runs processes, and provides inter-process communication, CPU time slice management, interrupt, memory management, IO management, and the like.
  • the application program 143 includes programs related to instant messaging.
  • the electronic device 101 can download the application program 143 from the software deployment server 149 .
  • FIG. 4 is a schematic structural diagram of an application scenario provided by an embodiment of the present application.
  • satellite image collection is performed through multiple satellites deployed in the sky; multiple satellites photograph the same area on the ground, multiple satellite images are collected, and the collected satellite images are transmitted to the server on the ground.
  • the server After receiving the satellite image transmitted by the satellite, the server constructs a three-dimensional model based on the satellite image. Wherein, the three-dimensional model constructed by the server has not performed the mapping of the detail texture.
  • the user shoots buildings in the city on the ground through the terminal, and obtains ground images corresponding to each building.
  • the user uploads the captured ground image to the server.
  • the server Based on the constructed 3D model, satellite image and ground image uploaded by the user, the server performs texture mapping on the 3D model to obtain a 3D real scene model. Finally, the server sends the constructed 3D real-scene model to the user's terminal, so as to realize functions such as presenting the real-scene model, visual positioning, and AR interaction on the terminal side.
  • FIG. 5 is a schematic diagram of satellite tilt imaging provided by an embodiment of the present application.
  • 8 satellites can be used to realize oblique imaging of the same area, and multiple satellite images taken of the same area can be obtained.
  • the satellite imaging orbit, imaging position, and imaging angle follow the following criteria: at least 8 oblique imaging, at least 1 in each of the 8 quadrants of north, northeast, east, southeast, south, southwest, west, and northwest;
  • the imaging tilt angle is greater than 35° and less than 45°.
  • FIG. 6 is a schematic diagram of a terminal collecting ground images provided in an embodiment of the present application.
  • an image acquisition module is deployed in the terminal for photographing buildings in the city, so as to acquire ground images of the buildings in the city.
  • the terminal may refer to a terminal device such as a smart phone, a tablet computer, a smart TV, a mobile Internet device, a wearable device, a VR device, and an AR device (such as AR glasses or an AR helmet).
  • the user After acquiring images of buildings in the city through the terminal, the user uploads the acquired images to the server.
  • the server receives the image uploaded by the terminal in real time, and feeds back to the terminal in real time whether the image is in an aligned state, that is, whether the overall structure of the building is completely captured in the feedback image, so that the user can adjust the shooting angle in time to obtain an effective image.
  • users can also complete the shooting and uploading operations of buildings during the process of AR interactive experience through the terminal.
  • the server will feed back to the terminal in real time whether the image is in alignment.
  • the surface of the building is lit on the screen of the terminal (an AR special effect), and the name of the building is presented.
  • FIG. 7 is a schematic flowchart of a method for constructing a 3D real scene model provided by an embodiment of the present application.
  • the method for constructing a three-dimensional real scene model includes the following steps 701 - 704 .
  • Step 701 acquiring a first corresponding relationship between a sub-image in a satellite image and a building plane in a three-dimensional model, where the satellite image includes a plurality of sub-images, and the three-dimensional model is constructed based on the satellite image.
  • the server may construct a 3D model based on the acquired satellite image in advance, or the server may obtain the satellite image in advance and construct the 3D model based on the satellite image.
  • the 3D model constructed by the server may be a model without texture maps, that is, the 3D model has only a specific 3D structure without color and detailed textures.
  • FIG. 8 is a schematic structural diagram of a three-dimensional model provided by an embodiment of the present application. As shown in Figure 8, the 3D model has a specific 3D structure of the building and some light and shadow effects, while the building surface of the building has no color and detailed texture.
  • the 3D model constructed by the server may also be a 3D model in which texture mapping is performed based on satellite images, that is, the 3D model has general color and texture, but the resolution of the texture of the building surface in the 3D model is low.
  • the server can determine the position of the building plane of a certain building in the three-dimensional model in the satellite image, so as to cut out a sub-image in the satellite image, and the sub-image includes the building plane of the building.
  • the sub-image may be a building plane including only one building.
  • the server can determine the position of each position coordinate in the sub-image on the building plane, thereby establishing an accurate first correspondence between the sub-image and the building plane. Based on the first correspondence, the server can obtain which position of the building plane in the three-dimensional model each area in the sub-image corresponds to.
  • FIG. 9 is a schematic diagram of a first corresponding relationship between a sub-image in a satellite image and a building plane in a three-dimensional model according to an embodiment of the present application. As shown in FIG. 9 , for a building on the right in the 3D model, the building plane of the building has a corresponding sub-image in the satellite image. The server may obtain the first corresponding relationship between the sub-image and the building plane of the building.
  • the server may establish multiple sets of first correspondences, and each set of first correspondences is used to indicate a sub-image and a building plane in the 3D model Correspondence between.
  • the satellite images are obtained by oblique imaging.
  • Most of the subsequent ground images captured by the terminal are obtained by photographing the building from the front of the building. That is, satellite images are taken from a high-altitude oblique perspective, while ground images are taken from a frontal perspective of the building's front.
  • geometric correction may be performed on the sub-image to eliminate projection distortion caused by oblique imaging.
  • the server may perform geometric correction on the sub-image based on a geometric imaging model of the satellite image, such as a rational polynomial model, so as to correct the shooting perspective of the sub-image to an orthographic perspective.
  • a geometric imaging model of the satellite image such as a rational polynomial model
  • the shooting angle of the sub-image can be converted from a high-altitude oblique angle to a ground orthographic angle. This makes the sub-image appear to be taken from the front of the building after geometric correction. For example, refer to FIG.
  • FIG. 10 is a schematic diagram of a corrected sub-image provided in an embodiment of the present application. It can be seen from Fig. 10 that after the geometric correction is performed on the sub-image, the sub-image eliminates the projection distortion caused by oblique imaging, making the sub-image appear to be taken from the front of the building.
  • Step 702 acquire a first image, the first image includes the image of the building plane, the first image is obtained from the ground, and the resolution of the first image is higher than that of the sub-image.
  • the server can obtain the first image (that is, the ground image) uploaded by the terminal, the first image is taken by the terminal from the ground, and the first image includes the above-mentioned building plane image.
  • the first image is obtained from the ground may refer to: the first image is obtained by the user through a terminal such as a smart phone from the ground such as a road or street; or, the first image is obtained by the user through a smart phone. Terminals such as mobile phones are obtained by taking pictures from one building to another.
  • the fact that the first image is taken from the ground may also mean that the first image is taken by the user from low altitude to the building through a terminal such as a drone.
  • the first image is captured at a relatively short distance from the building, which is different from the satellite image captured from a high altitude. Therefore, the resolution of the first image is also higher than the sub-images in the satellite image.
  • the definition of the building plane captured by the first image is higher, and the detailed texture is more prominent.
  • Step 703 matching the first image and the sub-image to obtain a second corresponding relationship between the first area and the second area, the first area is the building represented in the first image A plane area, the second area is an area representing the building plane in the sub-image.
  • the server may perform image matching on the first image and the sub-image to obtain the difference between the first area and the second area.
  • the first area is an area representing the building plane in the first image
  • the second area is an area representing the building plane in the sub-image. Based on the second correspondence, the correspondence between the sub-image and the regions representing the same building plane in the first image may be determined.
  • the process for the server to perform image matching on the first image and the sub-image may be: the server searches for matching feature points in the first image and the sub-image, and obtains multiple sets of matching feature points ; Then, the server determines the first region in the first image and the second region in the sub-image based on the feature points obtained by searching.
  • the feature point refers to a representative point in the image, that is, a point with characteristic properties.
  • Feature points can represent some unique and salient features in the image. Specifically, feature points can represent objects in a similar or identical form in other similar images containing the same object. To put it simply, for the same object or scene, multiple pictures are collected from different angles, if the same place can be identified as the same; then, these scale-invariant points or blocks are called feature points.
  • Feature points are points that are analyzed by algorithms and contain rich local information, usually appearing at the corner positions in the image or the positions where the texture changes drastically.
  • the server matches the first image and the sub-image to obtain the second corresponding relationship between the first area and the second area, including: the server matching the first image and the sub-image
  • the images are matched to obtain a plurality of first feature points in the first image and a plurality of second feature points in the sub-image, and the plurality of first feature points are identical to the plurality of second feature points One to one correspondence.
  • the server determines a second corresponding relationship between the first area and the second area based on the multiple first feature points and the multiple second feature points, and the first area is based on the multiple The first feature points are obtained, and the second area is obtained based on the plurality of second feature points.
  • the server connects the outermost first feature points among the plurality of first feature points of the first image to obtain the first area enclosed by the outermost first feature points.
  • the first area can be a convex polygon, which can include all the first feature points in the first image.
  • the server connects the outermost second feature points among the multiple second feature points of the sub-images to obtain a second area surrounded by the outermost second feature points.
  • the second area can be a convex polygon, which can contain all the second feature points in the sub-image.
  • the server can regard the matching problem of the sub-image and the first image as a multi-modal image matching problem, and the feature description is in the form of logarithmic polar coordinates, including but not limited to the Gradient Location-Orientation Histogram (Gradient Location-Orientation Histogram, GLOH) etc.
  • GLOH is an extension of the SIFT feature descriptor, and its purpose is to increase the robustness and uniqueness of the feature descriptor.
  • SIFT is a widely used image feature descriptor.
  • the feature point matching between the sub-image and the first image is realized based on GLOH, so that all the matched feature points are located on the building. Since the building is rigid and difficult to change, it can ensure a high alignment accuracy between the first image and the sub-image, so as to facilitate subsequent texture mapping.
  • FIG. 11 is a schematic diagram of correspondence between a first feature point and a second feature point provided in an embodiment of the present application.
  • a one-to-one correspondence between multiple first feature points in the first image and multiple second feature points in the sub-image can be established.
  • a first region surrounded by the first feature points in the outermost layer is obtained.
  • a second area surrounded by the outermost second feature points is obtained.
  • Step 704 based on the first correspondence and the second correspondence, perform texture mapping on the building plane in the 3D model according to the first image, to obtain a 3D real scene model.
  • the server since the server establishes the first corresponding relationship between each area in the sub-image and the building plane in the 3D model, and the relationship between the second area in the sub-image and the first area in the first image The second correspondence. Therefore, the server may determine the correspondence between the first region in the first image and the building plane in the three-dimensional model based on the above-mentioned first correspondence and second correspondence.
  • the server can establish an accurate correspondence between the first image and the building plane in the three-dimensional model by means of sub-images in the satellite image, so that the The building plane performs texture mapping to obtain a three-dimensional real scene model.
  • texture mapping refers to a process of mapping texture pixels in a two-dimensional space to pixels in a three-dimensional space. To put it simply, performing texture mapping on the building plane in the 3D model according to the first image is to paste the first image onto the surface of the building plane in the 3D space, so as to enhance the realism of the building plane in the 3D space.
  • the server may directly perform texture mapping on the 3D model according to the first image, so as to realize the construction of a 3D real scene model.
  • the server may also perform texture mapping of the 3D model based on the first image and sub-images in the satellite image at the same time. For example, based on the first area in the first image, the server can determine the target area corresponding to the first area in the building plane of the three-dimensional model; The target area performs texture mapping. For example, the server may specifically perform texture mapping on the target area based on the central projection perspective transformation model. For areas other than the target area in the same building plane, the server may perform texture mapping according to sub-images in the satellite image. For example, the server may specifically perform texture mapping on areas other than the target area based on a rational polynomial model.
  • the server may receive images of the same building taken by different terminals, that is, sub-images of satellite images may have a corresponding relationship with multiple images. That is to say, for a building plane in a three-dimensional model, the building plane may have a corresponding relationship with multiple ground images, and the multiple ground images can be used to perform texture mapping.
  • the candidate image’s pose, distance, signal-to-noise ratio and other parameters can be comprehensively selected.
  • the perspective transformation, occlusion analysis, and texture selection algorithms involved in this process can be Based on existing methods, such as the energy minimization algorithm method based on the Markov random field model, etc.
  • the texture mapping of the 3D model is performed by using the first image and the sub-image in the satellite image, which can ensure the overall texture integrity of the building plane of the 3D model on the basis of improving the resolution of the building plane of the 3D model as much as possible.
  • the server may perform texture mapping again on the building plane in the 3D model according to the first image, so as to cover the original texture in the 3D model , and then obtain a 3D reality model with high texture resolution.
  • the 3D model is constructed based on satellite images, an accurate correspondence between a certain sub-image in the satellite image and a certain building plane in the 3D model can be obtained.
  • the ground image is usually obtained based on the terminal shooting one or more buildings at different positions and angles, so it is difficult to directly establish an accurate correspondence between the picture content in the ground image and the building plane in the 3D model.
  • the resolution of the ground image is higher than that of the sub-image in the satellite image. Therefore, after the accurate correspondence between the ground image and the building plane of the 3D model is established, the texture mapping of the 3D reality model can be performed according to the ground image, and the 3D reality model with high texture resolution can be obtained, which improves the 3D reality model. accuracy.
  • the above describes the process of the server performing texture mapping on the 3D model based on the first correspondence between the sub-image of the satellite image and the building plane, and the second correspondence between the sub-image and the first image.
  • the process for the server to obtain the first corresponding relationship between the sub-image of the satellite image and the building plane will be described in detail below.
  • the server may use the corner points of houses in the satellite images as observation objects, and use the epipolar relationship and the vanishing line relationship as constraints to realize the extraction and matching of the corner points of houses in the satellite images. Then, based on the roof corner points matched in each satellite image, the spatial plane corresponding to the sub-image in the satellite image is determined, that is, the building plane in the three-dimensional model.
  • the server obtains the first corresponding relationship between the sub-image in the satellite image and the building plane in the three-dimensional model, which may include the following steps 7011-7013.
  • Step 7011 the server acquires multiple intersection points in the first satellite image and the second satellite image, each intersection point in the multiple intersection points is represented by a vertex and two branch vectors, the starting point of the two branch vectors is For the vertex, the satellite image includes the first satellite image and the second satellite image, and the first satellite image and the second satellite image are obtained from shooting the same region from different locations.
  • the intersection point of a house is usually a vertex of a square-shaped building plane
  • the intersection point can be used as a feature description manner of the corner point of the house.
  • the intersection point is an image structure composed of two intersecting line segments, and the intersection point may be specifically represented by a vertex and two branch vectors.
  • the starting point of each of the two branch vectors in the same intersection point is the vertex, and each branch vector can be represented by a length and a direction.
  • the observation object can be focused on the quadrilateral inflection point in three-dimensional space. Further, by constraining the length of the branch vector of the intersection point, the observation object can be focused on the corner of the house. For example, by constraining the length of the branch vectors of intersection points, quadrilateral corner points such as window corner points and door corner points can be excluded, and then the corner points of houses in satellite images can be obtained as much as possible.
  • the length of the branch vector in real space may be limited to be greater than or equal to 10 meters. Since the length of a line segment in the satellite image in the real space can be converted from the spatial resolution of the satellite image and the imaging attitude of the satellite, the length of each branch vector in the satellite image in the real space can be determined.
  • the server may respectively obtain intersection points in the first satellite image and the second satellite image, so as to obtain multiple intersection points in the first satellite image and multiple intersection points in the second satellite image.
  • the first satellite image and the second satellite image are a stereo pair, that is, a pair of images of the same area taken from two different positions.
  • Step 7012 the server matches the intersection points in the first satellite image and the intersection points in the second satellite image to obtain multiple sets of matching intersection points.
  • the server may first determine an intersection point in a certain satellite image, and then search for an intersection point matching the determined intersection point in another satellite image based on a constraint condition of intersection point matching. For example, the server first determines the first intersection point in the first satellite image, and then the server finds the second intersection point in the second satellite image based on the constraints of intersection point matching, wherein the first intersection point and the second intersection point satisfy the intersection Constraints for point matching. Wherein, the first intersection point and the second intersection point belong to one group of the multiple groups of matching intersection points.
  • the constraint condition of intersection matching may be that the epipolar line where the vertex of the first intersection is located and the epipolar line where the vertex of the second intersection is located are epipolar lines with the same name.
  • the server determines that the epipolar line where the vertex of the first intersection point in the first satellite image is located is the first epipolar line, then for the second intersection point matching the first intersection point in the second satellite image, the second intersection The second epipolar line where the vertex of the point is located must be the same epipolar line as the first epipolar line.
  • the epipolar line passed by the two branch vectors of the first intersection point and the epipolar line passed by the two branch vectors of the second intersection point are epipolar lines with the same name. That is to say, assuming that the first intersection point includes the first branch vector and the second branch vector, and the second intersection point includes the third branch vector and the fourth branch vector, the epipolar line passed by the first branch vector of the first intersection point.
  • the epipolar line passed by the third branch vector of the second intersection point is the epipolar line of the same name
  • the epipolar line passed by the second branch vector of the first intersection point is the epipolar line passed by the fourth branch vector of the second intersection point
  • the epipolar line is also the epipolar line of the same name.
  • FIG. 12 is a schematic diagram of selecting matching intersection points based on epipolar constraints provided by an embodiment of the present application.
  • the vertex c of the intersection point J is located on the epipolar line Fc
  • the branch vector cq of the intersection point passes through the epipolar line Fq
  • the branch vector cp of the intersection point passes through the epipolar line Fq.
  • the vertices c1', c2', c3' of the three intersection points J1', J2', J3'
  • epipolar line Fc and epipolar line Fc' have the same name nuclear line.
  • both the branch vector c1'q1' and the branch vector c1'p1' in the intersection J1' cross the epipolar line Fq', so J1' does not match the intersection J.
  • the epipolar line passed by the branch vector c3'q3' corresponding to the branch vector cq of the intersection J in the intersection J3' is Fp'
  • the branch corresponding to the branch vector cp of the intersection J in the intersection J3' The epipolar line crossed by vector c3'p3' is Fq', so intersection point J3' does not match intersection point J.
  • the epipolar line crossed by the branch vector c2'q2' corresponding to the branch vector cq of the intersection J in the intersection J2' is Fq'
  • the epipolar line passed by the branch vector c2'p2' corresponding to cp is Fp'.
  • epipolar line Fq and epipolar line Fq' are epipolar lines with the same name
  • epipolar line Fp and epipolar line Fp' are epipolar lines with the same name. Therefore, in the second satellite image, it can be determined that the intersection point J2' matches the intersection point J in the first satellite image.
  • the matching branch vectors in the matching intersection points must pass through the vanishing point with the same name. In this way, a stronger constraint can be imposed on the direction angle range of the branch vector, and the matching efficiency and accuracy rate can be improved.
  • the vanishing point passed by the two branch vectors of the first intersection point and the vanishing point passed by the two branch vectors of the second intersection point are vanishing points with the same name. That is to say, assuming that the first intersection point includes the first branch vector and the second branch vector, the second intersection point includes the third branch vector and the fourth branch vector, the vanishing point passed by the first branch vector of the first intersection point.
  • the vanishing point passed by the third branch vector of the second intersection point is the same name as the vanishing point, and the vanishing point passed by the second branch vector of the first intersection point is the same as that passed by the fourth branch vector of the second intersection point
  • the vanishing point of is also the vanishing point of the same name.
  • the constraint conditions of the intersection matching can be further strengthened, the accuracy of the intersection matching can be ensured, and the feasibility of the scheme can be improved.
  • FIG. 13 is a schematic diagram of selecting matching intersection points based on the constraints of vanishing points provided by the embodiment of the present application.
  • the vertex c of the intersection point is located on the epipolar line e, and the two branch vectors of the intersection point pass through the vanishing point v1 and the vanishing point v2 respectively.
  • the server can determine three intersection points satisfying the constraint conditions in the second satellite image based on the same-name vanishing point constraint relationship.
  • the vertices c 1 ', c 2 ', and c 3 ' of the three intersection points are all located on epipolar line e' of epipolar line 2 with the same name.
  • the two branch vectors of the three intersection points respectively pass through the vanishing point with the same name of the vanishing point v1 (ie, the vanishing point v1') and the vanishing point of the same name as the vanishing point v2 (ie, the vanishing point v2').
  • the server may use texture feature descriptors to describe the local plane range of the matching intersections, so as to evaluate the texture similarity between the two intersections.
  • texture similarity of the two intersections is greater than or equal to the preset threshold, it is determined that the two intersections are indeed matching intersections, thereby achieving robust matching of the intersections.
  • feature descriptors include but not limited to SIFT, SUR and other descriptors; feature matching methods include but not limited to least square method and the like.
  • Step 7013 the server obtains the first corresponding relationship between the sub-image and the building plane based on the multiple sets of matching intersection points, the sub-image is obtained from the multiple sets of matching intersection points, and the building A plane is obtained by the intersection of the multiple sets of matched intersections.
  • the server can perform intersection on the multiple sets of matching intersections, so as to obtain multiple sets of matching intersections in the actual The corresponding coordinate points in the three-dimensional space, and the space plane formed by these coordinate points, that is, the building plane in the three-dimensional model.
  • the sub-image in the satellite image is obtained based on multiple sets of matching intersection points. For example, in the first satellite image, four intersection points respectively located at the upper left corner of the house, the upper right corner of the house, the lower left corner of the house, and the lower right corner of the house may indicate a certain building plane in the first satellite image. Therefore, based on the four intersection points, a sub-image in the first satellite image can be determined, and the sub-image includes the above four intersection points.
  • FIG. 14 is a schematic diagram of obtaining a spatial plane through forward intersection of intersections provided by an embodiment of the present application.
  • the matching intersection points in the first satellite image and in the second satellite image such as house corner J and house corner J '.
  • the spatial planes corresponding to the four sets of house corner points can be obtained by means of forward intersection.
  • the space plane is the building plane of the three-dimensional model corresponding to the sub-images in the first satellite image and the second satellite image.
  • the corner point of the house is taken as the observation object, the feature description of the corner point of the house is realized through the intersection point, and the matching intersection point is searched in the first satellite image and the second satellite image. Finally, based on the matched intersection points, the first correspondence between the sub-image in the satellite image and the building plane in the 3D model is determined, which improves the matching accuracy between the satellite image and the 3D model.
  • the process for the server to obtain the first correspondence between the sub-image in the satellite image and the building plane in the 3D model is described above.
  • the process for the server to obtain the ground image corresponding to the building plane in the 3D model will be described in detail below.
  • the server may receive a large number of ground images uploaded by different terminals, and these ground images usually capture different buildings.
  • the 3D model acquired by the server includes a large number of buildings. The server needs to confirm which building in the three-dimensional model corresponds to each ground image among the large number of acquired ground images, so that the server performs matching based on the sub-image of the satellite image corresponding to the building and the ground image.
  • the building plane of each building has a sub-image corresponding to the satellite image.
  • the server acquires a large number of ground images, since the ground images usually only capture one or several buildings, the server needs to determine which building is captured in each ground image, so as to be able to select Subimages of satellite imagery corresponding to buildings are used to perform matching with ground imagery.
  • the acquiring the first image by the server may include: acquiring the second image and shooting location information corresponding to the second image by the server.
  • the second image is taken from the ground, and the second image is uploaded to the server by the terminal.
  • the terminal uploads the second image, it also uploads the shooting location information corresponding to the second image, that is, the location information of the terminal when the second image is taken.
  • the terminal may acquire and record the current location information through a built-in sensor in the terminal, so as to upload the shooting location information corresponding to the second image to the server.
  • the server determines the first image in the second image based on the shooting location information and the location information of the building plane in the three-dimensional model. Specifically, based on the shooting location information corresponding to the second image, the server can determine which buildings are specifically photographed in the second image; then, based on the location information of these buildings in the 3D model, it can determine which of these buildings The position of the building plane in the second image, so that the first image is obtained by cropping the second image, and the first image includes the building plane.
  • the server since the server has roughly determined the position of the building plane in the 3D model in the second image, in order to ensure that the cropped first image can include a complete building plane, the server can place the building plane in the second image Based on the position in , the cropped area is slightly expanded, so that the first cropped image can include a complete building plane.
  • the shooting location information acquired by the server may include the specific location information of the terminal when taking the second image (for example, XX Building, XX Road, XX District, XX City), the orientation of the terminal when taking the second image (for example, northeast orientation or southwest orientation) and the shooting angle of the terminal when shooting the second image (for example, an elevation angle of 20°).
  • the server may perform imaging simulation on the three-dimensional model, that is, shoot the three-dimensional model at the shooting location information to obtain a two-dimensional image corresponding to the three-dimensional model. In this way, based on the position of each building plane in the 3D model in the 2D image, the terminal can determine the position of a certain building plane in the second image.
  • FIG. 15 is a schematic diagram of obtaining a first image based on a second image according to an embodiment of the present application.
  • the terminal acquires the shooting location information of the second image, it shoots the 3D model based on the shooting location information to obtain a 2D image corresponding to the 3D model.
  • the server determines the rightmost position of the building plane of the target building in the two-dimensional image. Therefore, based on the location information of the building plane of the target building in the two-dimensional image, the server may crop the right side of the second image to obtain the first image, which includes the building plane of the target building.
  • the corresponding relationship between a certain building plane in the 3D model and the first image in the second image is confirmed based on the shooting position information of the second image, so as to roughly establish the relationship between the building plane and the first image. so that the subsequent image matching process can directly select the sub-image of the satellite image corresponding to the building plane.
  • the process of matching the second image with multiple sub-images of the satellite image one by one is avoided, and the efficiency of image matching is improved.
  • the acquiring the first image by the server may include: acquiring a third image by the server, the third image is taken from the ground, and the third image is uploaded to the server by the terminal .
  • the server performs semantic segmentation on the third image to obtain a semantic segmentation result of the third image.
  • semantic segmentation refers to classifying each pixel in the third image, so as to obtain a classification result of each pixel.
  • the server classifies some pixels in the third image as buildings, some pixels as plants, and some pixels as sky.
  • the server determines the first image in the third image based on the semantic segmentation result of the third image and the two-dimensional image of the three-dimensional model.
  • the server may generate two-dimensional images of the three-dimensional model at various angles based on the three-dimensional model.
  • the server can perform matching on the semantic segmentation result of the third image and the two-dimensional image of the three-dimensional model, so as to obtain a two-dimensional image matched with the semantic segmentation result of the third image.
  • the server can determine the position of the building plane in the third image based on the position of the building plane of the target building in the two-dimensional image, and then crop the first image.
  • FIG. 16 is a schematic diagram of obtaining a first image based on a third image according to an embodiment of the present application.
  • the terminal after acquiring the third image, the terminal performs semantic segmentation on the third image to obtain a semantic segmentation result of the third image.
  • the server acquires the 2D image corresponding to the 3D model, and matches the 2D image of the 3D model with the semantic segmentation result of the third image, so as to obtain the 2D image matched with the semantic segmentation result of the third image.
  • the server can determine the position of the building plane in the third image based on the position of the building plane of the target building in the two-dimensional image, and then crop the first image.
  • the corresponding relationship between a certain building plane in the 3D model and the first image in the third image is confirmed based on the semantic segmentation results of the third image, so as to roughly establish the relationship between the building plane and the first image. so that the subsequent image matching process can directly select the sub-image of the satellite image corresponding to the building plane.
  • the process of matching the second image with multiple sub-images of the satellite image one by one is avoided, and the efficiency of image matching is improved.
  • the ground image acquired by the terminal may have the same color due to different conditions such as the terminal model, shooting time, and shooting weather. Sex is poor.
  • satellite images usually image a large-scale surface area at one time, and each building included in the satellite image has better global color consistency. Therefore, the server may use the satellite image as a reference to perform color correction on the ground image uploaded by the terminal, so as to achieve consistent processing of the brightness and hue of each ground image.
  • the server may perform color correction on the first area in the first image based on the second area in the sub-image.
  • the manners for the server to perform color correction include but are not limited to Gamma transformation, Wallis transformation and other manners.
  • the server when the server receives each ground image uploaded by the terminal, the server can perform color correction on each ground image based on the corresponding sub-image in the satellite image, so as to ensure the brightness and tone of the ground image. Consistency processing to ensure the color consistency of the 3D real scene model.
  • the server performs color correction on the first region in the first image, including: the server constructs a triangulation network based on a plurality of second feature points in the sub-image of the satellite image, and the triangulation network is based on a plurality of consecutive A network of triangles.
  • the method for the server to construct the triangulation includes but is not limited to the Delaunay triangulation method.
  • the server constructs a color transformation equation based on the gray value of the center of gravity of the triangle in the triangulation; wherein, the center of gravity of the triangle refers to the intersection of the three median lines of the triangle.
  • the server performs color correction on the first region in the first image based on the color transformation equation. For example, refer to FIG.
  • each point in the sub-image of the satellite image is the second feature point
  • the solid line connected by the second feature point in the outer layer is the outer boundary of the TIN
  • the dotted line in the outer boundary of the TIN It is the side of the triangulation network
  • the point in the triangle in the triangulation network is the center of gravity of the triangle.
  • the coefficients of the color transformation equation may be constructed by matching points with the same name, and the matching points with the same name refer to points that match between the sub-image of the satellite image and the first image.
  • the server may respectively construct a triangulation network in the sub-image and the first image of the satellite image.
  • the sub-images of the satellite image and the gray value of the first image are respectively obtained by bilinear interpolation, and then substituted into the color transformation equation to construct an equation set.
  • the solution of these coefficients is realized by the method of least squares.
  • color correction can be performed on the first region in the first image.
  • the reason why the center of gravity of the triangular network is used to construct the color transformation equations is that the center of gravity of the triangular network can better meet the characteristics of the same material.
  • the same material is an important prerequisite for color correction, that is, use the sky template to correct the color of the sky image, and use the green leaf template to correct the color of the green leaf image.
  • the feature point matching performed on the sub-images of the satellite image and the first image can guarantee the geometric consistency, which is the basis for the selection of the same material area.
  • the sub-images of the satellite image and the feature points matched in the first image are often located in the corners of walls, window corners, and architectural patterns. These places usually have mixed pixel phenomena, which cannot satisfy the assumption of the same material.
  • the mixed pixel means that there are different types of ground objects in a pixel, mainly appearing at the boundary of land types.
  • the ground reflection or emission spectrum signal obtained by the remote sensor is recorded in units of pixels.
  • a cell contains only one type, which is called a pure cell.
  • a pixel often contains multiple surface types, and such a pixel is a mixed pixel.
  • FIG. 18a is a schematic flow diagram of a construction process of a 3D real scene model provided by the embodiment of the present application.
  • the server first constructs a three-dimensional model based on satellite images, and the three-dimensional model is a model without texture maps. Then, the server extracts the architectural planes of the buildings in the three-dimensional model, and texture mapping needs to be performed on the architectural planes of these buildings.
  • the server determines the sub-images corresponding to each building plane in the satellite image, so as to realize the correction of the satellite image, that is, obtain the satellite sub-image corresponding to each building plane.
  • the server may determine the sub-image corresponding to each building plane in the satellite image based on the intersection point matching method described in the above embodiment.
  • the server can acquire ground images uploaded by the terminal, and the ground images include images of buildings taken from different angles. These ground images can be collected through crowdsourcing, that is, taken by different users through different terminals.
  • the server determines the sub-images corresponding to each building plane in the ground image, so as to realize the correction of the ground image, that is, obtain the ground sub-images corresponding to each building plane.
  • the server can determine the sub-image corresponding to each building plane in the ground image based on the shooting location information of the ground image; the server can also perform semantic segmentation on the ground image, and determine the ground image based on the semantic segmentation result of the ground image The subimages corresponding to the individual building planes in .
  • the server may perform feature point matching on the satellite sub-image and the ground sub-image, thereby obtaining multiple pairs of matching feature points in the satellite sub-image and the ground sub-image. Specifically, there are multiple feature points in the satellite sub-image and the ground sub-image respectively, and each feature point in the satellite sub-image has a corresponding feature point in the ground sub-image.
  • the server can construct a triangulation network in the ground sub-image, and determine the center of gravity of the triangles in the triangulation network. Further, the server may use the satellite sub-image as a reference and construct a color transformation equation set based on the center of gravity of the triangulation in the ground sub-image, so as to perform color correction on the ground sub-image, so that the color of the ground sub-image remains consistent.
  • the server performs texture mapping on the 3D model according to the color-corrected ground sub-image to obtain a 3D real scene model.
  • FIG. 18b is a comparative schematic diagram of various model construction methods provided in the embodiment of the present application.
  • the 3D real scene model constructed based on the method provided by the embodiment of the present application has higher structural accuracy and higher texture Accuracy, higher texture resolution and lower price cost.
  • FIG. 19 is a schematic structural diagram of an electronic device 1900 provided in an embodiment of the present application.
  • the electronic device 1900 includes: an acquisition unit 1901 and a processing unit 1902; the acquisition unit 1901 is used to acquire satellite images The first corresponding relationship between the sub-image of the sub-image and the building plane in the three-dimensional model, the satellite image includes a plurality of sub-images, and the three-dimensional model is constructed based on the satellite image; the processing unit 1902 is configured to Acquire a first image, the first image includes an image of the building plane, the first image is taken from the ground, and the resolution of the first image is higher than that of the sub-image; the processing unit 1902, further matching the first image with the sub-image to obtain a second corresponding relationship between a first area and a second area, where the first area represents the The area of the building plane, the second area is the area representing the building plane in the sub-image; the processing unit 1902 is further configured to, based on the first correspondence and the second correspondence, according to
  • the acquiring unit 1901 is further configured to acquire a second image and shooting location information corresponding to the second image, where the second image is taken from the ground; the processing unit 1902. Further, determine the first image in the second image based on the shooting location information and the location information of the building plane in the three-dimensional model.
  • the acquiring unit 1901 is further configured to acquire a third image, and the third image is captured from the ground; the processing unit 1902 is also configured to process the third image performing semantic segmentation to obtain a semantic segmentation result of the third image; the processing unit 1902 is further configured to determine the third image based on the semantic segmentation result of the third image and the two-dimensional image of the three-dimensional model The first image in the image.
  • the processing unit 1902 is further configured to acquire multiple intersections in the first satellite image and the second satellite image respectively, each intersection of the multiple intersections passes through the vertex and two branch vectors, the starting point of the two branch vectors is the vertex, the satellite image includes the first satellite image and the second satellite image, the first satellite image and the second satellite image The satellite images are obtained from shooting the same area from different locations; the processing unit 1902 is also configured to match the intersection points in the first satellite image and the intersection points in the second satellite image to obtain multiple sets of Matched intersections; the processing unit 1902 is further configured to obtain a first correspondence between the sub-image and the building plane based on the multiple sets of matched intersections, the sub-image is composed of the multiple A set of matching intersection points is obtained, and the building plane is obtained by the intersection of the multiple sets of matching intersection points.
  • the epipolar line where the vertex of the first intersection point is located and the epipolar line where the vertex of the second intersection point is located are epipolar lines with the same name, and the two branch vectors of the first intersection point pass through
  • the epipolar line passed by the epipolar line and the epipolar line passed by the two branch vectors of the second intersection point are epipolar lines with the same name; the first intersection point and the second intersection point belong to the multiple sets of matching intersection points A group that.
  • the vanishing point passed by the two branch vectors of the first intersection and the vanishing point passed by the two branch vectors of the second intersection are vanishing points with the same name.
  • the processing unit 1902 is further configured to perform image feature point matching on the first image and the sub-image to obtain a plurality of first feature points in the first image and multiple second feature points in the sub-image, the multiple first feature points correspond to the multiple second feature points one by one, and the multiple first feature points are used to represent the first Features of the image, the multiple second feature points are used to represent the features of the sub-image; the processing unit 1902 is further configured to, based on the multiple first feature points and the multiple second feature points, determining a second corresponding relationship between the first area and the second area, the first area is obtained based on the plurality of first feature points, and the second area is obtained based on the plurality of first feature points Two feature points are obtained.
  • the processing unit 1902 is further configured to perform color correction on the first area in the first image based on the second area in the sub-image.
  • the processing unit 1902 is further configured to construct a triangulation network based on the plurality of second feature points, where the triangulation network is a network graph formed based on a plurality of continuous triangles; the The processing unit 1902 is further configured to construct a color transformation equation based on the gray value of the center of gravity of the triangle in the triangulation; the processing unit 1902 is further configured to perform a color transformation on the first image based on the color transformation equation The first zone performs color correction.
  • the method for constructing a three-dimensional real-scene model provided by the embodiment of the present application may be specifically executed by a chip in an electronic device, and the chip includes: a processing unit and a communication unit.
  • the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface. , pins or circuits, etc.
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chip in the server executes the method for constructing the three-dimensional real scene model described in the embodiment shown in FIGS. 1 to 18b above.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit can also be a storage unit located outside the chip in the wireless access device end, such as a read-only memory (read-only memory, ROM) Or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM read-only memory
  • RAM random access memory
  • FIG. 20 is a schematic structural diagram of a computer-readable storage medium 2000 provided by an embodiment of the present application.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • the method disclosed in FIG. Computer program instructions on a non-transitory medium or article of manufacture.
  • Figure 20 schematically illustrates a conceptual partial view of an example computer-readable storage medium comprising a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein.
  • computer readable storage medium 2000 is provided using signal bearing medium 2001 .
  • the signal-bearing medium 2001 may include one or more program instructions 2002 which, when executed by one or more processors, may provide the functions or part of the functions described above with respect to FIG. 7 . Additionally, program instructions 2002 in FIG. 20 also describe example instructions.
  • signal bearing medium 2001 may comprise computer readable medium 2003 such as, but not limited to, a hard drive, compact disc (CD), digital video disc (DVD), digital tape, memory, ROM or RAM, and the like.
  • computer readable medium 2003 such as, but not limited to, a hard drive, compact disc (CD), digital video disc (DVD), digital tape, memory, ROM or RAM, and the like.
  • signal bearing media 2001 may comprise computer recordable media 2004 such as, but not limited to, memory, read/write (R/W) CDs, R/W DVDs, and the like.
  • signal bearing medium 2001 may include communication media 2005, such as, but not limited to, digital and/or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).
  • the signal bearing medium 2001 may be conveyed by a wireless form of communication medium 2005 (eg, a wireless communication medium complying with the IEEE 802.9 standard or other transmission protocol).
  • One or more program instructions 2002 may be, for example, computer-executable instructions or logic-implemented instructions.
  • the computing device may be configured to respond to program instructions 2002 communicated to the computing device via one or more of computer-readable media 2003, computer-recordable media 2004, and/or communication media 2005 , providing various operations, functions, or actions.
  • the disclosed system, device and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种三维实景模型的构建方法，应用于电子设备上。本申请方法包括：获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系；获取第一图像，第一图像中包括建筑平面的图像，第一图像是从地面拍摄得到的，第一图像的分辨率高于子图像；对第一图像与子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，第一区域为第一图像中表示建筑平面的区域，第二区域为子图像中表示建筑平面的区域；基于第一对应关系以及第二对应关系，根据第一图像对三维模型中的建筑平面执行纹理映射，得到三维实景模型。基于本方法，能够得到具有高纹理分辨率的三维实景模型，提高了三维实景模型的精度。

Description

一种三维实景模型的构建方法及相关装置
本申请要求于2021年7月5日提交中国专利局、申请号为202110759173.1、发明名称为“一种三维实景模型的构建方法及相关装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种三维实景模型的构建方法及相关装置。
背景技术
数字城市代表了城市信息化的发展方向，是推动整个社会信息化的重要手段。目前，数字城市得到了较快的发展，已成为当前最具发展潜力的高技术领域之一。在数字城市的各类应用系统中，城市的三维实景模型正逐渐取代二维城市地图，成为城市规划、城市管理、公共安全、遗产保护、交通导航、旅游度假、军事国防以及影视娱乐等诸多领域的基础地理空间信息表达形式。
基于卫星测绘系统构建城市三维实景模型是目前比较主流的模型构建方法。在卫星测绘系统中，主要是通过高分辨率的光学卫星对城市区域进行多次倾斜成像，并基于拍摄得到的卫星图像构建三维实景模型。
但是，由于高空中的卫星是对较大区域的地面进行拍摄，难以获取到地面中建筑的具体细节。因此，基于卫星图像构建得到的三维实景模型通常较为粗糙，仅能辨识建筑的整体色彩，而无法提取建筑墙面的细节纹理，导致构建得到的三维实景模型的精度较差。
发明内容
本申请提供了一种三维实景模型的构建方法，能够得到具有高纹理分辨率的三维实景模型，提高了三维实景模型的精度。
本申请第一方面提供一种三维实景模型的构建方法，该方法应用于电子设备上，例如应用于服务器上。该方法包括：获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，所述卫星图像中包括多个子图像，所述三维模型是基于所述卫星图像构建得到的。例如，服务器确定三维模型中的某一个建筑物的建筑平面在卫星图像中的位置，从而在该卫星图像中裁切出一个子图像，该子图像中包括了该建筑物的建筑平面。
获取第一图像，所述第一图像中包括所述建筑平面的图像，所述第一图像是从地面拍摄得到的，所述第一图像的分辨率高于所述子图像。所述第一图像是从地面拍摄得到是指所述第一图像是在距离建筑物较近的距离内拍摄得到的，与从高空中拍摄得到的卫星图像有所区别。
对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面的区域。其中，第二对应关系表示了子图像和第一图像中表示相同建筑平面的图像区域中的坐标点之间的对应关系。例如，第二对应关系表示了子图像中建筑平面的拐角点和第一图像中建筑平面的拐角点具有对应关系。
基于所述第一对应关系以及所述第二对应关系，根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。也就是说，服务器可以借助于卫星图像中的子图像，建立第一图像与所述三维模型中的建筑平面之间准确的对应关系，从而能够根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
其中，纹理映射是指将二维空间中的纹理像素映射到三维空间中的像素的过程。简单来说，根据第一图像对三维模型中的建筑平面执行纹理映射就是将第一图像中贴到三维空间中的建筑平面的表面，以增强三维空间中的建筑平面的真实感。
本方案中，由于三维模型是基于卫星图像构建得到的，因此能够获取到卫星图像中的某个子图像与三维模型中的某个建筑平面之间精确的对应关系。而地面图像通常是基于终端在不同位置、不同角度对一个或多个建筑拍摄得到的，因此难以直接建立地面图像中的画面内容与三维模型中的建筑平面之间准确的对应关系。
因此，在服务器获取到卫星图像中的某个子图像与三维模型中的某个建筑平面之间精确的对应关系之后，通过对卫星图像的子图像与地面图像执行匹配，实现了两个二维图像之间的画面内容的准确匹配。这样一来，基于子图像与地面图像之间的对应关系，以及子图像与三维模型中的建筑平面之间的对应关系，即可确定地面图像的画面内容与三维模型中的建筑平面之间精确的对应关系，从而实现将地面图像中反映建筑平面的细节纹理映射到三维模型的建筑平面中，实现高精度的三维实景模型的构建。
在一种可能的实现方式中，所述获取第一图像，包括：获取第二图像以及所述第二图像对应的拍摄位置信息，所述第二图像是从地面拍摄得到的，且所述第二图像是由终端上传给服务器的。基于所述拍摄位置信息以及所述建筑平面在所述三维模型中的位置信息，确定所述第二图像中的所述第一图像。
具体地，基于第二图像对应的拍摄位置信息，服务器可以确定第二图像中具体拍摄了哪些建筑物；然后，基于这些建筑物在三维模型中的位置信息，可以确定这些建筑物中的某一个建筑平面在第二图像中的位置，从而在第二图像中裁切得到第一图像，该第一图像包括了该建筑平面。
本方案中，通过基于第二图像的拍摄位置信息来确认三维模型中的某一个建筑平面与第二图像中的第一图像之间的对应关系，从而粗略地建立建筑平面与第一图像之间的对应关系，以便于后续的图像匹配过程能够直接选择该建筑平面对应的卫星图像的子图像。避免了将第二图像分别与卫星图像的多个子图像一一匹配的过程，提高了图像匹配的效率。
在一种可能的实现方式中，所述获取第一图像，包括：获取第三图像，所述第三图像是从地面拍摄得到的，且所述第三图像是由终端上传给服务器的；对所述第三图像执行语义分割，以得到所述第三图像的语义分割结果；基于所述第三图像的语义分割结果以及所述三维模型的二维图像，确定所述第三图像中的所述第一图像。
本方案中，通过基于第三图像的语义分割结果来确认三维模型中的某一个建筑平面与第三图像中的第一图像之间的对应关系，从而粗略地建立建筑平面与第一图像之间的对应关系，以便于后续的图像匹配过程能够直接选择该建筑平面对应的卫星图像的子图像。避免了将第二图像分别与卫星图像的多个子图像一一匹配的过程，提高了图像匹配的效率。
在一种可能的实现方式中，所述获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，包括：分别获取第一卫星图像和第二卫星图像中的多个交叉点，所述多个交叉点中的每个交叉点通过顶点和两个分支向量表示，所述两个分支向量的起点为所述顶点，所述卫星图像包括所述第一卫星图像和所述第二卫星图像，所述第一卫星图像和所述第二卫星图像是从不同位置对同一地区拍摄得到的。由于房屋拐角点通常是方形形状的建筑平面的一个顶点，因此可以基于交叉点来作为房屋拐角点的特征描述方式。其中，交叉点是两个相交线段组成的图像结构，交叉点具体可以由一个顶点和两个分支向量表示。
对所述第一卫星图像中的交叉点以及所述第二卫星图像中的交叉点进行匹配，得到多组匹配的交叉点。其中，每一组匹配的交叉点均包括第一卫星图像中的一个交叉点以及第二卫星图像中的一个交叉点，且第一卫星图像中的交叉点与第二卫星图像中的交叉点是匹配的。
基于所述多组匹配的交叉点，得到所述子图像与所述建筑平面之间的第一对应关系，所述子图像由所述多组匹配的交叉点得到，所述建筑平面由所述多组匹配的交叉点交会得到。
本实施例中，以房屋拐角点为观测对象，通过交叉点来实现房屋拐角点的特征描述，并在第一卫星图像和第二卫星图像中查找匹配的交叉点。最终，基于匹配的交叉点确定卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，提高了卫星图像与三维模型之间的匹配准确性。
在一种可能的实现方式中，第一交叉点的顶点所处的核线和第二交叉点的顶点所处的核线为同名核线，所述第一交叉点的两个分支向量所穿过的核线与所述第二交叉点的两个分支向量所穿过的核线为同名核线；所述第一交叉点和所述第二交叉点属于所述多组匹配的交叉点中的一组。
本方案中，通过以同名核线关系来约束相匹配的交叉点的顶点的位置以及分支向量的方向，能够便于快速在两个卫星图像中查找到两个互相匹配的交叉点，提高了方案的可行性。
在一种可能的实现方式中，所述第一交叉点的两个分支向量所穿过的灭点与所述第二交叉点的两个分支向量所穿过的灭点为同名灭点。
本方案中，通过以同名灭点关系约束相匹配的交叉点的分支向量的方向，能够进一步加强交叉点匹配的约束条件，保证交叉点匹配的准确率，提高了方案的可行性。
在一种可能的实现方式中，所述对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，包括：对所述第一图像与所述子图像进行图像特征点的匹配，得到所述第一图像中的多个第一特征点以及所述子图像中的多个第二特征点，所述多个第一特征点与所述多个第二特征点一一对应，所述多个第一特征点用于表示所述第一图像的特征，所述多个第二特征点用于表示所述子图像的特征；基于所述多个第一特征点以及所述多个第二特征点，确定所述第一区域与所述第二区域之间的第二对应关系， 所述第一区域是基于所述多个第一特征点得到的，所述第二区域是基于所述多个第二特征点得到的。
其中，特征点是指图像中具有代表性的点，也就是具有特征性质的点。特征点能够表示图像中一些特有的、显著的特征。具体地，特征点能够在其他含有相同目标的相似图像中以一种相似或相同的形式来表示目标。简单来说，对于同一个物体或场景，从不同的角度采集多幅图片，如果相同的地方能够被识别出来是相同的；那么，这些具有尺度不变性的点或块则称为特征点。特征点是经过算法分析出来的，含有丰富局部信息的点，通常出现在图像中的拐角位置或纹理剧烈变化的位置。
在一种可能的实现方式中，所述方法还包括：以所述子图像中的第二区域为基准，对所述第一图像中的第一区域进行色彩纠正。
在一种可能的实现方式中，所述对所述第一图像中的第一区域进行色彩纠正，包括：基于所述多个第二特征点构建三角网，所述三角网是基于多个连续的三角形构成的网状图形；基于所述三角网中三角形的重心的灰度值，构建色彩变换方程；基于所述色彩变换方程，对所述第一图像中的第一区域执行色彩纠正。
本申请第二方面提供一种模型构建装置，包括：获取单元和处理单元；所述获取单元，用于获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，所述卫星图像中包括多个子图像，所述三维模型是基于所述卫星图像构建得到的；所述处理单元，用于获取第一图像，所述第一图像中包括所述建筑平面的图像，所述第一图像是从地面拍摄得到的，所述第一图像的分辨率高于所述子图像；所述处理单元，还用于对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面的区域；所述处理单元，还用于基于所述第一对应关系以及所述第二对应关系，根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
在一种可能的实现方式中，所述获取单元，还用于获取第二图像以及所述第二图像对应的拍摄位置信息，所述第二图像是从地面拍摄得到的；所述处理单元，还用于基于所述拍摄位置信息以及所述建筑平面在所述三维模型中的位置信息，确定所述第二图像中的所述第一图像。
在一种可能的实现方式中，所述获取单元，还用于获取第三图像，所述第三图像是从地面拍摄得到的；所述处理单元，还用于对所述第三图像执行语义分割，以得到所述第三图像的语义分割结果；所述处理单元，还用于基于所述第三图像的语义分割结果以及所述三维模型的二维图像，确定所述第三图像中的所述第一图像。
在一种可能的实现方式中，所述处理单元，还用于分别获取第一卫星图像和第二卫星图像中的多个交叉点，所述多个交叉点中的每个交叉点通过顶点和两个分支向量表示，所述两个分支向量的起点为所述顶点，所述卫星图像包括所述第一卫星图像和所述第二卫星图像，所述第一卫星图像和所述第二卫星图像是从不同位置对同一地区拍摄得到的；所述处理单元，还用于对所述第一卫星图像中的交叉点以及所述第二卫星图像中的交叉点进行 匹配，得到多组匹配的交叉点；所述处理单元，还用于基于所述多组匹配的交叉点，得到所述子图像与所述建筑平面之间的第一对应关系，所述子图像由所述多组匹配的交叉点得到，所述建筑平面由所述多组匹配的交叉点交会得到。
在一种可能的实现方式中，第一交叉点的顶点所处的核线和第二交叉点的顶点所处的核线为同名核线，所述第一交叉点的两个分支向量所穿过的核线与所述第二交叉点的两个分支向量所穿过的核线为同名核线；所述第一交叉点和所述第二交叉点属于所述多组匹配的交叉点中的一组。
在一种可能的实现方式中，所述第一交叉点的两个分支向量所穿过的灭点与所述第二交叉点的两个分支向量所穿过的灭点为同名灭点。
在一种可能的实现方式中，所述处理单元，还用于对所述第一图像与所述子图像进行图像特征点的匹配，得到所述第一图像中的多个第一特征点以及所述子图像中的多个第二特征点，所述多个第一特征点与所述多个第二特征点一一对应，所述多个第一特征点用于表示所述第一图像的特征，所述多个第二特征点用于表示所述子图像的特征；所述处理单元，还用于基于所述多个第一特征点以及所述多个第二特征点，确定所述第一区域与所述第二区域之间的第二对应关系，所述第一区域是基于所述多个第一特征点得到的，所述第二区域是基于所述多个第二特征点得到的。
在一种可能的实现方式中，所述处理单元，还用于以所述子图像中的第二区域为基准，对所述第一图像中的第一区域进行色彩纠正。
在一种可能的实现方式中，所述处理单元，还用于基于所述多个第二特征点构建三角网，所述三角网是基于多个连续的三角形构成的网状图形；所述处理单元，还用于基于所述三角网中三角形的重心的灰度值，构建色彩变换方程；所述处理单元，还用于基于所述色彩变换方程，对所述第一图像中的第一区域执行色彩纠正。
本申请第三方面提供一种电子设备，该电子设备包括：存储器和处理器；所述存储器存储有代码，所述处理器被配置为执行所述代码，当所述代码被执行时，所述电子设备执行如第一方面中的任意一种实现方式的方法。
本申请第四方面提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行如第一方面中的任意一种实现方式的方法。
本申请第五方面提供一种计算机程序产品，当其在计算机上运行时，使得计算机执行如第一方面中的任意一种实现方式的方法。
本申请第六方面提供一种芯片，包括一个或多个处理器。处理器中的部分或全部用于读取并执行存储器中存储的计算机程序，以执行上述任一方面任意可能的实现方式中的方法。
可选地，该芯片该包括存储器，该存储器与该处理器通过电路或电线与存储器连接。可选地，该芯片还包括通信接口，处理器与该通信接口连接。通信接口用于接收需要处理的数据和/或信息，处理器从该通信接口获取该数据和/或信息，并对该数据和/或信息进行 处理，并通过该通信接口输出处理结果。该通信接口可以是输入输出接口。本申请提供的方法可以由一个芯片实现，也可以由多个芯片协同实现。
附图说明
图1为本申请实施例提供的一种灭点的示意图；
图2a为本申请实施例提供的一种人工贴图得到的建筑三维模型；
图2b为本申请实施例提供的一种基于卫星图像构建得到的三维实景模型；
图3为本申请实施例提供的一种电子设备101的结构示意图；
图4为本申请实施例提供的一种应用场景的架构示意图；
图5为本申请实施例提供的一种卫星倾斜成像的示意图；
图6为本申请实施例提供的一种终端采集地面图像的示意图；
图7为本申请实施例提供的一种三维实景模型的构建方法的流程示意图；
图8为本申请实施例提供的一种三维模型的结构示意图；
图9为本申请实施例提供的一种卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系的示意图；
图10为本申请实施例提供的一种纠正后的子图像的示意图；
图11为本申请实施例提供的一种第一特征点与第二特征点之间的对应示意图；
图12为本申请实施例提供的一种基于核线的约束条件来选择匹配的交叉点的示意图；
图13为本申请实施例提供的一种基于灭点的约束条件来选择匹配的交叉点的示意图；
图14为本申请实施例提供的一种通过交叉点前方交会获得空间平面的示意图；
图15为本申请实施例提供的一种基于第二图像得到第一图像的示意图；
图16为本申请实施例提供的一种基于第三图像得到第一图像的示意图；
图17为本申请实施例提供的一种基于特征点构建三角网的示意图；
图18a为本申请实施例提供的一种三维实景模型的构建流程示意图；
图18b为本申请实施例提供的多种模型构建方法的对比示意图；
图19为本申请实施例提供的一种电子设备1900的结构示意图；
图20为本申请实施例提供的一种计算机可读存储介质2000的结构示意图。
具体实施方式
下面结合附图，对本申请的实施例进行描述。本领域普通技术人员可知，随着技术的发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便 包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
为了便于理解，以下先介绍本申请实施例所涉及的技术术语。
三维实景模型：借助计算机三维建模技术构建出的仿真模型，能够通过虚拟化的方式展示现实世界中实际的景物。
数字摄影测量：是利用计算机对数字影像或数字化影像进行处理，用计算机视觉(其核心是影像匹配与影像识别)代替人眼的立体量测与识别，完成影像几何与物理信息的自动提取。
在摄影测量中，利用单张像片是不能确定地面点的空间位置的，只能确定地面点所在的摄影方向。要获得地面点的空间位置，必须利用两张相互重叠的像片构成立体像对。它是立体摄影测量的基本单元，由其构成的立体模型是立体摄影测量的基础。
立体像对：从两个不同位置对同一地区所摄取的一对相片。用立体观测法和专用的工具可以在像对重叠影像部分内看出所摄目标的立体视模型。
内方位元素：内方位元素是描述摄影中心与像片之间相关位置的参数，包括三个参数，即摄影中心到像片的垂距，及像主点在框标坐标系中的坐标。
外方位元素：在恢复内方位元素(即恢复了摄影光束)的基础上，确定摄影光束在摄影瞬间的空间位置和姿态的参数，称为外方位元素。一张像片的外方位元素包括六个参数，其中三个是直线元素，用于描述摄影中心的空间坐标值；另外三个是角元素，用于描述像片的空间姿态。外方位元素是确定摄影光束在物方的几何关系的基本数据。用于表征摄影光束在摄影瞬间的空间位置，包括摄影中心在某一空间直角坐标系中的三维坐标值(即线元素)和确定摄影光束空间方位的三个角定向元素，共6个数据。
同名像点：立体像对重叠范围内，左右像片上同一物点所构成的像称为同名像点。同名像点是由于航空摄影时在(二个)不同摄影点对同一物点二次摄影得到的，在立体观察及量测像点高程时，必须准确确定和量测同名像点的位置及坐标值，才能保证立体观察及量测的质量和精度。
同名光线：同一物点与立体像对上它的两个同名像点联接形成的两条方向线。可分为两种情况，摄影时由同一物点向不同摄影站(或投影中心)投射出构成同名像点的光线；利用投影方式建立立体模型时，在投影光束中，分别通过两同名像点的投影光线。
前方交会：指恢复立体像对摄影时的光束和建立几何模型后，利用同名光线的交会确定模型点空间位置的方法。由立体像对左右两影像的内、外方位元素和同名像点的影像坐标测量值来确定该点的物方空间坐标，称为立体像对的空间前方交会。
交会角：同名光线的夹角为交会角。
核线：两摄站的连线被称为投影基线，包含基线的任意平面即为核面。如，每个物点与基线决定一个核面，每个同名光线对也同样决定一个核面。核面与像面的交线，即为核线。
同名核线：同一核面与左右影像相交形成的两条核线。
灭点：中心投影中，两条或多条代表平行线的线条向远处地平线伸展直至聚合的那一 点，即为灭点。如铁路的两条路轨看上去确实像在地平线上汇合到了一起。画面中可有一个或多个灭点，这取决于构图的座标位置和方向。如图1所示，图1为本申请实施例提供的一种灭点的示意图。在图1中，柱子A顶部的边、柱子A底部的边、柱子B顶部的边以及柱子B底部的边互相平行，且向远处地平线伸展直至聚合，得到灭点1。此外，柱子A顶部的另一边、柱子A底部的另一边、柱子C顶部的边以及柱子C底部的边互相平行，且向远处地平线伸展直至聚合，得到灭点2。
同名灭点：立体像对中相匹配的两个灭点。
灭线：平面上所有直线的灭点的轨迹，即为灭线。
三维实景：运用数码相机对现有场景进行多角度环视拍摄，然后进行后期缝合并加载播放程序来完成的一种三维虚拟展示技术。
星下点：星下点是地球中心与卫星的连线在地球表面上的交点，用地理经、纬度表示。卫星正下方的地面点称为星下点。星下点的集合称为星下点轨迹。
纹理映射：给三维模型表面赋予纹理和色彩信息的过程，纹理一般来源于二维图像。
极坐标系：一个二维坐标系统。该坐标系统中任意位置可由一个夹角和一段相对原点—极点的距离来表示。极坐标系的应用领域十分广泛，包括数学、物理、工程、航海、航空、电脑以及机器人领域。在两点间的关系用夹角和距离很容易表示时，极坐标系便显得尤为有用；而在平面直角坐标系中，这样的关系就只能使用三角函数来表示。对于很多类型的曲线，极坐标方程是最简单的表达形式，甚至对于某些曲线来说，只有极坐标方程能够表示。
增强现实(Augmented Reality，AR)：AR是一种将虚拟信息与真实世界巧妙融合的技术，广泛运用了多媒体、三维建模、实时跟踪及注册、智能交互、传感等多种技术手段，将计算机生成的文字、图像、三维模型、音乐、视频等虚拟信息模拟仿真后，应用到真实世界中，两种信息互为补充，从而实现对真实世界的“增强”。
增强现实技术不仅能够有效体现出真实世界的内容，也能够促使虚拟的信息内容显示出来，这些细腻内容相互补充和叠加。在视觉化的增强现实中，用户需要在头盔显示器的基础上，促使真实世界能够和电脑图形之间重合在一起，在重合之后可以充分看到真实的世界围绕着它。增强现实技术中主要有多媒体和三维建模以及场景融合等新的技术和手段，增强现实所提供的信息内容和人类能够感知的信息内容之间存在着明显不同。
在传统方法中，一种常见的三维实景模型的建模方案为人工贴图。首先，使用三维建模软件的点线面编辑功能，人工构建建筑结构模型；然后，获取现场照片，并使用三维建模软件的材质和纹理编辑功能，人工手动将照片上的纹理贴附到建筑结构上。示例性地，可以参阅图2a，图2a为本申请实施例提供的一种人工贴图得到的建筑三维模型。
人工贴图的方式通常需要大量的人工参与，且人工贴纹理的过程不可避免地有着畸变、错位。三维实景模型的纹理真实性不能保证，仅能用于非专业展示，不能用于几何特征提取。
由于人工贴图的方式需要大量的人工参与，自动化程度低且成本高，因此目前城市三 维实景模型的构建主要依靠于卫星测绘系统。在卫星测绘系统中，主要是通过高分辨率的光学卫星对城市区域多次侧摆倾斜成像，并基于拍摄得到的卫星图像构建三维实景模型。
但是，由于高空中的卫星是对较大区域的地面进行拍摄，难以获取到地面中建筑的具体细节。因此，基于卫星图像构建得到的三维实景模型通常较为粗糙，仅能辨识建筑的整体色彩，而无法提取建筑墙面的细节纹理，导致构建得到的三维实景模型的精度较差。
示例性地，可以参阅图2b，图2b为本申请实施例提供的一种基于卫星图像构建得到的三维实景模型。由图2b可以看出，基于卫星图像构建得到的三维实景模型能够大概展示建筑的整体色彩和形状，但是建筑的细节纹理却无法辨识。
有鉴于此，本申请实施例提供一种三维实景模型的构建方法，先获取卫星图像中小块的子图像与三维模型中的建筑平面之间精确的第一对应关系，然后将子图像与地面拍摄得到的高分辨率图像进行匹配，从而确定子图像中的内容与地面图像中的内容之间的第二对应关系；最后，基于第一对应关系和第二对应关系，将地面图像中反映建筑平面的细节纹理映射到三维模型的建筑平面中，实现高精度的三维实景模型的构建。由于从地面拍摄得到的地面图像的分辨率高于卫星图像中的子图像，因此基于地面图像对三维模型执行纹理映射，能够得到具有高纹理分辨率的三维实景模型，提高了三维实景模型的精度。
本申请实施例所提供的三维实景模型的构建方法可以应用于电子设备上。示例性地，该电子设备例如可以是服务器、智能手机(mobile phone)、个人电脑(personal computer，PC)、笔记本电脑、平板电脑、智慧电视、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线电子设备、无人驾驶(self driving)中的无线电子设备、远程手术(remote medical surgery)中的无线电子设备、智能电网(smart grid)中的无线电子设备、运输安全(transportation safety)中的无线电子设备、智慧城市(smart city)中的无线电子设备、智慧家庭(smart home)中的无线电子设备等。为了便于叙述，以下将以本申请实施例提供的方法应用于服务器上为例，对本申请实施例所提供的方法进行介绍。
可以参阅图3，图3为本申请实施例提供的一种电子设备101的结构示意图。如图3所示，电子设备101包括处理器103，处理器103和系统总线105耦合。处理器103可以是一个或者多个处理器，其中每个处理器都可以包括一个或多个处理器核。显示适配器(video adapter)107，显示适配器可以驱动显示器109，显示器109和系统总线105耦合。系统总线105通过总线桥111和输入输出(I/O)总线耦合。I/O接口115和I/O总线耦合。I/O接口115和多种I/O设备进行通信，比如输入设备117(如：触摸屏等)，外存储器121,(例如，硬盘、软盘、光盘或优盘)，多媒体接口等)。收发器123(可以发送和/或接收无线电通信信号)，摄像头155(可以捕捉静态和动态数字视频图像)和外部USB端口125。其中，可选地，和I/O接口115相连接的接口可以是USB接口。
其中，处理器103可以是任何传统处理器，包括精简指令集计算(reduced instruction set Computing，RISC)处理器、复杂指令集计算(complex instruction set computing，CISC)处理器或上述的组合。可选地，处理器可以是诸如ASIC的专用装置。
电子设备101可以通过网络接口129和软件部署服务器149通信。示例性的，网络接口129是硬件网络接口，比如，网卡。网络127可以是外部网络，比如因特网，也可以是内部网络，比如以太网或者虚拟私人网络(virtual private network，VPN)。可选地，网络127还可以是无线网络，比如WiFi网络，蜂窝网络等。
硬盘驱动器接口131和系统总线105耦合。硬件驱动接口和硬盘驱动器133相连接。内存储器135和系统总线105耦合。运行在内存储器135的数据可以包括电子设备101的操作系统(OS)137、应用程序143和调度表。
操作系统包括Shell 139和内核(kernel)141。Shell 139是介于使用者和操作系统的内核间的一个接口。shell是操作系统最外面的一层。shell管理使用者与操作系统之间的交互：等待使用者的输入，向操作系统解释使用者的输入，并且处理各种各样的操作系统的输出结果。
内核141由操作系统中用于管理存储器、文件、外设和系统资源的那些部分组成。内核141直接与硬件交互，操作系统内核通常运行进程，并提供进程间的通信，提供CPU时间片管理、中断、内存管理和IO管理等等。
示例性地，在电子设备101为智能手机的情况下，应用程序143包括即时通讯相关的程序。在一个实施例中，在需要执行应用程序143时，电子设备101可以从软件部署服务器149下载应用程序143。
以上介绍了本申请实施例提供的方法所应用的设备，以下将介绍本申请实施例提供的方法所应用的场景。
可以参阅图4，图4为本申请实施例提供的一种应用场景的架构示意图。如图4所示，首先通过部署于高空中的多个卫星来执行卫星图像的采集；多个卫星对地面的同一个地区进行拍摄，采集得到多个卫星图像，并将采集得到的卫星图像传输给地面的服务器。
服务器在接收到卫星所传输的卫星图像之后，则基于卫星图像构建三维模型。其中，服务器所构建的三维模型还没有执行细节纹理的映射。
用户通过终端在地面拍摄城市中的建筑，得到各个建筑对应的地面图像。用户将拍摄得到的地面图像上传至服务器。
服务器基于构建得到的三维模型、卫星图像以及用户上传的地面图像，对三维模型执行纹理映射，得到三维实景模型。最终，服务器将构建得到的三维实景模型下发到用户的终端，从而实现在终端侧呈现实景模型、视觉定位以及AR交互等功能。
可以参阅图5，图5为本申请实施例提供的一种卫星倾斜成像的示意图。如图5所示，在一种具体的应用中，可以通过8个卫星来对同一地区实现倾斜成像，得到对同一个地区所拍摄的多个卫星图像。
具体地，卫星成像轨道、成像位置以及成像角度遵循以下的准则：至少8次倾斜成像，在北、东北、东、东南、南、西南、西、西北8个象限内各至少1次；各个卫星的成像倾斜角大于35°且小于45°。
可以参阅图6，图6为本申请实施例提供的一种终端采集地面图像的示意图。如图6 所示，在终端中部署有图像采集模块，用于对城市中的建筑物进行拍摄，从而采集得到城市中的建筑的地面图像。其中，终端可以是指智能手机、平板电脑、智慧电视、移动互联网设备、可穿戴设备、VR设备以及AR设备(例如AR眼镜或AR头盔)等终端设备。用户通过终端采集得到城市中的建筑物的图像之后，将采集得到的图像上传到服务器中。服务器实时接收终端所上传的图像，并且实时向终端反馈图像是否处于对齐状态，即反馈图像中是否完整地采集到了建筑的整体结构，以便于用户能够适时调整拍摄角度，从而拍摄得到有效的图像。
此外，用户也可以在通过终端进行AR交互体验的过程中，一并完成对建筑物的拍摄和上传操作。当用户通过终端向服务器上传建筑物的图像之后，服务器实时向终端反馈图像是否处于对齐状态。当图像处于对齐状态的情况下，在终端的屏幕上该建筑物的表面被点亮(一种AR特效)，并呈现该建筑物的名称。
可以参阅图7，图7为本申请实施例提供的一种三维实景模型的构建方法的流程示意图。如图7所示，该三维实景模型的构建方法包括以下的步骤701-步骤704。
步骤701，获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，所述卫星图像中包括多个子图像，所述三维模型是基于所述卫星图像构建得到的。
本实施例中，服务器可以预先基于获取到的卫星图像构建得到三维模型，服务器也可以是预先获取到卫星图像以及基于该卫星图像构建得到的三维模型。
可选的，服务器构建得到的三维模型可以是没有纹理贴图的模型，即该三维模型只有具体的三维结构，而没有颜色以及细节纹理。示例性地，可以参阅图8，图8为本申请实施例提供的一种三维模型的结构示意图。如图8所示，该三维模型具有建筑物具体的三维结构以及一些光影效果，而建筑物的建筑表面是没有颜色以及细节纹理的。
服务器构建得到的三维模型也可以是基于卫星图像执行了纹理映射的三维模型，即该三维模型具有大体的颜色以及纹理，但该三维模型中的建筑表面的纹理的分辨率较低。
由于位于高空中的卫星是对城市中一个面积较大的区域进行拍摄，得到卫星图像。因此，在一个卫星图像中，通常包括有大量的建筑物。服务器可以确定三维模型中的某一个建筑物的建筑平面在卫星图像中的位置，从而在该卫星图像中裁切出一个子图像，该子图像中包括了该建筑物的建筑平面。可选的，该子图像中可以是只包括一个建筑物的建筑平面。
进一步地，由于三维模型是基于卫星图像构建得到的，因此服务器可以确定子图像中的各个位置坐标在建筑平面上的位置，从而建立子图像与建筑平面之间精确的第一对应关系。基于第一对应关系，服务器可以获取到子图像中的各个区域对应于三维模型中的建筑平面的哪个位置。示例性地，可以参阅图9，图9为本申请实施例提供的一种卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系的示意图。如图9所示，对于三维模型中右侧的一栋大厦，该大厦的建筑平面在卫星图像具有对应的子图像。服务器可以获取到该子图像与该大厦的建筑平面之间的第一对应关系。
可以理解的是，在实际应用中，由于三维模型中包括大量的建筑物，因此服务器可以 建立多组第一对应关系，每组第一对应关系用于指示一个子图像以及三维模型中的建筑平面之间的对应关系。
由于高空中的卫星是倾斜对地面中的建筑物进行拍摄，因此卫星图像是倾斜成像得到的。而后续由终端所拍摄得到的地面图像大都是从建筑物的正面对建筑物进行拍摄得到的。即卫星图像是从高空的倾斜视角拍摄得到的，而地面图像则是从建筑物正面的正视视角拍摄得到的。
因此，作为可选的，为了便于后续对卫星图像的子图像和地面图像进行匹配，可以对子图像执行几何纠正，以消除倾斜成像所带来的投影畸变。具体地，服务器可以基于卫星图像的几何成像模型，例如有理多项式模型，来对子图像执行几何纠正，从而将子图像的拍摄视角纠正为正视视角。简单来说，通过对子图像执行几何纠正之后，可以使得子图像的拍摄角度从高空倾斜角转换为地面正视角度。这样一来，子图像在执行几何纠正之后，就看起来像是从建筑物的正面拍摄得到的。示例性地，可以参阅图10，图10为本申请实施例提供的一种纠正后的子图像的示意图。由图10可以看出，在对子图像执行几何纠正之后，子图像消除了倾斜成像所带来的投影畸变，使得子图像看起来像是从建筑物的正面拍摄得到的。
步骤702，获取第一图像，所述第一图像中包括所述建筑平面的图像，所述第一图像是从地面拍摄得到的，所述第一图像的分辨率高于所述子图像。
本实施例中，服务器可以获取到由终端所上传的第一图像(即地面图像)，所述第一图像是终端从地面拍摄得到的，且所述第一图像中包括了上述的建筑平面的图像。
其中，所述第一图像是从地面拍摄得到可以是指：所述第一图像是用户通过智能手机等终端从马路、街边等地面拍摄得到的；或者，所述第一图像是用户通过智能手机等终端从一个建筑物内对另一个建筑物拍摄得到的。所述第一图像是从地面拍摄得到还可以是指：第一图像是用户通过无人机等终端从低空中对建筑物拍摄得到的。总的来说，所述第一图像是在距离建筑物较近的距离内拍摄得到的，与从高空中拍摄得到的卫星图像有所区别。因此，所述第一图像的分辨率也要高于所述卫星图像中的子图像。
也就是说，相较于卫星图像中的子图像，第一图像所拍摄得到的建筑平面的清晰度更高，细节纹理更为突出。
步骤703，对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面的区域。
在得到均包括有同一建筑物的建筑平面的第一图像和卫星图像的子图像之后，服务器可以对所述第一图像和所述子图像进行图像匹配，以得到第一区域与第二区域之间的第二对应关系。其中，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面的区域。基于第二对应关系，可以确定子图像和第一图像中表示相同的建筑平面的区域之间的对应关系。
可选的，服务器对所述第一图像和所述子图像进行图像匹配的过程可以是：服务器查找所述第一图像和所述子图像中相互匹配的特征点，得到多组匹配的特征点；然后，服务器基于查找得到的特征点，确定第一图像中的第一区域以及子图像中的第二区域。
其中，特征点是指图像中具有代表性的点，也就是具有特征性质的点。特征点能够表示图像中一些特有的、显著的特征。具体地，特征点能够在其他含有相同目标的相似图像中以一种相似或相同的形式来表示目标。简单来说，对于同一个物体或场景，从不同的角度采集多幅图片，如果相同的地方能够被识别出来是相同的；那么，这些具有尺度不变性的点或块则称为特征点。特征点是经过算法分析出来的，含有丰富局部信息的点，通常出现在图像中的拐角位置或纹理剧烈变化的位置。
具体地，所述服务器对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，包括：服务器对所述第一图像与所述子图像进行匹配，得到所述第一图像中的多个第一特征点以及所述子图像中的多个第二特征点，所述多个第一特征点与所述多个第二特征点一一对应。服务器基于所述多个第一特征点以及所述多个第二特征点，确定所述第一区域与所述第二区域之间的第二对应关系，所述第一区域是基于所述多个第一特征点得到的，所述第二区域是基于所述多个第二特征点得到的。
例如，服务器将第一图像的多个第一特征点中位于最外层的第一特征点连接起来，得到由最外层的第一特征点所围成的第一区域。该第一区域可以为一个凸多边形，能够包含第一图像中的所有第一特征点。类似地，服务器将子图像的多个第二特征点中位于最外层的第二特征点连接起来，得到由最外层的第二特征点所围成的第二区域。该第二区域可以为一个凸多边形，能够包含子图像中的所有第二特征点。
示例性地，由于卫星图像与地面图像的拍摄角度以及拍摄位置均不相同，因此卫星图像的子图像与第一图像之间还存在分辨率和光照的较大差异。基于此，服务器可以将子图像与第一图像的匹配问题视为多模态影像匹配问题，特征描述为对数极坐标的方式，包括但不限于梯度位置定向直方图(Gradient Location-Orientation Histogram,GLOH)等方式。其中，GLOH是SIFT特征描述子的一种扩展，其目的是为了增加特征描述子的鲁棒性和独特性。SIFT则是应用较为广泛的图像特征描述子。
基于GLOH来实现子图像和第一图像之间的特征点匹配，可以使得匹配到的特征点都位于建筑物上。由于建筑物具备刚性、不易变化的特点，可以保证第一图像与子图像之间具有较高的对齐精度，以便于执行后续的纹理映射。
可以参阅图11，图11为本申请实施例提供的一种第一特征点与第二特征点之间的对应示意图。如图11所示，在对子图像和第一图像执行匹配后，能够建立第一图像中的多个第一特征点与子图像中的多个第二特征点之间一一对应的关系。并且，通过将第一图像的多个第一特征点中位于最外层的第一特征点连接起来，得到由最外层的第一特征点所围成的第一区域。通过将子图像的多个第二特征点中位于最外层的第二特征点连接起来，得到由最外层的第二特征点所围成的第二区域。
步骤704，基于所述第一对应关系以及所述第二对应关系，根据所述第一图像对所述 三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
本实施例中，由于服务器建立了子图像中的各个区域与三维模型中的建筑平面之间的第一对应关系，以及子图像中的第二区域与第一图像中的第一区域之间的第二对应关系，因此，服务器可以基于上述的第一对应关系和第二对应关系，确定第一图像中的第一区域与三维模型中的建筑平面之间的对应关系。
也就是说，服务器可以借助于卫星图像中的子图像，建立第一图像与所述三维模型中的建筑平面之间准确的对应关系，从而能够根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
其中，纹理映射是指将二维空间中的纹理像素映射到三维空间中的像素的过程。简单来说，根据第一图像对三维模型中的建筑平面执行纹理映射就是将第一图像中贴到三维空间中的建筑平面的表面，以增强三维空间中的建筑平面的真实感。
可选的，在服务器获取到的三维模型是没有纹理贴图的模型的情况下，服务器可以直接根据第一图像对三维模型执行纹理映射，以实现三维实景模型的构建。
此外，服务器也可以是同时基于第一图像和卫星图像中的子图像来执行三维模型的纹理映射。例如，服务器可以基于第一图像中的第一区域，确定所述三维模型的建筑平面中与第一区域对应的目标区域；然后，服务器根据第一图像中的第一区域对三维模型中的该目标区域执行纹理映射。比如，服务器具体可以基于中心投影透视变换模型对目标区域执行纹理映射。对于同一个建筑平面中除了目标区域之外的其他区域，服务器则可以是根据卫星图像中的子图像来执行纹理映射。比如，服务器具体可以基于有理多项式模型来对目标区域之外的区域执行纹理映射。
在一些情况下，服务器可能会接收到不同终端对同一个建筑物进行拍摄的图像，即卫星图像的子图像可能与多个图像具有对应关系。也就是说，对于三维模型中的建筑平面来说，该建筑平面可能与多个地面图像具有对应关系，该多个地面图像都能够用于执行纹理映射。在这种情况下，如果同一位置具备两组或以上的候选纹理，则可以候选图像的姿态、距离、信噪比等参数综合择优，该过程所涉及的透视变换、遮挡分析、纹理择优算法可以基于现有的方法，如基于马尔科夫随机场模型的能量最小化算法方法等。
通过第一图像和卫星图像中的子图像来执行三维模型的纹理映射，能够在尽可能提高三维模型的建筑平面的分辨率的基础上，保证三维模型的建筑平面的整体纹理完整性。
在服务器获取到的三维模型是基于卫星图像执行了纹理映射的三维模型的情况下，服务器则可以根据第一图像对三维模型中的建筑平面再次执行纹理映射，以覆盖该三维模型中原有的纹理，进而得到具有高纹理分辨率的三维实景模型。
本实施例中，由于三维模型是基于卫星图像构建得到的，因此能够获取到卫星图像中的某个子图像与三维模型中的某个建筑平面之间精确的对应关系。而地面图像通常是基于终端在不同位置、不同角度对一个或多个建筑拍摄得到的，因此难以直接建立地面图像中的画面内容与三维模型中的建筑平面之间准确的对应关系。
因此，在服务器获取到卫星图像中的某个子图像与三维模型中的某个建筑平面之间精 确的对应关系之后，通过对卫星图像的子图像与地面图像执行匹配，实现了两个二维图像之间的画面内容的准确匹配。这样一来，基于子图像与地面图像之间的对应关系，以及子图像与三维模型中的建筑平面之间的对应关系，即可确定地面图像的画面内容与三维模型中的建筑平面之间精确的对应关系，从而实现将地面图像中反映建筑平面的细节纹理映射到三维模型的建筑平面中，实现高精度的三维实景模型的构建。
此外，针对于拍摄得到的建筑平面而言，地面图像的分辨率要高于卫星图像中的子图像。因此，在建立了地面图像与三维模型的建筑平面之间的准确对应关系之后，根据地面图像来执行三维实景模型的纹理映射，能够得到具有高纹理分辨率的三维实景模型，提高了三维实景模型的精度。
以上介绍了服务器基于卫星图像的子图像与建筑平面之间的第一对应关系，以及子图像与第一图像之间的第二对应关系，对三维模型执行纹理映射的过程。以下将详细介绍服务器获取卫星图像的子图像与建筑平面之间的第一对应关系的过程。
在一个可能的实施例中，服务器可以是以卫星图像中的房屋拐角点作为观测对象，以核线关系和灭线关系作为约束条件，实现卫星图像中的房屋拐角点的提取以及匹配。然后，基于各个卫星图像中匹配的屋顶拐角点，确定卫星图像中的子图像对应的空间平面，即三维模型中的建筑平面。
由于建筑物中的建筑平面大都是方形的形状，因此基于建筑平面中的房屋拐角点可以确定建筑平面的多个顶点，进而确定整个建筑平面在图像中的位置。最终，基于建筑平面在图像中的位置，可以确定卫星图像中的子图像与建筑平面的第一对应关系。
示例性地，所述服务器获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，可以包括以下的步骤7011-7013。
步骤7011，服务器获取第一卫星图像和第二卫星图像中的多个交叉点，所述多个交叉点中的每个交叉点通过顶点和两个分支向量表示，所述两个分支向量的起点为所述顶点，所述卫星图像包括所述第一卫星图像和所述第二卫星图像，所述第一卫星图像和所述第二卫星图像是从不同位置对同一地区拍摄得到的。
本实施例中，由于房屋拐角点通常是方形形状的建筑平面的一个顶点，因此可以基于交叉点来作为房屋拐角点的特征描述方式。其中，交叉点是两个相交线段组成的图像结构，交叉点具体可以由一个顶点和两个分支向量表示。同一个交叉点中的两个分支向量中的每个分支向量的起点都为所述顶点，且每个分支向量可以由长度和方向来表示。
由于在卫星图像中，图像中的交叉点都是由三维空间中的相交线段投影而成的。那么，以交叉点作为特征描述，则可以将观测对象聚焦到三维空间中的四边形拐点。进一步地，通过对交叉点的分支向量的长度进行约束，即可将观测对象聚焦到房屋拐角点。例如，通过约束交叉点的分支向量的长度，即可排除窗户拐角点、门的拐角点等四边形拐点，进而尽可能地获取到卫星图像中的房屋拐角点。示例性地，分支向量在真实空间的长度可以是限定为大于或等于10米。由于卫星图像中的某一条线段在真实空间中的长度可以由卫星图像的空间分辨率以及卫星的成像姿态换算得到，因此可以确定卫星图像中的各个分支向量 在真实空间的长度。
其中，服务器可以是分别获取第一卫星图像以及第二卫星图像中的交叉点，从而得到第一卫星图像中的多个交叉点以及第二卫星图像中的多个交叉点。第一卫星图像和第二卫星图像为立体像对，即从两个不同位置对同一地区所拍摄的一对图像。
步骤7012，服务器对所述第一卫星图像中的交叉点以及所述第二卫星图像中的交叉点进行匹配，得到多组匹配的交叉点。
具体地，服务器可以是在某一个卫星图像中先确定一个交叉点，然后基于交叉点匹配的约束条件在另一个卫星图像中查找与所确定的交叉点相匹配的交叉点。例如，服务器先在第一卫星图像中确定第一交叉点，然后服务器基于交叉点匹配的约束条件在第二卫星图像中查找得到第二交叉点，其中第一交叉点和第二交叉点满足交叉点匹配的约束条件。其中，所述第一交叉点和所述第二交叉点属于所述多组匹配的交叉点中的一组。
示例性地，交叉点匹配的约束条件可以为第一交叉点的顶点所处的核线和第二交叉点的顶点所处的核线为同名核线。例如，假设服务器确定第一卫星图像中的第一交叉点的顶点所处的核线为第一核线，则对于第二卫星图像中与第一交叉点匹配的第二交叉点，第二交叉点的顶点所处的第二核线必须是与第一核线为同名核线。
并且，所述第一交叉点的两个分支向量所穿过的核线与所述第二交叉点的两个分支向量所穿过的核线为同名核线。也就是说，假设第一交叉点包括第一分支向量和第二分支向量，第二交叉点包括第三分支向量和第四分支向量，第一交叉点的第一分支向量所穿过的核线与第二交叉点的第三分支向量所穿过的核线为同名核线，且第一交叉点的第二分支向量所穿过的核线与第二交叉点的第四分支向量所穿过的核线也为同名核线。
通过以同名核线关系来约束相匹配的交叉点的顶点的位置以及分支向量的方向，能够便于快速在两个卫星图像中查找到两个互相匹配的交叉点，提高了方案的可行性。
可以参阅图12，图12为本申请实施例提供的一种基于核线的约束条件来选择匹配的交叉点的示意图。如图12所示，在第一卫星图像中，交叉点J的顶点c位于核线Fc上，交叉点的分支向量cq穿过核线Fq，交叉点的分支向量cp穿过核线Fq。在第二卫星图像中，三个交叉点(J1’，J2’，J3’)的顶点c1’，c2’，c3’均位于核线Fc’上，其中核线Fc与核线Fc’为同名核线。然而，交叉点J1’中的分支向量c1’q1’以及分支向量c1’p1’均穿过核线Fq’，因此J1’与交叉点J并不匹配。此外，交叉点J3’中与交叉点J的分支向量cq所对应的分支向量c3’q3’穿过的核线为Fp’，交叉点J3’中与交叉点J的分支向量cp所对应的分支向量c3’p3’穿过的核线为Fq’，因此交叉点J3’与交叉点J并不匹配。
在第二卫星图像中，交叉点J2’中与交叉点J的分支向量cq所对应的分支向量c2’q2’穿过的核线为Fq’，交叉点J2’中与交叉点J的分支向量cp所对应的分支向量c2’p2’穿过的核线为Fp’。其中，核线为Fq与核线为Fq’为同名核线，核线为Fp与核线为Fp’为同名核线。因此，在第二卫星图像中，可以确定交叉点J2’与第一卫星图像中的交叉点J匹配。
可选的，由于城市建筑存在平行线的特征，即房屋拐角点的分支所在直线会穿过灭点。因此，在上述前提下，相匹配的交叉点中相匹配的分支向量必须穿过同名灭点。这样一来，能够对分支向量的方向角范围施加了更强的约束，提高匹配效率和正确率。
示例性地，所述第一交叉点的两个分支向量所穿过的灭点与所述第二交叉点的两个分支向量所穿过的灭点为同名灭点。也就是说，假设第一交叉点包括第一分支向量和第二分支向量，第二交叉点包括第三分支向量和第四分支向量，第一交叉点的第一分支向量所穿过的灭点与第二交叉点的第三分支向量所穿过的灭点为同名灭点，且第一交叉点的第二分支向量所穿过的灭点与第二交叉点的第四分支向量所穿过的灭点也为同名灭点。
通过以同名灭点关系约束相匹配的交叉点的分支向量的方向，能够进一步加强交叉点匹配的约束条件，保证交叉点匹配的准确率，提高了方案的可行性。
可以参阅图13，图13为本申请实施例提供的一种基于灭点的约束条件来选择匹配的交叉点的示意图。如图13所示，对于第一卫星图像中的一个交叉点，该交叉点的顶点c位于核线e上，且该交叉点的两个分支向量分别穿过灭点v1和灭点v2。这样，服务器基于同名灭点约束关系可以确定第二卫星图像中满足约束条件的三个交叉点。其中，三个交叉点的顶点c 1’，c 2’，c 3’均位于核线2的同名核线—核线e’上。并且，这三个交叉点的两个分支向量分别穿过灭点v1的同名灭点(即灭点v1’)和灭点v2的同名灭点(即灭点v2’)上。
可选的，在基于上述的约束条件确定了匹配的交叉点之后，服务器可以通过纹理特征描述子对匹配的交叉点的局部平面范围进行描述，以评价两个交叉点之间的纹理相似程度。当两个交叉点的纹理相似程度大于或等于预设阈值时，则确定这两个交叉点确实为匹配的交叉点，从而实现交叉点的稳健匹配。其中，特征描述子包括但不限于SIFT、SUR等描述子；特征匹配方法包括但不限于最小二乘法等。
步骤7013，服务器基于所述多组匹配的交叉点，得到所述子图像与所述建筑平面之间的第一对应关系，所述子图像由所述多组匹配的交叉点得到，所述建筑平面由所述多组匹配的交叉点交会得到。
在服务器基于上述的约束条件，确定第一卫星图像和第二卫星图像中多组匹配的交叉点之后，服务器可以对多组匹配的交叉点执行前方交会，从而得到多组匹配的交叉点在实际三维空间中对应的坐标点，以及由这些坐标点所构成的空间平面，即三维模型中的建筑平面。其中，卫星图像中的子图像是基于多组匹配的交叉点得到的。例如，在第一卫星图像中，分别位于房屋左上角、房屋右上角、房屋左下角以及房屋右下角的四个交叉点可以指示第一卫星图像中的某一个建筑平面。因此，基于这四个交叉点则可以确定第一卫星图像中的子图像，该子图像包括上述的四个交叉点。
可以参阅图14，图14为本申请实施例提供的一种通过交叉点前方交会获得空间平面的示意图。如图14所示，通过对第一卫星图像和第二卫星图像执行交叉点匹配，得到了第一卫星图像中与第二卫星图像中匹配的交叉点，例如房屋拐角点J和房屋拐角点J’。然后，基于第一卫星图像和第二卫星图像中匹配的四组交叉点，即房屋表面的四组房屋拐角点，可以通过前方交会的方式得到四组房屋拐角点对应的空间平面。该空间平面即为与第一卫星图像和第二卫星图像中的子图像所对应的三维模型的建筑平面。
本实施例中，以房屋拐角点为观测对象，通过交叉点来实现房屋拐角点的特征描述，并在第一卫星图像和第二卫星图像中查找匹配的交叉点。最终，基于匹配的交叉点确定卫 星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，提高了卫星图像与三维模型之间的匹配准确性。
以上介绍了服务器获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系的过程，以下将详细介绍服务器获取与三维模型中的建筑平面对应的地面图像的过程。
可以理解的是，在实际应用中，服务器可以接收到由不同终端所上传的大量地面图像，这些地面图像中通常是拍摄了不同的建筑物。并且，在服务器获取到的三维模型中，包括了大量的建筑物。服务器需要在获取到的大量地面图像中，确认各个地面图像与三维模型中的哪个建筑物具有对应关系，以便于服务器基于该建筑物对应的卫星图像的子图像与地面图像执行匹配。
简单来说，假设三维模型中具有1000个建筑物，分别为建筑物1-建筑物1000，每个建筑物的建筑平面均有对应的卫星图像的子图像。在服务器获取到了大量的地面图像的情况下，由于地面图像通常只是拍摄了一个或数个建筑物，因此服务器需要确定每个地面图像具体是拍摄了哪个建筑物，从而能够选择与地面图像所拍摄的建筑物对应的卫星图像的子图像来与地面图像执行匹配。
在一种可能的实现方式中，服务器获取第一图像，可以包括：服务器获取第二图像以及所述第二图像对应的拍摄位置信息。所述第二图像是从地面拍摄得到的，且所述第二图像是由终端上传给服务器的。在终端上传第二图像的同时，还上传了第二图像对应的拍摄位置信息，即终端在拍摄该第二图像时的位置信息。具体地，终端可以是在拍摄第二图像时，通过终端中内置的传感器获取并记录当前的位置信息，以便于将第二图像对应的拍摄位置信息一并上传给服务器。
服务器基于所述拍摄位置信息以及所述建筑平面在所述三维模型中的位置信息，确定所述第二图像中的所述第一图像。具体地，基于第二图像对应的拍摄位置信息，服务器可以确定第二图像中具体拍摄了哪些建筑物；然后，基于这些建筑物在三维模型中的位置信息，可以确定这些建筑物中的某一个建筑平面在第二图像中的位置，从而在第二图像中裁切得到第一图像，该第一图像包括了该建筑平面。此外，由于服务器是粗略地确定了三维模型中的建筑平面在第二图像中的位置，因此为了保证裁切得到的第一图像中能够包括完整的建筑平面，服务器可以在建筑平面在第二图像中的位置的基础上稍微扩大裁切的面积，从而使得裁切得到的第一图像中能够包括完整的建筑平面。
示例性地，服务器所获取到的拍摄位置信息可以是包括终端在拍摄第二图像时具体的位置信息(例如XX市XX区XX路XX大厦)，终端在拍摄第二图像时的朝向(例如东北朝向或西南朝向)以及终端在拍摄第二图像时的拍摄角度(例如仰角20°)。基于获取到的拍摄位置信息，服务器可以对三维模型进行成像模拟，即在该拍摄位置信息对三维模型进行拍摄，得到三维模型对应的二维图像。这样一来，基于三维模型中的各个建筑平面在该二维图像中的位置，终端可以确定某一个建筑平面在第二图像中的位置。
可以参阅图15，图15为本申请实施例提供的一种基于第二图像得到第一图像的示意图。如图15所示，终端获取到第二图像的拍摄位置信息之后，基于该拍摄位置信息对三维 模型进行拍摄，得到三维模型对应的二维图像。然后，服务器确定目标建筑的建筑平面在该二维图像中的最右侧位置。因此，服务器可以基于目标建筑的建筑平面在二维图像中的位置信息，对第二图像中的右侧位置进行裁切，从而得到第一图像，该第一图像包括了目标建筑的建筑平面。
本方案中，通过基于第二图像的拍摄位置信息来确认三维模型中的某一个建筑平面与第二图像中的第一图像之间的对应关系，从而粗略地建立建筑平面与第一图像之间的对应关系，以便于后续的图像匹配过程能够直接选择该建筑平面对应的卫星图像的子图像。避免了将第二图像分别与卫星图像的多个子图像一一匹配的过程，提高了图像匹配的效率。
在另一种可能的实现方式中，服务器获取第一图像，可以包括：服务器获取第三图像，所述第三图像是从地面拍摄得到的，且所述第三图像是由终端上传给服务器的。服务器对所述第三图像执行语义分割，以得到所述第三图像的语义分割结果。其中，语义分割是指对第三图像中的每一个像素进行分类，从而得到各个像素的分类结果。例如，服务器将第三图像中的部分像素分类为建筑物，部分像素分类为植物，部分像素分类为天空。
然后，服务器基于所述第三图像的语义分割结果以及所述三维模型的二维图像，确定所述第三图像中的所述第一图像。具体地，服务器可以基于三维模型，生成三维模型在各个角度下的二维图像。这样一来，服务器可以对第三图像的语义分割结果以及三维模型的二维图像执行匹配，从而得到与第三图像的语义分割结果所匹配的二维图像。这样一来，服务器可以基于目标建筑物的建筑平面在二维图像中的位置，确定该建筑平面在第三图像中的位置，进而裁切得到第一图像。
可以参阅图16，图16为本申请实施例提供的一种基于第三图像得到第一图像的示意图。如图16所示，终端获取到第三图像之后，对第三图像执行语义分割，得到第三图像的语义分割结果。然后服务器获取三维模型对应的二维图像，并将三维模型的二维图像与第三图像的语义分割结果进行匹配，从而得到与第三图像的语义分割结果所匹配的二维图像。这样一来，服务器可以基于目标建筑物的建筑平面在二维图像中的位置，确定该建筑平面在第三图像中的位置，进而裁切得到第一图像。
本方案中，通过基于第三图像的语义分割结果来确认三维模型中的某一个建筑平面与第三图像中的第一图像之间的对应关系，从而粗略地建立建筑平面与第一图像之间的对应关系，以便于后续的图像匹配过程能够直接选择该建筑平面对应的卫星图像的子图像。避免了将第二图像分别与卫星图像的多个子图像一一匹配的过程，提高了图像匹配的效率。
在本实施例中，由于服务器所获取到的第一图像是终端所上传的地面图像，而终端所获取的地面图像会由于终端的型号、拍摄时间、拍摄天气等各个条件的不同，导致色彩一致性较差。然而，卫星图像通常是对大范围内的地表区域进行一次性成像，卫星图像中所包括的各个建筑物具有更好的全局色彩一致性。因此，服务器可以是以卫星图像为基准，对终端所上传的地面图像执行色彩纠正，以实现各个地面图像的亮度和色调的一致性处理。
示例性地，服务器可以是以所述子图像中的第二区域为基准，对所述第一图像中的第 一区域进行色彩纠正。其中，服务器执行色彩纠正的方式包括但不限于Gamma变换以及Wallis变换等方式。
这样一来，服务器在接收到终端所上传的各个地面图像时，服务器都可以是以卫星图像中对应的子图像为基准，分别对各个地面图像执行色彩纠正，从而保证地面图像的亮度和色调的一致性处理，进而保证三维实景模型的色彩一致性。
具体地，服务器对所述第一图像中的第一区域进行色彩纠正，包括：服务器基于卫星图像的子图像中的多个第二特征点构建三角网，所述三角网是基于多个连续的三角形构成的网状图形。服务器构建三角网的方法包括但不限于Delaunay三角网方法。然后，服务器基于所述三角网中三角形的重心的灰度值，构建色彩变换方程；其中，三角形的重心是指三角形三条中线的交点。最后，服务器基于所述色彩变换方程，对所述第一图像中的第一区域执行色彩纠正。示例性地，可以参阅图17，图17为本申请实施例提供的一种基于特征点构建三角网的示意图。在图17中，卫星图像的子图像中的各点即为第二特征点，由外层的第二特征点连接起来的实线即为三角网的外边界，三角网的外边界内的虚线即为三角网的边，三角网中的三角形内的点即为三角形的重心。
其中，色彩变换方程的系数可以是由同名匹配点构建，同名匹配点是指卫星图像的子图像与第一图像中匹配的点。具体地，服务器可以分别在卫星图像的子图像和第一图像中构建三角网。在基于特征点构建的三角网的重心上，分别使用双线性插值的方式，获取卫星图像的子图像以及第一图像的灰度值，代入色彩变换方程构建方程组。通过将方程系数看作待求解的变量，以最小二乘方法实现这些系数的解算。最后，基于色彩变换方程，可以对第一图像中的第一区域执行色彩纠正。
本实施例中，取三角网的重心构建色彩变换方程组的原因是：三角网的重心能够更好地满足材质相同的特点。其中，材质相同是色彩纠正工作的重要前提，即用天空模板去纠正天空图像的色彩，用绿叶模板去纠正绿叶图像的色彩。对卫星图像的子图像以及第一图像所执行的特征点匹配能够保证几何一致性，从而为选定相同材质区域的依据。但是，卫星图像的子图像和第一图像中匹配得到的特征点往往位于墙角、窗户拐角、建筑花纹处，这些地方通常会出现混合像元现象，无法满足材质相同的假设。因此，基于三角网的重心来构建色彩变换方程组则可以很好地规避混合像元现象，提高色彩纠正的准确率。其中，混合像元是指在一个像元内存在有不同类型的地物，主要出现在地类的边界处。遥感器所获取的地面反射或发射光谱信号是以像元为单位记录的。一个像元内仅包含一种类型，这种像元称为纯像元。然而，多数情况下一个像元内往往包含多种地表类型，这种像元就是混合像元。
可以参阅图18a，图18a为本申请实施例提供的一种三维实景模型的构建流程示意图。如图18a所示，服务器先基于卫星图像构建得到三维模型，该三维模型为没有纹理贴图的模型。然后，服务器提取三维模型中的各个建筑物的建筑平面，这些建筑物的建筑平面需要执行纹理映射。
基于提取得到的建筑平面，服务器确定卫星图像中与各个建筑平面对应的子图像，以实现卫星图像的纠正，即得到与各个建筑平面对应的卫星子图像。具体地，服务器可以基 于上述实施例所述的交叉点匹配方法来确定卫星图像中与各个建筑平面对应的子图像。
此外，服务器可以获取终端所上传的地面图像，地面图像包括了从不同角度所拍摄的建筑物的图像。这些地面图像可以是通过众包的方式搜集得到的，即由不同的用户通过不同的终端拍摄得到的。基于提取得到的建筑平面，服务器确定地面图像中与各个建筑平面对应的子图像，以实现地面图像的纠正，即得到与各个建筑平面对应的地面子图像。具体地，服务器可以是基于地面图像的拍摄位置信息来确定地面图像中与各个建筑平面对应的子图像；服务器还可以是对地面图像执行语义分割，并基于地面图像的语义分割结果来确定地面图像中与各个建筑平面对应的子图像。
在得到每个建筑平面对应的卫星子图像以及地面子图像之后，服务器可以对卫星子图像和地面子图像执行特征点匹配，从而得到卫星子图像和地面子图像中的多对匹配的特征点。具体地，卫星子图像和地面子图像中分别有多个特征点，且卫星子图像中的每个特征点在地面子图像中均有对应的特征点。
基于地面子图像中的多个特征点，服务器可以在地面子图像中构建三角网，并且确定三角网中的三角形的重心。进一步地，服务器可以以卫星子图像为基准，基于地面子图像中三角网的重心构建色彩变换方程组，从而对地面子图像执行色彩纠正，使得地面子图像的色彩保持一致性。
最后，服务器根据色彩纠正后的地面子图像，对三维模型执行纹理映射，得到三维实景模型。
以上介绍了本申请实施例提供的一种三维实景模型的构建方法，为了便于理解本申请实施例所提供的方法的有益效果，以下将结合相关技术对比介绍本申请实施例所提供的方法的优势。
示例性地，可以参阅图18b，图18b为本申请实施例提供的多种模型构建方法的对比示意图。如图18b所示，相较于人工构建的三维贴图模型以及基于卫星系统构建的三维实景模型，基于本申请实施例提供的方法所构建的三维实景模型具有更高的结构精度、更高的纹理精度、更高的纹理分辨率以及更低的价格成本。
在图1至图18b所对应的实施例的基础上，为了更好的实施本申请实施例的上述方案，下面还提供用于实施上述方案的相关设备。
具体可以参阅图19，图19为本申请实施例提供的一种电子设备1900的结构示意图，该电子设备1900包括：获取单元1901和处理单元1902；所述获取单元1901，用于获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，所述卫星图像中包括多个子图像，所述三维模型是基于所述卫星图像构建得到的；所述处理单元1902，用于获取第一图像，所述第一图像中包括所述建筑平面的图像，所述第一图像是从地面拍摄得到的，所述第一图像的分辨率高于所述子图像；所述处理单元1902，还用于对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面 的区域；所述处理单元1902，还用于基于所述第一对应关系以及所述第二对应关系，根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
在一种可能的实现方式中，所述获取单元1901，还用于获取第二图像以及所述第二图像对应的拍摄位置信息，所述第二图像是从地面拍摄得到的；所述处理单元1902，还用于基于所述拍摄位置信息以及所述建筑平面在所述三维模型中的位置信息，确定所述第二图像中的所述第一图像。
在一种可能的实现方式中，所述获取单元1901，还用于获取第三图像，所述第三图像是从地面拍摄得到的；所述处理单元1902，还用于对所述第三图像执行语义分割，以得到所述第三图像的语义分割结果；所述处理单元1902，还用于基于所述第三图像的语义分割结果以及所述三维模型的二维图像，确定所述第三图像中的所述第一图像。
在一种可能的实现方式中，所述处理单元1902，还用于分别获取第一卫星图像和第二卫星图像中的多个交叉点，所述多个交叉点中的每个交叉点通过顶点和两个分支向量表示，所述两个分支向量的起点为所述顶点，所述卫星图像包括所述第一卫星图像和所述第二卫星图像，所述第一卫星图像和所述第二卫星图像是从不同位置对同一地区拍摄得到的；所述处理单元1902，还用于对所述第一卫星图像中的交叉点以及所述第二卫星图像中的交叉点进行匹配，得到多组匹配的交叉点；所述处理单元1902，还用于基于所述多组匹配的交叉点，得到所述子图像与所述建筑平面之间的第一对应关系，所述子图像由所述多组匹配的交叉点得到，所述建筑平面由所述多组匹配的交叉点交会得到。
在一种可能的实现方式中，第一交叉点的顶点所处的核线和第二交叉点的顶点所处的核线为同名核线，所述第一交叉点的两个分支向量所穿过的核线与所述第二交叉点的两个分支向量所穿过的核线为同名核线；所述第一交叉点和所述第二交叉点属于所述多组匹配的交叉点中的一组。
在一种可能的实现方式中，所述第一交叉点的两个分支向量所穿过的灭点与所述第二交叉点的两个分支向量所穿过的灭点为同名灭点。
在一种可能的实现方式中，所述处理单元1902，还用于对所述第一图像与所述子图像进行图像特征点的匹配，得到所述第一图像中的多个第一特征点以及所述子图像中的多个第二特征点，所述多个第一特征点与所述多个第二特征点一一对应，所述多个第一特征点用于表示所述第一图像的特征，所述多个第二特征点用于表示所述子图像的特征；所述处理单元1902，还用于基于所述多个第一特征点以及所述多个第二特征点，确定所述第一区域与所述第二区域之间的第二对应关系，所述第一区域是基于所述多个第一特征点得到的，所述第二区域是基于所述多个第二特征点得到的。
在一种可能的实现方式中，所述处理单元1902，还用于以所述子图像中的第二区域为基准，对所述第一图像中的第一区域进行色彩纠正。
在一种可能的实现方式中，所述处理单元1902，还用于基于所述多个第二特征点构建三角网，所述三角网是基于多个连续的三角形构成的网状图形；所述处理单元1902，还用于基于所述三角网中三角形的重心的灰度值，构建色彩变换方程；所述处理单元1902，还用于基于所述色彩变换方程，对所述第一图像中的第一区域执行色彩纠正。
本申请实施例提供的三维实景模型的构建方法具体可以由电子设备中的芯片来执行，该芯片包括：处理单元和通信单元，处理单元例如可以是处理器，通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使服务器内的芯片执行上述图1至图18b所示实施例描述的三维实景模型的构建方法。可选的，存储单元为芯片内的存储单元，如寄存器、缓存等，存储单元还可以是无线接入设备端内的位于芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
可以参阅图20，图20为本申请实施例提供的一种计算机可读存储介质2000的结构示意图。本申请实施例还提供了一种计算机可读存储介质，在一些实施例中，上述图7所公开的方法可以实施为以机器可读格式被编码在计算机可读存储介质上或者被编码在其它非瞬时性介质或者制品上的计算机程序指令。
图20示意性地示出根据这里展示的至少一些实施例而布置的示例计算机可读存储介质的概念性局部视图，示例计算机可读存储介质包括用于在计算设备上执行计算机进程的计算机程序。
在一个实施例中，计算机可读存储介质2000是使用信号承载介质2001来提供的。信号承载介质2001可以包括一个或多个程序指令2002，其当被一个或多个处理器运行时可以提供以上针对图7描述的功能或者部分功能。此外，图20中的程序指令2002也描述示例指令。
在一些示例中，信号承载介质2001可以包含计算机可读介质2003,诸如但不限于，硬盘驱动器、紧密盘(CD)、数字视频光盘(DVD)、数字磁带、存储器、ROM或RAM等等。
在一些实施方式中，信号承载介质2001可以包含计算机可记录介质2004，诸如但不限于，存储器、读/写(R/W)CD、R/W DVD、等等。在一些实施方式中，信号承载介质2001可以包含通信介质2005,诸如但不限于，数字和/或模拟通信介质(例如，光纤电缆、波导、有线通信链路、无线通信链路、等等)。因此，例如，信号承载介质2001可以由无线形式的通信介质2005(例如，遵守IEEE 802.9标准或者其它传输协议的无线通信介质)来传达。
一个或多个程序指令2002可以是，例如，计算机可执行指令或者逻辑实施指令。在一些示例中，计算设备的计算设备可以被配置为，响应于通过计算机可读介质2003、计算机可记录介质2004、和/或通信介质2005中的一个或多个传达到计算设备的程序指令2002，提供各种操作、功能、或者动作。
应该理解，这里描述的布置仅仅是用于示例的目的。因而，本领域技术人员将理解，其它布置和其它元素(例如，机器、接口、功能、顺序、和功能组等等)能够被取而代之地使用，并且一些元素可以根据所期望的结果而一并省略。另外，所描述的元素中的许多是可以被实现为离散的或者分布式的组件的、或者以任何适当的组合和位置来结合其它组件实施的功能实体。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (12)

  1. 一种三维实景模型的构建方法，其特征在于，包括：
    获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，所述卫星图像中包括多个子图像，所述三维模型是基于所述卫星图像构建得到的；
    获取第一图像，所述第一图像中包括所述建筑平面的图像，所述第一图像是从地面拍摄得到的，所述第一图像的分辨率高于所述子图像；
    对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，所述第一区域为所述第一图像中表示所述建筑平面的区域，所述第二区域为所述子图像中表示所述建筑平面的区域；
    基于所述第一对应关系以及所述第二对应关系，根据所述第一图像对所述三维模型中的所述建筑平面执行纹理映射，得到三维实景模型。
  2. 根据权利要求1所述的方法，其特征在于，所述获取第一图像，包括：
    获取第二图像以及所述第二图像对应的拍摄位置信息，所述第二图像是从地面拍摄得到的；
    基于所述拍摄位置信息以及所述建筑平面在所述三维模型中的位置信息，确定所述第二图像中的所述第一图像。
  3. 根据权利要求1所述的方法，其特征在于，所述获取第一图像，包括：
    获取第三图像，所述第三图像是从地面拍摄得到的；
    对所述第三图像执行语义分割，以得到所述第三图像的语义分割结果；
    基于所述第三图像的语义分割结果以及所述三维模型的二维图像，确定所述第三图像中的所述第一图像。
  4. 根据权利要求1-3任意一项所述的方法，其特征在于，所述获取卫星图像中的子图像与三维模型中的建筑平面之间的第一对应关系，包括：
    分别获取第一卫星图像和第二卫星图像中的多个交叉点，所述多个交叉点中的每个交叉点通过顶点和两个分支向量表示，所述两个分支向量的起点为所述顶点，所述卫星图像包括所述第一卫星图像和所述第二卫星图像，所述第一卫星图像和所述第二卫星图像是从不同位置对同一地区拍摄得到的；
    对所述第一卫星图像中的交叉点以及所述第二卫星图像中的交叉点进行匹配，得到多组匹配的交叉点；
    基于所述多组匹配的交叉点，得到所述子图像与所述建筑平面之间的第一对应关系，所述子图像由所述多组匹配的交叉点得到，所述建筑平面由所述多组匹配的交叉点交会得到。
  5. 根据权利要求4所述的方法，其特征在于，第一交叉点的顶点所处的核线和第二交 叉点的顶点所处的核线为同名核线，所述第一交叉点的两个分支向量所穿过的核线与所述第二交叉点的两个分支向量所穿过的核线为同名核线；
    所述第一交叉点和所述第二交叉点属于所述多组匹配的交叉点中的一组。
  6. 根据权利要求4或5所述的方法，其特征在于，所述第一交叉点的两个分支向量所穿过的灭点与所述第二交叉点的两个分支向量所穿过的灭点为同名灭点。
  7. 根据权利要求1-6任意一项所述的方法，其特征在于，所述对所述第一图像与所述子图像进行匹配，以得到第一区域与第二区域之间的第二对应关系，包括：
    对所述第一图像与所述子图像进行图像特征点的匹配，得到所述第一图像中的多个第一特征点以及所述子图像中的多个第二特征点，所述多个第一特征点与所述多个第二特征点一一对应，所述多个第一特征点用于表示所述第一图像的特征，所述多个第二特征点用于表示所述子图像的特征；
    基于所述多个第一特征点以及所述多个第二特征点，确定所述第一区域与所述第二区域之间的第二对应关系，所述第一区域是基于所述多个第一特征点得到的，所述第二区域是基于所述多个第二特征点得到的。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    以所述子图像中的第二区域为基准，对所述第一图像中的第一区域进行色彩纠正。
  9. 根据权利要求8所述的方法，其特征在于，所述对所述第一图像中的第一区域进行色彩纠正，包括：
    基于所述多个第二特征点构建三角网，所述三角网是基于多个连续的三角形构成的网状图形；
    基于所述三角网中三角形的重心的灰度值，构建色彩变换方程；
    基于所述色彩变换方程，对所述第一图像中的第一区域执行色彩纠正。
  10. 一种电子设备，其特征在于，包括存储器和处理器；所述存储器存储有代码，所述处理器被配置为执行所述代码，当所述代码被执行时，所述终端执行如权利要求1至9任一项所述的方法。
  11. 一种计算机可读存储介质，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机上运行时，使得所述计算机执行如权利要求1至9中任一项所述的方法。
  12. 一种计算机程序产品，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机上运行时，使得所述计算机执行如权利要求1至9任一项所述的方法。
PCT/CN2022/102581 2021-07-05 2022-06-30 一种三维实景模型的构建方法及相关装置 WO2023280038A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110759173.1A CN115641401A (zh) 2021-07-05 2021-07-05 一种三维实景模型的构建方法及相关装置
CN202110759173.1 2021-07-05

Publications (1)

Publication Number Publication Date
WO2023280038A1 true WO2023280038A1 (zh) 2023-01-12

Family

ID=84801254

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/102581 WO2023280038A1 (zh) 2021-07-05 2022-06-30 一种三维实景模型的构建方法及相关装置

Country Status (2)

Country Link
CN (1) CN115641401A (zh)
WO (1) WO2023280038A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116681848A (zh) * 2023-05-26 2023-09-01 青岛市勘察测绘研究院 一种道路注记与实景三维模型的融合显示方法
CN117078982A (zh) * 2023-10-16 2023-11-17 山东建筑大学 基于深度学习的大倾角立体像对准密集特征匹配方法
CN117437372A (zh) * 2023-12-20 2024-01-23 中国电子科技集团公司第十五研究所 塔台模拟用三维视景地形图处理方法、服务器及存储介质
CN117664088A (zh) * 2024-01-31 2024-03-08 中国人民解放军战略支援部队航天工程大学 超宽幅垂轨环扫式卫星影像确定同名点方法、系统及设备
CN117876506A (zh) * 2023-12-26 2024-04-12 北京建筑大学 多视影像的色彩处理方法及装置、存储介质、计算机设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117274536B (zh) * 2023-11-22 2024-02-20 北京飞渡科技股份有限公司 一种实景三维模型重构方法及装置
CN117308821B (zh) * 2023-11-28 2024-02-06 江苏华辉建筑装饰工程有限公司 一种基于扫描仪的建筑装饰造型精度检验方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106327573A (zh) * 2016-08-25 2017-01-11 成都慧途科技有限公司 一种针对城市建筑的实景三维建模方法
CN107798731A (zh) * 2017-11-10 2018-03-13 泰瑞数创科技(北京)有限公司 一种基于卫星影像自动建模的方法
CN108053474A (zh) * 2017-12-29 2018-05-18 湖南城市学院 一种新型城市三维建模控制系统及方法
CN109269472A (zh) * 2018-08-08 2019-01-25 广州市城市规划勘测设计研究院 倾斜摄影测量建筑物特征线提取方法、装置与存储介质
US10430961B2 (en) * 2015-12-16 2019-10-01 Objectvideo Labs, Llc Using satellite imagery to enhance a 3D surface model of a real world cityscape

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10430961B2 (en) * 2015-12-16 2019-10-01 Objectvideo Labs, Llc Using satellite imagery to enhance a 3D surface model of a real world cityscape
CN106327573A (zh) * 2016-08-25 2017-01-11 成都慧途科技有限公司 一种针对城市建筑的实景三维建模方法
CN107798731A (zh) * 2017-11-10 2018-03-13 泰瑞数创科技(北京)有限公司 一种基于卫星影像自动建模的方法
CN108053474A (zh) * 2017-12-29 2018-05-18 湖南城市学院 一种新型城市三维建模控制系统及方法
CN109269472A (zh) * 2018-08-08 2019-01-25 广州市城市规划勘测设计研究院 倾斜摄影测量建筑物特征线提取方法、装置与存储介质

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116681848A (zh) * 2023-05-26 2023-09-01 青岛市勘察测绘研究院 一种道路注记与实景三维模型的融合显示方法
CN116681848B (zh) * 2023-05-26 2024-02-27 青岛市勘察测绘研究院 一种道路注记与实景三维模型的融合显示方法
CN117078982A (zh) * 2023-10-16 2023-11-17 山东建筑大学 基于深度学习的大倾角立体像对准密集特征匹配方法
CN117078982B (zh) * 2023-10-16 2024-01-26 山东建筑大学 基于深度学习的大倾角立体像对准密集特征匹配方法
CN117437372A (zh) * 2023-12-20 2024-01-23 中国电子科技集团公司第十五研究所 塔台模拟用三维视景地形图处理方法、服务器及存储介质
CN117876506A (zh) * 2023-12-26 2024-04-12 北京建筑大学 多视影像的色彩处理方法及装置、存储介质、计算机设备
CN117664088A (zh) * 2024-01-31 2024-03-08 中国人民解放军战略支援部队航天工程大学 超宽幅垂轨环扫式卫星影像确定同名点方法、系统及设备
CN117664088B (zh) * 2024-01-31 2024-04-02 中国人民解放军战略支援部队航天工程大学 超宽幅垂轨环扫式卫星影像确定同名点方法、系统及设备

Also Published As

Publication number Publication date
CN115641401A (zh) 2023-01-24

Similar Documents

Publication Publication Date Title
WO2023280038A1 (zh) 一种三维实景模型的构建方法及相关装置
US10867430B2 (en) Method and system of 3D reconstruction with volume-based filtering for image processing
US11410320B2 (en) Image processing method, apparatus, and storage medium
CN108665536B (zh) 三维与实景数据可视化方法、装置与计算机可读存储介质
Zhang et al. A UAV-based panoramic oblique photogrammetry (POP) approach using spherical projection
US10297074B2 (en) Three-dimensional modeling from optical capture
US20190026400A1 (en) Three-dimensional modeling from point cloud data migration
WO2021103137A1 (zh) 室内场景光照估计模型、方法、装置、存储介质以及渲染方法
Wei et al. Applications of structure from motion: a survey
Pylvanainen et al. Automatic alignment and multi-view segmentation of street view data using 3d shape priors
JP2006053694A (ja) 空間シミュレータ、空間シミュレート方法、空間シミュレートプログラム、記録媒体
CN109387186A (zh) 测绘信息获取方法、装置、电子设备及存储介质
JP2015022510A (ja) 自由視点画像撮像装置およびその方法
CN114125310B (zh) 拍照方法、终端设备及云端服务器
JP2016537901A (ja) ライトフィールド処理方法
CN109523622A (zh) 一种非结构化的光场渲染方法
JP2019045991A (ja) 生成装置、生成方法、及びプログラム
US8509522B2 (en) Camera translation using rotation from device
CN109754463B (zh) 三维建模融合方法及装置
CN112270736A (zh) 增强现实处理方法及装置、存储介质和电子设备
CN115330594A (zh) 一种基于无人机倾斜摄影3d模型的目标快速识别与标定方法
WO2022040868A1 (zh) 全景拍摄方法、电子设备及存储介质
CN113129422A (zh) 一种三维模型构建方法、装置、存储介质和计算机设备
Bui et al. Integrating videos with LIDAR scans for virtual reality
CN114663599A (zh) 一种基于多视图的人体表面重建方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22836791

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE