WO2023083154A1 - 用于三维重建的方法、系统和存储介质 - Google Patents

用于三维重建的方法、系统和存储介质 Download PDF

Info

Publication number
WO2023083154A1
WO2023083154A1 PCT/CN2022/130480 CN2022130480W WO2023083154A1 WO 2023083154 A1 WO2023083154 A1 WO 2023083154A1 CN 2022130480 W CN2022130480 W CN 2022130480W WO 2023083154 A1 WO2023083154 A1 WO 2023083154A1
Authority
WO
WIPO (PCT)
Prior art keywords
plane mirror
mirror
camera
target object
information
Prior art date
Application number
PCT/CN2022/130480
Other languages
English (en)
French (fr)
Inventor
尚弘
李翔
施展
许宽宏
Original Assignee
索尼集团公司
尚弘
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 索尼集团公司, 尚弘 filed Critical 索尼集团公司
Priority to CN202280073308.1A priority Critical patent/CN118202392A/zh
Publication of WO2023083154A1 publication Critical patent/WO2023083154A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/514—Depth or shape recovery from specularities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Definitions

  • the present disclosure relates generally to three-dimensional reconstruction techniques, and in particular to deep neural network-based three-dimensional reconstruction techniques.
  • High-precision 3D reconstruction can play an important role in some occasions where plane vision is difficult or even impossible to solve, such as industrial automation, medical assistance applications, virtual reality applications, and visual navigation.
  • 3D reconstruction technology needs to obtain image information or depth information of the target object under multiple perspectives.
  • accuracy of 3D reconstruction is directly related to the density of angles. The sparser the angle, the lower the accuracy of the 3D reconstruction, or even impossible to model.
  • the method for three-dimensional reconstruction includes: obtaining a composite image obtained by photographing the body of the target object and its mirror image; Based on relative position and attitude information, 3D reconstruction of the target object is performed.
  • a system for three-dimensional reconstruction includes: an information processing device configured to execute the steps of the methods according to the embodiments of the present disclosure.
  • the plane mirror group includes a position and posture acquisition module disposed thereon, and the position and posture acquisition module is configured to obtain and transmit information related to the position and posture of the plane mirror group.
  • Yet another aspect of the present disclosure relates to an electronic device, including: a memory and a processor coupled to the memory, the processor configured to execute the method for three-dimensional reconstruction according to an embodiment of the present disclosure based on instructions stored in the memory.
  • Yet another aspect of the present disclosure relates to a non-transitory computer-readable storage medium having one or more instructions stored thereon, the instructions, when executed by a processor, cause the processor to perform a method for three-dimensional reconstruction according to an embodiment of the present disclosure. method.
  • Yet another aspect of the present disclosure relates to a computer program product comprising one or more instructions that, when executed by a processor, cause the processor to perform a method for three-dimensional reconstruction according to an embodiment of the present disclosure.
  • FIG. 1 is a schematic diagram showing an example of the configuration of a system for three-dimensional reconstruction according to an embodiment of the present disclosure.
  • FIG. 2A is a schematic diagram showing an example of the configuration of a plane mirror group and a plane mirror unit according to an embodiment of the present disclosure.
  • FIG. 2B shows a schematic diagram of an example of an arrangement of plane mirror groups according to an embodiment of the present disclosure.
  • FIG. 2C shows a schematic diagram of still another example of arrangement of plane mirror groups according to an embodiment of the present disclosure.
  • FIG. 3 is a flowchart illustrating an example of steps of a method for three-dimensional reconstruction according to an embodiment of the present disclosure.
  • Fig. 4 is a flow chart illustrating an example of sub-steps of some steps of the method for three-dimensional reconstruction according to an embodiment of the present disclosure.
  • FIG. 5A is a flowchart illustrating an example of a step of extracting global features according to an embodiment of the present disclosure.
  • FIG. 5B is a flowchart illustrating an example of steps of establishing a geometric association according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram illustrating an example of obtaining a composite image according to an embodiment of the present disclosure.
  • FIG. 7 is a block diagram illustrating an example of an electronic device according to an embodiment of the present disclosure.
  • FIG. 1 An example of the configuration of a system for three-dimensional reconstruction according to an embodiment of the present disclosure is exemplarily described below with reference to FIG. 1 .
  • the system 100 for three-dimensional reconstruction may include an information processing device 110 .
  • the information processing device 110 is used for three-dimensional reconstruction.
  • the information processing device 110 may be configured to perform at least a part of the steps of the method for three-dimensional reconstruction to be described later.
  • the system 100 for three-dimensional reconstruction may further include a plane mirror group 120 .
  • each plane mirror group 120 can provide a mirror surface.
  • the number L of plane mirror groups 120 in the system 100 can be selected according to needs. L is a positive integer.
  • the mirror surface of the plane mirror group 120 can generate a mirror image of the target object 140 .
  • the applicant realizes that, like the target object ontology, the mirror image of the target object in the mirror can also include feature information of the target object under a specific viewing angle. Therefore, the present disclosure proposes to use single or multiple cameras combined with the configuration of the plane mirror group to provide multi-view images required for 3D reconstruction of the target object, thereby reducing the requirement and dependence on the number of cameras.
  • the plane mirror group 120 may be formed by one or more plane mirror units.
  • multiple plane mirror units can be spliced together, placed adjacent to each other, or placed in a "Tian” character, a "Tick” character or a honeycomb-shaped cell, etc.
  • FIG. 2A schematically shows an example of a configuration of a plane mirror group according to an embodiment of the present disclosure, and a partially enlarged view thereof schematically shows an example of a configuration of plane mirror units constituting the plane mirror group.
  • each plane mirror group 120 may include one or more plane mirror units 210 .
  • the plane mirror unit 210 may include a plane mirror 214, and the plane mirrors in all the plane mirror units included in any plane mirror group may jointly constitute the mirror surface of the plane mirror group.
  • the plane mirror unit 210 can be spliced by a joint member 212 .
  • coupling members 212 may be disposed at four sides of the plane mirror unit 210, as shown in FIG. 2A.
  • the plane mirror unit 210 can be arbitrarily expanded in four directions as required, and the imaging of the plane mirror group is hardly affected.
  • Those skilled in the art should understand that there is no particular limitation on the arrangement position and joining method of the joining member 212, but can be selected according to the actual application, as long as the joining member 212 can splice the plane mirror units into a plane mirror group to provide a larger mirror surface And reduce the impact on mirror imaging.
  • using splicable plane mirror units to provide the plane mirror group can make the plane mirror group detachable, easy to expand, and flexibly adapt to different requirements.
  • At least a part of the following can be adjusted as required: the number of plane mirror groups, the position or attitude of at least part of the plane mirror groups relative to the camera.
  • the system 100 may further include a positioning device (not shown) for performing at least a part of the arrangement of the above-mentioned plane mirror group 120 and the arrangement of the camera 130 which will be described later.
  • the positioning device may be configured to adjust the position or attitude of one or more plane mirror groups relative to the camera.
  • the positioning means may be implemented using, for example, a robotic arm.
  • the positioning device may acquire information related to the arrangement from the information processing device 110, and control the arrangement of the plane mirror group and the camera based on the information.
  • the information related to the arrangement may be generated, for example, according to one or more of the following methods: designing for application scenarios, selecting according to experience, or presetting, and the like.
  • At least a portion of the arrangement of the mirror set and the camera may be performed manually by an operator.
  • the plane mirror group can also be realized by a whole plane mirror.
  • the plane mirror group 120 includes a position and posture acquisition module 220 disposed thereon.
  • the position and posture acquisition module 220 may be configured to obtain and send information related to the position and posture of the plane mirror group.
  • the position and posture acquisition module 220 may send information related to the position and posture of the plane mirror group to the information processing device 110 .
  • the information related to the position and posture of the plane mirror group may directly or indirectly indicate the position and posture of the plane mirror group.
  • the position and posture acquisition module 220 may include a sensing element (not shown) for sensing position and posture related parameters, so as to obtain position and posture related information of the plane mirror group.
  • the position and posture acquisition module 220 may also include a communication element (not shown) for communication, so as to transmit position and posture related information of the plane mirror group.
  • the position and posture acquisition module 220 can actively report the position and posture related information of the plane mirror group.
  • the position and posture acquiring module 220 may reply information related to the position and posture of the plane mirror group when receiving an inquiry.
  • the position and posture acquisition module 220 may be arranged at the edge or the back of the plane mirror group, so as to reduce the impact on mirror imaging.
  • the position and posture acquisition module 220 may be configured to use a specific pattern to calibrate the position and posture information of the plane mirror group.
  • the position and posture acquisition module 220 may be configured to provide a specific pattern including marking features on the plane mirror group, such as a barcode, a two-dimensional code or a checkerboard pattern. Once these specific patterns are photographed, the position and attitude of the plane mirror group relative to the camera can be identified, that is, relative position and attitude information.
  • the position and posture acquisition module 220 can be arranged at the edges (especially at the four corners) of each plane mirror unit constituting the plane mirror group, so as to form a suitable pattern and reduce the impact on mirror imaging.
  • the position and posture information of the plane mirror group 120 may be predetermined.
  • the system 100 for three-dimensional reconstruction may further include a camera 130 .
  • the camera 130 may be configured to photograph the body of the target object 130 and its mirror image, thereby obtaining a composite image.
  • the composite image described in this application refers to an image captured by the camera 130 that simultaneously includes the body of the target object and at least a part of its mirror image.
  • information from the camera 130 may be transmitted to the information processing device 110 .
  • the camera 130 may include a position and posture acquisition module (not shown). For example, combining the information sent by the respective position and posture acquisition modules of the camera 130 and the plane mirror group 120 , the position and posture of the plane mirror group 120 relative to the camera 130 can be determined, that is, relative position and posture information. Alternatively, camera 130 may be pre-calibrated.
  • the information processing device 110 may be deployed near the camera 130 (plane mirror group 120). Alternatively, in some embodiments, at least a part of the information processing device 110 may be deployed separately from the camera 130 . For example, in some embodiments, at least a part of the information processing apparatus 110 may be deployed at a remote server. Alternatively, in some embodiments, the information processing device 110 can also be integrated with the camera or the positioning device into the same device or module. Those skilled in the art should understand that there is no particular limitation on the positional relationship between the information processing device 110 and the camera 130 , but can be selected according to actual applications, as long as the information processing device 110 can obtain the information to be processed.
  • system 100 illustrated in FIG. 1 includes a plane mirror group 120 and a camera 130
  • the system 100 itself may not include the plane mirror group 120 and/or the camera 130, but may instead use a plane mirror external to the system.
  • group 120 and/or camera 130 may directly use composite images taken by cameras external to the system.
  • the method for three-dimensional reconstruction according to an embodiment of the present disclosure is exemplarily described below with reference to FIG. 3 , FIG. 4 , FIG. 5A-5B and FIG. 6 .
  • the method can be applied to any electronic device with processing capabilities.
  • the contents described above in conjunction with FIG. 1 , and FIG. 2A-2C are also applicable to corresponding features.
  • all or any part of the features of the plane mirror group described above may also be applicable to the method for three-dimensional reconstruction described below or the system for executing the method.
  • a method 300 for three-dimensional reconstruction may mainly include the following steps:
  • step 306 obtain a composite image obtained by photographing the target object body and its mirror image.
  • step 308 the target object is reconstructed three-dimensionally by using the composite image and the relative position and attitude information representing the position and attitude of the mirror surface that produces the mirror image relative to the camera that takes the picture.
  • the method for three-dimensional reconstruction may further include arranging the camera and at least a part of the plane mirror group providing the mirror, so that the shooting range of the camera covers the target object and the mirror.
  • “arrangement” may refer to “installation”, “replacement”, “adjustment” and the like.
  • the basic requirement of “arrangement” is to make the shooting range of the camera cover the target object and at least part of the mirror surface, so that the camera can capture the target object itself and at least part of its mirror image in the mirror surface.
  • arranging the cameras and at least a part of the plane mirror group may include sending an instruction, and controlling the arrangement of the plane mirror group and the camera based on information related to the arrangement included in the instruction.
  • the positioning device can be controlled to adjust the position or attitude of one or more plane mirror groups relative to the camera.
  • the positioning device can be realized by using, for example, a mechanical arm.
  • the information related to the arrangement may be generated according to one or more of the following methods: designed for the application scenario, selected according to experience, or preset.
  • an operator manually arranges at least some of the camera and plane mirror sets before performing the method for three-dimensional reconstruction.
  • arranging the camera and at least a part of the plane mirror groups may include adjusting at least one of the following as required: the number of the plane mirror groups, the position or attitude of at least part of the plane mirror groups relative to the camera, and the like. Examples of such adjustments have been described above in conjunction with FIGS. 2B-2C , and will not be repeated here. Properly arranging the camera and the plane mirror group according to application requirements and scene characteristics can advantageously achieve at least one of performance improvement and cost reduction.
  • the camera and plane mirror groups are pre-set at suitable positions, so no arrangement needs to be performed.
  • the camera and flat mirror may have been pre-installed and debugged.
  • the three-dimensional reconstruction of the target object can be directly performed without arranging a camera and a plane mirror group.
  • the method for 3D reconstruction may further include calibrating at least one of the camera and the mirror to obtain relative position and pose information indicating the position and pose of the mirror relative to the camera (step 304 ).
  • the position and attitude of the mirror relative to the camera can be determined in combination with the information sent by the respective position and attitude acquisition modules of the camera and the plane mirror group providing the mirror, that is, relative position and attitude information.
  • a specific pattern including marking features such as barcodes, QR codes, or checkerboards, may be arranged on the mirror set. By photographing these specific patterns with a camera, relative position and attitude information can be identified.
  • one of the camera and the mirror is pre-calibrated, and only the position and posture acquisition module on the other can be used to obtain relative position and posture information.
  • the relative positions and poses of the camera and the mirror are preset and known, so step 304 does not need to be performed.
  • step 304 does not need to be performed.
  • multiple 3D reconstructions can be performed without changing the arrangement of the two. In this case, there is no need to reacquire relative position and pose information in each 3D reconstruction.
  • step 306 a composite image obtained by photographing the body of the target object and its mirror image is obtained.
  • the mirror image of the target object is equivalent to the image of the target object "photographed" under one or even several specific viewing angles, that is, each mirror surface replaces one or more cameras (hereinafter referred to as is a mirror virtual camera).
  • a mirror virtual camera Through proper arrangement of the camera and the mirror, as shown in FIG. 6 , the target object body and its mirror image in the mirror can be photographed on the same image (referred to as a composite image herein).
  • the composite image can be used to obtain multi-view image information required for three-dimensional reconstruction of the target object.
  • the accuracy of 3D modeling is directly related to the number of cameras, so it is generally necessary to use multiple cameras to obtain multi-view images.
  • the composite image obtained by shooting the target body and its mirror image is used for three-dimensional modeling, which can reduce the requirement and dependence on the number of cameras, and realize high-precision processing at a lower cost.
  • step 308 three-dimensional reconstruction is performed using the compound image and relative position and attitude information representing the position and attitude of the mirror surface relative to the camera.
  • step 308 An example of sub-steps of performing three-dimensional reconstruction (step 308 ) according to an embodiment of the present disclosure will be described in detail below in conjunction with FIG. 4 .
  • points within a given three-dimensional space including a target object may be randomly sampled (step 402).
  • a given three-dimensional space including the target object may be appropriately restricted to improve sampling efficiency.
  • the number N of sampling points can be selected as required. Where N is a positive integer.
  • Fig. 5 illustrates in detail an example of specific steps for extracting global features corresponding to sampling points from a compound image according to an embodiment of the present disclosure.
  • step 502 feature extraction is performed on the composite image to obtain a global feature map.
  • the composite image may be input into a feature extractor for feature extraction.
  • the feature extractor may include but not limited to any one or combination of neural network, automatic codec, SIFT, HOG, etc.
  • a global feature map for the composite image can be obtained.
  • the global feature map may consist of feature elements.
  • Each feature element can be expressed in the form of a multidimensional vector.
  • the feature elements in the global feature map can correspond to the pixels on the image respectively.
  • the "correspondence" between a feature element and a pixel point means that the feature element can represent the feature of the corresponding pixel point.
  • performing global feature extraction on the image further includes performing preprocessing on the image such as downsampling before inputting the image to the feature extractor to reduce the resolution of the image. For example, in some embodiments, an image with a resolution of 512*512 may be compressed into an image with a resolution of 64*64 before the image is input to the feature extractor.
  • step 504 the geometric relationship between the sampling point and the pixel point of the composite image is determined.
  • the corresponding relationship between the world coordinate system and the image coordinate system can be determined.
  • the pixels corresponding to the sampling points in the composite image may not only include pixels obtained by photographing the body of the target object (hereinafter referred to as body pixels), but also include pixels obtained by photographing the mirror image of the target object (hereinafter referred to as mirror images). pixel). Therefore, establishing the geometric association between the sampling point and the pixel of the composite image may specifically include: determining the geometric association between the sampling point and the pixel of the body (substep 510), and determining the geometric association between the sampling point and the mirror pixel Geometric association (sub-step 520).
  • the geometric relationship between the sampling point and each pixel of the composite image can be determined by calculating the projection matrix of the body camera (real camera) and the projection matrix of the mirror virtual camera (virtual camera provided by the mirror).
  • the projection matrix of the body camera and the projection matrix of the mirror virtual camera can be calculated using relative position and attitude information known by the system or obtained through step 304 .
  • the position of camera C 0 can be set as the origin O of the world coordinate system.
  • T 1 2((T 0 ⁇ n)n+dn)
  • the attitude rotation matrix R 1 of the mirror virtual camera C 1 can be expressed as:
  • V is the mirror matrix
  • R0 is the pose rotation matrix of camera C0 .
  • R 0 can be expressed as:
  • the internal reference matrix I 0 of the camera C 0 is only related to the internal parameters of the camera, and can generally be obtained according to factory parameters or by calibrating the camera itself.
  • the internal reference matrix I 0 can be expressed as:
  • f x , f y represent the focal length related values of camera C 0
  • s x , s y represent the principal point offset related values of camera C 0 in imaging.
  • the internal parameter matrix I 1 of the mirror virtual camera C 1 can be expressed as:
  • the projection matrix P 0 of the camera C 0 can be expressed as:
  • the projection matrix P1 of the specular virtual camera C1 can be expressed as:
  • the above example sets the camera C 0 at the origin of the world coordinate system.
  • the coordinate position of the camera C 0 there is no particular limitation on the coordinate position of the camera C 0 . Also, if there are multiple real cameras, they can each be distributed at any location including the origin.
  • step 506 based on the geometric association, the global features corresponding to the sampling points are determined from the global feature map.
  • the feature elements in the global feature map may respectively correspond to the pixels on the composite image.
  • the geometric relationship between the sampling point and the pixel point of the composite image can be determined. Therefore, based on the geometric association, the corresponding relationship between the sampling point and the feature element can be determined.
  • the composite image may also include mirror pixels obtained by photographing the mirror image of the target object. Therefore, in some embodiments, the number of global features corresponding to each sampling point may be greater than 1, so that the total number of global features may be greater than the number of sampling points. Due to the limitation of viewing angle and possible stitching gaps, not every sampling point can be captured by each of the body camera and the mirror virtual camera, but the total number of global features can still be much larger than that of the target object captured only by the camera The condition of the body.
  • the imaging-related geometric information of the sampling points may be encoded to generate geometric encoding information (step 406 ).
  • the imaging-related geometric information of the sampling point may include at least a part of the spatial coordinates of the sampling point and the internal and external orientation information of the camera imaging the sampling point.
  • the camera includes not only the body camera, but also the specular virtual camera defined in this disclosure.
  • the imaging-related geometric information of the sampling point may only include the spatial coordinates of the sampling point.
  • the generated geometrically encoded information may only relate to the sample points themselves. Pixels corresponding to the same sampling point or the corresponding global features, whether obtained through the body camera or the specular virtual camera, will be associated with the same geometrically encoded information.
  • the imaging-related geometric information of the sampling point may include not only the spatial coordinates of the sampling point but also the internal and external orientation information of the camera.
  • the pixels or corresponding global features obtained by imaging the same sampling point from different viewing angles through the body camera or the specular virtual camera are associated with different geometric encoding information.
  • the generated geometry encoding information may be a multi-dimensional vector.
  • the geometric encoding information may include a multidimensional vector corresponding to the spatial coordinates of the sampling points and a multidimensional vector corresponding to the internal and external orientation information of the camera.
  • the inventors of the present application realized that the geometric coding information includes the above-mentioned multiple aspects of information, and can represent geometric features more accurately than intuitive geometric information. Therefore, using geometric coding information to represent geometric features is beneficial to improve the accuracy of 3D reconstruction.
  • the global features and corresponding geometric encoding information can be input into the model to determine the geometric relationship between the sampling point and the surface of the target object (step 408).
  • the global features corresponding to the sampling point and corresponding geometric encoding information can be input into the model.
  • the number of global features corresponding to each sampling point may be an integer greater than 1.
  • multiple global features corresponding to different viewing angles and corresponding geometric coding information may be input for one sampling point, which advantageously improves the accuracy of judgment.
  • the geometric coding information can include the internal and external orientation information of the camera to reflect the imaging angle of view, it will also be beneficial to further improve the accuracy of the judgment.
  • any model capable of judging the geometric relationship between the corresponding sampling point and the surface of the target object according to the above input can be used. Based on the global features for any sampling point and the corresponding geometric encoding information, the used model can output a judgment result indicating the geometric relationship between the sampling point and the surface of the target object.
  • the judgment result may be numerical.
  • the judgment result may be a numerical value indicating the probability that the sampling point is located inside/outside the surface of the target object. For example, when the determination result is 1, it may indicate that the sampling point is located within the surface of the target object. Relatively, when the determination result is 0, it may indicate that the sampling point is located outside the surface of the target object. vice versa. In other cases, the judgment result can be between 0 and 1.
  • the method for three-dimensional reconstruction may further include three-dimensionally reconstructing the voxel envelope of the target object based on the geometric relationship between the sampling points and the surface of the target object.
  • the three-dimensional reconstructed target voxel envelope can be obtained. For example, in the case where the judgment result is a numerical value indicating the probability that the sampling point is located inside/outside the surface of the target object, by extracting the 0.5 isosurface, the surface of the target object can be determined.
  • performing 3D reconstruction further includes training a model (step 410 ).
  • the model can be represented by an implicit function f that outputs the probability that the sampling point is located inside/outside the surface of the target object according to the above-mentioned input. This situation is described below as an example, but those skilled in the art can understand that the present disclosure is not limited thereto.
  • the discriminant error of the model can be calculated according to the judgment result of step 408 .
  • the judgment error can be obtained by comparing the judgment result output by the model with the real surface condition of the target object.
  • an implicit function f * may be used to describe the real target object surface.
  • the real target object surface may be the 0.5 isosurface of the function value of f * .
  • F Gi (X), Z i (X) (1 ⁇ i ⁇ m) respectively refer to the i-th global feature and the corresponding geometric coding information among one or more global features corresponding to the sampling point X, and m represents The number of global features corresponding to the sampling point X, m is a positive integer not greater than (L+1).
  • Z i (X) may be the same.
  • the discrimination error of the judgment result can be calculated by calculating the difference between the function value representing the implicit function f of the model and the function value of the implicit function f * representing the real target object surface.
  • the model is iteratively optimized by, for example, updating the parameters of the model, so that the discriminant error meets the accuracy requirement, and the training of the model is completed.
  • any suitable method can be used for iterative optimization of the model, including but not limited to gradient descent method, stochastic gradient descent method and the like.
  • the method for three-dimensional reconstruction according to the embodiments of the present disclosure can reduce the dependence on the number of cameras and enhance the adaptability to different application scenarios, and has low cost, high portability, scalability and flexible adjustment capabilities.
  • the application proposes Accurate and reliable information extraction and processing methods are established, and the model is trained and used to perform 3D reconstruction of the target object based on the processed information.
  • the present disclosure can realize high-precision three-dimensional reconstruction of a target object by using only a single or sparse camera.
  • the cost of three-dimensional modeling can be reduced and/or the accuracy of three-dimensional modeling can be improved.
  • Figure 7 shows a block diagram of some embodiments of an electronic device of the present disclosure.
  • the electronic device 7 of this embodiment includes: a memory 71 and a processor 72 coupled to the memory 71 , the processor 72 is configured to, based on the instructions stored in the memory 71 , execute the method for 3D reconstruction.
  • the memory 71 may include, for example, a system memory, a fixed non-volatile storage medium, and the like.
  • the system memory stores, for example, an operating system, an application program, a boot loader (Boot Loader), a database, and other programs.
  • Embodiments of the present disclosure also provide a non-transitory computer-readable storage medium, on which one or more instructions are stored, and when these instructions are executed by a processor, the processor can execute the method for 3D reconstruction methods.
  • the instructions in the computer-readable storage medium according to the embodiments of the present disclosure may be configured to perform operations corresponding to the above-mentioned system and method embodiments.
  • Embodiments of computer-readable storage media will be apparent to those skilled in the art when referring to the above-described system and method embodiments, and thus will not be described again.
  • Computer-readable storage media for carrying or including the above-described instructions also fall within the scope of the present disclosure.
  • Such computer-readable storage media may include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.
  • the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.
  • the above-described embodiments may be fully or partially implemented in the form of computer program products.
  • An embodiment of the present disclosure also provides a computer program product, including one or more computer instructions or computer programs, when these instructions or computer programs are executed by a processor, the processor can execute the three-dimensional method of reconstruction. Specifically, when computer instructions or computer programs are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. .
  • Embodiments of the present disclosure also provide various devices including components or units for performing the steps of the three-dimensional reconstruction method in the above embodiments.
  • each of the above components or units may be implemented as an independent physical entity, or may also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.).
  • a plurality of functions included in one unit in the above embodiments may be realized by separate devices.
  • a plurality of functions implemented by a plurality of units in the above embodiments may be respectively implemented by separate devices.
  • one of the above functions may be realized by a plurality of units.
  • a method for three-dimensional reconstruction comprising:
  • the target object is three-dimensionally reconstructed by using the composite image and the relative position and attitude information representing the position and attitude of the mirror surface that produces the mirror image relative to the camera that takes pictures.
  • the global features corresponding to the sampling points are determined from the global feature map.
  • establishing the geometric association between the sampling points and the pixels of the composite image comprises:
  • the geometric relationship between the sampling point and the mirror pixel obtained by shooting the mirror image of the target object in the composite image is determined.
  • At least one of the camera and the mirror is calibrated to obtain relative position and attitude information.
  • An electronic device comprising:
  • a processor coupled to the memory, the processor configured to execute the method for three-dimensional reconstruction according to any one of items 1-5 based on instructions stored in the memory.
  • a non-transitory computer-readable storage medium having stored thereon one or more instructions which, when executed by a processor, cause the processor to perform the method according to any one of items 1-5. 3D reconstruction methods.
  • a computer program product comprising one or more instructions which, when executed by a processor, cause the processor to perform the method for three-dimensional reconstruction according to any one of items 1-5.
  • a system for three-dimensional reconstruction comprising:
  • An information processing device configured to execute the steps of the method according to any one of items 1-5.
  • a flat mirror assembly of said mirror is provided.
  • the plane mirror group includes a position and posture acquisition module disposed thereon, and the position and posture acquisition module is configured to obtain and send information related to the position and posture of the plane mirror group.
  • a flat mirror assembly comprising one or more flat mirror units, wherein:
  • the plane mirror group includes a position and posture acquisition module disposed thereon, and the position and posture acquisition module is configured to obtain and send information related to the position and posture of the plane mirror group.
  • the set of plane mirrors is used to perform the method according to any one of items 1-5, or is included in the system according to any one of items 9-13.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本公开内容涉及用于三维重建的方法、系统和存储介质。描述了关于三维重建的各种实施例。在一个实施例中，用于三维重建的方法包括：获得通过拍摄目标对象本体及其镜像得到的复合图像；以及利用复合图像和表示产生镜像的镜面相对于进行拍摄的相机的位置和姿态的相对位置姿态信息，对所述目标对象进行三维重建。

Description

用于三维重建的方法、系统和存储介质
相关申请的交叉引用
本申请是以申请号为202111324566.6，申请日为2021年11月10日的中国申请为基础，并主张其优先权，该中国申请的公开内容在此作为整体引入本申请中。
技术领域
本公开一般地涉及三维重建技术，并且具体地涉及基于深度神经网络的三维重建技术。
背景技术
高精度的三维重建能够在诸如工业自动化、医疗辅助应用、虚拟现实应用、视觉导航等一些平面视觉难以解决甚至无法解决的场合起到重要的作用。
传统的高精度的三维重建技术需要获取目标对象在多视角下的图像信息或深度信息，通常情况下，三维重建的精度与角度的稠密程度直接相关。角度越稀疏，三维重建的精度越低，甚至无法建模。
发明内容
本公开的一个方面涉及用于三维重建的方法。根据本公开的实施例，用于三维重建的方法包括：获得通过拍摄目标对象本体及其镜像得到的复合图像；以及利用复合图像和表示产生镜像的镜面相对于进行拍摄的相机的位置和姿态的相对位置姿态信息，对目标对象进行三维重建。
本公开的一个方面涉及用于三维重建的系统。根据本公开的实施例，用于三维重建的系统包括：信息处理装置，被配置为执行根据本公开实施例的各方法的步骤。
本公开的一个方面涉及包括一个或多个平面镜单元的平面镜组。根据本公开的实施 例，平面镜组包括设置在上面的位置姿态获取模块，该位置姿态获取模块被配置为获得并发送与平面镜组的位置姿态相关的信息。
本公开的再一个方面涉及电子设备，包括：存储器和耦接至存储器的处理器，该处理器被配置为基于存储在存储器中的指令，执行根据本公开实施例的用于三维重建的方法。
本公开的再一个方面涉及非瞬态计算机可读存储介质，其上存储有一个或多个指令，该指令在由处理器执行时，使处理器执行根据本公开实施例的用于三维重建的方法。
本公开的再一个方面涉及计算机程序产品，包括一个或多个指令，该指令在由处理器执行时，使处理器执行根据本公开实施例的用于三维重建的方法。
提供上述概述是为了总结一些示例性的实施例，以提供对本文所描述的主题的各方面的基本理解。因此，上述特征仅仅是例子并且不应该被解释为以任何方式缩小本文所描述的主题的范围或精神。本文所描述的主题的其他特征、方面和优点将从以下结合附图描述的具体实施方式而变得明晰。
附图说明
当结合附图考虑实施例的以下具体描述时，可以获得对本公开内容更好的理解。在各附图中使用了相同或相似的附图标记来表示相同或者相似的部件。各附图连同下面的具体描述一起包含在本说明书中并形成说明书的一部分，用来例示说明本公开的实施例和解释本公开的原理和优点。其中：
图1是示出根据本公开实施例的用于三维重建的系统的配置的示例的示意图。
图2A是示出根据本公开实施例的平面镜组及平面镜单元的配置的示例的示意图。
图2B示出了根据本公开实施例的平面镜组的布置方式的示例的示意图。
图2C示出了根据本公开实施例的平面镜组的布置方式的又一示例的示意图。
图3是示出根据本公开实施例的用于三维重建的方法的步骤的示例的流程图。
图4是示出根据本公开实施例的用于三维重建的方法的部分步骤的子步骤的示例的流程图。
图5A是示出根据本公开实施例的提取全局特征的步骤的示例的流程图。
图5B是示出根据本公开实施例的建立几何关联的步骤的示例的流程图。
图6是示出根据本公开实施例的获得复合图像的示例的示意图。
图7是示出根据本公开实施例的电子设备的示例的框图。
虽然在本公开内容中所描述的实施例可能易于有各种修改和另选形式，但是其具体实施例在附图中作为例子示出并且在本文中被详细描述。但是，应当理解，附图以及对其的详细描述不是要将实施例限定到所公开的特定形式，而是相反，目的是要涵盖属于权利要求的精神和范围内的所有修改、等同和另选方案。
具体实施方式
以下描述根据本公开的设备和方法等各方面的代表性应用。这些例子的描述仅是为了增加上下文并帮助理解所描述的实施例。因此，对本领域技术人员而言明晰的是，以下所描述的实施例可以在没有具体细节当中的一些或全部的情况下被实施。在其他情况下，众所周知的过程步骤没有详细描述，以避免不必要地模糊所描述的实施例。其他应用也是可能的，本公开的方案并不限制于这些示例。
下面结合图1示例性地描述根据本公开实施例的用于三维重建的系统的配置的示例。
根据本公开的实施例，用于三维重建的系统100可以包括信息处理装置110。
信息处理装置110用于进行三维重建。特别地，信息处理装置110可以被配置为执行稍后将描述的用于三维重建的方法的步骤中的至少一部分。
在一些实施例中，用于三维重建的系统100还可以包括平面镜组120。具体地，每个平面镜组120可以提供一个镜面。系统100中的平面镜组120的数量L可以根据需要进行选择。L为正整数。
根据镜面成像原理，平面镜组120的镜面可以产生目标对象140的镜像。申请人认识到，与目标对象本体一样，目标对象在镜面中的镜像也可以包括目标对象在特定视角下的特征信息。因此，本公开提出利用单个或多个相机结合平面镜组的配置来提供三维重建目标对象所需的多视角图像，从而减少对相机数量的要求和依赖。
在一些实施例中，平面镜组120可以由一个或多个平面镜单元形成。例如，可以将多个平面镜单元拼接在一起，将多个平面镜单元邻近放置在一起，或者将多个平面镜单元放置在例如“田”字、“井”字或蜂窝状的单元格中等等。
图2A示意性地示出了根据本公开实施例的平面镜组的配置的示例，其中的局部放大图中示意性地示出了构成平面镜组的平面镜单元的配置的示例。
如图所示，在一些实施例中，每个平面镜组120可以包括一个或多个平面镜单元210。平面镜单元210可以包括平面镜214，任一平面镜组所包括的所有平面镜单元中的平面镜可以共同构成该平面镜组的镜面。
如图所示，在一些实施例中，平面镜单元210可以通过接合构件212进行拼接。例如，根据一些实施例的接合构件212可以设置在平面镜单元210的四个侧边处，如图2A中所示。利用这种布置，平面镜单元210可以根据需要在四个方向上任意扩展，并且几乎不影响平面镜组的成像。本领域的技术人员应当理解，对于接合构件212的布置位置和接合方式没有特别的限制，而是可以根据实际应用进行选择，只要接合构件212能够将平面镜单元拼接成平面镜组以提供更大的镜面并减少对镜面成像的影响即可。
有利地，使用可拼接的平面镜单元来提供平面镜组能够使平面镜组可拆装、易扩展，灵活适应于不同的需求。
特别地，在一些实施例中，以下中的至少一部分可以根据需要调整：平面镜组的数量、至少部分平面镜组相对于相机的位置或姿态。
下面结合图2B和图2C分别示出的根据本公开实施例的平面镜组的布置方式的示例来说明平面镜组的布置方式的调整。在图2B所示的布置方式的示例中，使用了两个平面镜组，它们以两种不同的位置姿态对目标对象进行镜像成像，其中每个平面镜组由四个平面镜单元拼接而成，平面镜组的尺寸较大。经过调整，在图2C所示的布置方式的示例中，使用了四个平面镜组，它们以四种不同的位置姿态对目标对象进行镜像成像，其中每个平面镜组只包括一个平面镜单元，平面镜组的尺寸较小。本领域的技术人员应当理解，对于平面镜组的布置方式没有特别的限制，而是可以根据实际应用进行设计和调整。
在一些实施例中，系统100还可以包括定位装置(未示出)，用于执行上述平面镜组 120的布置和稍后将描述的相机130的布置中的至少一部分。例如，定位装置可以被配置为调整一个或多个平面镜组相对于相机的位置或姿态。在一些实施例中，定位装置可以使用例如机械臂实现。
在一些实施例中，定位装置可以从信息处理装置110获取与布置相关的信息，并基于该信息控制平面镜组和相机的布置。其中，与布置相关的信息可以例如根据以下方式中的一个或多个产生：针对应用场景设计、根据经验选择或预先设定等。
可替代地，在一些实施例中，平面镜组和相机的布置中的至少一部分可以由操作者手动进行。
本领域的技术人员应当理解，虽然附图例示了由一个或多个平面镜单元拼接形成平面镜组的情况，但本申请不限于此。例如，在一些实施例中，平面镜组也可以用整块的平面镜来实现。
在本公开的一些实施例中，平面镜组120包括设置在上面的位置姿态获取模块220。
在一些实施例中，位置姿态获取模块220可以被配置为获得并发送与平面镜组的位置姿态相关的信息。例如，位置姿态获取模块220可以将与平面镜组的位置姿态相关的信息发送给信息处理装置110。其中，与平面镜组的位置姿态相关的信息(平面镜组的位置姿态相关信息)可以直接或者间接地指示平面镜组的位置姿态。在一些实施例中，位置姿态获取模块220可以包括用于感测位置和姿态相关参数的感测元件(未示出)，从而获得平面镜组的位置姿态相关信息。位置姿态获取模块220还可以包括用于通信的通信元件(未示出)，从而发送平面镜组的位置姿态相关信息。位置姿态获取模块220可以主动报告平面镜组的位置姿态相关信息。或者，位置姿态获取模块220可以在收到询问时回复平面镜组的位置姿态相关信息。
在这些实施例中，位置姿态获取模块220可以设置在平面镜组的边缘处或者背面，以减少对镜面成像的影响。
在另一些实施例中，位置姿态获取模块220可以被配置为利用特定图案标定平面镜组的位置姿态信息。例如，位置姿态获取模块220可以被配置为在平面镜组上提供包含标记特征的特定图案，诸如条形码、二维码或棋盘格。一旦拍摄到这些特定图案，就可以识别出平面镜组相对于相机的位置和姿态，即，相对位置姿态信息。
在这些实施例中，位置姿态获取模块220可以设置在构成平面镜组的各个平面镜单元的边缘处(特别是四个角处)，以构成合适的图案并减少对镜面成像的影响。
可替代地，在一些实施例中，平面镜组120的位置姿态信息可以被预先确定。
在一些实施例中，用于三维重建的系统100还可以包括相机130。相机130可以被配置为拍摄目标对象130的本体及其镜像，从而获得复合图像。本申请所述的复合图像指的是由相机130捕捉的、同时包含目标对象的本体和至少一部分镜像的图像。如图1所示，来自相机130的信息可以被传送给信息处理装置110。
相机130可以包括位置姿态获取模块(未示出)。例如，结合相机130和平面镜组120各自的位置姿态获取模块所发送的信息，可以确定平面镜组120相对于相机130的位置和姿态，即，相对位置姿态信息。或者，相机130可以被预先标定。
在一些实施例中，信息处理装置110可以部署在相机130(平面镜组120)附近。可替代地，在一些实施例中，信息处理装置110中的至少一部分可以与相机130分开部署。例如，在一些实施例中，信息处理装置110中的至少一部分可以部署在远程的服务器处。可替代地，在一些实施例中，信息处理装置110还可以与相机或定位装置集成为同一设备或模块。本领域的技术人员应当理解，对于信息处理装置110与相机130的位置关系没有特别的限制，而是可以根据实际应用进行选择，只要信息处理装置110能够获取待处理的信息即可。
虽然图1中例示的系统100包括平面镜组120和相机130，本领域的技术人员应当理解，系统100本身可以不包括平面镜组120和/或相机130，而是可以代替地使用由系统外部的平面镜组120和/或相机130。例如，系统100可以直接使用由系统外部的相机拍摄的复合图像。
下面参考图3、图4、图5A-图5B以及图6来示例性地描述根据本公开实施例的用于三维重建的方法。该方法可应用于任何具有处理能力的电子设备中。上面结合图1、图2A-图2C所描述的内容也可以适用于对应的特征。例如，以上描述的平面镜组的全部或任意部分特征也可以适用于下面描述的用于三维重建的方法或执行该方法的系统。
如图3所示，根据本公开的实施例，用于三维重建的方法300可以主要包括以下步骤：
在步骤306，获得通过拍摄目标对象本体及其镜像得到的复合图像；以及
在步骤308，利用复合图像和表示产生镜像的镜面相对于进行拍摄的相机的位置和姿态的相对位置姿态信息，对目标对象进行三维重建。
在一些实施例中，用于三维重建的方法还可以包括布置相机和提供镜面的平面镜组中的至少一部分，使得相机的拍摄范围覆盖目标对象以及镜面。这里，“布置”可以指代“安装”、“替换”、“调整”等。“布置”的基本要求是使得相机的拍摄范围覆盖目标对象以及至少部分镜面，从而使得相机能够拍摄到目标对象本体及其在镜面中的至少部分镜像。
在一些实施例中，布置相机和平面镜组中的至少一部分可以包括发送指令，并基于指令中包括的与布置相关的信息来控制平面镜组和相机的布置。例如，可以控制定位装置来调整一个或多个平面镜组相对于相机的位置或姿态。其中，定位装置可以利用例如机械臂实现。
在一些实施例中，与布置相关的信息可以根据以下方式中的一个或多个产生：针对应用场景设计、根据经验选择或预先设定等。
可替代地，在一些实施例中，在执行用于三维重建的方法之前，由操作者手动布置相机和平面镜组中的至少一部分。
在一些实施例中，布置相机和平面镜组中的至少一部分可以包括根据需要调整以下中的至少一项：平面镜组的数量、至少部分平面镜组相对于相机的位置或姿态等。上面已经结合图2B-图2C地描述了这种调整的示例，这里不再重复。根据应用需求和场景特点相应地布置相机和平面镜组可以有利地实现提高性能和降低成本中的至少一者。
可替代地，在一些实施例中，相机和平面镜组被预先设置在合适的位置处，因此不需要执行布置。例如，在某个应用场景下，相机和平面镜组可能已经被预先安装调试好。在这种情况下，无需布置相机和平面镜组就可以直接执行对目标对象的三维重建。
在一些实施例中，用于三维重建的方法还可以包括对相机和镜面中的至少一个进行标定，以获取指示镜面相对于相机的位置和姿态的相对位置姿态信息(步骤304)。
例如，在一些实施例中，可以结合相机和提供镜面的平面镜组各自的位置姿态获取模块所发送的信息，确定镜面相对于相机的位置和姿态，即，相对位置姿态信息。或者，在一些实施例中，可以在平面镜组上布置包含标记特征的特定图案，诸如条形码、二维码或棋盘格。通过用相机拍摄这些特定图案，可以识别出相对位置姿态信息。又或者， 在一些实施例中，相机和镜面中的一者被预先标定，仅使用另一者上的位置姿态获取模块，就可以获取相对位置姿态信息。
可替代地，在一些实施例中，相机和镜面的相对位置和姿态被预先设定并且已知，因此不需要执行步骤304。例如，在某个应用场景下，一旦安装好相机和镜面，可以在不改变二者的布置方式的前提下执行多次三维重建。在这种情况下，无需在每次三维重建中重新获取相对位置姿态信息。
但是，如果进行布置改变了相机和镜面的相对位置和姿态，则需要重新进行标定。
在步骤306中，获得通过拍摄目标对象本体及其镜像得到的复合图像。
在现有技术中，为了对目标对象进行3D重建，需要多个相机从多个视角捕捉图像。如上所述，目标对象的镜像相当于在一个或甚至几个特定视角下“拍摄”的目标对象的像，即，相当于每个镜面代替了现有技术的一个或者多个相机(下文中称为镜面虚拟相机)。通过相机和镜面的适当布置，如图6所示，目标对象本体及其在镜面中的镜像可以被拍摄到同一张图像(本文中称为复合图像)上。由此，利用复合图像就可以获得三维重建目标对象所需的多视角图像信息。
在相关的三维重建技术中，三维建模的精度和相机数量直接相关，因此一般需要使用多个相机来获得多视角图像。本申请利用拍摄目标对象本体及其镜像得到的复合图像进行三维建模，可以减少对相机数量的要求和依赖，用较低的成本实现高精度的处理。
在步骤308，利用复合图像和表示镜面相对于相机的位置和姿态的相对位置姿态信息，进行三维重建。
下面结合图4详细描述根据本公开实施例的进行三维重建(步骤308)的子步骤的示例。
如图4所例示的，可以对包括目标对象的给定三维空间内的点进行随机采样(步骤402)。
在一些实施例，包括目标对象的给定三维空间可以被适当地限制，以提高采样的效率。另外，采样点的数量N可以根据需要进行选择。其中N为正整数。
随后，可以从复合图像中提取与采样点对应的全局特征(步骤404)。
图5详细例示了根据本公开实施例从复合图像中提取与采样点对应的全局特征的具 体步骤的示例。
如图5所例示的，在步骤502，对复合图像进行特征提取，获得全局特征图。
具体地，在一些实施例中，可以将复合图像输入到特征提取器中进行特征提取。
在一些实施例中，特征提取器可以包括但不限于神经网络、自动编解码、SIFT、HOG等中的任意一个或组合。
作为特征提取器的输出，可以获得针对复合图像的全局特征图。
在一些实施例中，全局特征图可以由特征元素组成。各个特征元素可以采取多维向量的形式表示。全局特征图中的特征元素可以分别与图像上的像素点对应。这里，特征元素与像素点的“对应”指该特征元素可以表示对应像素点的特征。本领域技术人员容易理解，图像的分辨率越高或像素点越小，提取得到的全局特征图就越能准确地表示图像，但相应的工作量就越大。
在一些实施例中，为了避免巨大的计算开销，对图像进行全局特征提取还包括在将图像输入到特征提取器之前对图像进行诸如降采样之类的预处理，以降低图像的分辨率。例如，在一些实施例中，在将图像输入到特征提取器之前，可以将分辨率为512*512的图像压缩为分辨率为64*64的图像。
在步骤504，确定采样点与复合图像的像素点之间的几何关联。由此，可以确定世界坐标系与图像坐标系的对应关系。
复合图像中与采样点对应的像素点可以不仅包括通过拍摄目标对象本体得到的像素点(下文中称为本体像素点)，还包括通过拍摄目标对象的镜像得到的像素点(下文中称为镜像像素点)。因此，建立采样点与复合图像的像素点之间的几何关联可以具体地包括：确定采样点与本体像素点之间的几何关联(子步骤510)，以及确定采样点与镜像像素点之间的几何关联(子步骤520)。
在一些实施例中，可以通过计算本体相机(真实相机)的投影矩阵以及镜面虚拟相机(镜面提供的虚拟相机)的投影矩阵来确定采样点与复合图像的各个像素点之间的几何关联。本体相机投影矩阵以及镜面虚拟相机投影矩阵可以利用系统已知的或者通过步骤304获得的相对位置姿态信息来计算。
下面结合图5B，解释投影矩阵的确定方法的示例。
为了便于计算和描述，可以将相机C 0的位置设定为世界坐标系的原点O。利用相对位置姿态信息，可以获得平面镜组W 1的法线向量n＝(n x,n y,n z)以及平面镜组W 1所在平面到相机C 0的距离d。
相机C 0关于平面镜组W 1的镜像的位置可以被表示为：
[式1]
T 1＝2((T 0·n)n+dn)
其中，T 0表示相机C 0在世界坐标系中的位置。由于相机C 0被假设位于世界坐标系的原点O，上式可以被进一步简化为T 1＝2dn。结合以上分析可知，该镜像的位置T 1就是本公开中定义的镜面虚拟相机C 1的位置。
镜面虚拟相机C 1的姿态旋转矩阵R 1可以被表示为：
[式2]
Figure PCTCN2022130480-appb-000001
其中，V为镜像矩阵，R 0为相机C 0的姿态旋转矩阵。在本示例中，R 0可以被表示为：
[式3]
Figure PCTCN2022130480-appb-000002
基于在世界坐标系中的位置和姿态旋转矩阵，相机C 0的外参矩阵M 0可以被表示为：M 0＝[R 0 T 0]，类似地，镜面虚拟相机C 1的外参矩阵M 1可以被表示为：M 1＝[R 1 T 1]。
相机C 0的内参矩阵I 0仅与相机内部参数有关，一般可以根据出厂参数或标定相机本身来获得。一般来说，内参矩阵I 0可以被表示为：
[式4]
Figure PCTCN2022130480-appb-000003
其中，f x、f y表示相机C 0的焦距相关值，s x、s y表示相机C 0在成像中的主点偏移相关值。
因此，根据镜面成像原理，镜面虚拟相机C 1的内参矩阵I 1可以被表示为：
[式5]
Figure PCTCN2022130480-appb-000004
利用内参矩阵和外参矩阵，相机C 0的投影矩阵P 0可以被表示为：
[式6]
P 0＝M 0·I 0＝[R 0 T 0]·I 0
类似地，镜面虚拟相机C 1的投影矩阵P 1可以被表示为：
[式7]
P 1＝M 1·I 1＝[R 1 T 1]·I 1
为了简化描述，以上示例将相机C 0设定在了世界坐标系中的原点位置。本领域的技术人员应当理解，对于相机C 0的坐标位置没有特别的限制。而且，如果有多个真实相机，它们可以分别分布在包括原点在内的任何位置处。
虽然上述式1-式7仅示例性地描述了针对一个平面镜组W 1的投影矩阵的计算方法，但类似的计算方法也适用于在包括多个平面镜组的情况下其他平面镜组(诸如W 2)的投影矩阵的计算。
此外，虽然本文详细描述了使用投影矩阵来确定采样点与复合图像的像素点之间的几何关联的具体示例，但本公开不限于此。
在步骤506，基于几何关联，从全局特征图中确定与采样点对应的全局特征。
如上所述，全局特征图中的特征元素可以分别与复合图像上的像素点对应。此外，根据步骤504中的处理，可以确定采样点与复合图像的像素点之间的几何关联。由此， 基于该几何关联，可以确定采样点与特征元素的对应关系。
应注意，除了通过拍摄目标对象本体得到的本体像素点，复合图像还可以包括通过拍摄目标对象的镜像得到的镜像像素点。因此，在一些实施例中，与每个采样点对应的全局特征的数量可以大于1，使得全局特征的总数量可以大于采样点的数量。受视角的限制以及可能的拼接缝隙影响，不一定每个采样点都能被本体相机和镜面虚拟相机中的每一个拍摄到，但是全局特征的总数量依然可以远远大于仅用相机拍摄目标对象本体的情况。
如图4所例示的，可以对采样点的成像相关几何信息进行编码处理，生成几何编码信息(步骤406)。
在一些实施例中，采样点的成像相关几何信息可以包括采样点的空间坐标以及对采样点成像的相机的内外方位信息中的至少一部分。其中，相机不仅包括本体相机，还包括本公开定义的镜面虚拟相机。
例如，在一些实施例中，采样点的成像相关几何信息可以仅包括采样点的空间坐标。在这些实施例中，所生成的几何编码信息可以仅与采样点本身相关。对应于同一采样点的像素点或相应的全局特征，无论是通过本体相机还是镜面虚拟相机得到的，都将与同一几何编码信息相关联。
例如，在另一些实施例中，采样点的成像相关几何信息可以不仅包括采样点的空间坐标还包括相机的内外方位信息。在这些实施例中，通过本体相机或镜面虚拟相机从不同视角对同一采样点成像得到的像素点或相应的全局特征都与各自不同的几何编码信息相关联。
在一些实施例中，所生成的几何编码信息可以为多维向量。例如，作为示例，几何编码信息可以包括分别与采样点的空间坐标对应的多维向量和与相机的内外方位信息对应的多维向量。
本申请的发明人认识到，几何编码信息包含了例如上述多方面的信息，能够相对于直观的几何信息更准确地表示几何特征。由此，利用几何编码信息来表示几何特征有利于提高三维重建的准确度。
如图4所例示的，可以将全局特征和相应的几何编码信息输入模型，判断采样点与 目标对象表面的几何关系(步骤408)。
具体地，可以针对每个采样点，将与采样点对应的全局特征和相应的几何编码信息输入模型。
如以上所分析的，使用镜面能够与使用附加相机类似地实现增加视角的效果。由此，在引入镜面以提供镜面虚拟相机的情况下，与每个采样点对应的全局特征的数量可以是大于1的整数。在一些实施例中，可以针对一个采样点输入对应于不同视角的多个全局特征以及相应的几何编码信息，有利地提高了判断的准确度。
另外，如果几何编码信息能够包括相机的内外方位信息以反映成像视角，也将有利地进一步提高判断的准确度。
本领域技术人员能够理解，可以使用能够根据上述输入来判断相应采样点与目标对象表面的几何关系的任意模型。基于针对任一采样点的全局特征以及相应的几何编码信息，所使用的模型可以输出指示该采样点与目标对象表面的几何关系的判断结果。
在一些实施例中，判断结果可以是数值的。例如，在一些实施例中，判断结果可以是指示该采样点位于目标对象表面内/外的概率的数值。举例来说，当判断结果为1时，可以指示该采样点位于目标对象表面内。相对地，当判断结果为0时，可以指示该采样点位于目标对象表面外。反之亦然。在其它情况下，判断结果可以在0和1之间。
在一些实施例中，用于三维重建的方法还可以包括基于采样点与目标对象表面的几何关系，三维重建目标对象的体素包络。
在一些实施例中，通过判断所有采样点与目标对象表面的几何关系，可以获得三维重建的目标体素包络。例如，在判断结果是指示该采样点位于目标对象表面内/外的概率的数值的情况下，通过提取0.5等值面，就可以确定目标对象表面。
如图4所例示的，在一些实施例，进行三维重建还包括训练模型(步骤410)。
在一些实施例中，模型可以使用根据上述输入而输出采样点位于目标对象表面内/外的概率的隐函数f来表示。下面以这种情况为例进行描述，但本领域技术人员能够理解，本公开不限于此。
首先，可以根据步骤408的判断结果，计算模型的判别误差。
在一些实施例中，通过将模型输出的判断结果与真实的目标对象表面情况进行比较，可以获得判别误差。
例如，在一些实施例中，可以使用隐函数f *来描述真实的目标对象表面。
[式8]
Figure PCTCN2022130480-appb-000005
即，如果点X在目标对象表面内部，f *的函数值为1，如果在外部，f *的函数值为0。真实的目标对象表面可以为f *的函数值的0.5等值面。
类似地，描述模型的隐函数f可以如下表示。
[式9]
f(X)＝(F G1(X),…,F Gm(X),Z 1(X),…,Z m(X))
其中，F Gi(X)、Z i(X)(1≤i≤m)分别指与采样点X对应的一个或多个全局特征中的第i个全局特征和相应的几何编码信息，m表示与采样点X对应的全局特征的数量，m为不大于(L+1)的正整数。如上所述，当几何编码信息所指示的采样点的成像相关几何信息仅包括采样点的空间坐标时，Z i(X)可以是相同的。
由此，在一些实施例中，可以通过计算表示模型的隐函数f的函数值与表示真实的目标对象表面的隐函数f *的函数值的差值，计算判断结果的判别误差。
随后，通过例如更新模型的参数来迭代地优化模型，使得判别误差满足精度要求，完成对模型的训练。
本领域技术人员应该理解，可以使用任意合适的方法来进行模型的迭代优化，包括但不限于梯度下降法、随机梯度下降法等。
根据本公开实施例的用于三维重建的方法能够减少对相机数量的依赖并增强对不同应用场景的适应能力，具备低成本、高便携性、可扩展性和灵活调整能力，此外，本申请提出了准确可靠的信息提取和处理方法，训练并使用模型根据经处理的信息对目标对象进行三维重建。在此基础上，本公开能够仅利用单个或稀疏的相机就实现对目标对象的高精度三维重建。能够降低三维建模的成本和/或提高三维建模的准确度。
值得注意的是，在以上描述的方法中的各个步骤之间的边界仅仅是说明性的。在实际操作中，各个步骤之间可以任意组合，甚至合成单个步骤。此外，各个步骤的执行顺序不受描述顺序的限制，并且部分步骤可以省略。各个实施例的操作步骤也可以以任何适当的顺序相互组合，从而类似地实现比所描述的更多或更少的操作。
图7示出本公开的电子设备的一些实施例的框图。
如图7所示，该实施例的电子设备7包括：存储器71以及耦接至该存储器71的处理器72，处理器72被配置为基于存储在存储器71中的指令，执行根据本公开实施例的用于三维重建的方法。
存储器71例如可以包括系统存储器、固定非易失性存储介质等。系统存储器例如存储有操作系统、应用程序、引导装载程序(Boot Loader)、数据库以及其他程序等。
本公开实施例还提供了一种非瞬态计算机可读存储介质，其上存储有一个或多个指令，这些指令在由处理器执行时，可以使处理器执行根据本公开实施例的用于三维重建的方法。
具体地，根据本公开实施例的计算机可读存储介质中的指令可以被配置为执行与上述系统和方法实施例相应的操作。当参考上述系统和方法实施例时，计算机可读存储介质的实施例对于本领域技术人员而言是明晰的，因此不再重复描述。用于承载或包括上述指令的计算机可读存储介质也落在本公开的范围内。这样的计算机可读存储介质可以包括但不限于软盘、光盘、磁光盘、存储卡、存储棒等等。
本领域内的技术人员应当明白，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。在使用软件实现时，上述实施例可以全部或部分地以计算机程序产品的形式实现。本公开实施例还提供了一种计算机程序产品，包括一个或多个计算机指令或计算机程序，这些指令或计算机程序在由处理器执行时，可以使处理器执行根据本公开实施例的用于三维重建的方法。具体地，在计算机上加载或执行计算机指令或计算机程序时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用非瞬时性存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本公开实施例还提供了包括用于执行上述实施例中的三维重建方法的步骤的部件或单元的各种装置。
应注意，上述各个部件或单元仅是根据其所实现的具体功能划分的逻辑模块，而不是用于限制具体的实现方式，例如可以以软件、硬件或者软硬件结合的方式来实现。在实际实现时，上述各个部件或单元可被实现为独立的物理实体，或者也可由单个实体(例如，处理器(CPU或DSP等)、集成电路等)来实现。例如，在以上实施例中包括在一个单元中的多个功能可以由分开的装置来实现。替选地，在以上实施例中由多个单元实现的多个功能可分别由分开的装置来实现。另外，以上功能之一可由多个单元来实现。
以上参照附图描述了本公开的示例性实施例，但是本公开当然不限于以上示例。本领域技术人员可在所附权利要求的范围内得到各种变更和修改，并且应理解这些变更和修改自然将落入本公开的技术范围内。
虽然已经详细说明了本公开及其优点，但是应当理解在不脱离由所附的权利要求所限定的本公开的精神和范围的情况下可以进行各种改变、替代和变换。而且，本公开实施例的术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本公开的实施例还包括：
1.一种用于三维重建的方法，所述方法包括：
获得通过拍摄目标对象本体及其镜像得到的复合图像；以及
利用复合图像和表示产生镜像的镜面相对于进行拍摄的相机的位置和姿态的相对位置姿态信息，对所述目标对象进行三维重建。
2.根据项目1所述的方法，其中，进行三维重建包括：
对包括目标对象的给定三维空间内的点进行随机采样；
从复合图像中提取与采样点对应的全局特征；
对采样点的成像相关几何信息进行编码处理，生成几何编码信息；以及
将全局特征和几何编码信息输入模型，判断采样点与目标对象表面的几何关系。
3.根据项目2所述的方法，其中，从复合图像中提取与采样点对应的全局特征包括：
对复合图像进行特征提取，获得全局特征图；
确定采样点与复合图像的像素点之间的几何关联；以及
基于几何关联，从全局特征图中确定与采样点对应的全局特征。
4.根据项目3述的方法，其中，建立采样点与复合图像的像素点之间的几何关联包括：
确定采样点与复合图像中通过拍摄目标对象本体得到的本体像素点之间的几何关联；以及
根据相对位置姿态信息，确定采样点与复合图像中通过拍摄目标对象的镜像得到的镜像像素点之间的几何关联。
5.根据项目1所述的方法，还包括：
对相机和镜面中的至少一者进行标定，以获取相对位置姿态信息。
6.一种电子设备，包括：
存储器；和
耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器中的指令，执行根据项目1-5任一项所述的用于三维重建的方法。
7.一种非瞬态计算机可读存储介质，其上存储有一个或多个指令，所述指令在由处理器执行时，使处理器执行根据项目1-5任一项所述的用于三维重建的方法。
8.一种计算机程序产品，包括一个或多个指令，所述指令在由处理器执行时，使处理器执行根据项目1-5任一项所述的用于三维重建的方法。
9.一种用于三维重建的系统，包括：
信息处理装置，被配置为执行根据项目1-5任一项所述方法的步骤。
10.根据项目9所述的系统，还包括：
提供所述镜面的平面镜组。
11.根据项目10所述的系统，其中，平面镜组由平面镜单元形成。
12.根据项目10所述的系统，其中，以下中的至少一项可以根据需要进行调整：平面镜组的数量、至少部分平面镜组相对于相机的位置或姿态。
13.根据项目10所述的系统，其中，平面镜组包括设置在上面的位置姿态获取模块， 所述位置姿态获取模块被配置为获得并发送与平面镜组的位置姿态相关的信息。
14.一种平面镜组，包括一个或多个平面镜单元，其中：
所述平面镜组包括设置在上面的位置姿态获取模块，所述位置姿态获取模块被配置为获得并发送与平面镜组的位置姿态相关的信息。
15.根据项目14所述的平面镜组，其中：
所述平面镜组被用于执行根据项目1-5中任一项所述的方法，或被包括在根据项目9-13中任一项所述的系统中。

Claims (15)

  1. 一种用于三维重建的方法，所述方法包括：
    获得通过拍摄目标对象本体及其镜像得到的复合图像；以及
    利用复合图像和表示产生镜像的镜面相对于进行拍摄的相机的位置和姿态的相对位置姿态信息，对所述目标对象进行三维重建。
  2. 根据权利要求1所述的方法，其中，进行三维重建包括：
    对包括目标对象的给定三维空间内的点进行随机采样；
    从复合图像中提取与采样点对应的全局特征；
    对采样点的成像相关几何信息进行编码处理，生成几何编码信息；以及
    将全局特征和几何编码信息输入模型，判断采样点与目标对象表面的几何关系。
  3. 根据权利要求2所述的方法，其中，从复合图像中提取与采样点对应的全局特征包括：
    对复合图像进行特征提取，获得全局特征图；
    确定采样点与复合图像的像素点之间的几何关联；以及
    基于几何关联，从全局特征图中确定与采样点对应的全局特征。
  4. 根据权利要求3述的方法，其中，建立采样点与复合图像的像素点之间的几何关联包括：
    确定采样点与复合图像中通过拍摄目标对象本体得到的本体像素点之间的几何关联；以及
    根据相对位置姿态信息，确定采样点与复合图像中通过拍摄目标对象的镜像得到的镜像像素点之间的几何关联。
  5. 根据权利要求1所述的方法，还包括：
    对相机和镜面中的至少一者进行标定，以获取相对位置姿态信息。
  6. 一种电子设备，包括：
    存储器；和
    耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器中的指令，执行根据权利要求1-5任一项所述的用于三维重建的方法。
  7. 一种非瞬态计算机可读存储介质，其上存储有一个或多个指令，所述指令在由处理器执行时，使处理器执行根据权利要求1-5任一项所述的用于三维重建的方法。
  8. 一种计算机程序产品，包括一个或多个指令，所述指令在由处理器执行时，使处理器执行根据权利要求1-5任一项所述的用于三维重建的方法。
  9. 一种用于三维重建的系统，包括：
    信息处理装置，被配置为执行根据权利要求1-5任一项所述方法的步骤。
  10. 根据权利要求9所述的系统，还包括：
    提供所述镜面的平面镜组。
  11. 根据权利要求10所述的系统，其中，平面镜组由平面镜单元形成。
  12. 根据权利要求10所述的系统，其中，以下中的至少一项可以根据需要进行调整：平面镜组的数量、至少部分平面镜组相对于相机的位置或姿态。
  13. 根据权利要求10所述的系统，其中，平面镜组包括设置在上面的位置姿态获取模块，所述位置姿态获取模块被配置为获得并发送与平面镜组的位置姿态相关的信息。
  14. 一种平面镜组，包括一个或多个平面镜单元，其中：
    所述平面镜组包括设置在上面的位置姿态获取模块，所述位置姿态获取模块被配置 为获得并发送与平面镜组的位置姿态相关的信息。
  15. 根据权利要求14所述的平面镜组，其中：
    所述平面镜组被用于执行根据权利要求1-5中任一项所述的方法，或被包括在根据权利要求9-13中任一项所述的系统中。
PCT/CN2022/130480 2021-11-10 2022-11-08 用于三维重建的方法、系统和存储介质 WO2023083154A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202280073308.1A CN118202392A (zh) 2021-11-10 2022-11-08 用于三维重建的方法、系统和存储介质

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111324566.6A CN116129031A (zh) 2021-11-10 2021-11-10 用于三维重建的方法、系统和存储介质
CN202111324566.6 2021-11-10

Publications (1)

Publication Number Publication Date
WO2023083154A1 true WO2023083154A1 (zh) 2023-05-19

Family

ID=86294232

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/130480 WO2023083154A1 (zh) 2021-11-10 2022-11-08 用于三维重建的方法、系统和存储介质

Country Status (2)

Country Link
CN (2) CN116129031A (zh)
WO (1) WO2023083154A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117315152A (zh) * 2023-09-27 2023-12-29 杭州一隅千象科技有限公司 双目立体成像方法及其系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182095A (zh) * 2014-08-13 2014-12-03 长春理工大学 移动式自定位激光3d投影系统
CN104567818A (zh) * 2014-12-31 2015-04-29 浙江工业大学 一种便携式全天候主动全景视觉传感器
WO2019200419A2 (de) * 2018-04-16 2019-10-24 ATENSOR Engineering and Technology Systems GmbH 3d-sensorsystem
CN110443888B (zh) * 2019-08-15 2021-03-26 华南理工大学 一种形成多次反射成像的结构光三维重建装置及方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182095A (zh) * 2014-08-13 2014-12-03 长春理工大学 移动式自定位激光3d投影系统
CN104567818A (zh) * 2014-12-31 2015-04-29 浙江工业大学 一种便携式全天候主动全景视觉传感器
WO2019200419A2 (de) * 2018-04-16 2019-10-24 ATENSOR Engineering and Technology Systems GmbH 3d-sensorsystem
CN110443888B (zh) * 2019-08-15 2021-03-26 华南理工大学 一种形成多次反射成像的结构光三维重建装置及方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117315152A (zh) * 2023-09-27 2023-12-29 杭州一隅千象科技有限公司 双目立体成像方法及其系统
CN117315152B (zh) * 2023-09-27 2024-03-29 杭州一隅千象科技有限公司 双目立体成像方法及其系统

Also Published As

Publication number Publication date
CN116129031A (zh) 2023-05-16
CN118202392A (zh) 2024-06-14

Similar Documents

Publication Publication Date Title
US20210004933A1 (en) Method and system for image generation
CN110135455B (zh) 影像匹配方法、装置及计算机可读存储介质
EP2236980B1 (en) A method for determining the relative position of a first and a second imaging device and devices therefore
US9530235B2 (en) Aligning panoramic imagery and aerial imagery
CN107155341B (zh) 三维扫描系统和框架
KR101694969B1 (ko) 카메라 캘리브레이션 방법 및 장치
JP6883608B2 (ja) 深度マップに対して画像位置合せを行って深度データを最適化することができる深度データ処理システム
WO2018228436A1 (zh) 双视角图像校准及图像处理方法、装置、存储介质和电子设备
CN106530358A (zh) 仅用两幅场景图像标定ptz摄像机的方法
EP2960859B1 (en) Constructing a 3d structure
CN113256718B (zh) 定位方法和装置、设备及存储介质
US20220237880A1 (en) System and method of generating a 3d representation of an object
JP2019032218A (ja) 位置情報記録方法および装置
WO2023083154A1 (zh) 用于三维重建的方法、系统和存储介质
JP7220785B2 (ja) 測量用サンプリング点の計画方法、装置、制御端末及び記憶媒体
Hafeez et al. Image based 3D reconstruction of texture-less objects for VR contents
CN113361365A (zh) 定位方法和装置、设备及存储介质
CN113496503B (zh) 点云数据的生成及实时显示方法、装置、设备及介质
CN115345942A (zh) 空间标定方法、装置、计算机设备和存储介质
CN111127556A (zh) 基于3d视觉的目标物体识别和位姿估算方法以及装置
KR101598399B1 (ko) 로드뷰 사진이미지의 좌표정보를 이용한 자동 이미지 합성 시스템
KR101673144B1 (ko) 부분 선형화 기반의 3차원 영상 정합 방법
EP2879090B1 (en) Aligning ground based images and aerial imagery
Brunken et al. Deep learning self-calibration from planes
JP6890422B2 (ja) 情報処理装置、情報処理装置の制御方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22891948

Country of ref document: EP

Kind code of ref document: A1