WO2023103884A1 - 对象模型建立方法、装置、电子设备及存储介质 - Google Patents

对象模型建立方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023103884A1
WO2023103884A1 PCT/CN2022/135983 CN2022135983W WO2023103884A1 WO 2023103884 A1 WO2023103884 A1 WO 2023103884A1 CN 2022135983 W CN2022135983 W CN 2022135983W WO 2023103884 A1 WO2023103884 A1 WO 2023103884A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
marker
sample
acquisition device
image acquisition
Prior art date
Application number
PCT/CN2022/135983
Other languages
English (en)
French (fr)
Inventor
税国知
钟传琦
李扬
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2023103884A1 publication Critical patent/WO2023103884A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the field of augmented reality technology, and in particular relates to an object model establishment method, device, electronic equipment and storage medium.
  • AR Augmented Reality, Augmented Reality
  • 3D modeling and registration intelligent interaction, sensing and other technical fields.
  • 3D models, music, video and other virtual information are simulated and applied to the real world.
  • the two kinds of information complement each other, thus realizing the "enhancement" of the real world.
  • the main element of AR technology is content production, which focuses on the establishment of 3D models of physical objects.
  • Related technologies usually use CAD (Computer Aided Design, computer-aided design) software for manual modeling, or use precision reconstruction equipment such as 3D lasers Scanning modeling such as scanners.
  • Manual modeling requires staff to draw the 3D model of the object through CAD software according to the measurement parameters of the object, and the workload is huge.
  • the reconstruction equipment is expensive, and the modeling process is complex and time-consuming.
  • the output model needs secondary and tertiary processing before it can be applied to the production of AR content.
  • the purpose of the embodiments of the present application is to provide an object model establishment method, device, electronic device and storage medium, so as to reduce the period of object modeling and reduce the workload of modeling.
  • the specific technical scheme is as follows:
  • the embodiment of the present application provides a method for establishing an object model, the method comprising:
  • For each sample image determine the position of the marker in the sample image in the world coordinate system according to the pose information when the image acquisition device collects the sample image, and the position of the marker in the sample image;
  • a three-dimensional sparse point cloud model of the target object is established according to the positions of the landmarks in each of the sample images in the world coordinate system.
  • the embodiment of the present application provides an object model establishment device, the device includes:
  • a sample image acquisition module configured to acquire a plurality of sample images including markers and target objects collected by an image acquisition device, wherein, in the plurality of sample images, the markers are set on multiple keys of the target object point;
  • a marker position determination module configured to determine the positions of the markers in each of the sample images respectively
  • a pose information acquisition module configured to acquire pose information when the image acquisition device collects each of the sample images
  • the world coordinate determination module is used to determine, for each sample image, where the marker in the sample image is based on the pose information when the image acquisition device collects the sample image and the position of the marker in the sample image. position in the world coordinate system;
  • the 3D point cloud model building module is used to create a 3D sparse point cloud model of the target object according to the positions of the landmarks in each of the sample images in the world coordinate system.
  • the embodiment of the present application provides an electronic device, including a processor and a memory;
  • the memory is used to store computer programs
  • the processor is configured to implement any of the object model building methods described in the present application when executing the program stored on the memory.
  • the embodiment of the present application provides a computer-readable storage medium, which is characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any The method for establishing the object model.
  • the embodiment of the present application provides a computer program product containing instructions, which is characterized in that, when the computer program product is run on a computer, the computer is made to execute any one of the object model building methods described in the present application .
  • the object model establishment method, device, electronic equipment, and storage medium acquire a plurality of sample images including markers and target objects collected by an image acquisition device, wherein the markers are set in the plurality of sample images At multiple key points of the target object; respectively determine the position of the markers in each sample image; obtain the pose information when the image acquisition device collects each sample image; for each sample image, according to the image acquisition device when collecting the sample image Pose information, the position of the marker in the sample image, determine the position of the marker in the sample image in the world coordinate system; according to the position of the marker in each sample image in the world coordinate system, establish the position of the target object 3D sparse point cloud model.
  • the automatic modeling of the object model is realized, which can reduce the workload of modeling; the number of point clouds of the three-dimensional sparse point cloud model established by the embodiment of the present application is small, but the embodiment of the present application can realize the rapid modeling of the object, which can reduce the The cycle of object modeling reduces the workload of modeling. Moreover, the three-dimensional sparse point cloud model established in the embodiment of the present application has small data scale and low production difficulty, and is especially suitable for application in actual scenarios. Of course, implementing any product or method of the present application does not necessarily need to achieve all the above-mentioned advantages at the same time.
  • Fig. 1 is the first schematic diagram of the method for establishing an object model in the embodiment of the present application
  • FIG. 2 is a second schematic diagram of the method for establishing an object model in the embodiment of the present application.
  • FIG. 3 is a third schematic diagram of the method for establishing an object model in the embodiment of the present application.
  • Fig. 4 is the first schematic diagram of the object to be marked in the embodiment of the present application.
  • FIG. 5 is a second schematic diagram of the object to be marked in the embodiment of the present application.
  • Fig. 6 is a kind of schematic diagram of the three-dimensional sparse point cloud model of the embodiment of the present application.
  • FIG. 7 is a schematic diagram of an object model building device according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an electronic device according to an embodiment of the present application.
  • an embodiment of the present application provides a method for establishing an object model, see Figure 1, the method includes:
  • S101 Acquire a plurality of sample images collected by an image acquisition device that include markers and a target object, where the markers are set at multiple key points of the target object in the plurality of sample images.
  • the method for establishing an object model in the embodiment of the present application can be implemented by an electronic device with an image processing function.
  • the electronic device can be a handheld electronic device, such as a smart camera, a hard disk video recorder, a smart phone, etc.
  • the The electronic device may also be a personal computer or a server.
  • the sample image includes landmarks and target objects.
  • the landmarks need to have obvious appearance characteristics so that computer vision technology can be used to accurately identify the landmarks from the image.
  • the specific types of landmarks can be customized according to the actual situation.
  • the marker may be a two-dimensional code, a black and white checkerboard pattern, or other specific images.
  • the target object can be any object for which a 3D model needs to be established, for example, it can be a vehicle, a building, an industrial component, an animal or a plant, and the like.
  • the image acquisition device can be a monocular camera, a binocular camera, or a smart phone with a camera function.
  • Each sample image includes at least one marker, and the positions of the markers on the target object in different sample images can be the same or different, but for all sample images, the markers in these sample images need to be able to represent the target object position of multiple key points.
  • the positions of the markers and the target object in different sample images are not all the same.
  • the markers are set at different positions on the target object in different sample images and/or the angles at which the target object is captured are different in different sample images.
  • the key points of the target object can be customized according to the actual situation.
  • the key points are used to represent the outline of the target object, which can be points on the outline of the target object.
  • a landmark edge on the target object can be selected Corner positions etc. are used as key points of the target object. It is understandable that there may be some errors in the setting of markers in actual scenes.
  • the markers may be set exactly on the key points, or there may be a small distance from the key points, as long as they can represent the outline of the target object.
  • the computer vision technology is used to identify the markers in the sample image, and the positions of the markers contained therein are obtained, that is, the positions of the markers in the sample image.
  • the markers are preset two-dimensional codes
  • the respectively determining the positions of the markers in each of the sample images includes: for each sample image, using a two-dimensional code to identify The technology performs two-dimensional code recognition on the sample image, and obtains the position of the preset two-dimensional code in the sample image.
  • the marker is a black and white checkerboard pattern
  • determining the positions of the markers in each of the sample images includes: for each sample image, using computer vision to identify the The sample image is subjected to black and white checkerboard pattern recognition, and the position of the black and white checkerboard pattern in the sample image is obtained.
  • the pose information of the image acquisition device may include the position information of the image acquisition device (for example, the position of the image acquisition device in the world coordinate system) and attitude information (for example, the shooting angle when the image acquisition device collects the sample image).
  • position information of the image acquisition device for example, the position of the image acquisition device in the world coordinate system
  • attitude information for example, the shooting angle when the image acquisition device collects the sample image.
  • one or more of a gyroscope, a geomagnetic sensor, an acceleration sensor, and a wireless communication positioning module is installed in the image acquisition device, so as to obtain pose information of the image acquisition device.
  • S104 for each sample image, determine the position of the marker in the sample image in the world coordinate system according to the pose information when the image acquisition device collects the sample image and the position of the marker in the sample image Location.
  • the world coordinate system in the embodiment of the present application refers to the coordinate system of the real world where the sample to be marked (target object) is located.
  • the coordinate system of latitude and longitude plus height can be used, or it can be a three-dimensional custom built for the scene where the sample to be marked is located. coordinate system etc.
  • the external parameters of the image acquisition device can be obtained, and according to the position of the marker in the sample image, the attitude information when the image acquisition device collects the sample image, and the external parameters of the image acquisition device, the three-dimensional position of the marker in the image acquisition device can be obtained.
  • the position in the coordinate system then according to the position information of the image acquisition device in the world coordinate system and the position of the marker in the three-dimensional coordinate system of the image acquisition device, the position of the marker in the world coordinate system is obtained.
  • the marker is set at the key point of the target object, so the position of the marker in the world coordinate system is also the position of the key point of the target object in the world coordinate system; the markers in multiple sample images can be set at different target objects
  • the position of the target object in the world coordinate system can be represented by the position of the key points of the target object in the world coordinate system. Therefore, the position of the key points of the target object in the world coordinate system can be used to establish a 3D sparse point cloud model of the target object in the world coordinate system, that is, a 3D sparse point cloud model composed of key points represented by markers .
  • the marker information of the markers can also be obtained, and the corresponding relationship between the marker information of the markers and the 3D sparse point cloud model of the target object can be established .
  • the marker information of the marker may be an identifier of the marker, etc., and an identifier may be pre-set for each marker as the marker information of the marker.
  • the marker information of markers with the same visual feature is the same, and the marker information of markers with different visual features is different.
  • the marker is a two-dimensional code
  • the marker information is two-dimensional code information, so that the two-dimensional codes of the same type of target objects can be the same, and the two-dimensional codes of different types of target objects are different, so as to facilitate the subsequent use of two-dimensional code to call the model of the target object.
  • the corresponding relationship between the marker information of the marker and the 3D sparse point cloud model of the target object is established, and the marker information of the marker can be used to quickly call the 3D sparse point cloud model of the target object later.
  • the automatic modeling of the object model is realized by using images containing landmarks and target objects, which can reduce the workload of modeling; although compared with the 3D models established by reconstruction equipment such as 3D laser scanners,
  • the 3D sparse point cloud model established by the embodiment of the present application has a small number of point clouds, but the embodiment of the present application can realize rapid object modeling, reduce the cycle of object modeling, and reduce the workload of modeling.
  • the three-dimensional sparse point cloud model established in the embodiment of the present application has small data scale and low production difficulty, and is especially suitable for application in actual scenarios.
  • the SLAM Simultaneous Localization And Mapping
  • the acquisition of pose information when the image acquisition device acquires each of the sample images includes:
  • each of the sample images use a SLAM algorithm to determine pose information when the image acquisition device acquires each of the sample images.
  • the SLAM algorithm is also called CML (Concurrent Mapping and Localization, real-time positioning and map construction) algorithm or concurrent mapping and localization algorithm.
  • the SLAM algorithm refers to placing a robot in an unknown position in an unknown environment, so that the robot can gradually draw a complete map of the environment while moving.
  • the SLAM algorithm can use the two-dimensional image collected by the image acquisition device to model the unknown environment, obtain the position and attitude of the image acquisition device in the unknown environment, and obtain the position of each object (object) in the position environment.
  • the target object is required to be a static object, that is, the target object will not move or deform in the world coordinate system.
  • the specific calculation process of the SLAM algorithm refer to the implementation process of the SLAM algorithm in the related art, which is not specifically limited in this application.
  • the pose information of the image acquisition device and the position of the landmark in the world coordinate system can be obtained quickly and accurately by using the SLAM algorithm.
  • the method further includes:
  • a preset two-dimensional code is set as a marker at a key point of the target object, as shown in FIG. 5 .
  • S303 Repeat the above steps: S302 adjust the position of the image acquisition device and/or the position of the marker at the target object, and use the image acquisition device to capture images including the marker and the target object sample images until the acquisition termination condition is satisfied.
  • Step S302 is repeatedly executed until the acquisition termination condition is satisfied.
  • the acquisition termination condition can be customized according to the actual situation.
  • the acquisition termination condition can be that a preset number of sample images has been collected, and the preset number can be customized according to the actual situation, but it is necessary to ensure that the sample images of the preset data are sufficient to establish the target
  • the 3D sparse point cloud model of the object for example, the acquisition termination condition can trigger an instruction to stop the acquisition for the user, etc.
  • the 3D sparse point cloud model of the eight key points of the natural gas pipeline interface can be shown in FIG. 6 .
  • the three-dimensional sparse point cloud model is obtained by two-dimensional image sampling, which can realize the automatic generation of the three-dimensional sparse point cloud model. It has great advantages in scenes with different effects, indoor and outdoor shooting scenes, etc.
  • the two-dimensional image can be used to obtain the position of the key point of the target object in the three-dimensional world coordinate system.
  • the SLAM algorithm By integrating the SLAM algorithm and the two-dimensional code recognition technology, the difficulty of interacting between the two-dimensional image and the three-dimensional scene The problem.
  • the method for establishing an object model in the embodiment of the present application has low equipment cost, simple deployment, simple modeling process, and high success rate.
  • the key points of the marked objects are obtained from the two-dimensional code, and the accurate outline description method is used.
  • the three-dimensional sparse point cloud model has high precision and can be used without subsequent cutting and processing.
  • the method further includes:
  • Step 1 according to the obtained position of the marker in the world coordinate system and the current pose information of the image acquisition device, determine that the key point of the target object is in the image coordinate system of the image acquisition device The position in gets the keypoint image position.
  • the marker represents the key point of the target object
  • the position of the marker in the world coordinate system is the position of the key point of the target object in the world coordinate system.
  • Step 2 based on the obtained image positions of the key points, a rectangular frame is obtained by fitting.
  • the fitting of the rectangular box is not performed.
  • rectangle fitting can be performed on each key point image position to obtain a rectangular frame.
  • the way to obtain a rectangle by fitting multiple points can refer to the rectangle fitting method in the related art.
  • the position of the key point image can be used as the corner point of the rectangle to fit the largest rectangle, and make each The image positions of the key points all fall on the inside and on the rectangular frame.
  • Step 3 displaying the position of the key point image and the rectangular frame on the display screen corresponding to the image acquisition device.
  • the display screen corresponding to the image acquisition device may be a built-in display screen of the image acquisition device, or may be an external display screen connected to the image acquisition device. Displaying the position of the key point image and the rectangular frame on the display screen corresponding to the image acquisition device can enable the user to intuitively perceive the establishment effect of the 3D model, and can intuitively perceive the labeling result of the rectangular frame, which is convenient for the user to adjust the position of the marker in real time. In order to obtain a 3D model with better annotation effect.
  • the embodiment of the present application also provides an object model building device, see Figure 7, the device includes:
  • a sample image acquisition module 701 configured to acquire a plurality of sample images collected by an image acquisition device that include markers and target objects, wherein, in the plurality of sample images, the markers are set on multiple key points;
  • a marker position determination module 702 configured to respectively determine the positions of the markers in each of the sample images
  • a pose information acquisition module 703, configured to acquire pose information when the image acquisition device collects each of the sample images
  • the world coordinate determining module 704 is configured to, for each sample image, determine the marker in the sample image according to the pose information when the image acquisition device collects the sample image and the position of the marker in the sample image position in the world coordinate system;
  • the 3D point cloud model building module 705 is configured to create a 3D sparse point cloud model of the target object according to the positions of the landmarks in each of the sample images in the world coordinate system.
  • the marker is a preset two-dimensional code
  • the marker position determining module is specifically configured to: for each sample image, use two-dimensional code recognition technology to perform two-dimensional code identification on the sample image. QR code recognition to obtain the position of the preset QR code in the sample image.
  • the pose information acquisition module is specifically configured to: use the SLAM algorithm for synchronous positioning and mapping according to each of the sample images to determine when the image acquisition device collects each of the sample images pose information.
  • the world coordinate determination module is specifically configured to: for each sample image, according to the pose information when the image acquisition device collects the sample image, and the position of the marker in the sample image, use the SLAM algorithm to determine The position of the marker in the sample image in the world coordinate system.
  • the device also includes:
  • a marker setting module configured to set the marker at key points of the target object, and use the image acquisition device to collect a sample image including the marker and the target object;
  • a sample image acquisition module configured to adjust the position of the image acquisition device and/or the position of the marker at the target object, and use the image acquisition device to acquire images containing the marker and the target object sample image;
  • the collection completion judging module is used to call the sample image collection module to repeatedly collect sample images until the collection termination condition is met.
  • the device further includes: a rectangular frame display module, configured to obtain the position of the marker in the world coordinate system and the current pose information of the image acquisition device , determine the position of the key point of the target object in the image coordinate system of the image acquisition device to obtain the image position of the key point; based on the obtained image position of the key point, fit a rectangular frame; in the image acquisition The image position of the key point and the rectangular frame are displayed on the display screen corresponding to the device.
  • a rectangular frame display module configured to obtain the position of the marker in the world coordinate system and the current pose information of the image acquisition device , determine the position of the key point of the target object in the image coordinate system of the image acquisition device to obtain the image position of the key point; based on the obtained image position of the key point, fit a rectangular frame; in the image acquisition The image position of the key point and the rectangular frame are displayed on the display screen corresponding to the device.
  • the embodiment of the present application also provides an electronic device, including: a processor and a memory;
  • the above-mentioned memory is used to store computer programs
  • the above-mentioned processor When the above-mentioned processor is used to execute the computer program stored in the above-mentioned memory, it can implement any one of the methods for establishing an object model in the present application.
  • the electronic device in this embodiment of the present application further includes a communication interface 802 and a communication bus 804 , where the processor 801 , the communication interface 802 , and the memory 803 communicate with each other through the communication bus 804 .
  • the communication bus mentioned in the above-mentioned electronic equipment may be a PCI (Peripheral Component Interconnect, Peripheral Component Interconnect Standard) bus or an EISA (Extended Industry Standard Architecture, Extended Industry Standard Architecture) bus, etc.
  • the communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used for communication between the electronic device and other devices.
  • the memory may include RAM (Random Access Memory, random access memory), and may also include NVM (Non-Volatile Memory, non-volatile memory), such as at least one disk memory.
  • the memory may also be at least one storage device located far away from the aforementioned processor.
  • processor can be general-purpose processor, comprises CPU (Central Processing Unit, central processing unit), NP (Network Processor, network processor) etc.; Can also be DSP (Digital Signal Processing, digital signal processor), ASIC ( Application Specific Integrated Circuit (ASIC), FPGA (Field-Programmable Gate Array, Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor, network processor
  • DSP Digital Signal Processing, digital signal processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array, Field Programmable Gate Array
  • other programmable logic devices discrete gate or transistor logic devices, and discrete hardware components.
  • the embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored in the above-mentioned computer-readable storage medium, and when the above-mentioned computer program is executed by a processor, the method for establishing an object model described in any one of the present application is implemented.
  • a computer program product including instructions is also provided, which, when run on a computer, causes the computer to execute the method for establishing an object model described in any one of the above embodiments.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media.
  • the available media may be magnetic media, (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), and the like.
  • each embodiment in this specification is described in a related manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments.
  • the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiments.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种对象模型建立方法、装置、电子设备及存储介质，获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在多个样本图像中标志物设置在目标对象的多个关键点处；分别确定各样本图像中标志物的位置；获取图像采集设备采集各样本图像时的位姿信息；针对每一个样本图像，根据图像采集设备采集该样本图像时的位姿信息、标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；根据各样本图像中的标志物在世界坐标系中的位置，建立目标对象的三维稀疏点云模型。实现了对象模型的自动建模，能够减少建模的工作量，模型数据规模小，制作难度低，特别适用于实际场景的应用。

Description

对象模型建立方法、装置、电子设备及存储介质
本申请要求于2021年12月10日提交中国专利局、申请号为202111506107.X发明名称为“对象模型建立方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及增强现实技术领域，特别是涉及对象模型建立方法、装置、电子设备及存储介质。
背景技术
AR(Augmented Reality，增强现实)技术是一种将虚拟信息与真实世界巧妙融合的技术，广泛运用于多媒体、三维建模及注册、智能交互、传感等技术领域，将计算机生成的文字、图像、三维模型、音乐、视频等虚拟信息模拟仿真后，应用到真实世界中，两种信息互为补充，从而实现对真实世界的“增强”。
AR技术的主要要素为内容制作，内容制作的重点在于实物对象三维模型的建立，相关技术中通常是利用CAD(Computer Aided Design，计算机辅助设计)软件人工建模，或使用精密重建设备例如三维激光扫描仪等扫描建模。人工建模需要工作人员按照对象的测量参数通过CAD软件来绘制对象的三维模型，工作量庞大。而重建设备价格昂贵，且建模过程复杂、耗时长，产出的模型还需要二次、三次加工才能应用于AR内容的制作。
可见通过上述建模方法，对象建模周期较长，建模工作量大。
发明内容
本申请实施例的目的在于提供一种对象模型建立方法、装置、电子设备及存储介质，以实现降低对象建模的周期，减少建模的工作量。具体技术方案如下：
第一方面，本申请实施例提供了一种对象模型建立方法，所述方法包括：
获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处；
分别确定各所述样本图像中所述标志物的位置；
获取所述图像采集设备采集各所述样本图像时的位姿信息；
针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；
根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
第二方面，本申请实施例提供了一种对象模型建立装置，所述装置包括：
样本图像获取模块，用于获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处；
标志物位置确定模块，用于分别确定各所述样本图像中所述标志物的位置；
位姿信息获取模块，用于获取所述图像采集设备采集各所述样本图像时的位姿信息；
世界坐标确定模块，用于针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；
三维点云模型建立模块，用于根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
第三方面，本申请实施例提供了一种电子设备，包括处理器及存储器；
所述存储器，用于存放计算机程序；
所述处理器，用于执行所述存储器上所存放的程序时，实现本申请中任 一所述的对象模型建立方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现本申请中任一所述的对象模型建立方法。
第五方面，本申请实施例提供了一种包含指令的计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得计算机执行本申请中任一所述的对象模型建立方法。
本申请实施例有益效果：
本申请实施例提供的对象模型建立方法、装置、电子设备及存储介质，获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在多个样本图像中标志物设置在目标对象的多个关键点处；分别确定各样本图像中标志物的位置；获取图像采集设备采集各样本图像时的位姿信息；针对每一个样本图像，根据图像采集设备采集该样本图像时的位姿信息、标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；根据各样本图像中的标志物在世界坐标系中的位置，建立目标对象的三维稀疏点云模型。实现了对象模型的自动建模，能够减少建模的工作量；本申请实施例建立的三维稀疏点云模型的点云数量较少，但是本申请实施例可以实现对象的快速建模，能够降低对象建模的周期，减少建模的工作量。并且本申请实施例建立的三维稀疏点云模型，数据规模小，制作难度低，特别适用于实际场景的应用。当然，实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1为本申请实施例的对象模型建立方法的第一种示意图；
图2为本申请实施例的对象模型建立方法的第二种示意图；
图3为本申请实施例的对象模型建立方法的第三种示意图；
图4为本申请实施例的待标注对象的第一种示意图；
图5为本申请实施例的待标注对象的第二种示意图；
图6为本申请实施例的三维稀疏点云模型的一种示意图；
图7为本申请实施例的对象模型建立装置的一种示意图；
图8为本申请实施例的电子设备的一种示意图。
具体实施方式
为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了降低对象建模的周期，减少建模的工作量，本申请实施例提供了一种对象模型建立方法，参见图1，所述方法包括：
S101，获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处。
本申请实施例的对象模型建立方法可以通过具备图像处理功能的电子设备实现，一个例子中，该电子设备可以为手持电子设备，例如，智能摄像机、硬盘录像机、智能手机等，一个例子中，该电子设备还可以为个人电脑或服务器等。
样本图像中包括标志物及目标对象，其中，标志物需要具有明显的外观特征，以便于能够利用计算机视觉技术从图像中准确识别出标志物，标志物的具体类型可以根据实际情况自定义设置，例如，标志物可以为二维码、黑 白棋盘图案或其他特定图像等。目标对象可以为任意需要建立三维模型的对象，例如，可以为车辆、建筑、工业零部件、动物或植物等。
图像采集设备可以为单目摄像机、双目摄像机或包含摄像功能的智能手机等设备。每个样本图像中包括至少一个标志物，不同样本图像中标志物在目标对象上的位置可以相同也可以不同，但是对于全部的样本图像而言，这些样本图像中的标志物需要能够表示目标对象上多个关键点的位置。为了防止重复采样，一个例子中，不同样本图像中标志物与目标对象的位置不全部相同。一个例子中，不同样本图像中所述标志物设置在目标对象处的位置不同和/或不同样本图像中采集目标对象的角度不同。目标对象的关键点可以根据实际情况自定义设置，具体的，关键点用于表示目标对象的轮廓，可以为目标对象轮廓线上的点，一个例子中，可以选取目标对象上有标志性的边角位置等作为目标对象的关键点。可以理解的是，实际场景中标志物的设置会有一定的误差，标志物可能恰好设置在关键点上，也可以距离关键点有一个较小的距离，只要能够表示目标对象的轮廓即可。
S102，分别确定各所述样本图像中所述标志物的位置。
利用计算机视觉技术对样本图像进行标志物识别，得到其中包含的标志物的位置，也即标志物在样本图像中的位置。
在一种可能的实施方式中，所述标志物为预设二维码，所述分别确定各所述样本图像中所述标志物的位置，包括：针对每一样本图像，利用二维码识别技术对该样本图像进行二维码识别，得到该样本图像中预设二维码的位置。
在一种可能的实施方式中，所述标志物为黑白棋盘图案，所述分别确定各所述样本图像中所述标志物的位置，包括：针对每一样本图像，利用计算机视觉别技术对该样本图像进行黑白棋盘图案识别，得到该样本图像中黑白棋盘图案的位置。
S103，获取所述图像采集设备采集各所述样本图像时的位姿信息。
图像采集设备的位姿信息可以包括图像采集设备的位置信息(例如图像 采集设备在世界坐标系中的位置)以及姿态信息(例如图像采集设备采集样本图像时的拍摄角度)。一个例子中，图像采集设备中安装有陀螺仪、地磁传感器、加速度传感器、无线通讯定位模块中的一种或多种，从而获取图像采集设备的位姿信息。
S104，针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置。
本申请实施例中的世界坐标系是指待标注样本(目标对象)所在的真实世界的坐标系，可以采用经纬度加高度的坐标系，也可以为针对待标注样本所在的场景自定义建立的三维坐标系等。
一个例子中，可以获取图像采集设备的外参，根据标志物在样本图像中的位置、图像采集设备采集该样本图像时的姿态信息、图像采集设备的外参，得到标志物在图像采集设备三维坐标系中的位置；然后根据图像采集设备在世界坐标系中的位置信息以及标志物在图像采集设备三维坐标系中的位置，得到标志物在世界坐标系中的位置。
S105，根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
标志物设置在目标对象的关键点处，因此标志物在世界坐标系中的位置也即目标对象的关键点在世界坐标系中的位置；多个样本图像中的标注物可以设置在目标对象不同的关键点处，目标对象在世界坐标系中的位置便可以通过目标对象的关键点在世界坐标系中的位置表示。因此，可以利用目标对象的关键点在世界坐标系中的位置，来建立目标对象在世界坐标系中的三维稀疏点云模型，也即由标志物表示的关键点所组成的三维稀疏点云模型。
一个例子中，在建立目标对象的三维稀疏点云模型后，还可以获取标志物的标志物信息，并建立所述标志物的标志物信息与所述目标对象的三维稀疏点云模型的对应关系。一个例子中，标志物的标志物信息可以为该标志物的标识等，可以预先为每一标志物设置一个标识作为该标志物的标志物信息。 一个例子中，具有相同视觉特征的标志物的标志物信息相同，不同视觉特征的标志物的标志物信息不同。一个例子中，标志物为二维码，标志物信息为二维码信息，可以令同一类型的目标对象的二维码相同，不同类型的目标对象的二维码不同，从而方便后续利用二维码来调取目标对象的模型。在本申请实施例中，建立标志物的标志物信息与目标对象的三维稀疏点云模型的对应关系，后续可以利用标志物的标志物信息快速调取目标对象的三维稀疏点云模型。
在本申请实施例中，利用包含标志物及目标对象的图像，实现了对象模型的自动建模，能够减少建模的工作量；虽然与三维激光扫描仪等重建设备建立的三维模型相比，本申请实施例建立的三维稀疏点云模型的点云数量较少，但是本申请实施例可以实现对象的快速建模，能够降低对象建模的周期，减少建模的工作量。并且本申请实施例建立的三维稀疏点云模型，数据规模小，制作难度低，特别适用于实际场景的应用。
一个例子中，可以利用SLAM(Simultaneous Localization And Mapping，同步定位与建图)算法来建立目标对象的三维稀疏点云模型。在一种可能的实施方式中，参见图2，所述获取所述图像采集设备采集各所述样本图像时的位姿信息，包括：
S201，根据各所述样本图像，利用SLAM算法，确定所述图像采集设备采集各所述样本图像时的位姿信息。
在一种可能的实施方式中，参见图2，所述针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置，包括：
S202，针对每一样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，利用SLAM算法，确定该样本图像中的标志物在世界坐标系中的位置。
SLAM算法也称为CML(Concurrent Mapping and Localization，即时定位与地图构建)算法或并发建图与定位算法。SLAM算法是指将一个机器人放入 未知环境中的未知位置，使机器人一边移动一边逐步描绘出此环境完全的地图。具体的，SLAM算法可以利用图像采集设备采集的二维图像进行未知环境的建模，获取图像采集设备在该未知环境中的位置及姿态，并获取该位置环境中各物体(对象)的位置。使用SLAM算法，要求目标对象为静止的对象，即目标对象在世界坐标系中不会发生移动及形变。SLAM算法的具体计算过程可以参见相关技术中的SLAM算法实现过程，本申请中不做具体限定。
在本申请实施例中，使用SLAM算法，可以快速、准确的得到图像采集设备的位姿信息及标志物在世界坐标系中的位置。
下面对样本图像的采集过程进行说明，在一种可能的实施方式中，参见图3，所述方法还包括：
S301，将所述标志物设置在所述目标对象的关键点处，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像。
一个例子中，参见图4，以天然气管道接口为目标对象为例，将预设二维码作为标志物设置在目标对象的关键点处，例如图5所示。
S302，调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像。
调整图像采集设备拍摄目标对象的角度及位置，从而得到不同位姿下包含标志物及目标对象的样本图像；还可以将标志物放置在目标对象不同的关键点上，从而得到目标对象不同的关键点的位置。
S303，重复执行上述步骤：S302调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像，直至满足采集终止条件。
重复执行步骤S302直至满足采集终止条件。采集终止条件可以根据实际情况自定义设置，例如采集终止条件可以为采集到预设数量的样本图像，其中预设数量可以根据实际情况自定义设置，但是需要保证预设数据的样本图像足够建立目标对象的三维稀疏点云模型；例如，采集终止条件可以为用户 触发停止采集的指令等。一个例子中，天然气管道接口的八个关键点的三维稀疏点云模型可以如图6所示。
本申请实施例中，通过二维图像采样的方式来得到三维稀疏点云模型，能够实现三维稀疏点云模型的自动生成，对场景的适应性好，针对光线亮暗程度不同的场景、相机成像效果不同的场景、室内外拍摄场景等均有较大优势。
结合二维码及SLAM算法，可以利用二维图像来得到目标对象的关键点在三维世界坐标系中的位置，通过融合SLAM算法与二维码识别技术，解决了二维图像与三维场景交互难的问题。本申请实施例的对象模型建立方法与利用三维激光扫描仪等重建设备进行建模的方式相比，设备成本低廉，部署简单，建模过程简单，成功率高。由二维码来得到对标注对象的关键点，使用了准确的轮廓描述方法，三维稀疏点云模型的精度高，并且无需后续的裁剪加工，即可进行使用。
为了更加方便用户感知三维模型的建立效果，在一种可能的实施方式中，所述方法还包括：
步骤一，根据已得到的所述标志物在所述世界坐标系中的位置及所述图像采集设备的当前位姿信息，确定所述目标对象的关键点在所述图像采集设备的图像坐标系中的位置得到关键点图像位置。
标志物代表目标对象的关键点，标志物在世界坐标系中的位置即目标对象的关键点在世界坐标系中的位置。根据图像采集设备的实时位姿信息，可以得到图像采集设备的图像坐标系与世界坐标系的转换关系，因此可以得到关键点在图像坐标系中的位置，也即关键点图像位置。
步骤二，基于已得到的所述关键点图像位置，拟合得到矩形框。
一个例子中，在仅有一个关键点图像位置的情况下，不进行矩形框的拟合。在有至少两个关键点图像位置的情况下，可以对各关键点图像位置进行矩形拟合，得到矩形框。利用多个点拟合得到矩形的方式可以参见相关技术中的矩形拟合方式，一个例子中，可以将关键点图像位置作为矩形框的角点， 来拟合得到最大的矩形框，并使得各关键点图像位置均落在矩形框的内部及矩形框上。
步骤三，在所述图像采集设备对应的显示屏中显示所述关键点图像位置及所述矩形框。
图像采集设备对应的显示屏可以为该图像采集设备内置的显示屏，也可以为该图像采集设备外接的显示屏。在图像采集设备对应的显示屏中显示关键点图像位置及矩形框，能够使得用户直观的感知三维模型的建立效果，并且能够直观的感知到矩形框标注结果，便于用户实时调整标志物的位置，以得到标注效果更好的三维模型。
本申请实施例还提供了一种对象模型建立装置，参见图7，所述装置包括：
样本图像获取模块701，用于获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处；
标志物位置确定模块702，用于分别确定各所述样本图像中所述标志物的位置；
位姿信息获取模块703，用于获取所述图像采集设备采集各所述样本图像时的位姿信息；
世界坐标确定模块704，用于针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；
三维点云模型建立模块705，用于根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
在一种可能的实施方式中，所述标志物为预设二维码，所述标志物位置确定模块，具体用于：针对每一样本图像，利用二维码识别技术对该样本图像进行二维码识别，得到该样本图像中预设二维码的位置。
在一种可能的实施方式中，所述位姿信息获取模块，具体用于：根据各所述样本图像，利用同步定位与建图SLAM算法，确定所述图像采集设备采集各所述样本图像时的位姿信息。
所述世界坐标确定模块，具体用于：针对每一样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，利用SLAM算法，确定该样本图像中的标志物在世界坐标系中的位置。
在一种可能的实施方式中，所述装置还包括：
标志物设置模块，用于将所述标志物设置在所述目标对象的关键点处，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
样本图像采集模块，用于调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
采集完成判断模块，用于调用所述样本图像采集模块重复采集样本图像，直至满足采集终止条件。
在一种可能的实施方式中，所述装置还包括：矩形框显示模块，用于根据已得到的所述标志物在所述世界坐标系中的位置及所述图像采集设备的当前位姿信息，确定所述目标对象的关键点在所述图像采集设备的图像坐标系中的位置得到关键点图像位置；基于已得到的所述关键点图像位置，拟合得到矩形框；在所述图像采集设备对应的显示屏中显示所述关键点图像位置及所述矩形框。
本申请实施例还提供了一种电子设备，包括：处理器及存储器；
上述存储器，用于存放计算机程序；
上述处理器用于执行上述存储器存放的计算机程序时，实现本申请中任一所述的对象模型建立方法。
可选的，参见图8，本申请实施例的电子设备还包括通信接口802和通信总线804，其中，处理器801，通信接口802，存储器803通过通信总线804 完成相互间的通信。
上述电子设备提到的通信总线可以是PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(Extended Industry Standard Architecture，扩展工业标准结构)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
通信接口用于上述电子设备与其他设备之间的通信。
存储器可以包括RAM(Random Access Memory，随机存取存储器)，也可以包括NVM(Non-Volatile Memory，非易失性存储器)，例如至少一个磁盘存储器。可选的，存储器还可以是至少一个位于远离前述处理器的存储装置。
上述的处理器可以是通用处理器，包括CPU(Central Processing Unit，中央处理器)、NP(Network Processor，网络处理器)等；还可以是DSP(Digital Signal Processing，数字信号处理器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field-Programmable Gate Array，现场可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本申请实施例还提供了一种计算机可读存储介质，上述计算机可读存储介质内存储有计算机程序，上述计算机程序被处理器执行时实现本申请中任一所述的对象模型建立方法。
在本申请提供的又一实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一所述的对象模型建立方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或 者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)等。
需要说明的是，在本文中，各个可选方案中的技术特征只要不矛盾均可组合来形成方案，这些方案均在本申请公开的范围内。诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、电子设备及存储介质的实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (13)

  1. 一种对象模型建立方法，所述方法包括：
    获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处；
    分别确定各所述样本图像中所述标志物的位置；
    获取所述图像采集设备采集各所述样本图像时的位姿信息；
    针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置；
    根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
  2. 根据权利要求1所述的方法，其中，所述标志物为预设二维码，所述分别确定各所述样本图像中所述标志物的位置，包括：
    针对每一样本图像，利用二维码识别技术对该样本图像进行二维码识别，得到该样本图像中预设二维码的位置。
  3. 根据权利要求1所述的方法，其中，所述获取所述图像采集设备采集各所述样本图像时的位姿信息，包括：
    根据各所述样本图像，利用同步定位与建图SLAM算法，确定所述图像采集设备采集各所述样本图像时的位姿信息；
    所述针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该样本图像中的标志物在世界坐标系中的位置，包括：
    针对每一样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，利用SLAM算法，确定该样本图像中的标志物在世界坐标系中的位置。
  4. 根据权利要求1所述的方法，所述方法还包括：
    将所述标志物设置在所述目标对象的关键点处，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
    调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
    重复执行上述步骤：调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像，直至满足采集终止条件。
  5. 根据权利要求1-4任一所述的方法，所述方法还包括：
    根据已得到的所述标志物在所述世界坐标系中的位置及所述图像采集设备的当前位姿信息，确定所述目标对象的关键点在所述图像采集设备的图像坐标系中的位置得到关键点图像位置；
    基于已得到的所述关键点图像位置，拟合得到矩形框；
    在所述图像采集设备对应的显示屏中显示所述关键点图像位置及所述矩形框。
  6. 一种对象模型建立装置，所述装置包括：
    样本图像获取模块，用于获取由图像采集设备采集的包含标志物及目标对象的多个样本图像，其中，在所述多个样本图像中所述标志物设置在所述目标对象的多个关键点处；
    标志物位置确定模块，用于分别确定各所述样本图像中所述标志物的位置；
    位姿信息获取模块，用于获取所述图像采集设备采集各所述样本图像时的位姿信息；
    世界坐标确定模块，用于针对每一个样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，确定该 样本图像中的标志物在世界坐标系中的位置；
    三维点云模型建立模块，用于根据各所述样本图像中的标志物在所述世界坐标系中的位置，建立所述目标对象的三维稀疏点云模型。
  7. 根据权利要求6所述的装置，其中，所述标志物为预设二维码，所述标志物位置确定模块，具体用于：针对每一样本图像，利用二维码识别技术对该样本图像进行二维码识别，得到该样本图像中预设二维码的位置。
  8. 根据权利要求6所述的装置，其中，所述位姿信息获取模块，具体用于：根据各所述样本图像，利用同步定位与建图SLAM算法，确定所述图像采集设备采集各所述样本图像时的位姿信息；
    所述世界坐标确定模块，具体用于：针对每一样本图像，根据所述图像采集设备采集该样本图像时的位姿信息、所述标志物在该样本图像中的位置，利用SLAM算法，确定该样本图像中的标志物在世界坐标系中的位置。
  9. 根据权利要求6所述的装置，所述装置还包括：
    标志物设置模块，用于将所述标志物设置在所述目标对象的关键点处，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
    样本图像采集模块，用于调整所述图像采集设备的位置和/或所述标志物在所述目标对象处的位置，并利用所述图像采集设备采集包含所述标志物及所述目标对象的样本图像；
    采集完成判断模块，用于调用所述样本图像采集模块重复采集样本图像，直至满足采集终止条件。
  10. 根据权利要求6-9任一所述的装置，所述装置还包括：
    矩形框显示模块，用于根据已得到的所述标志物在所述世界坐标系中的位置及所述图像采集设备的当前位姿信息，确定所述目标对象的关键点在所述图像采集设备的图像坐标系中的位置得到关键点图像位置；基于已得到的所述关键点图像位置，拟合得到矩形框；在所述图像采集设备对应的显示屏中显示所述关键点图像位置及所述矩形框。
  11. 一种电子设备，包括处理器及存储器；
    所述存储器，用于存放计算机程序；
    所述处理器，用于执行所述存储器上所存放的程序时，实现权利要求1-5任一所述的对象模型建立方法。
  12. 一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-5任一所述的对象模型建立方法。
  13. 一种包含指令的计算机程序产品，当所述计算机程序产品在计算机上运行时，使得计算机执行权利要求1-5任一所述的对象模型建立方法。
PCT/CN2022/135983 2021-12-10 2022-12-01 对象模型建立方法、装置、电子设备及存储介质 WO2023103884A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111506107.X 2021-12-10
CN202111506107.XA CN116258812A (zh) 2021-12-10 2021-12-10 对象模型建立方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023103884A1 true WO2023103884A1 (zh) 2023-06-15

Family

ID=86681309

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/135983 WO2023103884A1 (zh) 2021-12-10 2022-12-01 对象模型建立方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN116258812A (zh)
WO (1) WO2023103884A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110221690A (zh) * 2019-05-13 2019-09-10 Oppo广东移动通信有限公司 基于ar场景的手势交互方法及装置、存储介质、通信终端
WO2019169540A1 (zh) * 2018-03-06 2019-09-12 斯坦德机器人（深圳）有限公司 紧耦合视觉slam的方法、终端及计算机可读存储介质
CN111627043A (zh) * 2020-04-13 2020-09-04 浙江工业大学 一种基于标志物和特征筛选器的简易人体曲线获取方法
CN112509125A (zh) * 2020-12-14 2021-03-16 广州迦恩科技有限公司 一种基于人工标志物和立体视觉的三维重建方法
CN113132717A (zh) * 2019-12-31 2021-07-16 华为技术有限公司 数据处理方法、终端和服务器

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019169540A1 (zh) * 2018-03-06 2019-09-12 斯坦德机器人（深圳）有限公司 紧耦合视觉slam的方法、终端及计算机可读存储介质
CN110221690A (zh) * 2019-05-13 2019-09-10 Oppo广东移动通信有限公司 基于ar场景的手势交互方法及装置、存储介质、通信终端
CN113132717A (zh) * 2019-12-31 2021-07-16 华为技术有限公司 数据处理方法、终端和服务器
CN111627043A (zh) * 2020-04-13 2020-09-04 浙江工业大学 一种基于标志物和特征筛选器的简易人体曲线获取方法
CN112509125A (zh) * 2020-12-14 2021-03-16 广州迦恩科技有限公司 一种基于人工标志物和立体视觉的三维重建方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
TANG PU, HE FU, TANG CHUNLIN: "Design of Dynamic Modeling System Inside Special Test Equipment", COMPUTER MEASUREMENT ＆ CONTROL., vol. 28, no. 9, 25 September 2020 (2020-09-25), pages 229 - 232, XP093071175, DOI: 10.16526/j.cnki.11-4762/tp.2020.09.045 *

Also Published As

Publication number Publication date
CN116258812A (zh) 2023-06-13

Similar Documents

Publication Publication Date Title
CN112894832B (zh) 三维建模方法、装置、电子设备和存储介质
CN110568447B (zh) 视觉定位的方法、装置及计算机可读介质
JP7236565B2 (ja) 位置姿勢決定方法、装置、電子機器、記憶媒体及びコンピュータプログラム
JP5766795B2 (ja) 拡張現実環境のためのモバイルデバイスベースのコンテンツマッピング
CN112258567B (zh) 物体抓取点的视觉定位方法、装置、存储介质及电子设备
TW201915943A (zh) 對圖像內目標物體進行自動標註的方法、裝置及系統
EP2782027A1 (en) Apparatus and method providing augmented reality contents based on web information structure
CN113592989B (zh) 一种三维场景的重建系统、方法、设备及存储介质
CN104781849A (zh) 单眼视觉同时定位与建图(slam)的快速初始化
CN113741698A (zh) 一种确定和呈现目标标记信息的方法与设备
Gomez-Jauregui et al. Quantitative evaluation of overlaying discrepancies in mobile augmented reality applications for AEC/FM
CN112489099B (zh) 点云配准方法、装置、存储介质及电子设备
CN111986214B (zh) 一种地图中人行横道的构建方法和电子设备
CN111476894A (zh) 三维语义地图构建方法、装置、存储介质及电子设备
KR20200136723A (ko) 가상 도시 모델을 이용하여 객체 인식을 위한 학습 데이터 생성 방법 및 장치
CN111340960B (zh) 图像的建模方法、装置、存储介质和电子设备
WO2019164830A1 (en) Apparatus, systems, and methods for tagging building features in a 3d space
CN112733641A (zh) 物体尺寸测量方法、装置、设备及存储介质
CN110657760B (zh) 基于人工智能的测量空间面积的方法、装置及存储介质
WO2023103883A1 (zh) 对象自动标注方法、装置、电子设备及存储介质
CN112669392B (zh) 一种应用于室内视频监控系统的地图定位方法及系统
WO2023103884A1 (zh) 对象模型建立方法、装置、电子设备及存储介质
WO2023088127A1 (zh) 室内导航方法、服务器、装置和终端
CN114089836B (zh) 标注方法、终端、服务器和存储介质
CN117456550B (zh) 基于mr的cad文件查看方法、装置、介质和设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22903306

Country of ref document: EP

Kind code of ref document: A1