CN113724370A - Three-dimensional reconstruction method, device and system based on graphics processor - Google Patents

Three-dimensional reconstruction method, device and system based on graphics processor Download PDF

Info

Publication number
CN113724370A
CN113724370A CN202110889415.9A CN202110889415A CN113724370A CN 113724370 A CN113724370 A CN 113724370A CN 202110889415 A CN202110889415 A CN 202110889415A CN 113724370 A CN113724370 A CN 113724370A
Authority
CN
China
Prior art keywords
distance
actual marking
graphics processor
thread group
dimensional reconstruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110889415.9A
Other languages
Chinese (zh)
Other versions
CN113724370B (en
Inventor
吴昆临
许秋子
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Realis Multimedia Technology Co Ltd
Original Assignee
Shenzhen Realis Multimedia Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Realis Multimedia Technology Co Ltd filed Critical Shenzhen Realis Multimedia Technology Co Ltd
Priority to CN202110889415.9A priority Critical patent/CN113724370B/en
Publication of CN113724370A publication Critical patent/CN113724370A/en
Application granted granted Critical
Publication of CN113724370B publication Critical patent/CN113724370B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/20Processor architectures; Processor configuration, e.g. pipelining

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请公开了一种基于图形处理器的三维重建方法、装置及系统,其中方法包括:确定环境中的物体;确定物体的各实际标记点;针对每个实际标记点,将实际标记点与其他的实际标记点之间的最大距离,确定为实际标记点的指定距离;按照指定步长,对指定距离进行划分,得到指定数量个距离区间;将图形处理器的各线程划分为若干个线程组;针对每个距离区间,将动辅模块采集的对应于距离区间的数据,分配至距离区间对应的线程组;根据各线程组各自得到的处理结果,执行三维重建。本说明书中的方法有效的对图形处理器提供的数据处理能力进行利用,有利于提高数据处理效率。此外,图形处理器的多线程处理架构也有利于加快数据处理速度。

Figure 202110889415

The present application discloses a three-dimensional reconstruction method, device and system based on a graphics processor, wherein the method includes: determining an object in the environment; determining each actual marking point of the object; for each actual marking point, comparing the actual marking point with other The maximum distance between the actual marking points is determined as the specified distance of the actual marking points; according to the specified step size, the specified distance is divided to obtain a specified number of distance intervals; each thread of the graphics processor is divided into several thread groups ; For each distance interval, assign the data corresponding to the distance interval collected by the moving auxiliary module to the thread group corresponding to the distance interval; perform three-dimensional reconstruction according to the processing results obtained by each thread group. The method in this specification effectively utilizes the data processing capability provided by the graphics processor, which is beneficial to improve the data processing efficiency. In addition, the multi-threaded processing architecture of the graphics processor also helps to speed up data processing.

Figure 202110889415

Description

Three-dimensional reconstruction method, device and system based on graphics processor
Technical Field
The present application relates to the field of simulation display technologies, and in particular, to a three-dimensional reconstruction method, apparatus, and system based on a graphics processor.
Background
The english term name for three-dimensional Reconstruction is 3D Reconstruction. Three-dimensional reconstruction refers to the creation of suitable calculations for three-dimensional objectsThe mathematical model of machine representation and processing is the basis for processing, operating and analyzing the properties of the machine in a computer environment, and is also the key technology for establishing virtual reality expressing an objective world in a computer. The effect of three-dimensional reconstruction will directly affect the user experience. In the related art, it is assumed that one rigid body has NRA mark point, the identified 3D point has NSIn the three-dimensional reconstruction engineering, the three-dimensional reconstruction method needs to be executed for the rigid body
Figure BDA0003193688480000011
Secondly, the problems of long time consumption and low efficiency exist.
Disclosure of Invention
The embodiment of the application provides a three-dimensional reconstruction method, a three-dimensional reconstruction device and a three-dimensional reconstruction system based on a graphics processor, so as to solve the technical problems at least partially.
The embodiment of the application adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides a three-dimensional reconstruction method based on a graphics processor, where the method includes:
determining an object in the environment;
determining each actual marking point of the object;
determining the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point for each actual marking point;
dividing the designated distance according to the designated step length to obtain a designated number of distance intervals;
dividing each thread of the graphics processor into a plurality of thread groups, such that each distance interval corresponds to one thread group;
for each distance interval, distributing the data corresponding to the distance interval acquired by the auxiliary module to a thread group corresponding to the distance interval, so that the thread group processes the distributed data;
and executing three-dimensional reconstruction according to the processing result obtained by each thread group.
In an optional embodiment of the present description, the auxiliary module includes an auxiliary camera for image capturing of the object.
In an alternative embodiment of the present description, the graphics processor includes at least one of: GPU and multi-core CPU; and/or the presence of a gas in the gas,
the object is a rigid body.
In an optional embodiment of this specification, allocating data corresponding to the distance interval, acquired by the secondary module, to a thread group corresponding to the distance interval includes:
identifying each reference mark point of the object in each frame of image collected by the dynamic auxiliary module;
determining, for each reference marker point, the respective available distances between the reference marker point and the other reference marker points;
and allocating the available distance corresponding to the distance interval in the available distances to the thread group corresponding to the distance interval.
In an optional embodiment of the present specification, allocating an available distance corresponding to the distance interval in the available distances to a thread group corresponding to the distance interval includes:
determining an available distance smaller than the specified distance of the actual marking point in each available distance as an available distance corresponding to the actual marking point for each actual marking point;
determining an available distance matched with the distance interval from available distances corresponding to the actual marking points in the available distances corresponding to the actual marking points as an available distance corresponding to the distance interval for each distance interval of the actual marking points;
and distributing the available distance corresponding to the distance interval to the thread group corresponding to the distance interval.
In an optional embodiment of this specification, after performing three-dimensional reconstruction according to a processing result obtained by each thread group, the method further includes:
and displaying the 3D image obtained by three-dimensional reconstruction to a user.
In a second aspect, an embodiment of the present application further provides a three-dimensional reconstruction system based on a graphics processor, where the system includes a reconstruction processor and an active-auxiliary module;
the reconstruction processor is configured to: determining an object in the environment; determining each actual marking point of the object; determining the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point for each actual marking point; dividing the designated distance according to the designated step length to obtain a designated number of distance intervals; dividing each thread of the graphics processor into a plurality of groups, such that each distance interval corresponds to a thread group; for each distance interval, distributing the data corresponding to the distance interval acquired by the auxiliary module to a thread group corresponding to the distance interval, so that the thread group processes the distributed data; executing three-dimensional reconstruction according to the processing result obtained by each thread group;
the active-auxiliary module is configured to: the dynamic auxiliary module comprises a dynamic auxiliary camera which is used for acquiring images of the object.
In a third aspect, an embodiment of the present application further provides a three-dimensional reconstruction apparatus based on a graphics processor, configured to implement any one of the foregoing methods in the first aspect.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform any one of the methods of the first aspect.
In a fifth aspect, embodiments of the present application further provide a computer-readable storage medium storing one or more programs which, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform any of the methods of the first aspect.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects: in the three-dimensional reconstruction process in this specification, in the reconstruction process, the data processing capability provided by the threads of the graphics processor is divided according to the distance between the actual mark points on the object (i.e., the side length between the actual mark points), so as to obtain a plurality of thread groups, and different thread groups process data corresponding to different distance intervals. The data processing capacity provided by the graphic processor is effectively utilized, and the data processing efficiency is improved. In addition, the multithreading processing architecture of the graphics processor is also beneficial to increasing the data processing speed.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic diagram of a three-dimensional reconstruction process based on a graphics processor according to an embodiment of the present disclosure;
fig. 2 is a schematic view of a holographic display scene provided in an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a three-dimensional reconstruction apparatus based on a graphics processor according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
The graphics processor-based three-dimensional reconstruction method in the present specification may include the steps of:
s100: objects in the environment are determined.
In an alternative embodiment of the present description, the processes in the present description may be performed by a three-dimensional reconstruction system of a graphics processor. In particular, a three-dimensional reconstruction system of a graphics processor may include a reconstruction processor and an active auxiliary module, and the processes in this specification may be performed by the reconstruction processor.
The dynamic auxiliary module in the specification is used for acquiring images of a specified environment, different dynamic auxiliary modules are different in pose when acquiring the images of the specified environment, and the images acquired by the dynamic auxiliary module comprise a plurality of two-dimensional points.
The three-dimensional reconstruction process in this specification does not specifically limit the objects in the environment involved. Alternatively, the objects in the environment may be two or more.
The dynamic assistant module in this specification may be a physical camera having a physical structure, or may be a virtual camera corresponding to a viewpoint of a user.
The specific circumstances in this specification may be determined according to actual needs. Illustratively, the specified environment may be a certain room, or a certain area in a certain room.
When the three-dimensional reconstruction process based on the graphic processor in this specification is applied to the field of the holographic display technology, the specified environment may be an environment in which a holographic sand table (sand table display device) is located, as shown in fig. 2 for example. In this example, the execution subject of the process in this specification may be a control module of a holographic presentation system to which the holographic sand table belongs.
S102: determining each actual marking point of the object.
The actual marking point mode can be obtained through a manual calibration mode, and then the calibrated actual marking point is input into a reconstruction processor.
Since the designated environment is a certain space, the actual marked points in the designated environment can be represented by three-dimensional coordinates. Optionally, in the holographic display scene, an origin of a three-dimensional coordinate system for determining three-dimensional coordinates is at the center of the holographic sand table.
S104: for each actual marking point, determining the maximum distance between the actual marking point and other actual marking points as the designated distance (L) of the actual marking pointMax)。
Suppose there is N on an objectRAn actual mark point corresponds to an object
Figure BDA0003193688480000061
And (4) arranging edges. The edges of these rigid bodies are matched to the N identified by the systemsThe edges of each reference mark point are provided with
Figure BDA0003193688480000062
And (4) respectively. The edge of each actual mark point is longer than the edge of the reference mark point, if the lengths of the two reference mark points are the same, the two current reference mark points correspond to the two current actual mark points, and then the two actual mark points are displayed on the screenA match may be made. Thereafter, a three-dimensional reconstruction may be performed based on the matched reference marker points.
Since a plurality of actual marking points are determined for each object, the designated distance of each actual marking point needs to be determined.
S106: and dividing the specified distance according to the specified step length to obtain a specified number of distance intervals.
For example: l isMaxThe value of (d) is 1.0 meter and the specified step size is 0.1 meter. 10 individual distance intervals are obtained, the specified number being 10.
S108: the threads of the graphics processor are divided into thread groups such that each distance interval corresponds to a thread group.
For example, each thread group corresponds to a bucket, divided into 10 buckets, bucket 1 contains 0 to 0.1 meter long edges, bucket 2 contains 0.1 to 0.2 meter long edges, and so on. Alternatively, the threads may be divided equally into thread groups. The resulting number of thread groups is greater than or equal to the specified number.
Side length (i.e. distance) less than LMaxThe edges of (a) can be allocated to different buckets by specific lengths, and the operation can also realize multithread acceleration.
S110: and for each distance interval, distributing the data corresponding to the distance interval acquired by the auxiliary module to the thread group corresponding to the distance interval, so that the thread group processes the distributed data.
When performing the processing, if the distance between the reference mark points is 0.15 m long, only the sides of the reference mark points in the bucket 2 need to be matched, greatly reducing the amount of calculation.
Optionally, the graphics processor is a GPU. The graphics processor processes all points in the data simultaneously to increase the computation speed. The GPU has the advantage over a Central Processing Unit (CPU) that a large number of threads can be generated, which reduces the total computation time when computing in large numbers in parallel.
S112: and executing three-dimensional reconstruction according to the processing result obtained by each thread group.
And then, displaying the 3D image obtained by three-dimensional reconstruction to a user.
Therefore, through the three-dimensional reconstruction process based on the graphics processor in the specification, a corresponding relationship can be established between the thread group and each distance interval, and the reasonable distribution of the processing resources of the graphics processing among the distance intervals is realized.
Since the rigid body is mobile, the process is applied to each acquired frame image. Generation of RSThe side lengths of all 3D points are calculated by each thread. If the side length is less than LMaxAnd reserving the reference mark points corresponding to the edges. Less than LMaxNumber of sides (by E'STo represent) is much smaller than ES. Using multiple threads to make a side length less than LMaxIs collected to another variable, and occupies only E 'of memory'sLength of (E) compared to occupation of E at subsequent accessesSThe long memory size reduces the computation time by reducing the memory bandwidth required to access the data.
In an optional embodiment of the present description, the auxiliary module includes an auxiliary camera for image capturing of the object. Further, the graphics processor may be a multi-core CPU and the object may be a rigid body. Each reference marker point corresponds to an actual marker point on the object.
When the collected image packet comprises a plurality of frames which are continued according to time, each reference mark point of the object in each frame of image collected by the auxiliary module can be identified; determining, for each reference marker point, the respective available distances between the reference marker point and the other reference marker points; determining an available distance smaller than the specified distance of the actual marking point in each available distance as an available distance corresponding to the actual marking point for each actual marking point; determining an available distance matched with the distance interval from available distances corresponding to the actual marking points in the available distances corresponding to the actual marking points as an available distance corresponding to the distance interval for each distance interval of the actual marking points; and distributing the available distance corresponding to the distance interval to the thread group corresponding to the distance interval.
Based on the same idea, the embodiments of the present specification further provide a graphics processor-based three-dimensional reconstruction apparatus corresponding to the partial process shown in fig. 1. As shown in fig. 3, the graphics processor-based three-dimensional reconstruction apparatus may include one or more of the following modules:
an object determination module 300 configured to: objects in the environment are determined.
An actual marker determination module 302 configured to: determining each actual marking point of the object.
A specified distance determination module 304 configured to: and determining the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point for each actual marking point.
A distance interval determination module 306 configured to: and dividing the specified distance according to the specified step length to obtain a specified number of distance intervals.
A thread group determination module 308 configured to: the threads of the graphics processor are divided into thread groups such that each distance interval corresponds to a thread group.
An assignment module 310 configured to: and for each distance interval, distributing the data corresponding to the distance interval acquired by the auxiliary module to the thread group corresponding to the distance interval, so that the thread group processes the distributed data.
A three-dimensional reconstruction module 312 configured to: and executing three-dimensional reconstruction according to the processing result obtained by each thread group.
In an optional embodiment of the present description, the auxiliary module includes an auxiliary camera for image capturing of the object.
In an alternative embodiment of the present description, the graphics processor includes at least one of: GPU and multi-core CPU; and/or, the object is a rigid body.
In an optional embodiment of the present description, the allocation module is specifically configured to: identifying each reference mark point of the object in each frame of image collected by the dynamic auxiliary module;
determining, for each reference marker point, the respective available distances between the reference marker point and the other reference marker points;
and allocating the available distance corresponding to the distance interval in the available distances to the thread group corresponding to the distance interval.
In an optional embodiment of the present description, the allocation module is specifically configured to: determining an available distance smaller than the specified distance of the actual marking point in each available distance as an available distance corresponding to the actual marking point for each actual marking point;
determining an available distance matched with the distance interval from available distances corresponding to the actual marking points in the available distances corresponding to the actual marking points as an available distance corresponding to the distance interval for each distance interval of the actual marking points;
and distributing the available distance corresponding to the distance interval to the thread group corresponding to the distance interval.
In an alternative embodiment of the present description, the apparatus further comprises a display module. And the display module is configured to display the three-dimensional reconstructed 3D image to a user.
It can be understood that, the three-dimensional reconstruction apparatus based on a graphics processor can implement the steps of the three-dimensional reconstruction process based on a graphics processor performed by the three-dimensional reconstruction apparatus based on a graphics processor provided in the foregoing embodiments, and the related explanations regarding the three-dimensional reconstruction method based on a graphics processor are applicable to the three-dimensional reconstruction apparatus based on a graphics processor, and are not repeated herein.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 4, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form a three-dimensional reconstruction device based on the graphics processor on a logic level. And the processor is used for executing the program stored in the memory and is particularly used for executing any one of the three-dimensional reconstruction processes based on the graphics processor.
The method performed by the three-dimensional reconstruction device based on the graphics processor according to the embodiment shown in fig. 1 of the present application can be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may further perform at least a part of the method steps performed by the three-dimensional reconstruction apparatus based on the graphics processor in fig. 1, and implement a three-dimensional reconstruction apparatus based on the graphics processor, which is not described herein again in this embodiment of the present application.
The present application also provides a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the method performed by the graphics processor-based three-dimensional reconstruction apparatus in the embodiment shown in fig. 1, and in particular to perform any one of the aforementioned graphics processor-based three-dimensional reconstruction methods.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1.一种基于图形处理器的三维重建方法,其中,所述方法包括:1. A three-dimensional reconstruction method based on a graphics processor, wherein the method comprises: 确定环境中的物体;Identify objects in the environment; 确定所述物体的各实际标记点;determining the actual marking points of the object; 针对每个实际标记点,将所述实际标记点与其他的实际标记点之间的最大距离,确定为所述实际标记点的指定距离;For each actual marking point, determine the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point; 按照指定步长,对所述指定距离进行划分,得到指定数量个距离区间;According to the specified step size, the specified distance is divided to obtain a specified number of distance intervals; 将所述图形处理器的各线程划分为若干个线程组,使得每个距离区间对应于一个线程组;dividing each thread of the graphics processor into several thread groups, so that each distance interval corresponds to one thread group; 针对每个距离区间,将动辅模块采集的对应于所述距离区间的数据,分配至所述距离区间对应的线程组,使得所述线程组对分配到的数据进行处理;For each distance interval, the data corresponding to the distance interval collected by the moving auxiliary module is allocated to the thread group corresponding to the distance interval, so that the thread group processes the allocated data; 根据各线程组各自得到的处理结果,执行三维重建。Three-dimensional reconstruction is performed according to the processing results obtained by each thread group. 2.如权利要求1所述方法,其中,所述动辅模块包括动辅相机,所述动辅相机用于对所述物体进行图像采集。2 . The method of claim 1 , wherein the moving auxiliary module comprises a moving auxiliary camera, and the moving auxiliary camera is used for image acquisition of the object. 3 . 3.如权利要求1所述方法,其中,所述图形处理器包括以下至少一种:GPU、多核CPU;和/或,3. The method of claim 1, wherein the graphics processor comprises at least one of the following: a GPU, a multi-core CPU; and/or, 所述物体是刚体。The object is a rigid body. 4.如权利要求1所述方法,其中,将动辅模块采集的对应于所述距离区间的数据,分配至所述距离区间对应的线程组,包括:4. The method according to claim 1, wherein the data corresponding to the distance interval collected by the moving auxiliary module is allocated to the thread group corresponding to the distance interval, comprising: 针对动辅模块采集的每一帧图像,识别出所述物体在该帧图像中的各参照标记点;For each frame of image collected by the motion-assist module, identify each reference mark point of the object in the frame of image; 针对每个参照标记点,确定该参照标记点与其他的参照标记点之间的各可用距离;for each reference marker, determining the available distances between the reference marker and other reference markers; 将所述可用距离中对应于所述距离区间的可用距离,分配至所述距离区间对应的线程组。Allocating available distances corresponding to the distance intervals among the available distances to thread groups corresponding to the distance intervals. 5.如权利要求4所述方法,其中,将所述可用距离中对应于所述距离区间的可用距离,分配至所述距离区间对应的线程组,包括:5. The method according to claim 4, wherein allocating the available distances corresponding to the distance intervals in the available distances to the thread groups corresponding to the distance intervals, comprising: 针对每个实际标记点,将各可用距离中小于所述实际标记点的指定距离的可用距离,确定为所述实际标记点对应的可用距离;For each actual marking point, determine the available distance that is less than the specified distance of the actual marking point among the available distances as the available distance corresponding to the actual marking point; 针对所述实际标记点的每个距离区间,在所述实际标记点对应的可用距离中,确定出与所述距离区间匹配的可用距离,作为所述距离区间对应的可用距离;For each distance interval of the actual marker point, among the available distances corresponding to the actual marker point, determine the available distance matching the distance interval as the available distance corresponding to the distance interval; 将所述距离区间对应的可用距离,分配至所述距离区间对应的线程组。The available distance corresponding to the distance interval is allocated to the thread group corresponding to the distance interval. 6.如权利要求1所述方法,其中,根据各线程组各自得到的处理结果,执行三维重建之后,所述方法还包括:6. The method according to claim 1, wherein, after performing the three-dimensional reconstruction according to the processing results obtained by each thread group, the method further comprises: 将三维重建得到的3D影像,展示至用户。The 3D image obtained by the 3D reconstruction is displayed to the user. 7.一种基于图形处理器的三维重建系统,所述系统包括重建处理器和动辅模块;7. A three-dimensional reconstruction system based on a graphics processor, the system comprising a reconstruction processor and a motion-assist module; 所述重建处理器,配置为:确定环境中的物体;确定所述物体的各实际标记点;针对每个实际标记点,将所述实际标记点与其他的实际标记点之间的最大距离,确定为所述实际标记点的指定距离;按照指定步长,对所述指定距离进行划分,得到指定数量个距离区间;将所述图形处理器的各线程划分为若干个组,使得每个距离区间对应于一个线程组;针对每个距离区间,将动辅模块采集的对应于所述距离区间的数据,分配至所述距离区间对应的线程组,使得所述线程组对分配到的数据进行处理;根据各线程组各自得到的处理结果,执行三维重建;The reconstruction processor is configured to: determine an object in the environment; determine each actual marking point of the object; for each actual marking point, determine the maximum distance between the actual marking point and other actual marking points, Determine the specified distance of the actual marking point; divide the specified distance according to the specified step size to obtain a specified number of distance intervals; divide each thread of the graphics processor into several groups, so that each distance The interval corresponds to a thread group; for each distance interval, the data corresponding to the distance interval collected by the moving auxiliary module is assigned to the thread group corresponding to the distance interval, so that the thread group performs the data on the assigned data. processing; according to the processing results obtained by each thread group, perform three-dimensional reconstruction; 所述动辅模块,配置为:所述动辅模块包括动辅相机,所述动辅相机用于对所述物体进行图像采集。The moving auxiliary module is configured as follows: the moving auxiliary module includes a moving auxiliary camera, and the moving auxiliary camera is used for image acquisition of the object. 8.一种基于图形处理器的三维重建装置,所述装置用于实现权利要求1~6之任一所述方法。8 . A three-dimensional reconstruction device based on a graphics processor, which is used to implement the method of any one of claims 1 to 6 . 9.一种非易失性计算机可读存储介质,其特征在于,所述非易失性计算机可读存储介质存储有计算机可执行指令,该计算机可执行指令被一个或多个处理器执行时,可使得所述一个或多个处理器执行权利要求1~6之任一所述的方法。9. A non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions that when executed by one or more processors , which can cause the one or more processors to perform the method of any one of claims 1-6. 10.一种电子设备,包括:10. An electronic device comprising: 处理器;以及processor; and 被安排成存储计算机可执行指令的存储器,所述可执行指令在被执行时使所述处理器执行所述权利要求1~6之任一所述方法。A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the method of any one of claims 1-6.
CN202110889415.9A 2021-08-03 2021-08-03 A three-dimensional reconstruction method, device and system based on graphics processor Active CN113724370B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110889415.9A CN113724370B (en) 2021-08-03 2021-08-03 A three-dimensional reconstruction method, device and system based on graphics processor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110889415.9A CN113724370B (en) 2021-08-03 2021-08-03 A three-dimensional reconstruction method, device and system based on graphics processor

Publications (2)

Publication Number Publication Date
CN113724370A true CN113724370A (en) 2021-11-30
CN113724370B CN113724370B (en) 2024-11-08

Family

ID=78674807

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110889415.9A Active CN113724370B (en) 2021-08-03 2021-08-03 A three-dimensional reconstruction method, device and system based on graphics processor

Country Status (1)

Country Link
CN (1) CN113724370B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104501740A (en) * 2014-12-18 2015-04-08 杭州鼎热科技有限公司 Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking
CN110728754A (en) * 2019-10-10 2020-01-24 深圳市瑞立视多媒体科技有限公司 Rigid body mark point identification method, device, equipment and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104501740A (en) * 2014-12-18 2015-04-08 杭州鼎热科技有限公司 Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking
CN110728754A (en) * 2019-10-10 2020-01-24 深圳市瑞立视多媒体科技有限公司 Rigid body mark point identification method, device, equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YANG JU ET AL: "Multi-thread parallel algorithm for Reconstruction 3d large-scale porous Structures", 《COMPUTERS & GEOSCIENCES》, 31 December 2017 (2017-12-31), pages 10 - 20 *
李响: "浅海水下目标物检测与位姿估计方法研究", 《中国硕士学位论文全文数据库 基础科学》, no. 2, 15 February 2020 (2020-02-15), pages 1 - 74 *
路子;高隽;韩晓新;李援;卢鹏;: "反馈调节的局部正则化鲁棒光流估计", 光子学报, no. 08, 15 August 2008 (2008-08-15), pages 1703 - 1707 *

Also Published As

Publication number Publication date
CN113724370B (en) 2024-11-08

Similar Documents

Publication Publication Date Title
US12217151B2 (en) Layout parasitics and device parameter prediction using graph neural networks
US10140123B2 (en) SIMD processing lanes storing input pixel operand data in local register file for thread execution of image processing operations
CN110675480B (en) Method and apparatus for acquiring sampling position of texture operation
CN106779057B (en) Method and device for calculating binary neural network convolution based on GPU
US20200210805A1 (en) Neural Network Generator
CN112288619A (en) Technique for preloading textures when rendering graphics
CN117271136B (en) Data processing method, device, equipment and storage medium
US9626285B2 (en) Storage resource allocation to dataflows based on data requirements and attributes
WO2022166293A1 (en) Target detection method and apparatus
CN106484532B (en) GPGPU parallel calculating method towards SPH fluid simulation
CN108280135B (en) Method and device for realizing visualization of data structure and electronic equipment
US10593103B2 (en) Method and apparatus for managing graphics layers within a data processing system
CN110866127A (en) Method for establishing index and related device
CN113724370A (en) Three-dimensional reconstruction method, device and system based on graphics processor
CN118519768A (en) Method, device, equipment and storage medium for overflowing data to shared buffer memory
US9183435B2 (en) Feature generalization using topological model
CN114860460B (en) Database acceleration method and device and computer equipment
US20220188380A1 (en) Data processing method and apparatus applied to graphics processing unit, and electronic device
CN113610959B (en) A three-dimensional reconstruction method and device based on graphics processor
CN109522238B (en) Sorting method and device
US12034852B2 (en) Block operation method, electronic device, and storage medium
CN103559263A (en) Method and system for merging plurality of images
HK40080368A (en) Processing method of graph data, device, electronic equipment and storage medium
CN115619631A (en) Graph data processing method and device, electronic equipment and storage medium
CN114972676A (en) Grid construction method, apparatus, electronic device, and computer-readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant