WO2022052893A1 - 一种5d纹理网格数据结构 - Google Patents

一种5d纹理网格数据结构 Download PDF

Info

Publication number
WO2022052893A1
WO2022052893A1 PCT/CN2021/116728 CN2021116728W WO2022052893A1 WO 2022052893 A1 WO2022052893 A1 WO 2022052893A1 CN 2021116728 W CN2021116728 W CN 2021116728W WO 2022052893 A1 WO2022052893 A1 WO 2022052893A1
Authority
WO
WIPO (PCT)
Prior art keywords
point
point cloud
row
points
data structure
Prior art date
Application number
PCT/CN2021/116728
Other languages
English (en)
French (fr)
Inventor
高磊
田希文
Original Assignee
熵智科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 熵智科技(深圳)有限公司 filed Critical 熵智科技(深圳)有限公司
Publication of WO2022052893A1 publication Critical patent/WO2022052893A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
    • B07C5/3422—Sorting according to other particular properties according to optical properties, e.g. colour using video scanning devices, e.g. TV-cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Definitions

  • the present disclosure belongs to the technical field of computer graphics and computer vision, and in particular, relates to a grid data structure used to characterize three-dimensional objects by gridding point cloud data.
  • Robot disorder sorting In the fields of industrial manufacturing and logistics, unordered sorting tasks such as machine tool loading and unloading, parts sorting, and goods dismantling and picking are high labor-intensity, low sorting efficiency, and difficult to recruit for traditional manual operations.
  • Robot disorder sorting technology is increasingly replacing manual labor, and has become an important means for related industries to reduce costs, increase efficiency, and upgrade intelligently.
  • Robot disorder sorting relies on visual recognition to locate objects to be sorted, and then guide the end of the robot to complete the grasping and placing operations of objects.
  • 2D images, 3D point clouds or grids are used to represent object information.
  • the visual recognition methods of objects can be divided into 2D methods and 3D methods.
  • the workload of the template is too large; when the workpieces to be inspected are closely placed, it is difficult for the 3D vision method to segment different objects in the scene, and it is difficult to adapt to the workpieces to be inspected with small changes in the shape of the objects, such as recognizing the packaging of mobile phones that are closely arranged in the turnover box box.
  • Only by identifying the complete shape of the workpiece can the workpiece be accurately stacked and avoid damage to the workpiece when grasping the valuable workpiece, while the traditional 2D/3D visual recognition methods can only be based on the local information on the upper surface of the workpiece. It is difficult to judge the complete shape of the workpiece to be tested, and it is difficult to adapt to the accurate stacking.
  • the object of the present invention is to provide a grid data structure (SeizetColorMesh) for characterizing three-dimensional objects, which is used to gridize point clouds. It can effectively represent the all-round information of three-dimensional objects, so both 2D and 3D image processing algorithms can be applied, thus achieving a more powerful perception and recognition effect, and also laying a data structure foundation for robots to face massive disordered workpiece grasping. .
  • a grid data structure SeizetColorMesh
  • the present disclosure provides a mesh data structure for characterizing a three-dimensional object, the mesh data structure being used for quadrangular mesh reconstruction of point cloud data, the mesh data structure including a plurality of half-edge quadrangles A grid, each of the half-side quadrangular grids includes a quadrilateral patch, four vertices S mn , S m,n+1 , S m+1,n , S m+1,n arranged on the quadrangular patch +1 and multiple half-edges, wherein the corresponding four vertices are obtained based on any point P i,j and the three adjacent points of P i,j in the point cloud data, and each of the vertices stores a point cloud
  • obtaining the corresponding four vertices based on any point P i,j and the three adjacent points of P i,j in the point cloud data includes:
  • the point cloud is an ordered point cloud
  • obtaining the row and column ordering of each point in the point cloud includes: traversing the ordered point cloud, and reading the row of each point P i,j in the ordered point cloud. Coordinate i and column coordinate j to obtain the row and column order of each point in the point cloud data.
  • the point cloud is an unordered point cloud, and obtaining the row and column ordering of each point in the point cloud is to sort the points in the point cloud, and obtaining each point P i,j in the point cloud data respectively. Sort by row and by column.
  • the obtaining by row sorting includes: for any two points in the disordered point cloud, sorting the points P i, j in the disordered point cloud according to the size of x/z and y/z, and comparing the two points. If the y/z value of the two is the same, compare the size of the x/z value, and arrange the point with the larger x/z value at the back ; If the two x/z values are also the same, delete one of them to obtain the row-by-row sorting;
  • the obtaining by column sorting includes: for any two points in the unordered point cloud, sorting the points P i, j in the unordered point cloud according to the sizes of x/z and y/z, and comparing two points. If the x/z value of the two is the same, the y/z value is compared, and the point with the larger y/z value is placed in the back. Later; if both y/z values are the same, delete one of them to get the sort by column.
  • search for the adjacent points P i,j+1 ,P of P i,j in the unordered point cloud according to the index rules in the row sorting and the column sorting i+1,j+1 , P i+1,j include the following steps:
  • the four-sided patch whose side length exceeds the threshold is deleted.
  • a new quadrilateral patch is created and four vertices are added to the quadrilateral patch in a predetermined order to generate Corresponding half-edge quadrangular grid.
  • the vertices are added in clockwise or counterclockwise order.
  • the present disclosure provides a mesh data structure for characterizing a three-dimensional object, the mesh data structure being used for quadrangular mesh reconstruction of point cloud data, the mesh data structure comprising topologically connected multiple four quadrangular grids, each of which includes a quadrangular patch, four vertices S m,n , S m,n+1 , S m+1,n , S m+ set on the quadrangular patch 1,n+1 , wherein the corresponding four vertices are obtained based on any point P i,j and the three adjacent points of P i,j in the point cloud data, and each of the vertices stores point cloud data
  • the 3D position coordinates (x, y, z) of the corresponding points P i, j in the middle, and the grid data structure also stores the adjacency relationship, and the adjacency relationship is the topological relationship between adjacent vertices and the adjacent vertices.
  • obtaining the corresponding four vertices based on any point P i,j and the three adjacent points of P i,j in the point cloud data includes:
  • the point cloud is an ordered point cloud
  • obtaining the row and column ordering of each point in the point cloud includes: traversing the ordered point cloud, and reading the row of each point P i,j in the ordered point cloud. Coordinate i and column coordinate j to obtain the row and column order of each point in the point cloud data.
  • the point cloud is an unordered point cloud, and obtaining the row and column ordering of each point in the point cloud is to sort the points in the point cloud, and obtaining each point P i,j in the point cloud data respectively. Sort by row and by column.
  • the obtaining by row sorting includes: for any two points in the unordered point cloud, sorting the points P i, j in the unordered point cloud according to the sizes of x/z and y/z, and comparing two points. If the y/z value of the two is the same, compare the size of the x/z value, and rank the point with the larger x/z value in the back. Later; if the x/z values of the two are also the same, delete one of them to obtain the row-by-row sorting;
  • the obtaining by column sorting includes: for any two points in the unordered point cloud, sorting the points P i, j in the unordered point cloud according to the sizes of x/z and y/z, and comparing two points. If the x/z value of the two is the same, the y/z value is compared, and the point with the larger y/z value is placed in the back. Later; if both y/z values are the same, delete one of them to get the sort by column.
  • search for the adjacent points P i,j+1 ,P of P i,j in the unordered point cloud according to the index rules in the row sorting and the column sorting. i+1,j+1 , P i+1,j include the following steps:
  • the four-sided patch whose side length exceeds the threshold is deleted.
  • a new four-corner patch is created and four vertices are added to the four-corner patch in a predetermined order to generate corresponding four-corner grid.
  • the vertices are added in clockwise or counterclockwise order.
  • the vertex also stores one or more combinations of normal vector, color, curvature, and flag bit of the corresponding point P i,j .
  • the present disclosure provides a method for generating a mesh data structure for characterizing a three-dimensional object, wherein the mesh data structure includes a plurality of half-edge quadrangular meshes, each of the half-edge quadrangular meshes It includes a quadrangular patch, four vertices S mn , S m,n+1 , S m+1,n , S m+1,n+1 and a plurality of half edges set on the quadrangular patch,
  • the method includes:
  • each vertex For each vertex, store the 3D position coordinates (x, y, z), normal vector, color, and pointer information of the half edge of the corresponding point P i, j in the point cloud data, and each half edge stores Pointer or index of start vertex, end vertex, adjoining face, previous half edge, next half edge, opposite half edge.
  • the present disclosure provides a method for generating a mesh data structure for characterizing a three-dimensional object, wherein the mesh data structure comprises a plurality of topologically connected quadrangular meshes, each of the quadrangular meshes
  • the lattice includes a quadrangular facet, four vertices S m,n , S m,n+1 , S m+1,n , S m+1,n+1 , S m,n+1 , S m+1,n+1 .
  • the method includes:
  • each said vertex For each said vertex, store the 3D position coordinates (x, y, z) of the corresponding point P i, j in the point cloud data; and store the adjacency relationship, which is the topological relationship between adjacent vertices , one or more combinations of the topological relationship between adjacent vertices and four-corner patches, and the topological relationship between adjacent four-corner patches and four-corner patches.
  • the present disclosure provides a neighborhood access method, comprising:
  • the method according to the embodiment of the present invention generates a mesh data structure for characterizing the three-dimensional object
  • Neighborhood access is performed on the mesh data structure to obtain information representing adjacent points of vertices in the mesh structure of the three-dimensional object.
  • the three-dimensional object based on the neighborhood visit, for example, based on the acquired information of the adjacent points of the vertices in the grid structure, it can be used for the detection of the three-dimensional object, including but not limited to the three-dimensional object.
  • the three-dimensional object may be manipulated based on the detection of the three-dimensional object, including but not limited to loading and unloading, sorting, unpacking, grabbing, placing and stacking of the three-dimensional object. one or more.
  • the present disclosure provides an electronic device, comprising: a processor and a memory storing a computer program, the processor is configured to run the computer program according to the method described in the embodiments of the present invention.
  • the present disclosure provides a storage medium, wherein the storage medium stores a computer program, and the computer program is configured to execute the method according to the embodiment of the present invention when executed.
  • the grid data structure shown in the present disclosure is used for the gridization of point clouds.
  • the point cloud is converted into a grid data structure, in addition to the original information of the point cloud, it also uses vertices, half edges, and four corners.
  • the connection relationship of the slice is indexed, which can effectively represent the comprehensive information of the three-dimensional object.
  • SeizetColorMesh proposed in this disclosure, after the 3D point cloud is converted into SeizetColorMesh, all algorithms such as 2D and 3D can be comprehensively used for the subsequent processing of point cloud data, which can achieve a more powerful perception and recognition effect, and then provide robots with a massive amount of The out-of-order artifact grabbing lays the foundation for the data structure.
  • the omni-directional template based on SeizetColorMesh can provide the complete shape and texture information of the object to be recognized (including the side and bottom surface), which improves the positioning accuracy, reliability and versatility, thereby enabling the robot to accurately grasp the massive disordered workpiece scene. and stacking, such as realizing the recognition and positioning of objects with large posture inclination and cluttered stacking scenes, etc.
  • the 2D image processing method can be extended to SeizetColorMesh, which can realize the segmentation of different objects in scenes where objects are closely placed.
  • FIG. 1 is a schematic structural diagram of a 5D texture mesh data structure according to an embodiment of the disclosure
  • FIG. 2 is a schematic structural diagram of a half-side quadrangular grid according to an embodiment of the present invention.
  • Fig. 3 is the disordered point cloud of the carton of the object to be identified
  • Fig. 4 is the 5D texture mesh data structure generated based on the disordered point cloud shown in Fig. 3;
  • FIG. 5 is an enlarged view of the structure in the dashed frame in FIG. 4 .
  • the mesh data structure may be referred to as a 5D texture mesh data structure, and by way of explanation and not limitation, the 5D means that the texture mesh data can be applied to both 2D (eg, 2D) texture) and 3D image processing algorithms.
  • point cloud data refers to a set of vectors in a three-dimensional coordinate system, including three-dimensional coordinates X, Y, Z , color, normal vector, curvature, and more.
  • the point cloud may include an unordered point cloud and an ordered point cloud.
  • the point cloud can be directly obtained by measurement, and each point can correspond to a measurement point.
  • the point cloud has not undergone other processing methods, so it contains the largest amount of information. This information is hidden in the point cloud and can be extracted by various extraction methods.
  • the extraction point The process of information in the cloud may be referred to as "3D point cloud processing".
  • the method may further include the step of acquiring a 3D point cloud, and in some embodiments, the acquisition of the 3D point cloud may be based on a 3D point cloud acquisition device, which may be obtained, for example, via 3D camera acquisition.
  • 2D image also referred to as a two-dimensional image, refers to a flat image that does not contain depth information.
  • “2D image processing” may include providing results based on characteristics of contrast in grayscale or color images, for various two-dimensional geometric analyses based on edge detection, and the like.
  • HalfEdge refers to a directed connection between vertices, such as but not limited to recording its end vertex and start vertex through TerminateVertex and OriginVertex pointers, respectively.
  • the present disclosure provides a mesh data structure for characterizing a three-dimensional object, which may be referred to as a 5D texture mesh data structure (SeizetColorMesh), for example, the 5D texture mesh data structure uses vertices, half edges, The connection relationship of the four corners is indexed, which is used for the four-corner grid reconstruction of the point cloud data.
  • the point cloud is represented based on the grid data structure, it not only retains the information stored in the point cloud, but also has a topology structure (including vertices, The mutual access method between the half-side and four-corner patches) can effectively represent the all-round information of three-dimensional objects.
  • the 5D texture mesh data structure includes a plurality of half-edge quadrangular meshes, each half-edge quadrangular mesh includes a quadrangular patch, and four vertices S m,n , S m,n+1 .S m set on the quadrangular patch +1,n , S m+1,n+1 and multiple half-edges, wherein, based on the point cloud data Pi ,j and the three adjacent points of Pi,j , four corresponding four-corner grids for each half-edge are obtained Vertex, each of which stores the 3D position coordinates (x, y, z), normal vector, color, and the pointer information of the half edge of the corresponding point P i, j in the point cloud data, and each half edge stores Pointer or index to the start vertex, end vertex, adjoining face, previous half edge, next half edge, opposite half edge.
  • each half-edge quadrilateral mesh generated based on point cloud data includes vertex 1), half-edge 2) and quadrilateral patch 3).
  • vertex S(1,0), S( 1,1), S(2,1), and S(2,0) are simplified to vertices A, B, C, and D, where AB, BC, CD, and DA are all half-edges, and ABCD is a quadrilateral patch.
  • Vertex (Vertex) A, B, C, D stores the 3D coordinates, normal vector, color and other information of corresponding points P i,j+1 , P i+1,j+1 , P i+1,j ;
  • half edge ( HalfEdge) is a directed connection between vertices, and its termination vertex and start vertex are recorded through TerminateVertex and OriginVertex pointers, respectively.
  • the termination vertex and start vertex of half edge AB are B and A, respectively.
  • the half-edge records its next half-edge and the previous half-edge respectively through the NextHalfEdge and PrevHalfEdge pointers.
  • the next and upper half-edges of the half-edge AB in Figure 2 are BC and DA respectively; the half-edge records its opposite half-edge through the OppositeHalfEdge pointer, and the half-edge AB in Figure 2
  • the opposite half-edge is BA; the half-edge records the adjacent four-corner patch through the AdjacentFace pointer.
  • the adjacent four-corner patch of the half-edge AB is ABCD; the vertex records the half-edge and the incoming half-edge respectively through the OutgoingHalfEdge and IncomingHalfEdge pointers, and the vertex A in Figure 2
  • the out and in half sides are AB and BA, respectively.
  • the QuadFace is composed of 4 sequentially connected half edges.
  • the quadrangle ABCD is composed of the half edges AB, BC, CD, and DA; the quadrangle is recorded by the OuterHalfEdge and InnerHalfEdge pointers.
  • the outer and inner halves of the four-corner patch ABCD in 2 are BA and AB, respectively.
  • a 5D texture mesh data structure shown in the present disclosure is used to represent all-round information of a three-dimensional object.
  • the access method between the corresponding neighborhood points builds a bridge for the promotion of 2D image processing methods to 3D morphology.
  • SeizetColorMesh proposed in the present disclosure, algorithms in 2D image processing and 3D point cloud processing can be comprehensively used to achieve a more powerful perception and recognition effect, thereby laying a data structure foundation for robots to grasp workpieces in a massive disordered state.
  • the omnidirectional template based on SeizetColorMesh can provide the complete shape and texture information of the object to be recognized (including side and bottom surfaces), The positioning accuracy, reliability and versatility are improved, so that the robot can be accurately grasped and stacked in the scene of massive disordered workpieces, that is, the recognition and positioning of objects with large inclination angles and cluttered stacking scenes can be realized;
  • the image processing method is extended to SeizetColorMesh. For example, 2D image processing method can be used to process SeizetColorMesh to realize the segmentation of different objects in scenes where objects are closely placed.
  • the four vertices of each half-edge quadrangular mesh are obtained according to the following steps:
  • S102 obtains the row and column order of each point in the point cloud
  • S104 obtains the four vertices S mn , S m,n+1 corresponding to P i,j , P i,j+1 , P i+1,j+1 , P i+1,j respectively according to the index relationship.
  • S m+1,n , S m+1,n+1 are the four vertices S mn , S m,n+1 corresponding to P i,j , P i,j+1 , P i+1,j respectively according to the index relationship.
  • step S103 for any point P i,j , if the corresponding three adjacent points P i,j+1 , P i+1,j+1 , P i+1,j cannot be found, then jump to Search for adjacent points through P i,j , and continue to search for adjacent points of the next point in the point cloud.
  • a new quadrilateral patch is created and four vertices are added to the quadrilateral patch in a predetermined order to generate the corresponding half-edge quadrilateral grid.
  • the topological relationship of these four vertices is established, that is, the four-corner patch.
  • the order of adding the vertices of the four-corner patch is S m,n , S m+1,n , S m+1,n+1 , S m,n+1 or S m, n , S m,n+1 , S m+1,n+1 , S m+1,n , i.e.
  • i row j column, i+1 row j column, i+1 row j in the point cloud data +1 column, i-th row j+1 column or i-th row j-column, i-th row j+1, i+1-th row j+1 column, i+1-th row j+1 column correspond to the four points in turn as four corners
  • the four vertices corresponding to the patch Q, the vertices are arranged in a clockwise or counterclockwise ring, which can simplify the structure as much as possible, ensure the same distance between vertices as possible, and improve the efficiency of mutual access between vertices as much as possible.
  • the half-edge quadrangle grid based on P i,j in the point cloud data After the half-edge quadrangle grid based on P i,j in the point cloud data is generated, continue to generate the half-edge quadrangle grid corresponding to the next point in the point cloud data until all points P i,j in the point cloud generate the corresponding half-edge Four-corner grid, the topological relationship between each half-edge four-corner grid is established through half-edges, vertices, and patches, and finally the 5D texture grid data structure corresponding to the point cloud of the object to be detected is formed.
  • the point cloud is an ordered point cloud. Since the points P i,j in the ordered point cloud are arranged in order, step S102 obtaining the row and column order of each point in the point cloud includes: traversing the ordered point cloud, Read the row coordinate i and column coordinate j of each point P i,j in the ordered point cloud to obtain the row and column order of each point in the point cloud data. In the corresponding step S103, for each point P i,j , its adjacent points P i,j ,P i,j+1 ,P i+1,j+1 ,P can also be easily found according to the row and column sorting. i+1,j .
  • the embodiment shown in FIG. 1 is a 5D texture grid data structure generated by a 4*4 ordered point cloud.
  • the point cloud is an unordered point cloud.
  • step S102 it is necessary to sort each point in the point cloud first, and then obtain the point cloud data respectively in the order of the points in the point cloud.
  • the points P i,j are sorted by row and by column.
  • the points P i,j in the disordered point cloud are sorted according to the sizes of x/z and y/z.
  • the acquisition by row order includes the following steps: for any two points in the unordered point cloud, compare the y/z size of the two, and arrange the point with the larger y/z value at the back, if the two points y/z If the z values are the same, compare the x/z values, and sort the point with the larger x/z value at the back; if the two x/z values are also the same, delete one of the points to obtain the row-by-row sorting.
  • the std::map class to create an empty container V1
  • the size of /z is sorted and stored in the container V1.
  • the sorting rule is: if the y/z value is larger, it will be in the back; if the y/z value is the same, the larger x/z value will be in the back.
  • the z value is also the same, then delete one of the points.
  • the container V1 is actually a matrix with one row and multiple columns, where the total number of columns is the number of corresponding points in the unordered point cloud (if the sorting rules no repeating points).
  • the acquisition of sorting by column includes the following steps: for any two points in the unordered point cloud, compare the x/z values of the two points, and sort the point with the larger x/z value at the back. If the y/z values are the same, compare the y/z values, and sort the point with the larger y/z value at the back; if the y/z values of the two are also the same, delete one of the points to obtain the sorting by column.
  • the std::map class to create an empty container V2, traverse each point P i,j of the original unordered point cloud whose coordinates are (x, y, z), according to x/z and y/z
  • the size of the s is sorted and stored in the container V2.
  • the sorting rule is: if the x/z value is larger, it will be ranked at the back; if the x/z value is the same, the larger y/z value will be ranked at the back. If the y/z values are the same, delete one of them.
  • the container V2 is also a matrix with one row and multiple columns, in which the total number of columns is the number of corresponding points in the unordered point cloud (if sorted by order) rules without repeating dots).
  • step 103 for any point P i,j , according to the indexing rules in row sorting and column sorting, the adjacent points P of P i, j in the unordered point cloud are searched.
  • i,j+1 ,P i+1,j+1 ,P i+1,j include the following steps:
  • SeizetColorMesh contains 6 sides of the carton and has a topological structure, which can represent all-round information of the carton.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • Image Generation (AREA)
  • Image Processing (AREA)

Abstract

一种5D纹理网格数据结构，用于点云数据的四角网格重构，5D纹理网格数据结构包括拓扑连接的多个半边四角网格，每个半边四角网格包括一四角面片、设于四角面片上的四个顶点S mn，S m,n+1，S m+1,n，S m+1,n+1以及多个半边，其中，基于点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个顶点，每个顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)、法向量、颜色以及其为起始顶点的半边的指针信息，每个半边存储起始顶点、终止顶点、邻接面、上个半边、下个半边、对立半边的指针或索引。3D点云转化为网格数据结构后，使用顶点、半边、四角面片的连接关系进行索引，可有效表征三维物体的全方位信息。

Description

一种5D纹理网格数据结构
本公开要求于2020年09月10日提交的中国专利申请202010944233.2的优先权，其全部内容通过援引加入本文。
技术领域
本公开属于涉及计算机图形和计算机视觉技术领域，特别涉及一种对点云的数据网格化以用于表征三维物体的网格数据结构。
背景技术
在工业制造、物流等领域，机床上下料、零件分拣、货品拆零拣选等无序分拣任务对于传统人工操作的方式来说，劳动强度大、分拣效率低、招工难度大。机器人无序分拣技术越来越多地替代人工，成为相关产业降本增效、智能化升级的重要手段。机器人无序分拣依靠视觉识别方法定位待分拣物体，进而引导机器人末端完成物体的抓取和放置操作。一般采用2D图像、3D点云或网格等方式表征物体信息，物体的视觉识别方法可分为2D方法和3D方法两类。
目前传统的2D视觉或3D视觉识别方法尚不能解决海量无序姿态工件的通用准确识别问题，难以应对电商物流的无序分拣需求。当待扫描工件的姿态倾角较大、杂乱堆叠时，比如进行位于周转箱侧放的桶装方便面识别，传统2D方法需要制作多个不同视角的模板进行匹配定位，这样面对海量工件时，建立模板工作量过于庞大；当待检测工件紧密放置时，3D视觉方法难以分割场景中的不同物体，且对物体形状变化较小的待检测工件难以适应，比如识别位于周转箱内紧密排列的手机包装盒。只有识别工件的完整形状(包括侧面和底面)等，才能工件实现准确码放，避免抓取贵重工件时损伤工件，而传统的2D/3D视觉识别方法，都只能基于工件上表面的局部信息进行识别，难以判断待测工件的完整形状，对准确码放难以适应。
上述问题的出现主要是由于现有视觉识别方法受限于表征三维物体的数据 结构，其中得益于2D图像的矩阵形式，2D视觉中图像处理方法都十分成熟，但2D图像难以表示三维物体的全方位信息；在3D视觉处理中，现有3D图像处理方法大多基于无序点云，难以实现类似2D的有序邻域访问，从而无法利用2D的纹理特征；基于有序点云的3D图像处理方法，由于有序点云仅能表达工件的上表面信息，对底面和侧面无法表达，也无法实现完整形状的表达。
发明内容
本发明的目的是提供一种用于表征三维物体的网格数据结构(SeizetColorMesh)，用以将点云网格化，点云数据基于纹理网格的数据结构的数据表示后，呈拓扑结构且可有效表征三维物体的全方位信息，故2D以及3D的图像处理算法均可适用，从而实现了更强大的感知识别效果，也为机器人面对海量无序状态的工件抓取奠定了数据结构基础。
在第一方面，本公开提供一种用于表征三维物体的网格数据结构，所述网格数据结构用于点云数据的四角网格重构，所述网格数据结构包括多个半边四角网格，每个所述半边四角网格包括一四角面片、设于四角面片上的四个顶点S mn，S m,n+1.S m+1,n，S m+1,n+1以及多个半边，其中，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，每个所述顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)、法向量、颜色以及其为起始顶点的半边的指针信息，每个半边存储起始顶点、终止顶点、邻接面、上个半边、下个半边、对立半边的指针或索引。
可选地，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点包括：
对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
获取所述点云中各点的行列排序；
对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1，P i+1,j+1，P i+1,j；
依据所述索引关系，获取P i,j，P i,j+1，P i+1,j+1，P i+1,j各自分别对应的四个顶点S m,n，S m,n+1，S m+1,n，S m+1,n+1。
进一步地，对任意一点P i,j，若无法查找到对应的三个相邻点P i,j+1、P i+1,j+1、P i+1,j，则跳过P i,j相邻点查找，继续点云中下一个点的相邻点查找。
可选地，所述点云为有序点云，获取所述点云中各点的行列排序包括：遍历所述有序点云，读取有序点云中各点P i,j的行坐标i和列坐标j，以获取所述点云数据中各点的行列排序。
可选地，所述点云为无序点云，获取所述点云中各点的行列排序为对点云中的各点进行排序，分别获取所述点云数据中各点P i,j的按行排序和按列排序。
进一步地，所述按行排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者的y/z大小，并将y/z值较大的点排在后面，若二者y/z值相同，则比较x/z值大小，并将x/z值较大的点排在后面；若二者x/z值也相同，则删除其中一点，以获取所述按行排序；
和/或，所述按列排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者x/z值大小，并将x/z值较大的点排在后面，若二者x/z值相同，则比较y/z值大小，并将y/z值较大的点排在后面；若二者y/z值也相同，则删除其中一点，以获取所述按列排序。
更进一步的，对于任一点P i,j，依照所述按行排序和按列排序中的索引规则查找所述无序点云中P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j包括如下步骤：
确定所述P i,j在所述按行排序中的位置，并将P i,j在所述按行排序中后一个点作为所述P i,j的相邻点P i,j+1；
确定所述P i,j+1在所述按列排序中的位置，并将所述P i,j+1在所述按列排序中 后一个点作为所述P i,j的相邻点P i+1,j+1；
确定P i,j在所述按列排序中的位置，并将P i,j在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j。
可选地，若判断所述四面面片的边长超出阈值，则删除边长超过阈值的所述四角面片。
可选地，基于所述点云数据获取每个所述半边四角网格的四个顶点后，新设一四角面片并按照预定顺序为所述四角面片添加四个所述顶点以生成对应的半边四角网格。
可选地，按照顺时针或逆时针顺序添加所述顶点。
在第二方面，本公开提供一种用于表征三维物体的网格数据结构，所述网格数据结构用于点云数据的四角网格重构，所述网格数据结构包括拓扑连接的多个四角网格，每个所述四角网格包括一四角面片、设于四角面片上的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1，其中，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，每个所述顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)，且所述网格数据结构还存储邻接关系，所述邻接关系为相邻的顶点与顶点之间拓扑关系、相邻的顶点与四角面片之间拓扑关系、相邻的四角面片与四角面片之间的拓扑关系中一种或多种组合。
可选地，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点包括：
对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
获取所述点云中各点的行列排序；
对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j；
依据所述索引关系，获取P i,j,P i,j+1,P i+1,j+1,P i+1,j各自分别对应的四个顶点S mn，S m,n+1.S m+1,n，S m+1,n+1。
可选地，对任意一点P i,j，若无法查找到对应的三个相邻点P i,j+1、P i+1,j+1、P i+1,j，则跳过P i,j相邻点查找，继续点云中下一个点的相邻点查找。
可选地，所述点云为有序点云，获取所述点云中各点的行列排序包括：遍历所述有序点云，读取有序点云中各点P i,j的行坐标i和列坐标j，以获取所述点云数据中各点的行列排序。
可选地，所述点云为无序点云，获取所述点云中各点的行列排序为对点云中的各点进行排序，分别获取所述点云数据中各点P i,j的按行排序和按列排序。
可选地，所述按行排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者的y/z大小，并将y/z值较大的点排在后面，若二者y/z值相同，则比较x/z值大小，并将x/z值较大的点排在后面；若二者x/z值也相同，则删除其中一点，以获取所述按行排序；
和/或，所述按列排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者x/z值大小，并将x/z值较大的点排在后面，若二者x/z值相同，则比较y/z值大小，并将y/z值较大的点排在后面；若二者y/z值也相同，则删除其中一点，以获取所述按列排序。
可选地，对于任一点P i,j，依照所述按行排序和按列排序中的索引规则查找所述无序点云中P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j包括如下步骤：
确定所述P i,j在所述按行排序中的位置，并将P i,j在所述按行排序中后一个点作为所述P i,j的相邻点P i,j+1；
确定所述P i,j+1在所述按列排序中的位置，并将所述P i,j+1在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j+1；
确定P i,j在所述按列排序中的位置，并将P i,j在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j。
可选地，若判断所述四面面片的边长超出阈值，则删除边长超过阈值的所述四角面片。
可选地，基于所述点云数据获取每个所述四角网格的四个顶点后，新设一四角面片并按照预定顺序为所述四角面片添加四个所述顶点以生成对应的四角网格。
可选地，按照顺时针或逆时针顺序添加所述顶点。
可选地，所述顶点还存储对应点P i,j的法向量、颜色、曲率、标志位中一种或多种组合。
在第三方面，本公开提供一种用于生成用于表征三维物体的网格数据结构的方法，其中，所述网格数据结构包括多个半边四角网格，每个所述半边四角网格包括一四角面片、设于四角面片上的四个顶点S mn，S m,n+1.S m+1,n，S m+1,n+1以及多个半边，
所述方法包括：
基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，包括：
对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
获取所述点云中各点的行列排序；
对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1，P i+1,j+1，P i+1,j；以及
依据所述索引关系，获取P i,j，P i,j+1，P i+1,j+1，P i+1,j各自分别对应的四个顶点S m,n，S m,n+1，S m+1,n，S m+1,n+1；以及
针对每个所述顶点，存储点云数据中对应点P i,j的3D位置坐标(x，y，z)、法向量、颜色以及其为起始顶点的半边的指针信息，每个半边存储起始顶点、 终止顶点、邻接面、上个半边、下个半边、对立半边的指针或索引。
在第四方面，本公开提供一种用于生成用于表征三维物体的网格数据结构的方法，其中，所述网格数据结构包括拓扑连接的多个四角网格，每个所述四角网格包括一四角面片、设于四角面片上的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1，
所述方法包括：
基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，包括：
对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
获取所述点云中各点的行列排序；
对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1，P i+1,j+1，P i+1,j；以及
依据所述索引关系，获取P i,j,，P i,j+1，P i+1,j+1，P i+1,j各自分别对应的四个顶点S m,n，S m,n+1，S m+1,n，S m+1,n+1；
针对每个所述顶点，存储点云数据中对应点P i,j的3D位置坐标(x，y，z)；以及存储邻接关系，所述邻接关系为相邻的顶点与顶点之间拓扑关系、相邻的顶点与四角面片之间拓扑关系、相邻的四角面片与四角面片之间的拓扑关系中一种或多种组合。
在第五方面，本公开提供一种邻域访问方法，包括：
根据本发明实施例所述的方法生成用于表征所述三维物体的网格数据结构；以及
对所述网格数据结构进行邻域访问，以获取表征所述三维物体的网格结构中顶点的临近点的信息。
在一些实施例中，可以基于所述邻域访问、如基于所获取的所述网格结构 中顶点的临近点的信息，用于所述三维物体的检测，包括但不限于所述三维物体的识别、定位和与邻近物体的区分或分割中的一种或多种。在进一步的实施例中，可以基于所述三维物体的检测，来操作所述三维物体，包括但不限于所述三维物体的上下料、分拣、拆开包装、抓取、放置和码放中的一种或多种。
在第六方面，本公开提供一种电子设备，包括：处理器和存储有计算机程序的存储器，所述处理器被配置为在运行计算机程序时根据本发明实施例所述的方法。
在第七方面，本公开提供一种存储介质，其特征在于，所述存储介质存储有计算机程序，所述计算机程序配置成被运行时执行本发明实施例所述的方法。
本公开所示的一种网格数据结构，用于点云的网格化，点云转化为网格数据结构后，除包括点云原有的信息外，其还使用顶点、半边、四角面片的连接关系进行索引，可有效表征三维物体的全方位信息。通过本公开提出的SeizetColorMesh，将3D点云转换成SeizetColorMesh后，可综合采用2D以及3D等所有算法，用于点云数据的后续处理，可实现更强大的感知识别效果，进而为机器人面对海量无序状态的工件抓取奠定了数据结构基础。
具体而言，可对待检测物各个面拍照，获取每个面的3D点云，将各3D点云转换成SeizetColorMesh后，然后基于SeizetColorMesh进行合成，制作1个基于SeizetColorMesh的全方位模板。基于SeizetColorMesh的全方位模板可提供待识别物体的完整形状和纹理信息(包括侧面和底面)，提高了定位精度、可靠性以及通用性，进而可实现机器人在海量无序工件场景下的精准抓取与码放，如实现物体姿态倾角较大、杂乱堆叠场景的识别定位等；此外，还可将2D图像处理方法拓展至SeizetColorMesh，即可实现物体紧密放置场景下的不同物体分割。
附图说明
图1为本公开一实施例的5D纹理网格数据结构的结构示意图；
图2为本发明一实施例的半边四角网格的结构示意图；
图3为待识别物纸盒的无序点云；
图4为基于图3所示无序点云生成的5D纹理网格数据结构；
图5为图4中虚框内结构的放大图。
具体实施方式
为使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例对本发明进行进一步的详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，而非以任何方式限制本发明的保护范围。
在说明书的全文中，相同的附图标号指代相同的元件。表述“和/或”包括相关联的所列相目中的一个或多个的任何和全部组合。在附图中，为了便于说明，已稍微夸大了物体的厚度、尺寸和形状。附图仅为示例而非严格按比例绘制。
还应理解的是，用语“包括”、“包括有”、“具有”、“包含”和/或“包含有”，当在本说明书中使用时表示存在所陈述的特征、步骤、整体、操作、元件和/或部件，但不排除存在或附加有一个或多个其它特征、步骤、整体、操作、元件、部件和/或它们的组合。
如在说明书中使用的用语“基本上”、“大约”以及类似的用于用作表示近似的用语，而不用作表示程度的用语，并且旨在说明将由本领域普通技术人员认识到的、测量值或计算值中的固有偏差。
除非另有限定，否则本文中使用的所有用语(包括技术用语和科学用语)均具有与本申请所属领域普通技术人员的通常理解相同的含义。还应理解的是，用语(例如在常用词典中定义的用语)应被解释为具有与它们在相关技术的上下文中的含义一致的含义，并且将不被以理想化或过度正式意义解释，除非本文中明确如此限定。
需要说明的是，在不冲突的情况下，本申请的实施例及实施例中的特征可以相互组合。
在本公开的一些实施例中，所述网格数据结构可被称为5D纹理网格数据结构，作为解释而非限制地，所述5D是指所述纹理网格数据可同时适用2D(如纹理)以及3D的图像处理算法。
在本公开中，术语“点云数据”(point cloud data)、“点云”或者“3D点云”是指在一个三维坐标系统中的一组向量的集合，包括三维坐标X,Y,Z、颜色、法向量、曲率等等信息。在本公开实施例中，点云可包括无序点云和有序点云。
在本公开实施例中，点云可以由测量直接得到，每个点可对应一个测量点。作为举例而非限制地，点云未经过其他处理手段，故包含了最大的信息量，这些信息隐藏在点云中可通过多种提取手段将其提取出来，在本公开实施例中，提取点云中信息的过程可称为“3D点云处理”。
由此，在本公开的一些实施例中，所述方法还可以包括获取3D点云的步骤，在一些实施例中，所述3D点云的获取可以基于3D点云获取装置，其例如可经由3D相机获取。
在本公开中，术语“2D图像”，也称二维图像，是指不包含深度信息的平面图像。在一些实施例中，“2D图像处理”可包括根据灰度或彩色图像中对比度的特征提供结果，用于基于边缘检测的各种二维几何分析等。
在本公开中，术语半边(HalfEdge)涉及顶点之间的有向连线，例如但不限于通过TerminateVertex和OriginVertex指针分别记录其终止顶点和起始顶点。
如图1所示，本公开提供了一种用于表征三维物体的网格数据结构，其例如可称为5D纹理网格数据结构(SeizetColorMesh)，该5D纹理网格数据结构使用顶点、半边、四角面片的连接关系进行索引，用于点云数据的四角网格重构，点云基于网格数据结构的数据表示后，不仅保留点云所存储的信息，且呈拓扑结构(包含顶点、半边、四角面片三者之间的互相访问方式)，可有效表 征三维物体的全方位信息。
5D纹理网格数据结构包括多个半边四角网格，每个半边四角网格包括一四角面片、设于四角面片上的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1以及多个半边，其中，基于点云数据P i,j以及P i,j的三个相邻点获取每个半边四角网格对应的四个顶点，每个所述顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)、法向量、颜色以及其为起始顶点的半边的指针信息，每个半边存储起始顶点、终止顶点、邻接面、上个半边、下个半边、对立半边的指针或索引。
如图2所示，基于点云数据所生成的每个半边四角网格中包括顶点1)、半边2)和四角面片3)，为方便说明，将顶点S(1,0)、S(1,1)、S(2,1)、S(2,0)简化为顶点A、B、C、D，其中，AB、BC、CD、DA均为半边，ABCD为四角面片。
顶点(Vertex)A、B、C、D中存储对应点P i,j+1,P i+1,j+1,P i+1,j的3D坐标、法向量、颜色等信息；半边(HalfEdge)是顶点之间的有向连线，通过TerminateVertex和OriginVertex指针分别记录其终止顶点和起始顶点，图2中半边AB的终止顶点和起始顶点分别为B和A。
半边通过NextHalfEdge和PrevHalfEdge指针分别记录其下个半边和上个半边，图2中半边AB的下个半边和上个半边分别为BC和DA；半边通过OppositeHalfEdge指针记录其对立半边，图2中半边AB的对立半边为BA；半边通过AdjacentFace指针记录与其邻接的四角面片，图2中半边AB的邻接四角面片为ABCD；顶点通过OutgoingHalfEdge和IncomingHalfEdge指针分别记录出半边和入半边，图2中顶点A的出半边和入半边分别为AB和BA。
四角面片(QuadFace)由4个顺序连接的半边组成，图2中四角面片ABCD由半边AB、BC、CD、DA组成；四角面片通过OuterHalfEdge和InnerHalfEdge指针记录其外半边和内半边，图2中四角面片ABCD的外半边和内半边分别为BA和AB。
本公开所示的一种5D纹理网格数据结构，用以表征三维物体的全方位信息，其使用顶点、半边、四角面片的连接关系进行索引，通过5D纹理网格数据结构，建立顶点与对应的邻域点之间的访问方式，为2D图像处理方法向3D形态推广建立桥梁。通过本公开提出的SeizetColorMesh，可综合采用2D图像处理以及3D点云处理中的算法，实现了更强大的感知识别效果，进而为机器人面对海量无序状态的工件抓取奠定了数据结构基础。
具体而言，将3D点云转换成SeizetColorMesh后，只需制作1个基于SeizetColorMesh的全方位模板，基于SeizetColorMesh的全方位模板，可提供待识别物体的完整形状和纹理信息(包括侧面和底面)，提高了定位精度、可靠性以及通用性，进而可实现机器人在海量无序工件场景下的精准抓取与码放,即实现物体姿态倾角较大、杂乱堆叠场景的识别定位；此外，还可将2D图像处理方法拓展至SeizetColorMesh，如可采用2D图像处理方法对SeizetColorMesh进行处理，以实现物体紧密放置场景下的不同物体分割。
作为一优选方案，本公开所示的5D纹理网格数据结构，依据如下步骤获取每个半边四角网格的四个顶点：
S101对于点云中的任意一点P i,j，添加P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S mn，建立P i,j与顶点S mn之间一一对应索引关系；
S102获取点云中各点的行列排序；
S103对点云数据中的任一点P i,j，依照行列排序的索引规则查找P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j；
S104依据索引关系，获取P i,j,P i,j+1,P i+1,j+1,P i+1,j各自分别对应的四个顶点S mn，S m,n+1.S m+1,n，S m+1,n+1。
其中，步骤S103中，对任意一点P i,j，若无法查找到对应的三个相邻点P i,j+1、P i+1,j+1、P i+1,j，则跳过P i,j相邻点查找，继续点云中下一个点的相邻点查找。
基于点云数据获取每个半边四角网格的四个顶点后，则新设一四角面片并 按照预定顺序为四角面片添加四个顶点以生成对应的半边四角网格，对点云数据排序后按行、按列依次找到4个相邻点对应的顶点后，则建立这4个顶点的拓扑关系，即四角面片。作为一优选方案，本实施例中，四角面片顶点添加顺序依次为S m,n、S m+1,n、S m+1,n+1、S m,n+1或为S m,n、S m,n+1、S m+1,n+1、S m+1,n，即将点云数据中第i行j列、第i+1行j列、第i+1行j+1列、第i行j+1列或第i行j列、第i行j+1、第i+1行j+1列、第i+1行j列对应的4个点依次作为四角面片Q对应的4个顶点，顶点以顺时针或逆时针环形设置，能够尽可能简化结构，尽可能保证顶点间距一致，同时尽可能提高顶点间相互访问的效率。
当点云数据中基于P i,j的半边四角网格生成后，继续点云数据中下一个点对应的半边四角网格生成，直至点云中全部的点P i,j均生成对应的半边四角网格，各半边四角网格之间经由半边、顶点、面片建立拓扑关系，最终形成待检测物点云所对应的5D纹理网格数据结构。
其中一实施例中，点云为有序点云，由于有序点云中各点P i,j是按顺序排列，步骤S102获取点云中各点的行列排序包括：遍历有序点云，读取有序点云中各点P i,j的行坐标i和列坐标j，以获取点云数据中各点的行列排序。对应的步骤S103中，对于各点P i,j，依据行列排序也可以很容易的找到它的相邻点P i,j,P i,j+1,P i+1,j+1,P i+1,j。图1所示实施例即为一个4*4的有序点云所生成的5D纹理网格数据结构。
在另外一实施例中，点云为无序点云，为获取点云中各点的行列排序，步骤S102中，需要先对点云中的各点进行排序，然后在分别获取点云数据中各点P i,j的按行排序和按列排序。
作为一优选方案，本实施例中，根据x/z和y/z的大小对无序点云中的点P i,j进行排序。其中，按行排序的获取包括如下步骤：对于无序点云中的任意两点，比较二者的y/z大小，并将y/z值较大的点排在后面，若二者y/z值相同，则比较x/z值大小，并将x/z值较大的点排在后面；若二者x/z值也相同，则删除其 中一点，以获取所述按行排序。具体而言，使用std::map类创建一个空的容器V1，然后遍历原始无序点云的每个点P i,j，其坐标为(x,y,z)，根据x/z和y/z的大小排序并存入容器V1，排序规则为：若y/z值较大，则排在后面；若y/z值相同，则x/z值较大的排在后面，若x/z值也相同，则删除其中一点。对无序点云中所有的点按照上述规则排序完毕后输出容器V1，容器V1实际上为一行多列的矩阵，其中总列数为无序点云中对应的点的数量(若按排序规则无重复的点)。
按列排序的获取包括如下步骤：对于无序点云中的任意两点，比较二者x/z值大小，并将x/z值较大的点排在后面，若二者x/z值相同，则比较y/z值大小，并将y/z值较大的点排在后面；若二者y/z值也相同，则删除其中一点，以获取所述按列排序。具体而言，使用std::map类创建一个空的容器V2，遍历原始无序点云的每个点P i,j其坐标为(x,y,z)，根据x/z和y/z的大小排序并存入容器V2，排序规则为：若x/z值较大，则排在后面；若x/z值相同，则y/z值较大的排在后面。若y/z值相同，则删除其中一点。对无序点云中所有的点按照上述规则排序完毕后输出容器V2，容器V2实质上也为一行多列的矩阵，其中总列数为无序点云中对应的点的数量(若按排序规则无重复的点)。
相应的，对无序点云而言，步骤103中，对于任一点P i,j，依照按行排序和按列排序中的索引规则查找无序点云中P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j包括如下步骤：
确定P i,j在按行排序中的位置，并将P i,j在按行排序中后一个点作为P i,j的相邻点P i,j+1；
确定P i,j+1在按列排序中的位置，并将P i,j+1在按列排序中后一个点作为P i,j的相邻点P i+1,j+1；
确定P i,j在按列排序中的位置，并将P i,j在按列排序中后一个点作为P i,j的相邻点P i+1,j。
当基于点云数据获取每个半边四角网格的四个顶点后，对于无序点云而言，在新设一四角面片并按照预定顺序为四角面片添加四个顶点以生成对应的半边四角网格时，还需要判断四面面片的边长是否超出阈值，若判断四面面片的边长超出阈值则删除边长超过阈值的四角面片。
如图3所示，其为待识别纸盒的原始无序点云，所生成的SeizetColorMesh如图4所示，图5为图4中虚框内结构的放大图。SeizetColorMesh包含纸盒的6个面，且呈拓扑结构，可表征纸盒的全方位信息。
上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。
以上仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。

Claims (20)

  1. 一种用于表征三维物体的网格数据结构，其特征在于，所述网格数据结构用于点云数据的四角网格重构，所述网格数据结构包括拓扑连接的多个半边四角网格，每个所述半边四角网格包括一四角面片、设于四角面片上的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1以及多个半边，其中，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，每个所述顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)、法向量、颜色以及其为起始顶点的半边的指针信息，每个半边存储起始顶点、终止顶点、邻接面、上个半边、下个半边、对立半边的指针或索引。
  2. 根据权利要求1所述的用于表征三维物体的网格数据结构，其特征在于，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点包括：
    对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
    获取所述点云中各点的行列排序；
    对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j；
    依据所述索引关系，获取P i,j,P i,j+1,P i+1,j+1,P i+1,j各自分别对应的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1。
  3. 根据权利要求2所述的用于表征三维物体的网格数据结构，其特征在于：对任意一点P i,j，若无法查找到对应的三个相邻点P i,j+1、P i+1,j+1、P i+1,j，则跳过P i,j相邻点查找，继续点云中下一个点的相邻点查找。
  4. 根据权利要求2所述的用于表征三维物体的网格数据结构，其特征在于： 所述点云为有序点云，获取所述点云中各点的行列排序包括：遍历所述有序点云，读取有序点云中各点P i,j的行坐标i和列坐标j，以获取所述点云数据中各点的行列排序。
  5. 根据权利要求2所述的用于表征三维物体的网格数据结构，其特征在于：所述点云为无序点云，获取所述点云中各点的行列排序为对点云中的各点进行排序，分别获取所述点云数据中各点P i,j的按行排序和按列排序。
  6. 根据权利要求5所述的用于表征三维物体的网格数据结构，其特征在于：所述按行排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者的y/z大小，并将y/z值较大的点排在后面，若二者y/z值相同，则比较x/z值大小，并将x/z值较大的点排在后面；若二者x/z值也相同，则删除其中一点，以获取所述按行排序；
    和/或，所述按列排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者x/z值大小，并将x/z值较大的点排在后面，若二者x/z值相同，则比较y/z值大小，并将y/z值较大的点排在后面；若二者y/z值也相同，则删除其中一点，以获取所述按列排序。
  7. 根据权利要求6所述的用于表征三维物体的网格数据结构，其特征在于：对于任一点P i,j，依照所述按行排序和按列排序中的索引规则查找所述无序点云中P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j包括如下步骤：
    确定所述P i,j在所述按行排序中的位置，并将P i,j在所述按行排序中后一个点作为所述P i,j的相邻点P i,j+1；
    确定所述P i,j+1在所述按列排序中的位置，并将所述P i,j+1在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j+1；
    确定P i,j在所述按列排序中的位置，并将P i,j在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j。
  8. 根据权利要求2所述的用于表征三维物体的网格数据结构，其特征在于： 若判断所述四面面片的边长超出阈值，则删除边长超过阈值的所述四角面片。
  9. 根据权利要求1所述的用于表征三维物体的网格数据结构，其特征在于，基于所述点云数据获取每个所述半边四角网格的四个顶点后，新设一四角面片并按照预定顺序为所述四角面片添加四个所述顶点以生成对应的半边四角网格。
  10. 根据权利要求1所述的用于表征三维物体的网格数据结构，其特征在于：按照顺时针或逆时针顺序添加所述顶点。
  11. 一种用于表征三维物体的网格数据结构，其特征在于，所述网格数据结构用于点云数据的四角网格重构，所述网格数据结构包括拓扑连接的多个四角网格，每个所述四角网格包括一四角面片、设于四角面片上的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1，其中，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点，每个所述顶点存储点云数据中对应点P i,j的3D位置坐标(x，y，z)，且所述网格数据结构还存储邻接关系，所述邻接关系为相邻的顶点与顶点之间拓扑关系、相邻的顶点与四角面片之间拓扑关系、相邻的四角面片与四角面片之间的拓扑关系中一种或多种组合。
  12. 根据权利要求11所述的用于表征三维物体的网格数据结构，其特征在于，基于所述点云数据中任意一点P i,j以及P i,j的三个相邻点获取对应的四个所述顶点包括：
    对于所述点云中的任意一点P i,j，添加所述P i,j的3D位置坐标(x，y，z)法向量、颜色信息以获取顶点S m,n，建立所述P i,j与所述顶点S m,n之间一一对应索引关系；
    获取所述点云中各点的行列排序；
    对点云数据中的任一点P i,j，依照所述行列排序的索引规则查找P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j；
    依据所述索引关系，获取P i,j,P i,j+1,P i+1,j+1,P i+1,j各自分别对应的四个顶点S m,n，S m,n+1.S m+1,n，S m+1,n+1。
  13. 根据权利要求12所述的用于表征三维物体的网格数据结构，其特征在于：对任意一点P i,j，若无法查找到对应的三个相邻点P i,j+1、P i+1,j+1、P i+1,j，则跳过P i,j相邻点查找，继续点云中下一个点的相邻点查找。
  14. 根据权利要求12所述的用于表征三维物体的网格数据结构，其特征在于：所述点云为有序点云，获取所述点云中各点的行列排序包括：遍历所述有序点云，读取有序点云中各点P i,j的行坐标i和列坐标j，以获取所述点云数据中各点的行列排序。
  15. 根据权利要求12所述的用于表征三维物体的网格数据结构，其特征在于：所述点云为无序点云，获取所述点云中各点的行列排序为对点云中的各点进行排序，分别获取所述点云数据中各点P i,j的按行排序和按列排序。
  16. 根据权利要求15所述的用于表征三维物体的网格数据结构，其特征在于：所述按行排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者的y/z大小，并将y/z值较大的点排在后面，若二者y/z值相同，则比较x/z值大小，并将x/z值较大的点排在后面；若二者x/z值也相同，则删除其中一点，以获取所述按行排序；
    和/或，所述按列排序获取包括：对于无序点云中的任意两点，根据x/z和y/z的大小对无序点云中的点P i,j进行排序，比较二者x/z值大小，并将x/z值较大的点排在后面，若二者x/z值相同，则比较y/z值大小，并将y/z值较大的点排在后面；若二者y/z值也相同，则删除其中一点，以获取所述按列排序。
  17. 根据权利要求16所述的用于表征三维物体的网格数据结构，其特征在于：对于任一点P i,j，依照所述按行排序和按列排序中的索引规则查找所述无序点云中P i,j的相邻点P i,j+1,P i+1,j+1,P i+1,j包括如下步骤：
    确定所述P i,j在所述按行排序中的位置，并将P i,j在所述按行排序中后一个点作为所述P i,j的相邻点P i,j+1；
    确定所述P i,j+1在所述按列排序中的位置，并将所述P i,j+1在所述按列排序中 后一个点作为所述P i,j的相邻点P i+1,j+1；
    确定P i,j在所述按列排序中的位置，并将P i,j在所述按列排序中后一个点作为所述P i,j的相邻点P i+1,j。
  18. 根据权利要求12所述的用于表征三维物体的网格数据结构，其特征在于：若判断所述四面面片的边长超出阈值，则删除边长超过阈值的所述四角面片。
  19. 根据权利要求11所述的用于表征三维物体的网格数据结构，其特征在于，基于所述点云数据获取每个所述四角网格的四个顶点后，新设一四角面片并按照预定顺序为所述四角面片添加四个所述顶点以生成对应的四角网格。
  20. 根据权利要求11所述的一种用于表征三维物体的网格数据结构，其特征在于，所述顶点还存储对应点P i,j的法向量、颜色、曲率、标志位中一种或多种组合。
PCT/CN2021/116728 2020-09-10 2021-09-06 一种5d纹理网格数据结构 WO2022052893A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010944233.2A CN111814774B (zh) 2020-09-10 2020-09-10 一种5d纹理网格数据结构
CN202010944233.2 2020-09-10

Publications (1)

Publication Number Publication Date
WO2022052893A1 true WO2022052893A1 (zh) 2022-03-17

Family

ID=72860745

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/116728 WO2022052893A1 (zh) 2020-09-10 2021-09-06 一种5d纹理网格数据结构

Country Status (2)

Country Link
CN (1) CN111814774B (zh)
WO (1) WO2022052893A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814774B (zh) * 2020-09-10 2020-12-18 熵智科技(深圳)有限公司 一种5d纹理网格数据结构

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107194431A (zh) * 2017-05-31 2017-09-22 浙江国遥地理信息技术有限公司 机载LiDAR点云数据分类技术
CN107786958A (zh) * 2017-10-12 2018-03-09 中国科学院合肥物质科学研究院 一种基于深度学习模型的数据融合方法
CN109658515A (zh) * 2017-10-11 2019-04-19 阿里巴巴集团控股有限公司 点云网格化方法、装置、设备及计算机存储介质
CN111814774A (zh) * 2020-09-10 2020-10-23 熵智科技(深圳)有限公司 一种5d纹理网格数据结构
CN111815691A (zh) * 2020-09-10 2020-10-23 熵智科技(深圳)有限公司 一种网格数据结构的邻域访问方法、装置、设备及介质
CN111881892A (zh) * 2020-09-10 2020-11-03 熵智科技(深圳)有限公司 有序点云5d纹理网格数据结构生成方法、装置、设备及介质
CN111932570A (zh) * 2020-09-10 2020-11-13 熵智科技(深圳)有限公司 一种基于网格数据的边缘检测方法、装置、介质及设备
CN112002015A (zh) * 2020-09-10 2020-11-27 熵智科技(深圳)有限公司 无序点云生成网格数据结构的方法、装置、设备及介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140135409A (ko) * 2013-05-16 2014-11-26 최진원 시공 구조물의 3차원 형상정보를 포함한 시공도면 출력 시스템, 그리고 시공 구조물의 3차원 형상정보를 포함하는 시공도면을 인식하는 작업용 글래스 및 이를 구비한 작업모
CN110502919B (zh) * 2019-07-12 2022-11-18 武汉理工大学 一种基于深度学习的轨迹数据去匿名方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107194431A (zh) * 2017-05-31 2017-09-22 浙江国遥地理信息技术有限公司 机载LiDAR点云数据分类技术
CN109658515A (zh) * 2017-10-11 2019-04-19 阿里巴巴集团控股有限公司 点云网格化方法、装置、设备及计算机存储介质
CN107786958A (zh) * 2017-10-12 2018-03-09 中国科学院合肥物质科学研究院 一种基于深度学习模型的数据融合方法
CN111814774A (zh) * 2020-09-10 2020-10-23 熵智科技(深圳)有限公司 一种5d纹理网格数据结构
CN111815691A (zh) * 2020-09-10 2020-10-23 熵智科技(深圳)有限公司 一种网格数据结构的邻域访问方法、装置、设备及介质
CN111881892A (zh) * 2020-09-10 2020-11-03 熵智科技(深圳)有限公司 有序点云5d纹理网格数据结构生成方法、装置、设备及介质
CN111932570A (zh) * 2020-09-10 2020-11-13 熵智科技(深圳)有限公司 一种基于网格数据的边缘检测方法、装置、介质及设备
CN112002015A (zh) * 2020-09-10 2020-11-27 熵智科技(深圳)有限公司 无序点云生成网格数据结构的方法、装置、设备及介质

Also Published As

Publication number Publication date
CN111814774A (zh) 2020-10-23
CN111814774B (zh) 2020-12-18

Similar Documents

Publication Publication Date Title
CN111881892B (zh) 有序点云5d纹理网格数据结构生成方法、装置、设备及介质
CN112002015B (zh) 无序点云生成网格数据结构的方法、装置、设备及介质
CN112070818A (zh) 基于机器视觉的机器人无序抓取方法和系统及存储介质
US20160224858A1 (en) Method and device for recognizing a known object in a field of view of a three-dimensional machine vision system
CN115330819B (zh) 一种软包分割定位方法、工控机和机器人抓取系统
CN114783068B (zh) 姿态识别方法、装置、电子装置和存储介质
CN109829476B (zh) 基于yolo的端到端三维物体检测方法
WO2022052893A1 (zh) 一种5d纹理网格数据结构
CN113191174A (zh) 物品定位方法和装置、机器人及计算机可读存储介质
CN111242240A (zh) 物料检测方法、装置及终端设备
CN114359226A (zh) 基于分层叠加和区域增长的三维模型组可视面积提取方法
Sun et al. TriVoC: Efficient voting-based consensus maximization for robust point cloud registration with extreme outlier ratios
EP4202858A1 (en) Method and device for cargo counting, computer equipment, and storage medium
CN110490887B (zh) 一种基于3d视觉的对矩形包裹边缘快速识别定位方法
CN105894494A (zh) 基于gpu技术的三线阵立体航测相机并行谱段配准方法
CN113379826A (zh) 物流件的体积测量方法以及装置
JP7373700B2 (ja) 画像処理装置、ビンピッキングシステム、画像処理方法、画像処理プログラム、制御方法及び制御プログラム
CN111815691B (zh) 一种网格数据结构的邻域访问方法、装置、设备及介质
Ran et al. Normal estimation algorithm for point cloud using KD-Tree
WO2023005195A1 (zh) 地图数据的处理方法、装置、家用电器和可读存储介质
Luchowski et al. Using RANSAC for 3D point cloud segmentation
Kumar et al. Polygonal Approximation of Closed Curves across Multiple Views.
CN113111741A (zh) 一种基于三维特征点的装配状态识别方法
Zhang et al. Robotic grasp detection using effective graspable feature selection and precise classification
JP3706603B2 (ja) データ特徴抽出装置及びデータ照合装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21865947

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06.07.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21865947

Country of ref document: EP

Kind code of ref document: A1