WO2021078065A1 - 乳房三维点云重建方法、装置、存储介质及计算机设备 - Google Patents

乳房三维点云重建方法、装置、存储介质及计算机设备 Download PDF

Info

Publication number
WO2021078065A1
WO2021078065A1 PCT/CN2020/121234 CN2020121234W WO2021078065A1 WO 2021078065 A1 WO2021078065 A1 WO 2021078065A1 CN 2020121234 W CN2020121234 W CN 2020121234W WO 2021078065 A1 WO2021078065 A1 WO 2021078065A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
breast
point
transformation matrix
area
Prior art date
Application number
PCT/CN2020/121234
Other languages
English (en)
French (fr)
Inventor
谈继勇
李元伟
杨光耀
秦辉
Original Assignee
深圳瀚维智能医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳瀚维智能医疗科技有限公司 filed Critical 深圳瀚维智能医疗科技有限公司
Publication of WO2021078065A1 publication Critical patent/WO2021078065A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30068—Mammography; Breast

Definitions

  • This application relates to the technical field of point cloud data processing, and in particular to a method, device, storage medium, and computing device for breast 3D point cloud reconstruction.
  • Ultrasound is a non-surgical diagnostic test that is painless, harmless, and non-radioactive to the subject.
  • ultrasound can clearly display various cross-sectional images of internal organs and their surroundings. Since the images are rich in solidity and close to the true structure of anatomy, the application of ultrasound can be used for early diagnosis. Based on the many advantages of ultrasound examination, from professional medical disease diagnosis to daily health index assessment, its application range is becoming wider and wider.
  • the existing breast screening mode basically uses conventional ultrasound equipment to scan the breast area through the operator’s hand-held ultrasound probe.
  • the scanning trajectory of the probe is usually a subjective choice of the operator.
  • the probe posture is difficult to adjust according to the breast shape of the corresponding area of the breast.
  • a comprehensive breast scan has many drawbacks, which are not conducive to making accurate judgments on the physiological conditions of the breasts.
  • the latest breast screening mode is to use an automated robotic arm to drive the ultrasound probe to complete the breast area scan process, although it can rely on mechanized technical means.
  • the motion trajectory of the ultrasonic probe is obtained by manually calibrating several coordinates of the area to be scanned and inputting it into the corresponding computer software programming.
  • manually calibrating the coordinates is not only a complicated process, but also has poor accuracy, which adversely affects the results of breast screening.
  • the main purpose of this application is to provide a breast 3D point cloud reconstruction method, which aims to solve the technical problem that the existing mechanized breast ultrasound scanning method is difficult to accurately obtain the 3D spatial information of the breast region.
  • a method for reconstructing breast 3D point cloud including:
  • the 3D point clouds in all viewing angles are transformed into the same base coordinate system to generate a complete three-dimensional point cloud of the breast area.
  • the calculating a full-view transformation matrix according to the transformation matrix includes:
  • a full-view transformation matrix from the node at the end to the reference node is calculated along the shortest path.
  • the generating a complete three-dimensional point cloud of the breast region includes:
  • the optimal point cloud is selected from the point clouds from multiple shooting devices in the overlapping area to be used for the overlapping
  • the 3D reconstruction of the area includes:
  • the optimal point cloud is selected from a number of overlapping point clouds at the same position to be combined into a point cloud area for three-dimensional reconstruction.
  • the generating a complete three-dimensional point cloud of the breast region includes:
  • the effective point cloud pieces are filtered from the curved surface according to preset filtering conditions.
  • the method further includes:
  • the present application also provides a breast 3D point cloud reconstruction device, including:
  • Image acquisition module used to acquire 2D images and 3D point clouds in different perspectives
  • the feature matching module is used to perform feature extraction and feature matching on the 2D image under each view angle to obtain a number of 2D matching point pairs;
  • the matrix calculation module is used to obtain 3D matching point pairs according to the 2D matching point pairs, and calculate the coordinate transformation of the 3D matching point pairs to obtain the transformation matrix of every two 3D point clouds with overlapping regions;
  • a full-view matrix calculation module configured to calculate a full-view transformation matrix according to the transformation matrix
  • the model generation module is used to transform the 3D point clouds in all viewing angles into the same base coordinate system according to the full-view transformation matrix to generate a complete three-dimensional point cloud of the breast region.
  • the full-view matrix calculation module includes:
  • a topological map establishing unit configured to determine a pair of related photographing devices according to the transformation matrix, so as to establish a topological connection diagram of the photographing devices;
  • a path calculation unit configured to select a reference node from the topological connection graph, and calculate the shortest paths from the remaining nodes to the reference node;
  • the matrix calculation unit is configured to calculate a full-view transformation matrix from the node at the end to the reference node along the shortest path.
  • the present application also proposes a computer program storage medium in which computer program code is stored, and when the computer program code is executed by a processor, the steps of the above-mentioned breast three-dimensional point cloud reconstruction method are realized.
  • the present application also provides a computer device, including a processor, a memory, and computer program code stored in the memory.
  • the processor calls the computer program code, it implements the above-mentioned breast 3D point cloud reconstruction method. step.
  • this application develops a set of 3D point cloud reconstruction algorithms suitable for the characteristics of female breasts, which can quickly perform offline calibration and online reconstruction of views of the breast area under multiple different shooting angles to obtain breasts.
  • the three-dimensional spatial information of the entire surface of the region provides the basis for subsequent scan trajectory planning.
  • the ultrasound probe can be based on the shape of the contact area. Adjust the scanning posture to ensure that the information covered by each frame of ultrasound image acquired is comprehensive and accurate, so as to make a comprehensive and accurate judgment on the physiological condition of the breast and its surrounding organs and tissues, so as to avoid scanning caused by human operations Occurrence of incomplete coverage and lack of information in the obtained ultrasound images.
  • FIG. 1 is a schematic structural diagram of an example environment in which multiple embodiments disclosed in this application can be implemented;
  • FIG. 2 is a schematic diagram of offline calibration when collecting a point cloud of the chest area in multiple embodiments disclosed in this application;
  • FIG. 3 is an image of a breast area in a first viewing angle in multiple embodiments disclosed in this application;
  • FIG. 4 is an image of a breast area in a second viewing angle in multiple embodiments disclosed in this application;
  • FIG. 5 is an image of a breast area obtained through coordinate transformation in multiple embodiments disclosed in this application.
  • FIG. 6 is a schematic flowchart of an embodiment of a breast point cloud 3D reconstruction method according to this application.
  • FIG. 7 is a schematic flowchart of another embodiment of the breast point cloud 3D reconstruction method of this application.
  • FIG. 8 is a schematic diagram of functional modules of an embodiment of the breast point cloud three-dimensional reconstruction device of this application.
  • FIG. 9 is a schematic structural diagram of a computer device in which multiple embodiments disclosed in this application can be implemented.
  • This application provides a breast 3D point cloud reconstruction method, whose purpose is to collect 3D point cloud data according to the chest area of each user.
  • 2D images and 3D points are collected from multiple different perspectives.
  • Cloud where the 2D image can be an RGB image, at least two views from different perspectives are collected, and the two views should have a certain degree of overlap, so as to reconstruct all the acquired 3D point clouds according to the reconstruction model.
  • the point cloud corresponding to the area to be scanned on the chest can be extracted from the processed 3D point cloud data, and the ultrasound scanning trajectory can be formulated through the position and structure information represented by the point cloud, so that the breast ultrasound can be scanned
  • the device can execute the scanning process according to the ultrasonic scanning trajectory, and then generate a comprehensive and accurate ultrasonic image containing information.
  • the aforementioned breast ultrasound scanning equipment mainly includes a scanning actuator 10, a screening platform 20, and a photographing device 30.
  • the scanning actuator 10 includes a host 11, and The robotic arm 12 connected to the host 11 and the ultrasonic probe 13 installed at the execution end of the robotic arm 12.
  • the host 11 has corresponding hardware capable of implementing communication, data processing and motion control functions, and the host 11 also has The infrastructure for installing the robotic arm 12, for example, the robotic arm 12 is configured as a multi-axis structure capable of providing three linear motion degrees of freedom and two or more rotational degrees of freedom, thereby ensuring that the ultrasonic probe 13 can be based on the surface shape of the area to be scanned
  • the robotic arm 12 can be a five-axis robotic arm or a six-axis robotic arm.
  • the screening platform 20 can be a fixed support structure or a movable structure that can provide position adjustment.
  • the photographing equipment 30 is arranged above the screening platform 20.
  • two sets of photographing equipment 30 can be configured according to the structure shown in FIG. 1.
  • the horizontal of the user's body is used as a reference
  • the shooting device 30 can be arranged with the longitudinal direction of the user's body as the reference direction.
  • the shooting device 30 in this embodiment may be a structured light sensor, of course, it may also be a lidar. ;
  • the shooting device 30 is installed on a motion mechanism, through the motion mechanism to achieve the conversion of different shooting angles, thereby reducing the number of shooting devices 30, in the minimum case, only one shooting device 30 can be arranged,
  • the shooting device 30 realizes the transformation of the shooting angle of view by moving along a certain set circle, thereby obtaining 2D images and 3D point clouds in multiple angles of view, as shown in Figs. 3 and 4, by collecting in two different angles of view. 3D point clouds of the two breast regions obtained.
  • the breast 3D point cloud reconstruction method proposed in the present application mainly includes two data processing links.
  • the first is to perform offline calibration based on the collected image data to obtain the calibration for view reconstruction. Parameters, and then online reconstruction of the collected breast area point cloud according to the calibration parameters, so as to transform the multiple view point clouds collected online into a unified coordinate system.
  • offline calibration and “online reconstruction” are specific Including the following detailed processing steps:
  • Step S10 acquiring 2D images and 3D point clouds under different viewing angles.
  • the image data collection involved in the two data processing links of "offline calibration” and “online reconstruction” can be for the same subject or for different subjects.
  • the 2D images and 3D point clouds obtained in the "offline calibration” link are from specific calibration objects, such as calibration plates or other objects with rich texture features
  • the 2D images and 3D points obtained in the "online reconstruction” link The point cloud is from the chest area of the user to be scanned by ultrasound; for example, in the latter case, the 2D image and 3D point cloud obtained in the "offline calibration” and “online reconstruction” links are from the user to be scanned for ultrasound.
  • the chest area is from the chest area of the user to be scanned by ultrasound.
  • the user’s breast area (for women) is a part that is susceptible to changes in its shape due to the influence of its own posture and external forces.
  • the breast area needs to be beamed before performing a comprehensive scan.
  • Shape for example, by wearing a chest vest with a certain elasticity to adjust the shape of the chest area and maintain the stability of the shape. Therefore, for each ultrasound scanning process, generally speaking, three-dimensional point cloud data needs to be re-acquired to obtain an accurate three-dimensional structure of the breast surface.
  • the user first lays flat on the screening platform 20 and adjusts the position according to the actual situation until the requirements of 3D point cloud data collection and ultrasound scanning are met, and then the 2D image and 3D image of the chest area are collected through the shooting device 30 Point cloud.
  • multiple shooting devices 30 can be arranged around the screening platform 20.
  • image data from different perspectives can be collected at the same time; a shooting device 30 that can move around the screening platform 20 can also be arranged.
  • image data under different viewing angles can be collected in time-sharing, and one of the two solutions can be selected from the front and back according to the specific structure of the breast ultrasound scanning device.
  • the image data in this embodiment may be RGB-D images. That includes RGB images and point clouds.
  • the user’s position can be adjusted by a cursor positioning device (not shown) provided with the photographing equipment 30.
  • the cursor positioning device can generate a cross laser line. (Respectively orthogonal horizontal laser line C and longitudinal laser line L), the user's posture meets the alignment of the cross laser line is the guarantee for the accurate output of the point cloud processing algorithm.
  • the scanning mentioned here starts from The starting line is probably located at the position of the clavicle or a certain distance below the clavicle. In specific applications, a reasonable selection can be made according to the differences of the objects to be scanned.
  • the 3D point cloud of the breast area By collecting the 3D point cloud of the breast area, the three-dimensional structure of the breast area can be accurately described, and the motion trajectory of the ultrasound probe 13 conforming to the actual scanning contact surface can be generated through the later scanning trajectory planning algorithm.
  • the breast 3D point cloud reconstruction method further includes:
  • Preprocessing is performed on each 3D point cloud.
  • the preprocessing includes point cloud downsampling, point cloud filtering and point cloud smoothing.
  • This step is performed after the 3D point cloud data is obtained.
  • point cloud data that is more suitable for the ultrasound scanning application scenario can be obtained, while reducing the complexity of the data and improving the data of the device Processing efficiency.
  • the input point cloud is relatively dense, and all processing takes a long time. Therefore, the input point cloud is down-sampled first to reduce the density of the point cloud and speed up the processing speed.
  • point cloud downsampling is to take a point from the original point cloud at a certain spatial distance to represent other points in its neighborhood, so that a more sparse point cloud can be obtained.
  • the specific point cloud downsampling setting standard It can be selected according to the data collection specifications of the photographing device 30 and the accuracy of post-data processing, and there is no limitation here.
  • the point cloud in the chest area should form a smooth continuous surface, but due to various reasons, there will be some abnormal point clouds (such as several isolated discrete points). These abnormal point clouds can be filtered out by point cloud filtering. , Output a higher quality point cloud for use in subsequent steps.
  • the filtered point cloud will be unsmooth due to the measurement error of the sensor, such as water wave-like ripples. Therefore, the point cloud can be further smoothed to make the point cloud surface smoother.
  • Step S20 Perform feature extraction and feature matching on the 2D image in each view to obtain a number of 2D matching point pairs.
  • the calibration object is a calibration board
  • the surf features are extracted from each 2D image of the calibration board, and the surf features of every two 2D images are matched separately to obtain Several 2D matching point pairs.
  • the aforementioned surf feature can be replaced with sift or ORB feature.
  • step S30 a 3D matching point pair is obtained according to the 2D matching point pair, and the coordinate transformation of the 3D matching point pair is calculated to obtain a transformation matrix of the two 3D point clouds with overlapping regions.
  • all three-dimensional point clouds whose angles with the ray OX in the point cloud are less than a certain value are intercepted, and the point cloud pieces are fitted into a spatial plane, and then, The intersection point of the ray OX and the space plane is calculated as the 3D point corresponding to the feature point.
  • the above 2D matching point pairs can be converted into 3D matching point pairs, and finally the 3D matching point pairs are input into the ICP algorithm to calculate the transformation relationship to obtain the transformation matrix of the two views.
  • the calibration parameter ⁇ Hij ⁇ of the matrix represents the conversion relationship between different views, where i and j are positive integers.
  • Step S40 Calculate the full-view transformation matrix according to the transformation matrix.
  • the full-view transformation matrix is the transformation matrix of the two views; if more than two views are reconstructed, the full-view transformation matrix can be a subset of the transformation matrix, or based on one A combination of sub-collections and modified by parameters.
  • the full-view transformation matrix is associated with all the views used for reconstruction, so the calibration parameters in the base coordinate system with full coverage can be obtained.
  • step S40 includes:
  • Step S41 Determine, according to the transformation matrix, two related photographing devices to establish a topological connection diagram of the photographing devices;
  • this step it is mainly to establish a topological connection graph (graph) of the photographing device 30 to indicate the relationship between the interconnected nodes. Specifically, it is determined that there are associated pairs of photographing devices 30 through a transformation matrix. If there is an effective transformation matrix, an edge is established, and the distance of each edge is defined as the spatial distance between the two end points of the edge corresponding to the photographing device 30. This distance calculation method is only a preferred solution. The set of interconnected nodes thus obtained is the topological connection diagram of the photographing device 30.
  • Step S42 Select a reference node from the topological connection graph, and calculate the shortest paths from the remaining nodes to the reference node;
  • the reference node can be selected according to the number of views captured by the photographing device 30, that is, in the pairwise calibration parameter ⁇ Hij ⁇ , the node corresponding to the view with the most occurrences is the reference node, or In the reconstruction calculation link, a certain node is manually designated as a reference node. After the reference node is determined, all paths from the remaining nodes to the reference node can be calculated, and the shortest path can be selected from these paths. The specific selection calculation method can be implemented by directly calling the shortest path algorithm, which will not be repeated here.
  • Step S43 Calculate the full-view transformation matrix from the node at the end to the reference node along the shortest path.
  • the transformation matrix calculated along the shortest path can represent the transformation parameters of all views to the base coordinate system, that is, the full view transformation matrix is obtained.
  • Step S50 according to the full-view transformation matrix, transform the 3D point clouds in all views of the breast area into the same base coordinate system to generate a complete three-dimensional point cloud of the breast area.
  • the 3D point cloud of all breast regions can be transformed into the same base coordinate system. Then through the post-processing link, a three-dimensional point cloud suitable for point cloud segmentation and trajectory planning is generated.
  • the processing result can be seen in the image shown in Figure 5. It should be noted that after matrix transformation is performed on multiple views, calibration parameters that can cover all views are obtained, and based on the shooting angle of view set in "Offline Calibration", 2D images of the user's chest area in multiple angles of view are collected And 3D point cloud, and transform the 3D point cloud into the same base coordinate system.
  • the solution adopted in this embodiment is to select the image area formed by the shooting device 30 with higher shooting accuracy from the views with overlapping areas. Specifically, in the link of "generating a complete three-dimensional point cloud of the breast area", the overlapping area where the point cloud overlaps is determined, and according to the shooting parameters between the point cloud in the overlapping area and the shooting device 30, it is determined from the overlapping area. The best point cloud is selected from the point clouds from multiple shooting devices to be used for the three-dimensional reconstruction of the overlapping area. For example, the shooting parameter is the deflection angle of the optical axis of the shooting device 30 relative to the target point cloud.
  • the smaller the deflection angle the more accurate the spatial information represented by the pixel. That is, calculate the angle between each overlapping point cloud and the origin of each shooting device and the optical axis of the shooting device; according to the included angle, filter out the best point cloud from several overlapping point clouds at the same position to Combined into a point cloud area for 3D reconstruction. Therefore, when eliminating redundant point clouds, a point cloud that can represent accurate position information can be selected according to the algorithm of this embodiment, and the obtained three-dimensional structure of the breast surface is more accurate.
  • the breast 3D point cloud reconstruction method further includes:
  • the effective point cloud pieces are filtered out from the surface.
  • the noise point cloud is generally a small area, so by using the continuous surface feature as the dividing condition, the remaining point cloud can be divided Into several point cloud areas.
  • the point cloud area where the breast is located has the largest area. By calculating and comparing the area of each point cloud area, the one with the largest surface area can be used as the point cloud area that needs to be retained, so that the area is used as the filter condition to filter from multiple surfaces A valid point cloud piece.
  • this embodiment extracts all transition regions of the point cloud piece.
  • the point cloud smoothing operation is performed on the transition area, so that the point cloud of the main area is spliced into a continuous piece.
  • the transition area is the location of the fault between the point cloud pieces. If the data is severely missing, it will have a greater impact on the later data processing.
  • this application can quickly perform offline calibration and online reconstruction of views of the breast area under multiple different shooting angles to obtain the full surface of the breast area.
  • the three-dimensional spatial information which provides the basis for subsequent scanning trajectory planning, when using fully automatic mechanized scanning methods to perform ultrasound scanning on the user’s breast area, the ultrasound probe can adjust the scanning posture according to the shape of the contact area , To ensure that the information covered by each frame of ultrasound image acquired is comprehensive and accurate, so as to make a comprehensive and accurate judgment of the physiological condition of the breast and its surrounding organs and tissues, so as to avoid the ultrasound image obtained by scanning due to human manipulation Occurrence of incomplete coverage and missing information.
  • the present application also provides a breast 3D point cloud reconstruction device.
  • the breast 3D point cloud reconstruction device includes:
  • the image acquisition module 100 is used to acquire 2D images and 3D point clouds in different viewing angles;
  • the feature matching module 200 is configured to perform feature extraction and feature matching on the 2D image in each view angle to obtain a number of 2D matching point pairs;
  • the matrix calculation module 300 is configured to obtain a 3D matching point pair according to the 2D matching point pair, and calculate the coordinate transformation of the 3D matching point pair to obtain a transformation matrix of every two 3D point clouds with overlapping regions;
  • the full-view matrix calculation module 400 is configured to calculate the full-view transformation matrix according to the transformation matrix
  • the model generation module 500 is configured to transform the 3D point clouds in all viewing angles into the same base coordinate system according to the full-view transformation matrix to generate a complete three-dimensional point cloud of the breast region.
  • the full-view matrix calculation module 400 includes:
  • the topological map establishing unit is used to determine the related pair of photographing devices according to the transformation matrix, so as to establish a topological connection diagram of the photographing devices;
  • the path calculation unit is used to select reference nodes from the topological connection graph, and calculate the shortest paths from the remaining nodes to the reference nodes;
  • the matrix calculation unit is used to calculate the full-view transformation matrix from the node at the end to the reference node along the shortest path.
  • Each module in the above-mentioned breast three-dimensional point cloud reconstruction device can be implemented in whole or in part by software, hardware and a combination thereof.
  • the above-mentioned modules may be embedded in the computer equipment in the form of hardware or independent of the computer equipment, and may also be stored in the memory in the server in the form of software, so that the computer equipment can call and execute the operations corresponding to the above-mentioned modules.
  • the computer equipment may be a central processing unit (CPU), a microcomputer equipment, a single-chip microcomputer, etc.
  • CPU central processing unit
  • microcomputer equipment a single-chip microcomputer
  • This application also provides a computer program storage medium in which computer program codes are stored, and when the computer program codes are executed by a processor, the following steps are implemented:
  • the 3D point cloud under all viewing angles of the breast area is transformed into the same base coordinate system to generate a complete three-dimensional point cloud of the breast area.
  • the computer device includes a processor 40, a memory 50, and computer program code stored in the memory 50.
  • the processor 40 calls the computer program code, the foregoing The steps of a breast three-dimensional point cloud reconstruction method provided in each embodiment.
  • the computer device may be a personal computer or a server.
  • the computer device includes a processor 40, a memory 50, and a communication interface (not shown) connected by a system bus.
  • the processor 40 is used to provide calculation and control capabilities, and support the operation of the entire computer equipment.
  • the memory 50 includes a non-volatile storage medium and an internal memory. An operating system and a computer program are stored in the non-volatile storage medium, and the computer program is executed by the processor 40 to realize a breast three-dimensional point cloud reconstruction method.
  • the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium.
  • the communication interface is used to communicate with an external server or terminal through a network connection.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

本申请公开了一种乳房三维点云重建方法，其包括：获取不同视角下的2D图像和3D点云；对每一个视角下的2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；根据变换矩阵计算全视图变换矩阵；根据全视图变换矩阵，将乳房区域所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。本申请的技术方案能够将胸部区域在多个不同的拍摄视角下的视图快速进行离线标定和在线重建，以获取乳房区域的全表面的三维空间信息，从而给后续的扫查轨迹规划提供基础。

Description

乳房三维点云重建方法、装置、存储介质及计算机设备 技术领域
本申请涉及点云数据处理技术领域，尤其涉及一种乳房三维点云重建方法、装置、存储介质及计算设备。
背景技术
超声波这种非手术的诊断性检查，对受检者无痛苦、无损伤、无放射性。并且，超声可以清晰地显示内脏器官及器官周围的各种断面图像，由于图像富于实体感，接近于解剖的真实结构，所以应用超声检查可以早期明确诊断。基于超声检查的诸多优势，从专业的医疗疾病诊断到日常化的健康指标评估，其应用范围越来越广。
随着医疗诊断技术的发展以及经济水平的提高，越来越多女性开始关注乳腺健康，而通过超声可以方便快捷地对乳腺生理状况进行初步判断，因此不管是公立医疗机构，还是盈利性健康服务机构，推出了多种乳腺筛查服务，以满足女性用户的需求。现有的乳腺筛查模式，基本上是借助于常规的超声设备，通过操作人员手持超声探头进行乳房区域的扫查，在扫查过程中，探头的扫查轨迹通常是操作人员的主观选择，可能存在未扫描到的位置，并且探头姿态也难以根据乳腺对应区域的乳房形状进行适应性调整，还可能存在超声图像信息缺失的现象，总的来说，仅依靠操作人员的主观操作来实现对乳房的全面扫查，存在诸多弊端，不利于对乳腺生理状况做出准确判断。
为了克服手持超声探头对乳房区域进行扫查而存在的多种弊端，最新的一种乳腺筛查模式是采用自动化的机械臂带动超声探头完成乳房区域的扫查过程，虽然依靠机械化的技术手段能够避免前述人工扫查存在的一些弊端，但是超声探头的运动轨迹是通过人工标定待扫查区域的若干坐标，并输入至相应的计算机软件编程得到的。然而，在确定超声探头的运动轨迹这一环节，通过人工标定坐标的方式不仅过程复杂，而且准确性差，从而对乳腺筛查的结果造成不利影响。
技术问题
本申请的主要目的在于提供一种乳房三维点云重建方法，旨在解决现有的机械化乳房超声扫查方式难以准确获取乳房区域的三维空间信息的技术问题。
技术解决方案
本申请解决上述技术问题所采用的技术方案如下：
一种乳房三维点云重建方法，包括：
获取不同视角下的2D图像和3D点云；
对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
根据所述变换矩阵计算全视图变换矩阵；
根据所述全视图变换矩阵，将所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
优选地，所述根据所述变换矩阵计算全视图变换矩阵包括：
根据所述变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
从所述拓扑连接图中选择参考节点，计算其余节点分别到所述参考节点的最短路径；
沿所述最短路径计算位于末端的节点到所述参考节点的全视图变换矩阵。
优选地，所述生成乳房区域的完整三维点云包括：
确定存在点云重叠的重叠区域，并根据所述重叠区域内的点云与拍摄设备之间的拍摄参数，从所述重叠区域内来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建。
优选地，所述根据所述重叠区域内的点云与拍摄设备之间的拍摄参数，从所述重叠区域内来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建包括：
计算每一个所述重叠区域内的每一个重叠点云与每一个拍摄设备的原点 连线与该拍摄设备的光轴之间的夹角；
根据所述夹角从同一位置的若干重叠点云中筛选出最佳点云，以组合成用于三维重建的点云区域。
优选地，所述生成乳房区域的完整三维点云包括：
对所述基坐标系中的点云进行区域分割，以得到若干连续的曲面；
根据预设的过滤条件从所述曲面中筛选出有效的点云片。
优选地，在所述根据预设的过滤条件从所述曲面中筛选出有效的点云片的步骤之后，所述方法还包括：
提取所述点云片的所有过渡区域，并对所述过渡区域进行点云平滑操作。
此外，本申请还提供一种乳房三维点云重建装置，包括：
图像获取模块，用于获取不同视角下的2D图像和3D点云；
特征匹配模块，用于对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
矩阵计算模块，用于根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
全视图矩阵计算模块，用于根据所述变换矩阵计算全视图变换矩阵；
模型生成模块，用于根据所述全视图变换矩阵，将所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
优选地，所述全视图矩阵计算模块包括：
拓扑图建立单元，用于根据所述变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
路径计算单元，用于从所述拓扑连接图中选择参考节点，计算其余节点分别到所述参考节点的最短路径；
矩阵计算单元，用于沿所述最短路径计算位于末端的节点到所述参考节点的全视图变换矩阵。
此外，本申请还提出一种计算机程序存储介质，所述计算机程序存储介质中存储有计算机程序代码，该计算机程序代码被处理器执行时实现上述乳房三维点云重建方法的步骤。
此外，本申请还提供一种计算机设备，包括处理器、存储器和存储在所述存储器中的计算机程序代码，所述处理器在调用所述计算机程序代码时， 实现上述乳房三维点云重建方法的步骤。
有益效果
相较于现有技术，本申请通过制定一套适用于女性乳房特点的三维点云重建算法，能够将胸部区域在多个不同的拍摄视角下的视图快速进行离线标定和在线重建，以获取乳房区域的全表面的三维空间信息，从而给后续的扫查轨迹规划提供基础，在采用全自动机械化的扫查方式来对用户乳房区域进行超声扫查时，可以使超声探头能够根据接触区域的形状调整扫查姿态，保证获取到的每一帧超声图像所涵盖的信息全面、准确，从而对乳腺及其周边器官、组织的生理状况进行全面、准确的判断，以避免因人为操作而导致扫查得到的超声图像覆盖面不全、信息缺失等情况的发生。
附图说明
图1为本申请公开的多个实施例可以在其中实施的示例环境的结构示意图；
图2为本申请公开的多个实施例中在采集胸部区域点云时的离线标定示意图；
图3为本申请公开的多个实施例中第一视角下的乳房区域图像；
图4为本申请公开的多个实施例中第二视角下的乳房区域图像；
图5为本申请公开的多个实施例中通过坐标变换得到的乳房区域图像；
图6为本申请的乳房点云三维重建方法一实施例的流程示意图；
图7为本申请的乳房点云三维重建方法另一实施例的流程示意图；
图8为本申请的乳房点云三维重建装置一实施例的功能模块示意图；
图9为本申请公开的多个实施例能够在其中实施的计算机设备的结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
本申请的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限 定本申请。
本申请提供一种乳房三维点云重建方法，其目的在于根据每位用户的胸部区域情况采集三维点云数据，为了获得胸部区域的全面三维结构，在多个不同视角下采集2D图像和3D点云，其中2D图像可以是RGB图像，至少保证采集两幅不同视角下的视图，并且该两幅视图应当具有一定重合度，从而根据重建模型对获取到的所有3D点云进行重建。在重建模型的基础上，后期可以从经过处理的三维点云数据中提取与胸部待扫查区域对应的点云，通过点云表示的位置结构信息制定超声扫查轨迹，以使乳房超声扫查设备能够根据该超声扫查轨迹执行扫查过程，进而生成包含信息全面、准确的超声图像。
如图1所示，为了给该方法的实施提供环境基础，前述乳房超声扫查设备主要包括扫查执行机构10、筛查平台20和拍摄设备30，其中扫查执行机构10包括主机11、与主机11连接的机械臂12以及安装在该机械臂12的执行末端处的超声探头13，本实施例中，主机11具有能够实现通信、数据处理和运动控制功能的相应硬件，主机11还具有用于安装机械臂12的基础架构，比如机械臂12被构造成能够提供三个直线运动自由度和两个以上旋转自由度的多轴结构，从而保证超声探头13能够根据待扫查区域的表面形状作适应性的姿态变换，具体应用时，机械臂12可以是五轴机械臂，或者是六轴机械臂。筛查平台20可以是固定式的支撑结构，也可以设置成能够提供位置调节的活动结构，比如通过设置升降机构，以调节筛查平台20的支撑面的高度，又比如通过设置水平移动机构，以调节筛查平台20的支撑面的水平位置，从而在不需要用户挪动身躯的情况下调节用户的初始位置。拍摄设备30设置在筛查平台20的上方，为了更加全面地获取三维点云数据，可以按照图1所示结构的指引，配置两套拍摄设备30，此示例中是以用户身躯的横向为参照方向布置拍摄设备30的，在其它实施方案中，以用户身躯的纵向为参照方向布置拍摄设备30也是能够满足要求的，本实施例的拍摄设备30可以是结构光传感器，当然也可以是激光雷达；又比如，拍摄设备30是安装在一个运动机构上的，通过该运动机构实现不同拍摄视角的变换，从而减少拍摄设备30的数量，在最低限度的情况下，可以仅布置一个拍摄设备30，该拍摄设备30通过沿某一设定的圆周运动而实现拍摄视角的变换，从而获取多个视角下的2D图像和3D点云，如图3、4所示，通过在两个不同视角下采集到的两幅乳房区域的 3D点云。
至此，已经详细介绍了本申请各个实施例的应用环境和相关设备的硬件结构和功能，并且上述乳房超声扫查设备的结构组成仅为基本功能的示例，并不是对乳房超声扫查设备实现方式的限制。下面，将基于上述应用环境和相关设备，详细介绍乳房三维点云重建方法的各个实施例。
如图6所示，在一实施例中，本申请提出的乳房三维点云重建方法主要包括两个数据处理环节，首先是根据采集到的图像数据进行离线标定，以获取用于视图重建的标定参数，然后是根据标定参数对采集到的乳房区域点云进行在线重建，从而将在线采集到的多个视图点云变换至统一坐标系中，具体地，“离线标定”和“在线重建”具体包括以下详细处理步骤：
步骤S10，获取不同视角下的2D图像和3D点云。
在本实施例中，“离线标定”和“在线重建”两个数据处理环节中涉及的图像数据采集可以是针对同一标的物，也可以针对不同标的物，具体而言，比如在前一种情形下，“离线标定”环节中获取到的2D图像和3D点云是来自特定的标定物体，比如标定板或其它具有丰富纹理特征的物体，而“在线重建”环节中获取到的2D图像和3D点云是来自待超声扫查用户的胸部区域；又比如在后一种情形下，“离线标定”和“在线重建”环节中获取到的2D图像和3D点云是来自待超声扫查用户的胸部区域。
用户的胸部区域(针对女性)作为容易受到自身姿势和外力影响而产生形状变化的部位，为了满足前述乳房超声扫查设备的技术要求，在执行全面的扫查动作前，需要对胸部区域进行束形，比如通过穿上具有一定弹性的束胸背心来调整胸部区域的形状，并保持外形的稳定性。因此，针对每一次的超声扫查过程，一般而言，均需要重新采集三维点云数据，以获取准确的乳房表面三维结构。在实际应用时，用户先平躺在筛查平台20上，并根据实际情况调整位置，直至满足三维点云数据采集和超声扫查的要求，然后通过拍摄设备30采集胸部区域的2D图像和3D点云。在实际应用时，可以通过围绕筛查平台20布置多个拍摄设备30，这种情况下，可以同时采集不同视角下的图像数据；还可以布置一个可以围绕筛查平台20运动的拍摄设备30，这种情况下，可以分时采集不同视角下的图像数据，可以根据乳房超声扫查设备的具体结构从前后两种方案中选择其中一种。为了保证获取到乳房区域的全 面三维结构，应当保持一定数量的拍摄视角(比如至少保持两个不同的视角)，并且拍摄的视场足够重叠，本实施例的图像数据可以是RGB-D图像，即包含了RGB图像和点云。
如图2所示，用户平躺在筛查平台20之后，可以通过与拍摄设备30配套设置的光标定位装置(图未示)对用户的位置进行调整，比如该光标定位装置能产生十字激光线(分别是正交的横向激光线C和纵向激光线L)，用户的姿势满足十字激光线对齐是点云处理算法输出准确结果的保证。在具体操作时，使用户的身体纵向中心线与纵向激光线L足够重合，同时使用户的身体胸部上侧的扫查起始线与横向激光线C足够重合，该处提及的扫查起始线大概位于锁骨所在位置或锁骨下方一定距离的位置，具体应用时可根据待扫查对象的差异性进行合理选择。
考虑到获取的原始点云数据覆盖面较广，需要对原始点云数据进行界限过滤，以简化数据的后期处理难度。通过采集乳房区域的3D点云，可以准确地描述乳房区域的三维结构，由此通过后期的扫查轨迹规划算法生成符合实际扫查接触面的超声探头13运动轨迹。
进一步地，在一较佳实施例中，该乳房三维点云重建方法还包括：
对每一幅3D点云进行预处理，该预处理包括点云降采样、点云滤波和点云平滑等。
该步骤是在获取到三维点云数据之后执行的，通过对三维点云数据进行预处理操作，可以获得更加符合超声扫查应用场景的点云数据，同时降低数据的复杂程度，提高设备的数据处理效率。具体地，输入的点云比较稠密，全部处理的话耗时较长，因此先对输入点云进行降采样，降低点云的密度，加快处理速度。直观上来说，点云降采样就是对原始点云每间隔一定的空间距离取一个点代表其邻域内的其它点，这样就可以得到一个更稀疏的点云，具体的点云降采样设定标准可以根据拍摄设备30的数据采集规格和后期数据处理精度选择，在此不作限制。此外，理论上胸部区域的点云应当构成一个平滑连续的曲面，但由于各种原因会存在一些异常点云(如孤立的几个离散点)，通过点云滤波就可以滤除这些异常点云，输出一个更高质量的点云供后续步骤使用。滤波后的点云由于传感器的测量误差，会有不平滑的现象，如水浪般的波纹，因此，还可以进一步对点云进行平滑处理，使点云曲面更加 平滑。
步骤S20，对每一个视角下的2D图像进行特征提取和特征匹配，以得到若干2D匹配点对。
以“离线标定”环节采用标定物的图像数据为例，比如标定物为标定板，分别对每一幅标定板2D图像提取surf特征，并且，分别匹配每两幅2D图像的surf特征，从而得到若干2D匹配点对。
在其它实施例中，上述surf特征可以替换为sift或ORB特征。
步骤S30，根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到该两幅具有重叠区域的3D点云的变换矩阵。
在本实施例中，为了获得每个特征点在三维点云中的对应3D坐标，首先，根据特征点的像素坐标x计算出该点在拍摄设备30的焦平面上的三维坐标X，将拍摄设备30的原点标记为O＝[0 0 1] T，则射线OX与点云的交点即为特征点对应的3D点。具体地，在一较佳实施方式中，为求出该交点，截取点云中所有与射线OX的夹角小于一定值的三维点云，并将该点云片拟合成空间平面，然后，计算射线OX与该空间平面的交点作为特征点对应的3D点。
在获得特征点对应的3D点后，可以将上述2D匹配点对转换为3D匹配点对，最后将3D匹配点对输入到ICP算法计算出变换关系，以得到两幅视图的变换矩阵，利用变换矩阵的标定参数{Hij}表示不同视图之间的转换关系，其中，i和j为正整数。
步骤S40，根据变换矩阵计算全视图变换矩阵。
若仅对两视图进行重建，则全视图变换矩阵是该两幅视图的变换矩阵；若对两幅以上的视图进行重建，则全视图变换矩阵可以是变换矩阵的一个子集合，或者是基于一个子集合的组合，并经过参数修正的。全视图变换矩阵是与所有用于重建的视图关联的，因此可以得到全覆盖的基坐标系下的标定参数。
参见图7，在一具体实施方式中，上述步骤S40包括：
步骤S41，根据变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
在该步骤中，主要是建立拍摄设备30的拓扑连接图(graph)，以表示互连节点的关系，具体是通过变换矩阵确定存在关联的两两拍摄设备30，若两 两拍摄设备30之间存在有效的变换矩阵，则建立一条边，并且定义每条边的距离为该边两端点对应拍摄设备30之间的空间距离，这种距离的计算方式仅为优选方案。由此得到的互连节点的集合即为拍摄设备30的拓扑连接图。
步骤S42，从拓扑连接图中选择参考节点，计算其余节点分别到参考节点的最短路径；
在该步骤中，参考节点可以根据拍摄设备30拍摄得到的视图的数量选择，也就是说，在两两标定参数{Hij}中，出现次数最多的视图所对应的节点为参考节点，或者，在重建计算环节中通过人工指定某一节点为参考节点。在确定了参考节点后，即可计算其余节点分别到参考节点的所有路径，并从这些路径中选出最短路径，具体的选择计算方法可以通过直接调用最短路径算法实现，在此不作赘述。
步骤S43，沿最短路径计算位于末端的节点到参考节点的全视图变换矩阵。
沿最短路径计算得到的变换矩阵，可以表示所有视图到基坐标系的变换参数，即获得全视图变换矩阵。
步骤S50，根据全视图变换矩阵，将乳房区域所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
通过计算两两视图之间的变换矩阵，并采用最短路径的算法从标定参数{Hij}中确定全覆盖的标定参数，由此可以将所有乳房区域的3D点云变换到同一基坐标系中，再通过后期处理环节生成适用于点云分割和轨迹规划的三维点云，处理结果可参见图5所示图像。需要说明的是，通过对多个视图进行矩阵变换，得到了能够覆盖全部视图的标定参数后，基于与“离线标定”中设定的拍摄视角，采集用户胸部区域在多个视角下的2D图像和3D点云，并将3D点云变换到同一基坐标系中。
进一步地，为了减少拍摄设备30在某些拍摄角度下造成的精度误差，本实施例采用的方案是，从存在重叠区域的视图中选取拍摄精度较高的拍摄设备30所成的图像区域。具体地，在“生成乳房区域的完整三维点云”的环节中，确定存在点云重叠的重叠区域，并根据重叠区域内的点云与拍摄设备30之间的拍摄参数，从存在重叠区域内来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建。比如，拍摄参数是拍摄设备30的光轴相对于标的点云的偏转角，根据成像特性，偏转角越小，像素表示的空间信 息越准确。即，计算每一个重叠点云与每一个拍摄设备的原点连线与该拍摄设备的光轴之间的夹角；根据夹角从同一位置的若干重叠点云中筛选出最佳点云，以组合成用于三维重建的点云区域。因此，在剔除冗余点云时，可以根据本实施例的算法选择能够表示准确位置信息的点云，所获得的乳房表面三维结构更加准确。
进一步地，在上述对重叠区域的点云进行筛选的步骤之后，在“生成乳房区域的完整三维点云”的环节中，该乳房三维点云重建方法还包括：
对基坐标系中的点云进行区域分割，以得到若干连续的曲面；
根据预设的过滤条件从曲面中筛选出有效的点云片。
在本实施例中，主要是将空间中存在的噪声点云进一步滤除，而噪声点云一般是较小范围的区域，因此通过以连续曲面特征为划分条件，即可将剩余的点云分割成若干点云区域。乳房所在的点云区域的面积最大，通过计算各个点云区域的面积并进行比较，即可将曲面面积最大的一个作为需要保留的点云区域，从而以面积作为过滤条件从多个曲面中筛选出有效的点云片。
而在得到了有效的点云片后，为了克服由于标定误差、结构光测量误差等因素引起的多个视图的点云不能完全重合的问题，本实施例通过提取点云片的所有过渡区域，并对过渡区域进行点云平滑操作，以使主要区域的点云拼接成连续的一片。这里，过渡区域是点云片之间的断层位置，若数据缺失严重，则会对后期的数据处理造成较大影响。
由此可见，本申请通过制定一套适用于女性乳房特点的点云重建算法，能够将胸部区域在多个不同的拍摄视角下的视图快速进行离线标定和在线重建，以获取乳房区域的全表面的三维空间信息，从而给后续的扫查轨迹规划提供基础，在采用全自动机械化的扫查方式来对用户乳房区域进行超声扫查时，可以使超声探头能够根据接触区域的形状调整扫查姿态，保证获取到的每一帧超声图像所涵盖的信息全面、准确，从而对乳腺及其周边器官、组织的生理状况进行全面、准确的判断，以避免因人为操作而导致扫查得到的超声图像覆盖面不全、信息缺失等情况的发生。
此外，本申请还提供一种乳房三维点云重建装置，如图8所示，该乳房三维点云重建装置包括：
图像获取模块100，用于获取不同视角下的2D图像和3D点云；
特征匹配模块200，用于对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
矩阵计算模块300，用于根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
全视图矩阵计算模块400，用于根据变换矩阵计算全视图变换矩阵；
模型生成模块500，用于根据所述全视图变换矩阵，将所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
在一较佳实施例中，全视图矩阵计算模块400包括：
拓扑图建立单元，用于根据变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
路径计算单元，用于从拓扑连接图中选择参考节点，计算其余节点分别到参考节点的最短路径；
矩阵计算单元，用于沿最短路径计算位于末端的节点到参考节点的全视图变换矩阵。
上述乳房三维点云重建装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中，也可以以软件形式存储于服务器中的存储器中，以便于计算机设备调用执行以上各个模块对应的操作。该计算机设备可以为中央处理单元(CPU)、微计算机设备、单片机等。上述各功能模块所起到的工作原理及起到的作用可参见图6、7中所示的乳房三维点云重建方法的实现过程，在此不作赘述。
本申请还提供一种计算机程序存储介质，该计算机程序存储介质中存储有计算机程序代码，该计算机程序代码被处理器执行时实现如下步骤：
获取不同视角下的2D图像和3D点云；
对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
根据变换矩阵计算全视图变换矩阵；
根据全视图变换矩阵，将乳房区域所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
该计算机程序被处理器执行时还实现了乳房三维点云重建方法的其它步骤，具体可参见上述乳房三维点云重建方法实施例的说明，在此不作赘述。
本申请还提供了一种计算机设备，如图9所示，该计算机设备包括处理器40、存储器50和存储在存储器50中的计算机程序代码，处理器40在调用该计算机程序代码时，实现上述各实施例中提供的一种乳房三维点云重建方法的步骤。
具体地，该计算机设备可为个人计算机或服务器。该计算机设备包括通过系统总线连接的处理器40、存储器50和通信接口(图未示)。其中，处理器40用于提供计算和控制能力，支撑整个计算机设备的运行。存储器50包括非易失性存储介质和内存储器。非易失性存储介质中存储有操作系统和计算机程序，该计算机程序被处理器40执行时以实现一种乳房三维点云重建方法。内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。通信接口用于与外部的服务器或终端通过网络连接通信。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种乳房三维点云重建方法，其特征在于，包括：
    获取不同视角下的2D图像和3D点云；
    对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
    根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
    根据所述变换矩阵计算全视图变换矩阵；
    根据所述全视图变换矩阵，将乳房区域所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
  2. 根据权利要求1所述的乳房三维点云重建方法，其特征在于，所述根据所述变换矩阵计算全视图变换矩阵包括：
    根据所述变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
    从所述拓扑连接图中选择参考节点，计算其余节点分别到所述参考节点的最短路径；
    沿所述最短路径计算位于末端的节点到所述参考节点的全视图变换矩阵。
  3. 根据权利要求1所述的乳房三维点云重建方法，其特征在于，所述生成乳房区域的完整三维点云包括：
    确定存在点云重叠的重叠区域，并根据所述重叠区域内的点云与拍摄设备之间的拍摄参数，从所述重叠区域内来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建。
  4. 根据权利要求3所述的乳房三维点云重建方法，其特征在于，所述根据所述重叠区域内的点云与拍摄设备之间的拍摄参数，从所述重叠区域内来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建包括：
    计算每一个所述重叠区域内的每一个重叠点云与每一个拍摄设备的原点连线与该拍摄设备的光轴之间的夹角；
    根据所述夹角从同一位置的若干重叠点云中筛选出最佳点云，以组合成用于三维重建的点云区域。
  5. 根据权利要求1所述的乳房三维点云重建方法，其特征在于，所述生成乳房区域的完整三维点云包括：
    对所述基坐标系中的点云进行区域分割，以得到若干连续的曲面；
    根据预设的过滤条件从所述曲面中筛选出有效的点云片。
  6. 根据权利要求5所述的乳房三维点云重建方法，其特征在于，在所述根据预设的过滤条件从所述曲面中筛选出有效的点云片的步骤之后，所述方法还包括：
    提取所述点云片的所有过渡区域，并对所述过渡区域进行点云平滑操作。
  7. 一种乳房三维点云重建装置，其特征在于，包括：
    图像获取模块，用于获取不同视角下的2D图像和3D点云；
    特征匹配模块，用于对每一个视角下的所述2D图像进行特征提取和特征匹配，以得到若干2D匹配点对；
    矩阵计算模块，用于根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到每两幅具有重叠区域的3D点云的变换矩阵；
    全视图矩阵计算模块，用于根据所述变换矩阵计算全视图变换矩阵；
    模型生成模块，用于根据所述全视图变换矩阵，将所有视角下的3D点云变换到同一基坐标系中，以生成乳房区域的完整三维点云。
  8. 根据权利要求7所述的乳房三维点云重建装置，其特征在于，所述全视图矩阵计算模块包括：
    拓扑图建立单元，用于根据所述变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图；
    路径计算单元，用于从所述拓扑连接图中选择参考节点，计算其余节点 分别到所述参考节点的最短路径；
    矩阵计算单元，用于沿所述最短路径计算位于末端的节点到所述参考节点的全视图变换矩阵。
  9. 一种计算机程序存储介质，其特征在于，所述计算机程序存储介质中存储有计算机程序代码，该计算机程序代码被处理器执行时实现权利要求1至6中任一项所述乳房三维点云重建方法的步骤。
  10. 一种计算机设备，包括处理器、存储器和存储在所述存储器中的计算机程序代码，其特征在于，所述处理器在调用所述计算机程序代码时，实现权利要求1至6中任意一项所述乳房三维点云重建方法的步骤。
PCT/CN2020/121234 2019-10-22 2020-10-15 乳房三维点云重建方法、装置、存储介质及计算机设备 WO2021078065A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911008528.2 2019-10-22
CN201911008528.2A CN110751719B (zh) 2019-10-22 2019-10-22 乳房三维点云重建方法、装置、存储介质及计算机设备

Publications (1)

Publication Number Publication Date
WO2021078065A1 true WO2021078065A1 (zh) 2021-04-29

Family

ID=69279421

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/121234 WO2021078065A1 (zh) 2019-10-22 2020-10-15 乳房三维点云重建方法、装置、存储介质及计算机设备

Country Status (2)

Country Link
CN (1) CN110751719B (zh)
WO (1) WO2021078065A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117689538A (zh) * 2024-02-02 2024-03-12 深圳市宗匠科技有限公司 一种针对断层式错位点云的拼接方法、装置、设备及介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751719B (zh) * 2019-10-22 2023-09-12 深圳瀚维智能医疗科技有限公司 乳房三维点云重建方法、装置、存储介质及计算机设备
CN113532311B (zh) * 2020-04-21 2023-06-09 广东博智林机器人有限公司 点云拼接方法、装置、设备和存储设备
CN111932613A (zh) * 2020-06-30 2020-11-13 深圳瀚维智能医疗科技有限公司 多视角点云标定方法、装置、设备及计算机可读存储介质
CN113940662A (zh) * 2020-07-16 2022-01-18 沈阳曼德医疗系统有限公司 智能化乳房外形3d测量方法及系统
CN112075957B (zh) * 2020-07-27 2022-05-17 深圳瀚维智能医疗科技有限公司 乳腺环扫轨迹规划方法、装置及计算机可读存储介质
CN112085698A (zh) * 2020-07-27 2020-12-15 深圳瀚维智能医疗科技有限公司 左右乳腺超声图像自动分析方法及装置
CN112070816B (zh) * 2020-07-29 2024-04-05 深圳瀚维智能医疗科技有限公司 用于按摩设备的人体背部三维点云重建方法、装置及计算机可读存储介质
CN111887907B (zh) * 2020-08-28 2024-05-31 喻堃 一种基于3d建模打印的丰胸注射定位模型获取方法
CN114663590A (zh) * 2022-03-23 2022-06-24 安翰科技(武汉)股份有限公司 医疗设备、胃部三维模型重建方法及其装置
CN114939874B (zh) * 2022-06-29 2023-08-01 深圳艾摩米智能科技有限公司 一种机械手臂沿人体表面移动的姿态规划方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908231A (zh) * 2010-07-27 2010-12-08 清华大学 处理含有主平面场景的三维点云重建方法和系统
CN106846461A (zh) * 2016-12-30 2017-06-13 西安交通大学 一种人体三维扫描方法
GB2550567A (en) * 2016-05-20 2017-11-29 Nokia Technologies Oy Point Cloud Matching Method
CN107958481A (zh) * 2016-10-17 2018-04-24 杭州海康威视数字技术股份有限公司 一种三维重建方法及装置
CN110751719A (zh) * 2019-10-22 2020-02-04 深圳瀚维智能医疗科技有限公司 乳房三维点云重建方法、装置、存储介质及计算机设备
CN107230225B (zh) * 2017-04-25 2020-06-09 华为技术有限公司 三维重建的方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104183010A (zh) * 2013-05-22 2014-12-03 上海迪谱工业检测技术有限公司 多视角三维在线重建的方法
CN105678748B (zh) * 2015-12-30 2019-01-15 清华大学 三维监控系统中基于三维重构的交互式标定方法和装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908231A (zh) * 2010-07-27 2010-12-08 清华大学 处理含有主平面场景的三维点云重建方法和系统
GB2550567A (en) * 2016-05-20 2017-11-29 Nokia Technologies Oy Point Cloud Matching Method
CN107958481A (zh) * 2016-10-17 2018-04-24 杭州海康威视数字技术股份有限公司 一种三维重建方法及装置
CN106846461A (zh) * 2016-12-30 2017-06-13 西安交通大学 一种人体三维扫描方法
CN107230225B (zh) * 2017-04-25 2020-06-09 华为技术有限公司 三维重建的方法和装置
CN110751719A (zh) * 2019-10-22 2020-02-04 深圳瀚维智能医疗科技有限公司 乳房三维点云重建方法、装置、存储介质及计算机设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117689538A (zh) * 2024-02-02 2024-03-12 深圳市宗匠科技有限公司 一种针对断层式错位点云的拼接方法、装置、设备及介质
CN117689538B (zh) * 2024-02-02 2024-05-17 深圳市宗匠科技有限公司 一种针对断层式错位点云的拼接方法、装置、设备及介质

Also Published As

Publication number Publication date
CN110751719A (zh) 2020-02-04
CN110751719B (zh) 2023-09-12

Similar Documents

Publication Publication Date Title
WO2021078065A1 (zh) 乳房三维点云重建方法、装置、存储介质及计算机设备
WO2021078064A1 (zh) 超声扫查轨迹规划方法、装置、存储介质及计算机设备
WO2021078066A1 (zh) 乳腺超声筛查方法、装置及系统
CN110675398B (zh) 乳腺超声筛查方法、装置及计算机设备
US11576645B2 (en) Systems and methods for scanning a patient in an imaging system
US11576578B2 (en) Systems and methods for scanning a patient in an imaging system
WO2020042720A1 (zh) 一种人体三维模型重建方法、装置和存储介质
CN109009211B (zh) 智能设备、基于超声波检测的方法及装置
WO2023273093A1 (zh) 一种人体三维模型获取方法、装置、智能终端及存储介质
US20190121427A1 (en) Iris and pupil-based gaze estimation method for head-mounted device
CN112057107A (zh) 超声扫查方法、超声设备及系统
CN112767309B (zh) 超声扫查方法、超声设备、系统以及存储介质
JP2017225835A (ja) 画像処理装置
BR112021013021A2 (pt) Dispositivo portátil para aquisição de dados antropométricos, e método de coleta de dados antropométricos
JP2022111704A (ja) 画像処理装置、医用画像撮像装置、画像処理方法、およびプログラム
JP2022111705A (ja) 学習装置、画像処理装置、医用画像撮像装置、学習方法およびプログラム
O'Reilly et al. A novel remote eye gaze tracking system using line illumination sources
Hernandez-Matas et al. Super resolution for fundoscopy based on 3D image registration
CN116958443A (zh) 基于smplx重建数字人可量化检测模型、方法及应用
CN115457093B (zh) 一种牙齿图像的处理方法、装置、电子设备及存储介质
WO2020133236A1 (zh) 一种脊柱的成像方法以及超声成像系统
CN113658107A (zh) 一种基于ct图像的肝脏病灶诊断方法及装置
CN111904450A (zh) 左心室中心和感兴趣区域的提取方法、装置及系统
CN114757951B (zh) 体征数据的融合方法、数据融合设备以及可读存储介质
WO2024140749A1 (zh) 超声扫描方法、装置和系统、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20878303

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20878303

Country of ref document: EP

Kind code of ref document: A1