WO2021078066A1 - 乳腺超声筛查方法、装置及系统 - Google Patents

乳腺超声筛查方法、装置及系统 Download PDF

Info

Publication number
WO2021078066A1
WO2021078066A1 PCT/CN2020/121237 CN2020121237W WO2021078066A1 WO 2021078066 A1 WO2021078066 A1 WO 2021078066A1 CN 2020121237 W CN2020121237 W CN 2020121237W WO 2021078066 A1 WO2021078066 A1 WO 2021078066A1
Authority
WO
WIPO (PCT)
Prior art keywords
ultrasound
area
scanning
breast
point cloud
Prior art date
Application number
PCT/CN2020/121237
Other languages
English (en)
French (fr)
Inventor
谈继勇
李冬玲
李元伟
翟亚光
严梓阳
陈尚均
税国强
李彬
庄鹏飞
武小斐
谭明晓
陈春雨
杨光耀
秦辉
Original Assignee
深圳瀚维智能医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳瀚维智能医疗科技有限公司 filed Critical 深圳瀚维智能医疗科技有限公司
Publication of WO2021078066A1 publication Critical patent/WO2021078066A1/zh

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • A61B8/0825—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of the breast, e.g. mammography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
    • A61B8/4444—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to the probe
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5223—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for extracting a diagnostic or physiological parameter from medical diagnostic data

Definitions

  • This application relates to the technical field of ultrasound diagnosis, and in particular to a breast ultrasound screening method, device and system.
  • breast cancer is an increasing threat to the health of women around the world. According to the report of Global Cancer Statistics 2018, breast cancer has surpassed lung cancer with the highest incidence in humans and has become the cancer with the highest incidence in women. From the perspective of the characteristics of breast cancer, breast cancer develops slowly in the early stage, and the screening time is sufficient, which can be as long as ten years. As long as women guarantee to have breast cancer screening every year, they can basically ensure that they stay away from breast cancer.
  • the early stage of breast cancer is carcinoma in situ and does not require radiotherapy or chemotherapy. The success rate of direct intervention is very high, and the 5-year survival rate of patients can exceed 95%.
  • the main purpose of this application is to provide a breast ultrasound screening method, which aims to solve the technical problem that the existing breast ultrasound screening method is highly dependent on professional doctors.
  • a method of breast ultrasound screening including:
  • model reconstruction according to the depth image to obtain a three-dimensional structure model of the area to be scanned, and generating a scanning trajectory of the ultrasound probe according to the three-dimensional structure model;
  • the method before the step of obtaining the depth image of the chest region of the user, the method further includes:
  • a screening serial number is generated and added to the screening waiting queue.
  • the method further includes:
  • said performing validity analysis on the acquired ultrasound image and adjusting the scanning posture of the ultrasound probe according to the result of said validity analysis includes:
  • the posture compensation amount of the ultrasound probe is calculated to adjust the scanning posture of the ultrasound probe.
  • the analyzing and processing the acquired ultrasound image to generate a diagnosis result includes:
  • the diagnosis data is classified according to the BI-RADS classification to generate a diagnosis result.
  • the analyzing and processing the acquired ultrasound image to generate a diagnosis result further includes:
  • the acquired ultrasound images are sent to the remote diagnosis terminal for analysis and processing.
  • the performing model reconstruction according to the depth image to obtain a three-dimensional structure model of the area to be scanned, and generating the scanning trajectory of the ultrasound probe according to the three-dimensional structure model includes:
  • the present application also provides a breast ultrasound screening device, including:
  • the image acquisition module is used to acquire the depth image of the user's chest area
  • a trajectory generation module configured to perform model reconstruction according to the depth image to obtain a three-dimensional structure model of the area to be scanned, and to generate a scanning trajectory of the ultrasound probe according to the three-dimensional structure model;
  • the scanning control module is configured to generate a motion control code according to the scanning trajectory, and input the motion control code to a scanning mechanism to control the scanning mechanism to drive the ultrasound probe to perform ultrasound scanning on the user's breast area ;
  • the diagnosis module is used to analyze and process the acquired ultrasound images to generate diagnosis results.
  • the present application also provides a breast ultrasound screening system, including a host, a photographing device, a scanning mechanism and an ultrasound probe, in which:
  • the shooting device is used to collect a depth image of the chest area of the user
  • the host is used to reconstruct the model of the depth image to obtain the three-dimensional structure model of the area to be scanned, and to generate the scanning trajectory of the ultrasound probe according to the three-dimensional structure model; the host is also used to Check trajectory to generate motion control code;
  • the scanning mechanism is configured to receive the motion control code output from the host, and drive the ultrasound probe to perform ultrasound scanning on the breast area of the user according to the motion control code;
  • the host is also used to analyze and process the acquired ultrasound images to generate diagnosis results.
  • the breast ultrasound screening system further includes a user information entry device, and the user information entry device includes an information entry module and a number calling module, wherein:
  • the information entry module is used to enter user personal information
  • the number calling module is used to generate a screening serial number according to the user's personal information and add it to the screening waiting queue.
  • this application formulates a set of plans for mass ultrasound screening of breasts to reduce the degree of reliance on professional doctors, thereby reducing screening costs and expanding the scope of application.
  • the full-surface three-dimensional space information of the breast area is constructed, and the scanning trajectory is generated, and the scanning mechanism is controlled to drive the ultrasonic probe to move according to the motion control code converted from the scanning trajectory.
  • the whole process adopts fully automatic mechanization
  • the scanning method is used to perform ultrasound scanning on the user’s breast area, which enables the ultrasound probe to adjust the scanning posture according to the shape of the contact area, ensuring that the information covered by each frame of ultrasound image acquired is comprehensive and accurate, so that the breast and its Comprehensive and accurate judgments on the physiological conditions of surrounding organs and tissues. Therefore, the pipelined operation makes a large-scale mass breast cancer screening possible.
  • FIG. 1 is a schematic structural diagram of one example environment in which multiple embodiments disclosed in this application can be implemented;
  • FIG. 2 is a schematic structural diagram of another example environment in which multiple embodiments disclosed in this application can be implemented;
  • FIG. 3 is a flowchart of the operation of breast ultrasound screening in multiple implementations disclosed in this application;
  • FIG. 4 is a schematic diagram of offline calibration when collecting the point cloud of the chest area in multiple embodiments disclosed in the application;
  • FIG. 5 is a point cloud diagram from a first perspective in multiple embodiments disclosed in this application.
  • FIG. 6 is a point cloud diagram from a second perspective in multiple embodiments disclosed in this application.
  • FIG. 7 is a point cloud of the chest area obtained through coordinate transformation in multiple embodiments disclosed in this application.
  • FIG. 8 is a schematic diagram of a point cloud obtained after preprocessing the image of the chest region in multiple embodiments disclosed in the application;
  • FIG. 9 is a schematic diagram of a breast scanning area point cloud obtained after cropping the breast area point cloud in multiple embodiments disclosed in the application.
  • FIG. 10 is a schematic diagram of a curved skeleton obtained through skeleton model reconstruction in multiple embodiments disclosed in this application;
  • FIG. 11 is a schematic flowchart of an embodiment of the breast ultrasound screening method of this application.
  • FIG. 12 is a schematic flowchart of another embodiment of the breast ultrasound screening method of this application.
  • FIG. 13 is a schematic flowchart of another embodiment of the breast ultrasound screening method of this application.
  • FIG. 14 is a schematic flowchart of another embodiment of the breast ultrasound screening method of this application.
  • FIG. 15 is a schematic flowchart of another embodiment of the breast ultrasound screening method of this application.
  • FIG. 17 is a schematic structural diagram of a computer device in which multiple embodiments disclosed in this application can be implemented.
  • the present application provides a breast ultrasound screening system, as shown in FIG. 1, the breast ultrasound screening system mainly includes a host (not shown), a photographing device 30, a scanning mechanism 10, and an ultrasound probe 13
  • the breast ultrasound screening system also includes a horizontal screening platform 20, thereby adopting a supine position for ultrasound screening, and in other embodiments, an upright posture can also be used for ultrasound screening. Screening, thus the aforementioned screening platform 20 can be omitted.
  • the host computer can be an industrial computer or other applicable computer equipment.
  • the host computer is used as the upper computer
  • the scanning mechanism 10 communicating with the host is used as the lower computer
  • the host computer and the scanning mechanism 10 The connection can be established through the TCP/IP communication protocol.
  • the screening platform 20 can be a fixed support structure or a movable structure that can provide position adjustment. For example, by setting a lifting mechanism to adjust the height of the supporting surface of the screening platform 20, or by setting a horizontal moving mechanism, In order to adjust the horizontal position of the supporting surface of the screening platform 20, the lifting mechanism and the horizontal moving mechanism can be hydraulic devices, or screw or rack and pinion transmission devices driven by a motor, so that the user does not need to move the body. Adjust the user’s initial position.
  • the photographing device 30 is set above the screening platform 20.
  • the depth image of this embodiment contains RGB images and point cloud data, which can be structured as shown in FIG. 1
  • two sets of shooting equipment 30 are configured.
  • the shooting device 30 is arranged with the user's body as the reference direction. In other embodiments, it is also possible to arrange the shooting device 30 with the user's body as the reference direction.
  • the photographing device 30 of this embodiment may be a structured light sensor, of course, it may also be a lidar; for another example, the photographing device 30 is installed on a motion mechanism, through which the conversion of different shooting angles can be realized, thereby reducing shooting
  • the number of devices 30, at a minimum, can be arranged with only one shooting device 30, the shooting device 30 realizes the change of the shooting angle of view by moving along a certain set circle, thereby obtaining point cloud images in multiple angles of view, As shown in Figures 5 and 6, through two point cloud images of the chest area collected from two different perspectives.
  • the scanning mechanism 10 mainly includes a control device 11 and a robotic arm 12 communicatively connected with the control device 11.
  • the ultrasonic probe 13 is installed at the execution end of the robotic arm 12.
  • the control device 11 has the ability to realize communication and data processing.
  • the robotic arm 12 is constructed to provide a multi-axis structure with three linear motion degrees of freedom and two or more rotational degrees of freedom, thereby ensuring that the ultrasonic probe 13 can adapt to the surface shape of the area to be scanned
  • the robotic arm 12 can be a five-axis robotic arm or a six-axis robotic arm.
  • the scanning mechanism 10' collects the ultrasound images of the user's left and right breasts through the ultrasound probes 13' installed on the two robotic arms 12', wherein the two robotic arms 12' Each has at least three degrees of freedom in mutually perpendicular directions.
  • the two mechanical arms 12' are driven by a linear motion mechanism to move in the up and down (ie Z axis), front and back (ie Y axis) and left and right (ie X axis) directions.
  • the two mechanical arms 12' are both arranged on a support frame (not shown) through a linear motion mechanism, and the two mechanical arms 12' are arranged in a hoisted state to facilitate the mechanical arm 12' to drive the ultrasonic probe 13' to move.
  • the present application realizes that during the movement of the two mechanical arms 12', the two mechanical arms 12' do not interfere with each other during their respective movement and working processes.
  • the linear motion mechanism includes two first linear guides 121' arranged along the X-axis direction, two second linear guides 122' arranged along the Y-axis direction, and two third linear guides 123' arranged along the Z-axis direction,
  • the two first linear guides 121' are in a horizontal state and are arranged at intervals on the support frame;
  • the two second linear guides 122' are installed on the first linear guide 121 through sliding blocks that are slidingly matched with the first linear guide 121' ′On;
  • the two third linear guide rails 123′ are respectively mounted on the two second linear guide rails 121′ through the sliding block slidingly matched with the second linear guide rail 122′, and the two mechanical arms 12′ are respectively connected to the two third linear guide rails.
  • the sliding block on the guide rail 123' is connected.
  • This embodiment adopts the solution of dual mechanical arms 12', which can simultaneously drive two ultrasound probes 13' to perform scanning actions, thereby greatly reducing the time for performing one breast ultrasound screening.
  • the mechanical arm 12' provided in this embodiment includes a first rotating assembly 124', a second rotating assembly 125', and a clamp.
  • the first rotating assembly 124' and the output end of the linear motion mechanism that is, the third linear guide 123'
  • the first rotating assembly 124' is used to drive the second rotating group 125' to rotate around the X axis
  • the second rotating assembly 125' is used to drive the clamp to rotate around the Y axis
  • the clamp is used to clamp the ultrasonic
  • the probe 13', and the first rotating assembly 124' and the second rotating assembly 125' are arranged up and down.
  • both the first rotating component 124' and the second rotating component 125' can adopt the same structure or different structures, such as a synchronous wheel assembly, a rack and pinion, and a separate motor.
  • the breast ultrasound screening system further includes a user information entry device (not shown in the figure), and the user information entry device includes an information entry module and a number calling module.
  • the information entry module is used to enter the user's personal information.
  • the information entry module is an ID card information reader, which mainly completes the reading of the ID card information through the RFID chip.
  • the user's personal information can be in the database accordingly Create a new screening account, you can also match the established screening account in the database based on this.
  • the user's personal information can also be written manually by manual entry. For example, a touch screen is provided, and the user’s personal information is generated on the touch screen for entry.
  • the number calling module is used to generate the screening serial number according to the user's personal information and add it to the screening waiting queue. In this way, waiting for breast ultrasound screening can ensure that the screening work is carried out in an orderly manner.
  • the user terminal can also be connected to the breast ultrasound screening system through a wireless network (WI-FI, 4G, 5G wireless channels widely used by the public, etc.). For example, the user terminal can follow the "breast screening system" on WeChat. “Check” (the name of the official account here is only an example) to establish a communication connection with the data processing center of the breast ultrasound screening system; another example is that the user terminal is installed with an APP provided by a breast ultrasound screening service provider. Start the APP to establish a communication connection with the data processing center of the breast ultrasound screening system, so that information from the breast ultrasound screening system can be received through the official account or APP, which includes account information, call number information, and ultrasound images And the diagnosis result, etc.
  • the main process of using the breast ultrasound screening system of the present application to perform mass breast cancer screening on users includes: user information collection; scanning model establishment; ultrasound scanning; image analysis and diagnosis.
  • user information collection can be obtained through a user information input device
  • scanning model establishment can be obtained by collecting depth images at specific locations and processing the depth images according to the set algorithm model
  • ultrasound scanning It is the process of obtaining ultrasound images by inputting the planned scanning trajectory into the scanning organization, and driving the ultrasound probe through the scanning organization to obtain ultrasound images
  • image analysis and diagnosis is the use of deep learning-based algorithm models to input ultrasound images Perform analysis and processing to output the diagnosis result.
  • the full-surface three-dimensional space information of the breast area is constructed, and the scanning trajectory is generated, and the scanning mechanism is controlled to drive the ultrasonic probe to move according to the motion control code converted from the scanning trajectory.
  • the whole process adopts fully automatic mechanization
  • the scanning method is used to perform ultrasound scanning on the user's breast area, which enables the ultrasound probe to adjust the scanning posture according to the shape of the contact area, ensuring that the information covered by each frame of ultrasound image acquired is comprehensive and accurate. Therefore, the pipelined operation makes a large-scale mass breast cancer screening possible.
  • the present application provides a breast ultrasound screening method, including:
  • Step S10 Obtain a depth image of the chest area of the user.
  • the user's chest area (for women) is a part that is susceptible to changes in its shape due to the influence of its own posture and external forces.
  • the user first lays down on the screening platform and adjusts the position according to the actual situation until it meets the requirements of 3D point cloud data collection and ultrasound scanning, and then passes The camera captures a depth image of the chest area.
  • multiple shooting devices can be arranged around the screening platform. In this case, images of the chest area from different perspectives can be collected at the same time; a shooting device that can move around the screening platform can also be arranged. In this case Next, you can collect images of the chest area from different perspectives in time-sharing, and you can choose one of the two options according to the specific structure of the breast ultrasound screening system.
  • the image of the breast area in this embodiment may be an RGB-D image .
  • the user's position can be adjusted through the cursor positioning device (not shown) provided with the shooting equipment.
  • the cursor positioning device can generate cross laser lines (respectively). It is the orthogonal horizontal laser line C and the longitudinal laser line L).
  • the user's posture meets the alignment of the cross laser line is the guarantee of the accurate output of the point cloud processing algorithm.
  • the three-dimensional structure of the chest area can be accurately described, and the trajectory of the ultrasound probe that conforms to the actual scanning contact surface can be generated through the later scanning trajectory planning algorithm.
  • the breast ultrasound screening method further includes:
  • Each depth image is preprocessed, and the preprocessing includes point cloud downsampling, point cloud filtering, and point cloud smoothing.
  • This step is performed after the depth image is obtained.
  • point cloud data that is more suitable for the ultrasound scanning application scenario can be obtained, while reducing the complexity of the data and improving the data processing efficiency of the device .
  • the input point cloud is relatively dense, and all processing takes a long time. Therefore, the input point cloud is down-sampled first to reduce the density of the point cloud and speed up the processing speed.
  • point cloud downsampling is to take a point from the original point cloud at a certain spatial distance to represent other points in its neighborhood, so that a more sparse point cloud can be obtained.
  • the specific point cloud downsampling setting standard It can be selected according to the data acquisition specifications of the shooting equipment and the accuracy of post-data processing, and there is no restriction here.
  • the point cloud in the chest area should form a smooth continuous surface, but due to various reasons, there will be some abnormal point clouds (such as several isolated discrete points). These abnormal point clouds can be filtered out by point cloud filtering. , Output a higher quality point cloud for use in subsequent steps.
  • the filtered point cloud will be unsmooth due to the measurement error of the sensor, such as water wave-like ripples. Therefore, the point cloud can be further smoothed to make the point cloud surface smoother.
  • the breast ultrasound screening method further includes:
  • the user's personal information is entered through the ID card information reader.
  • the ID card information reader mainly uses the RFID chip to read the ID card information.
  • a new screen can be established in the database. Check the account, you can also match the established screening account in the database accordingly.
  • the user's personal information can also be written manually by manual entry, for example, a touch screen is provided, and the user's personal information is generated on the touch screen for entry. The interactive interface of the user's personal information. Waiting for breast ultrasound screening in this way can ensure that the screening work is carried out in an orderly manner.
  • the user terminal can also be connected to the breast ultrasound screening system through wireless networks (WI-FI, 4G, 5G, etc.).
  • WI-FI wireless networks
  • the user terminal can follow the "breast screening" (the public number here) on WeChat. The name is just an example.)
  • the official account is used to establish a communication connection with the data processing center of the breast ultrasound screening system; another example is that the user terminal is installed with an APP provided by a breast ultrasound screening service provider.
  • the communication connection of the data processing center of the breast ultrasound screening system can receive information from the breast ultrasound screening system through the official account or APP. This information includes account information, number information, ultrasound images, and diagnosis results.
  • step S20 the model is reconstructed according to the depth image to obtain a three-dimensional structure model of the area to be scanned, and the scanning trajectory of the ultrasound probe is generated according to the three-dimensional structure model.
  • the depth image is mainly processed to obtain the image processing link of the scan trajectory.
  • the image processing link mainly includes model reconstruction, region segmentation and trajectory planning.
  • model reconstruction multiple depth images from different perspectives can be transformed to a unified coordinate system; through region segmentation, it can be extracted from the original point cloud data
  • the point cloud of the area to be scanned is used for subsequent trajectory planning.
  • step S20 specifically includes:
  • Step S21 Perform coordinate transformation on multiple depth images under different viewing angles to obtain a three-dimensional point cloud of the chest area in the same base coordinate system.
  • the offline calibration method is used to calculate the calibration parameters of the coordinate transformation, and then the collected chest area point cloud is reconstructed online according to the calibration parameters, thereby transforming the multiple view point clouds collected online to the same base coordinate Department.
  • the 2D images and 3D point clouds obtained in the "offline calibration" link are from specific calibration objects, such as calibration plates or other objects with rich texture features, and "online reconstruction"
  • the 2D image and 3D point cloud obtained in the link are from the chest area of the user to be scanned by ultrasound.
  • the calibration object is a calibration board
  • the surf features are extracted from each 2D image of the calibration board, and the surf features of every two 2D images are matched separately to obtain Several 2D matching point pairs.
  • the 2D image is an RGB image included in the depth image.
  • the aforementioned surf feature can be replaced with sift or ORB feature.
  • the three-dimensional coordinates X of the point on the focal plane of the shooting device are calculated according to the pixel coordinates x of the feature point, and the shooting device
  • the intersection point of the ray OX and the point cloud is the 3D point corresponding to the feature point.
  • all three-dimensional point clouds whose angles with the ray OX in the point cloud are less than a certain value are intercepted, and the point cloud pieces are fitted into a spatial plane, and then, The intersection point of the ray OX and the space plane is calculated as the 3D point corresponding to the feature point.
  • the above 2D matching point pairs can be converted into 3D matching point pairs, and finally the 3D matching point pairs are input into the ICP algorithm to calculate the transformation relationship to obtain the transformation matrix of the two views, using the transformation
  • the calibration parameter ⁇ H ij ⁇ of the matrix represents the conversion relationship between different views, where i and j are positive integers.
  • the full view transformation matrix is the transformation matrix of the two views; if more than two views are reconstructed, the full view transformation matrix can be one of the set of transformation matrices, or the transformation matrix One of the set, which has been modified by parameters.
  • the full-view transformation matrix is associated with all the views used for reconstruction, so the calibration parameters in the base coordinate system with full coverage can be obtained.
  • the above step “calculating the full-view transformation matrix according to the transformation matrix” includes:
  • this step it is mainly to establish a topological connection graph (graph) of the photographing equipment to indicate the relationship between the interconnected nodes. Specifically, it is determined that there is a pair of related photographing equipment through a transformation matrix. If there is a valid connection between the two photographing equipment In the transformation matrix, one side is established, and the distance of each side is defined as the spatial distance between the two end points of the side corresponding to the photographing equipment. This distance calculation method is only a preferred solution. The set of interconnected nodes thus obtained is the topological connection diagram of the photographing device.
  • the reference node can be selected according to the number of views captured by the camera, that is, in the pairwise calibration parameter ⁇ H ij ⁇ , the node corresponding to the view with the most occurrences is the reference node, or A certain node is manually designated as a reference node in the reconstruction calculation link.
  • all the paths from the remaining nodes to the reference node can be calculated, and the shortest path can be selected from these paths.
  • the specific selection calculation method can be implemented by directly calling the shortest path algorithm, which will not be repeated here.
  • the transformation matrix calculated along the shortest path can represent the transformation parameters from all views to the base coordinate system, that is, the full view transformation matrix is obtained.
  • the 3D point cloud from all perspectives of the chest area is transformed into the same base coordinate system to generate a complete three-dimensional point cloud of the chest area:
  • the solution adopted in this embodiment is to select the image area formed by the shooting device with higher shooting accuracy from the views with overlapping areas.
  • the overlapping area where the point cloud overlaps is determined, and according to the shooting parameters between the point cloud in the overlapping area and the shooting device, the overlapped area is derived from multiple points.
  • the best point cloud is selected from the point clouds of the two shooting devices to be used for the 3D reconstruction of the overlapping area.
  • the shooting parameter is that the optical axis of the shooting device 30 is equivalent to the deflection angle of the target point cloud.
  • the smaller the deflection angle the more accurate the spatial information represented by the pixel. That is, calculate the angle between each overlapping point cloud and the origin of each shooting device and the optical axis of the shooting device; according to the included angle, filter out the best point cloud from several overlapping point clouds at the same position to Combined into a point cloud area for 3D reconstruction. Therefore, when eliminating redundant point clouds, a point cloud that can represent accurate position information can be selected according to the algorithm of this embodiment, and the obtained three-dimensional structure of the breast surface is more accurate.
  • the breast ultrasound screening method further includes:
  • the effective point cloud pieces are filtered out from the surface.
  • the noise point cloud is generally a small area, so by using the continuous surface feature as the dividing condition, the remaining point cloud can be divided Into several point cloud areas.
  • the point cloud area where the breast is located has the largest area. By calculating and comparing the area of each point cloud area, the one with the largest surface area can be used as the point cloud area that needs to be retained, so that the area is used as the filter condition to filter from multiple surfaces A valid point cloud piece.
  • this embodiment extracts all transition regions of the point cloud piece.
  • the point cloud smoothing operation is performed on the transition area, so that the point cloud of the main area is spliced into a continuous piece.
  • the transition area is the location of the fault between the point cloud pieces. If the data is severely missing, it will have a greater impact on the later data processing.
  • Step S22 Segment the three-dimensional point cloud of the breast area according to a preset point cloud segmentation algorithm to obtain a point cloud of the breast scanning area.
  • the point cloud segmentation algorithm used in this step mainly includes:
  • the upper division boundary of the chest constructs a first vertical tangent plane, and uses the first vertical tangent plane as a reference to offset a preset distance from the head to the foot of the human body to obtain the second vertical tangent plane.
  • the point clouds on the two vertical tangent planes are fitted into the lower chest segmentation boundary; corresponding to the left and right breasts, respectively, extract the upper chest segmentation boundary, the central segmentation boundary, the axillary segmentation boundary and the lower chest segmentation boundary in the enclosed area
  • the point cloud is used as the point cloud of the breast scanning area.
  • this embodiment can also add the step of cutting the 3D region of interest to the point cloud segmentation algorithm. Since the point cloud acquisition device 30 is fixed, and the 3D space where the person is lying on the bed is also in a certain limited area, only point cloud data within a certain space range can be considered.
  • the 3D region of interest is defined as a 3D cube bounding box. Specifically, according to the principle of including the bed surface and the human breast area within the range of the screening platform, the bounding box XYZ three directions are determined by offline calibration The maximum and minimum coordinate values. After the bounding box is calibrated offline, all the point clouds in the bounding box are directly cropped for use in subsequent algorithm steps.
  • Figure 8 shows the result of cropping the 3D region of interest on the point cloud shown in Figure 7.
  • the part shown in the area A in Figure 7 is the key chest area.
  • the cropped result mainly includes the chest area P1 and the bed.
  • the point cloud data has been greatly simplified. It should be noted that the point clouds shown in FIGS. 7 and 8 are only the chest region corresponding to one breast of the human body.
  • a point cloud set including the breast area P1 and the bed plane area P2 is obtained by cutting the point cloud into the 3D region of interest.
  • the bed plane and the human body surface have significant distinguishing features, that is, the bed plane is a plane area with a larger area in the point cloud collection space, and the human body The surface is a curved area with a large area in the point cloud collection space.
  • the exposed area of the bed plane is affected by the body coverage position, the area representing the bed plane in the point cloud will be within a certain range However, it will not affect the accurate detection of the bed plane in this embodiment.
  • the preset conditions here mainly include two points, one is the area of the plane area, and the other is whether the plane area is located at the lower part of the entire point cloud.
  • the relevant algorithm in PCL can be used to identify the point cloud belonging to the plane area (for example, using the feature vector of each point as the associated parameter), and to calculate the area of the plane area.
  • the calculation method for the area of the cloud coverage area has been a relatively common content in the algorithm library of PCL, so I will not repeat it here.
  • the remaining point cloud After deleting the point cloud corresponding to the bed plane area, the remaining point cloud includes the chest area point cloud and the noise point cloud. Then, according to the continuity, the remaining point cloud is divided into several continuous surfaces.
  • the noise point cloud existing in the space is further filtered, and the noise point cloud is generally a small area, so by using the continuous surface feature as the dividing condition, the remaining point cloud can be divided into several point cloud areas .
  • the point cloud area where the breast is located has the largest area.
  • the one with the largest surface area can be regarded as the most significant point cloud area, so that the point cloud contained in the most significant point cloud area is taken as the chest Regional point cloud.
  • the point cloud of the chest area can also be filtered to eliminate some point clouds that cannot be used in the later planning and scanning trajectory. For example, filter out all the vertical distances from the highest point (such as the nipple position) less than a certain value (such as 10cm) point cloud, which constitutes the optimized chest area point cloud.
  • the tangent plane is fixed and can be calibrated offline, that is, when collecting point cloud data, the longitudinal centerline of the user's body coincides with the longitudinal laser line L Line, and the coincidence line where the scanning start line on the upper side of the user’s body chest coincides with the transverse laser line C.
  • the horizontal vertical tangent plane of the upper side of the chest segmentation boundary and the longitudinal vertical tangent plane of the center segmentation boundary can be determined directly according to the offline calibration data.
  • the axillary segmentation boundary can be the mid-axillary line or a position close to the mid-axillary line.
  • the specific position can be determined according to the movement stroke of the scanning mechanism, and the selected position of the axillary segmentation boundary may change.
  • equidistant slicing is used to determine the position of the axillary segmentation boundary, which is specifically based on the bed plane.
  • the coordinate plane determined by the XY axis coincides with the bed plane, that is, along The Z-axis upwards constructs a horizontal tangent plane in a certain step (such as 0.5cm).
  • a horizontal cutting plane can be constructed from the preset height of the bed plane.
  • the preset height can be specifically selected according to the figure of each user and input into the data processing equipment, for example, the preset height is 5 ⁇ 8cm, by resetting the starting position of the horizontal cutting plane, the number of slices is greatly reduced.
  • the surface normal of the point cloud on the horizontal tangent plane represents the curve of the axillary surface. Therefore, by calculating the surface normal of the point cloud and calculating the angle between the surface normal and the horizontal tangent plane, the position of the axillary surface can be evaluated Whether it meets the itinerary requirements of the scanning agency.
  • the first vertical tangent plane is constructed according to the determined upper side of the chest segmentation boundary, and the first vertical tangent plane is used as a reference to offset the predetermined distance from the head to the foot of the human body to obtain the second Vertical cutting plane.
  • the offset distance of the first vertical tangent plane can be set to several sets of constants. In actual applications, one of the constants is selected from the database as the offset distance according to the user’s age, height, and weight. Yes, for example, the constant can be a value selected arbitrarily in the range of 20 to 30 cm.
  • the tangent planes of the four segmentation boundaries can be used to filter the breast scan.
  • the regional point cloud provides an accurate point cloud basis for the subsequent scanning trajectory planning algorithm.
  • the cloud segmentation algorithm further includes:
  • the ideal lying position of the subject is that the centerline of the body is parallel to the centerline of the bed.
  • the scan will be incomplete or unexpected. Therefore, in order to ensure the safety of the scan and obtain comprehensive and accurate ultrasound images, it is necessary to check whether the subject's pose meets the requirements. If it does not meet the requirements of sufficient parallelism, the program returns and prompts to adjust the pose. By finding the angle bisector, the actual situation of the lying position can be evaluated.
  • the point cloud of the chest area is divided into horizontal and equal intervals of the body, and the interval distance can be adjusted (for example, 0.5cm), so as to obtain a series of horizontal slices of the point cloud of the chest area. Then, select the extreme point of the edge of the body from each transverse slice, that is, the lowest point of each slice and the point closest to the edge of the body. For the situation shown in Figure 9 (representing the left chest), the Y coordinate is the largest and the Z coordinate is The smallest point, but for the right breast, it is the point with the smallest Y coordinate and the smallest Z coordinate. Finally, all the extracted points are projected to the plane where the XY axis is located and a straight line is fitted to obtain a straight line equation.
  • the interval distance can be adjusted (for example, 0.5cm)
  • RANSAC or the least square method can be used to fit the point cloud to a linear equation.
  • the above-mentioned preset reference line is parallel to the X axis. If the angle between the angle bisector and the X axis is small enough, the parallel check is passed. For example, the included angle used as a reference is preset to 0 ⁇ 5°, otherwise it returns to fail.
  • the offset distance of the vertical tangent plane can also be calculated according to the two straight line equations.
  • the size of the preset distance is calculated according to the following formula:
  • W bd is the body width
  • r is the scale factor
  • d min is the minimum scan length.
  • the body width can be determined according to the equation of two straight lines, such as taking the midpoint of two edge straight lines, and calculating the distance between the two midpoints as the body width.
  • Step S23 Perform skeleton model reconstruction on the breast area structure according to the breast scan area point cloud to obtain a curved skeleton.
  • the data volume of the acquired three-dimensional point cloud data is relatively large, and the model needs to be reconstructed to meet the application requirements of the scanning trajectory planning algorithm while simplifying the data.
  • the point cloud is sliced according to a preset direction, and the direction of the human body is taken as a reference, and the slice operation is mainly performed along the horizontal and vertical directions of the body, and in the optimal slicing constraint condition, the slicing operation is performed at equal intervals. Slicing, so as to obtain a segment of the sub-point cloud of equal width, the width of each segment of the sub-point cloud can be flexibly adjusted according to the actual situation.
  • the ultrasonic probe adopts a strip scanning method, and the direction of the strip scanning is along the longitudinal direction of the body, so point cloud slices are performed along the transverse direction of the body.
  • This scanning method affects the movement mechanism. The requirements are low and the quality of ultrasound images can be guaranteed.
  • the three-dimensional point cloud data is sliced horizontally to obtain a number of sub-point clouds; the Bezier curve is used to curve-fit each sub-point cloud to obtain the curve skeleton.
  • the reconstructed curved skeleton is a more stable and reliable representation of the structure of the chest region, which is conducive to the post-processing of the algorithm.
  • this step for the fitting operation of the Bezier curve, reference can be made to the detailed description of this aspect in the prior art, which will not be repeated here.
  • step S24 each curve in the curve skeleton is divided according to the preset curve dividing condition, and all the dividing points on each curve are taken.
  • the horizontally distributed curves are divided into equal arc lengths, and the division distance is set according to the coverage of the ultrasound probe, so as to ensure that the ultrasound probe is scanning
  • the complete area to be scanned can be covered, and the overlap area can be reduced at the same time.
  • the obtained segmentation point is expressed as ⁇ S ij , 0 ⁇ i ⁇ A, 0 ⁇ j ⁇ B i ⁇ , where A is the number of curves in the curve skeleton, and B i is the i-th curve
  • the number of division points on the above, i and j are both positive integers.
  • Step S25 selecting multiple groups of segmentation points from the segmentation point set according to the preset ultrasonic scanning direction, and connecting each group of segmentation points into a scanning trajectory curve.
  • the simplest grouping method is to select The dividing points with the same number j on each curve in the curve skeleton form a group, so that a complete trajectory ⁇ S 0j , S 1j , S 2j ,..., S Aj ⁇ can be obtained.
  • any other applicable method may also be used to group the dividing points.
  • Step S26 Extract multiple trajectory points from the scanning trajectory curve, and calculate the attitude angle of each trajectory point.
  • the trajectory points are the aforementioned segmentation points.
  • This method of extracting trajectory points can simplify the data processing process.
  • one or more points between adjacent segmentation points can also be extracted as trajectory points.
  • the motion parameters of the scanning mechanism need to be combined to avoid data redundancy.
  • the ultrasonic probe By determining the five coordinates of each track point, the ultrasonic probe can be controlled to move to the specific position of the area to be scanned in the motion control program according to the XYZ coordinate value, and the ultrasonic probe can be controlled in the motion control program according to the attitude angle of Roll and Pitch.
  • the angle and posture of the probe should be adjusted to make the probe surface closely fit the surface of the area to be scanned.
  • the aforementioned calculation of the attitude angle of each trajectory point mainly adopts the following algorithm, and the specific steps include:
  • the unit direction vector of the XYZ coordinate axis of the track point is converted into the representation form of Euler angle, and the attitude angle is extracted.
  • the boundary radius of the neighborhood point set extracted with the track point as the center can be selected according to the desired calculation accuracy. There is no restriction on the range setting of the neighborhood point set. After setting the extraction range of the neighborhood point set, you can pass Calculate the PCA of the neighborhood point set to obtain the unit direction vector Vz of the track point on the Z axis.
  • attitude angles in three directions can actually be obtained. Specifically, several attitude angles can be extracted, which can be combined with the degree of freedom of motion provided by the ultrasound probe. Take the extraction of Roll and Pitch attitude angles as examples.
  • the motion limit of the ultrasonic probe can be calibrated and stored in the form of a data table for later use.
  • the track points are verified according to the data table, which can avoid accidents in the scanning process of the equipment.
  • smooth filtering is performed on each scanning trajectory curve, so that the ultrasonic probe moves more smoothly during the scanning process, and reduces local squeezing of the human body.
  • step S30 a motion control code is generated according to the scanning trajectory, and the motion control code is input to the scanning mechanism to control the scanning mechanism to drive the ultrasound probe to perform ultrasound scanning on the user's breast area.
  • the motion control code can be generated according to the position information represented by the points on the scanning trajectory.
  • the motion control code is expressed in the form of G code.
  • the specific can be It is realized by the configured multi-axis linkage motion control card, thereby controlling the scanning mechanism to drive the ultrasound probe to perform ultrasound scanning on the user's breast area.
  • the realization process of converting the coordinate information of the point into the motion control code is well known to those of ordinary skill in the art, so it will not be repeated here.
  • the ultrasound probe is controlled to contact the breast surface with a certain pressure, and in each scanning path, the posture of the acoustic wave emitting surface of the ultrasound probe is adjusted according to the characteristics of the breast surface to ensure that the height is obtained. Quality ultrasound images.
  • step S40 the acquired ultrasound image is analyzed and processed to generate a diagnosis result.
  • step S40 specifically includes:
  • Step S41 input the acquired ultrasound image into the AI diagnosis algorithm model for analysis and processing, so as to obtain diagnosis data;
  • Step S42 Perform classification processing on the diagnosis data according to the BI-RADS classification to generate a diagnosis result.
  • a convolutional neural network ie, AI diagnosis algorithm model
  • the convolutional neural network can be trained by providing training samples for a variety of lesions, and use random test samples to verify the convolutional nerve The reliability of the network.
  • the rapid detection and tracking of lesions adopts target detection and tracking algorithms based on convolutional neural networks to detect benign/malignant lesion targets in ultrasound images in real time and perform target tracking.
  • the diagnosis result can be embodied in text or graphic form.
  • the diagnosis result can be sent to the user terminal. Specifically, it can be presented in the form of WeChat official account, applet, APP, SMS, MMS, etc., so as to facilitate users to view.
  • the classification and recognition of lesions is based on the basic features of the extracted lesions, and a classification algorithm is used to give a benign and malignant classification or a more detailed classification.
  • a classification algorithm is used to give a benign and malignant classification or a more detailed classification.
  • the system will be classified into different levels according to BI-RADS (Breast imaging and data system, that is, breast imaging report and data system), so as to provide users with more standardized and easy-to-understand diagnostic reports.
  • BI-RADS Breast imaging and data system, that is, breast imaging report and data system
  • Level 0 Need to be recalled and evaluated after combining with other inspections
  • Level II Considering benign changes, it is recommended to follow-up regularly (such as once a year);
  • Grade IV There are abnormalities, malignant lesions cannot be completely ruled out, and biopsy is needed to confirm;
  • Level IVa Low probability of prone to malignancy
  • Level IVb The possibility of prone to malignancy is moderate
  • Level IVc The tendency to be malignant is highly likely
  • Grade V Highly suspected of malignant disease (almost identified as malignant disease), surgical resection and biopsy are required;
  • Grade VI Malignant lesions have been confirmed pathologically.
  • the acquired ultrasound images can be analyzed and processed by a local data processing device, such as a host, or sent to a remote data processing device via a wired/wireless network for analysis and processing.
  • a local data processing device such as a host
  • a remote data processing device via a wired/wireless network for analysis and processing.
  • the remote data processing device is Server or remote diagnostic terminal.
  • the diagnosis results of each user are stored in the database, and the diagnosis results are associated with account information, and follow-up screening arrangements and breast-related medical treatments are pushed to users based on the user's diagnosis results.
  • News In an example, if the user's screening results are BI-RADS 1 and BI-RADS 2, it can be considered that the user's current breast is in a normal state, but there is no guarantee that there will be no breast disease in the future. In this case, 11 months after this inspection, the system will send a reminder message to the user to remind the user to carry out the second-year breast cancer screening in time.
  • the breast ultrasound screening method further includes:
  • Step S50 Perform validity analysis on the acquired ultrasound image, and adjust the scanning attitude of the ultrasound probe according to the result of the validity analysis.
  • the image evaluation feedback strategy is used to adjust the scanning attitude of the ultrasound probe to ensure that the sound emission surface of the ultrasound probe is close to the breast surface as much as possible.
  • the ultrasound probe is deflected according to the position of the invalid region, and then the posture adjustment is realized.
  • step S50 specifically includes:
  • Step S51 Divide the acquired ultrasound image into multiple sub-regions, and calculate the proportion of the number of black pixels in each sub-region.
  • the pixel value of the black pixels is zero, and the proportion of the number of black pixels is the ratio of the number of black pixels in a divided single sub-region to the number of all pixels in the single sub-region.
  • the ultrasound image is divided into a number of continuous rectangular areas (sub-areas), each rectangular area is distributed with a certain number of black pixels, by calculating the number of black pixels in each rectangular area and all pixels The number of black pixels in the rectangular area can be calculated.
  • Step S52 judging whether the corresponding sub-region is an invalid imaging region according to the proportion of the number of black pixels.
  • the threshold of the proportion of black pixels in the invalid imaging area proposed in the present application is 75% to 88%, which is used as a judgment basis to confirm whether a sub-area is an invalid imaging area. For example, suppose that the ultrasound image is divided into several rectangular areas of the same size and continuously distributed. There are 15960 pixels in the rectangular area. If there are 1,2000-14000 black pixels in the rectangular area, it indicates the rectangular area. It is an invalid imaging area.
  • Step S53 Divide the left and right regions with the center line of the ultrasound image as a reference, count the number of invalid imaging regions in the left and right regions respectively, and calculate the area ratio of all invalid imaging regions in the left and right regions in the ultrasound image.
  • the ultrasound image is divided into a left area and a right area, the number of invalid imaging areas in the left area and the right area are respectively counted, and the area ratio of each to the ultrasound image is calculated. It can be understood that by dividing the obtained ultrasound image into a left area and a right area, the position of each invalid imaging area can be confirmed accordingly.
  • Step S54 Calculate the pose compensation amount of the ultrasound probe according to the area ratio, so as to adjust the scanning posture of the ultrasound probe.
  • the pose compensation amount of the ultrasound probe is calculated, which is actually to compensate the pose of the robotic arm with multiple degrees of freedom to pass the mechanical
  • the arm adjusts the pose of the ultrasound probe in real time, and specifically controls the rotation and/or depression of the ultrasound probe according to the calculated compensation amount.
  • the area ratio of the invalid imaging area in the left area is equal to the area ratio of the invalid imaging area in the right area, it indicates that there is no need to perform rotation compensation on the ultrasound probe, and only need to perform down pressure compensation on the ultrasound probe. Specifically, by multiplying the area ratio of the invalid imaging area in the left area or the area ratio of the invalid imaging area in the right area with the preset depression coefficient, the depression compensation amount of the ultrasound probe can be obtained.
  • the downward pressure compensation can make the ultrasound probe fit the human skin, reduce the invalid imaging area in the ultrasound image, and improve the imaging quality of the ultrasound image.
  • the ultrasonic probe is controlled to depress a preset height to reach the preset position, so that the ultrasonic probe can be attached to the breast surface, thereby obtaining a clearer ultrasound image, thereby improving diagnosis The accuracy of the results.
  • the area ratio of the invalid imaging area in the left area is not equal to the area ratio of the invalid imaging area in the right area
  • the area ratio of the invalid imaging area in the left area and the area ratio of the invalid imaging area in the right area need to be calculated. And multiply the obtained difference with the preset rotation coefficient to obtain the rotation compensation amount of the ultrasonic probe.
  • the ultrasonic probe is controlled to rotate a certain angle so that the area ratio of the invalid imaging area in the left area is the same as the area ratio of the invalid imaging area in the right area. It should be noted that if the area of the invalid imaging area in the left area is larger than the area of the invalid imaging area in the right area, the ultrasound probe is controlled to rotate to the left side of the human body by a certain angle; if the area of the invalid imaging area in the left area is If the area ratio is less than the area ratio of the invalid imaging area in the right area, the ultrasound probe is controlled to rotate a certain angle to the right side of the human body.
  • the area ratio of all invalid imaging regions in the left or right area at this time in the ultrasound image is calculated for use in the subsequent calculation of the depression compensation amount.
  • the breast ultrasound screening method of the present application with the help of automation technology and artificial intelligence technology, can make low-cost, large-scale mass breast cancer screening possible, and will greatly improve the breast cancer screening of Chinese women of the right age. Proportion, helps prevent and control breast cancer.
  • the present application also provides a breast ultrasound screening device.
  • the breast ultrasound screening device includes:
  • the image acquisition module 100 is used to acquire a depth image of the chest area of the user;
  • the trajectory generating module 200 is used to reconstruct the model according to the depth image to obtain the three-dimensional structure model of the area to be scanned, and to generate the scanning trajectory of the ultrasound probe according to the three-dimensional structure model;
  • the scanning control module 300 is configured to generate a motion control code according to the scanning trajectory and input the motion control code to the scanning mechanism to control the scanning mechanism to drive the ultrasound probe to perform ultrasound scanning on the user's breast area;
  • the diagnosis module 400 is used to analyze and process the acquired ultrasound images to generate a diagnosis result.
  • Each module in the above breast ultrasound screening device can be implemented in whole or in part by software, hardware and a combination thereof.
  • the above-mentioned modules may be embedded in the computer equipment in the form of hardware or independent of the computer equipment, and may also be stored in the memory in the server in the form of software, so that the computer equipment can call and execute the operations corresponding to the above-mentioned modules.
  • the computer equipment may be a central processing unit (CPU), a microcomputer equipment, a single-chip microcomputer, etc.
  • CPU central processing unit
  • microcomputer equipment a single-chip microcomputer
  • This application also provides a computer program storage medium in which computer program codes are stored, and when the computer program codes are executed by a processor, the following steps are implemented:
  • the computer device includes a processor 40, a memory 50, and computer program code stored in the memory 50.
  • the processor 40 calls the computer program code, the above The steps of a breast ultrasound screening method provided in each embodiment.
  • the computer device may be a personal computer or a server.
  • the computer device includes a processor 40, a memory 50, and a communication interface (not shown) connected by a system bus.
  • the processor 40 is used to provide calculation and control capabilities, and support the operation of the entire computer equipment.
  • the memory 50 includes a non-volatile storage medium and an internal memory. An operating system and a computer program are stored in the non-volatile storage medium, and the computer program is executed by the processor 40 to realize a breast ultrasound screening method.
  • the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium.
  • the communication interface is used to communicate with an external server or terminal through a network connection.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Physiology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

一种乳腺超声筛查方法，其包括：获取用户胸部区域的深度图像（S10）；根据深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据三维结构模型生成超声探头的扫查轨迹（S20）；根据扫查轨迹生成运动控制代码，并将运动控制代码输入至扫查机构，以控制扫查机构带动超声探头对用户的乳房区域进行超声扫查（S30）；对获取到的超声图像进行分析处理，以生成诊断结果（S40）。乳腺超声筛查方法借助自动化技术及人工智能技术，可以使低成本、大范围的群体性乳腺癌筛查成为可能，将大幅度提高我国适龄女性参加乳腺癌筛查的比例，有助于对乳腺癌防控。

Description

乳腺超声筛查方法、装置及系统 技术领域
本申请涉及超声诊断技术领域，尤其涉及一种乳腺超声筛查方法、装置及系统。
背景技术
乳腺癌对全球女性健康的威胁日益增大，据《2018年全球癌症统计数据》报告显示，乳腺癌超过人类发病率最高的肺癌，成为目前女性发病率占比最高的癌症。从乳腺癌的发病特点来看，乳腺癌在早期阶段发展缓慢，筛查时间充足，可长达十年，只要女性保证每年做一次乳腺癌筛查，基本就能确保远离乳腺癌。乳腺癌早期是属于原位癌，不需要进行放疗或化疗，直接干预的成功率非常高，患者5年存活率能超过95％。
2009年，国家开始全国范围内推动乳腺癌早期筛查。但截至目前，我国每年的乳腺癌早期筛查量非常有限，同时还存在地域分布不均的问题。什么原因导致了中国乳腺癌群体性筛查的普及性不足？主要是基层医生资源和设备的配备不足造成。超声技术是公认的适合做乳腺癌筛查的技术，在中国的乳腺癌筛查指南中，超声检查被列为检查乳腺癌的主要手段之一。因此，按照传统的乳腺癌筛查方式，受制于医生资源不足和超声设备成本高等因素，难以缓解当前乳腺癌群体性筛查的困境。
技术问题
本申请的主要目的在于提供一种乳腺超声筛查方法，旨在解决现有的乳腺超声筛查方式对专业医生依赖程度高的技术问题。
技术解决方案
本申请解决上述技术问题所采用的技术方案如下：
一种乳腺超声筛查方法，包括：
获取用户胸部区域的深度图像；
根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；
根据所述扫查轨迹生成运动控制代码，并将所述运动控制代码输入至扫 查机构，以控制所述扫查机构带动超声探头对用户的乳房区域进行超声扫查；
对获取到的超声图像进行分析处理，以生成诊断结果。
优选地，在所述获取用户胸部区域的深度图像的步骤之前，所述方法还包括：
录入用户个人信息；
根据所述用户个人信息生成筛查序列号并加入筛查等候队列。
优选地，在所述对获取到的超声图像进行分析处理，以生成诊断结果的步骤之前，所述方法还包括：
对获取到的超声图像进行有效性分析，并根据所述有效性分析的结果调整超声探头的扫查姿态。
优选地，所述对获取到的超声图像进行有效性分析，并根据所述有效性分析的结果调整超声探头的扫查姿态包括：
将获取到的超声图像划分成多个子区域，并计算每个子区域的黑色像素点的数量占比；
根据所述黑色像素点的数量占比判断其对应的子区域是否为无效成像区域；
以所述超声图像的居中线为参照划分出左右区域，统计所述无效成像区域分别在所述左右区域的数量，并计算所述左右区域中所有无效成像区域在所述超声图像中的面积占比；
根据所述面积占比，计算超声探头的位姿补偿量，以调整超声探头的扫查姿态。
优选地，所述对获取到的超声图像进行分析处理，以生成诊断结果包括：
将获取到的超声图像输入至AI诊断算法模型中进行分析处理，以得到诊断数据；
根据BI-RADS分级对所述诊断数据进行分级处理，以生成诊断结果。
优选地，所述对获取到的超声图像进行分析处理，以生成诊断结果还包括：
将获取到的超声图像发送至远程诊断终端进行分析处理。
优选地，所述根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹包括：
对不同视角下的多幅所述深度图像的点云数据进行坐标变换，以得到位于同一基坐标系下的胸部区域三维点云；
根据预设的点云分割算法对所述胸部区域三维点云进行分割，以得到乳房扫查区域点云；
根据所述乳房扫查区域点云对乳房区域结构进行骨架模型重建，以得到 曲线骨架；
按照预设的曲线分割条件对所述曲线骨架中的各条曲线进行分割，并取各条曲线上的所有分割点；
根据预设的超声扫查方向从分割点集合中选取多组分割点，并将每一组分割点连接成一条扫查轨迹曲线；
从所述扫查轨迹曲线中提取多个轨迹点，并计算各轨迹点的姿态角。
为实现上述目的，本申请还提供一种乳腺超声筛查装置，包括：
图像获取模块，用于获取用户胸部区域的深度图像；
轨迹生成模块，用于根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；
扫查控制模块，用于根据所述扫查轨迹生成运动控制代码，并将所述运动控制代码输入至扫查机构，以控制所述扫查机构带动超声探头对用户的乳房区域进行超声扫查；
诊断模块，用于对获取到的超声图像进行分析处理，以生成诊断结果。
为实现上述目的，本申请还提供一种乳腺超声筛查系统，包括主机、拍摄设备、扫查机构和超声探头，其中：
所述拍摄设备用于采集用户胸部区域的深度图像；
所述主机用于对所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；所述主机还用于根据所述扫查轨迹生成运动控制代码；
所述扫查机构用于接收来自所述主机输出的运动控制代码，并根据所述运动控制代码带动所述超声探头对用户的乳房区域进行超声扫查；
所述主机还用于对获取到的超声图像进行分析处理，以生成诊断结果。
优选地，所述乳腺超声筛查系统还包括用户信息录入装置，所述用户信息录入装置包括信息录入模块和叫号模块，其中：
所述信息录入模块用于录入用户个人信息；
所述叫号模块用于根据所述用户个人信息生成筛查序列号并加入筛查等候队列。
有益效果
相较于现有技术，本申请通过制定一套适用于对乳腺进行群体性超声筛查方案，以降低对专业医生的依赖程度，从而降低筛查成本和扩大应用范围。针对每一位用户的乳房特点构建乳房区域的全表面三维空间信息，并生成扫查轨迹，根据扫查轨迹转换得到的运动控制代码控制扫查机构带动超声探头运动，整个过程采用全自动机械化的扫查方式来对用户乳房区域进行超声扫 查，可以使超声探头能够根据接触区域的形状调整扫查姿态，保证获取到的每一帧超声图像所涵盖的信息全面、准确，从而对乳腺及其周边器官、组织的生理状况进行全面、准确的判断。因此，流水线式的操作使得大范围的群体性乳腺癌筛查成为可能。
附图说明
图1为本申请公开的多个实施例可以在其中实施的其中一种示例环境的结构示意图；
图2为本申请公开的多个实施例可以在其中实施的另一种示例环境的结构示意图；
图3为本申请公开的多个实施中乳腺超声筛查的操作流程图；
图4为本申请公开的多个实施例中在采集胸部区域点云时的离线标定示意图；
图5为本申请公开的多个实施例中第一视角下的点云图；
图6为本申请公开的多个实施例中第二视角下的点云图；
图7为本申请公开的多个实施例中通过坐标变换得到的胸部区域点云；
图8为本申请公开的多个实施例中对胸部区域图像进行预处理后得到的点云的示意图；
图9为本申请公开的多个实施例中对胸部区域点云进行裁剪后得到的乳房扫查区域点云的示意图；
图10为本申请公开的多个实施例中经过骨架模型重建得到的曲线骨架示意图；
图11为本申请的乳腺超声筛查方法一实施例的流程示意图；
图12为本申请的乳腺超声筛查方法另一实施例的流程示意图；
图13为本申请的乳腺超声筛查方法又一实施例的流程示意图；
图14为本申请的乳腺超声筛查方法又一实施例的流程示意图；
图15为本申请的乳腺超声筛查方法又一实施例的流程示意图；
图16为本申请的乳腺超声筛查装置一实施例的功能模块示意图；
图17为本申请公开的多个实施例能够在其中实施的计算机设备的结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
本申请的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限 定本申请。
为解决上述技术问题，本申请提供一种乳腺超声筛查系统，如图1所示，该乳腺超声筛查系统主要包括主机(图未示)、拍摄设备30、扫查机构10和超声探头13，在本实施例中，该乳腺超声筛查系统还包括水平放置的筛查平台20，由此采用平卧的姿势进行超声筛查，而在其它实施例中，还可以采用直立的姿势进行超声筛查，由此可以省略前述的筛查平台20。主机可以是工控机，或者是其它适用的计算机设备，在本实施例的硬件配置中，主机用作上位机，而与主机通信连接的扫查机构10用作下位机，主机与扫查机构10之间可通过TCP/IP通信协议建立连接。筛查平台20可以是固定式的支撑结构，也可以设置成能够提供位置调节的活动结构，比如通过设置升降机构，以调节筛查平台20的支撑面的高度，又比如通过设置水平移动机构，以调节筛查平台20的支撑面的水平位置，其中升降机构和水平移动机构可以是液压装置，也可以是由电机驱动的丝杠或齿轮齿条传动装置，从而在不需要用户挪动身躯的情况下调节用户的初始位置。
拍摄设备30设置在筛查平台20的上方，为了更加全面地获取深度图像(包含三维点云数据)，比如本实施例的深度图像包含RGB图像和点云数据，可以按照图1所示结构的指引，配置两套拍摄设备30，此示例中是以用户身躯的横向为参照方向布置拍摄设备30的，在其它实施方案中，以用户身躯的纵向为参照方向布置拍摄设备30也是能够满足要求的，本实施例的拍摄设备30可以是结构光传感器，当然也可以是激光雷达；又比如，拍摄设备30是安装在一个运动机构上的，通过该运动机构实现不同拍摄视角的变换，从而减少拍摄设备30的数量，在最低限度的情况下，可以仅布置一个拍摄设备30，该拍摄设备30通过沿某一设定的圆周运动而实现拍摄视角的变换，从而获取多个视角下的点云图，如图5、6所示，通过在两个不同视角下采集到的两幅胸部区域的点云图。
扫查机构10主要包括控制装置11和与控制装置11通信连接的机械臂12，超声探头13安装在该机械臂12的执行末端处，本实施例中，控制装置11具有能够实现通信、数据处理和运动控制功能的相应硬件，机械臂12被构造成能够提供三个直线运动自由度和两个以上旋转自由度的多轴结构，从而保证超声探头13能够根据待扫查区域的表面形状作适应性的姿态变换，具体应用时，机械臂12可以是五轴机械臂，或者是六轴机械臂。
如图2所示，在另一实施例中，扫查机构10′通过在两个机械臂12′上安装的超声探头13′对应采集用户左右两乳房的超声图像，其中两个机械臂12′均至少具有在相互垂直的三个方向上的自由度。两个机械臂12′均由直线运动机构驱动在上下(即Z轴)、前后(即Y轴)及左右(即X轴)方向上运动。
两个机械臂12′均通过直线运动机构设置在支撑架(图未示)上，且两个机械臂12′呈吊装状态布置，以方便机械臂12′驱动超声探头13′运动。具体地，本申请实现了两个机械臂12′在运动过程中，两个机械臂12′在各自的运动工作过程中互不干扰。
直线运动机构包括两个沿X轴方向布置的第一直线导轨121′、两个沿Y轴方向布置的第二直线导轨122′和两个沿Z轴方向布置的第三直线导轨123′，两个第一直线导轨121′呈水平状态且间隔布置在支撑架上；两个第二直线导轨122′通过与第一直线导轨121′滑动配合的滑块安装在第一直线导轨121′上；两个第三直线导轨123′通过与第二直线导轨122′滑动配合的滑块分别安装在两个第二直线导轨121′上，两个机械臂12′分别与两个第三直线导轨123′上的滑块连接。本实施例采用双机械臂12′的方案，可以同时驱动两个超声探头13′执行扫查动作，由此可以大大缩减执行一次乳腺超声筛查的时间。
具体地，本实施例提供的机械臂12′包括第一旋转组件124′、第二旋转组件125′以及夹具，第一旋转组件124′与直线运动机构的输出端(即第三直线导轨123′上的滑块)连接，且第一旋转组件124′用于驱动第二旋转组125′件绕X轴转动，第二旋转组件125′用于驱动夹具绕Y轴转动，夹具用于夹持超声探头13′，且第一旋转组件124′与第二旋转组件125′上下状态布置。其中，第一旋转组件124′和第二旋转组件125′均可采用同样的结构或不同的结构，如同步轮组件、齿轮齿条以及单独的电机等方式。
进一步地，该乳腺超声筛查系统还包括用户信息录入装置(图未示)，该用户信息录入装置包括信息录入模块和叫号模块。其中，信息录入模块用于录入用户个人信息，比如信息录入模块为身份证信息读取器，主要通过RFID芯片完成对身份证信息的读取，通过读取用户个人信息，可以据此在数据库中建立一个新的筛查账户，也可以据此在数据库中匹配已经建立的筛查账户。在其它实施例中，除了采用前述的非接触式信息读取技术，还可以采用人工录入的方式写入用户个人信息，比如提供一个触控显示屏，通过在触控显示屏上生成用于录入用户个人信息的交互界面。叫号模块用于根据用户个人信息生成筛查序列号并加入筛查等候队列，采用这种方式轮候乳腺超声筛查，可以保证筛查工作有序进行。此外，还可以通过无线网络(WI-FI、4G、5G被公众广泛使用的无线频道等)将用户终端接入乳腺超声筛查系统中，比如，用户终端通过在微信(WeChat)关注“乳腺筛查”(这里的公众号名称仅为示例)公众号，以建立其与乳腺超声筛查系统的数据处理中心的通信连接；又比如，用户终端安装有乳腺超声筛查服务商提供的APP，通过启动该APP以建立其与乳腺超声筛查系统的数据处理中心的通信连接，由此可以通过公众号或APP接收来自乳腺超声筛查系统的信息，这些信息包括账户信息、叫号 信息、超声图像以及诊断结果等。
如图3所示，采用本申请的乳腺超声筛查系统对用户进行群体性乳腺癌筛查的主要过程包括：用户信息采集；扫查模型建立；超声扫查；图像分析诊断。其中，“用户信息采集”可以通过用户信息录入装置获取得到；“扫查模型建立”可以通过采集特定位置的深度图像，并根据设定的算法模型对深度图像进行处理获得；“超声扫查”是通过将规划好的扫查轨迹输入至扫查机构中，通过扫查机构带动超声探头运动，以获取超声图像的过程；“图像分析诊断”是利用基于深度学习的算法模型对输入的超声图像进行分析处理，从而输出诊断结果。针对每一位用户的乳房特点构建乳房区域的全表面三维空间信息，并生成扫查轨迹，根据扫查轨迹转换得到的运动控制代码控制扫查机构带动超声探头运动，整个过程采用全自动机械化的扫查方式来对用户乳房区域进行超声扫查，可以使超声探头能够根据接触区域的形状调整扫查姿态，保证获取到的每一帧超声图像所涵盖的信息全面、准确。因此，流水线式的操作使得大范围的群体性乳腺癌筛查成为可能。
至此，已经详细介绍了本申请各个实施例的应用环境和相关设备的硬件结构和功能，并且上述乳腺超声筛查系统的结构组成仅为本申请的一部分实施例，并不是全部实施例。基于本申请中的实施例，本领域的普通技术人员在没有做出创造性劳动的前提下所获得的所有其它实施例，都属于本申请保护的范围。
下面，将基于上述应用环境和相关设备，详细介绍乳腺超声筛查方法的各个实施例。
如图11所示，本申请提供一种乳腺超声筛查方法，包括：
步骤S10，获取用户胸部区域的深度图像。
用户的胸部区域(针对女性)作为容易受到自身姿势和外力影响而产生形状变化的部位，为了满足前述扫查机构的技术要求，在执行全面的扫查动作前，需要对胸部区域进行束形，比如通过穿上具有一定弹性的束胸背心来调整胸部区域的形状，并保持外形的稳定性。因此，针对每一次的超声扫查过程，一般而言，均需要重新采集三维点云数据(深度图像)，以获取准确的乳房表面三维结构。在实际应用时，以平卧的姿势进行超声扫查为例，用户先平躺在筛查平台上，并根据实际情况调整位置，直至满足三维点云数据采集和超声扫查的要求，然后通过拍摄设备采集胸部区域的深度图像。在实际应用时，可以通过围绕筛查平台布置多个拍摄设备，这种情况下，可以同时采集不同视角下的胸部区域图像；还可以布置一个可以围绕筛查平台运动的拍摄设备，这种情况下，可以分时采集不同视角下的胸部区域图像，可以根据乳腺超声筛查系统的具体结构从前后两种方案中选择其中一种。为了保证 获取到乳房区域的全面三维结构，应当保持一定数量的拍摄视角(比如至少保持两个不同的视角)，并且拍摄的视场足够重叠，本实施例的胸部区域图像可以是RGB-D图像。
如图4所示，用户平躺在筛查平台之后，可以通过与拍摄设备配套设置的光标定位装置(图未示)对用户的位置进行调整，比如该光标定位装置能产生十字激光线(分别是正交的横向激光线C和纵向激光线L)，用户的姿势满足十字激光线对齐是点云处理算法输出准确结果的保证。在具体操作时，使用户的身体纵向中心线与纵向激光线L足够重合，同时使用户的身体胸部上侧的扫查起始线与横向激光线C足够重合，该处提及的扫查起始线大概位于锁骨所在位置或锁骨下方一定距离的位置，具体应用时可根据待扫查对象的差异性进行合理选择。
考虑到获取的原始点云数据覆盖面较广，需要对原始点云数据进行界限过滤，以简化数据的后期处理难度。通过采集胸部区域的三维点云数据，可以准确地描述胸部区域的三维结构，由此通过后期的扫查轨迹规划算法生成符合实际扫查接触面的超声探头运动轨迹。
进一步地，在一较佳实施例中，该乳腺超声筛查方法还包括：
对每一幅深度图像进行预处理，该预处理包括点云降采样、点云滤波和点云平滑等。
该步骤是在获取到深度图像之后执行的，通过对三维点云数据进行预处理操作，可以获得更加符合超声扫查应用场景的点云数据，同时降低数据的复杂程度，提高设备的数据处理效率。具体地，输入的点云比较稠密，全部处理的话耗时较长，因此先对输入点云进行降采样，降低点云的密度，加快处理速度。直观上来说，点云降采样就是对原始点云每间隔一定的空间距离取一个点代表其邻域内的其它点，这样就可以得到一个更稀疏的点云，具体的点云降采样设定标准可以根据拍摄设备的数据采集规格和后期数据处理精度选择，在此不作限制。此外，理论上胸部区域的点云应当构成一个平滑连续的曲面，但由于各种原因会存在一些异常点云(如孤立的几个离散点)，通过点云滤波就可以滤除这些异常点云，输出一个更高质量的点云供后续步骤使用。滤波后的点云由于传感器的测量误差，会有不平滑的现象，如水浪般的波纹，因此，还可以进一步对点云进行平滑处理，使点云曲面更加平滑。
为了提高乳腺超声筛查的自动化程度，在上述步骤S10之前，该乳腺超声筛查方法还包括：
录入用户个人信息；根据用户个人信息生成筛查序列号并加入筛查等候队列。
比如通过身份证信息读取器录入用户个人信息，身份证信息读取器主要 通过RFID芯片完成对身份证信息的读取，通过读取用户个人信息，可以据此在数据库中建立一个新的筛查账户，也可以据此在数据库中匹配已经建立的筛查账户。在其它实施例中，除了采用前述的非接触式信息读取技术，还可以采用人工录入的方式写入用户个人信息，比如提供一个触控显示屏，通过在触控显示屏上生成用于录入用户个人信息的交互界面。采用这种方式轮候乳腺超声筛查，可以保证筛查工作有序进行。此外，还可以通过无线网络(WI-FI、4G、5G等)将用户终端接入乳腺超声筛查系统中，比如，用户终端通过在微信(WeChat)关注“乳腺筛查”(这里的公众号名称仅为示例)公众号，以建立其与乳腺超声筛查系统的数据处理中心的通信连接；又比如，用户终端安装有乳腺超声筛查服务商提供的APP，通过启动该APP以建立其与乳腺超声筛查系统的数据处理中心的通信连接，由此可以通过公众号或APP接收来自乳腺超声筛查系统的信息，这些信息包括账户信息、叫号信息、超声图像以及诊断结果等。
步骤S20，根据深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据三维结构模型生成超声探头的扫查轨迹。
在该步骤中，主要是对深度图像进行处理，以得到扫查轨迹的图像处理环节。图像处理环节主要包括模型重建、区域分割和轨迹规划等，通过模型重建，可以将多个不同视角下的深度图像变换至统一的坐标系下；通过区域分割，可以从原始点云数据中提取出待扫查区域的点云，以供后续的轨迹规划使用。
如图15所示，在一较佳实施例中，步骤S20具体包括：
步骤S21，对不同视角下的多幅深度图像进行坐标变换，以得到位于同一基坐标系下的胸部区域三维点云。
在本实施例中，采用离线标定的方式计算坐标变换的标定参数，然后根据标定参数对采集到的胸部区域点云进行在线重建，从而将在线采集到的多个视图点云变换至同一基坐标系中。具体而言，比如在一种情形下，“离线标定”环节中获取到的2D图像和3D点云是来自特定的标定物体，比如标定板或其它具有丰富纹理特征的物体，而“在线重建”环节中获取到的2D图像和3D点云是来自待超声扫查用户的胸部区域。
对深度图像进行特征提取和特征匹配，以得到若干匹配点对：
以“离线标定”环节采用标定物的图像数据为例，比如标定物为标定板，分别对每一幅标定板2D图像提取surf特征，并且，分别匹配每两幅2D图像的surf特征，从而得到若干2D匹配点对。这里，2D图像是深度图像中包含的RGB图像。
在其它实施例中，上述surf特征可以替换为sift或ORB特征。
根据2D匹配点对得到3D匹配点对，并计算3D匹配点对的坐标变换，以得到该两幅具有重叠区域的3D点云的变换矩阵：
在本实施例中，为了获得每个特征点在三维点云中的对应3D坐标，首先，根据特征点的像素坐标x计算出该点在拍摄设备的焦平面上的三维坐标X，将拍摄设备的原点标记为O＝[0 0 1] T，则射线OX与点云的交点即为特征点对应的3D点。具体地，在一较佳实施方式中，为求出该交点，截取点云中所有与射线OX的夹角小于一定值的三维点云，并将该点云片拟合成空间平面，然后，计算射线OX与该空间平面的交点作为特征点对应的3D点。
在获得特征点对应的3D点后，可以将上述2D匹配点对转换为3D匹配点对，最后将3D匹配点对输入到ICP算法计算出变换关系，以得到两幅视图的变换矩阵，利用变换矩阵的标定参数{H ij}表示不同视图之间的转换关系，其中，i和j为正整数。
根据变换矩阵计算全视图变换矩阵：
若仅对两视图进行重建，则全视图变换矩阵是该两幅视图的变换矩阵；若对两幅以上的视图进行重建，则全视图变换矩阵可以是变换矩阵集合中的一个，或者是变换矩阵集合中的一个，并经过参数修正的。全视图变换矩阵是与所有用于重建的视图关联的，因此可以得到全覆盖的基坐标系下的标定参数。
在一具体实施方式中，上述步骤“根据变换矩阵计算全视图变换矩阵”包括：
根据变换矩阵确定存在关联的两两拍摄设备，以建立拍摄设备的拓扑连接图：
在该步骤中，主要是建立拍摄设备的拓扑连接图(graph)，以表示互连节点的关系，具体是通过变换矩阵确定存在关联的两两拍摄设备，若两两拍摄设备之间存在有效的变换矩阵，则建立一条边，并且定义每条边的距离为该边两端点对应拍摄设备之间的空间距离，这种距离的计算方式仅为优选方案。由此得到的互连节点的集合即为拍摄设备的拓扑连接图。
从拓扑连接图中选择参考节点，计算其余节点分别到参考节点的最短路径：
在该步骤中，参考节点可以根据拍摄设备拍摄得到的视图的数量选择，也就是说，在两两标定参数{H ij}中，出现次数最多的视图所对应的节点为参考节点，或者，在重建计算环节中通过人工指定某一节点为参考节点。在确定了参考节点后，即可计算其余节点分别到参考节点的所有路径，并从这些路径中选出最短路径，具体的选择计算方法可以通过直接调用最短路径算法实现，在此不作赘述。
沿最短路径计算位于末端的节点到参考节点的全视图变换矩阵：
沿最短路径计算得到的变换矩阵，可以表示所有视图到及基坐标系的变换参数，即获得全视图变换矩阵。
根据全视图变换矩阵，将胸部区域所有视角下的3D点云变换到同一基坐标系中，以生成胸部区域的完整三维点云：
通过计算两两视图之间的变换矩阵，并采用最短路径的算法从标定参数{H ij}中确定全覆盖的标定参数，由此可以将所有胸部区域的3D点云变换到同一基坐标系中，再通过后期处理环节生成适用于点云分割和轨迹规划的三维点云，处理结果可参见图7所示图像。需要说明的是，通过对多个视图进行矩阵变换，得到了能够覆盖全部视图的标定参数后，基于与“离线标定”中设定的拍摄视角，采集用户胸部区域在多个视角下的2D图像和3D点云，并将3D点云变换到同一基坐标系中。
进一步地，为了减少拍摄设备在某些拍摄角度下造成的精度误差，本实施例采用的方案是：从存在重叠区域的视图中选取拍摄精度较高的拍摄设备所成的图像区域。具体地，在“生成胸部区域的完整三维点云”的环节中，确定存在点云重叠的重叠区域，并根据重叠区域内的点云与拍摄设备之间的拍摄参数，从存在重叠区域来自多个拍摄设备的点云中筛选出最佳点云，以用于该重叠区域的三维重建。比如，拍摄参数是拍摄设备30的光轴相当于标的点云的偏转角，根据成像特性，偏转角越小，像素表示的空间信息越准确。即，计算每一个重叠点云与每一个拍摄设备的原点连线与该拍摄设备的光轴之间的夹角；根据夹角从同一位置的若干重叠点云中筛选出最佳点云，以组合成用于三维重建的点云区域。因此，在剔除冗余点云时，可以根据本实施例的算法选择能够表示准确位置信息的点云，所获得的乳房表面三维结构更加准确。
进一步地，在上述对重叠区域的点云进行筛选的步骤之后，在“生成胸部区域的完整三维点云”的环节中，该乳腺超声筛查方法还包括：
对基坐标系中的点云进行区域分割，以得到若干连续的曲面；
根据预设的过滤条件从曲面中筛选出有效的点云片。
在本实施例中，主要是将空间中存在的噪声点云进一步滤除，而噪声点云一般是较小范围的区域，因此通过以连续曲面特征为划分条件，即可将剩余的点云分割成若干点云区域。乳房所在的点云区域的面积最大，通过计算各个点云区域的面积并进行比较，即可将曲面面积最大的一个作为需要保留的点云区域，从而以面积作为过滤条件从多个曲面中筛选出有效的点云片。
而在得到了有效的点云片后，为了克服由于标定误差、结构光测量误差等因素引起的多个视图的点云不能完全重合的问题，本实施例通过提取点云 片的所有过渡区域，并对过渡区域进行点云平滑操作，以使主要区域的点云拼接成连续的一片。这里，过渡区域是点云片之间的断层位置，若数据缺失严重，则会对后期的数据处理造成较大影响。
步骤S22，根据预设的点云分割算法对所述胸部区域三维点云进行分割，以得到乳房扫查区域点云。
通过上述的模型重建操作，实现了将多幅不同拍摄视角下的深度图像统一到同一坐标系中，因此给本步骤中的点云分割提供了基础。具体地，该步骤中采用的点云分割算法主要包括：
根据预设条件从三维点云数据中将床平面区域对应的点云删除，以获得胸部区域点云；通过离线标定确定胸部区域点云的胸部上侧分割边界和中心分割边界；以床平面为基准，按照预设的高度递增值向上构建水平切平面，直到水平切平面上的点云满足预设的边界分割条件时，将当前水平切平面上的点云拟合成腋侧分割边界；根据胸部上侧分割边界构建第一竖直切平面，并以第一竖直切平面为基准往人体的头部至脚部的方向偏移预设距离，以得到第二竖直切平面，将第二竖直切平面上的点云拟合成胸部下侧分割边界；分别对应左右两侧乳房提取以胸部上侧分割边界、中心分割边界、腋侧分割边界和胸部下侧分割边界围合区域内的点云作为乳房扫查区域点云。
为了更进一步提高点云数据的处理效率，减少冗余数据的影响，本实施例还可以在点云分割算法中增加感兴趣3D区域裁剪这一环节。由于点云获取装置30是固定的，而且人躺在床上后所处的3D空间也是在一个确定的有限区域内，因此可以仅考虑一定空间范围内的点云数据。在本实施例中，感兴趣3D区域定义为一个3D立方体包围盒，具体地，按照能够包含筛查平台行程范围内的床面和人体胸部区域的原则，通过离线标定确定包围盒XYZ三个方向的最大和最小坐标值。离线标定出包围盒后，直接裁剪出包围盒内的所有点云供后续算法步骤使用。图8给出了对图7所示点云进行感兴趣3D区域裁剪后的结果，在图7中A区域示出的部分为关键的胸部区域，该裁剪后的结果主要包含胸部区域P1和床平面区域P2，点云数据得到了极大简化。需要说明的是，图7和图8中所示点云仅为人体其中一侧乳房对应的胸部区域。
如图8所示，以其中一侧的乳房位置为例，通过对点云进行感兴趣3D区域裁剪后，得到包含胸部区域P1和床平面区域P2的点云集合。在本实施例中，还需要将床平面区域P2的点云删除，床平面和人体表面有着显著的区别特征，即床平面在点云采集空间内是一片具有较大面积的平面区域，而人体表面在点云采集空间内是一片具有较大面积的曲面区域，在实际应用时，因为床平面显露出来的面积受到人体覆盖位置的影响，所以点云中表示床平面的区域会在一定范围内变化，但是不会对本实施例中准确检测床平面造成影 响。
具体地，这里的预设条件主要包括两点，一是平面区域的面积，二是平面区域是否位于整个点云的下部，通过从整个点云中分离出平面区域，并利用该预设条件对平面区域进行判断即可。在一较佳实施方式中，可以采用PCL(Point Cloud Library)中的相关算法识别属于平面区域的点云(比如利用每个点的特征向量作为关联参数)，以及计算平面区域的面积，关于点云覆盖区域的面积计算方法已为PCL的算法库中较为常见的内容，在此不作赘述。
将床平面区域对应的点云删除后，剩下的点云包含胸部区域点云和噪声点云。然后，根据连续性将剩下的点云分割成若干连续的曲面。
进一步地，将空间中存在的噪声点云进一步滤除，而噪声点云一般是较小范围的区域，因此通过以连续曲面特征为划分条件，即可将剩余的点云分割成若干点云区域。乳房所在的点云区域的面积最大，通过计算各个点云区域的面积并进行比较，即可将曲面面积最大的一个作为最显著点云区域，从而以最显著点云区域包含的点云作为胸部区域点云。进一步地，还可以对胸部区域点云进行筛选处理，剔除一些在后期规划扫查轨迹中使用不到的点云，比如筛选出所有离最高点(比如乳头位置)的垂直距离小于一定值(比如10cm)的点云，构成优化后的胸部区域点云。
如图9所示，对于胸部上侧分割边界和中心分割边界，其切平面是固定的，可以离线标定出来，也即采集点云数据时，用户的身体纵向中心线与纵向激光线L的重合线，以及用户的身体胸部上侧的扫查起始线与横向激光线C重合的重合线。由此，可以直接根据离线标定数据确定胸部上侧分割边界的横向竖直切平面，和中心分割边界的纵向竖直切平面。
腋侧分割边界可以是腋中线或接近腋中线的位置，具体位置可以根据扫查机构的运动行程确定，腋侧分割边界的选取位置是可能产生变化的。在本实施例中，采用等距切片的方式来确定腋侧分割边界的位置，具体是以床平面为基准，结合图9，比如由XY轴确定的坐标平面与床平面重合，也就是沿着Z轴往上按一定步长(比如0.5cm)构建水平切平面，针对每一次构建的水平切平面，均判断水平切平面上的点云是否满足预设的边界分割条件，当满足时停止向上的切片操作，并将当前水平切平面上的点云拟合成腋侧分割边界。可以理解的是，因为胸部区域点云表现出来的是曲面特征，所以水平切平面与胸部区域点云相交时会形成一条相交线，即水平切平面上的点云为该相交线上的点云。
为了减少数据的运算量，可以从床平面的预设高度开始构建水平切平面，该预设高度可以根据每一位用户的身材作具体选择，并输入数据处理设备中，比如预设高度为5～8cm，通过重新设定构建水平切平面的起始位置，大大减 少了切片的数量。
水平切平面上的点云的曲面法线代表了腋侧表面的曲面走向，因此通过计算点云的曲面法线，并计算曲面法线与水平切平面的夹角，可以评估腋侧表面的位置是否满足扫查机构的行程要求。
因为水平切平面上的点云足够多，所以将夹角的平均值与预设角度值作比较，具有更高的准确性。
胸部下侧分割边界确定的原则是至少超过乳房下边界，从而保证超声扫查的范围能将整个乳房所在区域覆盖。因此，根据已经确定的胸部上侧分割边界构建第一竖直切平面，并以第一竖直切平面为基准往人体的头部至脚部的方向偏移预设距离，即可得到第二竖直切平面。作为一种实现方式，第一竖直切平面的偏移距离可以设置为若干组常量，在实际应用时，根据用户的年龄、身高和体重等信息从数据库中选择其中一个常量作为偏移距离即可，比如该常量可以是20～30cm范围中任意选择的数值。得到第二竖直切平面后，就可以从胸部区域点云中筛选出与第二竖直切平面相交的点云，并根据该部分点云拟合成胸部下侧分割边界。
针对每一侧的乳房，在获取了其对应的胸部上侧分割边界、中心分割边界、腋侧分割边界和胸部下侧分割边界后，就可以利用该四处分割边界的切平面来筛选胸部扫查区域点云，为后续的扫查轨迹规划算法提供精确的点云基础。
在本申请的另一实施例中，为了保证数据处理的准确性，增加一个平躺位姿校验环节，具体地，该云分割算法还包括：
根据胸部区域点云计算胸部左右两侧的直线方程，并根据左右两侧的直线方程确定出角平分线，若角平分线与预设参考线之间所成的夹角小于预设值，则根据左右两侧的直线方程计算身体宽度。
理想的受试者平躺位姿是身体中线与床体中线平行，当身体中线相对床体中线倾斜超过一定角度时，会造成扫查不完全或出现意外情况。因此，为了保证扫查的安全性和获取到全面、准确的超声图像，需要检测受试者的位姿是否符合要求，如果不符合足够平行的要求则程序返回并提示调整位姿。通过求得角平分线，就可以评估平躺位姿的实际情况。
对单侧胸部，首先对胸部区域点云进行身体横向等间隔分片，间隔距离可调(比如取0.5cm)，这样得到胸部区域点云的一系列横向切片。然后，从每个横向切片中选取身体边缘的极值点，也就是每个切片的最低且最靠身体边缘的点，对图9示出的情况(表示左胸)就是Y坐标最大且Z坐标最小的点，但对右胸则是Y坐标最小且Z坐标最小的点。最后，将提取出的所有点投影到XY轴所在平面并进行直线拟合，得到直线方程。在本实施例中，可 以采用RANSAC或最小二乘法将点云拟合成直线方程。以图9所示的坐标系为例，上述预设参考线与X轴平行，如果角平分线与X轴的角度足够小则通过平行校验，比如用作参照的夹角预设值为0～5°，否则返回失败。
此外，在获取到了胸部左右两侧的直线方程后，还可以根据该两直线方程计算竖直切平面偏移距离。具体地，根据如下公式计算所述预设距离的大小：
d＝max(W bd·r，d min)
其中，W bd为身体宽度，r为比例系数，d min为最小扫查长度。
身体宽度可以根据两直线方程确定，比如取两条边缘直线的中点，并计算两个中点之间的距离作为身体宽度。比例系数可以根据用户的个体差异设置，或者采用通用值，比如r＝0.7。最小扫查长度的设置是为了避免估算得到的身体宽度过小，而未能全面覆盖待扫查区域，比如d min＝20cm，或者是大于20cm的一些可用数值。因此，采用量化的计算方式确定第一竖直切平面的偏移距离，准确性更高。
步骤S23，根据乳房扫查区域点云对乳房区域结构进行骨架模型重建，以得到曲线骨架。
经过前述的点云分割操作，可以获得较小范围的点云区域，在这个基础上进行轨迹规划，可以得到更加精确的结果。
获取到的三维点云数据的数据量较为庞大，需要对其进行模型的重建，在简化数据的同时，满足扫查轨迹规划算法的应用要求。具体地，根据预设方向对点云进行切片，以人体身躯的方向为参照，主要沿身躯的横向和纵向这两个方向进行切片操作，并且在优选的切片约束条件中，以等间距的方式切片，从而获得一段段等宽的子点云，每段子点云的宽度可以根据实际情况灵活调整。作为一种可能的实施方式，超声探头采用条形扫查的方式，并且该条形扫查的方向沿身躯的纵向，因此沿身躯的横向进行点云切片，这种扫查方式对运动机构的要求较低，并且能够保证超声图像的质量。
通过对三维点云数据进行横向切片以得到若干段子点云；使用贝塞尔曲线对每一段子点云进行曲线拟合以得到曲线骨架。
如图10所示，重建出的曲线骨架是胸部区域结构的更稳定可靠的表示方式，有利于算法的后期处理。本步骤中，关于贝塞尔曲线的拟合操作可参照现有技术中关于这方面的详细说明，在此不作赘述。
步骤S24，按照预设的曲线分割条件对曲线骨架中的各条曲线进行分割，并取各条曲线上的所有分割点。
在该步骤中，以前述选择的纵向条形扫查方式为例，对横向分布的各条曲线进行等弧长分割，并且根据超声探头的覆盖面大小设定分割间距，从而 保证超声探头在扫查过程中能够覆盖完整的待扫查区域，同时又能减少重合区。在执行曲线分割的环节，所得到的分割点表示为{S ij，0≤i＜A，0≤j＜B i}，其中A是曲线骨架中曲线的条数，B i是第i条曲线上的分割点数，i和j均取正整数，通过对点云进行坐标变换，可以得到各个分割点在超声探头对应的运动坐标系下的XYZ坐标值，关于点云坐标变换的原理，可参照现有技术的详细说明，在此不作赘述。
步骤S25，根据预设的超声扫查方向从分割点集合中选取多组分割点，并将每一组分割点连接成一条扫查轨迹曲线。
在该步骤中，根据预设的超声扫查方向从分割点集合中选取能够组合成扫查轨迹曲线的多组分割点，以纵向的条形扫查为例，最简单的分组方式是，选取曲线骨架中每条曲线上同序号j的分割点为一组，这样即可得到了一条完整的轨迹{S 0j，S 1j，S 2j，...，S Aj}。除了上述作为示例的分割点组合方式，还可以采用其它任意适用的方式进行分割点分组。
步骤S26，从扫查轨迹曲线中提取多个轨迹点，并计算各轨迹点的姿态角。
在该步骤中，作为较佳的实施方式，轨迹点即为前述分割点，这种轨迹点的提取方式能够简化数据处理的过程。当然，除了提取前述的分割点，还可以在相邻分割点之间额外提取一个或多个点作为轨迹点，这里需要结合扫查机构的运动参数，避免造成数据冗余。以提取的分割点作为轨迹点为例，并且匹配五自由度的扫查机构，则需要获取每个轨迹点坐标值和相应的姿态角，将轨迹点表示为P i＝[X i，Y i，Z i，R i，P i]，这五个量分别表示P i的XYZ坐标值和P i的Roll、Pitch姿态角，其中P i的XYZ坐标值根据前述的点云数据计算可以得到，因此该步骤主要计算轨迹点的两个姿态角。但是，如果提取得到的轨迹点并非前述分割点，那么还需计算这些未知轨迹点的XYZ坐标值。通过确定每个轨迹点的五个坐标量，根据XYZ坐标值可以在运动控制程序中控制超声探头运动至待扫查区域的具体位置，而根据Roll、Pitch姿态角可以在运动控制程序中控制超声探头应当调整到哪种角度姿态，以使探头表面与待扫查区域表面紧密贴合。
在一较佳实施例中，前述计算各轨迹点的姿态角主要采用以下算法，具体步骤包括：
提取轨迹点的邻域点集，并通过对邻域点集求PCA得到轨迹点在Z轴的单位方向向量Vz；
按照公式Vy＝Vz×[0 0 1] T、Vx＝Vy×Vz计算轨迹点在XY轴的单位方向向量Vx、Vy；
将轨迹点XYZ坐标轴的单位方向向量转换为欧拉角的表示形式，并提取姿态角。
以轨迹点为中心提取的邻域点集可根据所期望的计算精度选择边界半径，这里不对邻域点集的范围设定作出限制，设定好邻域点集的提取范围后，即可通过对邻域点集求PCA得到轨迹点在Z轴的单位方向向量Vz。
在将单位方向向量转换为欧拉角的表示形式后，实际上可以获取到三个方向的姿态角，具体是提取几个姿态角，可以结合超声探头所能提供的运动自由度，本实施例以提取Roll、Pitch姿态角作为示例。
此外，在计算各轨迹点的姿态角的步骤之后，考虑到一些轨迹点可能位于超声探头末端运动范围之外，因此需要对各轨迹点进行校验。导入超声探头末端的行程极限数据，并根据行程极限数据从轨迹点中滤除超声探头末端不可到达的点。
通常，可标定出超声探头的运动极限，并以数据表的形式存储起来备用，根据该数据表来对轨迹点进行校验，可以避免设备在扫查过程中出现意外。同时，滤除了一些轨迹点后，对每条扫查轨迹曲线进行平滑滤波，使得超声探头在扫查过程中动作更加平滑，减少对人体的局部挤压。
步骤S30，根据扫查轨迹生成运动控制代码，并将运动控制代码输入至扫查机构，以控制扫查机构带动超声探头对用户的乳房区域进行超声扫查。
在确定了扫查轨迹后，即可根据扫查轨迹上的点表示的位置信息生成运动控制代码，比如运动控制代码采用G代码的表示形式，通过将运动控制代码输入至扫查机构，具体可通过配置的多轴联动运动控制卡实现，从而控制扫查机构带动超声探头对用户的乳房区域进行超声扫查。在该步骤中，点的坐标信息转换成运动控制代码的实现过程已为本领域的普通技术人员所熟知，故在此不作赘述。根据已规划好的扫查轨迹，控制超声探头以一定压力接触乳房表面，并且在每一道扫查路径上，均根据乳房表面的曲面特点调整超声探头的声波发射面的姿态，从而保证获取到高质量的超声图像。
步骤S40，对获取到的超声图像进行分析处理，以生成诊断结果。
如图14所示，上述步骤S40具体包括：
步骤S41，将获取到的超声图像输入至AI诊断算法模型中进行分析处理，以得到诊断数据；
步骤S42，根据BI-RADS分级对诊断数据进行分级处理，以生成诊断结果。
基于深度学习技术，建立卷积神经网络(即AI诊断算法模型)对超声图像进行分析处理，卷积神经网络可以通过提供多种病灶训练样本训练得到，并使用随机测试样本校验该卷积神经网络的可靠性。病灶快速检测跟踪采用基于卷积神经网络的目标检测和跟踪算法，实时检测超声图像中的良/恶性病灶目标，并进行目标跟踪。诊断结果可以通过文本形式体现，也可以通过图 文形式体现，比如将诊断结果发送到用户终端，具体可以通过微信公众号、小程序、APP、短信和彩信等方式呈现，从而方便用户查看。
病灶分级识别基于提取的病灶基础特征，通过分类算法来给出良恶性分类或更细致的分级。对于用户的超声筛查结果，系统会按照BI-RADS(Breast imaging reporting and data system，即乳腺影像报告和数据系统)分级归纳为不同等级，从而给用户提供更加规范、易懂的诊断报告。其中，各个分级含义如下：
0级：需要召回，结合其他检查后再评估；
I级：未见异常；
II级：考虑良性改变，建议定期随访(如每年一次)；
III级：良性疾病可能，但需要缩短随访周期(如3～6个月一次)；
IV级：有异常，不能完全排除恶性病变可能，需要活检明确；
IVa级：倾向恶性可能性低；
IVb级：倾向恶性可能性中等；
IVc级：倾向恶性可能性高；
V级：高度怀疑为恶性病变(几乎认定为恶性疾病)，需要手术切除活检；
VI级：已经由病理证实为恶性病变。
在本实施例中，对于获取到的超声图像，可以通过本地的数据处理设备进行分析处理，比如主机，也可以通过有线/无线网络发送到远程数据处理设备进行分析处理，比如远程数据处理设备是服务器或远程诊断终端。
此外，为了实现更好的健康管理，将每一位用户的诊断结果存储到数据库中，并且诊断结果与账户信息关联，根据用户的诊断结果给用户推送后续的筛查安排以及与乳腺相关的医疗资讯。在一示例中，如果用户的筛查结果为BI-RADS 1和BI-RADS 2，可以认为该用户目前乳腺为正常状态，但是并不能保证后续不会有乳腺疾病的发生。在这种情况下，在本次检查11个月后，本系统会给该用户发送提醒信息，提醒用户及时进行第二年度的乳腺癌筛查。
如图12所示，为了保证最后输出的超声图像满足分析诊断的要求，在对获取到的超声图像进行分析处理之前，该乳腺超声筛查方法还包括：
步骤S50，对获取到的超声图像进行有效性分析，并根据有效性分析的结果调整超声探头的扫查姿态。
在该步骤中，主要是对实时获取到的超声图像进行有效性分析，这里的有效性分析是指超声图像是否完整，而超声图像的缺失主要是超声探头未与乳房表面紧密接触导致的，无效图像的体现是出现大面积的黑色像素点，因此利用图像评估反馈策略来调整超声探头的扫查姿态，尽可能地保证超声探头的声波发射面紧贴于乳房表面。在本实施例中，主要通过识别超声图像中 出现无效区域的位置，根据无效区域的位置使超声探头发生偏转，进而实现姿态调整。
如图13所示，上述步骤S50具体包括：
步骤S51，将获取到的超声图像划分成多个子区域，并计算每个子区域的黑色像素点的数量占比。
应当理解的是，黑色像素点的像素值为零，黑色像素点的数量占比为被划分的单个子区域内的黑色像素点的数量与该单个子区域内的所有像素点的数量比。
具体地，超声图像被分割成连续分布的若干矩形区域(子区域)，每个矩形区域内都分布有一定数量的黑色像素点，通过计算各矩形区域内的黑色像素点的数量与所有像素点的数量，可计算出该矩形区域内的黑色像素点的数量占比。
步骤S52，根据黑色像素点的数量占比判断其对应的子区域是否为无效成像区域。
容易理解的是，各子区域中的黑色像素点的数量越多，则越容易在该区域内形成无效成像区域(黑色区域)，当子区域内的黑色像素点的数量占比超出某一阈值后，则可判定该黑色像素点所在子区域为无效成像区域。具体的，本申请所提出的无效成像区域内的黑色像素点的数量占比的阈值为75％～88％，以此作为判断依据，确认个子区域是否为无效成像区域。比如，假设超声图像被划分成连续分布且大小相同的若干矩形区域，在该矩形区域内分布有15960个像素点，若在矩形区域内分布有12000-14000个黑色像素点，则表明该矩形区域为无效成像区域。
步骤S53，以超声图像的居中线为参照划分出左右区域，统计无效成像区域分别在左右区域的数量，并计算左右区域中所有无效成像区域在超声图像中的面积占比。
本实施例中，以超声图像的居中线为参照，将超声图像划分为左区域和右区域，分别统计左区域和右区域内的无效成像区域的数量，并计算各自与超声图像的面积比。可以理解的是，通过将获得的超声图像划分为左区域和右区域，以据此确认各无效成像区域的位置。
步骤S54，根据面积占比，计算超声探头的位姿补偿量，以调整超声探头的扫查姿态。
本实施例中，根据超声图像左右区域内的无效成像区域的面积占比，计算得到超声探头的位姿补偿量，实为对具有多个自由度的机械臂的位姿进行补偿，以通过机械臂对超声探头的位姿进行实时的调整，具体根据计算得到的补偿量，控制超声探头的旋转和/或下压。
需要说明的是，当左区域中无效成像区域的面积占比等于右区域中无效成像区域的面积占比时，表明无需对超声探头进行旋转补偿，仅需对超声探头进行下压补偿即可。具体的，通过将左区域中无效成像区域的面积占比或右区域中无效成像区域的面积占比与预设的下压系数相乘，即可获得超声探头的下压补偿量，通过超声探头的下压补偿，可使得超声探头与人体皮肤贴合，减少超声图像中的无效成像区域，从而提高超声图像的成像质量。
本实施例中，根据计算得到的下压补偿量，控制超声探头下压预设高度以达到预设位置，从而使得超声探头能够与乳房表面贴合，从而获得较为清晰的超声图像，从而提高诊断结果的准确性。
当左区域中无效成像区域的面积占比不等于右区域中无效成像区域的面积占比时，需要将左区域中无效成像区域的面积占比与右区域中无效成像区域的面积占比进行作差，并将获得的差值与预设的旋转系数相乘，从而获得超声探头的旋转补偿量。
本实施例中，根据计算获得的旋转补偿量，控制超声探头旋转一定角度，以使得左区域中无效成像区域的面积占比与右区域中无效成像区域的面积占比相同。需要说明的是，若左区域中的无效成像区域的面积占比大于右区域中无效成像区域的面积占比，则控制超声探头向人体左侧旋转一定角度；若左区域中的无效成像区域的面积占比小于右区域中无效成像区域的面积占比，则控制超声探头向人体右侧旋转一定角度。
当超声探头按照旋转补偿量旋转一定角度后，计算此时左区域或右区域中所有无效成像区域在超声图像中的面积占比，以供后续下压补偿量的计算使用。
最后，将旋转后的左区域中无效成像区域的面积占比或右区域中无效成像区域的面积占比与预设的下压系数相乘，以获得超声探头的下压补偿量，据此控制超声探头的下压。
由此可见，本申请的乳腺超声筛查方法借助自动化技术及人工智能技术，可以使低成本、大范围的群体性乳腺癌筛查成为可能，将大幅度提高我国适龄女性参加乳腺癌筛查的比例，有助于对乳腺癌防控。
此外，本申请还提供一种乳腺超声筛查装置，如图16所示，该乳腺超声筛查装置包括：
图像获取模块100，用于获取用户胸部区域的深度图像；
轨迹生成模块200，用于根据深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据三维结构模型生成超声探头的扫查轨迹；
扫查控制模块300，用于根据扫查轨迹生成运动控制代码，并将运动控制代码输入至扫查机构，以控制扫查机构带动超声探头对用户的乳房区域进行 超声扫查；
诊断模块400，用于对获取到的超声图像进行分析处理，以生成诊断结果。
上述乳腺超声筛查装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中，也可以以软件形式存储于服务器中的存储器中，以便于计算机设备调用执行以上各个模块对应的操作。该计算机设备可以为中央处理单元(CPU)、微计算机设备、单片机等。上述各功能模块所起到的工作原理及起到的作用可参见图11-15中所示的乳腺超声筛查方法的实现过程，在此不作赘述。
本申请还提供一种计算机程序存储介质，该计算机程序存储介质中存储有计算机程序代码，该计算机程序代码被处理器执行时实现如下步骤：
获取用户胸部区域的深度图像；
根据深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据三维结构模型生成超声探头的扫查轨迹；
根据扫查轨迹生成运动控制代码，并将运动控制代码输入至扫查机构，以控制扫查机构带动超声探头对用户的乳房区域进行超声扫查；
对获取到的超声图像进行分析处理，以生成诊断结果。
该计算机程序被处理器执行时还实现了乳腺超声筛查方法的其它步骤，具体可参见上述乳腺超声筛查方法实施例的说明，在此不作赘述。
本申请还提供了一种计算机设备，如图17所示，该计算机设备包括处理器40、存储器50和存储在存储器50中的计算机程序代码，处理器40在调用该计算机程序代码时，实现上述各实施例中提供的一种乳腺超声筛查方法的步骤。
具体地，该计算机设备可为个人计算机或服务器。该计算机设备包括通过系统总线连接的处理器40、存储器50和通信接口(图未示)。其中，处理器40用于提供计算和控制能力，支撑整个计算机设备的运行。存储器50包括非易失性存储介质和内存储器。非易失性存储介质中存储有操作系统和计算机程序，该计算机程序被处理器40执行时以实现一种乳腺超声筛查方法。内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。通信接口用于与外部的服务器或终端通过网络连接通信。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种乳腺超声筛查方法，其特征在于，包括：
    获取用户胸部区域的深度图像；
    根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；
    根据所述扫查轨迹生成运动控制代码，并将所述运动控制代码输入至扫查机构，以控制所述扫查机构带动超声探头对用户的乳房区域进行超声扫查；
    对获取到的超声图像进行分析处理，以生成诊断结果。
  2. 根据权利要求1所述的乳腺超声筛查方法，其特征在于，在所述获取用户胸部区域的深度图像的步骤之前，所述方法还包括：
    录入用户个人信息；
    根据所述用户个人信息生成筛查序列号并加入筛查等候队列。
  3. 根据权利要求1所述的乳腺超声筛查方法，其特征在于，在所述对获取到的超声图像进行分析处理，以生成诊断结果的步骤之前，所述方法还包括：
    对获取到的超声图像进行有效性分析，并根据所述有效性分析的结果调整超声探头的扫查姿态。
  4. 根据权利要求3所述的乳腺超声筛查方法，其特征在于，所述对获取到的超声图像进行有效性分析，并根据所述有效性分析的结果调整超声探头的扫查姿态包括：
    将获取到的超声图像划分成多个子区域，并计算每个子区域的黑色像素点的数量占比；
    根据所述黑色像素点的数量占比判断其对应的子区域是否为无效成像区域；
    以所述超声图像的居中线为参照划分出左右区域，统计所述无效成像区域分别在所述左右区域的数量，并计算所述左右区域中所有无效成像区域在所述超声图像中的面积占比；
    根据所述面积占比，计算超声探头的位姿补偿量，以调整超声探头的扫 查姿态。
  5. 根据权利要求1所述的乳腺超声筛查方法，其特征在于，所述对获取到的超声图像进行分析处理，以生成诊断结果包括：
    将获取到的超声图像输入至AI诊断算法模型中进行分析处理，以得到诊断数据；
    根据BI-RADS分级对所述诊断数据进行分级处理，以生成诊断结果。
  6. 根据权利要求5所述的乳腺超声筛查方法，其特征在于，所述对获取到的超声图像进行分析处理，以生成诊断结果还包括：
    将获取到的超声图像发送至远程诊断终端进行分析处理。
  7. 根据权利要求1所述的乳腺超声筛查方法，其特征在于，所述根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹包括：
    对不同视角下的多幅所述深度图像的点云数据进行坐标变换，以得到位于同一基坐标系下的胸部区域三维点云；
    根据预设的点云分割算法对所述胸部区域三维点云进行分割，以得到乳房扫查区域点云；
    根据所述乳房扫查区域点云对乳房区域结构进行骨架模型重建，以得到曲线骨架；
    按照预设的曲线分割条件对所述曲线骨架中的各条曲线进行分割，并取各条曲线上的所有分割点；
    根据预设的超声扫查方向从分割点集合中选取多组分割点，并将每一组分割点连接成一条扫查轨迹曲线；
    从所述扫查轨迹曲线中提取多个轨迹点，并计算各轨迹点的姿态角。
  8. 一种乳腺超声筛查装置，其特征在于，包括：
    图像获取模块，用于获取用户胸部区域的深度图像；
    轨迹生成模块，用于根据所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；
    扫查控制模块，用于根据所述扫查轨迹生成运动控制代码，并将所述运 动控制代码输入至扫查机构，以控制所述扫查机构带动超声探头对用户的乳房区域进行超声扫查；
    诊断模块，用于对获取到的超声图像进行分析处理，以生成诊断结果。
  9. 一种乳腺超声筛查系统，其特征在于，包括主机、拍摄设备、扫查机构和超声探头，其中：
    所述拍摄设备用于采集用户胸部区域的深度图像；
    所述主机用于对所述深度图像进行模型重建，以得到待扫查区域的三维结构模型，并根据所述三维结构模型生成超声探头的扫查轨迹；所述主机还用于根据所述扫查轨迹生成运动控制代码；
    所述扫查机构用于接收来自所述主机输出的运动控制代码，并根据所述运动控制代码带动所述超声探头对用户的乳房区域进行超声扫查；
    所述主机还用于对获取到的超声图像进行分析处理，以生成诊断结果。
  10. 根据权利要求9所述的乳腺超声筛查系统，其特征在于，还包括用户信息录入装置，所述用户信息录入装置包括信息录入模块和叫号模块，其中：
    所述信息录入模块用于录入用户个人信息；
    所述叫号模块用于根据所述用户个人信息生成筛查序列号并加入筛查等候队列。
PCT/CN2020/121237 2019-10-22 2020-10-15 乳腺超声筛查方法、装置及系统 WO2021078066A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911007859.4A CN110786887B (zh) 2019-10-22 2019-10-22 乳腺超声筛查方法、装置及系统
CN201911007859.4 2019-10-22

Publications (1)

Publication Number Publication Date
WO2021078066A1 true WO2021078066A1 (zh) 2021-04-29

Family

ID=69440928

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/121237 WO2021078066A1 (zh) 2019-10-22 2020-10-15 乳腺超声筛查方法、装置及系统

Country Status (2)

Country Link
CN (1) CN110786887B (zh)
WO (1) WO2021078066A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344028A (zh) * 2021-05-10 2021-09-03 深圳瀚维智能医疗科技有限公司 乳腺超声序列图像分类方法及装置
CN113975661A (zh) * 2021-11-16 2022-01-28 神州医疗科技股份有限公司 一种监控治疗设备的质控方法、装置、系统及存储介质
CN114041828A (zh) * 2022-01-13 2022-02-15 深圳瀚维智能医疗科技有限公司 超声扫查控制方法、机器人及存储介质
CN114224381A (zh) * 2021-12-16 2022-03-25 中国人民解放军联勤保障部队北戴河康复疗养中心 一种超声检查中辅助支撑装置及其使用方法
CN116250863A (zh) * 2023-03-01 2023-06-13 山东大学 多模态多参数乳腺癌筛查干预系统
CN117258168A (zh) * 2023-10-16 2023-12-22 广州驰狐科技有限公司 一种超声美容仪的动态智能控制方法及系统

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110786887B (zh) * 2019-10-22 2021-11-26 深圳瀚维智能医疗科技有限公司 乳腺超声筛查方法、装置及系统
CN111528902A (zh) * 2020-04-16 2020-08-14 深圳瀚维智能医疗科技有限公司 乳腺自动扫查设备
CN111603199B (zh) * 2020-04-24 2023-03-14 中国人民解放军总医院第二医学中心 一种基于体表定位测量仪的三维重建超声诊断系统
CN112085698A (zh) * 2020-07-27 2020-12-15 深圳瀚维智能医疗科技有限公司 左右乳腺超声图像自动分析方法及装置
CN112419415B (zh) * 2020-12-08 2022-06-17 浙江德尚韵兴医疗科技有限公司 一种基于crs曲线拟合实现位姿规划的超声扫查方法
CN112472137B (zh) * 2020-12-15 2022-09-16 深圳市德力凯医疗设备股份有限公司 一种基于触摸屏滑动操作轨迹的超声扫描方法及系统
CN112603342B (zh) * 2020-12-23 2022-12-06 达影医疗(中山)有限公司 压迫装置、乳腺x射线成像机以及乳腺检测成像方法
CN112767237A (zh) * 2020-12-30 2021-05-07 无锡祥生医疗科技股份有限公司 基于点云数据的环形位姿控制方法、装置及超声设备
CN112716522A (zh) * 2020-12-30 2021-04-30 无锡祥生医疗科技股份有限公司 探头末端的轨迹跟踪方法、装置、电子设备及存储介质
EP4309588A1 (en) * 2021-03-17 2024-01-24 FUJIFILM Corporation Ultrasonic diagnosis device and method for controlling ultrasonic diagnosis device
CN113057678A (zh) * 2021-04-09 2021-07-02 哈尔滨理工大学 一种基于双目视觉与机器人的乳腺超声扫描方法及系统
CN113538665B (zh) * 2021-07-21 2024-02-02 无锡艾米特智能医疗科技有限公司 一种器官三维图像重建补偿方法
CN113838210A (zh) * 2021-09-10 2021-12-24 西北工业大学 一种将超声图像转换为3d模型的方法及装置

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080021317A1 (en) * 2006-07-24 2008-01-24 Siemens Medical Solutions Usa, Inc. Ultrasound medical imaging with robotic assistance for volume imaging
CN102743188A (zh) * 2011-04-22 2012-10-24 李百祺 超声波自动扫描系统及其扫描方法
CN103690191A (zh) * 2013-12-03 2014-04-02 华南理工大学 一种超声探头智能连续扫描装置及其扫描方法
CN103750864A (zh) * 2014-01-13 2014-04-30 华南理工大学 一种超声弹性成像的扫描装置及其扫描方法
CN104856720A (zh) * 2015-05-07 2015-08-26 东北电力大学 一种基于rgb-d传感器的机器人辅助超声扫描系统
CN109674494A (zh) * 2019-01-29 2019-04-26 深圳瀚维智能医疗科技有限公司 超声扫查实时控制方法、装置、存储介质及计算机设备
CN209107408U (zh) * 2018-06-25 2019-07-16 深圳瀚维智能医疗科技有限公司 乳房自动扫查装置
CN110664438A (zh) * 2019-10-22 2020-01-10 深圳瀚维智能医疗科技有限公司 超声扫查轨迹规划方法、装置、存储介质及计算机设备
CN110675398A (zh) * 2019-10-22 2020-01-10 深圳瀚维智能医疗科技有限公司 乳腺超声筛查方法、装置及计算机设备
CN110786887A (zh) * 2019-10-22 2020-02-14 深圳瀚维智能医疗科技有限公司 乳腺超声筛查方法、装置及系统

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101097635A (zh) * 2007-08-02 2008-01-02 郑志豪 一种基于身份认证的银行排队方法及管理系统
US20110301461A1 (en) * 2010-06-04 2011-12-08 Doris Nkiruka Anite Self-administered breast ultrasonic imaging systems
EP2790587B1 (en) * 2011-12-18 2018-04-11 Metritrack, Inc. Three dimensional mapping display system for diagnostic ultrasound machines
CN103908258A (zh) * 2013-12-09 2014-07-09 天津天视科技有限公司 一种奶牛乳腺组织体积的测量方法
CN105997149A (zh) * 2016-06-16 2016-10-12 深圳市前海安测信息技术有限公司 乳腺自动筛查系统及方法
CN106923862B (zh) * 2017-03-17 2020-11-27 苏州佳世达电通有限公司 一种超声扫描导引装置及超声扫描导引方法
CN107397557A (zh) * 2017-07-13 2017-11-28 深圳市前海博志信息技术有限公司 乳腺超声波检查报告生成系统及方法
CN108670305B (zh) * 2018-06-25 2024-01-16 深圳瀚维智能医疗科技有限公司 乳房自动扫查装置
CN109637630A (zh) * 2018-11-15 2019-04-16 上海联影医疗科技有限公司 自助式医疗影像系统及控制方法

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080021317A1 (en) * 2006-07-24 2008-01-24 Siemens Medical Solutions Usa, Inc. Ultrasound medical imaging with robotic assistance for volume imaging
CN102743188A (zh) * 2011-04-22 2012-10-24 李百祺 超声波自动扫描系统及其扫描方法
CN103690191A (zh) * 2013-12-03 2014-04-02 华南理工大学 一种超声探头智能连续扫描装置及其扫描方法
CN103750864A (zh) * 2014-01-13 2014-04-30 华南理工大学 一种超声弹性成像的扫描装置及其扫描方法
CN104856720A (zh) * 2015-05-07 2015-08-26 东北电力大学 一种基于rgb-d传感器的机器人辅助超声扫描系统
CN209107408U (zh) * 2018-06-25 2019-07-16 深圳瀚维智能医疗科技有限公司 乳房自动扫查装置
CN109674494A (zh) * 2019-01-29 2019-04-26 深圳瀚维智能医疗科技有限公司 超声扫查实时控制方法、装置、存储介质及计算机设备
CN110664438A (zh) * 2019-10-22 2020-01-10 深圳瀚维智能医疗科技有限公司 超声扫查轨迹规划方法、装置、存储介质及计算机设备
CN110675398A (zh) * 2019-10-22 2020-01-10 深圳瀚维智能医疗科技有限公司 乳腺超声筛查方法、装置及计算机设备
CN110786887A (zh) * 2019-10-22 2020-02-14 深圳瀚维智能医疗科技有限公司 乳腺超声筛查方法、装置及系统

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344028A (zh) * 2021-05-10 2021-09-03 深圳瀚维智能医疗科技有限公司 乳腺超声序列图像分类方法及装置
CN113975661A (zh) * 2021-11-16 2022-01-28 神州医疗科技股份有限公司 一种监控治疗设备的质控方法、装置、系统及存储介质
CN113975661B (zh) * 2021-11-16 2024-03-01 神州医疗科技股份有限公司 一种监控治疗设备的质控方法、装置、系统及存储介质
CN114224381A (zh) * 2021-12-16 2022-03-25 中国人民解放军联勤保障部队北戴河康复疗养中心 一种超声检查中辅助支撑装置及其使用方法
CN114224381B (zh) * 2021-12-16 2024-02-27 中国人民解放军联勤保障部队北戴河康复疗养中心 一种超声检查中辅助支撑装置及其使用方法
CN114041828A (zh) * 2022-01-13 2022-02-15 深圳瀚维智能医疗科技有限公司 超声扫查控制方法、机器人及存储介质
CN116250863A (zh) * 2023-03-01 2023-06-13 山东大学 多模态多参数乳腺癌筛查干预系统
CN116250863B (zh) * 2023-03-01 2023-09-08 山东大学 多模态多参数乳腺癌筛查干预系统
CN117258168A (zh) * 2023-10-16 2023-12-22 广州驰狐科技有限公司 一种超声美容仪的动态智能控制方法及系统
CN117258168B (zh) * 2023-10-16 2024-03-22 广州驰狐科技有限公司 一种超声美容仪的动态智能控制方法及系统

Also Published As

Publication number Publication date
CN110786887B (zh) 2021-11-26
CN110786887A (zh) 2020-02-14

Similar Documents

Publication Publication Date Title
WO2021078066A1 (zh) 乳腺超声筛查方法、装置及系统
CN110675398B (zh) 乳腺超声筛查方法、装置及计算机设备
WO2021078064A1 (zh) 超声扫查轨迹规划方法、装置、存储介质及计算机设备
WO2021078065A1 (zh) 乳房三维点云重建方法、装置、存储介质及计算机设备
CN105027163B (zh) 扫描区域确定装置
CN110969245B (zh) 医学图像的目标检测模型训练方法和装置
CN110766704B (zh) 乳房点云分割方法、装置、存储介质及计算机设备
JP6304970B2 (ja) 画像処理装置、画像処理方法
WO2022011616A1 (zh) 一种通过影像组学特征判断癌症治疗反应的方法及系统
CN111627521B (zh) 增强现实在放疗中的应用
CN108257120B (zh) 一种基于ct图像的三维肝脏包围盒的自动提取方法
CN108564044A (zh) 一种确定肺结节密度的方法及装置
KR20240013724A (ko) 다중 펄스 x-선 소스 이동 단층합성 영상화 시스템을 사용한 인공 지능 훈련
CN101661623B (zh) 基于线性规划的变形体三维跟踪方法
CN103345741A (zh) 一种非刚性多模医学图像精确配准方法
CN105551042B (zh) 一种扫描床标记点位置的确定方法和装置
CN111402231B (zh) 一种用于肺部ct影像质量的自动评估系统及方法
BR112021013021A2 (pt) Dispositivo portátil para aquisição de dados antropométricos, e método de coleta de dados antropométricos
CN111449684B (zh) 心脏超声标准扫查切面快速获取方法及系统
CN112450956A (zh) 自动摆位方法、装置、可读存储介质、电子设备及系统
CN116580819A (zh) 用于自动化地确定图像序列中的检查结果的方法和系统
CN112515705A (zh) 用于投影轮廓启用的计算机辅助检测(cad)的方法和系统
CN112085698A (zh) 左右乳腺超声图像自动分析方法及装置
King et al. Automatic cephalometric landmark detection on X-ray images using object detection
Yan et al. Automatic detection and localization of pulmonary nodules in ct images based on yolov5

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879486

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20879486

Country of ref document: EP

Kind code of ref document: A1