WO2024080291A1 - 医用ナビゲーション方法、医用ナビゲーションシステム、およびコンピュータプログラム - Google Patents

医用ナビゲーション方法、医用ナビゲーションシステム、およびコンピュータプログラム Download PDF

Info

Publication number
WO2024080291A1
WO2024080291A1 PCT/JP2023/036831 JP2023036831W WO2024080291A1 WO 2024080291 A1 WO2024080291 A1 WO 2024080291A1 JP 2023036831 W JP2023036831 W JP 2023036831W WO 2024080291 A1 WO2024080291 A1 WO 2024080291A1
Authority
WO
WIPO (PCT)
Prior art keywords
model data
subject
reference model
data
medical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/036831
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
遼介 津村
潔 葭仲
義彦 小関
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National Institute of Advanced Industrial Science and Technology AIST
Original Assignee
National Institute of Advanced Industrial Science and Technology AIST
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National Institute of Advanced Industrial Science and Technology AIST filed Critical National Institute of Advanced Industrial Science and Technology AIST
Priority to JP2024551709A priority Critical patent/JPWO2024080291A1/ja
Publication of WO2024080291A1 publication Critical patent/WO2024080291A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/13—Tomography
    • A61B8/14—Echo-tomography

Definitions

  • the present invention relates to a medical navigation method, a medical navigation system, and a computer program executed in a medical robot device.
  • a doctor or other clinical expert remotely operates a medical instrument such as an ultrasound probe or stethoscope via a medical robot or the like, it is necessary to guide the medical robot or the like as to where on the body surface the medical instrument should be placed in order to perform the examination efficiently. Furthermore, when a doctor provides remote medical care, it may be necessary for a person without medical knowledge, such as the patient or a close relative of the patient, to place the medical instrument such as a stethoscope in an appropriate position on the patient's body surface instead of the doctor. On the other hand, the position where the medical instrument should be placed, i.e. the position of the patient's organs to be examined, varies depending on the individual differences in the patient's body shape, so it is important to estimate the position taking individual differences into account.
  • Patent Documents 1 and 2 and Non-Patent Document 1 disclose a method for mapping functional areas of the brain by applying a standard model of the human brain to an image of the patient's brain, thereby performing alignment and deformation (registration).
  • the present invention aims to provide a medical navigation method, medical navigation system, and computer program that can guide medical instruments such as ultrasound probes and stethoscopes to appropriate positions on the body surface in accordance with individual differences in body shape using an easily usable RGB-D (image/depth) camera.
  • medical instruments such as ultrasound probes and stethoscopes
  • one aspect of the present invention is a medical navigation method executed by a computer system for positioning a medical instrument on a subject, the computer system retaining three-dimensional point cloud information of the subject as body model data of the subject, and retaining N pieces of reference model data indicating the shape of the human body and placement data indicating the placement position of the medical instrument in each piece of reference model data, the medical navigation method including the steps of: determining, as reference model data, the reference model data that is most similar to the body model data of the subject among the N pieces of reference model data; applying registration to the reference model data using a non-rigid ICP (interactive closest point) algorithm with the body model data of the subject as a target; and converting the placement data in the reference model data into placement data of the medical instrument in the body model data of the subject using a coordinate conversion formula for the three-dimensional point cloud information between the body model data of the subject obtained by the registration and the reference model data.
  • ICP interactive closest point
  • Another aspect of the present invention is a medical navigation system for positioning a medical instrument on a subject, which holds three-dimensional point cloud information of the subject as body model data of the subject, holds N pieces of reference model data representing the shape of the human body and position data representing the position of the medical instrument in each piece of reference model data, determines the reference model data of the N pieces of reference model data that is most similar to the body model data of the subject as reference model data, applies registration using a non-rigid ICP (interactive closest point) algorithm to the reference model data with the body model data of the subject as a target, and converts the position data in the reference model data into position data of the medical instrument in the body model data of the subject using a coordinate conversion formula for the three-dimensional point cloud information between the body model data of the subject obtained by the registration and the reference model data.
  • ICP interactive closest point
  • Another aspect of the present invention is a computer program causing a computer system to execute the above-described medical navigation method.
  • Another aspect of the present invention is a computer-readable recording medium storing a computer program for causing a computer system to execute the above-described medical navigation method.
  • FIG. 1 is a diagram showing an example of the configuration of a medical robot device (medical navigation system) according to an embodiment of the present invention.
  • 1 is a diagram showing an overview of a medical navigation method according to an embodiment of the present invention.
  • FIG. 1 is a diagram showing a human body model created in a simulation.
  • FIG. 13 shows the overall results of the registration error as a function of chamfer distance.
  • FIG. 2 is a diagram showing an example of a processing flow of a medical navigation method according to an embodiment of the present invention.
  • the medical navigation method and medical navigation system are a method and system for appropriately estimating the position of a medical instrument, such as an ultrasonic probe or a stethoscope, on the body surface of a subject when a medical procedure is performed on the subject, such as a patient, using a medical robot device.
  • a medical instrument such as an ultrasonic probe or a stethoscope
  • auscultation is performed on a specific part of the patient's body (subject).
  • the medical robot device 1 is a diagram showing an example of the configuration of a medical robot device (medical navigation system) that executes the medical navigation method according to the present embodiment.
  • the medical robot device 1 includes a robot arm 10, a constant load passive scanning mechanism (end effector) 20, and an RGB-D camera 30.
  • the robot arm 10 is configured to be movable around a patient (subject) 5 on which a medical procedure is to be performed (FIG. 1 is a diagram for explanation, and the patient 5 (specifically, the body of the patient 5. In the following, it may also be referred to as the "patient's body 5" is substituted here with a mannequin).
  • the tip of the robot arm 10 is provided with a constant load passive scanning mechanism (end effector) 20, either integrally or detachably.
  • the constant load passive scanning mechanism 20 is a mechanism that holds a medical instrument 40 and enables the medical instrument 40 to be pressed against the body surface of the patient 5 while maintaining a constant load and moved.
  • the medical instrument may be any instrument necessary for a medical procedure to be performed on the patient 5, such as an ultrasound probe or a stethoscope.
  • an example of a medical instrument that can be used in the medical robot device 1 of this embodiment is an electronic stethoscope, such as the electronic stethoscope (digital stethoscope) JPES-01 manufactured by Mitrica Co., Ltd.
  • Sound collected by the electronic stethoscope coming into contact with the patient 5 can be transmitted as data to a computer device (such as a client computer device 60 described later).
  • a computer device such as a client computer device 60 described later.
  • other medical instruments may be configured so that sound and video data acquired by the medical instrument coming into contact with the patient 5 can be transmitted to a computer device.
  • the robot arm 10 can be provided with an RGB-D camera 30 at its tip, either integrally or detachably.
  • the RGB-D camera 30 is a camera that can simultaneously acquire a color image (RGB information) and a distance image (distance information).
  • the medical robot device 1 may also include a client computer device 60.
  • the client computer device 60 is directly or indirectly connected to each of the robot arm 10, constant load passive scanning mechanism (end effector) 20, and RGB-D camera 30 of the medical robot device 1, and transmits and receives data to and from these components.
  • the client computer device 60 transmits control signals for controlling each component and transmits data necessary to operate each component.
  • the client computer device 60 also receives data acquired or generated in each component from each component.
  • the client computer device 60 can be realized by a hardware configuration similar to that of a general computer device.
  • the client computer device 60 may include, for example, a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), an internal hard disk device, an external hard disk device, a removable memory such as a CD, a DVD, a USB memory, a memory stick, or an SD card, an input/output user interface (display, keyboard, mouse, touch panel, speaker, microphone, LED, etc.), and a wired/wireless communication interface capable of communicating with each configuration of the medical robot device 1 and other computer devices.
  • a processor for example, a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), an internal hard disk device, an external hard disk device, a removable memory such as a CD, a DVD, a USB memory, a memory stick, or an SD card, an input/output user interface (display, keyboard, mouse, touch panel, speaker, microphone, LED, etc.), and a wired/wireless communication interface capable of communicating with each configuration of the medical robot device 1 and other computer devices
  • the client computer device 60 may, for example, have a processor that reads into a memory such as a RAM a computer program for causing the client computer device 60 to execute the medical navigation method according to this embodiment, the computer program having been stored in advance in a hard disk device, a ROM, or a removable memory, and executes the necessary data while reading it appropriately from the hard disk device, the ROM, the removable memory, etc.
  • a processor that reads into a memory such as a RAM a computer program for causing the client computer device 60 to execute the medical navigation method according to this embodiment, the computer program having been stored in advance in a hard disk device, a ROM, or a removable memory, and executes the necessary data while reading it appropriately from the hard disk device, the ROM, the removable memory, etc.
  • Such operations of the client computer device 60 realize various processes in the medical robot device 1 described in this embodiment.
  • various data used in each process described in this embodiment is stored in a storage device or storage medium such as a hard disk drive, RAM, or removable memory, and
  • the medical robot device 1 shown in FIG. 1 is a medical robot device actually constructed by the inventors of the present application.
  • a six-axis collaborative 6-DOF cooperative robot arm (UR5e, Universal Robot, Denmark) was used as the robot arm 10.
  • a LiDAR camera (L515 RealSense, Intel, USA) was used as the RGB-D camera 30 to obtain the three-dimensional contour of the body surface of the patient 5 as point cloud data.
  • a well-known computer device (Dell Precision 5380, Dell, USA) was used as the client computer device 60.
  • the medical robot device 1 in this example was constructed assuming that auscultation would be performed using a stethoscope, as an example. Note that, since FIG.
  • FIG. 1 is a medical robot device constructed experimentally, the patient 5 is substituted by a mannequin here, and since the nipples and navel, which are landmarks of the patient 5 in this example, are unclear on the mannequin as described below, the inventors attached markers to these positions. Furthermore, the configuration shown in FIG. 1 is merely an example, and it goes without saying that the configuration of the medical robot device 1 according to this embodiment is not limited to this. The same applies throughout this specification.
  • a constant-load passive scanning mechanism using a spring is implemented in the end effector 20 of the robot arm 10 in order to adaptively grasp the stethoscope (medical instrument) 40 relative to the body surface of the patient 5.
  • a six-axis force/torque sensor 25 (Axia-80-M20, ATI Industrial Automation, USA) is attached to the base of the constant-load passive scanning mechanism (end effector) 20, and is configured to measure the contact force when placing the stethoscope 40 on the body surface of the patient 5.
  • the client computer device 60 that controls the medical robot device 1 and an external server computer device are directly connected to a network (speed 1 GB/sec (gigabytes per second)), and the data transmission protocol is TCP/IP.
  • the RGB-D camera 30 is provided in the constant-load passive scanning mechanism (end effector) 20, but is not limited to this.
  • the RGB-D camera 30 is fixed so that its relative position with respect to the robot arm 10 does not change, and may be installed in a location different from the constant load passive scanning mechanism (end effector) 20.
  • the RGB-D camera 30 acquires depth and color information of the shape of the patient 5 as point cloud data, and the point cloud data is used for coordinate registration (alignment) using the positional relationship between the robot arm 10 and the patient 5. Because the RGB-D camera 30 is attached to the end effector 20 of the robot arm 10, the positional relationship between the RGB-D camera 30 and the robot arm 10 remains kinematically fixed even if the robot arm 10 moves around the patient 5. Therefore, the position of the acquired point cloud data of the patient 5 is linked to the coordinates of the robot arm 10.
  • the robot arm 10 can be controlled by URScript (Universal Robots A/S), a programming language used in the medical robot device 1.
  • URScript Universal Robots A/S
  • the client computer device 60 that controls the medical robot device 1 can send URScript commands to an external server computer device (not shown) via socket communication.
  • the point cloud data was acquired using Intel RealSense SDK 2.0.
  • a software system customized based on Python programming in Visual Studio Code can synchronize the control of the robot arm 10 with the reading of the point cloud data.
  • the medical navigation method in order to localize the auscultation area of the stethoscope (medical instrument) while compensating for individual differences in the patient's body shape, body surface registration between the patient's body 5 and a reference model of the body registered in advance in the medical navigation system is used.
  • the "reference model” is a model of the human body that serves as a reference for body surface registration.
  • the auscultation position (including the case of a region (auscultation area) with a certain degree of spread, the same applies below) where the stethoscope should be placed is determined in advance based on general medical knowledge of clinical experts, etc., and placement data indicating the auscultation position is stored in the medical navigation system.
  • a correspondence relationship between the surface of the patient's body 5 and the reference model is calculated. Based on the correspondence relationship calculated by the body surface registration, the auscultation area identified in the reference model can be projected onto the patient's body.
  • one of the commonly known registration methods is the ICP (interactive closest point) algorithm, which allows applying the input point cloud data of the body surface using an affine transformation to fit the reference model to the patient's body surface, so that the point cloud is considered to be rigid.
  • ICP interactive closest point
  • the rigid registration by ICP cannot find an exact correspondence between the surface of the patient's body 5 and the reference model, and the input point cloud data needs to be deformed to fit the body surface of the target (patient's body 5).
  • a non-rigid ICP is used as an improved ICP algorithm, which can fit point cloud data to the patient's body 5 non-rigidly using feature points and deformation constraints.
  • the auscultation position of the patient's body 5 is identified by applying body surface registration between the patient 5 and a reference model. More specifically, a number of human body models (reference models) that can serve as several types of reference models are prepared in advance, and a reference model that is close to the patient's body is selected, thereby making it possible to improve the accuracy of localizing the auscultation area.
  • the auscultation position where the stethoscope should be placed (including cases where the area is a certain extent (auscultation area); the same applies below) is determined in advance based on general medical knowledge of clinical experts, etc., and placement data indicating the auscultation position is stored in the medical navigation system.
  • valves aortic valve, pulmonary valve, tricuspid valve, and mitral valve
  • the tricuspid valve, mitral valve, pulmonary valve, and aortic valve are generally located on the left side of the lower sternum close to the fifth intercostal space, at the left fifth intercostal cusp on the midclavicular line (approximately 10 cm from the midline), on the inner edge of the left second intercostal space, and in the right third intercostal space, respectively.
  • the medical navigation method and medical navigation system estimates the positions corresponding to each valve on the patient's body surface (the positions where the stethoscope should be placed) from the reference model by applying non-rigid registration between the patient's body 5 and a reference model of the human body (registered in advance in the medical navigation system).
  • FIG. 2 is a diagram showing an example of an outline of the medical navigation method according to this embodiment.
  • the medical navigation method according to this embodiment includes two processing steps, step (A) and step (B).
  • step (A) a reference model that is most similar to the patient's body data (patient body model) 50 obtained by photographing the patient's body 5 with the RGB-D camera 30 is selected as the reference model 52 from among a plurality of reference models 52-1, 52-2, ..., 52-N (N is an integer of 2 or more) that are the basis of the reference model and are registered in advance in the medical navigation system.
  • N is an integer of 2 or more
  • step (B) body surface registration by non-rigid ICP is performed on the reference model 52 selected in step (A), and the reference model 52 is deformed so as to fit the patient's body 5 (patient body model 50).
  • the position (reference position) 55 of each valve is projected from the reference model 52 to the patient's body 5 (patient body model 50), and the position (estimated position) 56 of each valve on the body surface of the patient's body 5 is estimated.
  • Step (A) Selection of the reference model most similar to the patient's body
  • the point cloud data of the patient's body model 50 and the point cloud data of each reference model 52-k are superimposed, and the degree of superimposition is calculated to measure the similarity between the patient's body model 50 and each reference model 52-k.
  • the Chamfer Distance is widely adopted to measure the similarity between two point sets, and is defined as follows:
  • S1 and S2 are subsets of point cloud data.
  • x and y are point data contained in S1 and S2 .
  • dCD represents the chamfer distance between S1 and S2 .
  • dCD can be calculated by summing the squares of the distances between the nearest neighbors of the two point clouds. When the shape deviation between the two point clouds is large, the distance between the nearest neighbors of the two point clouds is large, and thus the chamfer distance is large.
  • the reference model 52-k reference model 52 that is most similar to the patient body model 50 is identified.
  • Non-rigid ICP is used to deform the reference model 52 identified in step (A) to the patient body model 50.
  • non-rigid ICP is applied to the mesh data converted from the point cloud data.
  • the source mesh converted from the point cloud data of the reference model 52 is given as a set V of n vertices (n is a natural number equal to or greater than 3) and a set ⁇ of m edges (m is a natural number equal to or greater than 3).
  • the registration involves finding the detection parameters X, which represent the set V(X) of displaced source vertices, which is the deformation mesh relative to the body surface of the patient 5.
  • the objective function E was defined as follows:
  • E d , E s , and E l denote the distance objective function, stiffness objective function, and sensitivity mark distance objective function, respectively.
  • ⁇ and ⁇ denote stiffness and landmark parameters.
  • the reliability of the match is weighted by W i . If there is no corresponding vertex, the weight is set to zero. If a corresponding vertex is found, the weight is set to one.
  • a rigidity objective function is used to regularize the deformation by penalizing the weighted difference in the transformations of adjacent vertices.
  • the rigidity objective function is the Frobenius norm
  • G was used to weight the difference between the rotational and distortion parts of the deformation against the translational part of the deformation.
  • a landmark distance objective function was used for registration initialization and guidance, and was defined as follows:
  • the total objective function E can be solved by the method disclosed in Amberg B, Romdhani S, Vetter T (2007) Optimal Step Nonrigid ICP Algorithms for Surface Registration. 2007 IEEE Conference on Computer Vision and Pattern Recognition 1-8.
  • the inventors of the medical navigation method and medical navigation system according to the present embodiment conducted a simulation to verify that the accuracy of registration is improved by selecting a reference model that is close to the patient's body 5 (patient body model 50).
  • patient body model 50 a reference model that is close to the patient's body 5
  • the inventors created several types of human body models using the digital human platform software "DhaibaWorks" (National Institute of Advanced Industrial Science and Technology, https://www.dhaibaworks.com/). This software makes it possible to generate human body models of various shapes as mesh data. As shown in FIG.
  • the inventors used this software to create nine types of human body models in advance, ranging from short models (#1, #2) to tall models (#3, #4) and from light models (#5, #6) to heavy models (#7, #8), compared to a model (#0) with standard height and weight.
  • the reference model 52 (source mesh) was fixed to the standard model (#0 in FIG. 3), and the patient body model 50 (target mesh) was set to each of the other models (#1 to #8 in FIG. 3). Then, non-rigid ICP was applied to each pair of the source mesh and each target mesh. The registration accuracy was evaluated by the error between the position of each valve projected in the registration and the actual position of each valve determined by the medical knowledge of a clinical expert. Furthermore, the chamfer distance was calculated for each pair of the source mesh and each target mesh.
  • Table 1 shows the results of the non-rigid ICP registration error and the chamfer distance for each valve position in the simulated human body model.
  • Auscultation areas I to IV in Table 1 refer to the auscultation areas for the aortic valve, pulmonary valve, tricuspid valve, and mitral valve, respectively.
  • the results suggest that the larger the deviation in body shape between the target model (#1 to #8 in Figure 3) and the standard model (#0 in Figure 3), the larger the registration error.
  • the chamfer distance also indicates the degree of shape deviation, which corresponds to the result of the registration error.
  • the simulation results show that selecting a reference model 52 that is close to the patient's body 5 improves the accuracy of positioning.
  • the inventors of the medical navigation method and medical navigation system according to this embodiment conducted an experiment on the medical navigation method according to this embodiment using an actual human body.
  • eight body surface data sets were obtained from eight healthy male volunteers.
  • the male volunteers were selected taking into consideration the diversity of body shapes.
  • the average height, weight, and body mass index (BMI) of the volunteers were 1.73 ⁇ 0.06 m, 69.1 ⁇ 5.87 kg, and 23.2 ⁇ 1.98 kg/ m2 , respectively.
  • Table 2 shows the detailed information of the recruited volunteers.
  • Table 3 shows the results of the non-rigid ICP registration performed between the physical data of each volunteer.
  • Auscultation areas I to IV in Table 3 refer to the auscultation areas of the aortic valve, pulmonary valve, tricuspid valve, and mitral valve, respectively.
  • the minimum and maximum values of the chamfer distance and the average registration error calculated for each source model are shown in bold (shaded area). According to the results, it can be seen that in approximately 7/8 of the source model conditions, the minimum chamfer distance corresponds to the minimum registration error, and the maximum chamfer distance corresponds to the maximum registration error.
  • Figure 4 shows the overall results of registration error as a function of chamfer distance.
  • a linear regression analysis was performed to examine the strength of the association between the accuracy of the non-rigid ICP registration and the similarity of the compared models.
  • the linear coefficient of determination R2 was approximately 0.66, indicating that there is some relationship between the chamfer distance and the registration error.
  • the non-rigid ICP registration according to this example is capable of estimating the auscultation area with an average error of 5 to 19 mm, and is capable of providing a more accurate auscultation area that takes into account individual differences in body shape. An error of less than 20 mm is considered to be in a range that does not qualitatively affect auscultation.
  • the registration accuracy may depend on the similarity between the body surface of the patient's body 5 (patient body model 50) and the body surface of the reference model 52.
  • Statistical results showed that there is a correlation between the registration accuracy and the similarity of the models used and the equivalent chamfer distance.
  • the nipple and umbilicus were used as landmarks in the non-rigid ICP registration process.
  • other easily identifiable points on the body surface such as rib boundaries, can be used as landmarks.
  • the registration accuracy can be further improved.
  • the number of reference models (data) is seven, but this is not limiting.
  • body types can be classified into six types according to the obesity assessment criteria (for both men and women) set by the Japan Society for the Study of Obesity, as shown in the table below.
  • the number N of reference models may be an integer of 6 or more, and more preferably an integer of 12 or more (6 for men and 6 for women). Note that the above-mentioned criteria of the Japan Society for the Study of Obesity are only an example, and the number of reference models may be determined based on similar medical grounds, etc.
  • Fig. 5 is a diagram showing an example of a processing flow of the medical navigation method according to this embodiment. The following steps S102 to S106 correspond to the processing of step (A) (selection of a reference model most similar to the patient's body) described in Fig. 2, and steps S108 to S110 correspond to the processing of step (B) (non-rigid ICP for body surface registration) described in Fig. 2.
  • the RGB-D camera 30 acquires three-dimensional point cloud information of the patient's body 5.
  • the method of acquiring the three-dimensional point cloud information of the patient's body 5 is, for example, as follows. That is, the RGB-D camera 30 moves to a predetermined initial position.
  • the RGB-D camera 30 acquires three-dimensional point cloud information (acquired three-dimensional point cloud information) at each imaging location while moving within the imaging area (the entire chest of the subject 50 in this embodiment). At this time, the RGB-D camera 30 also acquires two-dimensional color image information at each imaging location and position and angle information (camera position coordinate information) of the RGB-D camera 30 at each imaging position. These processes are repeated until the entire imaging area has been moved.
  • the body surface shape of the subject 50 is reconstructed using the multiple pieces of acquired three-dimensional point cloud information and the multiple camera position coordinate information acquired at the multiple imaging positions.
  • the acquired multiple pieces of acquired 3D point cloud information are converted from local coordinates to global coordinates based on the position and angle information (position coordinate information) of the RGB-D camera 30 at each imaging position.
  • Correspondence between overlapping points in each piece of acquired 3D point cloud information is searched for.
  • the acquired multiple pieces of acquired 3D point cloud information are integrated into one piece of 3D point cloud information based on the correspondence between the searched overlapping points.
  • the 3D point cloud information acquired in this way is stored in a storage area such as a hard disk drive of the client computer device 60 as data for the patient body model 50.
  • step S104 the client computer device 60 calculates the chamfer distance between the N-body reference model 52-k, which has been stored in advance on a hard disk drive or the like of the client computer device 60, and the body model of the patient 5 acquired in step S102.
  • step S106 the client computer device 60 determines the reference model with the smallest chamfer distance as the base model 52.
  • step S108 the client computer device 60 applies registration using a non-rigid ICP to the reference model 52 with the patient body model 50 as a target.
  • step S110 the client computer device 60 converts the reference position 55, which is the position (position where the stethoscope should be placed) of the diagnostic part (aortic valve, pulmonary valve, tricuspid valve, mitral valve) previously stored in the medical navigation system, into a position on the patient body model 50 using a coordinate conversion formula of the three-dimensional point cloud between the patient body model 50 obtained by registration and the reference model 52, and determines this as the estimated position 55 of the auscultation position.
  • the determined estimated position is displayed and output on a remote computer device operated by a doctor, for example, connected to the client computer device 60 via a network.
  • the doctor can then operate the medical robot device 1 (robot arm 10) while visually checking the displayed estimated position.
  • the estimated position determined in step S110 is displayed and output on the display of the client computer device 60, and the patient himself or a close relative of the patient can place the stethoscope at the displayed estimated position.
  • the medical robot 1 can automatically place the stethoscope 40 at the estimated position determined in step S110 to obtain auscultation data of the patient 5 and provide the data to a clinical expert such as a doctor.
  • a stethoscope is used as a medical instrument, but the present invention is not limited to this.
  • the present invention can be used in various situations, such as robotic diagnosis and treatment navigation, which requires the placement of other medical equipment on the patient's body 5, or preoperative planning.
  • robotic diagnosis and treatment navigation which requires the placement of other medical equipment on the patient's body 5, or preoperative planning.
  • the medical navigation method and medical navigation system according to the present embodiment can be used to estimate the scanning path or scanning area on the body surface of the patient 5 that the ultrasound probe should scan, taking into account individual differences in body shape.
  • the nipple and navel are used as landmarks on the body surface of the subject 5, but it is possible to use any externally noticeable part of the human body as a landmark.
  • the shoulder, clavicle, or pelvis could be used as a landmark.
  • a medical navigation method executed by a computer system for positioning a medical instrument in a subject comprising: the computer system holds three-dimensional point cloud information of the subject as body model data of the subject, holds N pieces of reference model data indicating a shape of a human body, and position data indicating a position of the medical instrument in each reference model data;
  • the medical navigation method includes: determining, from among the N pieces of reference model data, the reference model data having the highest similarity to the subject's body model data as a reference model data; applying a registration to the reference model data using a non-rigid interactive closest point (ICP) algorithm with the subject's body model data as a target; converting the arrangement data in the reference model data into arrangement data of the medical instrument in the body model data of the subject, using a coordinate conversion equation of the three-dimensional point cloud information between the body model data of the subject obtained by the registration and the reference model data;
  • a medical navigation method comprising: (2) The medical navigation method of (1) above, wherein
  • a medical navigation system for positioning a medical instrument on a subject comprising: storing three-dimensional point cloud information of the subject as body model data of the subject, and storing N pieces of reference model data indicating a shape of a human body and arrangement data indicating an arrangement position of the medical instrument in each of the reference model data; Among the N pieces of reference model data, the reference model data having the highest similarity to the subject's body model data is determined as a reference model data; Applying registration to the reference model data using a non-rigid ICP (interactive closest point) algorithm with the subject's body model data as a target; converting the placement data in the reference model data into placement data of the medical instrument in the body model data of the subject, using a coordinate conversion formula of the three-dimensional point cloud information between the body model data of the subject obtained by the registration and the reference model data.
  • ICP interactive closest point
  • the medical navigation system includes an RGB-D camera; The medical navigation system according to (5) above, wherein the RGB-D camera captures images of the subject from a plurality of imaging positions to obtain the three-dimensional point cloud information of the subject.
  • a computer-readable recording medium storing a computer program for causing a computer system to execute any one of the medical navigation methods described above in (1) to (4).
  • Medical robot device (medical navigation system) 5...Patient (subject) 10...Robot arm 20...Constant load passive scanning mechanism (end effector) 25... Force/torque sensor 30... LiDAR camera 40... Medical instrument (stethoscope) 50: Patient body model 55: Reference position 56: Estimated position 60: Client computer device

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Robotics (AREA)
  • Image Processing (AREA)
PCT/JP2023/036831 2022-10-14 2023-10-11 医用ナビゲーション方法、医用ナビゲーションシステム、およびコンピュータプログラム Ceased WO2024080291A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024551709A JPWO2024080291A1 (enExample) 2022-10-14 2023-10-11

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022165133 2022-10-14
JP2022-165133 2022-10-14

Publications (1)

Publication Number Publication Date
WO2024080291A1 true WO2024080291A1 (ja) 2024-04-18

Family

ID=90669554

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/036831 Ceased WO2024080291A1 (ja) 2022-10-14 2023-10-11 医用ナビゲーション方法、医用ナビゲーションシステム、およびコンピュータプログラム

Country Status (2)

Country Link
JP (1) JPWO2024080291A1 (enExample)
WO (1) WO2024080291A1 (enExample)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113397578A (zh) * 2020-07-27 2021-09-17 上海联影医疗科技股份有限公司 一种成像系统和方法
JP2021166593A (ja) * 2020-04-09 2021-10-21 株式会社メディカロイド ロボット手術支援システム、ロボット手術支援方法、及びプログラム

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021166593A (ja) * 2020-04-09 2021-10-21 株式会社メディカロイド ロボット手術支援システム、ロボット手術支援方法、及びプログラム
CN113397578A (zh) * 2020-07-27 2021-09-17 上海联影医疗科技股份有限公司 一种成像系统和方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
RYOSUKE TSUMURA; YOSHIHIKO KOSEKI; NAOTAKA NITTA; KIYOSHI YOSHINAKA: "Toward Fully Automated Robotic Platform for Remote Auscultation", ARXIV.ORG, 14 January 2022 (2022-01-14), XP091138628 *
YIFAN ZHU: "Automated Heart and Lung Auscultation in Robotic Physical Examinations", IEEE ROBOTICS AND AUTOMATION LETTERS, IEEE, vol. 7, no. 2, 1 April 2022 (2022-04-01), pages 4204 - 4211, XP093159410, ISSN: 2377-3774, DOI: 10.1109/LRA.2022.3149576 *

Also Published As

Publication number Publication date
JPWO2024080291A1 (enExample) 2024-04-18

Similar Documents

Publication Publication Date Title
Huang et al. Robotic arm based automatic ultrasound scanning for three-dimensional imaging
Suligoj et al. RobUSt–an autonomous robotic ultrasound system for medical imaging
CN109077752B (zh) 移动机器臂
Hennersperger et al. Towards MRI-based autonomous robotic US acquisitions: a first feasibility study
US10507002B2 (en) X-ray system and method for standing subject
JP5866136B2 (ja) イメージデータ又はモデルにカテーテル案内システムの局所的非剛体位置合わせのためのシステム及び方法
Kim et al. Ultrasound probe and needle-guide calibration for robotic ultrasound scanning and needle targeting
Ma et al. A-see: Active-sensing end-effector enabled probe self-normal-positioning for robotic ultrasound imaging applications
CN110325141A (zh) 影像整合装置及影像整合方法
JP2003265408A (ja) 内視鏡誘導装置および方法
CN101262830A (zh) 用于把对象的虚拟模型映射到对象的方法和系统
Wang et al. Robotic ultrasound: View planning, tracking, and automatic acquisition of transesophageal echocardiography
JP5995408B2 (ja) 情報処理装置、撮影システム、情報処理方法および情報処理をコンピュータに実行させるためのプログラム
US20250064534A1 (en) Medical support method, medical support robot device, and computer program
US20210077069A1 (en) Methods and systems for a medical imaging device
US9633433B1 (en) Scanning system and display for aligning 3D images with each other and/or for detecting and quantifying similarities or differences between scanned images
Tsumura et al. Towards fully automated robotic platform for remote auscultation
Ferguson et al. Toward image-guided partial nephrectomy with the da Vinci robot: exploring surface acquisition methods for intraoperative re-registration
Lathrop et al. Minimally invasive holographic surface scanning for soft-tissue image registration
Zhang et al. Visual perception and convolutional neural network-based robotic autonomous lung ultrasound scanning localization system
KR20160057024A (ko) 마커리스 ３차원 객체추적 장치 및 그 방법
WO2024080291A1 (ja) 医用ナビゲーション方法、医用ナビゲーションシステム、およびコンピュータプログラム
CN114930390B (zh) 用于将活体医学图像与解剖模型配准的方法和装置
JP7495216B2 (ja) 鏡視下手術支援装置、鏡視下手術支援方法、及びプログラム
Zhetpissov et al. A-SEE2. 0: Active-Sensing End-Effector for Robotic Ultrasound Systems with Dense Contact Surface Perception Enabled Probe Orientation Adjustment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23877294

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024551709

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23877294

Country of ref document: EP

Kind code of ref document: A1