CN116747015A

CN116747015A - Registration method and system for lung trachea and electronic equipment

Info

Publication number: CN116747015A
Application number: CN202310497824.3A
Authority: CN
Inventors: 佘文波; 吴晓军
Original assignee: Changzhou Lunghealth Medtech Co ltd
Current assignee: Changzhou Lunghealth Medtech Co ltd
Priority date: 2023-05-05
Filing date: 2023-05-05
Publication date: 2023-09-15

Abstract

The embodiment of the application provides a registration method, a registration system and electronic equipment for lung trachea. The method comprises the following steps: determining a moving path of the positioning sensor in a three-dimensional model of a tracheal tree according to the acquired first moving path point set of the positioning sensor in the lung trachea, wherein the three-dimensional model of the tracheal tree is obtained based on medical image reconstruction of the lung trachea; determining an airway path of the three-dimensional model of the tracheal tree according to the obtained central line information of the three-dimensional model of the tracheal tree; extracting a first key point set of the moving path, wherein the first key point set comprises a moving path bifurcation point and a moving path end point of the moving path; determining a second set of key points corresponding to the first set of key points in the airway path; and registering the three-dimensional model of the tracheal tree with the pulmonary trachea according to the first key point set and the second key point set. The scheme can effectively improve the registration precision between the three-dimensional model of the tracheal tree and the actual pulmonary trachea.

Description

Registration method and system for lung trachea and electronic equipment

Technical Field

The application relates to the technical field of medical instruments, in particular to a method and a system for registering lung trachea and electronic equipment.

Background

With the development of computer technology and medical imaging technology, the surgical navigation technology has the advantages of accuracy, flexibility, minimally invasive and the like, and is widely applied to disease diagnosis and treatment. For example, electromagnetic navigation bronchoscopy techniques (Electromagnetic Navigation Bronchoscope, ENB) are commonly used to examine peripheral diseases of the patient's lungs. In the process of diagnosing and treating the lung of a patient by using ENB (advanced open end system), one indispensable step is registration, and the purpose of the registration is to match the actual physical space of the lung and the trachea tree three-dimensional model reconstructed for the lung of the patient before operation, so as to realize the accurate positioning of the relative positions of surgical instruments and the like in the trachea tree three-dimensional model when the patient enters into real-time surgical navigation subsequently, thereby providing precondition guarantee for surgery.

The three-dimensional model of the tracheal tree is obtained by three-dimensional reconstruction of medical images acquired by the lung of a patient before operation. At present, during an operation, the shape of the trachea of the lung of a patient is often deformed due to the influence of various factors such as the respiration and the posture of the patient, and the shape of the trachea of the lung in a medical image acquired by the lung of the patient before the operation is not completely consistent with that of the trachea of the lung, so that a three-dimensional model of a trachea tree during the operation cannot be completely matched with an actual trachea of the patient, and particularly, the deviation of the bronchus at the tail end of the trachea of the lung is larger, which easily causes the decline of navigation precision.

Disclosure of Invention

In view of the above, the present application provides a method, system and electronic device for registration of pulmonary airways that solves or at least partially solves the above-mentioned problems.

Thus, in one embodiment of the present application, a method of registration for a pulmonary trachea is provided. The method comprises the following steps:

acquiring a first moving path point set of a positioning sensor in a lung trachea;

determining a moving path of the positioning sensor in a three-dimensional model of the tracheal tree according to the first moving path point set; the three-dimensional model of the tracheal tree is obtained based on medical image reconstruction of the lung trachea;

acquiring central line information of the three-dimensional model of the tracheal tree;

determining an airway path of the three-dimensional model of the tracheal tree according to the central line information;

extracting a first key point set of the moving path, wherein the first key point set comprises a moving path bifurcation point and a moving path end point of the moving path;

determining a second set of keypoints in the airway path corresponding to the first set of keypoints;

and registering the three-dimensional model of the tracheal tree with the pulmonary trachea according to the first key point set and the second key point set.

In another embodiment of the present application, a registration system for a pulmonary trachea is also provided. The system comprises:

a magnetic field generator for generating a positioning magnetic field; the lung trachea is in the positioning magnetic field;

a positioning sensor capable of generating a positioning signal by sensing the positioning magnetic field when moving in the lung trachea, and transmitting the positioning signal to a processing device, so that the processing device determines a moving path point of the positioning sensor in the lung trachea according to the positioning signal;

the processing device is configured to perform the steps in the registration method for pulmonary trachea provided in the above embodiment.

In one embodiment of the present application, an electronic device is provided. The electronic device includes: a memory and a processor, wherein the memory is used for storing a computer program; the processor is coupled to the memory and is configured to execute the computer program stored in the memory, so as to implement the steps in the registration method for pulmonary airways according to the above embodiment of the present application.

According to the technical scheme provided by the embodiments of the application, on the basis of obtaining a corresponding three-dimensional model of the tracheal tree based on the medical image reconstruction of the pulmonary tracheal, the airway path of the three-dimensional model of the tracheal tree is determined according to the obtained central line information of the three-dimensional model of the tracheal tree; determining a moving path of the positioning sensor in the three-dimensional model of the tracheal tree according to the acquired first moving path point set of the positioning sensor in the tracheal of the lung; then, a first key point set (comprising a moving path bifurcation point and a moving path end point of the moving path) of the moving path is extracted, a second key point set corresponding to the first key point set is determined in the airway path, and the three-dimensional model of the tracheal tree is registered with the lung trachea according to the first key point set and the second key point set. The moving path bifurcation point is generally a path point generated when the positioning sensor moves to the intersection of two corresponding different air pipes in the process of moving in the air pipe of the lung, in other words, the positioning position of the positioning sensor represented by the moving path bifurcation point in the air pipe of the lung is generally at one end, such as the tail end, of the corresponding actual air pipe. The scheme also combines a moving path end point and a point corresponding to the moving path end point on the airway path when the registration is realized, wherein the moving path end point is a point at the end of a corresponding path in the moving path, the characterized positioning position of the positioning sensor in the lung trachea is possibly in the corresponding trachea or at the end, and if the positioning sensor is at the corresponding trachea end, the registration accuracy of the trachea end can be further improved; if the model is in the corresponding trachea (such as the middle of the trachea), the registration accuracy between the three-dimensional model of the tracheal tree and the actual pulmonary trachea is improved to a certain extent. In addition, the number of the bifurcation points and the end points of the moving paths is limited and small, so that the scheme can ensure the registration accuracy and simultaneously effectively reduce the calculation amount of registration data. In conclusion, by adopting the technical scheme provided by the embodiment of the application, the registration precision between the three-dimensional model of the tracheal tree and the actual pulmonary trachea can be effectively improved, particularly the registration precision of the tail end of the trachea, which can provide guarantee for the follow-up accurate guiding operation.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings that are needed to be utilized in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application and that other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIGS. 1a to 1c are schematic structural views of a medical system for pulmonary diagnosis and treatment according to an embodiment of the present application;

fig. 2 is a flowchart of a registration method for pulmonary airways according to an embodiment of the present application;

FIG. 3 is a front plan view of a three-dimensional model of a tracheal tree with centerline information, centerline bifurcation points, etc., and a hierarchical division of the three-dimensional model of the tracheal tree according to an embodiment of the present application;

FIG. 4a is a front plan view of a three-dimensional model of a tracheal tree with a second set of path points provided by an embodiment of the present application;

FIG. 4b is a front plan view of a three-dimensional model of a tracheal tree with a path of movement provided by an embodiment of the present application;

fig. 5 is a schematic structural view of a registration system for pulmonary airways according to an embodiment of the present application;

Fig. 6 is a schematic structural view of a registration device for pulmonary airways according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a computer program product according to an embodiment of the present application;

fig. 9 is a schematic view of a surgical registration interface provided in an embodiment of the present application.

Detailed Description

In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings.

In some of the flows described in the description of the application, the claims, and the figures described above, a number of operations occurring in a particular order are included, and the operations may be performed out of order or concurrently with respect to the order in which they occur. The sequence numbers of operations such as 101, 102, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types. The term "or/and" in the present application is merely an association relationship describing the association object, which means that three relationships may exist, for example: a and/or B are three cases that A can exist alone, A and B exist together and B exists alone; the character "/" in the present application generally indicates that the front and rear associated objects are an "or" relationship. It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a product or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such product or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a commodity or system comprising such elements. Furthermore, the embodiments described below are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In the following embodiments of the present application, reference will be made to a positioning sensor, for example, a positioning sensor provided on a positioning catheter, which uses the principle of electromagnetic navigation (Electromagetic Navigation) unlike the ordinary principle of electromagnetic navigation. The common electromagnetic navigation principle is mainly as follows: the direction of the medical device into the human body is influenced by attracting or repelling the permanent magnet body in the catheter in dependence of an external magnetic field. In the embodiment of the application, the working principle of the positioning sensor is mainly as follows: an electrical current is output to an external control system in response to the magnetic field of the space in which it is located for the control system to locate the position of the corresponding catheter. In detail, the positioning sensor is (signal) connected to the control system by a wired or wireless means. The control system includes a magnetic field generator for generating a magnetic field in a range of positioning spaces. The positioning sensor itself is non-magnetic and the coils in the positioning sensor are used to sense the magnetic field generated by the magnetic field generator. Wherein the magnetic field generator generates a varying magnetic field in a range of positioning space and ensures that the magnetic field characteristics of each electric field in the positioning space are unique. The coils in the position sensor generate currents in the transformed magnetic fields, which are transmitted to the control system after acquisition of the generated signals (i.e. the position signals described in the context), which are analyzed by the control system to determine the exact position and orientation of the respective catheter.

For easy understanding, before describing the technical scheme provided by the embodiment of the application, a medical system for lung surgery navigation is described. In particular, the method comprises the steps of,

fig. 1a and fig. 1b are schematic perspective views of a medical system according to an embodiment of the present application. As shown with reference to fig. 1a and 1b, the medical system comprises: a console 10, a magnetic navigation device 20, a medical imaging device 30, and a positioning medical tool 40. Wherein, the liquid crystal display device comprises a liquid crystal display device,

the magnetic navigation device 20 is used for navigating the positioning medical tool 40 to a corresponding target point (or target area) on the medical path according to the medical path.

The medical path may be planned by a corresponding processing module having a path planning function. Specifically, the processing module may be configured to reconstruct a corresponding three-dimensional model according to medical image data (such as CT, MRI, etc.) of the whole body or a certain body part of the patient, and perform path planning based on the planned three-dimensional model to obtain a corresponding medical path. For example, a three-dimensional model of a tracheal tree including a main trachea and a bronchus can be reconstructed according to medical image data of the lung of a patient before operation, and a navigation path (or a planning path) from the main carina to a target point through the natural trachea of the lung is planned according to the three-dimensional model of the tracheal tree, wherein the navigation path is a medical path. The main carina refers to the bifurcation of the main trachea, which is divided into a left bronchus and a right bronchus.

In practice, the processing module may be provided on the magnetic navigation device 20, or may be provided on a main control trolley 50 for a doctor as shown in fig. 1 c. The above-mentioned processing module may be application software having the functions of reconstructing a three-dimensional model and path planning, registration, etc., which may be installed in a control device such as the master cart 50 or the magnetic navigation device 20, such as the control device 511 (e.g. a computer device) shown in fig. 1 c. However, to reduce the risk of radiation exposure and cross-infection of the doctor, the control device 511 is preferably provided on the main trolley 50 separately from the electromagnetic navigation apparatus 20 so that the doctor can leave the operating room to operate the control device. The master cart 50 can communicate with the magnetic navigation device 20, which can be considered an extension of the display and steering functions of the magnetic navigation device 20. For example: the reconstructed three-dimensional model (such as a three-dimensional model of a tracheal tree), a medical path, an operation progress (such as a registration progress and a real-time navigation progress), an endoscopic image and the like can be displayed through a display screen, in particular a human-computer interaction interface, contained in the control device 511 on the main control trolley 50; in addition, the doctor can trigger the registration operation to start execution by clicking the registration control displayed on the man-machine interaction interface; alternatively, the medical imaging device 30 is controlled to perform scanning through a human-computer interaction interface, and so on.

The medical imaging device 30 is configured to acquire a medical image of a patient. For example, a medical image of a patient's lungs is acquired.

In particular, the medical imaging device 30 may be, but is not limited to, a CT machine, an X-ray machine, etc., and may be broadly referred to as an imaging device for imaging a local or whole body of a patient, including but not limited to, a two-dimensional image, where the conditions allow.

In an example, the medical imaging apparatus 30 includes a C-arm 311 movable relative to the console 10, where two ends of the C-arm 311 are respectively configured to be provided with a radiation module (for emitting corresponding medical radiation, such as X-rays) and an imaging module (not shown in the figure, and may be specifically provided at an end of the C-arm 311 above the console), and the C-opening faces the console. The movement of the C-arm 311 relative to the console means that the opening of the C-arm can move along the length or width direction of the console 10, or that the opening of the C-arm rotates around the console 10, so that the shooting or scanning of the medical imaging device 30 covers the whole console and all directions thereof, and a medical image (i.e. a medical image) with a proper angle can be shot as required, thereby improving the use experience of the medical system.

It should be noted that the medical imaging apparatus 30 may be connected to the control device on the main control carriage 50 or the magnetic navigation apparatus 20, so that a processing module (application software) on the control device may receive and process the medical image obtained by the medical imaging apparatus 30.

The console 10 may be, but is not limited to, an operating table in an operating room, a patient bed for patient care, or the like. The console 10 is capable of transmitting radiation from the medical imaging device 30, and has the effect of not affecting the imaging effect of the medical imaging device 30. In addition, the medical imaging device 30 and the magnetic navigation device 20 may be disposed beside the console 10.

The medical positioning tool 40 refers to a medical tool having a positioning function, such as biopsy, treatment, etc., specifically: may be a guide wire or catheter for positioning, an ablation catheter, etc.

The positioning medical tool 40 is coupled to the magnetic navigation device 20 or to a portion of the magnetic navigation device 20. For example, in case the shape of the magnetic navigation device 20 is configured as shown in fig. 1b, the magnetic navigation device 20 comprises a delivery mechanism 201, which delivery mechanism 201 is connectable to the positioning medical tool 40, the delivery mechanism 201 driving the positioning medical tool 40 into the patient, such as the lung airways (or airways) of the patient, according to the medical path and reaching the respective target point (or target region) under control of the magnetic navigation device 20. The above-mentioned fig. 1b shows an example of delivering the positioning medical tool 40 into the patient by the delivery mechanism, in other embodiments, the positioning medical tool 40 may be delivered into the patient by a corresponding pushing device (a pulling mechanism 2012 described below) through the operation of a physician, and the physician manually controls the pushing device by means of a display device (a display device 511 shown in fig. 1 c) corresponding to the magnetic navigation device, so that the pushing device drives the positioning medical tool 40 to move according to the medical path, thereby achieving the corresponding target point (or target area).

What needs to be explained here is: the transport mechanism 201 described above may include a robotic arm 2011 and a pulling mechanism 2012. The pulling mechanism 2012 is connected to one end of the positioning medical tool 40. The pulling mechanism 2012 can be pulled by pushing with a hand or a mechanical arm 2011, so as to drive the positioning medical tool 40 to enter and exit the trachea of the human body. In addition, the electromagnetic navigation device 20 may have other shape structures besides the shape structure shown in fig. 1b, for example, the shape structure shown in fig. 1c, and the shape structure of the electromagnetic navigation device 20 is not particularly limited in the embodiment of the present application.

Further, the magnetic navigation device 20 may further include a magnetic field generator 202 disposed within the console 10, and a positioning sensor 203 disposed within the positioning medical tool 40. Wherein the magnetic field generator 202 is configured to generate a positioning magnetic field for positioning the positioning sensor 203.

In particular, the magnetic field generator 202 is disposed in the console 10 and is capable of emitting a positioning magnetic field in a direction of the bed surface, in which a corresponding body part (e.g., lung) of the patient is located when the patient is lying on the console 10, and the relative position of the corresponding body part (e.g., lung) of the patient in the positioning magnetic field is fixed because the patient is generally in a general anesthesia state on the console 10. Since the positioning magnetic field has its own coordinate system, which is called a magnetic field coordinate system, whereby the relative position of the corresponding body part (e.g. lung) of the patient is fixed in the magnetic field coordinate system, and the position and direction of the positioning medical tool 40 in the patient can be represented by the position coordinates of the head of the positioning medical tool 40 in the magnetic field coordinate system, in particular, the positioning magnetic field can position the positioning sensor 203 in the positioning medical tool 40, and since the positioning sensor 203 is disposed at the head end position of the medical tool 40, the position and direction of the positioning medical tool 40 in the patient can be obtained by positioning the positioning sensor 203 to obtain the position coordinates of the positioning sensor in the magnetic field coordinate system.

The following describes in detail a method for registering pulmonary airways provided by an embodiment of the present application.

Fig. 2 shows a flowchart of a registration method for pulmonary airways according to an embodiment of the present application. The execution subject of the method is a corresponding processing device, such as the magnetic navigation device 20 shown in fig. 1a to 1c or a control device 511 (such as a computer device) on the main control trolley 50 shown in fig. 1c, which is not particularly limited herein. As shown in fig. 2, the registration method for pulmonary trachea comprises the following steps:

101. acquiring a first moving path point set of a positioning sensor in the lung trachea;

102. and determining a moving path of the positioning sensor in a three-dimensional model of the tracheal tree according to the first moving path point set, wherein the three-dimensional model of the tracheal tree is obtained based on medical image reconstruction of the pulmonary trachea.

103. Acquiring central line information of the three-dimensional model of the tracheal tree;

104. determining an airway path of the three-dimensional model of the tracheal tree according to the central line information;

105. extracting a first key point set of the moving path, wherein the first key point set comprises a moving path bifurcation point and a moving path end point of the moving path;

106. Determining a second set of keypoints in the airway path corresponding to the first set of keypoints;

107. and registering the three-dimensional model of the tracheal tree with the pulmonary trachea according to the first key point set and the second key point set.

In the above-mentioned 101-104, the pulmonary trachea is an actual tracheal tree of the lung, and the corresponding three-dimensional model of the tracheal tree may be obtained by performing correlation processing on a plurality of medical image images (such as two-dimensional tomographic images (CT images) or magnetic resonance images (MRI images) acquired for the lung of the patient from different angles, using a three-dimensional reconstruction technique (such as three-dimensional reconstruction software).

For example, a plurality of two-dimensional CT images of a patient's lungs may be acquired, such as by CT tomography of the patient's lungs; then, the plurality of two-dimensional CT images are imported into a processing module (which may be application software having functions of reconstructing a three-dimensional model, path planning, registration, etc.) according to other embodiments of the present application, and image recognition, segmentation, etc. of tissues such as lung and trachea are performed, so as to obtain three-dimensional modeling parameters of lung and trachea, and then a three-dimensional model of a tracheal tree of the lung and trachea of the patient is constructed based on the three-dimensional modeling parameters.

From the above example, before executing the step 101, the present embodiment may further include the following related implementation steps for reconstructing a corresponding three-dimensional model of the tracheal tree for the pulmonary trachea:

100a, acquiring a plurality of medical image images acquired for a lung trachea;

100b, identifying the plurality of medical image images to obtain three-dimensional modeling parameters of the lung trachea;

100c, constructing the three-dimensional model of the tracheal tree based on the three-dimensional modeling parameters.

After the three-dimensional model of the tracheal tree is obtained, when the steps 103 to 104 are executed, the three-dimensional model of the tracheal tree can be skeletonized to extract the central line information of the three-dimensional model of the tracheal tree, and data support is provided for the subsequent hierarchical division of the three-dimensional model of the tracheal tree, the determination of the airway path and the like.

In particular embodiments, centerline information of the three-dimensional model of the tracheal tree may be extracted using, but not limited to, a corresponding refinement algorithm, such as a topology refinement algorithm. The refinement algorithm is mainly a method for extracting the central line of the target object by repeatedly eroding the surface pixels of the target object until only the skeleton is left. For a specific implementation of extracting centerline information of a three-dimensional model of a tracheal tree using a refinement algorithm, reference may be made to existing correlation schemes.

In the front plan view of the three-dimensional model of the tracheal tree 100 shown in fig. 3, a relatively thin solid line in the three-dimensional model of the tracheal tree 100 is shown, i.e., the centerline information of the three-dimensional model of the tracheal tree is represented.

After extracting the central line information of the three-dimensional model of the tracheal tree, the path formed by the central line information is used as the airway path of the three-dimensional model of the tracheal tree.

In addition, since in reality, the pulmonary trachea of a person is tree-shaped, the pulmonary trachea is generally divided into different levels from top to bottom according to bifurcation levels, for example: the main bronchi (level 1, the tube with one end connected to the larynx and the other end connected to the bronchi, the air passage) located in the trunk enter the lung through the hilum, and branch into two main bronchi (also called She Zhuzhi bronchi, level 2): a right main bronchus and a left main bronchus; further, the two main bronchi may be further bifurcated, respectively, such as the right main bronchus further bifurcated into the upper right bronchus, the lower right bronchus, and so on. Based on this, in the embodiment, the centerline bifurcation point in the centerline information can be determined by analyzing the centerline information of the extracted three-dimensional model of the tracheal tree, so that the three-dimensional model of the tracheal tree is divided into multiple stages of tracheal models according to the centerline bifurcation point and the actual anatomical structure of the pulmonary tracheal, so as to provide support for the subsequent processing of each step. That is, the method provided in this embodiment may further include the following steps:

S11, determining a center line bifurcation point in the center line information;

and S12, carrying out hierarchical division on the three-dimensional model of the tracheal tree from top to bottom according to the center line bifurcation point to obtain a multi-stage tracheal model.

In the above description, the center line bifurcation point refers to an intersection point of at least two center line segments. The centerline bifurcation point in the centerline information can be seen from the multiple bifurcation points shown with black circles "·" in fig. 3, namely: bifurcation point b0, bifurcation point b1, bifurcation point b2, bifurcation point b3, bifurcation point b4, bifurcation point b5, bifurcation point b6. Since in the present embodiment, the path formed by the centerline information is the airway path as the three-dimensional model of the airway tree, for this reason, in a key point description scene of determining the airway path in the following detailed description, the centerline bifurcation points (i.e., bifurcation points b0 to b 6) described above are also referred to as airway path bifurcation points.

Accordingly, with continued reference to fig. 3, according to the above-mentioned centerline bifurcation point, the multi-stage tracheal model obtained by performing hierarchical division on the tracheal tree three-dimensional model from top to bottom may include:

a primary tracheal model 1 (primary tracheal model);

two secondary tracheal models 2, which are connected to the primary tracheal model 1 in a bifurcated manner, namely: a right main bronchus model 21, a left main bronchus model 22;

Four three-level tracheal models 3 (which are leaf bronchial models); wherein two of the four tertiary gas pipe models are connected to one of the two secondary gas pipe models 2 and the other two tertiary gas pipe models are connected to the other of the two secondary gas pipe models 2. Specifically, the four three-stage tracheal model 3 includes: an upper right-leaf bronchial model 31, a lower right-leaf bronchial model 32, which are bifurcated from the right main bronchial model 21; an upper left-leaf bronchial model 33, a lower left-leaf bronchial model 34, which are bifurcated from the left main bronchial model 22.

8 four-level tracheal models 4 (segment bronchial models). Specifically, the 8 four-stage tracheal models 4 include: a first segment of bronchial model 41, a second segment of bronchial model 42 connected to the upper right leaf bronchial model 31, a third segment of bronchial model 43, a fourth segment of bronchial model 44 connected to the lower right leaf bronchial model 32; and fifth and sixth segment bronchial models 45, 46 connected to the upper left leaf bronchial model 33, seventh and eighth segment bronchial models 47, 48 connected to the lower left leaf bronchial model 34.

Correspondingly, the central line information of the three-dimensional model of the tracheal tree comprises central lines of all levels of tracheal models.

In performing step 101 described above, the first set of movement path points may be obtained from the magnetic navigation device 20 as shown in fig. 1a or 1 b. Specifically, the magnetic navigation device 20 shown in fig. 1a or 1b may determine a first moving path point set (i.e. an actual moving path point set) generated by the movement of the positioning sensor in the lung trachea according to the acquired positioning signal sent in real time when the positioning sensor moves (walks) in the lung trachea, and send the first moving path point set to the execution body of the embodiment of the present application; the positioning signal is generated by the positioning sensor according to the current generated by the acquired self-internal coil in the transformed magnetic field. The magnetic field is changed by the magnetic navigation device 20 controlling the magnetic field generator 202 arranged in the operation table 10, and because the patient is in the operation table 10 when lying on the back, the patient's lung is in the changing magnetic field, the sensor can sense the changing magnetic field and generate corresponding current when moving in the trachea of the patient's lung.

The first set of travel path points includes travel path points that characterize the respective positioning positions of the positioning sensor within the lung.

When executing the step 102, the coordinates of each moving path point included in the first moving path point set may be spatially transformed according to the current spatial coordinate transformation relationship (which is the spatial coordinate transformation relationship between the actual physical space (magnetic space) of the lung trachea and the image space of the three-dimensional model of the tracheal tree), so as to map (or divide) each moving path point included in the first moving path point set into a corresponding level of tracheal model in the three-dimensional model of the tracheal tree, thereby obtaining a second moving path point set of the positioning sensor in each level of tracheal model; then, a path search is performed for a second set of movement path points in each stage of the tracheal model, so as to determine a movement path of the positioning sensor in the tracheal tree three-dimensional model according to the path search result. The purpose of the path search is to find the shortest moving path of the positioning sensor moving in each level of tracheal models, and combine the shortest moving paths of the positioning sensor in each level of tracheal models to obtain the moving path of the positioning sensor finally in the tracheal tree three-dimensional model. In performing the path search, a point corresponding to a main carina of a lung trachea in a three-dimensional model of a tracheal tree may be used as a search starting point.

From the above, that is, the three-dimensional model of the tracheal tree in this embodiment includes a multi-stage tracheal model, and in one implementation technical solution, the determining 102 "the moving path of the positioning sensor in the three-dimensional model of the tracheal tree according to the first moving path point set" may specifically include:

1021. determining a second moving path point set of the positioning sensor in each level of tracheal model according to the first moving path point set;

1022. determining points corresponding to main carina of the lung trachea in the three-dimensional model of the tracheal tree as reference points;

1023. starting from the datum point, based on a second moving path point set of the positioning sensor in each level of tracheal models, respectively performing path searching for each level of tracheal models to obtain the shortest moving path of the positioning sensor in each level of tracheal models;

1024. and merging the shortest moving paths of the positioning sensor in each level of tracheal models to obtain the moving paths.

In 1021, considering that the amount of data acquired by the positioning sensor is relatively large and includes a large amount of redundant data, that is, the first moving path point set includes a large amount of redundant path points, if a coordinate space conversion method is directly adopted to map each moving path point included in the first moving path point set into a corresponding stage of the three-dimensional model of the tracheal tree, then no processing is performed, a second moving path point set of the positioning sensor in each stage of the tracheal model is directly obtained according to the mapping result, and then there are problems that the data is too much and the data processing amount is large. Specifically, in one implementation technical solution, 1021 "determining, according to the first moving path point set, a second moving path point set of the positioning sensor in each stage of the tracheal model, where the second moving path point set includes the following steps:

10211. Determining a first principal point set corresponding to the moving path points contained in the first moving path point set in each level of tracheal model;

10212. determining the trachea radius of each level of trachea model;

10213. performing downsampling processing on a first voxel set in each level of tracheal model based on the tracheal radius;

10214. and determining the first integral point set in each level of tracheal model after the downsampling treatment as a second moving path point set of the positioning sensor in each level of tracheal model.

In the implementation, the implementation of step 10211 is implemented by performing spatial coordinate transformation on each moving path point included in the first moving path point set, so as to map each moving path point included in the first moving path point set to a corresponding level of tracheal model in the three-dimensional model of the tracheal tree, thereby determining according to the mapping result. For the steps 10212 to 10214, as shown in fig. 3 or fig. 4a, taking a certain three-level tracheal model 3 such as an upper right-leaf bronchial model 31 as an example, the tracheal radius R of the upper right-leaf bronchial model 31 can be obtained by calculating the distance from each voxel point on the surface of the upper right-leaf bronchial model 31 to the center line thereof, and then averaging the calculated distances; further, based on the trachea radius R, the first set of voxels VG in the upper right-leaf bronchial model 31 may be downsampled using, but not limited to, voxel grid (voxel grid); wherein, in the down-sampling process, the size of the mesh division is determined based on the tracheal radius R of the upper-leaf bronchial model 31. Finally, the first set of voxel points VG after the downsampling process is the second set of moving path points of the positioning sensor in the upper right leaf bronchus model 31.

What should be additionally stated here is: the radius R of the bronchus of the upper right-leaf bronchus model 31 may be determined by other methods, for example, to reduce the calculation amount, two voxel points on the surface of the upper right-leaf bronchus model 31 corresponding to two end points of the center line of the upper right-leaf bronchus model 31 may be determined; then, taking the two voxel points as the head and tail points, taking a curve on the surface of the upper right bronchus model 31, and calculating the distance between each voxel point on the curve and the central line of the upper right bronchus model, thereby obtaining the tracheal radius R of the upper right bronchus model 31 by taking the average value of the calculated distances.

For a specific implementation of downsampling data by voxel grid method, reference may be made to the existing relevant content, and details will not be given here. The voxel grid method is used for realizing the downsampling of the data, so that the data volume can be reduced, and the morphological characteristics of the original data can be maintained.

Based on the foregoing, if the third target level tracheal model is one of the multi-level tracheal models, then: in a specific implementation, the determining the tracheal radius of the third target-level tracheal model may include:

Determining a second centerline of the third target level tracheal model from the centerline information;

calculating a second distance from each second voxel on the third target level tracheal model surface to the second centerline;

and determining the average value of the second distances from the second body points to the second central line as the tracheal radius of the third target-level tracheal model.

In another specific implementation, the determining the tracheal radius of the third target-level tracheal model may include:

determining two points on the surface of the second target-level tracheal model, which have the same ordinate as the two end points of the second central line; in practice, the two points can be understood as: leading a perpendicular to the surface of the third target-level tracheal model through two end points of the second central line respectively, and obtaining two perpendicular points corresponding to the two end points of the second central line;

taking one of the two points as a head point and the other as a tail point, and taking a curve on the surface of the third target level air pipe model;

Calculating a third distance from each point on the curve to the second center line;

and determining the average value of the third distance from each point on the curve to the second central line as the tracheal radius of the third target-level tracheal model.

To facilitate understanding of step 1021, an example is illustrated below. In particular, the method comprises the steps of,

assuming that the positioning sensor enters the actual pulmonary trachea of a patient under the control of a doctor, the main trachea, the right main bronchus, the right upper lobe bronchus and a section of bronchus which is connected with the right upper lobe bronchus and is located upwards are traversed by the current positioning sensor in the pulmonary trachea moving process. After the first moving path point set generated by the current positioning sensor in the pulmonary tracheal movement process is acquired, performing various processes such as spatial coordinate conversion on coordinates of each moving path point contained in the first moving path point set, and finally determining a point set corresponding to the first moving path point set in the tracheal tree three-dimensional model, wherein the point set comprises all virtual points shown in the tracheal tree three-dimensional model in fig. 4a, namely all virtual points shown by a virtual line L, and then: such as by a virtual line segment in the right main bronchus model 21 (which is the secondary bronchus model) in the virtual line L The virtual point set is shown as a second moving path point set of the positioning sensor in the primary tracheal model 1; similarly, a second moving path point set of the positioning sensor in other all levels of tracheal models can be obtained. Wherein, e.g. leftThe primary bronchus model 22 (which is a secondary bronchus model) does not include virtual points, i.e., it characterizes the current positioning sensor as not having traversed into the bronchus region of the lung corresponding to the left primary bronchus model 22, so that it does not have a corresponding second set of movement path points in the left primary bronchus model 22.

With the above example, the second set of moving path points of the positioning sensor in a certain level of tracheal model can be simply understood as: the method is determined according to a moving path point generated in the moving process of the positioning sensor in the lung trachea and a corresponding tracheal region in a certain level tracheal model; the first moving path point set includes moving path points generated in the process that the positioning sensor moves in the lung trachea with a corresponding trachea region in the certain level trachea model. In addition, the present application shows the second set of movement path points of the positioning sensor in each stage of the tracheal tree three-dimensional model in front plan view shown in fig. 4a by means of virtual points (i.e., points shown by virtual line L).

In the above 1022, since the main carina refers to the bifurcation of the main trachea in the lung trachea, the point corresponding to the main carina of the lung trachea in the three-dimensional model of the tracheal tree can be determined based on the intersection point of the centerlines of the primary tracheal model and the two secondary tracheal models, and thus can be used as the reference point.

For example, referring to fig. 4a or 3, the intersection point of the center lines of the primary airway model 1 (main airway model) and the center lines of the two secondary airway models 2 (including the right main airway model 21 and the left main airway model 22), that is, the bifurcation point b0, acquired from the center line information of the three-dimensional airway tree model may be determined as the reference point a.

From the reference point, it may be determined for which one or both of the hierarchical tracheal models should start performing the path search at the present time, and then perform the path search based on the determined second set of moving path points in the one or both hierarchical tracheal models, as described above 1023. Specifically, in a specific implementation manner, the 1023 "starting from the reference point, based on the second moving path point sets of the positioning sensor in each stage of the tracheal models, respectively, perform path searching for each stage of the tracheal models to obtain the shortest moving path of the positioning sensor in each stage of the tracheal models", which may be implemented by adopting the following specific steps:

10231. Starting from the datum point, determining a second target-level tracheal model of which the path search is currently required to be executed;

10232. determining a starting position and an ending position of the second target level tracheal model;

10233. searching a second moving path point set of the positioning sensor in the second target-level tracheal model, wherein the moving path point with the shortest distance from the initial position is determined as a searching starting point, and the moving path point with the shortest distance from the final position is determined as a searching end point;

10234. performing a path search based on the search start point, the search end point, and a second set of movement path points of the positioning sensor within the second target level tracheal model to determine a shortest path from the search start point to the search end point as a shortest movement path of the positioning sensor in the second target level tracheal model according to a search result;

10235. after the path searching is completed aiming at the second target-level air pipe model, judging whether an air pipe model needing to be subjected to the path searching exists in the multi-level air pipe model or not;

10236. if so, returning to the step of executing the second target level tracheal model determined in 10231 above that the path search should be currently executed.

For example, referring to fig. 4a, assuming that the reference point a is currently used as a starting point, a path search is performed around based on the second moving path point set of the positioning sensor in each level of tracheal model, and the utilized path search algorithm is a coordinate-based path search algorithm (such as a CPS algorithm), the specific search process may be as follows:

firstly, comparing the shape and the direction of central line information of a three-dimensional model of the tracheal tree, searching forward, backward and the like relative to a datum point according to the obtained coordinates of all second moving path points in a second moving path point set of a positioning sensor in each level of tracheal models to determine a second moving path point adjacent to the datum point A, and specifically determining two second moving path points adjacent to the datum point A, wherein one of the second moving path points is positioned in a first level tracheal model 1 and the other second level tracheal model 2 (namely a right main tracheal model 21) which is connected with the first level tracheal model 1 and is right-deviated; and then, according to the determined second moving path point adjacent to the reference point A, determining that a second target level tracheal model for which path searching is currently required to be performed is as follows: a primary tracheal model 1 and a right main tracheal model 21.

For the two second target level tracheal models determined above (i.e., primary tracheal model 1, right main tracheal model 21), a path search may be performed simultaneously.

As exemplified by the right main bronchus model 21: the centerline l1 of the right main bronchus model 21 (which is a solid line between the bifurcation point b0 and the bifurcation point b 1) may be determined according to the centerline information of the three-dimensional model of the bronchus tree, and then the starting position and the ending position of the right main bronchus model 21 may be determined according to the two end points (i.e., the bifurcation point b0 and the bifurcation point b 1) of the centerline l1, for example, when the path search is performed by taking the reference point as the starting point, the forward search should be performed with respect to the reference point with respect to the right main bronchus model 21, for this reason, the end point closest to the reference point according to the two end points of the centerline l1 may be taken as the starting position of the right main bronchus model 21, and the other end point may be taken as the ending position of the right main bronchus model 21, i.e., the starting position and the ending position of the right main bronchus model 21 are respectively the bifurcation point b0 and the bifurcation point b1. Further, a second set of moving path points (denoted as a set of moving points G1, i.e., a set of virtual points shown by way of virtual lines in the right main bronchus model 21 in fig. 4 a) of the positioning sensor within the right main bronchus model 21 may be searched for a moving path point c0 having the shortest distance from the bifurcation point b0 to determine as a search start point and a moving path point c1 having the shortest distance from the bifurcation point b1 to determine as a search end point; still further, with the searching start point (i.e. the moving path point c 0) as a first path point (i.e. an initial path point), performing forward searching with respect to the first path point, and determining at least one first adjacent point adjacent to the first path point in the moving point set G1; then, respectively calculating the distance (such as Euclidean distance) between the first path point and each first adjacent point, selecting the first adjacent point with the shortest corresponding distance as a second path point, and adding the second path point into a Search path point set search_G corresponding to the right main bronchus model 21; continuing to perform forward Search relative to the second path points, determining at least one second adjacent point adjacent to the second path points on the moving point set G1, selecting a second adjacent point with the shortest distance to the second path points from the at least one second adjacent point as a third path point, and adding the third path point into the Search path point set search_G corresponding to the right main bronchus model 21; and so on, until the search reaches the search end point (i.e., the moving path point c 1), the execution of the path search for the right main bronchus model 21 may end. Finally, each path point included in the Search path point set search_g is sequentially connected, so that a path can be generated, and the path is the shortest moving path of the positioning sensor in the right main bronchus model 21.

In the same way, the above path search may also be performed for the primary air pipe model 1 (primary air pipe model) to determine the shortest moving path of the position sensor in the primary air pipe model 1.

What should be additionally stated here is: the forward search described in the above example may be understood as searching in the increasing direction of the airway model level in the three-dimensional model of the airway tree. When a path search is performed for the primary tracheal model 1, a backward search is performed.

After the above-mentioned path searching for the right main bronchus model 21 is completed, the forward searching with respect to the moving path point c1 may be continued, so as to determine whether there is a second moving path point adjacent to the moving path point c1 according to the obtained coordinates of all the second moving path points in the second moving path point set of the positioning sensor in each stage of the bronchus models, if so, determine the corresponding stage of the bronchus model where the second moving path point adjacent to the moving path point c1 is located, as the second target stage of the bronchus model where the path searching should be performed next, and perform the path searching again for the second target stage of the bronchus model where the path searching should be performed next, and the detailed path searching implementation process may refer to the path searching process detailed for the right main bronchus model 21.

Based on the above example, a specific implementation technical solution of 10232 "determine the starting position and the ending position of the second target level tracheal model" is:

102321, determining a first centerline of the second target level tracheal model from the centerline information;

102322, determining a starting position and an ending position of the second target-level tracheal model according to two end points of the first central line.

In 1024, the shortest moving paths of the positioning sensor in each stage of the tracheal model can be combined, so that the moving path of the positioning sensor in the tracheal tree three-dimensional model can be finally obtained. Wherein, the merging can be realized in an end-to-end mode.

The above-mentioned resulting movement path of the positioning sensor in the three-dimensional model of the tracheal tree can be seen in the dashed line shown in fig. 4b in the three-dimensional model of the tracheal tree.

After the air passage path of the three-dimensional model of the tracheal tree and the moving path of the positioning sensor in the three-dimensional model of the tracheal tree are obtained through the steps 101-104, the key points of the moving path and the air passage path can be further extracted for subsequent registration. In order to ensure that the three-dimensional model of the tracheal tree and the actual pulmonary tracheal can be accurately registered later, particularly to effectively avoid deformation of the actual pulmonary tracheal caused by actions such as breathing of a patient in operation, particularly that deformation is relatively large at the tracheal end of the pulmonary tracheal, the problem that the tracheal end of the actual pulmonary tracheal cannot be well matched with the tracheal end of the tracheal model in the three-model of the tracheal tree and large deviation exists is solved, the embodiment considers that when the positioning sensor moves in the pulmonary tracheal, if the positioning sensor moves to the intersection of at least two different air pipes in the pulmonary tracheal, the positioning sensor is at the end of the corresponding air pipe end, for example, if the positioning sensor moves to the intersection of the main tracheal and the two main air pipes in the pulmonary tracheal, the positioning sensor can be considered to be at the end of the main tracheal at this time, and the positioning sensor uses a path point generated at the end of the main tracheal as a key point to perform registration analysis, thereby being beneficial to improving the registration accuracy of the tracheal end, and on the basis, the embodiment takes a bifurcation point on a path as a key point; in addition, it is also considered that a point on the end of the path (i.e., a path end point) may also be a path point generated by the positioning sensor moving to the end of the corresponding trachea, for example, a path point generated by the positioning sensor moving to the end of a certain segment of trachea (e.g., an actual trachea corresponding to the first segment of trachea model 41 shown in fig. 3) in the lung trachea is a path end point, so that the registration accuracy of the end of the trachea is further improved, and the present embodiment further regards the end of the path on the path as a key point, wherein the key point on the airway path may be determined based on the extracted key point on the moving path.

Specifically, when the above step 105 is performed to extract the key points (the first key points) on the moving path, the above reference points may be used as the starting points, and four-circle searching may be performed on the moving path, so as to extract the bifurcation points of the moving path on the moving path, until the end of each path included in the moving path is searched, and the corresponding moving path end point is extracted, so as to obtain the first key point set of the moving path.

Namely, the first key point set includes: the movement path bifurcation point and the movement path end point may include, in particular, at least one movement path bifurcation point and at least one movement path end point. The reason for including at least one movement path bifurcation point will be described in detail in the last part of the text embodiments below.

For example, referring to the movement path shown in fig. 4b with a dotted line, at least one movement path bifurcation point included in the first keypoint set of the movement path is: as in FIG. 4bThe bifurcation point c0', bifurcation point c1', bifurcation point c2' are shown, and at least one moving path end point is included as follows: as in fig. 4b +.>The path end point d1', the path end point d2' are shown.

In the above 106, the second set of key points specified for the airway path corresponds to the first set of key points, and for this purpose, the second set of key points includes an airway path bifurcation point and an airway path end point corresponding to the movement path bifurcation point and the movement path end point, respectively.

In the implementation, when determining the bifurcation point of the airway path, the bifurcation point of the airway path corresponding to the bifurcation point of the moving path can be searched on the airway path based on the target-level tracheal model where the bifurcation point of the moving path is located. Based on the above, the second set of key points includes airway path bifurcation points; accordingly, the step of determining 106 "the second set of key points corresponding to the first set of key points on the airway path" may specifically include the following steps:

1061. determining a first target-level tracheal model in which the bifurcation point of the moving path is located in the tracheal tree three-dimensional model;

1062. and searching an air pipe path bifurcation point which is located in the first target level air pipe model and has the shortest distance with the moving path bifurcation point on the air path.

For example, in the example of step 105, with continued reference to fig. 4b and with reference to fig. 3, taking the bifurcation point of the moving path as bifurcation point c1' as an example, according to the obtained coordinates of bifurcation point c1', it may be determined that bifurcation point c1' is in a right main bronchus model 21, which is a right-biased secondary bronchus model; further, by looking up on the airway path, it can be determined that the bifurcation point b1 on the airway path is in the right main bronchus model 21 and is closest to the bifurcation point c1', i.e., the bifurcation point b1 is the bifurcation point of the airway path corresponding to the bifurcation point c 1'. Accordingly, it is also possible to determine that the bifurcation points of the airway path corresponding to the bifurcation points c0 'and c2' of the other moving paths included in the first set of key points are: bifurcation point b0, bifurcation point b6.

Further, the second set of key points further includes an airway path endpoint; and "determining a second set of keypoints corresponding to the first set of keypoints in the airway path" in 106 above, further specifically including the steps of:

1063. along the moving path, determining a target moving path bifurcation point nearest to the moving path end point;

1064. calculating a first distance between the bifurcation point of the target moving path and the end point of the moving path;

1065. determining a searching direction according to the bifurcation point of the target moving path and the end point of the moving path;

1066. determining a target airway path bifurcation point corresponding to the target movement path bifurcation point from the airway path;

1067. and searching an airway path end point corresponding to the moving path end point on the airway path according to the searching direction, wherein the distance between the airway path end point and the bifurcation point of the target airway path is equal to the first distance.

For example, with continued reference to fig. 4b, taking the moving path end point as the path end point d2', then:

first, a bifurcation point of a target moving path closest to the path end point d2 'on the moving path (i.e., a dashed line shown in the figure) may be determined as a bifurcation point c2', and then a distance between the bifurcation point c2 'and the path end point d2' is calculated as a first distance distance_0 by using a corresponding distance calculation algorithm (e.g., manhattan distance) according to the coordinates of the bifurcation point c2 'and the coordinates of the path end point d 2'. And determines a search direction for subsequent use according to the bifurcation point c2 'and the path end point d 2'. The search direction is indicated by the solid arrow in fig. 4b, i.e. the increasing direction of the level of the different level of the tracheal model in the three-dimensional model of the tracheal tree, more specifically the increasing direction of the level which is biased to the right.

Then, the target airway path bifurcation point corresponding to bifurcation point c2' on the airway path may be determined as follows in accordance with the determined airway path bifurcation point scheme described in steps 1061-1062 above: bifurcation point b6. Alternatively, the steps 1061 to 1062 may be omitted, and the bifurcation points of the air path corresponding to the bifurcation points c0', c1', and c2' included in the first key information on the air path determined by executing the steps 1061 to 1062 may be: among the bifurcation points b0, b1, and b6, the target airway path bifurcation point corresponding to the bifurcation point c2' is determined as: bifurcation point b6.

Further, according to the above determined searching direction, a path point having a distance from the bifurcation point b6 equal to the first distance distance_0, such as the path point e1 shown in fig. 4b, is searched for as an airway path end point corresponding to the path end point d 2'.

Accordingly, the airway path end point corresponding to the path end point d1' which is the other moving path end point included in the first key point set on the airway path can also be determined, as the path point e2 shown in fig. 4 b. In determining the airway path end point corresponding to the path end point d1' on the airway path, the direction of the search is as indicated by the dashed arrow in fig. 4 b.

To sum up, the second set of key points of the finally determined airway path may include: the airway path bifurcation point b0, the airway path bifurcation point b1, the airway path bifurcation point b6, the path point e1, the path point e2.

What should be additionally stated here is: as can be appreciated in conjunction with the examples described above with respect to determining the bifurcation point and the end point of the airway path, embodiments of the present application may determine the end point of the airway path after determining the bifurcation point of the airway path, or may perform the determination of the bifurcation point of the airway path during the determination of the end point of the airway path, which is not limited herein. Such as: in the above example given for the above steps 1063 to 1067, in the implementation process of determining that the target airway path bifurcation point corresponding to the bifurcation point c2' (being the movement path bifurcation point) is the bifurcation point b6, if the following means are adopted: the respective airway path bifurcation points corresponding to the respective movement path bifurcation points, i.e., bifurcation point c0', bifurcation point c1', bifurcation point c2', respectively, on the airway path determined immediately before by executing the above steps 1061 to 1062 are: the bifurcation point b0, the bifurcation point b1 and the bifurcation point b6 are determined, and then the end point of the airway path is determined after the bifurcation point of the airway path is determined; if the method is as follows: the determination of the bifurcation point of the airway path is performed during the process of determining the endpoint of the airway path by determining the bifurcation point scheme of the airway path as described in steps 1061-1062.

In 107, based on the determined first set of key points of the moving path and the determined second set of key points of the airway path, a corresponding registration algorithm may be used to find a spatial coordinate transformation relationship (or transformation matrix) between the first set of key points from an actual physical spatial coordinate system (which is a magnetic spatial coordinate system) of the pulmonary trachea and the second set of key points from the spatial coordinate system of the image, so that the two may be spatially matched, and update the original spatial coordinate transformation relationship to the newly found spatial coordinate transformation relationship. And the spatial matching of the three-dimensional model of the tracheal tree and the actual pulmonary trachea can be accurately realized according to the newly found spatial coordinate transformation relation.

The registration algorithm described above may be, but is not limited to being: ICP (Iterative Closest Point), iterative closest point algorithm), NDT (Normal Distribution Transform, normal distributed point cloud algorithm), deep learning based point cloud registration, and other fine registration algorithms. The present embodiment is preferably an ICP algorithm, and specific implementation of finding a spatial coordinate transformation relationship between the first set of key points and the second set of key points by using the ICP algorithm can be seen in the related content.

According to the technical scheme provided by the embodiment, on the basis of obtaining a corresponding three-dimensional model of the tracheal tree based on the medical image graph reconstruction of the lung trachea, the airway path of the three-dimensional model of the tracheal tree is determined according to the obtained central line information of the three-dimensional model of the tracheal tree; determining the moving path of the positioning sensor in the three-dimensional model of the tracheal tree according to the acquired first moving path point set of the positioning sensor in the tracheal of the lung; then, a first key point set (comprising a moving path bifurcation point and a moving path end point on the path) of the moving path is extracted, a second key point set corresponding to the first key point set is determined in the airway path, and the three-dimensional model of the tracheal tree is registered with the lung trachea according to the first key point set and the second key point set. The moving path bifurcation point is generally a path point generated when the positioning sensor moves to the intersection of two corresponding different air pipes in the process of moving in the air pipe of the lung, in other words, the positioning position of the positioning sensor represented by the moving path bifurcation point in the air pipe of the lung is generally at one end, such as the tail end, of the corresponding actual air pipe. The scheme also combines a moving path end point and a point corresponding to the moving path end point on the airway path when the registration is realized, wherein the moving path end point is a point at the end of a corresponding path in the moving path, the characterized positioning position of the positioning sensor in the lung trachea is possibly in the corresponding trachea or at the end, and if the positioning sensor is at the corresponding trachea end, the registration accuracy of the trachea end can be further improved; if the model is in the corresponding trachea (such as the middle of the trachea), the registration accuracy between the three-dimensional model of the tracheal tree and the actual pulmonary trachea is improved to a certain extent. In addition, the number of the bifurcation points and the end points of the moving paths is limited and small, so that the scheme can ensure the registration accuracy and simultaneously effectively reduce the calculation amount of registration data. In conclusion, by adopting the scheme, the registration precision between the three-dimensional model of the tracheal tree and the actual pulmonary trachea can be effectively improved, particularly the registration precision of the tracheal end can be guaranteed for the follow-up accurate guiding operation.

Generally, the point cloud registration is generally divided into two registration steps, namely coarse registration and fine registration, wherein the coarse registration is the initial registration of the point cloud, and refers to the alignment of the point clouds in two different spaces as much as possible through an initial space transformation matrix. After coarse registration, the overlapped parts of the two point clouds can be approximately aligned, but the precision is far from reaching the subsequent positioning requirement, so that further fine registration is needed, and the fine registration refers to further calculating a space transformation matrix approximated by the two point clouds on the basis of initial registration. Based on this, before executing the step 102, the first coarse registration may be performed on the three-dimensional model of the tracheal tree and the first moving path point by using a corresponding coarse configuration algorithm, so that the first moving path point set and the three-dimensional model of the tracheal tree after coarse registration are used for performing subsequent processing, thereby implementing the above-mentioned fine registration which is beneficial to the first key point set and the second key point set. That is, further, the method provided in this embodiment further includes the following steps:

s21, registering the first moving path point set and the three-dimensional model of the tracheal tree to obtain the registered first moving path point set and the registered three-dimensional model of the tracheal tree;

S22, triggering and executing the step 102 based on the registered first moving path point set and the registered three-dimensional model of the tracheal tree.

In particular embodiments, the first set of mobile path points and the three-dimensional model of the tracheal tree may be coarsely registered using, but not limited to, more classical coarse registration algorithms, such as RANSAC (Random Sample Consensus ), 4PCS (4-Point Congruent Sets, congruent four-point set), etc., algorithms

Of course, in this rough registration process, the above method used in achieving the fine registration may be used, for example, ICP, NDT, etc., which is not limited herein. However, because the data processing amount often involved in the coarse registration process is relatively large, the problems of relatively low calculation speed, relatively low precision and the like often exist in the coarse registration realized by adopting the algorithms of ICP, NDT and the like, for the coarse registration in this embodiment, the algorithms of RANSAC, 4PCS and the like are preferably realized, and more preferably, the RANSAC algorithm is utilized, and the specific process of realizing the coarse registration by utilizing the RANSAC algorithm can be seen from the existing related contents.

Further, the method provided in this embodiment may further include the following steps:

108. and periodically updating the first moving path point set to re-execute the steps according to the updated first moving path point set.

The purpose of this embodiment to perform the above step 108 is to: realizing the real-time registration of the three-dimensional model of the tracheal tree and the pulmonary trachea. The period of updating the first moving path point set may be flexibly set according to practical situations, such as 1 second, 2 seconds, 1 minute, and the like, which is not limited herein.

In summary, the following should be added: considering that the point cloud data collected by the positioning sensor (i.e., the above-mentioned moving path point data (or referred to as the position data of the positioning sensor in the lung trachea) tends to be meaningless if the positioning sensor is always in a non-moving state in the lung trachea, and that the amount of the point cloud data collected by the positioning sensor tends to be relatively small at the time of the initial registration, in this case, the accuracy of the registration is relatively low if the point cloud data collected by the positioning sensor is processed for registration calculation or the like, for this reason, the present embodiment starts to process the point cloud data collected by the positioning sensor to determine the moving path of the current positioning sensor in the three-dimensional model of the tracheal tree, the first key point set of the moving path, and the like, if it is determined that the number of the point clouds collected by the positioning sensor is greater than or equal to the set number threshold, and the moving range of the positioning sensor is greater than the set range threshold.

In this embodiment, since the point corresponding to the main carina of the trachea of the lung in the three-dimensional model of the tracheal tree is taken as the reference point, as shown in fig. 4a, the registration scheme provided in this embodiment is triggered to be executed only when the positioning sensor is detected to reach the reference point, for example: in the lung operation navigation, if the positioning sensor is determined to reach the datum point A in the three-dimensional model of the tracheal tree according to the current positioning position of the positioning sensor in the lung trachea during the movement of the lung trachea of the patient by a doctor, a prompt message such as "the positioning sensor is placed in the main carina position and the registration is started by clicking the [ confirm ] button" is displayed on the operation registration interface as shown in fig. 9, and the registration is automatically executed after responding to the "confirm" button in the prompt message clicked by the doctor. Wherein the surgical registration interface described above may be displayed by a display screen of a control device 511 as shown in fig. 1 c; during registration, the physician may continue to manipulate the positioning sensor to move in the patient's lung airways. Based on the above-mentioned example, the technical solution provided in this embodiment, when starting to process the point cloud data collected by using the positioning sensor to perform the registration, the positioning sensor reaches at least an end of a primary tracheal model (main tracheal model) in the tracheal three-dimensional model, which also makes the first key point set described above include at least one movement path bifurcation point. In addition, based on the above example, when the threshold value of the setting range is set, the embodiment of the present application may be flexibly set according to practical situations, so long as it is ensured that when processing using the point cloud data acquired by the positioning sensor is started, the moving range of the positioning sensor has exceeded the area of the primary tracheal model (i.e., the primary tracheal model) of the three-dimensional model of the tracheal tree, that is, it is ensured that the positioning sensor is not currently in the primary tracheal model of the three-dimensional model of the tracheal tree.

In the processing process, the point cloud data acquired by using the positioning sensor is preprocessed first to delete some abnormal data, such as outlier data and invalid data generated by signal loss. For example, the coil in the positioning sensor is disturbed to sense the magnetic field, so that no current can be generated by itself, and at this time, the positioning sensor sends a default signal to the execution body of the embodiment, and the received default signal becomes invalid data or may be outlier data.

An embodiment of the application further provides a registration system for the pulmonary trachea, which corresponds to the embodiment of the method. As shown with reference to fig. 5, the registration system for the pulmonary airways may include:

the processing device is used for executing the steps in the embodiment of the lung tracheal registration method provided by the application.

The above description of the magnetic field generator and the positioning sensor may be related to other embodiments of the present application, and will not be described in detail herein.

The processing device may be a control device having data processing functions, such as the control device 511 shown in fig. 1c (e.g. a computer device). The control device may be provided independently on the main control carriage 50 for doctor as shown in fig. 1c, or may be provided integrally with the magnetic navigation device 20 as shown in fig. 1a, i.e. may be provided on the magnetic navigation device 20, which is not limited in this embodiment.

It should be noted that, in addition to the above devices or apparatuses, other devices or apparatuses may be included in the registration system for pulmonary airways according to the embodiment of the present application, and the related other devices or apparatuses may include other devices or apparatuses may participate in the medical system described in fig. 1a to 1 c.

Fig. 6 shows a block diagram of a registration apparatus for pulmonary airways according to an embodiment of the present application. As shown in fig. 6, the registration progress detection device for pulmonary trachea includes: an acquisition module 61, a determination module 62, an extraction module 63, and a registration module 64; wherein, the liquid crystal display device comprises a liquid crystal display device,

An acquisition module 61 for acquiring a first set of movement path points of a positioning sensor within the pulmonary trachea;

a determining module 62, configured to determine, according to the first set of movement path points, a movement path of the positioning sensor in a three-dimensional model of a tracheal tree, where the three-dimensional model of the tracheal tree is reconstructed based on a medical image of the pulmonary trachea;

the acquiring module 61 is further configured to acquire centerline information of the three-dimensional model of the tracheal tree;

the determining module 62 is further configured to determine an airway path of the three-dimensional model of the tracheal tree according to the centerline information;

an extracting module 63, configured to extract a first set of key points of the moving path, where the first set of key points includes a moving path bifurcation point and a moving path end point of the moving path;

the determining module 62 is further configured to determine a second set of keypoints corresponding to the first set of keypoints on the airway path;

and a registration module 64, configured to register the three-dimensional model of the tracheal tree with the pulmonary trachea according to the first set of key points and the second set of key points.

Further, the three-dimensional model of the tracheal tree comprises a multi-stage tracheal model; the second set of key points includes airway path bifurcation points; and the determining module 62, when configured to determine, in the airway path, a second set of keypoints corresponding to the first set of keypoints, is specifically configured to: determining a first target-level tracheal model in which the bifurcation point of the moving path is located in the tracheal tree three-dimensional model; and searching an airway path bifurcation point which is located in the first target-level airway model and has the shortest distance from the moving path bifurcation point on the airway path.

Further, the second set of key points includes an airway path endpoint; and the determining module 62, when configured to determine, in the airway path, a second set of keypoints corresponding to the first set of keypoints, is specifically configured to: along the moving path, determining a target moving path bifurcation point nearest to the moving path end point; calculating a first distance between the bifurcation point of the target moving path and the end point of the moving path; determining a searching direction according to the bifurcation point of the target moving path and the end point of the moving path; determining a target airway path bifurcation point corresponding to the target movement path bifurcation point on the airway path; searching an airway path end point corresponding to the moving path end point on the airway path according to the searching direction; and the distance between the airway path end point and the bifurcation point of the target airway path is equal to the first distance.

Further, the three-dimensional model of the tracheal tree comprises a multi-stage tracheal model; the determining module 62, when configured to determine, according to the first set of moving path points, a moving path of the positioning sensor in the three-dimensional model of the tracheal tree, is specifically configured to: determining a second moving path point set of the positioning sensor in each level of tracheal model according to the first moving path point set; determining points corresponding to main carina of the lung trachea in the three-dimensional model of the tracheal tree as reference points; starting from the datum point, based on a second moving path point set of the positioning sensor in each level of tracheal models, respectively performing path searching for each level of tracheal models to obtain the shortest moving path of the positioning sensor in each level of tracheal models; and merging the shortest moving paths of the positioning sensor in each level of tracheal models to obtain the moving paths.

Further, the determining module 62, when configured to perform path searching for each stage of the tracheal model based on the second set of moving path points of the positioning sensor in each stage of the tracheal model from the reference point, is specifically configured to: starting from the datum point, determining a second target-level tracheal model of which the path search is currently required to be executed; determining a starting position and an ending position of the second target level tracheal model; searching a second moving path point set of the positioning sensor in the second target-level tracheal model, wherein the moving path point with the shortest distance from the initial position is determined as a searching starting point, and the moving path point with the shortest distance from the final position is determined as a searching end point; performing a path search based on the search start point, the search end point, and a second set of movement path points of the positioning sensor within the second target level tracheal model to determine a shortest path from the search start point to the search end point as a shortest movement path of the positioning sensor in the second target level tracheal model according to a search result; after the path search is completed aiming at the second target-level air pipe model, judging whether an air pipe model needing to be subjected to the route search exists in the multi-level air pipe model or not; if so, returning to the step of executing the second target level tracheal model for determining the current path search to be executed.

Further, the determining module 62, when configured to determine the starting position and the ending position in the second target level tracheal model, is specifically configured to: determining a first centerline of the second target level tracheal model from the centerline information; and determining the starting position and the ending position of the second target-level tracheal model according to the two end points of the first central line.

Further, the determining module 62 is specifically configured to, when determining, according to the first set of moving path points, a second set of moving path points of the positioning sensor in each stage of tracheal models, respectively: determining a first principal point set corresponding to the moving path points contained in the first moving path point set in each level of tracheal model; determining the trachea radius of each level of trachea model; performing downsampling processing on a first voxel set in each level of tracheal model based on the tracheal radius; and determining the first integral point set in each level of tracheal model after the downsampling treatment as a second moving path point set of the positioning sensor in each level of tracheal model.

Further, the third target level tracheal model is one of the multi-level tracheal models; and, the determining module 62, when used for determining the tracheal radius of the third target-level tracheal model, is specifically configured to: determining a second centerline of the third target level tracheal model from the centerline information; calculating a second distance from each second voxel on the third target level tracheal model surface to the second centerline; and determining the average value of the second distances from the second body points to the second central line as the tracheal radius of the third target-level tracheal model.

Further, the registration module 64 is further configured to: registering the first moving path point set with the three-dimensional model of the tracheal tree to obtain the registered first moving path point set and the registered three-dimensional model of the tracheal tree; and, the device provided in this embodiment further includes: and the triggering module is used for triggering and executing the step of determining the moving path of the positioning sensor in the three-dimensional model of the tracheal tree reconstructed for the pulmonary trachea according to the first moving path point set based on the registered first moving path point set and the registered three-dimensional model of the tracheal tree.

Further, the apparatus provided in this embodiment may further include: updating a module; the updating module is used for updating the first moving path point set regularly so as to re-execute the step of registering the first moving path point set and the three-dimensional model of the tracheal tree according to the updated first moving path point set.

What needs to be explained here is: the registration device for pulmonary trachea provided by the above embodiment can implement the technical solution described in the method embodiment provided by the present application, and the specific implementation principle of each module or unit can be referred to the corresponding content in the method embodiment, which is not repeated here.

Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 7, the electronic device includes: a memory 71 and a processor 72. Wherein the memory 71 is for storing a computer program; the processor is coupled to the memory for executing the computer program stored in the memory for implementing the steps or functions in the embodiment of the registration method for pulmonary airways provided by the application.

The memory 71 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The electronic device may refer to the processing device shown in fig. 5 described above.

Further, as shown in fig. 7, the electronic device may further include: communication component 73, display 74, power component 75, audio component 76, and other components. Only some of the components are schematically shown in fig. 7, which does not mean that the electronic device only comprises the components shown in fig. 7.

Accordingly, the embodiment of the present application further provides a computer readable storage medium storing a computer program, where the computer program when executed by a computer can implement the steps or functions in the method for detecting the registration progress of the lung and the trachea provided by the embodiment of the present application.

The methods of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. Fig. 8 schematically shows a block diagram of a computer program product provided by the application. The computer program product comprises computer program/instructions 81 which, when the computer program/instructions 81 are executed by a processor 72 such as shown in fig. 7, may fully or partially perform the steps or functions of the method of registration for pulmonary airways provided by embodiments of the present application. The computer may be a general purpose computer, a special purpose computer, a computer network, a network device, a user device, a core network device, an OAM, or other programmable apparatus.

The computer program or instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program or instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that integrates one or more available media. The usable medium may be a magnetic medium, e.g., floppy disk, hard disk, tape; but also optical media such as digital video discs; but also semiconductor media such as solid state disks. The computer readable storage medium may be volatile or nonvolatile storage medium, or may include both volatile and nonvolatile types of storage medium.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of registration for a pulmonary trachea, comprising:

determining a moving path of the positioning sensor in a three-dimensional model of a tracheal tree according to the first moving path point set, wherein the three-dimensional model of the tracheal tree is obtained based on medical image reconstruction of the pulmonary trachea;

2. The method of claim 1, wherein the three-dimensional model of the tracheal tree comprises a multi-level tracheal model; the second set of key points includes airway path bifurcation points;

determining a second set of keypoints in the airway path corresponding to the first set of keypoints, comprising:

determining a first target-level tracheal model in which the bifurcation point of the moving path is located in the tracheal tree three-dimensional model;

and searching an airway path bifurcation point which is located in the first target-level airway model and has the shortest distance from the moving path bifurcation point on the airway path.

3. The method of claim 1, wherein the second set of key points comprises airway path endpoints;

along the moving path, determining a target moving path bifurcation point nearest to the moving path end point;

Calculating a first distance between the bifurcation point of the target moving path and the end point of the moving path;

determining a searching direction according to the bifurcation point of the target moving path and the end point of the moving path;

determining a target airway path bifurcation point corresponding to the target movement path bifurcation point on the airway path;

and searching an airway path end point corresponding to the moving path end point on the airway path according to the searching direction, wherein the distance between the airway path end point and the bifurcation point of the target airway path is equal to the first distance.

4. The method of claim 1, wherein the three-dimensional model of the tracheal tree comprises a multi-level tracheal model; determining a moving path of the positioning sensor in the three-dimensional model of the tracheal tree according to the first moving path point set, wherein the moving path comprises the following steps:

determining a second moving path point set of the positioning sensor in each level of tracheal model according to the first moving path point set;

determining points corresponding to main carina of the lung trachea in the three-dimensional model of the tracheal tree as reference points;

starting from the datum point, based on a second moving path point set of the positioning sensor in each level of tracheal models, respectively performing path searching for each level of tracheal models to obtain the shortest moving path of the positioning sensor in each level of tracheal models;

And merging the shortest moving paths of the positioning sensor in each level of tracheal models to obtain the moving paths.

5. The method of claim 4, wherein starting from the reference point, performing a path search for each stage of the tracheal model based on a second set of path points of movement of the positioning sensor in each stage of the tracheal model, respectively, to obtain a shortest path of movement of the positioning sensor in each stage of the tracheal model, respectively, comprising:

starting from the datum point, determining a second target-level tracheal model of which the path search is currently required to be executed;

determining a starting position and an ending position of the second target level tracheal model;

searching a second moving path point set of the positioning sensor in the second target-level tracheal model, wherein the moving path point with the shortest distance from the initial position is determined as a searching starting point, and the moving path point with the shortest distance from the final position is determined as a searching end point;

performing a path search based on the search start point, the search end point, and a second set of movement path points of the positioning sensor within the second target level tracheal model to determine a shortest path from the search start point to the search end point as a shortest movement path of the positioning sensor in the second target level tracheal model according to a search result;

After the path searching is completed aiming at the second target-level air pipe model, judging whether an air pipe model needing to be subjected to the path searching exists in the multi-level air pipe model or not;

if so, returning to the step of executing the second target level tracheal model for determining the current path search to be executed.

6. The method of claim 5, wherein determining a starting location and an ending location in the second target level tracheal model comprises:

determining a first centerline of the second target level tracheal model from the centerline information;

and determining the starting position and the ending position of the second target-level tracheal model according to the two end points of the first central line.

7. The method of claim 4, wherein determining a second set of movement path points for the positioning sensor in each stage of the tracheal model from the first set of movement path points comprises:

determining a first principal point set corresponding to the moving path points contained in the first moving path point set in each level of tracheal model;

determining the trachea radius of each level of trachea model;

performing downsampling processing on a first voxel set in each level of tracheal model based on the tracheal radius;

And determining the first pixel set after the downsampling processing in each level of tracheal models as a second moving path point set of the positioning sensor in each level of tracheal models respectively.

8. The method of claim 7, wherein a third target level tracheal model is one of the multi-level tracheal models;

determining a tracheal radius of the third target-level tracheal model, comprising:

9. A registration system for a pulmonary trachea, comprising:

Processing device for performing the steps of the registration method for pulmonary airways according to any of claims 1 to 8.

10. An electronic device, comprising: a memory and a processor, wherein,

the memory is used for storing a computer program;

the processor, coupled to the memory, for executing the computer program stored in the memory for implementing the steps in the method of registration for pulmonary airways according to any of the preceding claims 1 to 8.